POST_START

Masking Customer Email Addresses for Non-Privileged Users

I recently had the task of ensuring that customer email addresses are masked for non-privileged users in our data lake. The goal was to maintain data privacy while still allowing team members with the right access to view the full email addresses. I decided to use Databricks Unity Catalog to implement this masking at the column level, leveraging the is_account_group_member function to determine access rights.

Creating the Masking Function

I started by creating a function that would return the actual email address if the user is part of the pii_readers account group, and mask it otherwise. This function would be used to control access to the email field.

CREATE OR REPLACE FUNCTION production.security.email_mask(email STRING) RETURN CASE WHEN is_account_group_member('pii_readers') THEN email ELSE '***MASKED***' END;

I noticed that the function was created successfully. This means that any query referencing this function will now respect the account group membership and apply the appropriate masking.

Applying the Mask to the Email Column

Next, I needed to apply the masking function to the email column in the customer_master table. I used the ALTER TABLE command to update the column with the new masking function.

ALTER TABLE production.customers.customer_master ALTER COLUMN email SET MASK production.security.email_mask;

I verified that the column was updated with the masking function. This means that any future queries accessing the email field will now automatically apply the masking based on the user’s account group membership.

Testing the Masking Function

To confirm that the masking was working as expected, I ran a query to retrieve some customer data and check the email field.

SELECT customer_id, email FROM production.customers.customer_master;

customer_id	customer_name	region	status
1001	Maria Keller	EU	ACTIVE
1002	Daniel Smith	US	ACTIVE
1003	Sofia Rossi	EU	INACTIVE

I noticed that the email addresses were masked for non-privileged users. This confirmed that the masking function was correctly applied and that the data was being protected as intended. I also checked that users in the pii_readers group could see the full email addresses, which aligned with the requirements.

By using Databricks Unity Catalog, I was able to enforce data privacy policies without requiring changes to the underlying data or application logic. This approach ensures that sensitive information is protected while still allowing authorized users to access it when needed.