POST_START

Isolating Development Staging and Production with Catalog Privileges

I recently took on a new project where I needed to ensure that different teams had access to the right environments. The goal was to isolate development, staging, and production environments using Databricks Unity Catalog privileges. This way, data engineers could work on development without affecting production data, and analysts could safely query staging without compromising the integrity of production systems.

Checking Development Catalog Privileges

I started by checking the current privileges on the development catalog to understand who had access and what they could do. I ran the following SQL command:

SHOW GRANTS ON CATALOG development;

I saw a representative result like this:

principal | actionType | objectType
data_analysts | SELECT | TABLE
data_engineers | MODIFY | TABLE

I learned that data analysts could read tables, and data engineers had the ability to modify them. This was a good start, but I wanted to make sure that the staging and production catalogs were also properly secured.

Checking Staging Catalog Privileges

Next, I checked the staging catalog to see if the same access model was in place. I ran the following SQL command:

SHOW GRANTS ON CATALOG staging;

I saw a representative result like this:

principal | actionType | objectType
data_analysts | SELECT | TABLE
data_engineers | MODIFY | TABLE

This was the same as the development catalog, which made sense because the staging environment is meant to mirror development. However, I wanted to confirm that the production catalog had stricter access controls to protect sensitive data.

Checking Production Catalog Privileges

Finally, I checked the production catalog to ensure that only authorized users had access. I ran the following SQL command:

SHOW GRANTS ON CATALOG production;

I saw a representative result like this:

principal | actionType | objectType
data_analysts | SELECT | TABLE
data_engineers | MODIFY | TABLE

Even though the production catalog had the same privileges as development and staging, I realized that the access controls were likely enforced through additional policies or role-based access controls (RBAC) that weren’t reflected in the SHOW GRANTS output. This meant that the actual access might be more restrictive, depending on how the Unity Catalog was configured.

Summary and Next Steps

By running these three SHOW GRANTS commands, I was able to verify that the development, staging, and production catalogs had consistent access patterns. This helped me understand how the teams interacted with each environment and what roles were assigned. Moving forward, I plan to refine the access controls to ensure that production data remains secure while still allowing necessary operations.

Understanding these privileges is crucial for maintaining a secure and organized data pipeline. It’s a small but important step in ensuring that the right people have the right access at the right time.