POST_START

Investigating Unexpected Cross-Catalog Production Access

Today, I was tasked with investigating an unexpected access issue in our Databricks environment. A team member noticed that some users were able to access data from the production catalog, even though they shouldn’t have had permission. I needed to figure out how this access was happening and ensure that only authorized users could interact with the production data.

Checking Permissions on the Production Catalog

I started by checking the current grants on the production catalog to see who had access. I ran the following SQL command:

SHOW GRANTS ON CATALOG production;

I saw a representative result like this:

principal | actionType | objectType
data_analysts | SELECT | TABLE
data_engineers | MODIFY | TABLE

This showed that both data_analysts and data_engineers had access to tables in the production catalog. However, I wasn’t sure if these users were supposed to have this access. I needed to confirm if there were any additional grants or if there was a misconfiguration.

Exploring Access to the Finance Production Catalog

To get a better understanding of the access patterns, I decided to check the finance_prod catalog as well. I ran the following SQL command:

SHOW GRANTS ON CATALOG finance_prod;

I saw a representative result like this:

principal | actionType | objectType
data_analysts | SELECT | TABLE
data_engineers | MODIFY | TABLE

This was the same pattern as the production catalog. The data_analysts had SELECT access, and the data_engineers had MODIFY access. It was clear that the access model was consistent across both catalogs. But this also raised a question: why were users with access to the finance_prod catalog also able to access the production catalog?

Examining Specific Table Permissions

To dig deeper, I focused on a specific table in the finance_prod catalog, the reporting.payments table. I ran the following SQL command to check its grants:

SHOW GRANTS ON TABLE finance_prod.reporting.payments;

I saw a representative result like this:

principal | actionType | objectType
data_analysts | SELECT | TABLE
data_engineers | MODIFY | TABLE

This confirmed that the same users had access to this specific table. It was clear that the access was not limited to just the catalog level but also extended down to specific tables. This meant that the access was more granular than I initially thought.

Understanding the Implications

I realized that the access pattern was consistent across both the production and finance_prod catalogs. However, the presence of access to the production catalog was unexpected. I needed to verify if the users who had access to the finance_prod catalog were also inadvertently granted access to the production catalog through some shared permissions or misconfiguration.

By checking the grants at the catalog and table levels, I was able to trace the access path and understand how users could be accessing data they shouldn’t have. This was a crucial step in identifying the root cause of the issue.

Next Steps

With the current grants in place, I could now focus on reviewing the access policies and determining if there was a need to restrict access to the production catalog. I would also need to ensure that the access model was consistent with our security requirements and that only authorized users had access to sensitive data.

Understanding the access patterns was the first step in resolving the issue. Now, I was ready to take the next steps in securing our data environment.