data engineering
-
Removing Temporary Elevated Production Privileges
POST_START Removing Temporary Elevated Production Privileges I recently had to address a security concern where a group of incident responders had been granted temporary MODIFY privileges on the production.sales.orders table. These privileges were initially issued to allow them to investigate a data integrity issue, but the incident had been resolved, and the elevated access was…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, elevated, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, privileges, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, removing, sql, temporary, unity catalog -
Isolating Development Staging and Production with Catalog Privileges
POST_START Isolating Development Staging and Production with Catalog Privileges Isolating Development Staging and Production with Catalog Privileges I recently took on a new project where I needed to ensure that different teams had access to the right environments. The goal was to isolate development, staging, and production environments using Databricks Unity Catalog privileges. This way,…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, development, education, isolating, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, privileges, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, staging, unity catalog -
Building Read-Only Production Access for Analysts
POST_START Building Read-Only Production Access for Analysts Building Read-Only Production Access for Analysts I was recently tasked with setting up read-only access to a production catalog for a group of analysts. The goal was to ensure they could query the data they needed without being able to modify anything. I knew that using Databricks Unity…
A1, Access, analysts, building, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, read-only, sql, unity catalog -
Troubleshooting a Service Principal That Lost Production Table Access
POST_START Troubleshooting a Service Principal That Lost Production Table Access Today, I encountered an issue where a service principal that was previously able to access a table in the production catalog suddenly lost access. The team was trying to run a critical ETL job, and the error message indicated that the service principal didn’t have…
A1, Access, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, Lakehouse, Lost, paragraph ordering, paragraph structure, paragraphorder, principal, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, service, sql, table, That, troubleshooting, unity catalog -
Designing Service Principal Access for Production Data Pipelines
POST_START Designing Service Principal Access for Production Data Pipelines I recently took on the task of securing access for our ETL pipelines to a production data catalog in Databricks. The goal was to ensure that the service principals used by our data pipelines had the right level of access without exposing sensitive data unnecessarily. I…
A1, Access, category:business, category:education, category:software, category:technology, coding, data, data engineering, data governance, databricks, designing, education, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, Pipelines., principal, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, service, sql, unity catalog -
Detecting Privilege Escalation Through Production Group Grants
POST_START Understanding Privilege Escalation Through Production Group Grants I recently had the opportunity to audit the access controls for our production data catalog. One of the key concerns was identifying potential privilege escalation risks. I knew that understanding the grants assigned to the production catalog and its objects was essential. I started by examining the…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, detecting, education, escalation, grants, group, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, privilege, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, through, unity catalog -
Enforcing Separation of Duties for Sensitive Production Data with Unity Catalog Privileges
POST_START Enforcing Separation of Duties for Sensitive Production Data with Unity Catalog Privileges I recently took on the responsibility of securing access to a critical dataset in our production environment. The dataset, called customer_master, contains sensitive customer information that must be handled with care. My task was to enforce a separation of duties model using…
A1, category:business, category:education, category:software, category:technology, coding, data, data engineering, data governance, databricks, duties, education, enforcing, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, privileges, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sensitive, separation, sql, unity catalog
