data engineering
-
Validating Least-Privilege Access from Catalog Through Sensitive Columns
POST_START Validating Least-Privilege Access from Catalog Through Sensitive Columns I started my day by reviewing the access controls for the production catalog in our Databricks environment. My goal was to ensure that the least-privilege principle was being followed, especially when it came to sensitive columns in our customer data. I wanted to verify if the…
A1, Access, category:business, category:education, category:software, category:technology, coding, columns, data engineering, data governance, databricks, education, Lakehouse, least-privilege, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sensitive, sql, through, unity catalog, validating -
Investigating Unexpected Production Volume Write Access
POST_START Investigating Unexpected Production Volume Write Access Today, I was tasked with investigating an unexpected production volume write access issue. A team member reported that some data in the production.files.documents volume was being written to without proper authorization. My goal was to determine who had access and whether any unauthorized writes had occurred. Checking Volume…
A1, Access, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, investigating, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, Unexpected, unity catalog, Volume., Write -
Restricting Production Volume Writes to Approved Pipelines
POST_START Restricting Production Volume Writes to Approved Pipelines I recently had the task of ensuring that only approved pipelines could write to the production volume for document files. This was a critical step to maintain data integrity and prevent unauthorized or accidental modifications to our production data. I needed to configure Unity Catalog to enforce…
A1, approved, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, Pipelines., production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, restricting, sql, unity catalog, Volume., writes -
Restricting Production Volume Reads to Approved Consumers
POST_START Restricting Production Volume Reads to Approved Consumers I recently had the task of ensuring that only approved consumers could read from a specific volume in our production environment. This was part of a broader effort to enforce data governance and ensure that sensitive documents were only accessed by the right teams. The volume in…
A1, approved, category:business, category:education, category:software, category:technology, coding, Consumers, data engineering, data governance, databricks, education, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, reads, restricting, sql, unity catalog, Volume. -
Reviewing Production Schema Creation Privileges
POST_START Reviewing Production Schema Creation Privileges I recently needed to review the privileges associated with the production.finance schema in our Databricks environment. This was part of a routine security audit to ensure that only authorized users had access to sensitive financial data. I started by checking the current grants assigned to the schema using the…
A1, category:business, category:education, category:software, category:technology, coding, creation, data engineering, data governance, databricks, education, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, privileges, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, Reviewing, schema, sql, unity catalog -
Auditing Security Function Ownership Used by Row Filters
POST_START Auditing Security Function Ownership Used by Row Filters I recently had to audit the security ownership of a row filter function used in our production environment. The function, production.security.region_filter, is used to enforce regional access controls across multiple datasets. My goal was to understand who owns the function and what permissions are granted on…
A1, auditing, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, filters, function, Lakehouse, ownership, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, row, security, sql, unity catalog, Used
