profession:data engineer
-
Implementing Role-Aware Column Masking for Customer Contact Data
POST_START Implementing Role-Aware Column Masking for Customer Contact Data I recently had the task of ensuring that customer contact data in our Databricks environment is only accessible to authorized users. This is part of a broader initiative to enforce data privacy and compliance across our organization. The specific requirement was to implement role-aware column masking…
A1, category:business, category:education, category:software, category:technology, coding, column, contact, customer, data, data engineering, data governance, databricks, education, implementing, Lakehouse, masking, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, role-aware, sql, unity catalog -
Masking Multiple PII Columns with Separate Governance Functions
POST_START Masking Multiple PII Columns with Separate Governance Functions I recently had the task of masking multiple Personally Identifiable Information (PII) columns in the customer master table. The goal was to ensure that sensitive data like phone numbers and emails were only accessible to authorized users. I decided to use Unity Catalog’s masking capabilities along…
A1, category:business, category:education, category:software, category:technology, coding, columns, data engineering, data governance, databricks, education, Functions, Governance, Lakehouse, masking, multiple, paragraph ordering, paragraph structure, paragraphorder, pii, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, separate, sql, unity catalog -
Hardening Row-Level Security with a Controlled Administrator Bypass
POST_START Hardening Row-Level Security with a Controlled Administrator Bypass I recently had the task of hardening row-level security for a customer table in our data lake. The goal was to enforce a region-based filter, but also allow for a controlled bypass for administrators. I needed to implement a function that would restrict access to data…
A1, administrator, bypass, category:business, category:education, category:software, category:technology, coding, controlled, data engineering, data governance, databricks, education, hardening, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, row-level, security, sql, unity catalog -
Designing Multi-Group Row-Level Security for Regional Sales Data
POST_START Designing Multi-Group Row-Level Security for Regional Sales Data I’ve been tasked with implementing row-level security for the sales data in our Databricks environment. The goal is to ensure that users can only access sales data relevant to their region. This involves setting up a function that filters data based on user group membership and…
A1, category:business, category:education, category:software, category:technology, coding, data, data engineering, data governance, databricks, designing, education, Lakehouse, multi-group, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, regional, row-level, sales, security, sql, unity catalog -
End-to-End Production Security Review for Row and Column Protection
POST_START End-to-End Production Security Review for Row and Column Protection I recently had the opportunity to perform a comprehensive security review of a customer data pipeline in Databricks Unity Catalog. The goal was to ensure that both row-level and column-level data protection were properly implemented and enforced. I started by understanding the functions and tables…
A1, category:business, category:education, category:software, category:technology, coding, column, data engineering, data governance, databricks, education, End-to-End, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, protection, review, row, security, sql, unity catalog -
Documenting a Production Table and Sensitive Column for Governance Review
POST_START Documenting a Production Table and Sensitive Column for Governance Review I recently took on the responsibility of ensuring that our production data assets are properly documented for governance review. One of the key tables in our system is the customer_master table in the production.customers schema. This table holds the core customer information, and it’s…
A1, category:business, category:education, category:software, category:technology, coding, column, data engineering, data governance, databricks, documenting, education, Governance, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, review, sensitive, sql, table, unity catalog -
End-to-End Production Access Investigation from Namespace to Audit Trail
POST_START End-to-End Production Access Investigation from Namespace to Audit Trail I recently needed to investigate access to the production catalog in Databricks Unity Catalog. My goal was to understand who had access to what, what actions they could perform, and how their interactions were recorded in the audit trail. This is a common task when…
A1, Access, audit, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, End-to-End, investigation, Lakehouse, namespace, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, trail, unity catalog -
Testing a Production Table Schema Change Before Grant Review
POST_START Testing a Production Table Schema Change Before Grant Review I recently needed to test a schema change for a production table before submitting it for grant review. The table in question was production.sales.orders, and the goal was to add a new column to track the status of a review process. Before making any changes,…
A1, before, category:business, category:education, category:software, category:technology, change, coding, data engineering, data governance, databricks, education, grant, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, review, schema, sql, table, Testing, unity catalog -
Investigating a Production Table Rename Through Object Discovery
POST_START Investigating a Production Table Rename Through Object Discovery Today, I was tasked with investigating a recent table rename in the production environment. The team had renamed a table from orders to orders_archive, and I needed to verify that the change was correctly applied and understand the impact on the system. I started by checking…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, discovery, education, investigating, Lakehouse, object, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, rename, sql, table, through, unity catalog
