data engineering
-
Controlling Who Can Create Tables in Production Unity Catalog Schemas
POST_START Controlling Who Can Create Tables in Production Unity Catalog Schemas I’m working on a project where I need to ensure that only specific teams have the ability to create tables in our production Unity Catalog schemas. This is important to maintain data integrity and prevent accidental or unauthorized changes. Today, I’m going to walk…
A1, Can, category:business, category:education, category:software, category:technology, coding, controlling, Create, data engineering, data governance, databricks, education, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, schemas, sql, tables, unity catalog, Who -
Controlling Who Can Create Governed Views in Production Unity Catalog Schemas
POST_START Controlling Who Can Create Governed Views in Production Unity Catalog Schemas I recently had the responsibility of ensuring that only specific team members could create governed views in our production Unity Catalog schemas. The goal was to maintain data governance and prevent unauthorized access or modifications to sensitive data. I started by understanding the…
A1, Can, category:business, category:education, category:software, category:technology, coding, controlling, Create, data engineering, data governance, databricks, education, governed, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, schemas, sql, unity catalog, Views, Who -
Designing Least-Privilege Unity Catalog Read Access for Production Analytics Users
POST_START Designing Least-Privilege Unity Catalog Read Access for Production Analytics Users Designing Least-Privilege Unity Catalog Read Access for Production Analytics Users I started my day by reviewing the access requirements for the production analytics users. We needed to ensure that these users could read data from specific tables in the production catalog, but with the…
A1, Access, Analytics,, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, designing, education, Lakehouse, least-privilege, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, read, sql, unity catalog, users -
Troubleshooting Sensitive Column Visibility and Unity Catalog Column Mask Behavior
POST_START Troubleshooting Sensitive Column Visibility and Unity Catalog Column Mask Behavior I was working on a data pipeline that involved accessing the production.customers.customer_master table. My task was to understand why certain sensitive columns weren’t showing up in the results, even though I had the necessary permissions. I wanted to troubleshoot the issue using Unity Catalog’s…
A1, behavior, category:business, category:education, category:software, category:technology, coding, column, data engineering, data governance, databricks, education, Lakehouse, mask, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sensitive, sql, troubleshooting, unity catalog, visibility -
Safely Removing Column-Level Masking from a Production Unity Catalog Table
POST_START Safely Removing Column-Level Masking from a Production Unity Catalog Table Safely Removing Column-Level Masking from a Production Unity Catalog Table I’ve been working on a project where we needed to update a customer email column in our production table to remove column-level masking. The table in question is production.customers.customer_master, and the column we’re focusing…
A1, category:business, category:education, category:software, category:technology, coding, column-level, data engineering, data governance, databricks, education, Lakehouse, masking, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, removing, safely, sql, table, unity catalog -
Protecting Sensitive Production Columns with Unity Catalog Column Masks
POST_START Protecting Sensitive Production Columns with Unity Catalog Column Masks Setting the Stage I recently joined a team that’s responsible for maintaining a data warehouse built on Databricks. One of the key responsibilities I inherited was ensuring that sensitive data, especially in production environments, is properly protected. The team had a number of tables with…
A1, category:business, category:education, category:software, category:technology, coding, column, columns, data engineering, data governance, databricks, education, Lakehouse, masks, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, protecting, sensitive, sql, unity catalog -
Safely Removing Row-Level Security from a Production Unity Catalog Table
POST_START Safely Removing Row-Level Security from a Production Unity Catalog Table The Problem I recently inherited a table in the production environment that had row-level security (RLS) applied to it. The table was production.sales.customers, and it was used by multiple teams for reporting and analytics. While the RLS was initially implemented to enforce data access…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, removing, row-level, safely, security, sql, table, unity catalog
