data engineering
-
Implementing Production Row-Level Security with a Country-Based Unity Catalog Row Filter
POST_START Implementing Production Row-Level Security with a Country-Based Unity Catalog Row Filter The Problem: Securing Data Access by Country I recently had to implement row-level security for a customer database in Databricks. The requirement was simple but critical: only users in the global_sales account group should be able to access all data, while others should…
A1, category:business, category:education, category:software, category:technology, coding, country-based, data engineering, data governance, databricks, education, filter, implementing, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, row, row-level, security, sql, unity catalog -
Troubleshooting Unexpected Row-Level Access Results from Unity Catalog Row Filters
POST_START Troubleshooting Unexpected Row-Level Access Results from Unity Catalog Row Filters I was working on a query against the production.sales.customers table and noticed that some rows were not being returned as expected. I suspected that Unity Catalog row filters might be at play, but I wasn’t sure why the results didn’t align with what I…
A1, Access, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, filters, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, results, row, row-level, sql, troubleshooting, Unexpected, unity catalog -
Removing Production Unity Catalog RBAC Privileges During User or Team Offboarding
POST_START Removing Production Unity Catalog RBAC Privileges During User or Team Offboarding I recently had to handle the offboarding of a data engineering team from our production environment. One of the critical steps in this process was ensuring that all team members no longer had access to sensitive data and systems. As part of the…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, during, education, Lakehouse, offboarding, paragraph ordering, paragraph structure, paragraphorder, privileges, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, rbac, removing, sql, team, unity catalog, user -
Auditing Role-Based Access Control Assignments with Unity Catalog INFORMATION_SCHEMA
POST_START Auditing Role-Based Access Control Assignments with Unity Catalog INFORMATION_SCHEMA I recently had the task of auditing role-based access control (RBAC) assignments within our Unity Catalog environment. The goal was to ensure that all access to sensitive data in the ‘production’ catalog was properly scoped and assigned to roles, following our organization’s security policies. Since…
A1, Access, assignments, auditing, category:business, category:education, category:software, category:technology, coding, control, data engineering, data governance, databricks, education, information, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, role-based, schema, sql, unity catalog -
Granting Least-Privilege RBAC Write Access to Production Data Engineering Teams
POST_START Granting Least-Privilege RBAC Write Access to Production Data Engineering Teams Granting Least-Privilege RBAC Write Access to Production Data Engineering Teams I started my day by reviewing the latest security requirements for our data engineering team. We needed to grant them the ability to write to the production data catalog, but with the strictest access…
A1, Access, category:business, category:education, category:software, category:technology, coding, data, data engineering, data governance, databricks, education, engineering, granting, Lakehouse, least-privilege, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, rbac, sql, unity catalog, Write -
Implementing Role-Based Access Control Across Production Catalogs, Schemas, and Tables
POST_START Implementing Role-Based Access Control Across Production Catalogs, Schemas, and Tables Implementing Role-Based Access Control Across Production Catalogs, Schemas, and Tables I started my day by reviewing the access control requirements for our production data in Databricks. Our team needed to ensure that the right people could access the right data, without exposing sensitive information.…
A1, Access, Across, catalogs, category:business, category:education, category:software, category:technology, coding, control, data engineering, data governance, databricks, education, implementing, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, role-based, schemas, sql, unity catalog -
Finding Excessive or Unexpected Unity Catalog Privileges on Production Data
POST_START Identifying Excessive or Unexpected Unity Catalog Privileges on Production Data I recently had to investigate a concern raised by our security team about potential excessive or unexpected privileges on our production data in Unity Catalog. The team was worried that some users might have access to data they shouldn’t, which could lead to data…
A1, category:business, category:education, category:software, category:technology, coding, data, data engineering, data governance, databricks, education, excessive, finding, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, privileges, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, Unexpected, unity catalog -
Tracing a Production Table Query Failure Through Catalog, Schema, and Table Privileges
POST_START Tracing a Production Table Query Failure Through Catalog, Schema, and Table Privileges Tracing a Production Table Query Failure Through Catalog, Schema, and Table Privileges I was working on a critical data pipeline that was failing with an unexpected error. The query was attempting to access a table in the production catalog, but it was…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, failure, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, privileges, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, query, schema, sql, table, through, tracing, unity catalog -
Controlling File Access with Volume Privileges
POST_START Controlling File Access with Volume Privileges I started my day by reviewing the latest data governance requirements. Our team needed to ensure that only specific roles had access to certain volumes in Unity Catalog. The main goal was to grant read access to the training.sales.documents volume to the analysts group. This was a critical…
A1, Access, category:business, category:education, category:software, category:technology, coding, controlling, data engineering, data governance, databricks, education, file, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, privileges, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, unity catalog, Volume.
