data
-
Comparing Current and Historical Production Data After a Suspected Bad Write
POST_START Comparing Current and Historical Production Data After a Suspected Bad Write Yesterday, I noticed some inconsistencies in our sales reports. A few orders were showing up as completed when they should have been pending. I suspected a bad write might have occurred, so I decided to investigate using Databricks Unity Catalog’s time travel capabilities.…
A1, after, bad, category:business, category:education, category:software, category:technology, coding, Comparing, current, data, data engineering, data governance, databricks, education, historical, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, suspected, unity catalog -
Comparing Current and Historical Production Data After a Suspected Bad Write
POST_START Comparing Current and Historical Production Data After a Suspected Bad Write Today, I encountered an issue with the production data in our Databricks Unity Catalog. A team member reported that a recent write operation might have caused some data to be incorrect. My task was to investigate and compare the current data with a…
A1, after, bad, category:business, category:education, category:software, category:technology, coding, Comparing, current, data, data engineering, data governance, databricks, education, historical, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, suspected, unity catalog -
Implementing Role-Aware Column Masking for Customer Contact Data
POST_START Implementing Role-Aware Column Masking for Customer Contact Data I recently had the task of ensuring that customer contact data in our Databricks environment is only accessible to authorized users. This is part of a broader initiative to enforce data privacy and compliance across our organization. The specific requirement was to implement role-aware column masking…
A1, category:business, category:education, category:software, category:technology, coding, column, contact, customer, data, data engineering, data governance, databricks, education, implementing, Lakehouse, masking, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, role-aware, sql, unity catalog -
Designing Multi-Group Row-Level Security for Regional Sales Data
POST_START Designing Multi-Group Row-Level Security for Regional Sales Data I’ve been tasked with implementing row-level security for the sales data in our Databricks environment. The goal is to ensure that users can only access sales data relevant to their region. This involves setting up a function that filters data based on user group membership and…
A1, category:business, category:education, category:software, category:technology, coding, data, data engineering, data governance, databricks, designing, education, Lakehouse, multi-group, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, regional, row-level, sales, security, sql, unity catalog -
Validating an External Location Before Production Data Onboarding
POST_START Validating an External Location Before Production Data Onboarding I recently had the task of onboarding a new data source into our production environment. Before I could proceed, I needed to validate the external location to ensure it met our security, access, and governance standards. The first step was to check what external locations were…
A1, before, category:business, category:education, category:software, category:technology, coding, data, data engineering, data governance, databricks, education, External, Lakehouse, location, onboarding, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, unity catalog, validating -
Comparing Current and Historical Production Data After a Suspected Bad Write
POST_START Comparing Current and Historical Production Data After a Suspected Bad Write Yesterday, I noticed some unusual behavior in our production data pipeline. A new batch of data was written to the production.sales.orders table, but when I checked the latest records, it looked like some entries were missing or had incorrect values. I suspected a…
A1, after, bad, category:business, category:education, category:software, category:technology, coding, Comparing, current, data, data engineering, data governance, databricks, education, historical, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, suspected, unity catalog -
Inspecting Storage Credentials for External Data Access
POST_START Inspecting Storage Credentials for External Data Access I recently needed to understand how our team is accessing external storage systems for data processing. I started by checking the list of storage credentials that are currently registered in Unity Catalog. This would give me a high-level overview of what credentials are available and who owns…
A1, Access, category:business, category:education, category:software, category:technology, coding, credentials, data, data engineering, data governance, databricks, education, External, inspecting, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, Storage, unity catalog -
Controlling Table Data Changes with MODIFY Privileges
POST_START Controlling Table Data Changes with MODIFY Privileges I recently had the task of managing access to a table in our data warehouse called training.sales.customers. The goal was to ensure that only specific teams could modify the data, while others could only read it. I learned that using the MODIFY privilege is essential for allowing…
A1, category:business, category:education, category:software, category:technology, changes, coding, controlling, data, data engineering, data governance, databricks, education, Lakehouse, modify, paragraph ordering, paragraph structure, paragraphorder, privileges, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, table, unity catalog
