production
-
Comparing Current and Historical Production Data After a Suspected Bad Write
POST_START Comparing Current and Historical Production Data After a Suspected Bad Write Yesterday, I noticed some inconsistencies in our sales reports. A few orders were showing up as completed when they should have been pending. I suspected a bad write might have occurred, so I decided to investigate using Databricks Unity Catalog’s time travel capabilities.…
A1, after, bad, category:business, category:education, category:software, category:technology, coding, Comparing, current, data, data engineering, data governance, databricks, education, historical, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, suspected, unity catalog -
Establishing a Clean Production Catalog Context Before an Incident Investigation
POST_START Establishing a Clean Production Catalog Context Before an Incident Investigation Today, I’m tasked with investigating an unexpected data discrepancy in our production environment. To ensure I’m working with the correct and clean context, I need to establish a clear understanding of the current Unity Catalog setup. I know that starting with the right catalog…
A1, before, category:business, category:education, category:software, category:technology, clean, coding, context, data engineering, data governance, databricks, education, establishing, incident, investigation, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, unity catalog -
Recovering a Production Delta Table to a Known Good Version
POST_START Recovering a Production Delta Table to a Known Good Version Today, I encountered an issue with the production Delta table production.sales.orders. A recent update introduced some inconsistencies, and I needed to recover the table to a known good version. I decided to use Unity Catalog’s versioning features to identify and restore the table to…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, Delta, education, good, known, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, recovering, sql, table, unity catalog, version -
Comparing Current and Historical Production Data After a Suspected Bad Write
POST_START Comparing Current and Historical Production Data After a Suspected Bad Write Today, I encountered an issue with the production data in our Databricks Unity Catalog. A team member reported that a recent write operation might have caused some data to be incorrect. My task was to investigate and compare the current data with a…
A1, after, bad, category:business, category:education, category:software, category:technology, coding, Comparing, current, data, data engineering, data governance, databricks, education, historical, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, suspected, unity catalog -
Establishing a Clean Production Catalog Context Before an Incident Investigation
POST_START Establishing a Clean Production Catalog Context Before an Incident Investigation Today, I’m tasked with investigating an unexpected data inconsistency in our production environment. Before diving into the issue, I need to ensure I’m working with the correct and clean catalog context to avoid any misinterpretation of the data. The first step is to verify…
A1, before, category:business, category:education, category:software, category:technology, clean, coding, context, data engineering, data governance, databricks, education, establishing, incident, investigation, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, unity catalog -
End-to-End Production Security Review for Row and Column Protection
POST_START End-to-End Production Security Review for Row and Column Protection I recently had the opportunity to perform a comprehensive security review of a customer data pipeline in Databricks Unity Catalog. The goal was to ensure that both row-level and column-level data protection were properly implemented and enforced. I started by understanding the functions and tables…
A1, category:business, category:education, category:software, category:technology, coding, column, data engineering, data governance, databricks, education, End-to-End, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, protection, review, row, security, sql, unity catalog -
End-to-End Production Access Investigation from Namespace to Audit Trail
POST_START End-to-End Production Access Investigation from Namespace to Audit Trail I recently needed to investigate access to the production catalog in Databricks Unity Catalog. My goal was to understand who had access to what, what actions they could perform, and how their interactions were recorded in the audit trail. This is a common task when…
A1, Access, audit, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, End-to-End, investigation, Lakehouse, namespace, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, trail, unity catalog -
Documenting a Production Table and Sensitive Column for Governance Review
POST_START Documenting a Production Table and Sensitive Column for Governance Review I recently took on the responsibility of ensuring that our production data assets are properly documented for governance review. One of the key tables in our system is the customer_master table in the production.customers schema. This table holds the core customer information, and it’s…
A1, category:business, category:education, category:software, category:technology, coding, column, data engineering, data governance, databricks, documenting, education, Governance, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, review, sensitive, sql, table, unity catalog -
Testing a Production Table Schema Change Before Grant Review
POST_START Testing a Production Table Schema Change Before Grant Review I recently needed to test a schema change for a production table before submitting it for grant review. The table in question was production.sales.orders, and the goal was to add a new column to track the status of a review process. Before making any changes,…
A1, before, category:business, category:education, category:software, category:technology, change, coding, data engineering, data governance, databricks, education, grant, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, review, schema, sql, table, Testing, unity catalog
