Delta
-
Recovering a Production Delta Table to a Known Good Version
POST_START Recovering a Production Delta Table to a Known Good Version Today, I encountered an issue with the production Delta table production.sales.orders. A recent update introduced some inconsistencies, and I needed to recover the table to a known good version. I decided to use Unity Catalog’s versioning features to identify and restore the table to…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, Delta, education, good, known, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, recovering, sql, table, unity catalog, version -
Recovering a Production Delta Table to a Known Good Version
POST_START Recovering a Production Delta Table to a Known Good Version Today, I encountered an issue with a production Delta table that had been accidentally modified. The table in question was production.sales.orders, and the latest version contained incorrect data that was causing downstream processes to fail. I needed to recover the table to a known…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, Delta, education, good, known, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, recovering, sql, table, unity catalog, version -
Creating and Managing External Delta Tables
POST_START Creating and Managing External Delta Tables I recently needed to create an external Delta table in Databricks Unity Catalog to integrate data from an external source. This table would be used to store sales orders, and I wanted to ensure it was properly managed within the catalog. I started by defining the structure of…
A1, category:business, category:education, category:software, category:technology, coding, creating, data engineering, data governance, databricks, Delta, education, External, Lakehouse, Managing, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, tables, unity catalog -
Upserting Data with Delta MERGE
POST_START Upserting Data with Delta MERGE Introduction I recently needed to synchronize customer data between two tables in my Delta Lake environment. One table, called customers, was the source of truth, and the other, customer_updates, contained new or modified records. The challenge was to efficiently update existing records and insert new ones in a single…
A1, category:business, category:education, category:software, category:technology, coding, data, data engineering, data governance, databricks, Delta, education, Lakehouse, merge, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, unity catalog, upserting -
Deleting Selected Records from Delta Tables
POST_START Deleting Selected Records from Delta Tables I recently needed to clean up some customer data in my Delta table. The task was to remove a specific customer record by their ID. I started by checking the current state of the table to make sure I understood what I was working with. SELECT * FROM…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, deleting, Delta, education, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, records, selected, sql, tables, unity catalog -
Updating Existing Records in Delta Tables
POST_START Updating Existing Records in Delta Tables I recently needed to update a specific customer’s country in our Delta table. The customer had an ID of 1, and I wanted to change their country to the Netherlands. I knew that Delta Lake supports updates, but I wanted to make sure I did it correctly using…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, Delta, education, existing, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, records, sql, tables, unity catalog, Updating
