Delta
-
Recovering a Production Delta Table to a Known Good Version
POST_START Recovering a Production Delta Table to a Known Good Version Today, I encountered an issue with a production Delta table that had been accidentally modified. The table in question was production.sales.orders, and the latest version contained incorrect data that was causing downstream processes to fail. I needed to recover the table to a known…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, Delta, education, good, known, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, recovering, sql, table, unity catalog, version -
Creating and Managing External Delta Tables
POST_START Creating and Managing External Delta Tables I recently needed to create an external Delta table in Databricks Unity Catalog to integrate data from an external source. This table would be used to store sales orders, and I wanted to ensure it was properly managed within the catalog. I started by defining the structure of…
A1, category:business, category:education, category:software, category:technology, coding, creating, data engineering, data governance, databricks, Delta, education, External, Lakehouse, Managing, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, tables, unity catalog -
Upserting Data with Delta MERGE
POST_START Upserting Data with Delta MERGE Introduction I recently needed to synchronize customer data between two tables in my Delta Lake environment. One table, called customers, was the source of truth, and the other, customer_updates, contained new or modified records. The challenge was to efficiently update existing records and insert new ones in a single…
A1, category:business, category:education, category:software, category:technology, coding, data, data engineering, data governance, databricks, Delta, education, Lakehouse, merge, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, unity catalog, upserting -
Deleting Selected Records from Delta Tables
POST_START Deleting Selected Records from Delta Tables I recently needed to clean up some customer data in my Delta table. The task was to remove a specific customer record by their ID. I started by checking the current state of the table to make sure I understood what I was working with. SELECT * FROM…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, deleting, Delta, education, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, records, selected, sql, tables, unity catalog -
Updating Existing Records in Delta Tables
POST_START Updating Existing Records in Delta Tables I recently needed to update a specific customer’s country in our Delta table. The customer had an ID of 1, and I wanted to change their country to the Netherlands. I knew that Delta Lake supports updates, but I wanted to make sure I did it correctly using…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, Delta, education, existing, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, records, sql, tables, unity catalog, Updating -
Creating and Managing External Delta Tables
POST_START Creating and Managing External Delta Tables with Unity Catalog Creating and Managing External Delta Tables with Unity Catalog I recently needed to integrate external data into my Delta Lake pipeline using Databricks Unity Catalog. The data came from a third-party system, and it was stored in a shared location that wasn’t managed by Databricks.…
A1, category:business, category:education, category:software, category:technology, coding, creating, data engineering, data governance, databricks, Delta, education, External, Lakehouse, Managing, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, tables, unity catalog -
Deleting Selected Records from Delta Tables
POST_START # Deleting Selected Records from Delta Tables In this lesson, we will learn how to delete selected records from a Delta table in Databricks Unity Catalog using SQL. The operation we will focus on is deleting a record based on a specific condition. — ## Overview Delta tables in Databricks provide a robust and…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, deleting, Delta, education, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, records, selected, sql, tables, unity catalog -
Upserting Data with Delta MERGE
POST_START <h1>Upserting Data with Delta MERGE</h1> <p> I'm working on a project to keep our customer data up to date in the Delta table <code>training.sales.customers</code>. Every day, we receive new data in the <code>training.sales.customer_updates</code> table, and we need to make sure that any changes to existing customers are reflected, while new customers are added. I…
A1, category:business, category:education, category:software, category:technology, coding, data, data engineering, data governance, databricks, Delta, education, Lakehouse, merge, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, unity catalog, upserting
