data engineering
-
Reviewing Production Table Properties During an Operational Investigation
POST_START Reviewing Production Table Properties During an Operational Investigation I recently encountered an issue with the production table production.sales.orders that required me to investigate its properties to understand its structure and behavior. As part of the operational investigation, I needed to check the table’s metadata to determine if there were any settings that might be…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, during, education, investigation, Lakehouse, operational, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, properties, Reviewing, sql, table, unity catalog -
Transferring Production View Ownership to the Reporting Platform Team
POST_START Transferring Production View Ownership to the Reporting Platform Team I recently had the task of transferring ownership of a production view to the reporting platform team. This was necessary to ensure that the team had full control over the view, including the ability to manage access and make changes as needed. The view in…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, Lakehouse, ownership, paragraph ordering, paragraph structure, paragraphorder, Platform, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, Reporting, sql, team, transferring, unity catalog, view -
Creating a Restricted Production Reporting View for Analysts
POST_START Creating a Restricted Production Reporting View for Analysts I recently needed to create a restricted reporting view for the data analysts team so they could access only the active orders from our production system. This view would allow them to analyze current orders without exposing sensitive or outdated data. To achieve this, I followed…
A1, analysts, category:business, category:education, category:software, category:technology, coding, creating, data engineering, data governance, databricks, education, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, Reporting, restricted, sql, unity catalog, view -
Granting Read-Only Access to a Production Volume
POST_START Granting Read-Only Access to a Production Volume I recently needed to grant read-only access to a production volume in Databricks Unity Catalog so that the data analysts could explore the data without modifying it. The volume in question was production.files.landing, and the target principal was the data_analysts group. Granting Read-Only Access I started by…
A1, Access, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, granting, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, read-only, sql, unity catalog, Volume. -
Removing Write Access from a Production External Location
POST_START Removing Write Access from a Production External Location I recently had the task of securing a production external location in Databricks Unity Catalog. The location, named production_raw, was being used by several teams, but we needed to remove write access for a legacy ingestion service to prevent accidental or unauthorized data modifications. Understanding the…
A1, Access, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, External, Lakehouse, location, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, removing, sql, unity catalog, Write -
Creating a Governed External Volume for Production Landing Files
POST_START Creating a Governed External Volume for Production Landing Files I’m working on setting up a new data pipeline for our production environment. As part of the data governance process, I need to create a governed external volume for the landing files that will be ingested from S3. I started by checking what volumes already…
A1, category:business, category:education, category:software, category:technology, coding, creating, data engineering, data governance, databricks, education, External, files, governed, Lakehouse, landing, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, unity catalog, Volume.
