External
-
Removing Write Access from a Production External Location
POST_START Removing Write Access from a Production External Location I recently had the task of securing a production external location in Databricks Unity Catalog. The location, named production_raw, was being used by several teams, but we needed to remove write access for a legacy ingestion service to prevent accidental or unauthorized data modifications. Understanding the…
A1, Access, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, External, Lakehouse, location, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, removing, sql, unity catalog, Write -
Creating a Governed External Volume for Production Landing Files
POST_START Creating a Governed External Volume for Production Landing Files I’m working on setting up a new data pipeline for our production environment. As part of the data governance process, I need to create a governed external volume for the landing files that will be ingested from S3. I started by checking what volumes already…
A1, category:business, category:education, category:software, category:technology, coding, creating, data engineering, data governance, databricks, education, External, files, governed, Lakehouse, landing, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, unity catalog, Volume. -
Reviewing Storage Credentials Used by Production External Locations
POST_START Reviewing Storage Credentials Used by Production External Locations I recently needed to review the storage credentials used by our production external locations to ensure they were correctly configured and secure. This was part of a routine audit to verify that all external data sources are properly managed within the Databricks Unity Catalog. I started…
A1, category:business, category:education, category:software, category:technology, coding, credentials, data engineering, data governance, databricks, education, External, Lakehouse, locations, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, Reviewing, sql, Storage, unity catalog, Used -
Granting Read Access to a Governed Production External Location
POST_START Granting Read Access to a Governed Production External Location I recently had the task of granting read access to a governed production external location in Databricks Unity Catalog. The location, named production_raw, is used by data engineers to access raw data stored in an S3 bucket. My goal was to ensure that the data_engineers…
A1, Access, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, External, governed, granting, Lakehouse, location, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, read, sql, unity catalog -
Validating an External Location Before Production Data Onboarding
POST_START Validating an External Location Before Production Data Onboarding I recently had the task of onboarding a new data source into our production environment. Before I could proceed, I needed to validate the external location to ensure it met our security, access, and governance standards. The first step was to check what external locations were…
A1, before, category:business, category:education, category:software, category:technology, coding, data, data engineering, data governance, databricks, education, External, Lakehouse, location, onboarding, paragraph ordering, paragraph structure, paragraphorder, production, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, unity catalog, validating -
Creating and Managing External Volumes
POST_START Creating and Managing External Volumes in Databricks Unity Catalog I recently started working on a new data pipeline for the sales team, and I needed to organize the raw files they’re ingesting from various sources. I learned that Databricks Unity Catalog provides a way to manage external volumes, which are useful for storing and…
A1, category:business, category:education, category:software, category:technology, coding, creating, data engineering, data governance, databricks, education, External, Lakehouse, Managing, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, unity catalog, volumes -
Creating and Managing External Delta Tables
POST_START Creating and Managing External Delta Tables I recently needed to create an external Delta table in Databricks Unity Catalog to integrate data from an external source. This table would be used to store sales orders, and I wanted to ensure it was properly managed within the catalog. I started by defining the structure of…
A1, category:business, category:education, category:software, category:technology, coding, creating, data engineering, data governance, databricks, Delta, education, External, Lakehouse, Managing, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, tables, unity catalog -
Managing Read, Write, and Table-Creation Access on External Locations
POST_START Managing Read, Write, and Table-Creation Access on External Locations Setting Up Access for Data Engineers I recently needed to set up access for the data_engineers team to an external location called raw_data. The goal was to grant them the ability to read files from this location, but also to manage their permissions carefully to…
A1, Access, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, External, Lakehouse, locations, Managing, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, read, sql, table-creation, unity catalog, Write -
Inspecting Storage Credentials for External Data Access
POST_START Inspecting Storage Credentials for External Data Access I recently needed to understand how our team is accessing external storage systems for data processing. I started by checking the list of storage credentials that are currently registered in Unity Catalog. This would give me a high-level overview of what credentials are available and who owns…
A1, Access, category:business, category:education, category:software, category:technology, coding, credentials, data, data engineering, data governance, databricks, education, External, inspecting, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, Storage, unity catalog -
Creating and Inspecting Governed External Locations
POST_START Creating and Inspecting Governed External Locations I recently needed to set up a new external location in Unity Catalog to access raw data stored in cloud storage. This location would be used by multiple teams, so I wanted to ensure it was properly governed and secured. I started by creating the external location with…
A1, category:business, category:education, category:software, category:technology, coding, creating, data engineering, data governance, databricks, education, External, governed, inspecting, Lakehouse, locations, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, unity catalog
