POST_START

Creating and Inspecting Governed External Locations

I started my day by setting up a new external location in Databricks Unity Catalog to access raw data stored in cloud storage. This was part of a larger initiative to centralize data access and ensure all data sources are properly governed and secured.

Creating the External Location

I ran the CREATE EXTERNAL LOCATION command to define a new external location named raw_data. I used the cloud storage URL provided by our team and associated it with the storage credential my_storage_credential, which had already been created and configured in the Databricks workspace.

CREATE EXTERNAL LOCATION raw_data URL 's3://my-bucket/raw-data/' WITH (STORAGE CREDENTIAL my_storage_credential);

I noticed that the command didn’t return any errors, which was a good sign. This meant the syntax was correct and the storage credential was valid. I verified that the external location was successfully created and ready for use.

Verifying the External Location

To make sure the external location was properly registered in Unity Catalog, I ran the SHOW EXTERNAL LOCATIONS command. This allowed me to see a list of all external locations in the current catalog.

SHOW EXTERNAL LOCATIONS;

I checked the output and confirmed that raw_data was listed. This step was important to ensure that the location was visible and accessible to other users and notebooks in the workspace.

Inspecting the External Location

I wanted to get more details about the raw_data external location, so I ran the DESCRIBE EXTERNAL LOCATION command. This helped me understand the configuration and properties of the location, such as the URL, storage credential, and any additional parameters.

DESCRIBE EXTERNAL LOCATION raw_data;

The output showed me that the location was correctly pointing to the specified cloud storage URL and was using the designated storage credential. This step was crucial for verifying that the external location was properly configured and aligned with our governance requirements.

Checking Access Permissions

As part of the governance process, I needed to ensure that the raw_data external location was accessible to the right users and roles. I ran the SHOW GRANTS ON EXTERNAL LOCATION command to review the current access permissions.

SHOW GRANTS ON EXTERNAL LOCATION raw_data;

The output listed the existing grants, which helped me confirm that the location was secure and that only authorized users had access. This was an important step in ensuring that the data was governed and protected according to our policies.

By following these steps, I successfully created and inspected a governed external location in Databricks Unity Catalog. This process ensured that the raw data was accessible, secure, and aligned with our data governance strategy.