profession:cloud engineer
-
Creating and Managing Managed Volumes for Non-Tabular Files
POST_START Creating and Managing Managed Volumes for Non-Tabular Files I started my day by reviewing the data pipeline for the sales team. The analysts needed to process a large number of non-tabular files—PDFs, Word documents, and CSVs—stored in a shared directory. These files were not structured in a way that could be easily queried using…
A1, category:business, category:education, category:software, category:technology, coding, creating, data engineering, data governance, databricks, education, files, Lakehouse, Managed, Managing, non-tabular, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, unity catalog, volumes -
Creating and Managing External Volumes
POST_START Creating and Managing External Volumes in Databricks Unity Catalog I recently needed to set up a new data pipeline for the sales team, and part of that involved organizing raw files stored in an external location. I decided to use Databricks Unity Catalog to create an external volume that would make it easier to…
A1, category:business, category:education, category:software, category:technology, coding, creating, data engineering, data governance, databricks, education, External, Lakehouse, Managing, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, unity catalog, volumes -
Managing Read, Write, and Table-Creation Access on External Locations
POST_START Managing Read, Write, and Table-Creation Access on External Locations I started my day by reviewing the access control requirements for our raw data location in Unity Catalog. The team needed to grant specific permissions to the `data_engineers` role—read access to files, write access to files, and the ability to create external tables. I knew…
A1, Access, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, External, Lakehouse, locations, Managing, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, read, sql, table-creation, unity catalog, Write -
Creating and Managing External Delta Tables
POST_START Creating and Managing External Delta Tables with Unity Catalog Creating and Managing External Delta Tables with Unity Catalog I recently needed to integrate external data into my Delta Lake pipeline using Databricks Unity Catalog. The data came from a third-party system, and it was stored in a shared location that wasn’t managed by Databricks.…
A1, category:business, category:education, category:software, category:technology, coding, creating, data engineering, data governance, databricks, Delta, education, External, Lakehouse, Managing, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, tables, unity catalog -
Creating and Inspecting Governed External Locations
POST_START Creating and Inspecting Governed External Locations Creating and Inspecting Governed External Locations I started my day by setting up a new external location in Databricks Unity Catalog to access raw data stored in cloud storage. This was part of a larger initiative to centralize data access and ensure all data sources are properly governed…
A1, category:business, category:education, category:software, category:technology, coding, creating, data engineering, data governance, databricks, education, External, governed, inspecting, Lakehouse, locations, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, unity catalog -
Inspecting Storage Credentials for External Data Access
POST_START Inspecting Storage Credentials for External Data Access Inspecting Storage Credentials for External Data Access I recently needed to inspect the storage credentials used by my team to access external data sources. Our data pipeline relies on several external systems, and ensuring that the credentials are correctly configured and secured is crucial for both performance…
A1, Access, category:business, category:education, category:software, category:technology, coding, credentials, data, data engineering, data governance, databricks, education, External, inspecting, Lakehouse, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, Storage, unity catalog -
Transferring Ownership of Unity Catalog Objects
POST_START Transferring Ownership of Unity Catalog Objects I recently took on the responsibility of managing access to a set of Unity Catalog objects within our organization. As part of a team restructuring, several schemas, tables, and views needed to be reassigned to different teams to ensure proper ownership and governance. My task was to transfer…
A1, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, Lakehouse, objects:, ownership, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, transferring, unity catalog -
Managing Schema-Level SELECT Access Across Tables
POST_START Managing Schema-Level SELECT Access Across Tables I recently had to set up access controls for a team of analysts who needed to query a set of tables in the training.sales schema. Since the analysts would be working with multiple tables within this schema, I decided to grant them schema-level SELECT access rather than individual…
A1, Access, Across, category:business, category:education, category:software, category:technology, coding, data engineering, data governance, databricks, education, Lakehouse, Managing, paragraph ordering, paragraph structure, paragraphorder, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, schema-level, select, sql, tables, unity catalog -
Controlling Table Data Changes with MODIFY Privileges
POST_START Controlling Table Data Changes with MODIFY Privileges I recently had the task of ensuring that only specific data engineers could modify the customer data in our training database. The goal was to prevent accidental or unauthorized changes to the training.sales.customers table while still allowing the team to perform necessary updates. I decided to use…
A1, category:business, category:education, category:software, category:technology, changes, coding, controlling, data, data engineering, data governance, databricks, education, Lakehouse, modify, paragraph ordering, paragraph structure, paragraphorder, privileges, profession:cloud engineer, profession:data analyst, profession:data engineer, profession:data scientist, profession:database administrator, profession:software engineer, profession:solution architect, progressive learning, sql, table, unity catalog
