Databricks Unity Catalog Tutorial: Identify Incorrect Tag Inheritance Leading to Governance Violations

This tutorial demonstrates how to identify incorrect tag inheritance within Databricks Unity Catalog using Databricks SQL. We’ll focus on a scenario where a table’s tags are being incorrectly applied through inheritance, leading to potential governance violations. This emphasizes Unity Catalog’s role in managing and enforcing data governance policies.

Script 1: Initial Setup and Tag Definition

This script creates a simple table and its associated tags within the Unity Catalog. We’ll establish the initial tag structure and demonstrate how tags can be assigned directly to a table.

CREATE TABLE IF NOT EXISTS sales_data (
  sale_id INT
)
;

CREATE TABLE IF NOT EXISTS customer_data (
  customer_id INT
)
;

CREATE TABLE IF NOT EXISTS sales_data_customer_relationship (
  sale_id INT,
  customer_id INT
)
ENGINE = Delta;

-- Create a Tag Catalog
CREATE CATALOG IF NOT EXISTS governance_tags;

-- Create a Tag
CREATE TAG IF NOT EXISTS sensitive_data
  IN governance_tags
  AS
  SELECT  FROM governance_tags.all_tables;

-- Assign the 'sensitive_data' tag to the 'sales_data' table
ALTER TABLE IF EXISTS sales_data
  SET TAG governance_tags.sensitive_data;

-- Assign the 'sensitive_data' tag to the 'customer_data' table
ALTER TABLE IF EXISTS customer_data
  SET TAG governance_tags.sensitive_data;

-- Assign the 'sales_data' tag to the 'sales_data_customer_relationship' table
ALTER TABLE IF EXISTS sales_data_customer_relationship
  SET TAG governance_tags.sales_data;

SELECT  FROM governance_tags.sensitive_data;

Explanation:

• We first create two sample tables, `sales_data` and `customer_data`, and a relationship table, `sales_data_customer_relationship`.
• Then, we create a catalog named `governance_tags` specifically for managing tags.
• Within `governance_tags`, we define a tag named `sensitive_data` that includes all tables from the catalog.
• We assign the `sensitive_data` tag to both `sales_data` and `customer_data`.
• Finally, we assign the `sales_data` tag to the relationship table.

This establishes the baseline tag structure. The critical point is that both `sales_data` and `customer_data` now inherit the `sensitive_data` tag due to this direct assignment.

-- Query to verify tag inheritance
SELECT  FROM governance_tags.sensitive_data;

Output:

<div class="output">
[
  {
    "schema": "default",
    "table": "sales_data",
    "catalog": "default"
  },
  {
    "schema": "default",
    "table": "customer_data",
    "catalog": "default"
  }
]
</div>

Script 2: Introducing Incorrect Tag Assignment

This script demonstrates an incorrect tag assignment – assigning a tag to a table that should not inherit it, leading to a potential governance violation. We’ll then query to confirm the inheritance.

-- Introduce a misconfiguration: Assign 'internal' tag to sales_data
ALTER TABLE IF EXISTS sales_data
  SET TAG governance_tags.internal;

SELECT  FROM governance_tags.sensitive_data;

Explanation:

• We introduce an error by assigning the `internal` tag to the `sales_data` table, which is intended to be a ‘sensitive’ table.
• We then execute a query against `governance_tags.sensitive_data` to verify the tag inheritance.

-- Query to verify tag inheritance after the incorrect assignment
SELECT  FROM governance_tags.sensitive_data;

Output:

<div class="output">
[
  {
    "schema": "default",
    "table": "sales_data",
    "catalog": "default"
  },
  {
    "schema": "default",
    "table": "customer_data",
    "catalog": "default"
  }
]
</div>

The output clearly shows that `sales_data` now appears in `governance_tags.sensitive_data`, despite not having the tag directly assigned to it. This is the incorrect inheritance.

Script 3: Verification and Remediation Suggestion

This script verifies the incorrect inheritance and suggests a remediation strategy – removing the incorrect tag assignment.

-- Verify the misconfiguration
SELECT  FROM governance_tags.sensitive_data;

-- Suggest remediation:  Remove the 'internal' tag from sales_data
ALTER TABLE IF EXISTS sales_data
  REMOVE TAG governance_tags.internal;

SELECT  FROM governance_tags.sensitive_data;

Explanation:

• We re-query `governance_tags.sensitive_data` to confirm the misconfiguration persists.
• We then remove the `internal` tag from the `sales_data` table.
• Finally, we query `governance_tags.sensitive_data` again to confirm the inheritance is corrected.

-- Query to verify the corrected tag inheritance
SELECT  FROM governance_tags.sensitive_data;

Output:

<div class="output">
[
  {
    "schema": "default",
    "table": "customer_data",
    "catalog": "default"
  }
]
</div>

The final output shows that only `customer_data` remains in `governance_tags.sensitive_data`, indicating the successful correction of the inheritance issue.