Topic Overview

One of the biggest advantages of using Databricks is that it takes care of a lot of the heavy lifting when it comes to how your data is physically stored and queried. In traditional data warehouses and even early data lake setups, you had to manually think about things like partitioning strategies, file sizes, and indexing. With Delta Lake on Databricks, many of these concerns are handled automatically or with minimal configuration.

The exam wants you to understand which features exist to simplify data layout decisions and boost query performance. This includes things like Liquid Clustering, Predictive Optimization, data skipping, and file compaction. You do not need to memorize every internal detail, but you should know what each feature does, when it kicks in, and why it matters for performance.

At the Associate level, think of this section as understanding the "what" and "why" rather than deep implementation. Know what tools Databricks gives you so your data is laid out efficiently, and know how that translates into faster queries without you having to manually tune everything.

Key Concepts

• Delta Lake and the Lakehouse Format
• Liquid Clustering
• Predictive Optimization
• Data Skipping
• File Compaction (OPTIMIZE)
• Deletion Vectors

Code Examples

Creating a Table with Liquid Clustering

-- Create a new table with Liquid Clustering
CREATE TABLE catalog.schema.sales (
  sale_id BIGINT,
  sale_date DATE,
  region STRING,
  product_id INT,
  amount DECIMAL(10,2)
)
CLUSTER BY (region, sale_date);

-- Change clustering keys on an existing table (no rewrite needed)
ALTER TABLE catalog.schema.sales CLUSTER BY (product_id, sale_date);

-- Remove clustering entirely
ALTER TABLE catalog.schema.sales CLUSTER BY NONE;

Running OPTIMIZE Manually

-- Compact small files into larger ones
OPTIMIZE catalog.schema.sales;

-- VACUUM removes old files no longer referenced by the transaction log
-- Default retention is 7 days
VACUUM catalog.schema.sales;

-- Check table details including file count and size
DESCRIBE DETAIL catalog.schema.sales;

Enabling Predictive Optimization

-- Enable Predictive Optimization at schema level
ALTER SCHEMA catalog.schema
ENABLE PREDICTIVE OPTIMIZATION;

-- Enable at catalog level (applies to all schemas and tables)
ALTER CATALOG my_catalog
ENABLE PREDICTIVE OPTIMIZATION;