Table Stream

Table Stream is the change data capture (CDC) mechanism in Lakehouse. It captures INSERT, UPDATE, and DELETE changes on a table for downstream tasks to consume.

For a detailed introduction, see Table Stream Object Model.

In This Chapter

Page	Description
TABLE STREAM Overview	Full concepts, how it works, and best practices
CREATE TABLE STREAM	Create a stream; supports STANDARD and APPEND_ONLY modes
DESC TABLE STREAM	View stream details, including offset position and stale status
SHOW TABLE STREAMS	List all streams in the current schema
DROP TABLE STREAM	Drop a stream (source table data is unaffected)

Common Operations

Create a Stream

-- STANDARD mode: captures all DML changes (INSERT / UPDATE / DELETE) CREATE TABLE STREAM orders_stream ON TABLE orders WITH PROPERTIES ('TABLE_STREAM_MODE' = 'STANDARD'); -- APPEND_ONLY mode: captures only INSERT; suited for append-only log tables CREATE TABLE STREAM events_stream ON TABLE events WITH PROPERTIES ('TABLE_STREAM_MODE' = 'APPEND_ONLY');

Consume a Stream

-- Consume via INSERT INTO ... SELECT; offset advances automatically INSERT INTO dwd_orders SELECT order_id, user_id, amount, status FROM orders_stream;

⚠️ Note: SELECT alone does not consume: querying a stream without a DML statement does not advance the offset. The offset only advances through DML operations (INSERT/MERGE).

⚠️ Note: WHERE filters cause data loss: changes filtered out are permanently lost and cannot be re-consumed. When selective consumption is needed, write all changes to an intermediate table first, then filter.

View and Drop

-- List streams SHOW TABLE STREAMS; -- View stream details (including offset and stale status) DESC TABLE STREAM orders_stream; -- Drop a stream DROP TABLE STREAM orders_stream;

Document	Description
Table Stream Object Model	Core concepts, selection scenarios, FAQs, cost notes
Table Stream Best Practices	Configuration recommendations for production environments
Real-Time Pipeline Selection Guide	Comparison of Pipe / Stream / Dynamic Table