TECH Signal 243 2 feeds carried it
Dataflow Model authors call triggers over-engineered, say stream-table equivalence was missed
↗
Illustration only Photo by Adi Goldstein on Unsplash
The original Dataflow Model authors revisit their eleven-year-old paper on the occasion of its VLDB Test of Time award, finding core foundations sound but acknowledging that triggers were over-engineered and the stream-centric worldview missed that streams and tables are two representations of the same object.
For engineers building streaming or analytical systems, this retrospective confirms that event-time processing and strong consistency remain essential, while suggesting SQL-based approaches with materialized views and incremental maintenance have proven more practical than complex trigger mechanisms for analytical workloads.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
The core foundations of the Dataflow Model, event time primacy, futility of waiting for completeness, and strong consistency, aged well.
Triggers were an over-engineered answer to a question users should never have faced, and the stream-centric worldview missed that streams and tables are equivalent representations with different access semantics.
The mechanisms that delivered on the paper's analytical goals came from the database playbook: SQL, incremental view maintenance, and materialized views with explicit freshness contracts.
THE CLUSTER