Netflix Reworks Conductor for 420 Million Monthly Workflow Executions and 10X Larger Workflows

elseif has not written about this yet · InfoQ describes it this way

Netflix has reworked its Conductor workflow orchestration engine to handle larger workloads, increasing supported workflow size from about 2,500 to 30,000 tasks and reducing p99 workflow evaluation latency by about 40%. Conductor 4.0 separates workflow metadata from task data, moves evaluation to asynchronous processing, and introduces dynamic worker allocation and concurrency controls. By Leela Kumili