LANGUAGES Signal 396
Author shares techniques to achieve 500.000 lines per second compilation speed
↗
Illustration only Photo by Natalia Y. on Unsplash
The article explains how language design and implementation choices allow compilation rates of 500.000 lines per second on a single core.
Long compile times slow down iteration and frustrate developers, as seen with Rust’s common complaint. Fast compilation allows whole-program builds in less than 200ms, reducing the need for complex separate compilation schemes.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
Using a context-free grammar with a simple recursive descent parser speeds up the front-end.
Storing strings as (start_pointer, size) pairs avoids copying and lets substrings reference the input buffer directly.
Managing AST, objects and code generation in separate memory regions, with region pools for temporaries, makes allocation and de-allocation O(1).
THE READ
What the cluster adds up to.
The author targets a compilation rate of at least 500.000 lines of code per second on a single CPU core. To reach this rate, the article advocates designing the language itself for fast compilation. A context free grammar that works with a simple recursive descent parser is chosen. This grammar keeps parsing work minimal and human-readable.
Strings are represented by a pointer and length instead of zero-terminated buffers to avoid copying. Struct layout, flag packing, and small enums are used to lower memory footprint and improve cache fit. Memory regions hold the AST, program objects and generated code, with pools reused for temporary scopes. Error-related work is postponed until an actual error occurs, keeping the frequent path lean.
When compiling an existing language such as C++, the author notes that grammar, preprocessor and module system limit the applicability of these optimizations. Identifiers are handled as integer IDs rather than strings to avoid costly comparison and hashing. Memory regions require whole-region deallocation, making them inappropriate for objects with independent lifetimes. Consequently, the presented optimizations are most effective for newly designed languages where the designer can enforce simple syntax and region-based allocation.
Written by elseif from the cluster below · checked for specifics the sources never containedTHE CLUSTER