GCC removes trampolines for nested functions without capture in latest release

The event centers on GCC’s evolving handling of nested functions, a feature that traditionally required runtime trampolines. These trampolines posed security risks by requiring executable stacks or incurred performance penalties via heap allocations. The latest changes address these issues by distinguishing between nested functions that capture parent variables and those that do not. For the latter, GCC now guarantees trampoline-free execution, even in unoptimized builds, which simplifies writing secure callback code without manual workarounds.

For nested functions that *do* capture parent variables, the solution involves new built-ins (`__builtin_call_code_address` and `__builtin_call_static_chain`) merged into GCC’s development branch. These built-ins allow the compiler to pass captured data via a reserved register, eliminating the need for trampolines. The approach mirrors manual implementations of closures but automates the boilerplate, reducing the risk of errors. This change aligns with the ABI’s existing conventions, ensuring compatibility with other languages that use static chains.

The practical impact is twofold. First, engineers can now write nested functions for callbacks without worrying about executable stacks or heap leaks, provided they avoid capturing parent variables. Second, the built-ins for captured variables offer a path to trampoline-free execution in future GCC releases, though adoption requires using non-standard extensions. The trade-off is portability: while GCC 16’s guarantee is backward-compatible, the built-ins are GCC-specific and may not work with other compilers like Clang.

Performance and security are the primary beneficiaries. Trampoline-free nested functions reduce attack surfaces by avoiding executable stacks and eliminate heap allocation overhead. The built-ins also enable optimizations like the example where a nested function call collapses into a single arithmetic instruction. However, the reliance on ABI-specific registers means the feature may not be portable to architectures without reserved static-chain registers, limiting its use in cross-platform codebases.