DATABASES Signal 286
nixpkgs-multiverse explores three ways to query SQLite from inside Nix evaluation
↗
Illustration only Photo by Mario Gogh on Unsplash
Farid Zakaria's blog post describes three approaches to use SQLite as a query backend from within Nix, motivated by the fact that builtins.fromJSON eagerly parses entire JSON files even for single-key lookups.
Nix has no builtins.sqlite, and its JSON parsing is eager, meaning a 5.3 MB index file is fully parsed and materialized on the Nix heap for every lookup. For projects like nixpkgs-multiverse that index 305,492 package versions, this makes JSON impractical as the index grows. The post outlines escape hatches that trade safety or ergonomics for query efficiency.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
builtins.fromJSON is eager with no lazy or streaming parse, so a single attribute lookup materializes all 305,492 values from the 5.3 MB versions.json onto the Nix heap.
builtins.exec, available since Nix 1.11.9, can shell out to sqlite3 and parse its stdout as a Nix expression, but each query forks a process and re-parses output.
builtins.importNative, available since Nix 1.8, can dlopen a shared object exposing a C++ Nix PrimOp, enabling a native SQLite query function but requiring compilation against the Nix C++ API.
THE CLUSTER