release: merge release 2.94.2 back to mainline

This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh

Change-Id: I01c4b3caeaf2c180d69b406b0dbd663ed1e4392d

doc/manual/change-authors.yml

@@ -73,6 +73,9 @@ detroyejr:
   display_name: Jonathan De Troye
   github: detroyejr
 
+edef:
+  github: edef1c
+
 edolstra:
   display_name: Eelco Dolstra
   github: edolstra
@@ -86,6 +89,11 @@ ericson:
   display_name: John Ericson
   github: ericson2314
 
+getchoo:
+  display_name: Seth Flynn
+  forgejo: getchoo
+  github: getchoo
+
 gilice:
   forgejo: gilice
 
@@ -224,6 +232,9 @@ roberth:
   display_name: Robert Hensing
   github: roberth
 
+sandydoo:
+  github: sandydoo
+
 seppel3210:
   github: Seppel3210
 

doc/manual/rl-next/arg-identifier.md

@@ -1,16 +0,0 @@
----
-synopsis: First argument to `--arg`/`--argstr` must be a valid Nix identifier
-issues: [fj#496]
-category: "Breaking Changes"
-credits: [ma27]
----
-
-The first argument to `--arg`/`--argstr` must be a valid Nix identifier, i.e.
-`nix-build --arg config.allowUnfree true` is now rejected.
-
-This is because that invocation is a false friend since it doesn't set
-`{ config = { allowUnfree = true; }; }`, but `{ "config.allowUnfree" = true; }`.
-
-The idea is to change the behavior to the latter in the long-term. For that,
-non-identifiers started giving a warning since 2.92 and are now rejected to give people
-who depend on that a chance to notice and potentially weigh in on the discussion.

doc/manual/rl-next/async-query.md

@@ -1,20 +0,0 @@
----
-synopsis: "Improved susbtituter query speed"
-issues: []
-cls: []
-category: Improvements
-credits: [horrors]
----
-
-The code used to query substituters for derivations has been rewritten slightly
-to take advantage of our asynchronous runtime. Such queries run for every build
-that could download from substituters and processes every derivation that isn't
-yet present on the local system. Previously Lix would use `http-connections` to
-limit query concurrency, even for modern caches that support HTTP/2 and have no
-limit on how many queries can be run concurrently on one single connection. Lix
-no longer does this, resulting in approximately 60% reduction in query time for
-medium-sized closures (e.g. NixOS system closures) during testing, although the
-exact number depends greatly on local network latency and generally improves as
-latency increases. Unlike previously setting `http-connections` to `1` or other
-low values no longer brings a massive penalty in query performance if the cache
-in use by the querying system supports HTTP/2 (as e.g. `cache.nixos.org` does).

doc/manual/rl-next/build-dir-mandatory.md

@@ -1,12 +0,0 @@
----
-synopsis: "`build-dir` no longer defaults to `temp-dir`"
-cls: [3453]
-category: "Fixes"
-credits: [horrors]
----
-
-The directory in which temporary build directories are created no longer defaults
-to the value of the `temp-dir` setting to avoid builders making their directories
-world-accessible. This behavior has been used to escape the build sandbox and can
-cause build impurities even when not used maliciously. We now default to `builds`
-in `NIX_STATE_DIR` (which is `/nix/var/nix/builds` in the default configuration).

doc/manual/rl-next/certificate-authorities.md

@@ -1,60 +0,0 @@
----
-synopsis: "Global certificate authorities are copied inside the builder's environment"
-issues: [gh#12698, fj#885]
-cls: [3765]
-category: Fixes
-credits: [raito, emilazy]
----
-
-Previously, CA certificates were only installed at
-`/etc/ssl/certs/ca-certificates.crt` for sandboxed builds on Linux.
-
-This setup was insufficient in light of recent changes in `nixpkgs`, which now
-enforce HTTPS usage for `fetchurl`, even for fixed-output derivations, to
-mitigate confidentiality risks such as `netrc` or credentials leakage.
-`nixpkgs` still make use of a special package called `cacerts` which contains a
-copy of the CA certificates maintained by Nixpkgs and added as a reference for
-TLS-enabled fetchers.
-
-As a result, having a consistent and trusted certificate authority in all
-builder environments is becoming more essential.
-
-On `nix-darwin`, the `NIX_SSL_CERT_FILE` environment variable is always
-explicitly defined, but it is ignored by the sandbox setup.
-
-Simultaneously, Nix evaluates and propagates impure environment variables via
-`lib.proxyImpureEnvVars`, meaning that if `NIX_SSL_CERT_FILE` is set (which
-influences the default value for `ssl-cert-file`), it will be forwarded
-unchanged into the builder environment.
-
-However, on Linux, Nix also *copies* the CA file into the sandbox, creating a
-discrepancy between the value of `NIX_SSL_CERT_FILE` and the actual trusted
-certificate path used during the build.
-
-This divergence caused confusion and was partially addressed by attempts to
-whitelist the CA path in the Darwin sandbox (see cl/2906), but that approach
-involved a non-trivial path canonicalization step and is not as general as this one.
-
-To address this properly, we now emit a warning and override
-`NIX_SSL_CERT_FILE` inside the builder, explicitly pointing it to the CA file
-copied into the sandbox.
-
-This eliminates ambiguity between `NIX_SSL_CERT_FILE`
-and `ssl-cert-file`, ensuring consistent trust anchors across platforms.
-
-This warning might become a hard error as we figure out what to do regarding
-`lib.proxyImpureEnvVars` in nixpkgs.
-
-The behavior has been verified across sandboxed and unsandboxed builds on both
-Linux and Darwin.
-
-As a consequence of this change, approximately 500 KB of CA certificate data is
-now unconditionally copied into the build directory for fixed-output
-derivations.
-
-While this ensures consistent trust verification without having to restart the
-daemon after system upgrades, it may introduce a slight overhead in build
-performance. At present, no optimizations have been implemented to avoid this
-copy, but if this overhead proves noticeable in your workflows, please open an
-issue so we can evaluate and possibly implement different strategies to render
-trust anchors visible.

doc/manual/rl-next/cgroups-delegation.md

@@ -1,69 +0,0 @@
----
-synopsis: New cgroup delegation model
-issues: [fj#537, fj#77]
-cls: [3230]
-category: "Breaking Changes"
-credits: [raito, horrors, lheckemann]
----
-
-Builds using cgroups (i.e. `use-cgroups = true` and the experimental feature
-`cgroups`) now always delegate a cgroup tree to the sandbox.
-
-Compared to the original C++ Nix project, our delegation includes the
-`subtree_control` file as well, which means that the sandbox can disable
-certain controllers in its own cgroup tree.
-
-This is a breaking change because this requires the Nix daemon to run with an
-already delegated cgroup tree by the service manager.
-
-## How to setup the cgroup tree with systemd?
-
-systemd offers knobs to perform the required setup using:
-
-```
-[Service]
-Delegate=yes
-DelegateSubtree=supervisor
-```
-
-These directives are now included in our systemd packaging.
-
-## What about using Nix as root without connecting to the daemon?
-
-Builds run as `root` without connecting to the daemon relying on the cgroup
-feature are now broken, i.e.
-
-```console
-# nix-build --use-cgroups --sandbox ... # will not work
-```
-
-Consider doing instead:
-
-```console
-# systemd-run --same-dir --wait -p Delegate=yes -p DelegateSubgroup=supervisor nix-build --use-cgroups ...
-```
-
-If you need to disable cgroups temporarily, remember that you can do
-`NIX_CONF='include /etc/nix/nix.conf\nuse-cgroups = false' nix-build ...` or
-`nix-build --no-use-cgroups ...`.
-
-## What about other service managers than systemd?
-
-systemd has a [documentation](https://systemd.io/CGROUP_DELEGATION/) on how to
-handle cgroup delegation from service management perspective.
-
-If your service manager adheres to systemd semantics, e.g. writing an extended
-attribute `user.delegate=1` on the delegated cgroup tree directory and moving
-the `nix-daemon` process inside a cgroup tree to respect the inner process
-rule, then, the feature will work as well.
-
-## Why is the cgroup feature still experimental?
-
-While the cgroup feature unlocks many use cases, its behavior and integration (e.g. user experience), especially at scale on build farms or in multi-tenant environments, are not yet fully matured. There’s also potential for deeper systemd integration (e.g. using slices and scopes) that has not been fully explored.
-
-To avoid locking in an unstable interface, we’re keeping the experimental flag until we have validated the feature across a broader range of scenarios, including but not limited to:
-
-* Nix as root
-* Hydra-style build farms
-* Forgejo CI runners
-* Shared remote builders

doc/manual/rl-next/deprecated-derivation-types.md

@@ -1,16 +0,0 @@
----
-synopsis: Deprecation of CA derivations, dynamic derivations, and impure derivations
-issues: [fj#815]
-cls: []
-significance: significant
-category: Miscellany
-credits: []
----
-
-Content-addressed derivations are now deprecated and slated for removal in Lix 2.94.
-We're doing this because the CA derivation system has been a known cause of problems
-and inconsistencies, is unmaintained, habitually makes improving the store code very
-difficult (or blocks such improvements outright), and is beset by a number of design
-flaws that in our opinion cannot be fixed without a full reimplementation from zero.
-Dynamic derivations and impure derivations are built on the CA derivation framework,
-and owing to this they too are deprecated and slated for removal in another release.

doc/manual/rl-next/double-sigint.md

@@ -1,25 +0,0 @@
----
-synopsis: "Hitting Control-C twice always terminates Lix"
-cls: [3574]
-issues: []
-category: "Improvements"
-credits: [horrors]
----
-
-Hitting Control-C or sending `SIGINT` to Lix now prints an informational message
-if it is still running after on second, the second Control-C/`SIGINT` terminates
-Lix immediately without waiting for any shutdown code to finish running. Lix did
-not treat the second such event differently from first in the past; this made it
-impossible to easily terminate running Lix processes that got stuck in e.g. very
-expensive Nixlang code that never interacted with the store. We now terminate as
-soon as the user hits Control-C again without waiting any more, to much the same
-effect as putting Lix into the background and killing it immediately afterwards.
-
-This means you can now more conveniently break out of stuck Nixlang evaluations:
-```
-❯ nix-instantiate --eval --expr 'let f = n: if n == 0 then 0 else f (n - 1) + f (n - 1); in f 32'
-^CStill shutting down. Press ^C again to abort all operations immediately.
-^C
-
-❌130 ❯
-```

doc/manual/rl-next/exponential-backoff.md

@@ -1,12 +0,0 @@
----
-synopsis: "libstore: exponential backoff for downloads"
-issues: [lix#932]
-cls: [3856]
-category: Fixes
-credits: [ma27]
----
-
-The connection timeout when downloading from e.g. a binary cache is exponentially
-increased per failure. The option `connect-timeout` is now an alias to `max-connect-timeout`
-which is the maximum value for a timeout. The start value is controlled
-by `initial-connect-timeout` which is `5` by default.

doc/manual/rl-next/fix-develop-esc.md

@@ -1,9 +0,0 @@
----
-synopsis: Fix develop shells for derivations with escape codes
-issues: [fj#991]
-cls: [4154, 4155]
-category: Fixes
-credits: [Qyriad]
----
-
-ASCII control characters (including `\e`, used for ANSI escape codes) in derivation variables are now correctly escaped for `nix develop` and `nix print-dev-env`, instead of erroring.

doc/manual/rl-next/fix-gc-links.md

@@ -1,13 +0,0 @@
----
-synopsis: "nix-store --delete: always remove obsolete hardlinks"
-issues: []
-cls: [3188]
-category: Fixes
-credits: [lheckemann]
----
-
-Deleting specific paths using `nix-store --delete` or `nix store
-delete` previously did not delete hard links created by `nix-store
---optimise` even if they became obsolete, unless _all_ of the given
-paths were deleted successfully. Now, hard links are always cleaned
-up, even if some of the given paths could not be deleted.

doc/manual/rl-next/fix-gc-stats.md

@@ -1,24 +0,0 @@
----
-synopsis: "Report GC statistics correctly"
-issues: []
-cls: [3188]
-category: Fixes
-credits: [lheckemann]
----
-
-Deleting specific paths using `nix-store --delete` or `nix store delete` previously did
-not report statistics correctly when some of the paths could not be deleted, even if
-others were deleted:
-
-```
-$ nix store delete /nix/store/9bwryidal9q3g91cjm6xschfn4ikd82q-hello-2.12.1 --delete-closure -v
-finding garbage collector roots...
-deleting '/nix/store/9bwryidal9q3g91cjm6xschfn4ikd82q-hello-2.12.1'
-0 store paths deleted, 0.00 MiB freed
-error: Cannot delete some of the given paths because they are still alive. Paths not deleted:
-         k9bxzr1l92r5y6mihrkbpbr3fmc8qszx-libidn2-2.3.8
-         mbx9ii53lzjlrsnlrfmzpwm33ynljwdn-libunistring-1.3
-         rf8hcy6bldxdqc0g6q1dcka1vh47x69s-xgcc-14.2.1.20250322-libgcc
-         vbrdc5wgzn0w1zdp10xd2favkjn5fk7y-glibc-2.40-66
-       To find out why, use nix-store --query --roots and nix-store --query --referrers.
-```

doc/manual/rl-next/flake-self-attrs.md

@@ -1,16 +0,0 @@
----
-synopsis: Add `inputs.self.submodules` flake attribute
-issues: [fj#942]
-cls: [3839]
-category: Features
-credits: [edolstra, kasimeka]
----
-
-A port of <https://github.com/NixOS/nix/pull/12421> to Lix, which:
-
-- adds a general `inputs.self` flake attribute that retroactively applies
-  configurations to a flake after it's been fetched, then triggers a refetch of
-  the flake with the new config.
-- implements `inputs.self.submodules` that allows a flake to declare its need
-  for submodules, which are then fetched automatically with no need to pass
-  `?submodules=1` anywhere.

doc/manual/rl-next/hyperlinks-in-attrsets.md

@@ -1,16 +0,0 @@
----
-synopsis: Add hyperlinks in attr set printing
-issues: []
-cls: [3790]
-category: Features
-credits: [jade]
----
-
-The attribute set printer, such as is seen in `nix repl` or in type errors, now prints hyperlinks on each attribute name to its definition site if it is known.
-
-Example: all of the attributes shown here are hyperlinks to the exact definition site of the attribute in question:
-
-```
-$ nix eval -f '<nixpkgs>' lib.licenses.mit
-{ deprecated = false; free = true; fullName = "MIT License"; redistributable = true; shortName = "mit"; spdxId = "MIT"; url = "https://spdx.org/licenses/MIT.html"; }
-```

doc/manual/rl-next/ignore-try-default.md

@@ -1,24 +0,0 @@
----
-synopsis: Repl debugger uses `--ignore-try` by default
-issues: [lix#666]
-cls: [3488]
-category: Breaking Changes
-credits: [jade]
----
-Previously, using the debugger meant that exceptions thrown in `builtins.tryEval` would trigger the debugger.
-
-However, this caught nixpkgs initialization code, which is unhelpful in the majority of cases, so we changed the default.
-
-To get the old behaviour, use `--no-ignore-try`.
-
-```
-$ nix repl --debugger --expr 'with import <nixpkgs> {}; pkgs.hello'
-Lix 2.94.0-dev-pre20250625-9a59106
-Type :? for help.
-error: file 'nixpkgs-overlays' was not found in the Nix search path (add it using $NIX_PATH or -I)
-
-This exception occurred in a 'tryEval' call. Use --ignore-try to skip these.
-
-Added 13 variables.
-nix-repl>
-```

doc/manual/rl-next/infallible-build-dirs.md

@@ -1,25 +0,0 @@
----
-synopsis: "Fallback to safe temp dir when build-dir is unwritable"
-issues: [fj#876]
-cls: [3501]
-category: "Fixes"
-credits: ["raito", "horrors"]
----
-
-Non-daemon builds started failing with a permission error after introducing the `build-dir` option:
-
-```
-$ nix build --store ~/scratch nixpkgs#hello --rebuild
-error: creating directory '/nix/var/nix/builds/nix-build-hello-2.12.2.drv-0': Permission denied
-```
-
-This happens because:
-
-1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
-2. The user lacks permissions for that path.
-
-We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
-
-To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
-
-**Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.

doc/manual/rl-next/integer-coercion.md

@@ -1,47 +0,0 @@
----
-synopsis: Experimental integer coercion in interpolated strings
-issues: []
-cls: [3198]
-category: "Features"
-credits: [raito, delroth, horrors, winter]
----
-
-Ever tried interpolating a port number in Lix and ended up with something like this?
-
-```nix
-"http://${config.network.host}:${builtins.toString config.network.port}/"
-```
-
-You're not alone. Thousands of Lix users suffer every day from excessive `builtins.toString` syndrome. It’s 2025, and we still have to cast integers to use them in strings.
-
-To address this, Lix introduces the **`coerce-integers`** experimental feature. When enabled, interpolated integers within `"${...}"` are automatically coerced to strings. This allows writing:
-
-```nix
-"http://${config.network.host}:${config.network.port}/"
-```
-
-without additional conversion.
-
-To enable the feature, you need to add `coerce-integers` to your set of experimental features.
-
-### Stabilization criteria
-
-The `coerce-integers` feature is experimental and limited strictly to string interpolation (`"${...}"`). Before stabilization, the following must hold:
-
-1. **Interpolation-only**
-   Coercion must not occur outside interpolation. Expressions like `"" + 42` must continue to fail.
-
-2. **Expectation that no explicit cast are being observed**
-   Cases observing explicit coercion (e.g., via `tryEval` gadget or similar) are expected not to be load-bearing in actual production code.
-
-### Timeline for stabilization
-
-If the feature proves safe and is widely adopted across typical usage (e.g., actual configurations in the wild turning on the flag, non-trivial out-of-tree projects using it), the experimental flag will be removed **after six months of active use or two Lix releases**, whichever is longer.
-
-This avoids locking the feature in experimental status indefinitely, as happened with Flakes, while allowing time for validation and ecosystem integration.
-
-### What about coercing floats or more?
-
-Coercion beyond integers -- such as for floats or other types -- is **not planned**, even under an experimental flag. Questions like "what is the canonical string representation of a float?" involve subtle and context-dependent trade-offs. Without a robust and principled mechanism to define and audit such behavior, introducing broader coercion risks setting unintended and hard-to-reverse precedents. The scope of `coerce-integers` is intentionally narrow and will remain so.
-
-In terms of outlook, a proposal like https://git.lix.systems/lix-project/lix/issues/835 could pave the way for a better solution.

doc/manual/rl-next/json-number-overflow.md

@@ -1,16 +0,0 @@
----
-synopsis: Parse overflowing JSON number literals as floating‐point
-issues: []
-cls: [3919]
-category: "Fixes"
-credits: [emilazy]
----
-
-Previously, `builtins.fromJSON "-9223372036854775809"` would
-return a floating‐point number, while `builtins.fromJSON
-"9223372036854775808"` would cause an evaluation error. This was
-introduced with the banning of integer overflow in Lix 2.91; previously
-the latter would result in C++ undefined behaviour. These cases are
-now treated consistently with JSON’s model of a single numeric type,
-and JSON number literals that do not fit in a Nix‐language integer
-will be parsed as floating‐point numbers.

doc/manual/rl-next/keep-failed-chown.md

@@ -1,13 +0,0 @@
----
-synopsis: "`--keep-failed` chowns the build directory to the user that request the build"
-issues: []
-cls: []
-category: Improvements
-credits: [horrors]
----
-
-Running a build with `--keep-failed` now chowns the temporary directory from the
-builder user and group to the user that request the build if the build came from
-a local user connected to the daemon. This makes inspecting failed derivations a
-lot easier. On Linux the build directory made visible to the user will not be in
-the same path as it was in the sandbox and continuing builds will usually break.

doc/manual/rl-next/log-output-terminal-codes.md

@@ -1,11 +0,0 @@
----
-synopsis: Fix handling of OSC codes in terminal output
-issues: [fj#160]
-cls: [3143]
-category: Fixes
-credits: [lilyball]
----
-
-OSC codes in terminal output are now handled correctly, where OSC 8 (hyperlink) is preserved any
-time color codes are allowed and all other OSC codes are stripped out. This applies not only to
-output from build commands but also to rendered documentation in the REPL.

doc/manual/rl-next/mismatch-check-path.md

@@ -1,14 +0,0 @@
----
-synopsis: Better debuggability on fixed-output hash mismatches
-issues: []
-cls: []
-category: Improvements
-credits: [lheckemann]
----
-
-Fixed-output derivation hash mismatch error messages will now include the path that was
-produced unexpectedly, and this path will be registered as valid even if `--check`
-(`nix-store`, `nix-build`) or `--rebuild` (`nix build`) was passed. This makes comparing
-the expected path with the obtained path easier, and is useful for debugging when
-upstreams modify previously-published releases or when changes in fixed-output
-derivations' dependencies affect their output unexpectedly.

doc/manual/rl-next/nej-no-instantiate.md

@@ -1,10 +0,0 @@
----
-synopsis: "nix-eval-jobs: support `--no-instantiate` flag"
-issues: [fj#987]
-category: Features
-credits: [mic92,ma27]
----
-
-`nix-eval-jobs` now supports a flag called `--no-instantiate`. With this enabled,
-no write operations on the eval store are performed. That means, only evaluation is
-performed, but derivations (and their gcroots) aren't created.

doc/manual/rl-next/nix-develop-output-checks.md

@@ -1,29 +0,0 @@
----
-synopsis: "Fix nix develop for derivations that rejects dependencies with structured attrs"
-issues: [fj#997]
-cls: [4182]
-category: Fixes
-credits: [raito]
----
-
-For the sake of concision, we refer to `disallowedReferences` in what follows,
-but all output checks were equally fixed:
-`{dis,}allowed{References,Requisites}`.
-
-Derivations can define *output checks* to reject unwanted dependencies, such as
-interpreters like `bash` or compilers like `gcc`. This can be done in two ways:
-
-* **Legacy style**: `disallowedReferences = [ ... ]` in the environment.
-* **Structured attrs**: `outputChecks.<output>.disallowedReferences = [ ... ]`,
-  typically used in `__json`.
-
-Only the structured form supports derivations with multiple outputs.
-
-`nix develop` internally rewrites derivations to create development shells. It
-relied on the legacy `disallowedReferences`, and failed to honor the structured
-variant. This led to broken shells in cases where `bashInteractive` was
-explicitly disallowed using structured output checks, e.g. `nix develop
-nixpkgs#systemd` after the "bash-less NixOS" changes.
-
-This fix teaches `nix develop` to respect structured output checks, restoring
-support for such derivations.

doc/manual/rl-next/nix-instantiate-raw.md

@@ -1,12 +0,0 @@
----
-synopsis: "Add --raw flag to `nix-instantiate --eval` for unescaped output"
-issues: []
-prs: [gh#12119]
-cls: [2886]
-category: Improvements
-credits: [not-my-profile, infinisil, raito]
----
-
-The `nix-instantiate --eval` command now supports a `--raw` flag. When used,
-the result must be coercible to a string (as with `${...}`) and is printed
-verbatim, without quotes or escaping.

doc/manual/rl-next/nix-store-ls-read-nar-listings.md

@@ -1,12 +0,0 @@
----
-synopsis: Allow `nix store ls` to read nar listings from binary cache stores.
-issues: []
-cls: [3225]
-category: Improvements
-credits: [vlinkz]
----
-
-The `nix store ls` command now supports reading `.ls` nar listings from binary cache stores.
-If a listing is detected for the store path being queried, the nar is no longer downloaded.
-These nar listings are available in binary cache stores where the `write-nar-listing` option is
-enabled, such as cache.nixos.org.

doc/manual/rl-next/nix_path-nix-eval-jobs.md

@@ -1,10 +0,0 @@
----
-synopsis: "nix-eval-jobs: retain NIX_PATH"
-issues: []
-cls: [3859]
-category: Fixes
-credits: [ma27,mic92]
----
-
-`nix-eval-jobs` doesn't clear the `NIX_PATH` from the environment anymore. This matches the behavior
-of [upstream version `2.30`](https://github.com/nix-community/nix-eval-jobs/releases/tag/v2.30.0).

doc/manual/rl-next/output-cycle-refs.md

@@ -1,25 +0,0 @@
----
-synopsis: "show tree with references that lead to an output cycle"
-issues: [fj#551]
-category: Improvements
-credits: [ma27]
----
-
-When Lix determines a cyclic dependency between several outputs of a derivation,
-it now displays which files in which outputs lead to an output cycle:
-
-```
-error: cycle detected in build of '/nix/store/gc5h2whz3rylpf34n99nswvqgkjkigmy-demo.drv' in the references of output 'bar' from output 'foo'.
-
-       Shown below are the files inside the outputs leading to the cycle:
-       /nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar
-       └───lib/libfoo: …stuffbefore /nix/store/h680k7k53rjl9p15g6h7kpym33250w0y-demo-baz andafter.…
-           → /nix/store/h680k7k53rjl9p15g6h7kpym33250w0y-demo-baz
-           └───share/snenskek: …???? /nix/store/dm24c76p9y2mrvmwgpmi64rryw6x5qmm-demo-foo ....…
-               → /nix/store/dm24c76p9y2mrvmwgpmi64rryw6x5qmm-demo-foo
-               └───bin/alarm: …textexttext/nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar abcabcabc.…
-                   → /nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar
-```
-
-Please note that showing the files and its contents while displaying the cycles only works
-on Linux.

doc/manual/rl-next/pasta.md

@@ -1,20 +0,0 @@
----
-synopsis: "Fixed output derivations can be run using `pasta` network isolation"
-cls: [3452]
-issues: [fj#285]
-category: "Breaking Changes"
-credits: [horrors, puck]
----
-
-Fixed output derivations traditionally run in the host network namespace.
-On Linux this allows such derivations to communicate with other sandboxes
-or the host using the abstract Unix domains socket namespace; this hasn't
-been unproblematic in the past and has been used in two distinct exploits
-to break out of the sandbox. For this reason fixed output derivations can
-now run in a network namespace (provided by [`pasta`]), restricted to TCP
-and UDP communication with the rest of the world. When enabled this could
-be a breaking change and we classify it as such, even though we don't yet
-enable or require such isolation by default. We may enforce this in later
-releases of Lix once we have sufficient confidence that breakage is rare.
-
-[`pasta`]: https://passt.top/

doc/manual/rl-next/reliance-on-bash-for-ssh.md

@@ -1,21 +0,0 @@
----
-synopsis: Remove reliance on Bash for remote stores via SSH
-issues: [fj#830, fj#805, fj#304]
-cls: [3159]
-category: "Fixes"
-credits: [raito]
----
-
-The pre-flight `echo started` handshake -- added years ago to catch race conditions -- has been removed.
-
-After removal of connection sharing in Lix 2.93, it required a Bash-compatible shell and a standard `echo`, so it failed on:
-
-  * builders protected by `ForceCommand` wrappers (e.g. `nix-remote-build`),
-  * BusyBox / initrd images with no Bash,
-  * hosts using non-POSIX shells such as Nushell.
-
-The race the probe once addressed was tied to SSH connection-sharing -- since connection-sharing code has already been removed, the probe is now pointless.
-
-Real connection or protocol errors are now left to SSH/Nix to report directly.
-
-This is technically a breaking change if you had scripts that relied on the literal "started" which needs to be updated to rely on other signals, e.g., exit codes.

doc/manual/rl-next/remove-old-wires.md

@@ -1,20 +0,0 @@
----
-synopsis: Remove support for daemon protocols before 2.18
-issues: [fj#510]
-cls: [3249]
-significance: significant
-category: "Breaking Changes"
-credits: [horrors]
----
-
-Support for daemon wire protocols belonging to Nix 2.17 or older have been
-removed. This impacts clients connecting to the local daemon socket or any
-remote builder configured using the `ssh-ng` protocol. Builders configured
-with the `ssh` protocol are still accessible from clients such as Nix 2.3.
-Additionally Lix will not be able to connect to an old daemon locally, and
-remote build connections to old daemons is likewise limited to `ssh` urls.
-
-We have decided to take this step because the old protocols are very badly
-tested (if at all), maintenance overhead is high, and a number of problems
-with their design makes it infeasible to remain backwards compatible while
-we move Lix to a more modern RPC mechanism with better versioning support.

doc/manual/rl-next/remove-write-to.md

@@ -1,14 +0,0 @@
----
-synopsis: "`nix eval --write-to` has been removed"
-cls: [4045]
-issues: [fj#974, fj#227]
-category: "Breaking Changes"
-credits: [horrors]
----
-
-`nix eval --write-to` has been removed since it was underspecified, not widely
-useful, and prone to security-sensitive misbehaviors. The feature was added in
-Nix 2.4 purely for internal use in the build system. According to our research
-it hasn't found any use outside of some distribution packaging scripts. Please
-use structured outputs formats (such as JSON) instead as they have better type
-fidelity, don't conflate attributes with paths, and are useful to other tools.

doc/manual/rl-next/remove-xp-derivations.md

@@ -1,17 +0,0 @@
----
-synopsis: Remove impure derivations and dynamic derivations
-issues: [fj#815]
-cls: [3210]
-significance: significant
-category: "Breaking Changes"
-credits: [horrors]
----
-
-The `impure-derivations` and `dynamic-derivations` experimental feature have
-been removed.
-
-New impure or dynamic derivations cannot be created from this point forward, and
-any such pre-existing store derivations canot be read or built any more.
-Derivation outputs created by building such a derivation are still valid
-until garbage collected; existing store derivations can only be garbage
-collected.

doc/manual/rl-next/remove-xp-parse-toml-timestamps.md

@@ -1,17 +0,0 @@
----
-synopsis: Remove the `parse-toml-timestamps` experimental feature
-category: "Breaking Changes"
-credits: [emilazy]
----
-
-The `parse-toml-timestamps` experimental feature has been removed.
-
-This feature used in‐band signalling to mark timestamps, making it
-impossible to unambiguously parse TOML documents. It also exposed
-implementation‐defined behaviour in the TOML specification that
-changed in the toml11 parser library.
-
-Any interface for parsing TOML timestamps suitable for future
-stabilization would necessarily involve breaking changes, and there
-is no evidence this experimental feature is being relied upon in the
-wild, so it has been removed.

doc/manual/rl-next/repl-overlays-pure.md

@@ -1,20 +0,0 @@
----
-synopsis: "repl-overlays now work in the debugger for flakes"
-issues: [fj#777]
-cls: [3398]
-category: Fixes
-credits: [jade]
----
-Due to a bug, it was previously not possible to use the debugger on flakes with repl-overlays, or with pure evaluation in general:
-
-```
-$ nix repl --pure-eval
-Lix 2.94.0-dev-pre20250617-87d99da
-Type :? for help.
-Loading 'repl-overlays'...
-error: access to absolute path '/Users/jade/.config/nix/repl.nix' is forbidden in pure eval mode (use '--impure' to override)
-```
-
-This is now fixed.
-The contents of the repl-overlays file itself (i.e. most typically the top level lambda in it) will be evaluated in impure mode.
-It may be necessary to use `builtins.seq` to force the impure operations to happen first if one wants to do impure operations inside a repl-overlays file in pure evaluation mode.

doc/manual/rl-next/report-forbidden-path-as-chain.md

@@ -1,19 +0,0 @@
----
-synopsis: "`disallowedRequisites` now reports chains of disallowed requisites"
-issues: [fj#334,fj#626,gh#10877]
-category: Improvements
-credits: [ma27,roberth]
----
-
-When a build fails because of [`disallowedRequisites`](@docroot@/language/advanced-attributes.md#adv-attr-disallowedRequisites), the error message now includes the chain of references that led to the failure. This makes it easier to see in which derivations the chain can be broken, to resolve the problem.
-
-Example:
-
-```
-$ nix-build -A hello
-error: output '/nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2' is not allowed to refer to the following paths:
-       /nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-glibc-2.40-66
-       Shown below are chains that lead to the forbidden path(s).
-       /nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2
-       └───/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-glibc-2.40-66
-```

doc/manual/rl-next/snix-http-cache-integration.md

@@ -1,17 +0,0 @@
----
-synopsis: "libstore/binary-cache-store: don't cache narinfo on nix copy, remove negative entry"
-issues: []
-cls: [3789]
-category: Fixes
-credits: [ma27]
----
-
-When using e.g. [Snix's nar-bridge](https://snix.dev/docs/components/overview/#nar-bridge) via
-an `http`-store, Lix would create cache entries with a wrong URL to the NAR when uploading
-a store-path.
-
-This caused hard build failures for Hydra.
-
-Lix doesn't create these entries on upload anymore. Instead, it only removes negative cache entries.
-The cache entry for a narinfo is now created the first time, Lix queries the cache
-for the previously uploaded store-path again.

doc/manual/rl-next/static-linking.md

@@ -1,10 +0,0 @@
----
-synopsis: "Lix libraries can now be linked statically"
-issues: [fj#789]
-cls: [3775, 3778]
-category: Fixes
-credits: [alois31]
----
-Previously the pkg-config files distributed with Lix were only suitable for dynamic linkage, causing "undefined reference to…" linker errors when trying to link statically.
-Private dependency information has now been added to make static linkage work as expected without user intervention.
-In addition, relevant static libraries are now prelinked to avoid strange failures due to missing static initializers.

doc/manual/rl-next/symbol-value-reuse.md

@@ -1,34 +0,0 @@
----
-synopsis: Symbols reuses once-allocated Value to reduce garbage collected allocations
-issues: []
-cls: [3308, 3300, 3314, 3310, 3312, 3313]
-category: Improvements
-credits: [raito, horrors, thubrecht, xokdvium, nan-git]
----
-
-In the Lix evaluator, **symbols** represent immutable strings, like those used
-for attribute names.
-
-In evaluator design, such strings are typically [**interned**](https://en.wikipedia.org/wiki/String_interning), stored uniquely
-to save memory, and Lix inherits this approach from the original C++ codebase.
-
-However, some builtins, like `builtins.attrNames`, must return a `Value` type
-that can represent any Nix value (strings, integers, lists, etc.).
-
-Before this change, these builtins would create lists of `Value` objects by
-allocating them through the garbage collector, copying the symbol’s string
-content each time.
-
-This allocation is unnecessary if the interned symbols themselves also hold a
-`Value` representation allocated outside the garbage collector, since these
-live for the full duration of evaluation.
-
-As a result, this reduces the number of allocations, leading to:
-
-* A significant drop in maximum [resident set memory](https://en.wikipedia.org/wiki/Resident_set_size) (RSS), with some large-scale
-  tests showing up to 11% (about 500 MiB) savings in large colmena deployments.
-* A slight decrease in CPU usage during Nix evaluations.
-
-This change is inspired by https://github.com/NixOS/nix/pull/13258 but the approach is different.
-
-**Note** : [`xokdvium`](https://github.com/xokdvium) is the rightful author of https://gerrit.lix.systems/c/lix/+/3300 and the credit was missed on our end during the development process. We are deeply sorry for this mistake.

doc/manual/rl-next/toml-number-overflow.md

@@ -1,14 +0,0 @@
----
-synopsis: Reject overflowing TOML integer literals
-issues: []
-cls: [3916]
-category: "Breaking Changes"
-credits: [emilazy]
----
-
-The toml11 library used by Lix was updated. The new
-version aligns with the [TOML v1.0.0 specification’s
-requirement](https://toml.io/en/v1.0.0#integer) to reject integer
-literals that cannot be losslessly parsed. This means that code like
-`builtins.fromTOML "v=0x8000000000000000"` will now produce an error
-rather than silently saturating the integer result.

doc/manual/rl-next/uid-range-requires-cgroups.md

@@ -1,15 +0,0 @@
----
-synopsis: uid-range depends on cgroups
-issues: []
-cls: [3230]
-category: "Breaking Changes"
-credits: [raito, horrors]
----
-
-`uid-range` builds now depends on `cgroups`, an experimental feature.
-
-`uid-range` builds already depended upon `auto-allocate-uids`, another experimental feature.
-
-The rationale for doing so is that `uid-range` provides a sandbox with many
-UIDs, this is useful for re-mapping them into a nested namespace, e.g. a
-container.

doc/manual/rl-next/zsh-completions-desc.md

@@ -1,10 +0,0 @@
----
-synopsis: add description to zsh completions
-issues: [fj#910]
-cls: [3632]
-category: "Fixes"
-credits: [matthewbauer]
----
-
-Emit descriptions when completing args in zsh completions. This uses the descriptions we already
-provided in NIX\_GET\_COMPLETIONS.

doc/manual/src/SUMMARY.md

@@ -39,6 +39,9 @@
   - [Tuning Cores and Jobs](advanced-topics/cores-vs-jobs.md)
   - [Verifying Build Reproducibility](advanced-topics/diff-hook.md)
   - [Using the `post-build-hook`](advanced-topics/post-build-hook.md)
+  - [Pasta](advanced-topics/pasta.md)
+- [Known Issues](known-issues/known-issues.md)
+  - [Limitations around non-isolated builds](known-issues/non-isolated-build-limits.md)
 - [Command Reference](command-ref/command-ref.md)
   - [Common Options](command-ref/opt-common.md)
   - [Common Environment Variables](command-ref/env-common.md)
@@ -197,6 +200,7 @@
 - [Release Notes](release-notes/release-notes.md)
   - [Upcoming release](release-notes/rl-next.md)
     <!-- RELENG-AUTO-INSERTION-MARKER (see releng/release_notes.py) -->
+  - [Lix 2.94 (2025-11-17)](release-notes/rl-2.94.md)
   - [Lix 2.93 (2025-05-09)](release-notes/rl-2.93.md)
   - [Lix 2.92 (2025-01-18)](release-notes/rl-2.92.md)
   - [Lix 2.91 (2024-08-12)](release-notes/rl-2.91.md)

doc/manual/src/advanced-topics/pasta.md

@@ -0,0 +1,19 @@
+# [Pasta](https://passt.top/passt/about/): a network sandbox for fixed-output derivations
+
+## Introduction
+
+This section only applies to **Linux systems** as Pasta is a Linux-only measure.
+
+Since [CVE-2025-46416](https://lix.systems/blog/2025-06-24-lix-cves/), the Lix project decided to adopt [Pasta](https://passt.top/passt/about/) for all fixed-output derivations, protecting against various attack vectors such as UNIX abstract domain sockets or more manipulation at the network layer from a malicious fixed-output derivation code.
+
+Pasta acts as a translation layer between a layer-2 network interface and layer-4 sockets (TCP, UDP, ICMP/ICMPv6 echo) on the host. It requires no special privileges and can serve as a alternative to [SLiRP](https://en.wikipedia.org/wiki/Slirp) which was used [by Guix to mitigate the same problem](https://codeberg.org/guix/guix/commit/fb42611b8f27960304db5a1c0d33b8371dcde2a8).
+
+## How to disable Pasta?
+
+It's sufficient to pass `pasta-path = ""` in your `/etc/nix/nix.conf` or on the command line `--pasta-path ""` of a Lix invocation.
+
+## Known issues surrounding Pasta
+
+- Only the first DNS server in `/etc/resolv.conf` is considered: failover is not possible.
+- [Reduced feature set compared to the Linux kernel](https://passt.top/passt/about/#features)
+- [Performance overhead in multi-gigabits contexts and IMIX MTUs](https://passt.top/passt/about/#performance_1)

doc/manual/src/command-ref/nix-build.md

@@ -58,7 +58,7 @@ $ nix-build flake:nixpkgs -A firefox
 $ nix-build flake:github:NixOS/nixpkgs/release-23.11 -A firefox
 ```
 
-Finally, for legacy reasons, if a path starts with `channel:`, the rest of the argument is interpreted as the name of a *nixpkgs* channel tarball to fetch from `https://nixos.org/channels/$CHANNEL_NAME/nixexprs.tar.xz`.
+Finally, for legacy reasons, if a path starts with `channel:`, the rest of the argument is interpreted as the name of a *nixpkgs* channel tarball to fetch from `https://channels.nixos.org/$CHANNEL_NAME/nixexprs.tar.xz`.
 This is a **hard coded URL** pattern and is *not* related to the subscribed channels managed by the [nix-channel](./nix-channel.md) command.
 
 > **Note**: any of the special syntaxes may always be disambiguated by prefixing the path.

doc/manual/src/command-ref/nix-channel.md

@@ -11,7 +11,7 @@
 Channels are a mechanism for referencing remote Nix expressions and conveniently retrieving their latest version.
 
 The moving parts of channels are:
-- The official channels listed at <https://nixos.org/channels>
+- The official channels listed at <https://channels.nixos.org>
 - The user-specific list of [subscribed channels](#subscribed-channels)
 - The [downloaded channel contents](#channels)
 - The [Nix expression search path](@docroot@/command-ref/conf-file.md#conf-nix-path), set with the [`-I` option](#opt-I) or the [`NIX_PATH` environment variable](#env-NIX_PATH)
@@ -77,9 +77,9 @@ This command has the following operations:
 Subscribe to the Nixpkgs channel and run `hello` from the GNU Hello package:
 
 ```console
-$ nix-channel --add https://nixos.org/channels/nixpkgs-unstable
+$ nix-channel --add https://channels.nixos.org/nixpkgs-unstable
 $ nix-channel --list
-nixpkgs https://nixos.org/channels/nixpkgs
+nixpkgs https://channels.nixos.org/nixpkgs
 $ nix-channel --update
 $ nix-shell -p hello --run hello
 hello

doc/manual/src/contributing/hacking.md

@@ -170,6 +170,14 @@ Once you get a `Code-Review+2` vote on your change, it's rebased on `main` and C
 If the change appears as "Rebase Required", you need to rebase it on `main` locally or via the Gerrit UI and wait for `Verified+1` before the Submit button is made active
 The `Code-Review+2` from before will stick around through trivial rebases so no need to re-request review for a mere rebase.
 
+## Interacting with the CI, Buildkite
+
+We use Buildkite for our CI, usually you will not have to interact directly with it other than reviewing any errors it produces, which are linked from Gerrit.
+
+However in certain cases a CI run will fail due to transient issues not related to your code and you will need to rerun it by hand.
+You can log in to the CI via [SSO](https://buildkite.com/sso/lix-project). On your job you can then hit the "Retry failed" button to rerun it, normally you will not have a repeat of the transient issue.
+If the build still fails on CI issues or all builds are failing this should be reported via [Zulip on #T-infra](https://zulip.lix.systems/#narrow/channel/7-T-infra) or [Matrix on #dev](https://matrix.to/#/%23dev%3Alix.systems?via=lix.systems).
+
 ## Building Lix with `nix`
 
 To build a release version of Lix for the current operating system and CPU architecture:

doc/manual/src/contributing/testing.md

@@ -62,6 +62,12 @@ For `installcheck` specifically, first run `just install` before running the tes
 
 Finer-grained filtering within a test suite is also possible using the [--gtest_filter](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) command-line option to a test suite executable, or the `GTEST_FILTER` environment variable.
 
+### Inspecting failures
+
+The test suite emits logs in `build/meson-logs/`; the full textual failure logs are in `build/meson-logs/testlog.txt`.
+
+If you want a much nicer experience of viewing the logs in a structured manner, use `xunit-viewer --results build/meson-logs/testlog.junit.xml --server` to view them in a web browser.
+
 ### Unit test support libraries
 
 There are headers and code which are not just used to test the library in question, but also downstream libraries.

doc/manual/src/installation/installing-docker.md

@@ -60,3 +60,10 @@ Then:
 ```console
 $ docker run -ti lix
 ```
+
+# Known issues
+
+Lix in Docker is very sensitive to **functional** DNS resolution if you are running with [Pasta protections](../advanced-topics/pasta.md) which are enabled by default since Lix 2.93.0 on most distributions.
+If you notice failure to download things, double check whether your **first** DNS entry in `/etc/resolv.conf` is functional.
+
+Lix with [Pasta protections](../advanced-topics/pasta.md) does not support failing over the next entries.

doc/manual/src/known-issues/known-issues.md

@@ -0,0 +1 @@
+This section lists known issues around Lix.

doc/manual/src/known-issues/non-isolated-build-limits.md

@@ -0,0 +1,21 @@
+# Limitations of non-isolated builds
+
+## What are non-isolated builds?
+
+In Lix, only builds done on Linux with `sandbox = true` and a functioning
+`pasta-path` are isolated from the rest of the system, all other builds are
+considered non-isolated to some degree.
+
+For example, running Lix with [Pasta](@docroot@/advanced-topics/pasta.md)
+disabled makes the host network visible to fixed-output derivations, reducing
+isolation somewhat.
+
+## Clean termination of non-isolated builds
+
+Non-isolated builds may not terminate cleanly in all cases due to limitations in Lix's process management.
+
+This occurs when a build keeps the build log file descriptor open past the end of the actual build. A common cause of this are background tasks that aren't properly terminated before the main build process exits, for example: HTTP servers run as part of a test suite.
+
+See [issue #1018](https://git.lix.systems/lix-project/lix/issues/1018) for an example.
+
+The only solution is to manually terminate leftover processes in your derivation, including during failure scenarios.

doc/manual/src/package-management/basic-package-mgmt.md

@@ -41,7 +41,7 @@ install Lix. If this is not the case for some reason, you can add it
 as follows:
 
 ```console
-$ nix-channel --add https://nixos.org/channels/nixpkgs-unstable
+$ nix-channel --add https://channels.nixos.org/nixpkgs-unstable
 $ nix-channel --update
 ```
 
@@ -49,7 +49,7 @@ $ nix-channel --update
 >
 > On NixOS, you’re automatically subscribed to a NixOS channel
 > corresponding to your NixOS major release (e.g.
-> <http://nixos.org/channels/nixos-21.11>). A NixOS channel is identical
+> <https://channels.nixos.org/nixos-21.11>). A NixOS channel is identical
 > to the Nixpkgs channel, except that it contains only Linux binaries
 > and is updated only if a set of regression tests succeed.
 

doc/manual/src/release-notes/rl-2.94.md

@@ -0,0 +1,989 @@
+# Lix 2.94 "Açaí na tigela" (2025-11-17)
+# Lix 2.94.2 (2026-05-04)
+## Fixes
+
+- Fix unsigned overflow leading to out-of-band write in the NAR parser [cl/5553](https://gerrit.lix.systems/c/lix/+/5553)
+
+  The NAR parser contained an unsigned integer overflow that could be used by an
+  attacker to write arbitrary data to an unknown memory location and possibly
+  achieve code execution. A successful attack on the system-wide Lix daemon
+  could lead to privilege escalation to root. Any process that involves NAR
+  serialization could trigger this issue, including (but not limited to)
+
+    - local user interaction, whether the users are trusted or untrusted
+    - malicious substituters sending malformed NARs
+    - remote builders sending malformed build results
+    - remote daemons sending malformed inputs when requesting remote builds
+
+  Successful attacks using this bug require ASLR weakening of some sort, whether
+  by architecture constraints (e.g. on 32 bit systems, where little randomization
+  is possible) or system configuration (e.g. low ASLR entropy when loading
+  libraries), and millions of attempts. Local attacks can be mounted in less than
+  an hour. Remote builds typically require a fresh SSH connection for each build
+  and are thus less susceptible. Only one attempt can be made by substituters for
+  every build using substituters, they are thus not a likely vector for attacks.
+
+  At the time of writing, MITRE has not assigned this a CVE yet.
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae), [Raito Bezarius](https://git.lix.systems/raito), [edef](https://github.com/edef1c), and [sandydoo](https://github.com/sandydoo) for this.
+
+
+
+
+# Lix 2.94.1 (2026-03-13)
+
+
+
+
+# Lix 2.94.0 (2025-11-17)
+## Breaking Changes
+
+- Remove support for daemon protocols before 2.18 [fj#510](https://git.lix.systems/lix-project/lix/issues/510) [cl/3249](https://gerrit.lix.systems/c/lix/+/3249)
+
+  Support for daemon wire protocols belonging to Nix 2.17 or older have been
+  removed. This impacts clients connecting to the local daemon socket or any
+  remote builder configured using the `ssh-ng` protocol. Builders configured
+  with the `ssh` protocol are still accessible from clients such as Nix 2.3.
+  Additionally Lix will not be able to connect to an old daemon locally, and
+  remote build connections to old daemons is likewise limited to `ssh` urls.
+
+  We have decided to take this step because the old protocols are very badly
+  tested (if at all), maintenance overhead is high, and a number of problems
+  with their design makes it infeasible to remain backwards compatible while
+  we move Lix to a more modern RPC mechanism with better versioning support.
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+- Remove impure derivations and dynamic derivations [fj#815](https://git.lix.systems/lix-project/lix/issues/815) [cl/3210](https://gerrit.lix.systems/c/lix/+/3210)
+
+  The `impure-derivations` and `dynamic-derivations` experimental feature have
+  been removed.
+
+  New impure or dynamic derivations cannot be created from this point forward, and
+  any such pre-existing store derivations canot be read or built any more.
+  Derivation outputs created by building such a derivation are still valid
+  until garbage collected; existing store derivations can only be garbage
+  collected.
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+- First argument to `--arg`/`--argstr` must be a valid Nix identifier [fj#496](https://git.lix.systems/lix-project/lix/issues/496)
+
+  The first argument to `--arg`/`--argstr` must be a valid Nix identifier, i.e.
+  `nix-build --arg config.allowUnfree true` is now rejected.
+
+  This is because that invocation is a false friend since it doesn't set
+  `{ config = { allowUnfree = true; }; }`, but `{ "config.allowUnfree" = true; }`.
+
+  The idea is to change the behavior to the latter in the long-term. For that,
+  non-identifiers started giving a warning since 2.92 and are now rejected to give people
+  who depend on that a chance to notice and potentially weigh in on the discussion.
+
+  Many thanks to [ma27](https://git.lix.systems/ma27) for this.
+
+- New cgroup delegation model [fj#537](https://git.lix.systems/lix-project/lix/issues/537) [fj#77](https://git.lix.systems/lix-project/lix/issues/77) [cl/3230](https://gerrit.lix.systems/c/lix/+/3230)
+
+  Builds using cgroups (i.e. `use-cgroups = true` and the experimental feature
+  `cgroups`) now always delegate a cgroup tree to the sandbox.
+
+  Compared to the original C++ Nix project, our delegation includes the
+  `subtree_control` file as well, which means that the sandbox can disable
+  certain controllers in its own cgroup tree.
+
+  This is a breaking change because this requires the Nix daemon to run with an
+  already delegated cgroup tree by the service manager.
+
+  ## How to setup the cgroup tree with systemd?
+
+  systemd offers knobs to perform the required setup using:
+
+  ```
+  [Service]
+  Delegate=yes
+  DelegateSubtree=supervisor
+  ```
+
+  These directives are now included in our systemd packaging.
+
+  ## What about using Nix as root without connecting to the daemon?
+
+  Builds run as `root` without connecting to the daemon relying on the cgroup
+  feature are now broken, i.e.
+
+  ```console
+  # nix-build --use-cgroups --sandbox ... # will not work
+  ```
+
+  Consider doing instead:
+
+  ```console
+  # systemd-run --same-dir --wait -p Delegate=yes -p DelegateSubgroup=supervisor nix-build --use-cgroups ...
+  ```
+
+  If you need to disable cgroups temporarily, remember that you can do
+  `NIX_CONF='include /etc/nix/nix.conf\nuse-cgroups = false' nix-build ...` or
+  `nix-build --no-use-cgroups ...`.
+
+  ## What about other service managers than systemd?
+
+  systemd has a [documentation](https://systemd.io/CGROUP_DELEGATION/) on how to
+  handle cgroup delegation from service management perspective.
+
+  If your service manager adheres to systemd semantics, e.g. writing an extended
+  attribute `user.delegate=1` on the delegated cgroup tree directory and moving
+  the `nix-daemon` process inside a cgroup tree to respect the inner process
+  rule, then, the feature will work as well.
+
+  ## Why is the cgroup feature still experimental?
+
+  While the cgroup feature unlocks many use cases, its behavior and integration (e.g. user experience), especially at scale on build farms or in multi-tenant environments, are not yet fully matured. There’s also potential for deeper systemd integration (e.g. using slices and scopes) that has not been fully explored.
+
+  To avoid locking in an unstable interface, we’re keeping the experimental flag until we have validated the feature across a broader range of scenarios, including but not limited to:
+
+  * Nix as root
+  * Hydra-style build farms
+  * Forgejo CI runners
+  * Shared remote builders
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [eldritch horrors](https://git.lix.systems/pennae), and [lheckemann](https://git.lix.systems/lheckemann) for this.
+
+- Enable high compress ratio zstd compression by default for binary caches uploads [fj#945](https://git.lix.systems/lix-project/lix/issues/945) [cl/4503](https://gerrit.lix.systems/c/lix/+/4503)
+
+  The default compression method for binary cache uploads has been switched from
+  [`xz`](https://github.com/tukaani-project/xz) to
+  [`zstd`](https://github.com/facebook/zstd) to address performance and usability
+  issues related to modern hardware and high-speed connections.
+
+  ## Why?
+
+  `xz` offers compression ratios but is single-threaded in our implementation and
+  very slow (~10-20 Mbps in our test), preventing full utilization of 100Mbps+
+  connections and significantly slowing decompression for end users.
+
+  Lix is a "compress once, decompress many" application: build farms can afford
+  to spend more time compressing to achieve a faster download transfer for the
+  end user. More importantly, it matters that all end users spend the least
+  amount of time decompressing.
+
+  ## What about compression ratios?
+
+  `zstd` cannot achieve the same peaks as `xz`, nonetheless, `zstd` compression
+  level has been increased to level 12 by default to balance compression ratio
+  and performance.
+
+  ## Synthetic test case data
+
+  * **xz** (default compression level) on a 4.4GB file: ~632MB (77s)
+  * **zstd** (level 12) on the same file: ~775MB (18s), 18% larger but 50% faster
+  * **zstd** (level 14): ~773MB (37s)
+  * **zstd** (level 16): ~735MB (66s)
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [Raito Bezarius](https://git.lix.systems/raito) for this.
+
+- Repl debugger uses `--ignore-try` by default [lix#666](https://git.lix.systems/lix-project/lix/issues/666) [cl/3488](https://gerrit.lix.systems/c/lix/+/3488)
+
+  Previously, using the debugger meant that exceptions thrown in `builtins.tryEval` would trigger the debugger.
+
+  However, this caught nixpkgs initialization code, which is unhelpful in the majority of cases, so we changed the default.
+
+  To get the old behaviour, use `--no-ignore-try`.
+
+  ```
+  $ nix repl --debugger --expr 'with import <nixpkgs> {}; pkgs.hello'
+  Lix 2.94.0-dev-pre20250625-9a59106
+  Type :? for help.
+  error: file 'nixpkgs-overlays' was not found in the Nix search path (add it using $NIX_PATH or -I)
+
+  This exception occurred in a 'tryEval' call. Use --ignore-try to skip these.
+
+  Added 13 variables.
+  nix-repl>
+  ```
+
+  Many thanks to [jade](https://git.lix.systems/jade) for this.
+
+- Strings may now contain NUL bytes [cl/3968](https://gerrit.lix.systems/c/lix/+/3968)
+
+  Lix now allows strings to contain NUL bytes instead of silently truncating the
+  string before the first such byte. Notably NUL-bearing strings were allowed as
+  attribute names—even though the corresponding strings were not representable!—
+  leading to very surprising and incorrect behavior in corner cases, for example
+
+  ```
+  nix-repl> builtins.fromJSON ''{"a": 1, "a\u0000b": 2}''
+  {
+    a = 1;
+    "ab" = 2;
+  }
+
+  nix-repl> builtins.attrNames (builtins.fromJSON ''{"a": 1, "a\u0000b": 2}'')
+  [
+    "a"
+    "a"
+  ]
+  ```
+
+  rather than the more correct but still with the terminal eating NUL on display
+
+  ```
+  nix-repl> builtins.fromJSON ''{"a": 1, "a\u0000b": 2}''
+  {
+    a = 1;
+    "ab" = 2;
+  }
+
+  nix-repl> builtins.attrNames (builtins.fromJSON ''{"a": 1, "a\u0000b": 2}'')
+  [
+    "a"
+    "ab"
+  ]
+  ```
+
+  We consider this a breaking change since eval results *will* change if strings
+  with embedded NUL bytes were used, but we also consider the old behavior to be
+  not intentional (seeing how inconsistent it was) but merely fallout from a old
+  and misguided implementation decision to be worked around, not actually fixed.
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+- Fixed output derivations can be run using `pasta` network isolation [fj#285](https://git.lix.systems/lix-project/lix/issues/285) [cl/3452](https://gerrit.lix.systems/c/lix/+/3452)
+
+  Fixed output derivations traditionally run in the host network namespace.
+  On Linux this allows such derivations to communicate with other sandboxes
+  or the host using the abstract Unix domains socket namespace; this hasn't
+  been unproblematic in the past and has been used in two distinct exploits
+  to break out of the sandbox. For this reason fixed output derivations can
+  now run in a network namespace (provided by [`pasta`]), restricted to TCP
+  and UDP communication with the rest of the world. When enabled this could
+  be a breaking change and we classify it as such, even though we don't yet
+  enable or require such isolation by default. We may enforce this in later
+  releases of Lix once we have sufficient confidence that breakage is rare.
+
+  [`pasta`]: https://passt.top/
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [puck](https://git.lix.systems/puck) for this.
+
+- Function equality semantics are more consistent, but still bad [cl/4556](https://gerrit.lix.systems/c/lix/+/4556) [cl/4244](https://gerrit.lix.systems/c/lix/+/4244)
+
+  Lix has inherited a historic misfeature from CppNix in the form of pointer
+  equality checks built into the `==` operator. These checks were originally
+  meant to optimize comparison for large sets, but they have the unfortunate
+  side effect of producing unexpected results when sets containing functions
+  are compared. **Lix 2.93 and earlier** behave as shown in the repl session
+
+  ```
+  Lix 2.93.3
+  Type :? for help.
+  nix-repl> f = x: x
+  Added f.
+
+  nix-repl> f == f
+  false
+
+  nix-repl> let s.f = f; in s.f == s.f
+  false
+
+  nix-repl> # however!
+            { inherit f; } == { inherit f; }
+  true
+
+  nix-repl> [ f ] == [ f ]
+  true
+
+  nix-repl> # and, in another twist:
+            [ f ] == map f [ f ]
+  false
+  ```
+
+  Nixpkgs relies on sets containing functions being comparable, so we cannot
+  simply deprecate this behavior. Due to changes to the object model used by
+  Lix ***all* comparisons above now evaluate to `true`**. This is considered
+  a breaking change because eval results may differ, but we also consider it
+  minor because the optimization is unsound (c.f. `let l = [NaN]; in l == l`
+  evaluates to `true` even though floating point `NaN` is incomparable). Lix
+  intends to remove this optimization altogether in the future, but until we
+  can do that we instead make it slightly less broken to allow other, *real*
+  optimizations. Function equality comparison remains **undefined behavior**
+  and should not be relied upon in Nixlang code that intends to be portable.
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+- `nix eval --write-to` has been removed [fj#974](https://git.lix.systems/lix-project/lix/issues/974) [fj#227](https://git.lix.systems/lix-project/lix/issues/227) [cl/4045](https://gerrit.lix.systems/c/lix/+/4045)
+
+  `nix eval --write-to` has been removed since it was underspecified, not widely
+  useful, and prone to security-sensitive misbehaviors. The feature was added in
+  Nix 2.4 purely for internal use in the build system. According to our research
+  it hasn't found any use outside of some distribution packaging scripts. Please
+  use structured outputs formats (such as JSON) instead as they have better type
+  fidelity, don't conflate attributes with paths, and are useful to other tools.
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+- Remove the `parse-toml-timestamps` experimental feature
+
+  The `parse-toml-timestamps` experimental feature has been removed.
+
+  This feature used in‐band signalling to mark timestamps, making it
+  impossible to unambiguously parse TOML documents. It also exposed
+  implementation‐defined behaviour in the TOML specification that
+  changed in the toml11 parser library.
+
+  Any interface for parsing TOML timestamps suitable for future
+  stabilization would necessarily involve breaking changes, and there
+  is no evidence this experimental feature is being relied upon in the
+  wild, so it has been removed.
+
+  Many thanks to [Emily](https://git.lix.systems/emilazy) for this.
+
+- Reject overflowing TOML integer literals [cl/3916](https://gerrit.lix.systems/c/lix/+/3916)
+
+  The toml11 library used by Lix was updated. The new
+  version aligns with the [TOML v1.0.0 specification’s
+  requirement](https://toml.io/en/v1.0.0#integer) to reject integer
+  literals that cannot be losslessly parsed. This means that code like
+  `builtins.fromTOML "v=0x8000000000000000"` will now produce an error
+  rather than silently saturating the integer result.
+
+  Many thanks to [Emily](https://git.lix.systems/emilazy) for this.
+
+- uid-range depends on cgroups [cl/3230](https://gerrit.lix.systems/c/lix/+/3230)
+
+  `uid-range` builds now depends on `cgroups`, an experimental feature.
+
+  `uid-range` builds already depended upon `auto-allocate-uids`, another experimental feature.
+
+  The rationale for doing so is that `uid-range` provides a sandbox with many
+  UIDs, this is useful for re-mapping them into a nested namespace, e.g. a
+  container.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+
+## Features
+
+- Add `inputs.self.submodules` flake attribute [fj#942](https://git.lix.systems/lix-project/lix/issues/942) [cl/3839](https://gerrit.lix.systems/c/lix/+/3839)
+
+  A port of <https://github.com/NixOS/nix/pull/12421> to Lix, which:
+
+  - adds a general `inputs.self` flake attribute that retroactively applies
+    configurations to a flake after it's been fetched, then triggers a refetch of
+    the flake with the new config.
+  - implements `inputs.self.submodules` that allows a flake to declare its need
+    for submodules, which are then fetched automatically with no need to pass
+    `?submodules=1` anywhere.
+
+  Many thanks to [Eelco Dolstra](https://github.com/edolstra) and [ورد](https://git.lix.systems/janw4ld) for this.
+
+- Lix supports HTTP/3 behind `--http3` [fj#1033](https://git.lix.systems/lix-project/lix/issues/1033)
+
+  Lix now supports HTTP/3 for file transfers when the linked curl version
+  supports it.
+
+  By default, HTTP/3 is disabled notably due to performance issues reported in
+  mid-2024. [More details
+  here](https://daniel.haxx.se/blog/2024/06/10/http-3-in-curl-mid-2024/).
+
+  As of 2025-11-14, [NixOS official cache](https://cache.nixos.org) supports
+  HTTP/3 via Fastly. [More info
+  here](https://github.com/NixOS/infra/commit/157fa70e46afbd6338a32407be461fce05c57bf8).
+
+  To enable HTTP/3:
+
+  * Use `--http3` for individual transfers.
+  * Add `http3 = true` in your Nix configuration for permanent activation.
+
+  To disable it, use `--no-http3`.
+
+  **Note**:
+
+  * `--no-http2 --http3` will still enable both HTTP/2 and HTTP/3.
+  * `--http2 --http3` will prioritize HTTP/3 and fall back to HTTP/2 (and then
+    HTTP/1.1).
+
+  These are current CLI limitations. In the future, we plan to replace `--httpX`
+  options with `--max-http-version [1,2,3]` for easier version selection in Lix
+  transfers.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+- Add hyperlinks in attr set printing [cl/3790](https://gerrit.lix.systems/c/lix/+/3790)
+
+  The attribute set printer, such as is seen in `nix repl` or in type errors, now prints hyperlinks on each attribute name to its definition site if it is known.
+
+  Example: all of the attributes shown here are hyperlinks to the exact definition site of the attribute in question:
+
+  ```
+  $ nix eval -f '<nixpkgs>' lib.licenses.mit
+  { deprecated = false; free = true; fullName = "MIT License"; redistributable = true; shortName = "mit"; spdxId = "MIT"; url = "https://spdx.org/licenses/MIT.html"; }
+  ```
+
+  Many thanks to [jade](https://git.lix.systems/jade) for this.
+
+- Experimental integer coercion in interpolated strings [cl/3198](https://gerrit.lix.systems/c/lix/+/3198)
+
+  Ever tried interpolating a port number in Lix and ended up with something like this?
+
+  ```nix
+  "http://${config.network.host}:${builtins.toString config.network.port}/"
+  ```
+
+  You're not alone. Thousands of Lix users suffer every day from excessive `builtins.toString` syndrome. It’s 2025, and we still have to cast integers to use them in strings.
+
+  To address this, Lix introduces the **`coerce-integers`** experimental feature. When enabled, interpolated integers within `"${...}"` are automatically coerced to strings. This allows writing:
+
+  ```nix
+  "http://${config.network.host}:${config.network.port}/"
+  ```
+
+  without additional conversion.
+
+  To enable the feature, you need to add `coerce-integers` to your set of experimental features.
+
+  ### Stabilization criteria
+
+  The `coerce-integers` feature is experimental and limited strictly to string interpolation (`"${...}"`). Before stabilization, the following must hold:
+
+  1. **Interpolation-only**
+     Coercion must not occur outside interpolation. Expressions like `"" + 42` must continue to fail.
+
+  2. **Expectation that no explicit cast are being observed**
+     Cases observing explicit coercion (e.g., via `tryEval` gadget or similar) are expected not to be load-bearing in actual production code.
+
+  ### Timeline for stabilization
+
+  If the feature proves safe and is widely adopted across typical usage (e.g., actual configurations in the wild turning on the flag, non-trivial out-of-tree projects using it), the experimental flag will be removed **after six months of active use or two Lix releases**, whichever is longer.
+
+  This avoids locking the feature in experimental status indefinitely, as happened with Flakes, while allowing time for validation and ecosystem integration.
+
+  ### What about coercing floats or more?
+
+  Coercion beyond integers -- such as for floats or other types -- is **not planned**, even under an experimental flag. Questions like "what is the canonical string representation of a float?" involve subtle and context-dependent trade-offs. Without a robust and principled mechanism to define and audit such behavior, introducing broader coercion risks setting unintended and hard-to-reverse precedents. The scope of `coerce-integers` is intentionally narrow and will remain so.
+
+  In terms of outlook, a proposal like https://git.lix.systems/lix-project/lix/issues/835 could pave the way for a better solution.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [delroth](https://github.com/delroth), [eldritch horrors](https://git.lix.systems/pennae), and [winter](https://git.lix.systems/winter) for this.
+
+- nix-eval-jobs: support `--no-instantiate` flag [fj#987](https://git.lix.systems/lix-project/lix/issues/987)
+
+  `nix-eval-jobs` now supports a flag called `--no-instantiate`. With this enabled,
+  no write operations on the eval store are performed. That means, only evaluation is
+  performed, but derivations (and their gcroots) aren't created.
+
+  Many thanks to [mic92](https://github.com/mic92) and [ma27](https://git.lix.systems/ma27) for this.
+
+
+## Improvements
+
+- Assess current profile generations pointers in `nix doctor` [cl/3108](https://gerrit.lix.systems/c/lix/+/3108)
+
+  Added a new check to `nix doctor` that verifies whether the current generation of
+  a Nix profile can be resolved. This helps users diagnose issues with broken or
+  misconfigured profile symlinks.
+
+  This helps determining if you have broken symlinks or misconfigured packaging.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
+
+- Improved susbtituter query speed
+
+  The code used to query substituters for derivations has been rewritten slightly
+  to take advantage of our asynchronous runtime. Such queries run for every build
+  that could download from substituters and processes every derivation that isn't
+  yet present on the local system. Previously Lix would use `http-connections` to
+  limit query concurrency, even for modern caches that support HTTP/2 and have no
+  limit on how many queries can be run concurrently on one single connection. Lix
+  no longer does this, resulting in approximately 60% reduction in query time for
+  medium-sized closures (e.g. NixOS system closures) during testing, although the
+  exact number depends greatly on local network latency and generally improves as
+  latency increases. Unlike previously setting `http-connections` to `1` or other
+  low values no longer brings a massive penalty in query performance if the cache
+  in use by the querying system supports HTTP/2 (as e.g. `cache.nixos.org` does).
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+- Hitting Control-C twice always terminates Lix [cl/3574](https://gerrit.lix.systems/c/lix/+/3574)
+
+  Hitting Control-C or sending `SIGINT` to Lix now prints an informational message
+  if it is still running after on second, the second Control-C/`SIGINT` terminates
+  Lix immediately without waiting for any shutdown code to finish running. Lix did
+  not treat the second such event differently from first in the past; this made it
+  impossible to easily terminate running Lix processes that got stuck in e.g. very
+  expensive Nixlang code that never interacted with the store. We now terminate as
+  soon as the user hits Control-C again without waiting any more, to much the same
+  effect as putting Lix into the background and killing it immediately afterwards.
+
+  This means you can now more conveniently break out of stuck Nixlang evaluations:
+  ```
+  ❯ nix-instantiate --eval --expr 'let f = n: if n == 0 then 0 else f (n - 1) + f (n - 1); in f 32'
+  ^CStill shutting down. Press ^C again to abort all operations immediately.
+  ^C
+
+  ❌130 ❯
+  ```
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+- `--keep-failed` chowns the build directory to the user that request the build
+
+  Running a build with `--keep-failed` now chowns the temporary directory from the
+  builder user and group to the user that request the build if the build came from
+  a local user connected to the daemon. This makes inspecting failed derivations a
+  lot easier. On Linux the build directory made visible to the user will not be in
+  the same path as it was in the sandbox and continuing builds will usually break.
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+- Better debuggability on fixed-output hash mismatches
+
+  Fixed-output derivation hash mismatch error messages will now include the path that was
+  produced unexpectedly, and this path will be registered as valid even if `--check`
+  (`nix-store`, `nix-build`) or `--rebuild` (`nix build`) was passed. This makes comparing
+  the expected path with the obtained path easier, and is useful for debugging when
+  upstreams modify previously-published releases or when changes in fixed-output
+  derivations' dependencies affect their output unexpectedly.
+
+  Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
+
+- Add --raw flag to `nix-instantiate --eval` for unescaped output [gh#12119](https://github.com/NixOS/nix/pull/12119) [cl/2886](https://gerrit.lix.systems/c/lix/+/2886)
+
+  The `nix-instantiate --eval` command now supports a `--raw` flag. When used,
+  the result must be coercible to a string (as with `${...}`) and is printed
+  verbatim, without quotes or escaping.
+
+  Many thanks to [Martin Fischer](https://github.com/not-my-profile), [infinisil](https://github.com/infinisil), and [Raito Bezarius](https://git.lix.systems/raito) for this.
+
+- Allow `nix store ls` to read nar listings from binary cache stores. [cl/3225](https://gerrit.lix.systems/c/lix/+/3225)
+
+  The `nix store ls` command now supports reading `.ls` nar listings from binary cache stores.
+  If a listing is detected for the store path being queried, the nar is no longer downloaded.
+  These nar listings are available in binary cache stores where the `write-nar-listing` option is
+  enabled, such as cache.nixos.org.
+
+  Many thanks to [Victor Fuentes](https://git.lix.systems/vlinkz) for this.
+
+- show tree with references that lead to an output cycle [fj#551](https://git.lix.systems/lix-project/lix/issues/551)
+
+  When Lix determines a cyclic dependency between several outputs of a derivation,
+  it now displays which files in which outputs lead to an output cycle:
+
+  ```
+  error: cycle detected in build of '/nix/store/gc5h2whz3rylpf34n99nswvqgkjkigmy-demo.drv' in the references of output 'bar' from output 'foo'.
+
+         Shown below are the files inside the outputs leading to the cycle:
+         /nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar
+         └───lib/libfoo: …stuffbefore /nix/store/h680k7k53rjl9p15g6h7kpym33250w0y-demo-baz andafter.…
+             → /nix/store/h680k7k53rjl9p15g6h7kpym33250w0y-demo-baz
+             └───share/snenskek: …???? /nix/store/dm24c76p9y2mrvmwgpmi64rryw6x5qmm-demo-foo ....…
+                 → /nix/store/dm24c76p9y2mrvmwgpmi64rryw6x5qmm-demo-foo
+                 └───bin/alarm: …textexttext/nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar abcabcabc.…
+                     → /nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar
+  ```
+
+  Please note that showing the files and its contents while displaying the cycles only works
+  on Linux.
+
+  Many thanks to [ma27](https://git.lix.systems/ma27) for this.
+
+- Lix now enables parallel marking in boehm-gc [fj#983](https://git.lix.systems/lix-project/lix/issues/983) [cl/3880](https://gerrit.lix.systems/c/lix/+/3880)
+
+  This brings a fairly modest performance improvement (~38% for `nixpkgs search hello`) to evaluation, especially in scenarios that necessitate larger heap sizes.
+
+  Many thanks to [Eelco Dolstra](https://github.com/edolstra) and [Seth Flynn](https://git.lix.systems/getchoo) for this.
+
+- `disallowedRequisites` now reports chains of disallowed requisites [fj#334](https://git.lix.systems/lix-project/lix/issues/334) [fj#626](https://git.lix.systems/lix-project/lix/issues/626) [gh#10877](https://github.com/NixOS/nix/issues/10877)
+
+  When a build fails because of [`disallowedRequisites`](@docroot@/language/advanced-attributes.md#adv-attr-disallowedRequisites), the error message now includes the chain of references that led to the failure. This makes it easier to see in which derivations the chain can be broken, to resolve the problem.
+
+  Example:
+
+  ```
+  $ nix-build -A hello
+  error: output '/nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2' is not allowed to refer to the following paths:
+         /nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-glibc-2.40-66
+         Shown below are chains that lead to the forbidden path(s).
+         /nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2
+         └───/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-glibc-2.40-66
+  ```
+
+  Many thanks to [ma27](https://git.lix.systems/ma27) and [Robert Hensing](https://github.com/roberth) for this.
+
+- Stack traces now summarize involved derivations at the bottom [cl/4493](https://gerrit.lix.systems/c/lix/+/4493)
+
+  When evaluation errors and a stack trace is printed,
+
+  For example, if I add Nheko to a NixOS `environment.systemPackages` without adding `olm-3.2.16` `nixpkgs.config.permittedInsecurePackages`, then without `--show-trace`, I previously got this:
+
+  ```
+  error:
+         … while calling the 'head' builtin
+           at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/attrsets.nix:1701:13:
+           1700|           if length values == 1 || pred here (elemAt values 1) (head values) then
+           1701|             head values
+               |             ^
+           1702|           else
+
+         … while evaluating the attribute 'value'
+           at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/modules.nix:1118:7:
+           1117|     // {
+           1118|       value = addErrorContext "while evaluating the option `${showOption loc}':" value;
+               |       ^
+           1119|       inherit (res.defsFinal') highestPrio;
+
+         (stack trace truncated; use '--show-trace' to show the full trace)
+
+         error: Package ‘olm-3.2.16’ in /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/pkgs/by-name/ol/olm/package.nix:37 is marked as insecure, refusing to evaluate.
+
+         < -snip the whole explanation about olm's CVEs- >
+  ```
+
+  This doesn't tell me anything about where `olm-3.2.16` came from.
+  With `--show-trace`, there's 1 155 lines to sift through, but does contain lines like "while evaluating derivation 'nheko-0.12.1'".
+
+  With this change, those lines are summarized and collected at the bottom, regardless of `--show-trace`:
+
+  ```
+  error:
+         … while calling the 'head' builtin
+           at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/attrsets.nix:1701:13:
+           1700|           if length values == 1 || pred here (elemAt values 1) (head values) then
+           1701|             head values
+               |             ^
+           1702|           else
+
+         … while evaluating the attribute 'value'
+           at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/modules.nix:1118:7:
+           1117|     // {
+           1118|       value = addErrorContext "while evaluating the option `${showOption loc}':" value;
+               |       ^
+           1119|       inherit (res.defsFinal') highestPrio;
+
+         (stack trace truncated; use '--show-trace' to show the full trace)
+
+         error: Package ‘olm-3.2.16’ in /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/pkgs/by-name/ol/olm/package.nix:37 is marked as insecure, refusing to evaluate.
+
+
+         < -snip the whole explanation about olm's CVEs- >
+
+
+         note: trace involved the following derivations:
+         derivation 'etc'
+         derivation 'dbus-1'
+         derivation 'system-path'
+         derivation 'nheko-0.12.1'
+         derivation 'mtxclient-0.10.1'
+  ```
+
+  Now we finally know that olm was evaluated because of Nheko, without sifting through *thousands* of lines of error message.
+
+  Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
+
+- Symbols reuses once-allocated Value to reduce garbage collected allocations [cl/3308](https://gerrit.lix.systems/c/lix/+/3308) [cl/3300](https://gerrit.lix.systems/c/lix/+/3300) [cl/3314](https://gerrit.lix.systems/c/lix/+/3314) [cl/3310](https://gerrit.lix.systems/c/lix/+/3310) [cl/3312](https://gerrit.lix.systems/c/lix/+/3312) [cl/3313](https://gerrit.lix.systems/c/lix/+/3313)
+
+  In the Lix evaluator, **symbols** represent immutable strings, like those used
+  for attribute names.
+
+  In evaluator design, such strings are typically [**interned**](https://en.wikipedia.org/wiki/String_interning), stored uniquely
+  to save memory, and Lix inherits this approach from the original C++ codebase.
+
+  However, some builtins, like `builtins.attrNames`, must return a `Value` type
+  that can represent any Nix value (strings, integers, lists, etc.).
+
+  Before this change, these builtins would create lists of `Value` objects by
+  allocating them through the garbage collector, copying the symbol’s string
+  content each time.
+
+  This allocation is unnecessary if the interned symbols themselves also hold a
+  `Value` representation allocated outside the garbage collector, since these
+  live for the full duration of evaluation.
+
+  As a result, this reduces the number of allocations, leading to:
+
+  * A significant drop in maximum [resident set memory](https://en.wikipedia.org/wiki/Resident_set_size) (RSS), with some large-scale
+    tests showing up to 11% (about 500 MiB) savings in large colmena deployments.
+  * A slight decrease in CPU usage during Nix evaluations.
+
+  This change is inspired by https://github.com/NixOS/nix/pull/13258 but the approach is different.
+
+  **Note** : [`xokdvium`](https://github.com/xokdvium) is the rightful author of https://gerrit.lix.systems/c/lix/+/3300 and the credit was missed on our end during the development process. We are deeply sorry for this mistake.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [eldritch horrors](https://git.lix.systems/pennae), [Tom Hubrecht](https://git.lix.systems/tom-hubrecht), [xokdvium](https://github.com/xokdvium), and [NaN-git](https://github.com/NaN-git) for this.
+
+
+## Fixes
+
+- `build-dir` no longer defaults to `temp-dir` [cl/3453](https://gerrit.lix.systems/c/lix/+/3453)
+
+  The directory in which temporary build directories are created no longer defaults
+  to the value of the `temp-dir` setting to avoid builders making their directories
+  world-accessible. This behavior has been used to escape the build sandbox and can
+  cause build impurities even when not used maliciously. We now default to `builds`
+  in `NIX_STATE_DIR` (which is `/nix/var/nix/b` in the default configuration).
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+- Global certificate authorities are copied inside the builder's environment [gh#12698](https://github.com/NixOS/nix/issues/12698) [fj#885](https://git.lix.systems/lix-project/lix/issues/885) [cl/3765](https://gerrit.lix.systems/c/lix/+/3765)
+
+  Previously, CA certificates were only installed at
+  `/etc/ssl/certs/ca-certificates.crt` for sandboxed builds on Linux.
+
+  This setup was insufficient in light of recent changes in `nixpkgs`, which now
+  enforce HTTPS usage for `fetchurl`, even for fixed-output derivations, to
+  mitigate confidentiality risks such as `netrc` or credentials leakage.
+  `nixpkgs` still make use of a special package called `cacerts` which contains a
+  copy of the CA certificates maintained by Nixpkgs and added as a reference for
+  TLS-enabled fetchers.
+
+  As a result, having a consistent and trusted certificate authority in all
+  builder environments is becoming more essential.
+
+  On `nix-darwin`, the `NIX_SSL_CERT_FILE` environment variable is always
+  explicitly defined, but it is ignored by the sandbox setup.
+
+  Simultaneously, Nix evaluates and propagates impure environment variables via
+  `lib.proxyImpureEnvVars`, meaning that if `NIX_SSL_CERT_FILE` is set (which
+  influences the default value for `ssl-cert-file`), it will be forwarded
+  unchanged into the builder environment.
+
+  However, on Linux, Nix also *copies* the CA file into the sandbox, creating a
+  discrepancy between the value of `NIX_SSL_CERT_FILE` and the actual trusted
+  certificate path used during the build.
+
+  This divergence caused confusion and was partially addressed by attempts to
+  whitelist the CA path in the Darwin sandbox (see cl/2906), but that approach
+  involved a non-trivial path canonicalization step and is not as general as this one.
+
+  To address this properly, we now emit a warning and override
+  `NIX_SSL_CERT_FILE` inside the builder, explicitly pointing it to the CA file
+  copied into the sandbox.
+
+  This eliminates ambiguity between `NIX_SSL_CERT_FILE`
+  and `ssl-cert-file`, ensuring consistent trust anchors across platforms.
+
+  This warning might become a hard error as we figure out what to do regarding
+  `lib.proxyImpureEnvVars` in nixpkgs.
+
+  The behavior has been verified across sandboxed and unsandboxed builds on both
+  Linux and Darwin.
+
+  As a consequence of this change, approximately 500 KB of CA certificate data is
+  now unconditionally copied into the build directory for fixed-output
+  derivations.
+
+  While this ensures consistent trust verification without having to restart the
+  daemon after system upgrades, it may introduce a slight overhead in build
+  performance. At present, no optimizations have been implemented to avoid this
+  copy, but if this overhead proves noticeable in your workflows, please open an
+  issue so we can evaluate and possibly implement different strategies to render
+  trust anchors visible.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [Emily](https://git.lix.systems/emilazy) for this.
+
+- libstore: exponential backoff for downloads [lix#932](https://git.lix.systems/lix-project/lix/issues/932) [cl/3856](https://gerrit.lix.systems/c/lix/+/3856)
+
+  The connection timeout when downloading from e.g. a binary cache is exponentially
+  increased per failure. The option `connect-timeout` is now an alias to `max-connect-timeout`
+  which is the maximum value for a timeout. The start value is controlled
+  by `initial-connect-timeout` which is `5` by default.
+
+  Many thanks to [ma27](https://git.lix.systems/ma27) for this.
+
+- Fix develop shells for derivations with escape codes [fj#991](https://git.lix.systems/lix-project/lix/issues/991) [cl/4154](https://gerrit.lix.systems/c/lix/+/4154) [cl/4155](https://gerrit.lix.systems/c/lix/+/4155)
+
+  ASCII control characters (including `\e`, used for ANSI escape codes) in derivation variables are now correctly escaped for `nix develop` and `nix print-dev-env`, instead of erroring.
+
+  Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
+
+- nix-store --delete: always remove obsolete hardlinks [cl/3188](https://gerrit.lix.systems/c/lix/+/3188)
+
+  Deleting specific paths using `nix-store --delete` or `nix store
+  delete` previously did not delete hard links created by `nix-store
+  --optimise` even if they became obsolete, unless _all_ of the given
+  paths were deleted successfully. Now, hard links are always cleaned
+  up, even if some of the given paths could not be deleted.
+
+  Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
+
+- Report GC statistics correctly [cl/3188](https://gerrit.lix.systems/c/lix/+/3188)
+
+  Deleting specific paths using `nix-store --delete` or `nix store delete` previously did
+  not report statistics correctly when some of the paths could not be deleted, even if
+  others were deleted:
+
+  ```
+  $ nix store delete /nix/store/9bwryidal9q3g91cjm6xschfn4ikd82q-hello-2.12.1 --delete-closure -v
+  finding garbage collector roots...
+  deleting '/nix/store/9bwryidal9q3g91cjm6xschfn4ikd82q-hello-2.12.1'
+  0 store paths deleted, 0.00 MiB freed
+  error: Cannot delete some of the given paths because they are still alive. Paths not deleted:
+           k9bxzr1l92r5y6mihrkbpbr3fmc8qszx-libidn2-2.3.8
+           mbx9ii53lzjlrsnlrfmzpwm33ynljwdn-libunistring-1.3
+           rf8hcy6bldxdqc0g6q1dcka1vh47x69s-xgcc-14.2.1.20250322-libgcc
+           vbrdc5wgzn0w1zdp10xd2favkjn5fk7y-glibc-2.40-66
+         To find out why, use nix-store --query --roots and nix-store --query --referrers.
+  ```
+
+  Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
+
+- Fallback to safe temp dir when build-dir is unwritable [fj#876](https://git.lix.systems/lix-project/lix/issues/876) [cl/3501](https://gerrit.lix.systems/c/lix/+/3501)
+
+  Non-daemon builds started failing with a permission error after introducing the `build-dir` option:
+
+  ```
+  $ nix build --store ~/scratch nixpkgs#hello --rebuild
+  error: creating directory '/nix/var/nix/builds/nix-build-hello-2.12.2.drv-0': Permission denied
+  ```
+
+  This happens because:
+
+  1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
+  2. The user lacks permissions for that path.
+
+  We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
+
+  To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
+
+  **Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+- Parse overflowing JSON number literals as floating‐point [cl/3919](https://gerrit.lix.systems/c/lix/+/3919)
+
+  Previously, `builtins.fromJSON "-9223372036854775809"` would
+  return a floating‐point number, while `builtins.fromJSON
+  "9223372036854775808"` would cause an evaluation error. This was
+  introduced with the banning of integer overflow in Lix 2.91; previously
+  the latter would result in C++ undefined behaviour. These cases are
+  now treated consistently with JSON’s model of a single numeric type,
+  and JSON number literals that do not fit in a Nix‐language integer
+  will be parsed as floating‐point numbers.
+
+  Many thanks to [Emily](https://git.lix.systems/emilazy) for this.
+
+- Fix handling of OSC codes in terminal output [fj#160](https://git.lix.systems/lix-project/lix/issues/160) [cl/3143](https://gerrit.lix.systems/c/lix/+/3143)
+
+  OSC codes in terminal output are now handled correctly, where OSC 8 (hyperlink) is preserved any
+  time color codes are allowed and all other OSC codes are stripped out. This applies not only to
+  output from build commands but also to rendered documentation in the REPL.
+
+  Many thanks to [lilyball](https://git.lix.systems/lilyball) for this.
+
+- Fix nix develop for derivations that rejects dependencies with structured attrs [fj#997](https://git.lix.systems/lix-project/lix/issues/997) [cl/4182](https://gerrit.lix.systems/c/lix/+/4182)
+
+  For the sake of concision, we refer to `disallowedReferences` in what follows,
+  but all output checks were equally fixed:
+  `{dis,}allowed{References,Requisites}`.
+
+  Derivations can define *output checks* to reject unwanted dependencies, such as
+  interpreters like `bash` or compilers like `gcc`. This can be done in two ways:
+
+  * **Legacy style**: `disallowedReferences = [ ... ]` in the environment.
+  * **Structured attrs**: `outputChecks.<output>.disallowedReferences = [ ... ]`,
+    typically used in `__json`.
+
+  Only the structured form supports derivations with multiple outputs.
+
+  `nix develop` internally rewrites derivations to create development shells. It
+  relied on the legacy `disallowedReferences`, and failed to honor the structured
+  variant. This led to broken shells in cases where `bashInteractive` was
+  explicitly disallowed using structured output checks, e.g. `nix develop
+  nixpkgs#systemd` after the "bash-less NixOS" changes.
+
+  This fix teaches `nix develop` to respect structured output checks, restoring
+  support for such derivations.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
+
+- nix-eval-jobs: retain NIX_PATH [cl/3859](https://gerrit.lix.systems/c/lix/+/3859)
+
+  `nix-eval-jobs` doesn't clear the `NIX_PATH` from the environment anymore. This matches the behavior
+  of [upstream version `2.30`](https://github.com/nix-community/nix-eval-jobs/releases/tag/v2.30.0).
+
+  Many thanks to [ma27](https://git.lix.systems/ma27) and [mic92](https://github.com/mic92) for this.
+
+- Remove reliance on Bash for remote stores via SSH [fj#830](https://git.lix.systems/lix-project/lix/issues/830) [fj#805](https://git.lix.systems/lix-project/lix/issues/805) [fj#304](https://git.lix.systems/lix-project/lix/issues/304) [cl/3159](https://gerrit.lix.systems/c/lix/+/3159)
+
+  The pre-flight `echo started` handshake -- added years ago to catch race conditions -- has been removed.
+
+  After removal of connection sharing in Lix 2.93, it required a Bash-compatible shell and a standard `echo`, so it failed on:
+
+    * builders protected by `ForceCommand` wrappers (e.g. `nix-remote-build`),
+    * BusyBox / initrd images with no Bash,
+    * hosts using non-POSIX shells such as Nushell.
+
+  The race the probe once addressed was tied to SSH connection-sharing -- since connection-sharing code has already been removed, the probe is now pointless.
+
+  Real connection or protocol errors are now left to SSH/Nix to report directly.
+
+  This is technically a breaking change if you had scripts that relied on the literal "started" which needs to be updated to rely on other signals, e.g., exit codes.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
+
+- repl-overlays now work in the debugger for flakes [fj#777](https://git.lix.systems/lix-project/lix/issues/777) [cl/3398](https://gerrit.lix.systems/c/lix/+/3398)
+
+  Due to a bug, it was previously not possible to use the debugger on flakes with repl-overlays, or with pure evaluation in general:
+
+  ```
+  $ nix repl --pure-eval
+  Lix 2.94.0-dev-pre20250617-87d99da
+  Type :? for help.
+  Loading 'repl-overlays'...
+  error: access to absolute path '/Users/jade/.config/nix/repl.nix' is forbidden in pure eval mode (use '--impure' to override)
+  ```
+
+  This is now fixed.
+  The contents of the repl-overlays file itself (i.e. most typically the top level lambda in it) will be evaluated in impure mode.
+  It may be necessary to use `builtins.seq` to force the impure operations to happen first if one wants to do impure operations inside a repl-overlays file in pure evaluation mode.
+
+  Many thanks to [jade](https://git.lix.systems/jade) for this.
+
+- `nix-shell` default shell directory is not `/tmp` anymore for `$NIX_BUILD_TOP` [fj#940](https://git.lix.systems/lix-project/lix/issues/940)
+
+  Previously, Lix `nix-shell`s could exit non-zero status when `stdenv`'s `dumpVars` phase failed to write to `$NIX_BUILD_TOP/env-vars`, despite `dumpVars` being intended as a debugging aid.
+
+  This happens when `TMPDIR` is not set and defaults therefore to `/tmp`, resulting in a `/tmp/env-vars` global file that every `nix-shell` wants to write.
+
+  We fix this issue by reusing a pre-created, unique, and writable location, as the build top directory, avoiding shell exiting from write failures silently.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
+
+- libstore/binary-cache-store: don't cache narinfo on nix copy, remove negative entry [cl/3789](https://gerrit.lix.systems/c/lix/+/3789)
+
+  When using e.g. [Snix's nar-bridge](https://snix.dev/docs/components/overview/#nar-bridge) via
+  an `http`-store, Lix would create cache entries with a wrong URL to the NAR when uploading
+  a store-path.
+
+  This caused hard build failures for Hydra.
+
+  Lix doesn't create these entries on upload anymore. Instead, it only removes negative cache entries.
+  The cache entry for a narinfo is now created the first time, Lix queries the cache
+  for the previously uploaded store-path again.
+
+  Many thanks to [ma27](https://git.lix.systems/ma27) for this.
+
+- Lix libraries can now be linked statically [fj#789](https://git.lix.systems/lix-project/lix/issues/789) [cl/3775](https://gerrit.lix.systems/c/lix/+/3775) [cl/3778](https://gerrit.lix.systems/c/lix/+/3778)
+
+  Previously the pkg-config files distributed with Lix were only suitable for dynamic linkage, causing "undefined reference to…" linker errors when trying to link statically.
+  Private dependency information has now been added to make static linkage work as expected without user intervention.
+  In addition, relevant static libraries are now prelinked to avoid strange failures due to missing static initializers.
+
+  Many thanks to [alois31](https://git.lix.systems/alois31) for this.
+
+- add description to zsh completions [fj#910](https://git.lix.systems/lix-project/lix/issues/910) [cl/3632](https://gerrit.lix.systems/c/lix/+/3632)
+
+  Emit descriptions when completing args in zsh completions. This uses the descriptions we already
+  provided in NIX\_GET\_COMPLETIONS.
+
+  Many thanks to [matthewbauer](https://github.com/matthewbauer) for this.
+
+
+## Miscellany
+
+- Deprecation of CA derivations, dynamic derivations, and impure derivations [fj#815](https://git.lix.systems/lix-project/lix/issues/815)
+
+  Content-addressed derivations are now deprecated and slated for removal in Lix 2.94.
+  We're doing this because the CA derivation system has been a known cause of problems
+  and inconsistencies, is unmaintained, habitually makes improving the store code very
+  difficult (or blocks such improvements outright), and is beset by a number of design
+  flaws that in our opinion cannot be fixed without a full reimplementation from zero.
+  Dynamic derivations and impure derivations are built on the CA derivation framework,
+  and owing to this they too are deprecated and slated for removal in another release.

docker.nix

@@ -8,7 +8,7 @@
   tag ? "latest",
   bundleNixpkgs ? true,
   channelName ? "nixpkgs",
-  channelURL ? "https://nixos.org/channels/nixpkgs-unstable",
+  channelURL ? "https://channels.nixos.org/nixpkgs-unstable",
   extraPkgs ? [ ],
   maxLayers ? 100,
   nixConf ? { },

flake.lock

@@ -108,11 +108,11 @@
     },
     "nixpkgs_2": {
       "locked": {
-        "lastModified": 1758391731,
-        "narHash": "sha256-UuwQoPWv13DVKMveeev+F0OC/N95AOmAz6SzCuGhxjQ=",
+        "lastModified": 1757198069,
+        "narHash": "sha256-m3VUcOD4rTs8J7S+3dOjWMrAjw6RcITC3XYQ98zhEFs=",
         "owner": "NixOS",
         "repo": "nixpkgs",
-        "rev": "3f00d36f15e16e0471d9ca1e8f88958941fa970a",
+        "rev": "0747026fc57ecb9c28901c7f7a2b5dc40e8af43c",
         "type": "github"
       },
       "original": {

flake.nix

@@ -88,7 +88,7 @@
         else
           "pre${
             builtins.substring 0 8 (self.lastModifiedDate or self.lastModified or "19700101")
-          }_${self.shortRev or "dirty"}";
+          }-dev_${self.shortRev or "dirty"}";
 
       linux32BitSystems = [ "i686-linux" ];
       linux64BitSystems = [
@@ -100,6 +100,7 @@
         "x86_64-darwin"
         "aarch64-darwin"
       ];
+      nonDarwinSystems = linuxSystems;
       systems = linuxSystems ++ darwinSystems;
 
       # If you add something here, please update the list in doc/manual/src/contributing/hacking.md.
@@ -259,6 +260,19 @@
             nativeBuildInputs = prevAttrs.nativeBuildInputs ++ [ final.buildPackages.bmake ];
             postInstall = lib.replaceStrings [ "lowdown.so.1" ] [ "lowdown.so.2" ] prevAttrs.postInstall;
           });
+
+          capnproto = prev.capnproto.overrideAttrs (old: {
+            patches =
+              old.patches or [ ]
+              ++ [
+                # backport of https://github.com/capnproto/capnproto/pull/1810
+                ./misc/capnproto-promise-nodiscard.patch
+              ]
+              ++ lib.optionals (lib.versionOlder old.version "1.2.0") [
+                # backport of https://github.com/capnproto/capnproto/pull/2296
+                ./misc/capnproto-monotonic-clocks-are-a-lie.patch
+              ];
+          });
         };
     in
     {
@@ -371,7 +385,10 @@
               ;
           }
           // {
-            nix-eval-jobs = forAllSystems (system: self.packages.${system}.nix-eval-jobs.tests.nix-eval-jobs);
+            # the n-e-j test suite is unusably slow in darwin ci. disbled until anywho fixes this.
+            nix-eval-jobs = (lib.genAttrs nonDarwinSystems) (
+              system: self.packages.${system}.nix-eval-jobs.tests.nix-eval-jobs
+            );
 
             # This is x86_64-linux only, just because we have significantly
             # cheaper x86_64-linux compute in CI.

lix/legacy/build-remote.cc

@@ -1,15 +1,18 @@
 #include "lix/libstore/path.hh"
 #include "lix/libutil/async.hh"
+#include "lix/libutil/c-calls.hh"
 #include "lix/libutil/error.hh"
 #include "lix/libutil/file-descriptor.hh"
+#include "lix/libutil/logging-rpc.hh"
 #include "lix/libutil/logging.hh"
 #include "lix/libutil/rpc.hh"
-#include "lix/libutil/types-rpc.hh"
+#include "lix/libutil/types-rpc.hh" // IWYU pragma: keep
+#include "lix/libutil/types.hh"
 #include <algorithm>
 #include <capnp/rpc-twoparty.h>
-#include <chrono>
 #include <cstring>
-#include <future>
+#include <exception>
+#include <kj/async.h>
 #include <kj/time.h>
 #include <set>
 #include <memory>
@@ -43,9 +46,11 @@ namespace {
 struct Instance final : rpc::build_remote::HookInstance::Server
 {
     unsigned int maxBuildJobs;
+    bool initialized = false, used = false;
 
-    Instance(unsigned int maxBuildJobs) : maxBuildJobs(maxBuildJobs) {}
+    kj::Promise<void> init(InitContext context) override;
 
+    kj::Promise<void> buildImpl(BuildContext context);
     kj::Promise<void> build(BuildContext context) override;
 };
 }
@@ -183,27 +188,62 @@ struct BuilderConnection
 
     // start the thread that reads ssh stderr and turns it into log items.
     // this future *must* outlive sshStore, otherwise it will never finish
-    std::future<void> startLogThread(int intoFD)
-    {
+    kj::Promise<Result<void>> startLogThread(std::string buildDescription, std::string drvPath)
+    try {
         if (!logPipe.readSide) {
-            return {};
+            co_return result::success();
         }
 
         logPipe.writeSide.close();
 
-        return std::async(
-            std::launch::async,
-            [](int from, int to) {
-                AsyncIoRoot aio;
-
-                auto reader = AIO().lowLevelProvider.wrapInputFd(from);
-                auto writer = AIO().lowLevelProvider.wrapOutputFd(to);
-
-                reader->pumpTo(*writer).wait(aio.kj.waitScope);
-            },
-            logPipe.readSide.get(),
-            intoFD
+        // NOTE this is very similar to handleBuilderOutput in DerivationGoal, but unlike
+        // the derivation goal we do not need to handle EIO from a pty here. we also have
+        // no timeouts or limits to keep track of, which makes deduplication less useful.
+        auto act = logger->startActivity(
+            lvlInfo, actBuild, buildDescription, Logger::Fields{drvPath, storeUri, 1, 1}
         );
+
+        std::map<ActivityId, Activity> activities;
+
+        auto reader = AIO().lowLevelProvider.wrapInputFd(logPipe.readSide.get());
+
+        LogLineSplitter splitter;
+
+        auto flushLine = [&](const std::string & line) {
+            if (const auto state =
+                    handleJSONLogMessage(line, act, activities, "the derivation builder"))
+            {
+                return *state;
+            } else {
+                return act.result(resBuildLogLine, line);
+            }
+        };
+
+        auto buf = kj::heapArray<char>(4096);
+        while (true) {
+            const auto got = co_await reader->tryRead(buf.begin(), 1, buf.size());
+            if (got == 0) {
+                break;
+            }
+
+            std::string_view data{buf.begin(), got};
+            while (!data.empty()) {
+                if (auto line = splitter.feed(data)) {
+                    if (flushLine(*line) == Logger::BufferState::NeedsFlush) {
+                        TRY_AWAIT(act.getLogger().flush());
+                    }
+                }
+            }
+        }
+
+        if (auto line = splitter.finish(); !line.empty()) {
+            (void) flushLine(line);
+            TRY_AWAIT(act.getLogger().flush());
+        }
+
+        co_return result::success();
+    } catch (...) {
+        co_return result::current_exception();
     }
 };
 
@@ -212,21 +252,16 @@ struct AcceptedBuild final : rpc::build_remote::HookInstance::AcceptedBuild::Ser
     ref<Store> store;
     StorePath drvPath;
     BuilderConnection builder;
-    rpc::build_remote::HookInstance::BuildLogger::Client buildLogger;
+    bool used = false;
 
-    AcceptedBuild(
-        ref<Store> store,
-        StorePath drvPath,
-        BuilderConnection builder,
-        rpc::build_remote::HookInstance::BuildLogger::Client buildLogger
-    )
+    AcceptedBuild(ref<Store> store, StorePath drvPath, BuilderConnection builder)
         : store(store)
         , drvPath(drvPath)
         , builder(std::move(builder))
-        , buildLogger(std::move(buildLogger))
     {
     }
 
+    kj::Promise<void> runImpl(RunContext context);
     kj::Promise<void> run(RunContext context) override;
 };
 
@@ -235,7 +270,7 @@ enum class BuildRejected { Temporarily, Permanently };
 
 static kj::Promise<Result<std::variant<BuildRejected, BuilderConnection>>> connectToBuilder(
     const ref<Store> & store,
-    const std::optional<StorePath> & drvPath,
+    const StorePath & drvPath,
     Machines & machines,
     const unsigned int maxBuildJobs,
     const bool amWilling,
@@ -255,7 +290,7 @@ try {
     bool canBuildLocally = amWilling && couldBuildLocally;
 
     /* Error ignored here, will be caught later */
-    mkdir(currentLoad.c_str(), 0777);
+    (void) sys::mkdir(currentLoad, 0777);
 
     while (true) {
         bestSlotLock.reset();
@@ -272,7 +307,7 @@ try {
             } else {
                 printSelectionFailureMessage(
                     couldBuildLocally ? lvlChatty : lvlWarn,
-                    drvPath ? drvPath->to_string() : "<unknown>",
+                    drvPath.to_string(),
                     machines,
                     neededSystem,
                     requiredFeatures
@@ -294,8 +329,8 @@ try {
         Pipe logPipe;
 
         try {
-            Activity act(
-                *logger, lvlTalkative, actUnknown, fmt("connecting to '%s'", bestMachine->storeUri)
+            auto act = logger->startActivity(
+                lvlTalkative, actUnknown, fmt("connecting to '%s'", bestMachine->storeUri)
             );
 
             std::tie(sshStore, logPipe) = TRY_AWAIT(bestMachine->openStore());
@@ -321,8 +356,6 @@ try {
 static int main_build_remote(AsyncIoRoot & aio, std::string programName, Strings argv)
 {
     {
-        logger = makeJSONLogger(*logger);
-
         /* Ensure we don't get any SSH passphrase or host key popups. */
         unsetenv("DISPLAY");
         unsetenv("SSH_ASKPASS");
@@ -335,104 +368,146 @@ static int main_build_remote(AsyncIoRoot & aio, std::string programName, Strings
 
         verbosity = (Verbosity) std::stoll(argv.front());
 
-        FdSource source(STDIN_FILENO);
-
-        /* Read the parent's settings. */
-        while (readNum<unsigned>(source)) {
-            auto name = readString(source);
-            auto value = readString(source);
-            settings.set(name, value);
-        }
-
-        auto maxBuildJobs = settings.maxBuildJobs;
-        settings.maxBuildJobs.set("1"); // hack to make tests with local?root= work
-
-        initPlugins();
-
         auto conn = aio.kj.lowLevelProvider->wrapUnixSocketFd(1);
-        capnp::TwoPartyServer srv(kj::heap<Instance>(maxBuildJobs));
+        capnp::TwoPartyServer srv(kj::heap<Instance>());
         srv.accept(*conn, 1).wait(aio.kj.waitScope);
         return 0;
     }
 }
 
-kj::Promise<void> Instance::build(BuildContext context)
+kj::Promise<void> Instance::init(InitContext context)
 {
     try {
-        // FIXME this does not open a daemon connection for historical reasons.
-        // we may create a lot of build hook instances, and having each of them
-        // also create a daemon instance is inefficient and wasteful. in future
-        // versions of the build hook (where we don't need one hook process per
-        // build) we should change this to using a daemon connection, ideally a
-        // daemon connection provided by the parent via file descriptor passing
-        auto store = TRY_AWAIT(openStore(settings.storeUri, {}, AllowDaemon::Disallow));
-
-        /* It would be more appropriate to use $XDG_RUNTIME_DIR, since
-           that gets cleared on reboot, but it wouldn't work on macOS. */
-        auto currentLoadName = "/current-load";
-        if (auto localStore = store.try_cast_shared<LocalFSStore>())
-            currentLoad = std::string { localStore->config().stateDir } + currentLoadName;
-        else
-            currentLoad = settings.nixStateDir + currentLoadName;
-
-        auto machines = getMachines();
-        debug("got %d remote builders", machines.size());
-
-        if (machines.empty()) {
-            context.getResults().initResult().initGood().setDeclinePermanently();
-            co_return;
+        if (initialized) {
+            throw Error("build hook can only be initialized once");
         }
 
-        auto amWilling = context.getParams().getAmWilling();
-        auto neededSystem = rpc::to<std::string>(context.getParams().getNeededSystem());
-        auto drvPath = from(context.getParams().getDrvPath(), *store);
-        auto requiredFeatures =
-            rpc::to<std::set<std::string>>(context.getParams().getRequiredFeatures());
-        auto buildLogger = context.getParams().getBuildLogger();
+        logger = rpc::log::makeRpcLoggerClient(context.getParams().getLogger());
 
-        auto result = TRY_AWAIT(connectToBuilder(
-            store, drvPath, machines, maxBuildJobs, amWilling, neededSystem, requiredFeatures
-        ));
-
-        if (auto immediateResponse = std::get_if<BuildRejected>(&result)) {
-            switch (*immediateResponse) {
-            case BuildRejected::Temporarily:
-                context.getResults().initResult().initGood().setPostpone();
-                co_return;
-            case BuildRejected::Permanently:
-                context.getResults().initResult().initGood().setDecline();
-                co_return;
-            }
+        /* Read the parent's settings. */
+        for (const auto & [name, value] : rpc::to<StringMap>(context.getParams().getSettings())) {
+            settings.set(name, value);
         }
 
-        auto builder = std::get_if<BuilderConnection>(&result);
-        assert(builder);
+        maxBuildJobs = settings.maxBuildJobs;
+        settings.maxBuildJobs.set("1"); // hack to make tests with local?root= work
 
-        auto ac = context.getResults().initResult().initGood().initAccept();
-        RPC_FILL(ac, setMachineName, builder->storeUri);
-        ac.setMachine(kj::heap<AcceptedBuild>(store, drvPath, std::move(*builder), buildLogger));
+        initPlugins();
+
+        initialized = true;
+
+        context.getResults().initResult().setGood();
     } catch (...) {
         RPC_FILL(context.getResults(), initResult, std::current_exception());
     }
+
+    return kj::READY_NOW;
+}
+
+kj::Promise<void> Instance::buildImpl(BuildContext context)
+{
+    if (!initialized) {
+        throw Error("build hook not fully initialized");
+    }
+
+    // FIXME this does not open a daemon connection for historical reasons.
+    // we may create a lot of build hook instances, and having each of them
+    // also create a daemon instance is inefficient and wasteful. in future
+    // versions of the build hook (where we don't need one hook process per
+    // build) we should change this to using a daemon connection, ideally a
+    // daemon connection provided by the parent via file descriptor passing
+    auto store = TRY_AWAIT(openStore(settings.storeUri, {}, AllowDaemon::Disallow));
+
+    /* It would be more appropriate to use $XDG_RUNTIME_DIR, since
+       that gets cleared on reboot, but it wouldn't work on macOS. */
+    auto currentLoadName = "/current-load";
+    if (auto localStore = store.try_cast_shared<LocalFSStore>()) {
+        currentLoad = std::string{localStore->config().stateDir} + currentLoadName;
+    } else {
+        currentLoad = settings.nixStateDir + currentLoadName;
+    }
+
+    auto machines = getMachines();
+    debug("got %d remote builders", machines.size());
+
+    if (machines.empty()) {
+        context.getResults().initResult().initGood().setDeclinePermanently();
+        co_return;
+    }
+
+    auto amWilling = context.getParams().getAmWilling();
+    auto neededSystem = rpc::to<std::string>(context.getParams().getNeededSystem());
+    auto drvPath = from(context.getParams().getDrvPath(), *store);
+    auto requiredFeatures =
+        rpc::to<std::set<std::string>>(context.getParams().getRequiredFeatures());
+
+    auto result = TRY_AWAIT(connectToBuilder(
+        store, drvPath, machines, maxBuildJobs, amWilling, neededSystem, requiredFeatures
+    ));
+
+    if (auto immediateResponse = std::get_if<BuildRejected>(&result)) {
+        switch (*immediateResponse) {
+        case BuildRejected::Temporarily:
+            context.getResults().initResult().initGood().setPostpone();
+            co_return;
+        case BuildRejected::Permanently:
+            context.getResults().initResult().initGood().setDecline();
+            co_return;
+        }
+    }
+
+    auto builder = std::get_if<BuilderConnection>(&result);
+    assert(builder);
+
+    auto ac = context.getResults().initResult().initGood().initAccept();
+    ac.setMachine(kj::heap<AcceptedBuild>(store, drvPath, std::move(*builder)));
+}
+
+kj::Promise<void> Instance::build(BuildContext context)
+try {
+    if (used) {
+        throw Error("build hooks can only accept a single job");
+    }
+    used = true; // lock out other rpc calls during processing
+    co_await buildImpl(context);
+    TRY_AWAIT(logger->flush());
+    used = context.getResults().getResult().getGood().isAccept();
+} catch (...) {
+    RPC_FILL(context.getResults(), getResult, std::current_exception());
 }
 
 kj::Promise<void> AcceptedBuild::run(RunContext context)
 {
     try {
-        const int logFD = (co_await buildLogger.getFd()).orDefault(-1);
-        if (logFD < 0) {
-            throw Error("build-hook needs a logFD from the builder to build");
-        }
-
-        auto logThread = builder.startLogThread(logFD);
+        auto oldLogger = logger;
+        logger = rpc::log::makeRpcLoggerClient(context.getParams().getLogger());
+        TRY_AWAIT(oldLogger->flush());
         KJ_DEFER({
-            // drop any existing ssh connection so the log thread can exit
-            builder.sshStore = nullptr;
-            if (logThread.valid()) {
-                logThread.get();
-            }
+            delete logger;
+            logger = oldLogger;
         });
 
+        if (used) {
+            throw Error("build hooks builds are single-use items");
+        }
+        used = true;
+        co_await runImpl(context);
+        TRY_AWAIT(logger->flush());
+    } catch (...) {
+        RPC_FILL(context.getResults(), getResult, std::current_exception());
+    }
+}
+
+kj::Promise<void> AcceptedBuild::runImpl(RunContext context)
+{
+    try {
+        auto logHandler = builder.startLogThread(
+            fmt("%s on '%s'",
+                rpc::to<std::string_view>(context.getParams().getDescription()),
+                builder.storeUri),
+            store->printStorePath(drvPath)
+        );
+
         auto & sshStore = builder.sshStore;
         auto & storeUri = builder.storeUri;
 
@@ -444,7 +519,9 @@ kj::Promise<void> AcceptedBuild::run(RunContext context)
         AutoCloseFD uploadLock = openLockFile(lockFileName, true);
 
         {
-            Activity act(*logger, lvlTalkative, actUnknown, fmt("waiting for the upload lock to '%s'", storeUri));
+            auto act = logger->startActivity(
+                lvlTalkative, actUnknown, fmt("waiting for the upload lock to '%s'", storeUri)
+            );
 
             auto result = TRY_AWAIT(
                 AIO().timeoutAfter(15 * kj::MINUTES, lockFileAsync(uploadLock.get(), ltWrite))
@@ -457,7 +534,9 @@ kj::Promise<void> AcceptedBuild::run(RunContext context)
         auto substitute = settings.buildersUseSubstitutes ? Substitute : NoSubstitute;
 
         {
-            Activity act(*logger, lvlTalkative, actUnknown, fmt("copying dependencies to '%s'", storeUri));
+            auto act = logger->startActivity(
+                lvlTalkative, actUnknown, fmt("copying dependencies to '%s'", storeUri)
+            );
             TRY_AWAIT(copyPaths(*store, *sshStore, inputs, NoRepair, NoCheckSigs, substitute));
         }
 
@@ -527,7 +606,9 @@ kj::Promise<void> AcceptedBuild::run(RunContext context)
         }
 
         if (!missingPaths.empty()) {
-            Activity act(*logger, lvlTalkative, actUnknown, fmt("copying outputs from '%s'", storeUri));
+            auto act = logger->startActivity(
+                lvlTalkative, actUnknown, fmt("copying outputs from '%s'", storeUri)
+            );
             if (auto localStore = store.try_cast_shared<LocalStore>())
                 for (auto & path : missingPaths)
                     localStore->locksHeld.insert(store->printStorePath(path)); /* FIXME: ugly */
@@ -536,6 +617,10 @@ kj::Promise<void> AcceptedBuild::run(RunContext context)
             );
         }
 
+        // drop store connection, let log handler process any remaining input
+        builder.sshStore = nullptr;
+        TRY_AWAIT(logHandler);
+
         context.getResults().initResult().setGood();
     } catch (...) {
         RPC_FILL(context.getResults(), initResult, std::current_exception());

lix/legacy/nix-build.cc

@@ -9,6 +9,7 @@
 #include "lix/libstore/store-api.hh"
 #include "lix/libstore/local-fs-store.hh"
 #include "lix/libstore/globals.hh"
+#include "lix/libutil/c-calls.hh"
 #include "lix/libutil/current-process.hh"
 #include "lix/libstore/derivations.hh"
 #include "lix/libmain/shared.hh"
@@ -18,6 +19,8 @@
 #include "lix/libcmd/common-eval-args.hh"
 #include "lix/libexpr/attr-path.hh"
 #include "lix/libcmd/legacy.hh"
+#include "lix/libutil/finally.hh"
+#include "lix/libutil/processes.hh"
 #include "lix/libutil/regex.hh"
 #include "lix/libutil/shlex.hh"
 #include "nix-build.hh"
@@ -29,7 +32,7 @@ namespace nix {
 
 using namespace std::string_literals;
 
-static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings argv)
+static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings argv)
 {
     auto dryRun = false;
     auto runEnv = std::regex_search(programName, regex::parse("nix-shell$"));
@@ -187,7 +190,8 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
     if (packages && fromArgs)
         throw UsageError("'-p' and '-E' are mutually exclusive");
 
-    AutoDelete tmpDir(createTempDir("", myName));
+    AutoDelete tmpDir(createTempDir(myName));
+    AutoDelete buildTopTmpDir(createTempSubdir(tmpDir, "build-top"));
     if (outLink.empty())
         outLink = (Path) tmpDir + "/result";
 
@@ -224,8 +228,9 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
             left = {"default.nix"};
     }
 
-    if (runEnv)
-        setenv("IN_NIX_SHELL", pure ? "pure" : "impure", 1);
+    if (runEnv) {
+        (void) sys::setenv("IN_NIX_SHELL", pure ? "pure" : "impure", 1);
+    }
 
     DrvInfos drvs;
 
@@ -397,7 +402,9 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
 
         buildPaths(pathsToBuild);
 
-        if (dryRun) return;
+        if (dryRun) {
+            return 0;
+        }
 
         if (shellDrv) {
             auto shellDrvOutputs =
@@ -419,24 +426,24 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
         }
 
         // Don't use defaultTempDir() here! We want to preserve the user's TMPDIR for the shell
-        env["NIX_BUILD_TOP"] = env["TMPDIR"] = env["TEMPDIR"] = env["TMP"] = env["TEMP"] = getEnvNonEmpty("TMPDIR").value_or("/tmp");
+        env["NIX_BUILD_TOP"] = env["TMPDIR"] = env["TEMPDIR"] = env["TMP"] = env["TEMP"] =
+            getEnvNonEmpty("TMPDIR").value_or(buildTopTmpDir);
         env["NIX_STORE"] = store->config().storeDir;
         env["NIX_BUILD_CORES"] = std::to_string(settings.buildCores);
 
         auto passAsFile = tokenizeString<StringSet>(getOr(drv.env, "passAsFile", ""));
 
-        bool keepTmp = false;
         int fileNr = 0;
 
         for (auto & var : drv.env)
             if (passAsFile.count(var.first)) {
-                keepTmp = true;
                 auto fn = ".attr-" + std::to_string(fileNr++);
                 Path p = (Path) tmpDir + "/" + fn;
                 writeFile(p, var.second);
                 env[var.first + "Path"] = p;
-            } else
+            } else {
                 env[var.first] = var.second;
+            }
 
         std::string structuredAttrsRC;
 
@@ -469,7 +476,6 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
 
                 env["NIX_ATTRS_SH_FILE"] = attrsSH;
                 env["NIX_ATTRS_JSON_FILE"] = attrsJSON;
-                keepTmp = true;
             }
         }
 
@@ -479,24 +485,13 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
            lose the current $PATH directories. */
         auto rcfile = (Path) tmpDir + "/rc";
         auto tz = getEnv("TZ");
-        std::string rc = fmt(
-            R"(_nix_shell_clean_tmpdir() { command rm -rf %1%; }; )"
-            "%2%"
-            "%3%"
-            // always clear PATH.
-            // when nix-shell is run impure, we rehydrate it with the `p=$PATH` above
-            "unset PATH;"
-            "dontAddDisableDepTrack=1;\n",
-            shellEscape(tmpDir),
-            (keepTmp
-                ? "trap _nix_shell_clean_tmpdir EXIT; "
-                  "exitHooks+=(_nix_shell_clean_tmpdir); "
-                  "failureHooks+=(_nix_shell_clean_tmpdir); "
-                : "_nix_shell_clean_tmpdir; "),
-            (pure
-                ? ""
-                : "[ -n \"$PS1\" ] && [ -e ~/.bashrc ] && source ~/.bashrc; p=$PATH; ")
-        );
+        std::string rc =
+            fmt("%1%"
+                // always clear PATH.
+                // when nix-shell is run impure, we rehydrate it with the `p=$PATH` above
+                "unset PATH;"
+                "dontAddDisableDepTrack=1;\n",
+                (pure ? "" : "[ -n \"$PS1\" ] && [ -e ~/.bashrc ] && source ~/.bashrc; p=$PATH; "));
         rc += structuredAttrsRC;
         rc += fmt(
             "\n[ -e $stdenv/setup ] && source $stdenv/setup; "
@@ -526,29 +521,38 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
         vomit("Sourcing nix-shell with file %s and contents:\n%s", rcfile, rc);
         writeFile(rcfile, rc);
 
-        Strings envStrs;
-        for (auto & i : env)
-            envStrs.push_back(i.first + "=" + i.second);
-
-        auto args = interactive
-            ? Strings{"bash", "--rcfile", rcfile}
-            : Strings{"bash", rcfile};
-
-        auto envPtrs = stringsToCharPtrs(envStrs);
-
-        environ = envPtrs.data();
-
-        auto argPtrs = stringsToCharPtrs(args);
-
-        restoreProcessContext();
+        auto args = interactive ? Strings{"--rcfile", rcfile} : Strings{rcfile};
 
+        // We are going to run an interactive command, do not let the logger send a line.
         logger->pause();
 
         printMsg(lvlChatty, "running shell: %s", concatMapStringsSep(" ", args, shellEscape));
 
-        execvp(shell->c_str(), argPtrs.data());
+        RunningProgram proc = runProgram2(
+            {.program = *shell,
+             .searchPath = true,
+             .args = args,
+             .environment = env,
+             .dieWithParent = true}
+        );
 
-        throw SysError("executing shell '%s'", *shell);
+        // NOTE: we wait and return the status check immediately.
+        // If there's interruption, we will swallow it and wait again for termination.
+        auto toExitStatus = [](int waitRes) {
+            if (WIFEXITED(waitRes)) {
+                return WEXITSTATUS(waitRes);
+            } else if (WIFSIGNALED(waitRes)) {
+                return 128 + WTERMSIG(waitRes);
+            } else {
+                return 255;
+            }
+        };
+
+        try {
+            return toExitStatus(proc.wait());
+        } catch (Interrupted &) {
+            return toExitStatus(proc.wait());
+        }
     }
 
     else {
@@ -582,7 +586,9 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
 
         buildPaths(pathsToBuild);
 
-        if (dryRun) return;
+        if (dryRun) {
+            return 0;
+        }
 
         std::vector<StorePath> outPaths;
 
@@ -611,6 +617,8 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
         for (auto & path : outPaths)
             std::cout << store->printStorePath(path) << '\n';
     }
+
+    return 0;
 }
 
 void registerLegacyNixBuildAndNixShell() {

lix/legacy/nix-channel.cc

@@ -8,6 +8,7 @@
 #include "lix/libexpr/eval-settings.hh" // for defexpr
 #include "lix/libstore/temporary-dir.hh"
 #include "lix/libutil/async.hh"
+#include "lix/libutil/c-calls.hh"
 #include "lix/libutil/regex.hh"
 #include "lix/libutil/result.hh"
 #include "lix/libutil/users.hh"
@@ -44,7 +45,7 @@ static void readChannels()
 // Writes the list of channels.
 static void writeChannels()
 {
-    auto channelsFD = AutoCloseFD{open(channelsList.c_str(), O_WRONLY | O_CLOEXEC | O_CREAT | O_TRUNC, 0644)};
+    auto channelsFD = sys::open(channelsList, O_WRONLY | O_CLOEXEC | O_CREAT | O_TRUNC, 0644);
     if (!channelsFD)
         throw SysError("opening '%1%' for writing", channelsList);
     for (const auto & channel : channels)
@@ -174,11 +175,12 @@ static void update(AsyncIoRoot & aio, const StringSet & channelNames)
 
     // Make the channels appear in nix-env.
     struct stat st;
-    if (lstat(nixDefExpr.c_str(), &st) == 0) {
+    if (sys::lstat(nixDefExpr, &st) == 0) {
         if (S_ISLNK(st.st_mode))
             // old-skool ~/.nix-defexpr
-            if (unlink(nixDefExpr.c_str()) == -1)
+            if (sys::unlink(nixDefExpr) == -1) {
                 throw SysError("unlinking %1%", nixDefExpr);
+            }
     } else if (errno != ENOENT) {
         throw SysError("getting status of %1%", nixDefExpr);
     }

lix/legacy/nix-collect-garbage.cc

@@ -1,3 +1,4 @@
+#include "lix/libutil/c-calls.hh"
 #include "lix/libutil/file-system.hh"
 #include "lix/libstore/store-api.hh"
 #include "lix/libstore/store-cast.hh"
@@ -24,9 +25,11 @@ bool dryRun = false;
 
 static void removeOldGenerations(std::string dir, NeverAsync = {})
 {
-    if (access(dir.c_str(), R_OK) != 0) return;
+    if (sys::access(dir, R_OK) != 0) {
+        return;
+    }
 
-    bool canWrite = access(dir.c_str(), W_OK) == 0;
+    bool canWrite = sys::access(dir, W_OK) == 0;
 
     for (auto & i : readDirectory(dir)) {
         checkInterrupt();

lix/legacy/nix-store.cc

@@ -25,6 +25,7 @@
 #include <iostream>
 #include <algorithm>
 
+#include <ranges>
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <fcntl.h>
@@ -88,6 +89,16 @@ try {
             if (i == drv.outputs.end())
                 throw Error("derivation '%s' does not have an output named '%s'",
                     store2->printStorePath(path.path), j);
+            if (!outputPaths.contains(i->first)) {
+                throw Error(
+                    "Possible SQLite database corruption: derivation '%s' output map contains only "
+                    "outputs '{%s}', not '%s'\n"
+                    "Note: derivation output maps are stored in the SQLite database.",
+                    store2->printStorePath(path.path),
+                    concatStringsSep(", ", std::views::keys(outputPaths)),
+                    i->first
+                );
+            }
             auto outPath = outputPaths.at(i->first);
             auto retPath = store->printStorePath(outPath);
             if (store2) {

lix/libcmd/command.cc

@@ -5,6 +5,8 @@
 #include "lix/libstore/profiles.hh"
 #include "lix/libcmd/repl.hh"
 #include "lix/libutil/async.hh"
+#include "lix/libutil/c-calls.hh"
+#include "lix/libutil/error.hh"
 
 extern char * * environ __attribute__((weak));
 
@@ -303,8 +305,10 @@ void MixEnvironment::setEnviron() {
             throw UsageError("--unset does not make sense with --ignore-environment");
 
         for (const auto & var : keep) {
-            auto val = getenv(var.c_str());
-            if (val) stringsEnv.emplace_back(fmt("%s=%s", var.c_str(), val));
+            auto val = sys::getenv(var);
+            if (val) {
+                stringsEnv.emplace_back(fmt("%s=%s", var, val));
+            }
         }
 
         vectorEnv = stringsToCharPtrs(stringsEnv);
@@ -314,7 +318,7 @@ void MixEnvironment::setEnviron() {
             throw UsageError("--keep does not make sense without --ignore-environment");
 
         for (const auto & var : unset)
-            unsetenv(var.c_str());
+            (void) sys::unsetenv(var);
     }
 }
 

lix/libcmd/common-eval-args.cc

@@ -113,7 +113,7 @@ MixEvalArgs::MixEvalArgs()
 
   ```
   -I nixpkgs=channel:nixos-21.05
-  -I nixpkgs=https://nixos.org/channels/nixos-21.05/nixexprs.tar.xz
+  -I nixpkgs=https://channels.nixos.org/nixos-21.05/nixexprs.tar.xz
   ```
 
   You can also fetch source trees using [flake URLs](./nix3-flake.md#url-like-syntax) and add them to the

lix/libcmd/installable-flake.cc

@@ -60,7 +60,8 @@ InstallableFlake::InstallableFlake(
 
 DerivedPathsWithInfo InstallableFlake::toDerivedPaths(EvalState & state)
 {
-    Activity act(*logger, lvlTalkative, actUnknown, fmt("evaluating derivation '%s'", what()));
+    auto act =
+        logger->startActivity(lvlTalkative, actUnknown, fmt("evaluating derivation '%s'", what()));
 
     auto attr = getCursor(state);
 

lix/libcmd/legacy.hh

@@ -9,7 +9,7 @@
 
 namespace nix {
 
-typedef std::function<void(AsyncIoRoot &, std::string, std::list<std::string>)> MainFunction;
+typedef std::function<int(AsyncIoRoot &, std::string, std::list<std::string>)> MainFunction;
 
 struct LegacyCommandRegistry
 {

lix/libcmd/markdown.cc

@@ -74,7 +74,11 @@ std::string renderMarkdownToTerminal(std::string_view markdown, StandardOutputSt
         .vmargin = 0,
 #endif /* LOWDOWN_SEPARATE_TERM_OPTS */
         .feat = LOWDOWN_COMMONMARK | LOWDOWN_FENCED | LOWDOWN_DEFLIST | LOWDOWN_TABLES,
+#ifdef LOWDOWN_CONSOLIDATED_OFLAGS
+        .oflags = LOWDOWN_NOLINK,
+#else
         .oflags = LOWDOWN_TERM_NOLINK,
+#endif /* LOWDOWN_CONSOLIDATED_OFLAGS */
     };
     if (!shouldANSI(fileno)) {
         opts.oflags |= LOWDOWN_TERM_NOANSI;

lix/libcmd/repl-interacter.cc

@@ -1,3 +1,4 @@
+#include "lix/libutil/c-calls.hh"
 #include "lix/libutil/error.hh"
 #include "lix/libutil/file-system.hh"
 #include "lix/libutil/logging.hh"
@@ -110,7 +111,7 @@ static el_status_t doCompletion() {
     if (possible.size() == 1) {
         const auto completion = *possible.cbegin();
         if (completion.size() > s.size()) {
-            rl_insert_text(completion.c_str() + s.size());
+            rl_insert_text(requireCString(completion.substr(s.size())));
             return redisplay();
         }
 
@@ -134,7 +135,7 @@ static el_status_t doCompletion() {
     }
     if (len > 0) {
         auto commonPrefix = possible.begin()->substr(start, len);
-        rl_insert_text(commonPrefix.c_str());
+        rl_insert_text(requireCString(commonPrefix));
         el_ring_bell();
         return redisplay();
     }
@@ -154,7 +155,7 @@ ReadlineLikeInteracter::Guard ReadlineLikeInteracter::init(detail::ReplCompleter
         logWarning(e.info());
     }
     el_hist_size = 1000;
-    read_history(historyFile.c_str());
+    read_history(requireCString(historyFile));
     auto oldRepl = curRepl;
     curRepl = repl;
     Guard restoreRepl([oldRepl] { curRepl = oldRepl; });
@@ -202,7 +203,7 @@ bool ReadlineLikeInteracter::getLine(std::string & input, ReplPromptType promptT
     };
 
     setupSignals();
-    char * s = readline(promptForType(promptType));
+    char * s = readline(promptForType(promptType)); // NOLINT(lix-unsafe-c-calls)
     Finally doFree([&]() { free(s); });
     restoreSignals();
 
@@ -223,7 +224,7 @@ bool ReadlineLikeInteracter::getLine(std::string & input, ReplPromptType promptT
 
 void ReadlineLikeInteracter::writeHistory()
 {
-    int ret = write_history(historyFile.c_str());
+    int ret = write_history(requireCString(historyFile));
     int writeHistErr = errno;
 
     if (ret == 0) {

lix/libcmd/repl.cc

@@ -56,6 +56,7 @@ using NdString = std::unique_ptr<const char, decltype(&lixdoc_free_string)>;
  */
 NdString lambdaDocsForPos(SourcePath const path, nix::Pos const &pos) {
   std::string const file = path.to_string();
+  // NOLINTNEXTLINE(lix-unsafe-c-calls): paths are safe
   return NdString{lixdoc_get_function_docs(file.c_str(), pos.line, pos.column), &lixdoc_free_string};
 }
 
@@ -1003,8 +1004,8 @@ Value NixRepl::getReplOverlaysEvalFunction()
 Value NixRepl::replOverlays()
 {
     Value replInits;
-    auto replInitElems = evaluator.mem.newList(evalSettings.replOverlays.get().size());
-    replInits = {NewValueAs::list, replInitElems};
+    auto replInitStorage = evaluator.mem.newList(evalSettings.replOverlays.get().size());
+    replInits = {NewValueAs::list, replInitStorage};
 
     size_t i = 0;
     for (auto path : evalSettings.replOverlays.get()) {
@@ -1045,7 +1046,7 @@ Value NixRepl::replOverlays()
                 .debugThrow();
         }
 
-        replInitElems->elems[i] = replInit;
+        replInitStorage->elems[i] = replInit;
         i++;
     }
 

lix/libexpr/attr-set.hh

@@ -43,7 +43,7 @@ static_assert(sizeof(Attr) == 2 * sizeof(uint32_t) + sizeof(Value *),
  * elements allocated after this structure, while the size corresponds to
  * the number of elements already inserted in this structure.
  */
-class Bindings
+class alignas(Value::TAG_ALIGN) Bindings
 {
 public:
     using Size = uint32_t;

lix/libexpr/eval-inline.hh

@@ -20,18 +20,28 @@ inline Value::Value(app_t, EvalMemory & mem, Value & lhs, Value & rhs)
 }
 
 inline Value::Value(app_t, EvalMemory & mem, Value & lhs, std::span<Value> args)
+    : Value(app_t{}, mem, lhs, args, {})
 {
-    auto app = static_cast<Value::App *>(mem.allocBytes(sizeof(Value::App) + args.size_bytes()));
+}
+
+inline Value::Value(
+    app_t, EvalMemory & mem, const Value & lhs, std::span<Value> baseArgs, std::span<Value> moreArgs
+)
+{
+    auto app = static_cast<Value::App *>(
+        mem.allocBytes(sizeof(Value::App) + baseArgs.size_bytes() + moreArgs.size_bytes())
+    );
     app->_left = lhs;
-    app->_n = args.size();
-    std::copy(args.begin(), args.end(), app->_args);
+    app->_n = baseArgs.size() + moreArgs.size();
+    std::copy(baseArgs.begin(), baseArgs.end(), app->_args);
+    std::copy(moreArgs.begin(), moreArgs.end(), app->_args + baseArgs.size());
     raw = tag(tApp, app);
 }
 
 inline Value::Value(thunk_t, EvalMemory & mem, Env & env, Expr & expr)
 {
     auto thunk = mem.allocType<Thunk>();
-    *thunk = {._env = reinterpret_cast<uintptr_t>(&env), .expr = &expr};
+    *thunk = {._env = &env, .expr = &expr};
     raw = tag(tThunk, thunk);
 }
 
@@ -81,6 +91,7 @@ template<typename T>
 [[gnu::always_inline]]
 T * EvalMemory::allocType(size_t n)
 {
+    // NOLINTNEXTLINE(bugprone-sizeof-expression)
     return static_cast<T *>(allocBytes(checkedArrayAllocSize(sizeof(T), n)));
 }
 

lix/libexpr/eval-settings.cc

@@ -82,7 +82,7 @@ bool EvalSettings::isPseudoUrl(std::string_view s)
 std::string EvalSettings::resolvePseudoUrl(std::string_view url)
 {
     if (url.starts_with("channel:"))
-        return "https://nixos.org/channels/" + std::string(url.substr(8)) + "/nixexprs.tar.xz";
+        return "https://channels.nixos.org/" + std::string(url.substr(8)) + "/nixexprs.tar.xz";
     else
         return std::string(url);
 }

lix/libexpr/eval.cc

@@ -39,10 +39,26 @@
 #include <sys/resource.h>
 #include <fstream>
 #include <functional>
+#include <ranges>
 
 #include <sys/resource.h>
 #include <boost/container/small_vector.hpp>
 
+// Ignore all internal signals boehm uses for parallel marking
+// FIXME: Find out how to do this with LLDB for macOS!
+#ifndef __APPLE__
+[[gnu::section(".debug_gdb_scripts"), gnu::used, gnu::aligned(1)]]
+// TODO: We should use `SECTION_SCRIPT_ID_PYTHON_TEXT` from
+// `<gdb/section-scripts.h>` instead of hardcoding 4.
+// But why isn't this header exported by GDB?
+static const char printer_script[] =
+    "\4"
+    R"(lix-ignore-boehm-signals
+import gdb
+gdb.execute("handle SIGPWR SIGXCPU ignore")
+)";
+#endif
+
 #if HAVE_BOEHMGC
 
 #define GC_INCLUDE_NEW
@@ -177,6 +193,9 @@ void initLibExpr()
 
     GC_INIT();
 
+    // Enable parallel marking
+    GC_start_mark_threads();
+
     GC_set_oom_fn(oomHandler);
 
     /* Set the initial heap size to something fairly big (25% of
@@ -573,20 +592,20 @@ std::ostream & operator<<(std::ostream & output, const PrimOp & primOp)
     return output;
 }
 
-void EvalBuiltins::addPrimOp(PrimOpDetails && primOp)
+void EvalBuiltins::addPrimOp(PrimOpDetails primOp)
 {
     /* Hack to make constants lazy: turn them into a application of
        the primop to a dummy value. */
     if (primOp.arity == 0) {
         primOp.arity = 1;
-        Value vPrimOp{NewValueAs::primop, *new PrimOp(std::move(primOp))};
+        Value vPrimOp{NewValueAs::primop, *new PrimOp(primOp)};
         Value v{NewValueAs::app, mem, vPrimOp, vPrimOp};
         addConstant(
             vPrimOp.primOp()->name,
             v,
             {
                 .type = nFunction,
-                .doc = primOp.doc,
+                .doc = vPrimOp.primOp()->doc,
             }
         );
     }
@@ -1285,113 +1304,149 @@ static std::string showAttrPath(EvalState & state, Env & env, const AttrPath & a
     return out.str();
 }
 
+/** Returns `nullptr` if we should be using a default instead. */
+Attr const * ExprSelect::selectSingleAttr(
+    EvalState & state, Env & env, AttrName const & attrName, Value & vCurrent
+)
+{
+    Symbol const attrSym = getName(attrName, state, env);
+
+    try {
+        state.forceValue(vCurrent, pos);
+    } catch (Error & e) {
+        // clang-format off
+        e.addTrace(state.ctx.positions[attrName.pos], HintFmt(
+            "while evaluating an expression to select '%s' on it", state.ctx.symbols[attrSym]
+        ));
+        // clang-format on
+        throw;
+    }
+
+    if (vCurrent.type() != nAttrs) {
+        // If we have an `or` provided default, then it doesn't have to be an attrset.
+        // Let the caller know there's no attr value here.
+        if (def != nullptr) {
+            return nullptr;
+        }
+
+        // Otherwise, we must type error.
+        // clang-format off
+        state.ctx.errors.make<TypeError>(
+            "expected a set but found %s: %s",
+            showType(vCurrent),
+            ValuePrinter(state, vCurrent, errorPrintOptions)
+        ).addTrace(
+            attrName.pos,
+            HintFmt("while selecting '%s'", state.ctx.symbols[attrSym])
+        ).debugThrow();
+        // clang-format on
+    }
+
+    // Now that we know it's an attrset, we can actually look for the name.
+
+    auto const attrIt = vCurrent.attrs()->get(attrSym);
+    if (!attrIt) {
+
+        // Again if we have an `or` provided default, then missing attr is not an error.
+        if (def != nullptr) {
+            return nullptr;
+        }
+
+        // Otherwise, we collect all attr names and throw an attr missing error.
+
+        std::set<std::string> const allAttrNames = *vCurrent.attrs()
+            | std::views::transform([&state](auto const & attr) {
+                  return std::string{state.ctx.symbols[attr.name]};
+              })
+            | std::ranges::to<std::set>();
+
+        auto suggestions = Suggestions::bestMatches(allAttrNames, state.ctx.symbols[attrSym]);
+        state.ctx.errors.make<EvalError>("attribute '%s' missing", state.ctx.symbols[attrSym])
+            .atPos(attrName.pos)
+            .withSuggestions(suggestions)
+            .withFrame(env, *this)
+            .debugThrow();
+    }
+
+    // If we made it here, then we successfully found the attribute.
+    // Return it to our caller!
+
+    return attrIt;
+}
 
 void ExprSelect::eval(EvalState & state, Env & env, Value & v)
 {
-    Value vFirst;
-
-    // Pointer to the current attrset Value in this select chain.
-    Value * vCurrent = &vFirst;
     // Position for the current attrset Value in this select chain.
     PosIdx posCurrent;
     // Position for the current selector in this select chain.
     PosIdx posCurrentSyntax;
 
+    Value baseSelectee;
     try {
-        e->eval(state, env, vFirst);
+        // Evaluate the original thing we're selecting on.
+        e->eval(state, env, baseSelectee);
     } catch (Error & e) {
-        assert(this->e != nullptr);
-        e.addTrace(
-            state.ctx.positions[getPos()],
+        // clang-format off
+        e.addTrace(state.ctx.positions[getPos()], HintFmt(
             "while evaluating an expression to select '%s' on it",
-            showAttrPath(state.ctx.symbols, this->attrPath)
-        );
+            showAttrPath(state.ctx.symbols, attrPath)
+        ));
+        // clang-format on
         throw;
     }
 
     try {
-        for (auto const & [partIdx, currentAttrName] : enumerate(attrPath)) {
+        // With the original selectee evaluated, we'll walk the selection path starting
+        // with the evaluated original selectee.
+        std::reference_wrapper<Value> curSelectee = std::ref(baseSelectee);
+        for (AttrName const & attrName : attrPath) {
             state.ctx.stats.nrLookups++;
 
-            Symbol const name = getName(currentAttrName, state, env);
-
-            try {
-                state.forceValue(*vCurrent, pos);
-            } catch (Error & e) {
-                e.addTrace(
-                    state.ctx.positions[currentAttrName.pos],
-                    "while evaluating an expression to select '%s' on it",
-                    state.ctx.symbols[name]
-                );
-                throw;
-            }
-
-            if (vCurrent->type() != nAttrs) {
-
-                // If we have an `or` provided default,
-                // then this is allowed to not be an attrset.
-                if (def != nullptr) {
+            // Select `attrName` on `curSelectee`.
+            auto const attr = selectSingleAttr(state, env, attrName, curSelectee.get());
+            if (!attr) {
+                // Use default.
+                try {
                     this->def->eval(state, env, v);
-                    return;
+                } catch (Error & err) {
+                    err.addTrace(
+                        state.ctx.positions[this->def->pos],
+                        "while evaluating fallback for missing attribute '%s'",
+                        state.ctx.symbols[getName(attrName, state, env)]
+                    );
+                    throw;
                 }
-
-                // Otherwise, we must type error.
-                state.ctx.errors.make<TypeError>(
-                    "expected a set but found %s: %s",
-                    showType(*vCurrent),
-                    ValuePrinter(state, *vCurrent, errorPrintOptions)
-                ).addTrace(
-                    currentAttrName.pos,
-                    HintFmt("while selecting '%s'", state.ctx.symbols[name])
-                ).debugThrow();
+                return;
             }
 
-            // Now that we know this is actually an attrset, try to find an attr
-            // with the selected name.
-            auto attrIt = vCurrent->attrs()->get(name);
-            if (!attrIt) {
+            // The selection worked. If we have another iteration, then we use `attr->value`
+            // as the thing to select on. If this is the last iteration, then `attr->value`
+            // is the final value this ExprSelect evaluated to.
+            curSelectee = std::ref(attr->value);
 
-                // If we have an `or` provided default, then we'll use that.
-                if (def != nullptr) {
-                    this->def->eval(state, env, v);
-                    return;
-                }
-
-                // Otherwise, missing attr error.
-                std::set<std::string> allAttrNames;
-                for (auto const & attr : *vCurrent->attrs()) {
-                    allAttrNames.emplace(state.ctx.symbols[attr.name]);
-                }
-                auto suggestions = Suggestions::bestMatches(allAttrNames, state.ctx.symbols[name]);
-                state.ctx.errors.make<EvalError>("attribute '%s' missing", state.ctx.symbols[name])
-                    .atPos(currentAttrName.pos)
-                    .withSuggestions(suggestions)
-                    .withFrame(env, *this)
-                    .debugThrow();
+            posCurrent = attr->pos;
+            posCurrentSyntax = attrName.pos;
+            if (state.ctx.stats.countCalls) {
+                state.ctx.stats.attrSelects[posCurrent]++;
             }
-
-            // If we're here, then we successfully found the attribute.
-            // Set our currently operated-on attrset to this one, and keep going.
-            vCurrent = &attrIt->value;
-            posCurrent = attrIt->pos;
-            posCurrentSyntax = currentAttrName.pos;
-            if (state.ctx.stats.countCalls) state.ctx.stats.attrSelects[posCurrent]++;
         }
 
-        state.forceValue(*vCurrent, (posCurrent ? posCurrent : posCurrentSyntax));
+        state.forceValue(curSelectee.get(), posCurrent ? posCurrent : posCurrentSyntax);
+
+        v = curSelectee.get();
+
+    } catch (Error & err) {
+        auto const & lastPos = state.ctx.positions[posCurrent];
+        if (lastPos && !std::get_if<Pos::Hidden>(&lastPos.origin)) {
+            err.addTrace(
+                lastPos, "while evaluating the attribute '%s'", showAttrPath(state, env, attrPath)
+            );
+        }
 
-    } catch (Error & e) {
-        auto pos2r = state.ctx.positions[posCurrent];
-        if (pos2r && !std::get_if<Pos::Hidden>(&pos2r.origin))
-            e.addTrace(pos2r, "while evaluating the attribute '%1%'",
-                showAttrPath(state, env, attrPath));
         throw;
     }
-
-    v = *vCurrent;
 }
 
-
 void ExprOpHasAttr::eval(EvalState & state, Env & env, Value & v)
 {
     Value vTmp;
@@ -1584,7 +1639,14 @@ void EvalState::callFunction(Value & fun, std::span<Value> args, Value & vRes, c
 
     Value vCur(fun);
 
-    auto makeAppChain = [&]() { vRes = {NewValueAs::app, ctx.mem, vCur, args}; };
+    auto makeAppChain = [&]() {
+        if (vCur.isApp()) {
+            auto & app = vCur.app();
+            vRes = {NewValueAs::app, ctx.mem, app.left(), app.args(), args};
+        } else {
+            vRes = {NewValueAs::app, ctx.mem, vCur, args};
+        }
+    };
 
     const Attr * functor;
 
@@ -1996,17 +2058,16 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
         for (const auto & part : s) result += *part;
         return result;
     };
-    /* c_str() is not str().c_str() because we want to create a string
-       Value. allocating a GC'd string directly and moving it into a
-       Value lets us avoid an allocation and copy. */
-    const auto c_str = [&] {
-        char * result = gcAllocString(sSize + 1);
-        char * tmp = result;
+    /* build a gc'd value string directly instead of going through str()
+       and mkString to save an allocation and copy */
+    const auto gcStr = [&] {
+        auto result = Value::Str::gcAlloc(sSize);
+
+        char * tmp = result->contents;
         for (const auto & part : s) {
             memcpy(tmp, part->data(), part->size());
             tmp += part->size();
         }
-        *tmp = 0;
         return result;
     };
 
@@ -2080,7 +2141,7 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
             state.ctx.errors.make<EvalError>("a string that refers to a store path cannot be appended to a path").atPos(pos).withFrame(env, *this).debugThrow();
         v.mkPath(CanonPath(canonPath(str())));
     } else
-        v.mkStringMove(c_str(), context);
+        v.mkStringMove(gcStr(), context);
 }
 
 
@@ -2351,8 +2412,8 @@ BackedStringView EvalState::coerceToString(
 
     if (v.type() == nPath) {
         return !canonicalizePath && !copyToStore
-            ? v.string().content // FIXME: hack to preserve path literals that end in a slash, as in
-                                 // /foo/${x}.
+            // FIXME: hack to preserve path literals that end in a slash, as in /foo/${x}.
+            ? std::string(v.string().content->str())
             : (copyToStore
                    ? ctx.store->printStorePath(
                          aio.blockOn(ctx.paths.copyPathToStore(context, v.path(), ctx.repair))
@@ -2558,11 +2619,6 @@ bool EvalState::eqValues(Value & v1, Value & v2, const PosIdx pos, std::string_v
     forceValue(v1, pos);
     forceValue(v2, pos);
 
-    /* !!! Hack to support some old broken code that relies on pointer
-       equality tests between sets.  (Specifically, builderDefs calls
-       uniqList on a list of sets.)  Will remove this eventually. */
-    if (&v1 == &v2) return true;
-
     // Special case type-compatibility between float and int
     if (v1.type() == nInt && v2.type() == nFloat) {
         return v1.integer().value == v2.fpoint();
@@ -2574,6 +2630,11 @@ bool EvalState::eqValues(Value & v1, Value & v2, const PosIdx pos, std::string_v
     // All other types are not compatible with each other.
     if (v1.type() != v2.type()) return false;
 
+    /* !!! Hack to support some old broken code that relies on pointer
+       equality tests between sets.  (Specifically, builderDefs calls
+       uniqList on a list of sets.)  Will remove this eventually. */
+    auto pointerEq = [&] { return v1.pointerEqProxy() == v2.pointerEqProxy(); };
+
     switch (v1.type()) {
         case nInt:
             return v1.integer() == v2.integer();
@@ -2585,12 +2646,13 @@ bool EvalState::eqValues(Value & v1, Value & v2, const PosIdx pos, std::string_v
             return v1.str() == v2.str();
 
         case nPath:
-            return strcmp(v1.string().content, v2.string().content) == 0;
+            return v1.string().content->str() == v2.string().content->str();
 
         case nNull:
             return true;
 
         case nList:
+            if (pointerEq()) return true;
             if (v1.listSize() != v2.listSize()) return false;
             for (size_t n = 0; n < v1.listSize(); ++n) {
                 if (!eqValues(v1.listElems()[n], v2.listElems()[n], pos, errorCtx)) {
@@ -2600,6 +2662,7 @@ bool EvalState::eqValues(Value & v1, Value & v2, const PosIdx pos, std::string_v
             return true;
 
         case nAttrs: {
+            if (pointerEq()) return true;
             /* If both sets denote a derivation (type = "derivation"),
                then compare their outPaths. */
             if (isDerivation(v1) && isDerivation(v2)) {
@@ -2624,11 +2687,12 @@ bool EvalState::eqValues(Value & v1, Value & v2, const PosIdx pos, std::string_v
             return true;
         }
 
-        /* Functions are incomparable. */
+        /* Functions are incomparable, except for identity (see note above about this nonsense). */
         case nFunction:
-            return false;
+            return pointerEq();
 
         case nExternal:
+            if (pointerEq()) return true;
             return *v1.external() == *v2.external();
 
         case nFloat:

lix/libexpr/eval.hh

@@ -265,7 +265,7 @@ private:
 
     void addConstant(const std::string & name, const Value & v, Constant info);
 
-    void addPrimOp(PrimOpDetails && primOp);
+    void addPrimOp(PrimOpDetails primOp);
 
     Value prepareNixPath(const SearchPath & searchPath);
 

lix/libexpr/nixexpr.cc

@@ -10,7 +10,8 @@ namespace nix {
 
 ExprBlackHole eBlackHole;
 
-Value::Thunk Value::blackHole{{0}, &eBlackHole};
+static Env nullEnv;
+Value::Thunk Value::blackHole{{&nullEnv}, &eBlackHole};
 
 // FIXME: remove, because *symbols* are abstract and do not have a single
 //        textual representation; see printIdentifier()
@@ -321,74 +322,6 @@ JSON printAttrPathToJson(const SymbolTable & symbols, const AttrPath & attrPath)
 /* Computing levels/displacements for variables. */
 
 namespace {
-// This is a one-pass static analyzer for
-// various topics.
-struct StaticAnalyzer : ExprVisitor
-{
-    std::set<Symbol> staticallyUsedVariables;
-    bool usedDynamicVariables = false;
-
-    StaticAnalyzer() {}
-    using ExprVisitor::visit;
-
-    void visit(ExprDebugFrame & e, std::unique_ptr<Expr> & ptr) override
-    {
-        visit(e.inner);
-    }
-    void visit(ExprLiteral & e, std::unique_ptr<Expr> & ptr) override {}
-    void visit(ExprVar & e, std::unique_ptr<Expr> & ptr) override
-    {
-        staticallyUsedVariables.insert(e.name);
-    }
-    void visit(ExprInheritFrom & e, std::unique_ptr<Expr> & ptr) override {}
-    void visit(ExprSelect & e, std::unique_ptr<Expr> & ptr) override
-    {
-        if (e.isDynamic()) {
-            usedDynamicVariables = true;
-        }
-
-        visit(e.def);
-        visit(e.e);
-    }
-    void visit(ExprOpHasAttr & e, std::unique_ptr<Expr> & ptr) override
-    {
-        if (e.isDynamic()) {
-            usedDynamicVariables = true;
-        }
-
-        visit(e.e);
-    }
-    void visit(ExprSet & e, std::unique_ptr<Expr> & ptr) override
-    {
-        // TODO: oh bro, we need to analyze dynamic attributes for their value expressions.
-    }
-    void visit(ExprList & e, std::unique_ptr<Expr> & ptr) override {}
-    void visit(ExprLambda & e, std::unique_ptr<Expr> & ptr) override {}
-    void visit(ExprCall & e, std::unique_ptr<Expr> & ptr) override {}
-    void visit(ExprLet & e, std::unique_ptr<Expr> & ptr) override {}
-    void visit(ExprWith & e, std::unique_ptr<Expr> & ptr) override {}
-    void visit(ExprIf & e, std::unique_ptr<Expr> & ptr) override {}
-    void visit(ExprAssert & e, std::unique_ptr<Expr> & ptr) override {}
-    void visit(ExprOpNot & e, std::unique_ptr<Expr> & ptr) override {}
-#define BINOP(type)                                            \
-    /* NOLINTNEXTLINE(bugprone-macro-parentheses) */           \
-    void visit(type & e, std::unique_ptr<Expr> & ptr) override \
-    {                                                          \
-        visit(e.e1);                                           \
-        visit(e.e2);                                           \
-    }
-    BINOP(ExprOpEq)
-    BINOP(ExprOpNEq)
-    BINOP(ExprOpAnd)
-    BINOP(ExprOpOr)
-    BINOP(ExprOpImpl)
-    BINOP(ExprOpUpdate)
-    BINOP(ExprOpConcatLists)
-#undef BINOP
-    void visit(ExprConcatStrings & e, std::unique_ptr<Expr> & ptr) override {}
-    void visit(ExprPos & e, std::unique_ptr<Expr> & ptr) override {}
-    void visit(ExprBlackHole & e, std::unique_ptr<Expr> & ptr) override {}
-};
 struct VarBinder : ExprVisitor
 {
     Evaluator & es;
@@ -658,13 +591,6 @@ void VarBinder::visit(ExprLambda & e, std::unique_ptr<Expr> & ptr)
 {
     withEnv(e.pattern->buildEnv(env.get()), [&] {
         e.pattern->accept(*this);
-        /* TODO: If statically, e.body makes only use of some parameters and not the whole scope.
-         * We shouldn't have to keep around all the environment data which might contain trapped
-         * pointers. Analyze `e.body` and return its statically known set of used variables.
-         * */
-        DirectCallAnalyzer analyzer{es, env};
-        analyzer.visit(e.body);
-        e.shortcut = analyzer.shortcut;
         visit(e.body);
     });
 }

lix/libexpr/nixexpr.hh

@@ -204,21 +204,31 @@ struct ExprFloat : ExprLiteral
 
 struct ExprString : ExprLiteral
 {
-    std::string s;
-    Value::String strcb{.content = s.c_str(), .context = nullptr};
-    ExprString(const PosIdx pos, std::string && s) : ExprLiteral(pos), s(std::move(s))
+    std::unique_ptr<Value::Str, Value::Str::Deleter> contents;
+    Value::String strcb{.content = contents.get(), .context = nullptr};
+    ExprString(const PosIdx pos, std::string s) : ExprLiteral(pos), contents(Value::Str::copy(s))
     {
         v = {NewValueAs::string, &strcb};
     }
+
+    std::string_view str() const
+    {
+        return contents->str();
+    }
 };
 
 struct ExprPath : ExprLiteral
 {
-    std::string s;
-    Value::String strcb{.content = s.c_str(), .context = Value::String::path};
-    ExprPath(const PosIdx pos, std::string s) : ExprLiteral(pos), s(std::move(s))
+    std::unique_ptr<Value::Str, Value::Str::Deleter> contents;
+    Value::String strcb{.content = contents.get(), .context = Value::String::path};
+    ExprPath(const PosIdx pos, std::string s) : ExprLiteral(pos), contents(Value::Str::copy(s))
     {
-        v = {NewValueAs::path, &strcb};
+        v = Value{NewValueAs::path, &strcb};
+    }
+
+    std::string_view str() const
+    {
+        return contents->str();
     }
 };
 
@@ -279,6 +289,8 @@ struct ExprInheritFrom : Expr
     void accept(ExprVisitor & ev, std::unique_ptr<Expr> & ptr) override { ev.visit(*this, ptr); }
 };
 
+struct Attr;
+
 struct ExprSelect : Expr
 {
     /** The expression attributes are being selected on. e.g. `foo` in `foo.bar.baz`. */
@@ -292,38 +304,20 @@ struct ExprSelect : Expr
     /** The path of attributes being selected. e.g. `bar.baz` in `foo.bar.baz.` */
     AttrPath attrPath;
 
-    bool isDynamic() const
-    {
-        for (auto & name : attrPath) {
-            if (name.expr) {
-                return true;
-            }
-        }
-
-        return false;
-    }
-
     ExprSelect(const PosIdx & pos, std::unique_ptr<Expr> e, AttrPath attrPath, std::unique_ptr<Expr> def) : Expr(pos), e(std::move(e)), def(std::move(def)), attrPath(std::move(attrPath)) { };
     ExprSelect(const PosIdx & pos, std::unique_ptr<Expr> e, const PosIdx namePos, Symbol name) : Expr(pos), e(std::move(e)) { attrPath.push_back(AttrName(namePos, name)); };
     JSON toJSON(const SymbolTable & symbols) const override;
     void eval(EvalState & state, Env & env, Value & v) override;
     void accept(ExprVisitor & ev, std::unique_ptr<Expr> & ptr) override { ev.visit(*this, ptr); }
+
+private:
+    Attr const * selectSingleAttr(EvalState & state, Env & env, AttrName const & attrName, Value & vCurrent);
 };
 
 struct ExprOpHasAttr : Expr
 {
     std::unique_ptr<Expr> e;
     AttrPath attrPath;
-
-    bool isDynamic() const
-    {
-        for (auto & name : attrPath) {
-            if (name.expr) {
-                return true;
-            }
-        }
-        return false;
-    }
     ExprOpHasAttr(const PosIdx & pos, std::unique_ptr<Expr> e, AttrPath attrPath) : Expr(pos), e(std::move(e)), attrPath(std::move(attrPath)) { };
     JSON toJSON(const SymbolTable & symbols) const override;
     void eval(EvalState & state, Env & env, Value & v) override;
@@ -500,13 +494,6 @@ struct ExprLambda : Expr
     Symbol name;
     std::unique_ptr<Pattern> pattern;
     std::unique_ptr<Expr> body;
-    // This is a shortcut variant which
-    // exhausts the body further lambda constructions
-    // to transform x1: x2: …: xn: b
-    // into { x1, …, xn }: b
-    // This can be used when you know that you are
-    // passing all the arguments at once.
-    std::unique_ptr<ExprLambda> shortcut;
     ExprLambda(PosIdx pos, std::unique_ptr<Pattern> pattern, std::unique_ptr<Expr> body)
         : Expr(pos), pattern(std::move(pattern)), body(std::move(body))
     {

lix/libexpr/parser/parser-impl1.inc.cc

@@ -326,10 +326,11 @@ template<> struct BuildAST<grammar::v1::attr::simple> {
 template<> struct BuildAST<grammar::v1::attr::string> {
     static void apply(const auto & in, auto & s, State & ps) {
         auto e = s->popExprOnly();
-        if (auto str = dynamic_cast<ExprString *>(e.get()))
-            s.pushAttr(ps.symbols.create(str->s), ps.at(in));
-        else
+        if (auto estr = dynamic_cast<ExprString *>(e.get())) {
+            s.pushAttr(ps.symbols.create(estr->str()), ps.at(in));
+        } else {
             s.pushAttr(std::move(e), ps.at(in));
+        }
     }
 };
 
@@ -387,9 +388,9 @@ template<> struct BuildAST<grammar::v1::inherit> : change_head<InheritState> {
         for (auto & i : s.attrs) {
             if (i.symbol)
                 continue;
-            if (auto str = dynamic_cast<ExprString *>(i.expr.get()))
-                i = AttrName(i.pos, ps.symbols.create(str->s));
-            else {
+            if (auto estr = dynamic_cast<ExprString *>(i.expr.get())) {
+                i = AttrName(i.pos, ps.symbols.create(estr->str()));
+            } else {
                 throw ParseError({
                     .msg = HintFmt("dynamic attributes not allowed in inherit"),
                     .pos = ps.positions[i.pos]
@@ -770,13 +771,15 @@ template<> struct BuildAST<grammar::v1::path> : change_head<StringState> {
     template<typename E>
     static void check_slash(PosIdx end, StringState & s, State & ps) {
         auto e = dynamic_cast<E *>(s.parts.back().second.get());
-        if (!e || !e->s.ends_with('/'))
+        if (!e || !e->str().ends_with('/')) {
             return;
-        if (s.parts.size() > 1 || e->s != "/")
+        }
+        if (s.parts.size() > 1 || e->str() != "/") {
             throw ParseError({
                 .msg = HintFmt("path has a trailing slash"),
                 .pos = ps.positions[end],
             });
+        }
     }
 
     static void success(const auto & in, StringState & s, ExprState & e, State & ps) {

lix/libexpr/primops.cc

@@ -16,6 +16,7 @@
 #include "lix/libexpr/value-to-xml.hh"
 #include "lix/libexpr/primops.hh"
 #include "lix/libfetchers/fetch-to-store.hh"
+#include "lix/libutil/c-calls.hh"
 #include "lix/libutil/regex.hh"
 #include "lix/libutil/types.hh"
 #include "value.hh"
@@ -273,12 +274,12 @@ void prim_importNative(EvalState & state, Value * * args, Value & v)
 
     std::string sym(state.forceStringNoCtx(*args[1], noPos, "while evaluating the second argument passed to builtins.importNative"));
 
-    void *handle = dlopen(path.canonical().c_str(), RTLD_LAZY | RTLD_LOCAL);
+    void * handle = dlopen(requireCString(path.canonical().abs()), RTLD_LAZY | RTLD_LOCAL);
     if (!handle)
         state.ctx.errors.make<EvalError>("could not open '%1%': %2%", path, dlerror()).debugThrow();
 
     dlerror();
-    ValueInitializer func = reinterpret_cast<ValueInitializer>(dlsym(handle, sym.c_str()));
+    ValueInitializer func = reinterpret_cast<ValueInitializer>(dlsym(handle, requireCString(sym)));
     if(!func) {
         char *message = dlerror();
         if (message)
@@ -454,7 +455,12 @@ struct CompareValues : NeverAsync
         return (*this)(v1, v2, errorCtx);
     }
 
-    bool operator()(Value & v1, Value & v2, std::string_view errorCtx) const
+    bool operator()(Value const & v1, Value const & v2) const
+    {
+        return (*this)(v1, v2, errorCtx);
+    }
+
+    bool operator()(Value const & v1, Value const & v2, std::string_view errorCtx) const
     {
         try {
             if (v1.type() == nFloat && v2.type() == nInt) {
@@ -479,7 +485,7 @@ struct CompareValues : NeverAsync
             case nString:
                 return v1.str() < v2.str();
             case nPath:
-                return strcmp(v1.string().content, v2.string().content) < 0;
+                return v1.string().content->str() < v2.string().content->str();
             case nList:
                 // Lexicographic comparison
                 for (size_t i = 0;; i++) {
@@ -518,9 +524,9 @@ struct CompareValues : NeverAsync
 
 /// NOTE: this type must NEVER be outside of GC-scanned memory.
 #if HAVE_BOEHMGC
-using UnsafeValueList = std::list<Value *, gc_allocator<Value *>>;
+using UnsafeValueList = std::list<Value, gc_allocator<Value>>;
 #else
-using UnsafeValueList = std::list<Value *>;
+using UnsafeValueList = std::list<Value>;
 #endif
 
 static const Attr *
@@ -553,7 +559,7 @@ static void prim_genericClosure(EvalState & state, Value * * args, Value & v)
 
     UnsafeValueList workSet;
     for (auto & elem : startSet->value.listItems()) {
-        workSet.push_back(&elem);
+        workSet.push_back(elem);
     }
 
     if (startSet->value.listSize() == 0) {
@@ -581,36 +587,41 @@ static void prim_genericClosure(EvalState & state, Value * * args, Value & v)
     // `doneKeys' doesn't need to be a GC root, because its values are
     // reachable from res.
     auto cmp = CompareValues(state, "while comparing the `key` attributes of two genericClosure elements");
-    std::set<Value *, decltype(cmp)> doneKeys(cmp);
+    std::set<Value, decltype(cmp)> doneKeys(cmp);
     while (!workSet.empty()) {
-        Value * e = *(workSet.begin());
+        Value e = *(workSet.begin());
         workSet.pop_front();
 
-        state.forceAttrs(*e, noPos, "while evaluating one of the elements generated by (or initially passed to) builtins.genericClosure");
+        state.forceAttrs(
+            e,
+            noPos,
+            "while evaluating one of the elements generated by (or initially passed to) "
+            "builtins.genericClosure"
+        );
 
         auto key = getAttr(
             state,
             state.ctx.s.key,
-            e->attrs(),
+            e.attrs(),
             "in one of the attrsets generated by (or initially passed to) builtins.genericClosure"
         );
         state.forceValue(key->value, noPos);
 
-        if (!doneKeys.insert(&key->value).second) {
+        if (!doneKeys.insert(key->value).second) {
             continue;
         }
         res.push_back(e);
 
         /* Call the `operator' function with `e' as argument. */
         Value newElements;
-        state.callFunction(op->value, {e, 1}, newElements, noPos);
+        state.callFunction(op->value, {&e, 1}, newElements, noPos);
         state.forceList(newElements, noPos, "while evaluating the return value of the `operator` passed to builtins.genericClosure");
 
         /* Add the values returned by the operator to the work set. */
         for (auto & elem : newElements.listItems()) {
             state.forceValue(elem, noPos); // "while evaluating one one of the elements returned by
                                            // the `operator` passed to builtins.genericClosure");
-            workSet.push_back(&elem);
+            workSet.push_back(elem);
         }
     }
 
@@ -619,7 +630,7 @@ static void prim_genericClosure(EvalState & state, Value * * args, Value & v)
     v = {NewValueAs::list, result};
     unsigned int n = 0;
     for (auto & i : res)
-        result->elems[n++] = *i;
+        result->elems[n++] = i;
 }
 
 
@@ -842,10 +853,12 @@ static void prim_derivationStrict(EvalState & state, Value * * args, Value & v)
          * often results from the composition of several functions
          * (derivationStrict, derivation, mkDerivation, mkPythonModule, etc.)
          */
-        e.addTrace(nullptr, HintFmt(
-                "while evaluating derivation '%s'\n"
-                "  whose name attribute is located at %s",
-                drvName, pos));
+
+        e.pushTrace(Trace::fromDrv(
+            state.ctx.positions[nameAttr->pos],
+            drvName
+        ));
+
         throw;
     }
 }
@@ -1071,8 +1084,11 @@ drvName, Bindings * attrs, Value & v)
             }
 
         } catch (Error & e) {
-            e.addTrace(state.ctx.positions[i->pos],
-                HintFmt("while evaluating attribute '%1%' of derivation '%2%'", key, drvName));
+            e.pushTrace(Trace::fromDrvAttr(
+                state.ctx.positions[i->pos],
+                std::string(drvName),
+                std::string(key)
+            ));
             throw;
         }
     }
@@ -1344,11 +1360,6 @@ static void prim_readFile(EvalState & state, Value * * args, Value & v)
 {
     auto path = realisePath(state, *args[0]);
     auto s = path.readFile();
-    if (s.find((char) 0) != std::string::npos)
-        state.ctx.errors.make<EvalError>(
-            "the contents of the file '%1%' cannot be represented as a Nix string",
-            path
-        ).debugThrow();
     StorePathSet refs;
     if (state.ctx.store->isInStore(path.canonical().abs())) {
         try {

lix/libexpr/symbol-table.hh

@@ -45,7 +45,7 @@ private:
     /*
      * The type that actually stores the string contained inside of the Value.
      */
-    std::string contents;
+    std::unique_ptr<Value::Str, Value::Str::Deleter> contents;
 
     Value::String strcb;
 
@@ -56,8 +56,8 @@ private:
 
 public:
     explicit InternedSymbol(std::string_view s)
-        : contents(s)
-        , strcb{.content = contents.c_str(), .context = nullptr}
+        : contents(Value::Str::copy(s))
+        , strcb{.content = contents.get(), .context = nullptr}
         , underlyingValue(NewValueAs::string, &strcb)
     {
     }
@@ -69,17 +69,17 @@ public:
 
     operator SymbolStr() const
     {
-        return SymbolStr(contents);
+        return SymbolStr(contents->str());
     }
 
     bool operator==(std::string_view s2) const
     {
-        return contents == s2;
+        return contents->str() == s2;
     }
 
     operator std::string_view() const
     {
-        return contents;
+        return contents->str();
     }
 
     Value toValue() const

lix/libexpr/value.cc

@@ -14,6 +14,19 @@ Value Value::EMPTY_LIST{Value::list_t{}, &emptyListData};
 
 const Value::Null Value::NULL_ACB = {{Value::Acb::tNull}};
 
+static_assert(alignof(Value::String) >= Value::TAG_ALIGN);
+static_assert(alignof(Bindings) >= Value::TAG_ALIGN);
+static_assert(alignof(Value::List) >= Value::TAG_ALIGN);
+static_assert(alignof(Value::Thunk) >= Value::TAG_ALIGN);
+static_assert(alignof(Value::App) >= Value::TAG_ALIGN);
+
+static_assert(alignof(Value::External) >= Value::Acb::TAG_ALIGN);
+static_assert(alignof(Value::Float) >= Value::Acb::TAG_ALIGN);
+static_assert(alignof(Value::Null) >= Value::Acb::TAG_ALIGN);
+static_assert(alignof(Value::PrimOp) >= Value::Acb::TAG_ALIGN);
+static_assert(alignof(Value::Int) >= Value::Acb::TAG_ALIGN);
+static_assert(alignof(Value::Lambda) >= Value::Acb::TAG_ALIGN);
+
 static void copyContextToValue(Value::String & s, const NixStringContext & context)
 {
     if (!context.empty()) {
@@ -46,9 +59,11 @@ void Value::mkPrimOp(PrimOp * p)
     *this = {NewValueAs::primop, *p};
 }
 
-void Value::mkString(std::string_view s)
+void Value::mkString(std::string_view s, const char ** context)
 {
-    mkString(gcCopyStringIfNeeded(s));
+    auto block = gcAllocType<String>();
+    *block = {.content = Str::gcCopy(s), .context = context};
+    raw = tag(tString, block);
 }
 
 void Value::mkString(std::string_view s, const NixStringContext & context)
@@ -57,10 +72,12 @@ void Value::mkString(std::string_view s, const NixStringContext & context)
     copyContextToValue(*untag<String *>(), context);
 }
 
-void Value::mkStringMove(const char * s, const NixStringContext & context)
+void Value::mkStringMove(Str * s, const NixStringContext & context)
 {
-    mkString(s);
-    copyContextToValue(*untag<String *>(), context);
+    auto block = gcAllocType<String>();
+    *block = {.content = s, .context = nullptr};
+    raw = tag(tString, block);
+    copyContextToValue(*block, context);
 }
 
 void Value::mkPath(const SourcePath & path)
@@ -68,4 +85,147 @@ void Value::mkPath(const SourcePath & path)
     *this = Value(NewValueAs::path, path);
 }
 
+#ifndef __APPLE__
+[[gnu::section(".debug_gdb_scripts"), gnu::used, gnu::aligned(1)]]
+static const char printer_script[] =
+    "\4"
+    R"(lix-value-printer
+class ValuePrinter(gdb.ValuePrinter):
+    def __init__(self, val):
+          self._val = val
+          self._t_thunk = gdb.lookup_type('nix::Value::Thunk').pointer()
+          self._t_app = gdb.lookup_type('nix::Value::App').pointer()
+          self._t_int = gdb.lookup_type('intptr_t')
+          self._t_string = gdb.lookup_type('nix::Value::String').pointer()
+          self._t_attrs = gdb.lookup_type('nix::Bindings').pointer()
+          self._t_list = gdb.lookup_type('nix::Value::List').pointer()
+          self._t_aux = gdb.lookup_type('nix::Value::Acb').pointer()
+
+    def _addr(self, t = None):
+          addr = self._val['raw'] & ~self._val['TAG_MASK']
+          return addr.cast(t).referenced_value() if t else addr
+
+    def _tag(self):
+          return self._val['raw'] & self._val['TAG_MASK']
+    def _v_int(self):
+          return self._val['raw'].cast(self._t_int) >> self._val['TAG_BITS']
+
+    def to_string(self):
+          if self._tag() == 7:
+                return self._addr(self._t_aux)
+          elif not self.children():
+                return f'undecoded {self._val['raw'].format_string(format='x')}'
+          return None
+
+    def children(self):
+          match self._tag():
+                case 0:
+                      return [('thunk', self._addr(self._t_thunk))]
+                case 1:
+                      return [('app', self._addr(self._t_app))]
+                case 2:
+                      return [('int', self._v_int())]
+                case 3:
+                      return [('bool', (self._val['raw'] >> self._val['TAG_BITS']) != 0)]
+                case 4:
+                      return [('string', self._addr(self._t_string))]
+                case 5:
+                      return [('attrs', self._addr(self._t_attrs))]
+                case 6:
+                      return [('list', self._addr(self._t_list))]
+                case _:
+                      return []
+
+class ThunkPrinter(gdb.ValuePrinter):
+    def __init__(self, val):
+          self._val = val
+          self._t_value = gdb.lookup_type('nix::Value')
+          self._t_env = gdb.lookup_type('nix::Env').pointer()
+
+    def children(self):
+          if self._val['expr'] == 0:
+                return [('result', self._val['_result'].cast(self._t_value))]
+          else:
+                return [
+                      ('env', self._val['_env'].cast(self._t_env).referenced_value()),
+                      ('expr', self._val['expr'].referenced_value()),
+                ]
+
+class AppPrinter(gdb.ValuePrinter):
+    def __init__(self, val):
+          self._val = val
+          self._t_value = gdb.lookup_type('nix::Value')
+
+    def children(self):
+          if self._val['_n'] + 1 == 0:
+                return [('result', self._val['_left'].cast(self._t_value))]
+          else:
+                n = int(self._val['_n'])
+                arr = self._t_value.array(0, n - 1)
+                return [
+                      ('fn', self._val['_left'].cast(self._t_value)),
+                      ('n', n),
+                      ('args', self._val['_args'][0].cast(arr)),
+                ]
+
+class AcbPrinter(gdb.ValuePrinter):
+    def __init__(self, val):
+          self._val = val
+          self._t_external = gdb.lookup_type('nix::Value::External').pointer()
+          self._t_float = gdb.lookup_type('nix::Value::Float').pointer()
+          self._t_primop = gdb.lookup_type('nix::Value::PrimOp')
+          self._t_primopdetails = gdb.lookup_type('nix::PrimOpDetails')
+          self._t_lambda = gdb.lookup_type('nix::Value::Lambda').pointer()
+          self._t_int = gdb.lookup_type('nix::Value::Int').pointer()
+
+    def _addr(self, t = None):
+          addr = self._val['raw'] & ~self._val['TAG_MASK']
+          return addr.cast(t).referenced_value() if t else addr
+
+    def _tag(self):
+          return self._val['raw'] & self._val['TAG_MASK']
+
+    def to_string(self):
+          match self._tag():
+                case 2:
+                      return 'null'
+                case 3:
+                      op = self._val.cast(self._t_primop).cast(self._t_primopdetails)
+                      return f'primop {op['name']}'
+                case _:
+                      return None
+
+    def children(self):
+          match self._tag():
+                case 0:
+                      return [('external', self._addr(self._t_external))]
+                case 1:
+                      return [('float', self._addr(self._t_float)['value'])]
+                case 4:
+                      return [('lambda', self._addr(self._t_lambda))]
+                case 5:
+                      return [('int', self._addr(self._t_int)['value'])]
+                case _:
+                      return []
+
+def value_lookup_function(val):
+    lookup_tag = val.type.tag
+    if lookup_tag is None:
+        return None
+    if lookup_tag == 'nix::Value':
+        return ValuePrinter(val)
+    elif lookup_tag == 'nix::Value::Thunk':
+        return ThunkPrinter(val)
+    elif lookup_tag == 'nix::Value::App':
+        return AppPrinter(val)
+    elif lookup_tag == 'nix::Value::Acb':
+        return AcbPrinter(val)
+    return None
+
+def register_printers(objfile):
+      objfile.pretty_printers.append(value_lookup_function)
+
+register_printers(gdb.current_objfile())
+)";
+#endif
 }

lix/libexpr/value.hh

@@ -7,8 +7,10 @@
 #include <cstring>
 #include <functional>
 #include <limits>
+#include <memory>
 #include <ranges>
 #include <span>
+#include <string_view>
 #include <type_traits>
 
 #include "lix/libexpr/gc-alloc.hh"
@@ -72,6 +74,8 @@ struct PrimOpDetails
     std::optional<ExperimentalFeature> experimentalFeature;
 };
 
+// NOTE value.cc contains alignment assertions for pointers tagged thusly.
+// *always* ensure that these assertions match the tag types declared here
 typedef enum {
     // NOTE: tThunk *must* be 0, otherwise invalid value detection breaks
     // since invalid values are encoded as thunks with a null thunk state
@@ -247,10 +251,16 @@ struct Value
 private:
     mutable uintptr_t raw;
 
+public:
     static constexpr size_t TAG_BITS = 3;
     static constexpr size_t TAG_ALIGN = 1 << TAG_BITS;
     static constexpr uintptr_t TAG_MASK = (1 << TAG_BITS) - 1;
 
+private:
+    // boehmgc always allocate in two-word chunks, which means 8 bytes on 32 bit architectures.
+    // ensure that malloc must always use at least 8 byte chunks as well so our tags always fit
+    static_assert(alignof(std::max_align_t) >= Value::TAG_ALIGN);
+
     static uintptr_t tag(InternalType t, auto v)
     {
         if constexpr (std::is_null_pointer_v<decltype(v)>) {
@@ -266,7 +276,6 @@ private:
     T untag() const
     {
         if constexpr (std::is_pointer_v<T>) {
-            static_assert(alignof(T) >= TAG_BITS);
             return reinterpret_cast<T>(raw & ~TAG_MASK);
         } else {
             return static_cast<T>((raw & ~TAG_MASK) >> TAG_BITS);
@@ -282,6 +291,67 @@ private:
 
 public:
 
+    /**
+     * Underlying data storage for stringly values (i.e., strings and paths). Stores
+     * both the length of the string and its contents in a single GC-allocated block
+     * of memory to reduce overhead in the most common case. This and `String` could
+     * be merged into a single struct to decrease memory overhead further, but doing
+     * so precludes us from using atomic allocations that do not need to be scanned,
+     * increasing GC runtime overhead. We only use this struct to replace C strings.
+     */
+    struct Str
+    {
+        struct Deleter
+        {
+            void operator()(Str * s)
+            {
+                free(s);
+            }
+        };
+
+        size_t length;
+        char contents[0];
+
+        std::string_view str() const
+        {
+            return {contents, length};
+        }
+
+        static Str * gcAlloc(size_t size)
+        {
+            auto result = static_cast<Str *>(LIX_GC_MALLOC_ATOMIC(sizeof(Value::Str) + size));
+            if (result) {
+                result->length = size;
+                return result;
+            }
+            throw std::bad_alloc();
+        }
+
+        static std::unique_ptr<Str, Deleter> copy(std::string_view s)
+        {
+            auto result = alloc(s.size());
+            memcpy(result->contents, s.data(), s.size());
+            return {result, {}};
+        }
+
+        static Str * gcCopy(std::string_view s)
+        {
+            auto result = gcAlloc(s.size());
+            memcpy(result->contents, s.data(), s.size());
+            return result;
+        }
+
+    private:
+        static Str * alloc(size_t size)
+        {
+            if (auto result = static_cast<Str *>(malloc(sizeof(Value::Str) + size))) {
+                result->length = size;
+                return result;
+            }
+            throw std::bad_alloc();
+        }
+    };
+
     /**
      * Empty list constant.
      */
@@ -370,7 +440,7 @@ public:
     /// Neither the C-string nor the context array are copied; this constructor
     /// assumes suitable memory has already been allocated (with the GC if
     /// enabled), and string and context data copied into that memory.
-    Value(string_t, char const * strPtr, char const ** contextPtr = nullptr)
+    Value(string_t, const Str * strPtr, char const ** contextPtr = nullptr)
     {
         auto block = gcAllocType<String>();
         *block = {.content = strPtr, .context = contextPtr};
@@ -387,7 +457,7 @@ public:
     Value(string_t, std::string_view copyFrom, NixStringContext const & context = {})
     {
         auto block = gcAllocType<String>();
-        *block = {.content = gcCopyStringIfNeeded(copyFrom), .context = nullptr};
+        *block = {.content = Str::gcCopy(copyFrom), .context = nullptr};
         raw = tag(tString, block);
 
         if (context.empty()) {
@@ -408,39 +478,6 @@ public:
         block->context[n] = nullptr;
     }
 
-    /// Constructx a nix language value of type "string", with the value of the
-    /// C-string pointed to by @ref strPtr, and optionally with a set of string
-    /// context @ref context.
-    ///
-    /// The C-string is not copied; this constructor assumes suitable memory
-    /// has already been allocated (with the GC if enabled), and string data
-    /// has been copied into that memory. The context data *is* copied from
-    /// @ref context, and this constructor performs a dynamic (GC) allocation
-    /// to do so.
-    Value(string_t, char const * strPtr, NixStringContext const & context)
-    {
-        auto block = gcAllocType<String>();
-        *block = {.content = strPtr, .context = nullptr};
-        raw = tag(tString, block);
-
-        if (context.empty()) {
-            // It stays nullptr
-            return;
-        }
-
-        // Copy the context.
-        block->context = gcAllocType<char const *>(context.size() + 1);
-
-        size_t n = 0;
-        for (NixStringContextElem const & contextElem : context) {
-            block->context[n] = gcCopyStringIfNeeded(contextElem.to_string());
-            n += 1;
-        }
-
-        // Terminator sentinel.
-        block->context[n] = nullptr;
-    }
-
     /// Constructs a nix language value of type "path", with the value of the
     /// C-string pointed to by @ref strPtr.
     ///
@@ -460,7 +497,7 @@ public:
     Value(path_t, SourcePath const & path)
     {
         auto block = gcAllocType<String>();
-        *block = {.content = gcCopyStringIfNeeded(path.canonical().abs()), .context = String::path};
+        *block = {.content = Str::gcCopy(path.canonical().abs()), .context = String::path};
         raw = tag(tString, block);
     }
 
@@ -507,6 +544,10 @@ public:
     /// lazy and/or partial application of a function.
     Value(app_t, EvalMemory & mem, Value & lhs, std::span<Value> args);
 
+    /// Constructs a nix language value of type "lambda", which represents a
+    /// lazy and/or partial application of a function.
+    Value(app_t, EvalMemory & mem, const Value & lhs, std::span<Value> baseArgs, std::span<Value> moreArgs);
+
     /// Constructs a nix language value of type "external", which is only used
     /// by plugins. Do any existing plugins even use this mechanism?
     Value(external_t, ExternalValueBase & external)
@@ -569,10 +610,7 @@ public:
     {
         return internalType() == tApp;
     }
-    inline bool isBlackhole() const
-    {
-        return internalType() == tThunk && untag<const Thunk *>()->expr == blackHole.expr;
-    }
+    inline bool isBlackhole() const;
     inline bool isInvalid() const
     {
         return raw == 0;
@@ -616,7 +654,7 @@ public:
         /// marker location for paths, to be used as path context.
         static inline const char * path[] = {"\1<path>", nullptr};
 
-        const char * content;
+        const Str * content;
         const char ** context; // must be in sorted order
 
         bool isPath() const
@@ -631,6 +669,8 @@ public:
     /// these blocks are usually heap-allocated in GC memory space.
     struct alignas(TAG_ALIGN) Acb
     {
+        // NOTE value.cc contains alignment assertions for pointers tagged thusly.
+        // *always* ensure that these assertions match the tag types declared here
         enum Type {
             tExternal,
             tFloat,
@@ -661,7 +701,6 @@ public:
         T untag() const
         {
             if constexpr (std::is_pointer_v<T>) {
-                static_assert(alignof(T) >= TAG_BITS);
                 return reinterpret_cast<T>(raw & ~TAG_MASK);
             } else {
                 return static_cast<T>((raw & ~TAG_MASK) >> TAG_BITS);
@@ -685,7 +724,7 @@ public:
     {};
     struct PrimOp : Acb, PrimOpDetails
     {
-        explicit PrimOp(PrimOpDetails && p) : Acb{tPrimOp}, PrimOpDetails(std::move(p)) {}
+        explicit PrimOp(PrimOpDetails p) : Acb{tPrimOp}, PrimOpDetails(std::move(p)) {}
     };
     struct Int : Acb
     {
@@ -703,37 +742,6 @@ public:
             return untag<Env *>();
         }
     };
-    struct alignas(TAG_ALIGN) Thunk
-    {
-        union {
-            uintptr_t _env;
-            uintptr_t _result;
-        };
-        Expr * expr;
-
-        bool resolved() const
-        {
-            return expr == nullptr;
-        }
-
-        void resolve(Value v)
-        {
-            _result = v.raw;
-            expr = nullptr;
-        }
-
-        Env * env() const
-        {
-            return reinterpret_cast<Env *>(_env);
-        }
-
-        Value result() const
-        {
-            Value v;
-            v.raw = _result;
-            return v;
-        }
-    };
 
     /**
      * Returns the normal type of a Value. This only returns nThunk if
@@ -759,25 +767,18 @@ public:
         raw = tag(tBool, b);
     }
 
-    inline void mkString(const char * s, const char * * context = 0)
-    {
-        auto block = gcAllocType<String>();
-        *block = {.content = s, .context = context};
-        raw = tag(tString, block);
-    }
-
-    void mkString(std::string_view s);
+    void mkString(std::string_view s, const char ** context = 0);
 
     void mkString(std::string_view s, const NixStringContext & context);
 
-    void mkStringMove(const char * s, const NixStringContext & context);
+    void mkStringMove(Str * s, const NixStringContext & context);
 
     void mkPath(const SourcePath & path);
 
     inline void mkPath(const char * path)
     {
         auto block = gcAllocType<String>();
-        *block = {.content = path, .context = String::path};
+        *block = {.content = Str::gcCopy(path), .context = String::path};
         raw = tag(tString, block);
     }
 
@@ -838,13 +839,13 @@ public:
     SourcePath path() const
     {
         assert(internalType() == tString && untag<const String *>()->isPath());
-        return SourcePath{CanonPath(untag<const String *>()->content)};
+        return SourcePath{CanonPath(untag<const String *>()->content->str())};
     }
 
     std::string_view str() const
     {
         assert(internalType() == tString && !untag<const String *>()->isPath());
-        return std::string_view(untag<const String *>()->content);
+        return std::string_view(untag<const String *>()->content->str());
     }
 
     NixInt integer() const
@@ -911,6 +912,41 @@ public:
     {
         return untag<const Acb *>();
     }
+
+    uintptr_t pointerEqProxy() const
+    {
+        return raw;
+    }
+};
+
+struct alignas(Value::TAG_ALIGN) Value::Thunk
+{
+    union {
+        Env * _env;
+        Value _result;
+    };
+    Expr * expr;
+
+    bool resolved() const
+    {
+        return expr == nullptr;
+    }
+
+    void resolve(Value v)
+    {
+        _result = v;
+        expr = nullptr;
+    }
+
+    Env * env() const
+    {
+        return _env;
+    }
+
+    Value result() const
+    {
+        return _result;
+    }
 };
 
 struct alignas(Value::TAG_ALIGN) Value::List
@@ -1003,7 +1039,7 @@ again:
                 abort();
             }
         } else if (thunk().resolved()) {
-            raw = thunk()._env;
+            raw = thunk().result().raw;
             goto again;
         }
         return nThunk;
@@ -1016,6 +1052,11 @@ again:
     }
 }
 
+inline bool Value::isBlackhole() const
+{
+    return internalType() == tThunk && untag<const Thunk *>()->expr == blackHole.expr;
+}
+
 inline bool Value::isPrimOpApp() const
 {
     return internalType() == tApp && !app().resolved() && app().target().isPrimOp();

lix/libfetchers/fetch-to-store.cc

@@ -10,7 +10,7 @@ kj::Promise<Result<StorePath>> fetchToStoreFlat(
     std::string_view name,
     RepairFlag repair)
 try {
-    Activity act(*logger, lvlChatty, actUnknown, fmt("copying '%s' to the store", path));
+    auto act = logger->startActivity(lvlChatty, actUnknown, fmt("copying '%s' to the store", path));
     auto physicalPath = path.canonical().abs();
 
     co_return settings.readOnlyMode
@@ -26,8 +26,8 @@ kj::Promise<Result<StorePath>> fetchToStoreRecursive(
     std::string_view name,
     RepairFlag repair)
 try {
-    Activity act(
-        *logger, lvlChatty, actUnknown, fmt("copying '%s' to the store", contents.rootPath)
+    auto act = logger->startActivity(
+        lvlChatty, actUnknown, fmt("copying '%s' to the store", contents.rootPath)
     );
 
     co_return settings.readOnlyMode

lix/libfetchers/git.cc

@@ -1,6 +1,7 @@
 #include "lix/libutil/archive.hh"
 #include "lix/libutil/async-io.hh"
 #include "lix/libutil/async.hh"
+#include "lix/libutil/c-calls.hh"
 #include "lix/libutil/error.hh"
 #include "lix/libfetchers/fetchers.hh"
 #include "lix/libfetchers/cache.hh"
@@ -20,6 +21,7 @@
 
 #include "lix/libfetchers/fetch-settings.hh"
 
+#include <optional>
 #include <regex>
 #include <string.h>
 #include <sys/time.h>
@@ -51,7 +53,7 @@ bool touchCacheFile(const Path & path, time_t touch_time)
     times[1].tv_sec = touch_time;
     times[1].tv_usec = 0;
 
-    return lutimes(path.c_str(), times) == 0;
+    return sys::lutimes(path, times) == 0;
 }
 
 Path getCachePath(std::string_view key)
@@ -69,15 +71,13 @@ Path getCachePath(std::string_view key)
 //   ...
 static kj::Promise<Result<std::optional<std::string>>> readHead(const Path & path)
 try {
-    auto [status, output] = TRY_AWAIT(runProgram(RunOptions{
-        .program = "git",
+    auto output = TRY_AWAIT(runProgram(
+        "git",
+        true,
         // FIXME: use 'HEAD' to avoid returning all refs
-        .args = {"ls-remote", "--symref", path},
-        .isInteractive = true,
-    }));
-    if (status != 0) {
-        co_return std::nullopt;
-    }
+        {"ls-remote", "--symref", path},
+        true
+    ));
 
     std::string_view line = output;
     line = line.substr(0, line.find("\n"));
@@ -93,6 +93,8 @@ try {
         co_return parseResult->target;
     }
     co_return std::nullopt;
+} catch (ExecError &) {
+    co_return std::nullopt;
 } catch (...) {
     co_return result::current_exception();
 }
@@ -128,7 +130,7 @@ try {
     time_t now = time(0);
     struct stat st;
     std::optional<std::string> cachedRef;
-    if (stat(headRefFile.c_str(), &st) == 0) {
+    if (sys::stat(headRefFile, &st) == 0) {
         cachedRef = TRY_AWAIT(readHead(cacheDir));
         if (cachedRef != std::nullopt &&
             *cachedRef != gitInitialBranch &&
@@ -737,8 +739,7 @@ struct GitInputScheme : InputScheme
                         /* If the local ref is older than ‘tarball-ttl’ seconds, do a
                            git fetch to update the local ref to the remote ref. */
                         struct stat st;
-                        return stat(path.c_str(), &st) == 0 &&
-                            isCacheFileWithinTtl(now, st);
+                        return sys::stat(path, &st) == 0 && isCacheFileWithinTtl(now, st);
                     };
                     if (auto result = resolveRefToCachePath(
                         input,
@@ -758,7 +759,9 @@ struct GitInputScheme : InputScheme
             // Because git needs to figure out what we're fetching
             // (i.e. is it a rev? a branch? a tag?)
             if (doFetch) {
-                Activity act(*logger, lvlTalkative, actUnknown, fmt("fetching Git repository '%s'", actualUrl));
+                auto act = logger->startActivity(
+                    lvlTalkative, actUnknown, fmt("fetching Git repository '%s'", actualUrl)
+                );
 
                 auto ref = input.getRef();
                 std::string fetchRef;
@@ -893,7 +896,9 @@ struct GitInputScheme : InputScheme
                 // TODO: repoDir might lack the ref (it only checks if rev
                 // exists, see FIXME above) so use a big hammer and fetch
                 // everything to ensure we get the rev.
-                Activity act(*logger, lvlTalkative, actUnknown, fmt("making temporary clone of '%s'", repoDir));
+                auto act = logger->startActivity(
+                    lvlTalkative, actUnknown, fmt("making temporary clone of '%s'", repoDir)
+                );
                 TRY_AWAIT(runProgram(
                     "git",
                     true,
@@ -932,13 +937,17 @@ struct GitInputScheme : InputScheme
                source repo if it exists. */
             auto modulesPath = repoDir + "/" + gitDir + "/modules";
             if (pathExists(modulesPath)) {
-                Activity act(*logger, lvlTalkative, actUnknown, fmt("copying submodules of '%s'", actualUrl));
+                auto act = logger->startActivity(
+                    lvlTalkative, actUnknown, fmt("copying submodules of '%s'", actualUrl)
+                );
                 TRY_AWAIT(runProgram("cp", true, {"-R", "--", modulesPath, tmpGitDir + "/modules"})
                 );
             }
 
             {
-                Activity act(*logger, lvlTalkative, actUnknown, fmt("fetching submodules of '%s'", actualUrl));
+                auto act = logger->startActivity(
+                    lvlTalkative, actUnknown, fmt("fetching submodules of '%s'", actualUrl)
+                );
                 TRY_AWAIT(runProgram(
                     "git",
                     true,

lix/libfetchers/mercurial.cc

@@ -307,7 +307,9 @@ struct MercurialInputScheme : InputScheme
                       .second
                   == "1"))
         {
-            Activity act(*logger, lvlTalkative, actUnknown, fmt("fetching Mercurial repository '%s'", actualUrl));
+            auto act = logger->startActivity(
+                lvlTalkative, actUnknown, fmt("fetching Mercurial repository '%s'", actualUrl)
+            );
 
             if (pathExists(cacheDir)) {
                 try {

lix/libfetchers/path.cc

@@ -131,7 +131,7 @@ struct PathInputScheme : InputScheme
         } else
             absPath = path;
 
-        Activity act(*logger, lvlTalkative, actUnknown, fmt("copying '%s'", absPath));
+        auto act = logger->startActivity(lvlTalkative, actUnknown, fmt("copying '%s'", absPath));
 
         // FIXME: check whether access to 'path' is allowed.
         auto storePath = store->maybeParseStorePath(absPath);

lix/libmain/progress-bar.cc

@@ -104,31 +104,34 @@ bool ProgressBar::isVerbose()
     return printBuildLogs;
 }
 
-void ProgressBar::log(Verbosity lvl, std::string_view s)
+Logger::BufferState ProgressBar::log(Verbosity lvl, std::string_view s)
 {
-    if (lvl > verbosity) return;
+    if (lvl > verbosity) {
+        return BufferState::HasSpace;
+    }
     auto state(state_.lock());
-    log(*state, lvl, s);
+    return log(*state, lvl, s);
 }
 
-void ProgressBar::logEI(const ErrorInfo & ei)
+Logger::BufferState ProgressBar::logEI(const ErrorInfo & ei)
 {
     auto state(state_.lock());
 
     std::stringstream oss;
     showErrorInfo(oss, ei, loggerSettings.showTrace.get());
 
-    log(*state, ei.level, oss.str());
+    return log(*state, ei.level, oss.str());
 }
 
-void ProgressBar::log(State & state, Verbosity lvl, std::string_view s)
+Logger::BufferState ProgressBar::log(State & state, Verbosity lvl, std::string_view s)
 {
     if (state.paused == 0) eraseProgressDisplay(state);
     writeLogsToStderr(filterANSIEscapes(s + ANSI_NORMAL "\n", !isTTY));
     restoreProgressDisplay(state);
+    return BufferState::HasSpace;
 }
 
-void ProgressBar::startActivity(
+Logger::BufferState ProgressBar::startActivityImpl(
     ActivityId act,
     Verbosity lvl,
     ActivityType type,
@@ -140,7 +143,7 @@ void ProgressBar::startActivity(
     auto state(state_.lock());
 
     if (lvl <= verbosity && !s.empty() && type != actBuildWaiting)
-        log(*state, lvl, s + "...");
+        (void) log(*state, lvl, s + "...");
 
     state->activities.emplace_back(ActInfo {
         .s = s,
@@ -198,6 +201,7 @@ void ProgressBar::startActivity(
         i->visible = false;
 
     update(*state);
+    return BufferState::HasSpace;
 }
 
 /* Check whether an activity has an ancestore with the specified
@@ -213,7 +217,7 @@ bool ProgressBar::hasAncestor(State & state, ActivityType type, ActivityId act)
     return false;
 }
 
-void ProgressBar::stopActivity(ActivityId act)
+Logger::BufferState ProgressBar::stopActivityImpl(ActivityId act)
 {
     auto state(state_.lock());
 
@@ -233,9 +237,11 @@ void ProgressBar::stopActivity(ActivityId act)
     }
 
     update(*state);
+    return BufferState::HasSpace;
 }
 
-void ProgressBar::result(ActivityId act, ResultType type, const std::vector<Field> & fields)
+Logger::BufferState
+ProgressBar::resultImpl(ActivityId act, ResultType type, const std::vector<Field> & fields)
 {
     auto state(state_.lock());
 
@@ -256,7 +262,11 @@ void ProgressBar::result(ActivityId act, ResultType type, const std::vector<Fiel
                 if (type == resPostBuildLogLine) {
                     suffix = " (post)> ";
                 }
-                log(*state, lvlInfo, ANSI_FAINT + info.name.value_or("unnamed") + suffix + ANSI_NORMAL + lastLine);
+                (void) log(
+                    *state,
+                    lvlInfo,
+                    ANSI_FAINT + info.name.value_or("unnamed") + suffix + ANSI_NORMAL + lastLine
+                );
             } else {
                 if (!printMultiline) {
                     state->activities.erase(i->second);
@@ -310,6 +320,8 @@ void ProgressBar::result(ActivityId act, ResultType type, const std::vector<Fiel
         state->activitiesByType[type].expected += j;
         update(*state);
     }
+
+    return BufferState::HasSpace;
 }
 
 void ProgressBar::update(State & state)

lix/libmain/progress-bar.hh

@@ -75,13 +75,13 @@ struct ProgressBar : public Logger
 
     bool isVerbose() override;
 
-    void log(Verbosity lvl, std::string_view s) override;
+    BufferState log(Verbosity lvl, std::string_view s) override;
 
-    void logEI(const ErrorInfo & ei) override;
+    BufferState logEI(const ErrorInfo & ei) override;
 
-    void log(State & state, Verbosity lvl, std::string_view s);
+    BufferState log(State & state, Verbosity lvl, std::string_view s);
 
-    void startActivity(
+    BufferState startActivityImpl(
         ActivityId act,
         Verbosity lvl,
         ActivityType type,
@@ -92,9 +92,10 @@ struct ProgressBar : public Logger
 
     bool hasAncestor(State & state, ActivityType type, ActivityId act);
 
-    void stopActivity(ActivityId act) override;
+    BufferState stopActivityImpl(ActivityId act) override;
 
-    void result(ActivityId act, ResultType type, const std::vector<Field> & fields) override;
+    BufferState
+    resultImpl(ActivityId act, ResultType type, const std::vector<Field> & fields) override;
 
     void update(State & state);
 

lix/libmain/shared.cc

@@ -3,6 +3,7 @@
 #include "lix/libmain/shared.hh"
 #include "lix/libstore/store-api.hh"
 #include "lix/libstore/gc-store.hh"
+#include "lix/libutil/c-calls.hh"
 #include "lix/libutil/result.hh"
 #include "lix/libutil/signals.hh"
 #include "lix/libmain/loggers.hh"
@@ -307,13 +308,12 @@ void printVersion(const std::string & programName)
 void showManPage(const std::string & name)
 {
     restoreProcessContext();
-    setenv("MANPATH", settings.nixManDir.c_str(), 1);
-    execlp("man", "man", name.c_str(), nullptr);
+    (void) sys::setenv("MANPATH", settings.nixManDir, 1);
+    execlp("man", "man", requireCString(name).asCStr(), nullptr);
     throw SysError("command 'man %1%' failed", name.c_str());
 }
 
-
-int handleExceptions(const std::string & programName, std::function<void()> fun)
+int handleExceptions(const std::string & programName, std::function<int()> fun)
 {
     ReceiveInterrupts receiveInterrupts; // FIXME: need better place for this
 
@@ -323,7 +323,7 @@ int handleExceptions(const std::string & programName, std::function<void()> fun)
     bool onlyForSubcommands = baseNameOf(programName) == "lix";
 
     try {
-        fun();
+        return fun();
     } catch (Exit & e) {
         return e.status;
     } catch (UsageError & e) {

lix/libmain/shared.hh

@@ -14,7 +14,7 @@
 
 namespace nix {
 
-int handleExceptions(const std::string & programName, std::function<void()> fun);
+int handleExceptions(const std::string & programName, std::function<int()> fun);
 
 /**
  * Don't forget to call initPlugins() after settings are initialized!

lix/libstore/binary-cache-store.cc

@@ -225,6 +225,7 @@ try {
 
         auto * buildIdDir = std::get_if<nar_index::Directory>(&narIndex);
         for (auto subdir : { "lib", "debug", ".build-id" }) {
+            if (!buildIdDir) break;
             // get returns nullptr subdir does not exist, and std::get_if propagates it.
             buildIdDir = std::get_if<nar_index::Directory>(get(buildIdDir->contents, subdir));
         }
@@ -463,18 +464,17 @@ BinaryCacheStore::queryPathInfoUncached(const StorePath & storePath, const Activ
 try {
     auto uri = getUri();
     auto storePathS = printStorePath(storePath);
-    auto act = std::make_shared<Activity>(
-        *logger,
+    auto act = logger->startActivity(
         lvlTalkative,
         actQueryPathInfo,
         fmt("querying info about '%s' on '%s'", storePathS, uri),
         Logger::Fields{storePathS, uri},
-        context ? context->id : 0
+        context
     );
 
     auto narInfoFile = narInfoFileFor(storePath);
 
-    auto data = TRY_AWAIT(getFileContents(narInfoFile, act.get()));
+    auto data = TRY_AWAIT(getFileContents(narInfoFile, &act));
 
     if (!data) co_return result::success(nullptr);
 

lix/libstore/binary-cache-store.hh

@@ -19,8 +19,12 @@ struct BinaryCacheStoreConfig : virtual StoreConfig
 {
     using StoreConfig::StoreConfig;
 
-    const Setting<std::string> compression{this, "xz", "compression",
-        "NAR compression method (`xz`, `bzip2`, `gzip`, `zstd`, or `none`)."};
+    const Setting<std::string> compression{
+        this,
+        "zstd",
+        "compression",
+        "NAR compression method (`xz`, `bzip2`, `gzip`, `zstd`, or `none`)."
+    };
 
     const Setting<bool> writeNARListing{this, false, "write-nar-listing",
         "Whether to write a JSON file that lists the files in each NAR."};
@@ -40,12 +44,19 @@ struct BinaryCacheStoreConfig : virtual StoreConfig
     const Setting<bool> parallelCompression{this, false, "parallel-compression",
         "Enable multi-threaded compression of NARs. This is currently only available for `xz` and `zstd`."};
 
-    const Setting<int> compressionLevel{this, -1, "compression-level",
+    const Setting<int> compressionLevel{
+        this,
+        -1,
+        "compression-level",
         R"(
           The *preset level* to be used when compressing NARs.
           The meaning and accepted values depend on the compression method selected.
           `-1` specifies that the default compression level should be used.
-        )"};
+
+          Note: when using zstd `-1` will select level 12 to approximately match xz compression
+          ratios at default settings rather than the zstd library default of 3.
+        )"
+    };
 };
 
 

lix/libstore/build/child.cc

@@ -1,3 +1,4 @@
+#include "lix/libutil/c-calls.hh"
 #include "lix/libutil/current-process.hh"
 #include "lix/libutil/logging.hh"
 
@@ -22,12 +23,13 @@ void commonExecveingChildInit()
         throw SysError("cannot dup stderr into stdout");
 
     /* Reroute stdin to /dev/null. */
-    int fdDevNull = open(pathNullDevice.c_str(), O_RDWR);
-    if (fdDevNull == -1)
+    auto fdDevNull = sys::open(pathNullDevice, O_RDWR);
+    if (!fdDevNull) {
         throw SysError("cannot open '%1%'", pathNullDevice);
-    if (dup2(fdDevNull, STDIN_FILENO) == -1)
+    }
+    if (dup2(fdDevNull.get(), STDIN_FILENO) == -1) {
         throw SysError("cannot dup null device into stdin");
-    close(fdDevNull);
+    }
 }
 
 }

lix/libstore/build/derivation-goal.cc

@@ -1,6 +1,7 @@
 #include "lix/libstore/build/derivation-goal.hh"
 #include "lix/libutil/async-io.hh"
 #include "lix/libutil/async.hh"
+#include "lix/libutil/c-calls.hh"
 #include "lix/libutil/file-descriptor.hh"
 #include "lix/libutil/file-system.hh"
 #include "lix/libstore/build/hook-instance.hh"
@@ -22,12 +23,18 @@
 
 #include <boost/outcome/try.hpp>
 #include <capnp/rpc-twoparty.h>
+#include <cstdint>
+#include <exception>
 #include <fstream>
 #include <kj/array.h>
 #include <kj/async-unix.h>
 #include <kj/async.h>
 #include <kj/debug.h>
+#include <kj/exception.h>
+#include <kj/time.h>
 #include <kj/vector.h>
+#include <limits>
+#include <memory>
 #include <optional>
 #include <ranges>
 #include <sys/types.h>
@@ -120,7 +127,6 @@ DerivationGoal::~DerivationGoal() noexcept(false)
 void DerivationGoal::killChild()
 {
     hook.reset();
-    builderOutFD = nullptr;
 }
 
 
@@ -133,10 +139,13 @@ Goal::WorkResult DerivationGoal::timedOut(Error && ex)
 
 kj::Promise<Result<Goal::WorkResult>> DerivationGoal::workImpl() noexcept
 {
+    // always clear the slot token, no matter what happens. not doing this
+    // can cause builds to get stuck on exceptions (or other early exits).
+    // ideally we'd use scoped slot tokens instead of keeping them in some
+    // goal member variable, but we cannot do this yet for legacy reasons.
     KJ_DEFER({
-        act.reset();
         actLock.reset();
-        builderActivities.clear();
+        slotToken = {};
     });
 
     BOOST_OUTCOME_CO_TRY(auto result, co_await (useDerivation ? getDerivation() : haveDerivation()));
@@ -607,17 +616,14 @@ try {
     co_return result::current_exception();
 }
 
-void DerivationGoal::started()
+std::string DerivationGoal::buildDescription() const
 {
-    auto msg = fmt(
-        buildMode == bmRepair ? "repairing outputs of '%s'" :
-        buildMode == bmCheck ? "checking outputs of '%s'" :
-        "building '%s'", worker.store.printStorePath(drvPath));
-    fmt("building '%s'", worker.store.printStorePath(drvPath));
-    if (hook) msg += fmt(" on '%s'", machineName);
-    act = std::make_unique<Activity>(*logger, lvlInfo, actBuild, msg,
-        Logger::Fields{worker.store.printStorePath(drvPath), hook ? machineName : "", 1, 1});
-    mcRunningBuilds = worker.runningBuilds.addTemporarily(1);
+    return fmt(
+        buildMode == bmRepair      ? "repairing outputs of '%s'"
+            : buildMode == bmCheck ? "checking outputs of '%s'"
+                                   : "building '%s'",
+        worker.store.printStorePath(drvPath)
+    );
 }
 
 kj::Promise<Result<Goal::WorkResult>> DerivationGoal::tryToBuild() noexcept
@@ -648,8 +654,11 @@ retry:
     outputLocks = tryLockPaths(lockFiles);
     if (!outputLocks) {
         if (!actLock)
-            actLock = std::make_unique<Activity>(*logger, lvlWarn, actBuildWaiting,
-                fmt("waiting for lock on %s", Magenta(showPaths(lockFiles))));
+            actLock = logger->startActivity(
+                lvlWarn,
+                actBuildWaiting,
+                fmt("waiting for lock on %s", Magenta(showPaths(lockFiles)))
+            );
         co_await waitForAWhile();
         // we can loop very often, and `co_return co_await` always allocates a new frame
         goto retry;
@@ -705,8 +714,12 @@ retry:
             /* Not now; wait until at least one child finishes or
                 the wake-up timeout expires. */
             if (!actLock)
-                actLock = std::make_unique<Activity>(*logger, lvlTalkative, actBuildWaiting,
-                    fmt("waiting for a machine to build '%s'", Magenta(worker.store.printStorePath(drvPath))));
+                actLock = logger->startActivity(
+                    lvlTalkative,
+                    actBuildWaiting,
+                    fmt("waiting for a machine to build '%s'",
+                        Magenta(worker.store.printStorePath(drvPath)))
+                );
             outputLocks.reset();
             co_await waitForAWhile();
             goto retry;
@@ -763,11 +776,7 @@ void replaceValidPath(const Path & storePath, const Path & tmpPath)
        we're repairing (say) Glibc, we end up with a broken system. */
     Path oldPath;
     if (pathExists(storePath)) {
-        do {
-            oldPath = makeTempPath(storePath, ".old");
-            // store paths are often directories so we can't just unlink() it
-            // let's make sure the path doesn't exist before we try to use it
-        } while (pathExists(oldPath));
+        oldPath = makeTempSiblingPath(storePath);
         movePath(storePath, oldPath);
     }
 
@@ -791,16 +800,10 @@ void replaceValidPath(const Path & storePath, const Path & tmpPath)
 
 int DerivationGoal::getChildStatus()
 {
-    builderOutFD = nullptr;
-    return hook->pid.kill();
+    return hook->kill();
 }
 
-
-void DerivationGoal::closeReadPipes()
-{
-    hook->fromHook.reset();
-    builderOutFD = nullptr;
-}
+void DerivationGoal::closeReadPipes() {}
 
 void DerivationGoal::cleanupHookFinally()
 {
@@ -841,8 +844,7 @@ try {
         co_return result::success();
     }
 
-    Activity act(
-        logger,
+    auto act = logger.startActivity(
         lvlTalkative,
         actPostBuildHook,
         fmt("running post-build-hook '%s'", settings.postBuildHook),
@@ -881,7 +883,7 @@ try {
         const std::string_view data{buffer.data(), *got};
         for (auto c : data) {
             if (c == '\n') {
-                act.result(resPostBuildLogLine, currentLine);
+                ACTIVITY_RESULT(act, resPostBuildLogLine, currentLine);
                 currentLine.clear();
             } else {
                 currentLine += c;
@@ -891,7 +893,7 @@ try {
 
     if (currentLine != "") {
         currentLine += '\n';
-        act.result(resPostBuildLogLine, currentLine);
+        ACTIVITY_RESULT(act, resPostBuildLogLine, currentLine);
     }
 
     wait.run();
@@ -1035,15 +1037,20 @@ try {
 }
 
 namespace {
-struct BuildHookLogger final : rpc::build_remote::HookInstance::BuildLogger::Server
+struct ActivityTrackingHookLogger final : HookInstance::HookLogger
 {
-    AutoCloseFD fd;
+    kj::TimePoint & tracker;
 
-    BuildHookLogger(AutoCloseFD fd) : fd(std::move(fd)) {}
-
-    kj::Maybe<int> getFd() override
+    ActivityTrackingHookLogger(const Activity & act, FinishSink * logSink, kj::TimePoint & tracker)
+        : HookLogger(act, logSink)
+        , tracker(tracker)
     {
-        return fd.get();
+    }
+
+    kj::Promise<void> push(PushContext context) override
+    {
+        tracker = AIO().provider.getTimer().now();
+        return HookLogger::push(context);
     }
 };
 }
@@ -1061,24 +1068,16 @@ try {
         hook = std::move(worker.hook.instances.front());
         worker.hook.instances.pop_front();
     } else {
-        hook = TRY_AWAIT(HookInstance::create());
+        hook = TRY_AWAIT(HookInstance::create(worker.act));
     }
 
-    // open a pipe to receive logs directly from the hook
-    Pipe logPipe;
-    logPipe.create();
-    builderOutFD = &logPipe.readSide;
-    KJ_DEFER(builderOutFD = nullptr);
-
     KJ_DEFER(hook = nullptr);
-    auto output = handleChildOutput();
 
-    auto buildReq = hook->rpc.buildRequest();
+    auto buildReq = hook->rpc->buildRequest();
     RPC_FILL(buildReq, setAmWilling, slotToken.valid());
     RPC_FILL(buildReq, setNeededSystem, drv->platform);
     RPC_FILL(buildReq, initDrvPath, drvPath, worker.store);
     RPC_FILL(buildReq, initRequiredFeatures, parsedDrv->getRequiredSystemFeatures());
-    buildReq.setBuildLogger(kj::heap<BuildHookLogger>(std::move(logPipe.writeSide)));
     auto buildRespPromise = buildReq.send();
     auto buildResp = TRY_AWAIT_RPC(buildRespPromise);
 
@@ -1100,11 +1099,15 @@ try {
     // the build was accepted by the hook, we can free the slot for another build now
     hookSlot = {};
 
-    machineName = rpc::to<std::string>(buildResp.getAccept().getMachineName());
+    /* Create the log file and pipe. */
+    openLogFile();
 
     auto runReq = buildResp.getAccept().getMachine().runRequest();
     /* Tell the hook all the inputs that have to be copied to the
        remote system. */
+    runReq.setLogger(
+        kj::heap<ActivityTrackingHookLogger>(worker.act, logSink.get(), lastChildActivity)
+    );
     RPC_FILL(runReq, initInputs, inputPaths, worker.store);
 
     /* Tell the hooks the missing outputs that have to be copied back
@@ -1117,35 +1120,35 @@ try {
             missingOutputs.insert(outputName);
         }
         RPC_FILL(runReq, initWantedOutputs, missingOutputs);
+        RPC_FILL(runReq, setDescription, buildDescription());
     }
 
-    /* Create the log file and pipe. */
-    openLogFile();
-
     auto runPromise = runReq.send();
 
     // build via hook is now properly running. wait for it to finish
     actLock.reset();
     buildResult.startTime = time(0); // inexact
-    started();
+    mcRunningBuilds = worker.runningBuilds.addTemporarily(1);
 
-    auto result = co_await runPromise;
+    auto result = TRY_AWAIT(
+        wrapChildHandler(runPromise.then([&](auto result) -> kj::Promise<Result<WorkResult>> {
+            try {
+                std::shared_ptr<Error> remoteError;
+                if (result.getResult().isBad()) {
+                    remoteError = std::make_shared<Error>(from(result.getResult().getBad()));
+                    logErrorInfo(remoteError->info().level, remoteError->info());
+                }
+                // close the rpc connection to have the hook exit
+                hook->rpc = nullptr;
+                hook->wait();
+                return buildDone(remoteError);
+            } catch (...) {
+                return {result::current_exception()};
+            }
+        }))
+    );
 
-    // close the rpc connection to have the hook exit
-    hook->rpc = nullptr;
-    hook->client = std::nullopt;
-    hook->conn = nullptr;
-
-    if (auto error = TRY_AWAIT(output)) {
-        co_return HookResult::Accept{std::move(*error)};
-    }
-
-    std::shared_ptr<Error> remoteError;
-    if (result.getResult().isBad()) {
-        remoteError = std::make_shared<Error>(from(result.getResult().getBad()));
-        logErrorInfo(remoteError->info().level, remoteError->info());
-    }
-    co_return HookResult::Accept{TRY_AWAIT(buildDone(remoteError))};
+    co_return HookResult::Accept{std::move(result)};
 } catch (...) {
     co_return result::current_exception();
 }
@@ -1162,10 +1165,39 @@ kj::Promise<Result<SingleDrvOutputs>> DerivationGoal::registerOutputs()
     return assertPathValidity();
 }
 
+DerivationGoal::LogSink::LogSink(AutoCloseFD fd, ref<BufferedSink> file, bool compress, uint64_t limit)
+    : fd(std::move(fd))
+    , file(file)
+    , target(compress ? makeCompressionSink("bzip2", *file) : file)
+    , limit(limit)
+{
+}
+
+DerivationGoal::LogSink::~LogSink() noexcept
+{
+    signal.fulfiller->fulfill(false);
+}
+
+void DerivationGoal::LogSink::operator()(std::string_view data)
+{
+    writtenSoFar += data.size();
+    if (writtenSoFar <= limit) {
+        (*target)(data);
+    } else {
+        signal.fulfiller->fulfill(true);
+    }
+}
+
+void DerivationGoal::LogSink::finish()
+{
+    if (auto inner2 = dynamic_cast<FinishSink *>(&*target)) {
+        inner2->finish();
+    }
+    file->flush();
+}
+
 Path DerivationGoal::openLogFile()
 {
-    logSize = 0;
-
     if (!settings.keepLog) return "";
 
     auto baseName = std::string(baseNameOf(worker.store.printStorePath(drvPath)));
@@ -1182,27 +1214,26 @@ Path DerivationGoal::openLogFile()
     Path logFileName = fmt("%s/%s%s", dir, baseName.substr(2),
         settings.compressLog ? ".bz2" : "");
 
-    fdLogFile = AutoCloseFD{open(logFileName.c_str(), O_CREAT | O_WRONLY | O_TRUNC | O_CLOEXEC, 0666)};
+    auto fdLogFile = sys::open(logFileName, O_CREAT | O_WRONLY | O_TRUNC | O_CLOEXEC, 0666);
     if (!fdLogFile) throw SysError("creating log file '%1%'", logFileName);
 
-    logFileSink = std::make_shared<FdSink>(fdLogFile.get());
+    auto logFileSink = make_ref<FdSink>(fdLogFile.get());
+    const auto logLimit =
+        settings.maxLogSize ? settings.maxLogSize.get() : std::numeric_limits<uint64_t>::max();
 
-    if (settings.compressLog)
-        logSink = std::shared_ptr<CompressionSink>(makeCompressionSink("bzip2", *logFileSink));
-    else
-        logSink = logFileSink;
+    logSink = std::make_unique<LogSink>(
+        std::move(fdLogFile), logFileSink, settings.compressLog, logLimit
+    );
 
     return logFileName;
 }
 
-
 void DerivationGoal::closeLogFile()
 {
-    auto logSink2 = std::dynamic_pointer_cast<CompressionSink>(logSink);
-    if (logSink2) logSink2->finish();
-    if (logFileSink) logFileSink->flush();
-    logSink = logFileSink = 0;
-    fdLogFile.reset();
+    if (logSink) {
+        logSink->finish();
+    }
+    logSink = 0;
 }
 
 
@@ -1215,130 +1246,19 @@ Goal::WorkResult DerivationGoal::tooMuchLogs()
             getName(), settings.maxLogSize));
 }
 
-kj::Promise<Result<std::optional<Goal::WorkResult>>>
-DerivationGoal::handleBuilderOutput(AsyncInputStream & in) noexcept
-try {
-    auto buf = kj::heapArray<char>(4096);
-    while (true) {
-        std::string_view data;
-        try {
-            if (const auto got = TRY_AWAIT(in.read(buf.begin(), buf.size()))) {
-                data = {buf.begin(), *got};
-            } else {
-                co_return std::nullopt;
-            }
-        } catch (SysError & e) {
-            // the builder output stream may be a pty fd, and closing one pty
-            // endpoint sends EIO to the other endpoint. this is a good exit.
-            if (e.errNo == EIO) {
-                data = {};
-            } else {
-                throw;
-            }
-        }
-        lastChildActivity = AIO().provider.getTimer().now();
-
-        if (data.empty()) {
-            co_return std::nullopt;
-        }
-
-        logSize += data.size();
-        if (settings.maxLogSize && logSize > settings.maxLogSize) {
-            co_return tooMuchLogs();
-        }
-
-        for (auto c : data)
-            if (c == '\r')
-                currentLogLinePos = 0;
-            else if (c == '\n')
-                flushLine();
-            else {
-                if (currentLogLinePos >= currentLogLine.size())
-                    currentLogLine.resize(currentLogLinePos + 1);
-                currentLogLine[currentLogLinePos++] = c;
-            }
-
-        if (logSink) (*logSink)(data);
+kj::Promise<Result<Goal::WorkResult>>
+DerivationGoal::wrapChildHandler(kj::Promise<Result<WorkResult>> handler) noexcept
+{
+    if (respectsTimeouts() && settings.maxSilentTime != 0) {
+        handler = handler.exclusiveJoin(monitorForSilence());
     }
-} catch (...) {
-    co_return result::current_exception();
-}
-
-kj::Promise<Result<std::optional<Goal::WorkResult>>>
-DerivationGoal::handleHookOutput(AsyncInputStream & in) noexcept
-try {
-    auto buf = kj::heapArray<char>(4096);
-    while (true) {
-        const auto got = TRY_AWAIT(in.read(buf.begin(), buf.size()));
-        if (!got) {
-            co_return std::nullopt;
-        }
-
-        std::string_view data = {buf.begin(), *got};
-        lastChildActivity = AIO().provider.getTimer().now();
-
-        for (auto c : data)
-            if (c == '\n') {
-                auto json = parseJSONMessage(currentHookLine, "the derivation builder");
-                if (json) {
-                    auto s = handleJSONLogMessage(*json, worker.act, hook->activities, "the derivation builder", true);
-                    // ensure that logs from a builder using `ssh-ng://` as protocol
-                    // are also available to `nix log`.
-                    if (s && logSink) {
-                        const auto type = (*json)["type"];
-                        const auto fields = (*json)["fields"];
-                        if (type == resBuildLogLine) {
-                            const std::string logLine =
-                                (fields.size() > 0 ? fields[0].get<std::string>() : "") + "\n";
-                            logSize += logLine.size();
-                            if (settings.maxLogSize && logSize > settings.maxLogSize) {
-                                co_return tooMuchLogs();
-                            }
-                            (*logSink)(logLine);
-                        } else if (type == resSetPhase && ! fields.is_null()) {
-                            const auto phase = fields[0];
-                            if (! phase.is_null()) {
-                                // nixpkgs' stdenv produces lines in the log to signal
-                                // phase changes.
-                                // We want to get the same lines in case of remote builds.
-                                // The format is:
-                                //   @nix { "action": "setPhase", "phase": "$curPhase" }
-                                const auto logLine = JSON::object({
-                                    {"action", "setPhase"},
-                                    {"phase", phase}
-                                });
-                                (*logSink)("@nix " + logLine.dump(-1, ' ', false, JSON::error_handler_t::replace) + "\n");
-                            }
-                        }
-                    }
-                }
-                currentHookLine.clear();
-            } else
-                currentHookLine += c;
-    }
-} catch (...) {
-    co_return result::current_exception();
-}
-
-kj::Promise<Result<std::optional<Goal::WorkResult>>> DerivationGoal::handleChildOutput() noexcept
-try {
-    kj::Own<AsyncInputStream> builderIn, hookIn;
-    if (builderOutFD) {
-        builderIn = kj::heap<AsyncFdIoStream>(AsyncFdIoStream::shared_fd{}, builderOutFD->get());
-    }
-    if (hook) {
-        hookIn = kj::heap<AsyncFdIoStream>(AsyncFdIoStream::shared_fd{}, hook->fromHook.get());
-    }
-
-    auto handlers = handleChildStreams(builderIn.get(), hookIn.get())
-                        .attach(std::move(builderIn), std::move(hookIn));
 
     if (respectsTimeouts() && settings.buildTimeout != 0) {
-        handlers = handlers.exclusiveJoin(
+        handler = handler.exclusiveJoin(
             AIO()
                 .provider.getTimer()
                 .afterDelay(settings.buildTimeout.get() * kj::SECONDS)
-                .then([this]() -> Result<std::optional<WorkResult>> {
+                .then([this]() -> Result<WorkResult> {
                     return timedOut(
                         Error("%1% timed out after %2% seconds", name, settings.buildTimeout)
                     );
@@ -1346,16 +1266,29 @@ try {
         );
     }
 
-    return handlers.then([this](auto r) {
-        if (!currentLogLine.empty()) flushLine();
-        return r;
-    });
-} catch (...) {
-    return {result::current_exception()};
+    if (logSink) {
+        handler = handler.exclusiveJoin(
+            logSink->signal.promise.then([&](bool limitReached) -> kj::Promise<Result<WorkResult>> {
+                try {
+                    if (limitReached) {
+                        return {tooMuchLogs()};
+                    } else {
+                        return kj::NEVER_DONE;
+                    }
+                } catch (...) {
+                    return {result::current_exception()};
+                }
+            })
+        );
+    }
+
+    return handler;
 }
 
-kj::Promise<Result<std::optional<Goal::WorkResult>>> DerivationGoal::monitorForSilence() noexcept
+kj::Promise<Result<Goal::WorkResult>> DerivationGoal::monitorForSilence() noexcept
 {
+    lastChildActivity = AIO().provider.getTimer().now();
+
     while (true) {
         const auto stash = lastChildActivity;
         auto waitUntil = lastChildActivity + settings.maxSilentTime.get() * kj::SECONDS;
@@ -1368,57 +1301,6 @@ kj::Promise<Result<std::optional<Goal::WorkResult>>> DerivationGoal::monitorForS
     }
 }
 
-kj::Promise<Result<std::optional<Goal::WorkResult>>>
-DerivationGoal::handleChildStreams(AsyncInputStream * builderIn, AsyncInputStream * hookIn) noexcept
-{
-    assert(builderIn || hookIn);
-
-    lastChildActivity = AIO().provider.getTimer().now();
-
-    auto handlers = kj::joinPromisesFailFast([&] {
-        kj::Vector<kj::Promise<Result<std::optional<WorkResult>>>> parts{2};
-
-        if (builderIn) {
-            parts.add(handleBuilderOutput(*builderIn));
-        }
-        if (hookIn) {
-            parts.add(handleHookOutput(*hookIn));
-        }
-
-        return parts.releaseAsArray();
-    }());
-
-    if (respectsTimeouts() && settings.maxSilentTime != 0) {
-        handlers = handlers.exclusiveJoin(monitorForSilence().then([](auto r) {
-            return kj::arr(std::move(r));
-        }));
-    }
-
-    for (auto r : co_await handlers) {
-        if (r) {
-            co_return r;
-        }
-    }
-    co_return std::nullopt;
-}
-
-void DerivationGoal::flushLine()
-{
-    if (handleJSONLogMessage(currentLogLine, *act, builderActivities, "the derivation builder", false))
-        ;
-
-    else {
-        logTail.push_back(currentLogLine);
-        if (logTail.size() > settings.logLines) logTail.pop_front();
-
-        act->result(resBuildLogLine, currentLogLine);
-    }
-
-    currentLogLine = "";
-    currentLogLinePos = 0;
-}
-
-
 kj::Promise<Result<OutputPathMap>> DerivationGoal::queryDerivationOutputMap()
 try {
     OutputPathMap res;

lix/libstore/build/derivation-goal.hh

@@ -74,6 +74,25 @@ struct InitialOutput {
  */
 struct DerivationGoal : public Goal
 {
+    /**
+     * A sink that only passes a limited amount of data and signals exhaustion through a promise.
+     */
+    struct LogSink : FinishSink
+    {
+        AutoCloseFD fd;
+        ref<BufferedSink> file, target;
+        const uint64_t limit;
+        uint64_t writtenSoFar = 0;
+
+        kj::PromiseFulfillerPair<bool> signal = kj::newPromiseAndFulfiller<bool>();
+
+        LogSink(AutoCloseFD fd, ref<BufferedSink> file, bool compress, uint64_t limit);
+        ~LogSink() noexcept;
+
+        void operator()(std::string_view data) override;
+        void finish() override;
+    };
+
     /**
       * Whether this goal has completed. Completed goals can not be
       * asked for more outputs, a new goal must be created instead.
@@ -186,38 +205,20 @@ struct DerivationGoal : public Goal
     BuildResult buildResult;
 
     /**
-     * File descriptor for the log file.
+     * Sink for the log file.
      */
-    AutoCloseFD fdLogFile;
-    std::shared_ptr<BufferedSink> logFileSink, logSink;
-
-    /**
-     * Number of bytes received from the builder's stdout/stderr.
-     */
-    unsigned long logSize;
+    std::shared_ptr<LogSink> logSink;
 
     /**
      * The most recent log lines.
      */
     std::list<std::string> logTail;
 
-    std::string currentLogLine;
-    size_t currentLogLinePos = 0; // to handle carriage return
-
-    std::string currentHookLine;
-
     /**
      * The build hook.
      */
     std::unique_ptr<HookInstance> hook;
 
-    /**
-      * Builder output is pulled from this file descriptor when not null.
-      * Owned by the derivation goal or subclass, must not be reset until
-      * the build has finished and no more output must be processed by us
-      */
-    AutoCloseFD * builderOutFD = nullptr;
-
     /**
      * The sort of derivation we are building.
      */
@@ -227,19 +228,10 @@ struct DerivationGoal : public Goal
 
     NotifyingCounter<uint64_t>::Bump mcExpectedBuilds, mcRunningBuilds;
 
-    std::unique_ptr<Activity> act;
-
     /**
      * Activity that denotes waiting for a lock.
      */
-    std::unique_ptr<Activity> actLock;
-
-    std::map<ActivityId, Activity> builderActivities;
-
-    /**
-     * The remote machine on which we're building.
-     */
-    std::string machineName;
+    std::optional<Activity> actLock;
 
     /** Witness type to say that the drvPath has already been added as a temp root */
     struct DrvHasRoot { explicit DrvHasRoot() = default; };
@@ -254,7 +246,7 @@ struct DerivationGoal : public Goal
 
     WorkResult timedOut(Error && ex);
 
-    kj::Promise<Result<WorkResult>> workImpl() noexcept override;
+    kj::Promise<Result<WorkResult>> workImpl() noexcept override final;
 
     /**
      * Add wanted outputs to an already existing derivation goal.
@@ -315,17 +307,14 @@ struct DerivationGoal : public Goal
     virtual void cleanupPostOutputsRegisteredModeNonCheck();
 
 protected:
+    AsyncSemaphore::Token slotToken;
+
     kj::TimePoint lastChildActivity = kj::minValue;
 
-    kj::Promise<Result<std::optional<WorkResult>>> handleChildOutput() noexcept;
-    kj::Promise<Result<std::optional<WorkResult>>>
-    handleChildStreams(AsyncInputStream * builderIn, AsyncInputStream * hookIn) noexcept;
-    kj::Promise<Result<std::optional<WorkResult>>> handleBuilderOutput(AsyncInputStream & in
+    kj::Promise<Result<WorkResult>> wrapChildHandler(kj::Promise<Result<WorkResult>> handler
     ) noexcept;
-    kj::Promise<Result<std::optional<WorkResult>>> handleHookOutput(AsyncInputStream & in) noexcept;
-    kj::Promise<Result<std::optional<WorkResult>>> monitorForSilence() noexcept;
+    kj::Promise<Result<WorkResult>> monitorForSilence() noexcept;
     WorkResult tooMuchLogs();
-    void flushLine();
 
     virtual std::string buildErrorContents(const std::string & exitMsg, bool diskFull);
 
@@ -358,7 +347,7 @@ public:
 
     kj::Promise<Result<WorkResult>> repairClosure() noexcept;
 
-    void started();
+    std::string buildDescription() const;
 
     WorkResult done(
         BuildResult::Status status,

lix/libstore/build/goal.cc

@@ -22,12 +22,6 @@ kj::Promise<void> Goal::waitForAWhile()
 
 kj::Promise<Result<Goal::WorkResult>> Goal::work() noexcept
 try {
-    // always clear the slot token, no matter what happens. not doing this
-    // can cause builds to get stuck on exceptions (or other early exist).
-    // ideally we'd use scoped slot tokens instead of keeping them in some
-    // goal member variable, but we cannot do this yet for legacy reasons.
-    KJ_DEFER({ slotToken = {}; });
-
     BOOST_OUTCOME_CO_TRY(auto result, co_await workImpl());
 
     trace("done");

lix/libstore/build/goal.hh

@@ -82,9 +82,6 @@ struct Goal
      */
     std::string name;
 
-protected:
-    AsyncSemaphore::Token slotToken;
-
 public:
     struct [[nodiscard]] WorkResult {
         ExitCode exitCode;

lix/libstore/build/hook-instance.capnp

@@ -5,17 +5,16 @@ $Cxx.namespace("nix::rpc::build_remote");
 $Cxx.allowCancellation;
 
 using Types = import "/lix/libutil/types.capnp";
+using Log = import "/lix/libutil/logging.capnp";
 using StoreTypes = import "/lix/libstore/types.capnp";
 
 interface HookInstance {
-  interface BuildLogger {
-    # only used for fd passing
-  }
-
   interface AcceptedBuild {
     run @0 (
+      logger :Log.LogStream,
       inputs :List(StoreTypes.StorePath), # actual a set
       wantedOutputs :List(Data), # actually StringSet
+      description :Text, # root activity description for this build
     ) -> (result :Types.ResultV);
   }
 
@@ -23,19 +22,21 @@ interface HookInstance {
     union {
       accept :group {
         machine @0 :AcceptedBuild;
-        machineName @1 :Data;
       }
-      postpone @2 :Void;
-      decline @3 :Void;
-      declinePermanently @4 :Void;
+      postpone @1 :Void;
+      decline @2 :Void;
+      declinePermanently @3 :Void;
     }
   }
 
-  build @0 (
+  init @0 (
+    logger :Log.LogStream,
+    settings :Types.Settings,
+  ) -> (result :Types.ResultV);
+  build @1 (
     amWilling :Bool,
     neededSystem :Data,
     drvPath :StoreTypes.StorePath,
     requiredFeatures :List(Data),
-    buildLogger :BuildLogger,
   ) -> (result :Types.Result(BuildResponse));
 }

lix/libstore/build/hook-instance.cc

@@ -1,13 +1,66 @@
 #include "lix/libstore/build/child.hh"
+#include "lix/libutil/c-calls.hh"
 #include "lix/libutil/error.hh"
 #include "lix/libutil/file-system.hh"
 #include "lix/libstore/globals.hh"
 #include "lix/libstore/build/hook-instance.hh"
+#include "lix/libutil/json.hh"
+#include "lix/libutil/logging.hh"
+#include "lix/libutil/result.hh"
+#include "lix/libutil/rpc.hh"
+#include "lix/libutil/serialise.hh"
 #include "lix/libutil/strings.hh"
+#include "lix/libutil/logging-rpc.hh" // IWYU pragma: keep
+#include "lix/libutil/types-rpc.hh" // IWYU pragma: keep
+#include <kj/memory.h>
+#include <memory>
+#include <string_view>
 
 namespace nix {
 
-kj::Promise<Result<std::unique_ptr<HookInstance>>> HookInstance::create()
+void HookInstance::HookLogger::emitLog(rpc::log::Event::Result::Reader r)
+{
+    auto type = rpc::log::from(r.getType());
+    auto fields = r.getFields();
+    if (!type) {
+        return;
+    }
+
+    // ensure that logs from a builder using `ssh-ng://` as protocol
+    // are also available to `nix log`.
+    if (type == resBuildLogLine) {
+        if (fields.size() > 0 && fields[0].isS()) {
+            (*logSink)(fmt("%s\n", rpc::to<std::string_view>(fields[0].getS())));
+        } else {
+            (*logSink)("\n");
+        }
+    } else if (type == resSetPhase && fields.size() > 0 && fields[0].isS()) {
+        // nixpkgs' stdenv produces lines in the log to signal phase changes.
+        // We want to get the same lines in case of remote builds.
+        // The format is:
+        //   @nix { "action": "setPhase", "phase": "$curPhase" }
+        const auto phase = rpc::to<std::string_view>(fields[0].getS());
+        const auto logLine = JSON::object({{"action", "setPhase"}, {"phase", phase}});
+        (*logSink)("@nix " + logLine.dump(-1, ' ', false, JSON::error_handler_t::replace) + "\n");
+    }
+}
+
+kj::Promise<void> HookInstance::HookLogger::push(PushContext context)
+{
+    try {
+        auto e = context.getParams().getE();
+        if (logSink && e.isResult()) {
+            emitLog(e.getResult());
+        }
+    } catch (std::exception & e) { // NOLINT(lix-foreign-exceptions)
+        printError("error in log processor: %s", e.what());
+        throw; // NOLINT(lix-foreign-exceptions)
+    }
+
+    return RpcLoggerServer::push(context);
+}
+
+kj::Promise<Result<std::unique_ptr<HookInstance>>> HookInstance::create(const Activity & act)
 try {
     debug("starting build hook '%s'", concatStringsSep(" ", settings.buildHook.get()));
 
@@ -27,36 +80,23 @@ try {
 
     args.push_back(std::to_string(verbosity));
 
-    /* Create a pipe to get the output of the child. */
-    Pipe fromHook_;
-    fromHook_.create();
-
     /* Create the communication pipes. */
-    Pipe toHook_;
-    toHook_.create();
-
     auto [selfRPC, hookRPC] = SocketPair::stream();
 
     printMsg(lvlChatty, "running build hook: %s", concatMapStringsSep(" ", args, shellEscape));
 
     /* Fork the hook. */
     auto pid = startProcess([&]() {
-        if (dup2(fromHook_.writeSide.get(), STDERR_FILENO) == -1)
-            throw SysError("cannot pipe standard error into log file");
-
         commonExecveingChildInit();
 
         if (chdir("/") == -1) throw SysError("changing into /");
 
         /* Dup the communication pipes. */
-        if (dup2(toHook_.readSide.get(), STDIN_FILENO) == -1) {
-            throw SysError("dupping to-hook read side");
-        }
         if (dup2(hookRPC.get(), STDOUT_FILENO) == -1) {
             throw SysError("dupping to-hook read side");
         }
 
-        execv(buildHook.c_str(), stringsToCharPtrs(args).data());
+        sys::execv(buildHook, args);
 
         throw SysError("executing '%s'", buildHook);
     });
@@ -65,15 +105,20 @@ try {
 
     std::map<std::string, Config::SettingInfo> settings;
     globalConfig.getSettings(settings, true);
-    FdSink sink(toHook_.writeSide.get());
-    for (auto & setting : settings) {
-        sink << 1 << setting.first << setting.second.value;
+
+    auto conn = AIO().lowLevelProvider.wrapUnixSocketFd(kj::AutoCloseFd(selfRPC.release()));
+    auto client = std::make_unique<capnp::TwoPartyClient>(*conn, 1);
+    auto rpc = client->bootstrap().castAs<rpc::build_remote::HookInstance>();
+
+    {
+        auto initReq = rpc.initRequest();
+        initReq.setLogger(kj::heap<HookLogger>(act, nullptr));
+        RPC_FILL(initReq, initSettings, settings);
+        TRY_AWAIT_RPC(initReq.send());
     }
-    sink << 0;
-    sink.flush();
 
     co_return std::make_unique<HookInstance>(
-        std::move(fromHook_.readSide), std::move(selfRPC), std::move(pid)
+        kj::heap(std::move(rpc)).attach(std::move(conn), std::move(client)), std::move(pid)
     );
 } catch (...) {
     co_return result::current_exception();
@@ -82,7 +127,7 @@ try {
 HookInstance::~HookInstance()
 {
     try {
-        if (pid) pid.kill();
+        kill();
     } catch (...) {
         ignoreExceptionInDestructor();
     }

lix/libstore/build/hook-instance.hh

@@ -2,43 +2,75 @@
 ///@file
 
 #include "hook-instance.capnp.h"
+#include "lix/libutil/logging-rpc.hh"
 #include "lix/libutil/logging.hh"
 #include "lix/libutil/processes.hh"
 #include "lix/libutil/serialise.hh"
+#include "logging.capnp.h"
 #include <capnp/rpc-twoparty.h>
 #include <kj/async-io.h>
+#include <memory>
 #include <utility>
 
 namespace nix {
 
 struct HookInstance
 {
-    /**
-     * Pipe for the hook's standard output/error.
-     */
-    AutoCloseFD fromHook;
+    struct HookLogger : rpc::log::RpcLoggerServer
+    {
+        FinishSink * logSink;
 
-    kj::Own<kj::AsyncCapabilityStream> conn;
-    std::optional<capnp::TwoPartyClient> client;
-    rpc::build_remote::HookInstance::Client rpc;
+        HookLogger(const Activity & act, FinishSink * logSink)
+            : rpc::log::RpcLoggerServer(act)
+            , logSink(logSink)
+        {
+        }
 
-    /**
-     * The process ID of the hook.
-     */
-    Pid pid;
+        void emitLog(rpc::log::Event::Result::Reader r);
+        kj::Promise<void> push(PushContext context) override;
+    };
 
-    std::map<ActivityId, Activity> activities;
+    kj::Own<rpc::build_remote::HookInstance::Client> rpc;
 
-    static kj::Promise<Result<std::unique_ptr<HookInstance>>> create();
+    static kj::Promise<Result<std::unique_ptr<HookInstance>>> create(const Activity & act);
 
-    HookInstance(AutoCloseFD fromHook, AutoCloseFD rpc, Pid pid)
-        : fromHook(std::move(fromHook))
-        , conn(AIO().lowLevelProvider.wrapUnixSocketFd(kj::AutoCloseFd(rpc.release())))
-        , client(std::in_place, *this->conn, 1)
-        , rpc(client->bootstrap().castAs<rpc::build_remote::HookInstance>())
-        , pid(std::move(pid))
+    HookInstance(kj::Own<rpc::build_remote::HookInstance::Client> rpc, Pid pid)
+        : rpc(std::move(rpc))
+        , pidOrStatus(std::move(pid))
     {
     }
     ~HookInstance();
+
+    int wait()
+    {
+        return childStatusOr<&Pid::wait>();
+    }
+
+    int kill()
+    {
+        return childStatusOr<&Pid::kill>();
+    }
+
+private:
+    /**
+     * The process ID of the hook if it's running, or its exit status if not.
+     */
+    std::variant<Pid, int> pidOrStatus;
+
+    template<int (Pid::*fn)()>
+    int childStatusOr()
+    {
+        return std::visit(
+            overloaded{
+                [&](Pid & pid) {
+                    int status = (pid.*fn)();
+                    pidOrStatus = status;
+                    return status;
+                },
+                [](int status) { return status; },
+            },
+            pidOrStatus
+        );
+    }
 };
 }

lix/libstore/build/local-derivation-goal.cc

@@ -1,4 +1,5 @@
 #include "lix/libstore/build/local-derivation-goal.hh"
+#include "build/derivation-goal.hh"
 #include "lix/libutil/async-io.hh"
 #include "lix/libutil/async.hh"
 #include "lix/libutil/error.hh"
@@ -12,6 +13,7 @@
 #include "lix/libstore/path-references.hh"
 #include "lix/libutil/archive.hh"
 #include "lix/libstore/daemon.hh"
+#include "lix/libutil/c-calls.hh"
 #include "lix/libutil/fmt.hh"
 #include "lix/libutil/regex.hh"
 #include "lix/libutil/file-descriptor.hh"
@@ -239,8 +241,12 @@ retry:
 
         if (!buildUser) {
             if (!actLock)
-                actLock = std::make_unique<Activity>(*logger, lvlWarn, actBuildWaiting,
-                    fmt("waiting for a free build user ID for '%s'", Magenta(worker.store.printStorePath(drvPath))));
+                actLock = logger->startActivity(
+                    lvlWarn,
+                    actBuildWaiting,
+                    fmt("waiting for a free build user ID for '%s'",
+                        Magenta(worker.store.printStorePath(drvPath)))
+                );
             co_await waitForAWhile();
             // we can loop very often, and `co_return co_await` always allocates a new frame
             goto retry;
@@ -274,10 +280,8 @@ retry:
         /* Okay, we have to build. */
         TRY_AWAIT(startBuilder());
 
-        started();
-        if (auto error = TRY_AWAIT(handleChildOutput())) {
-            co_return std::move(*error);
-        }
+        mcRunningBuilds = worker.runningBuilds.addTemporarily(1);
+        co_return TRY_AWAIT(wrapChildHandler(handleRawChild()));
 
     } catch (BuildError & e) {
         outputLocks.reset();
@@ -286,8 +290,6 @@ retry:
         report.permanentFailure = true;
         co_return report;
     }
-
-    co_return co_await buildDone();
 } catch (...) {
     co_return result::current_exception();
 }
@@ -326,7 +328,6 @@ void LocalDerivationGoal::closeReadPipes()
         DerivationGoal::closeReadPipes();
     } else {
         builderOutPTY.close();
-        builderOutFD = nullptr;
     }
 }
 
@@ -370,13 +371,14 @@ bool LocalDerivationGoal::cleanupDecideWhetherDiskFull()
         auto & localStore = getLocalStore();
         uint64_t required = 8ULL * 1024 * 1024; // FIXME: make configurable
         struct statvfs st;
-        if (statvfs(localStore.config().realStoreDir.get().c_str(), &st) == 0 &&
-            (uint64_t) st.f_bavail * st.f_bsize < required)
-            diskFull = true;
-        if (statvfs(tmpDirRoot.c_str(), &st) == 0 && (uint64_t) st.f_bavail * st.f_bsize < required)
+        if (sys::statvfs(localStore.config().realStoreDir, &st) == 0
+            && (uint64_t) st.f_bavail * st.f_bsize < required)
         {
             diskFull = true;
         }
+        if (sys::statvfs(tmpDirRoot, &st) == 0 && (uint64_t) st.f_bavail * st.f_bsize < required) {
+            diskFull = true;
+        }
     }
 #endif
 
@@ -454,8 +456,7 @@ try {
 
         /* Create a temporary directory where the build will take
            place. */
-        tmpDirRoot =
-            createTempDir(buildDir, "nix-build-" + std::string(drvPath.name()), false, false, 0700);
+        tmpDirRoot = createTempSubdir(buildDir, std::nullopt, 0700);
     } catch (SysError & e) {
         /*
          * Fallback to the global tmpdir and create a safe space there
@@ -465,53 +466,41 @@ try {
             throw;
         }
 
-        auto globalTmp = defaultTempDir();
-        createDirs(globalTmp);
-#if __APPLE__
-        /* macOS filesystem namespacing does not exist, to avoid breaking builds, we need to weaken
-         * the mode bits on the top-level directory. This avoids issues like
-         * https://github.com/NixOS/nix/pull/11031. */
-        constexpr int toplevelDirMode = 0755;
-#else
-        constexpr int toplevelDirMode = 0700;
-#endif
-        auto nixBuildsTmp = createTempDir(
-            globalTmp, fmt("nix-builds-%s", geteuid()), false, false, toplevelDirMode
-        );
+        auto nixBuildsTmp = createTempDir(fmt("nix-builds-%s", geteuid()), 0700);
         printTaggedWarning(
             "Failed to use the system-wide build directory '%s', falling back to a temporary "
             "directory inside '%s'",
             settings.buildDir.get(),
             nixBuildsTmp
         );
+        tmpDirRoot = createTempSubdir(nixBuildsTmp, std::nullopt, 0700);
         worker.buildDirOverride = nixBuildsTmp;
-        tmpDirRoot = createTempDir(
-            nixBuildsTmp, "nix-build-" + std::string(drvPath.name()), false, false, 0700
-        );
     }
     /* The TOCTOU between the previous mkdir call and this open call is unavoidable due to
      * POSIX semantics.*/
-    tmpDirRootFd = AutoCloseFD{open(tmpDirRoot.c_str(), O_RDONLY | O_NOFOLLOW | O_DIRECTORY)};
+    tmpDirRootFd = sys::open(tmpDirRoot, O_RDONLY | O_NOFOLLOW | O_DIRECTORY);
     if (!tmpDirRootFd) {
         throw SysError("failed to open the build temporary directory descriptor '%1%'", tmpDirRoot);
     }
 
+#if __APPLE__
+    // The Darwin sandbox ensures that builds cannot change the
+    // permissions of their own build directory. Unsandboxed builds
+    // disable this, but have no isolation by design anyway. The
+    // minimal sandbox (applied even when `sandbox = false`, though not
+    // when `_NIX_TEST_NO_SANDBOX` is set) prevents the creation of
+    // `set{u,g}id` files regardless.
+    tmpDir = tmpDirRoot;
+    tmpDirFd = std::move(tmpDirRootFd);
+#else
     // place the actual build directory in a subdirectory of tmpDirRoot. if
     // we do not do this a build can `chown 777` its build directory and so
     // make it accessible to everyone in the system, breaking isolation. we
     // also need the intermediate level to be inaccessible to others. build
     // processes must be able to at least traverse to the directory though,
     // without being able to chmod. this means either mode 0750 or 0710. we
-    // cannot use 0710 because the libarchive we link with is compiled with
-    // an old apple sdk that does not have O_SEARCH, which makes libarchive
-    // try to open tmpDirRoot for *read* and fail because g+r is not set. a
-    // future update to nixpkgs may fix this. until then we do not lose any
-    // security by setting mode 0750 because we use only a single subdir in
-    // tmpDirRoot, so being able to list its parent doesn't break anything.
-    //
-    // use a short name to not increase the path length too much on darwin.
-    // darwin has a severe sockaddr_un path length limitation, so this does
-    // make a difference over more evocative names. we use `b` for `build`.
+    // use 0710 just to be extra safe; if we ever add more directories they
+    // will not be enumerable to other processes in the builder user group.
     tmpDir = tmpDirRoot + "/b";
     if (mkdirat(tmpDirRootFd.get(), "b", 0700)) {
         throw SysError("failed to create the build temporary directory '%1%'", tmpDir);
@@ -520,15 +509,17 @@ try {
     if (!tmpDirFd)
         throw SysError("failed to open the build temporary directory descriptor '%1%'", tmpDir);
 
-    chownToBuilder(tmpDirFd);
     if (buildUser) {
         if (fchown(tmpDirRootFd.get(), -1, buildUser->getGID()) == -1) {
             throw SysError("cannot change ownership of '%1%'", tmpDirRoot);
         }
-        if (fchmod(tmpDirRootFd.get(), 0750) == -1) {
+        if (fchmod(tmpDirRootFd.get(), 0710) == -1) {
             throw SysError("cannot change mode of '%1%'", tmpDirRoot);
         }
     }
+#endif
+
+    chownToBuilder(tmpDirFd);
 
     for (auto & [outputName, status] : initialOutputs) {
         /* Set scratch path we'll actually use during the build.
@@ -777,19 +768,21 @@ try {
 
     /* Create a pseudoterminal to get the output of the builder. */
     builderOutPTY = AutoCloseFD{posix_openpt(O_RDWR | O_NOCTTY)};
-    if (!builderOutPTY)
+    if (!builderOutPTY) {
         throw SysError("opening pseudoterminal master");
-    builderOutFD = &builderOutPTY;
+    }
 
     // FIXME: not thread-safe, use ptsname_r
     std::string slaveName = ptsname(builderOutPTY.get());
 
     if (buildUser) {
-        if (chmod(slaveName.c_str(), 0600))
+        if (sys::chmod(slaveName, 0600)) {
             throw SysError("changing mode of pseudoterminal slave");
+        }
 
-        if (chown(slaveName.c_str(), buildUser->getUID(), 0))
+        if (sys::chown(slaveName, buildUser->getUID(), 0)) {
             throw SysError("changing owner of pseudoterminal slave");
+        }
     }
 #if __APPLE__
     else {
@@ -802,9 +795,8 @@ try {
         throw SysError("unlocking pseudoterminal");
 
     /* Open the slave side of the pseudoterminal and use it as stderr. */
-    auto openSlave = [&]()
-    {
-        AutoCloseFD builderOut{open(slaveName.c_str(), O_RDWR | O_NOCTTY)};
+    auto openSlave = [&]() {
+        AutoCloseFD builderOut{sys::open(slaveName, O_RDWR | O_NOCTTY)};
         if (!builderOut)
             throw SysError("opening pseudoterminal slave");
 
@@ -896,7 +888,12 @@ void LocalDerivationGoal::initTmpDir() {
                 std::string fn = ".attr-" + hash.to_string(Base::Base32, false);
                 Path p = tmpDir + "/" + fn;
                 /* TODO(jade): we should have BorrowedFD instead of OwnedFD. */
-                AutoCloseFD passAsFileFd{openat(tmpDirFd.get(), fn.c_str(), O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC | O_EXCL | O_NOFOLLOW, 0666)};
+                AutoCloseFD passAsFileFd{sys::openat(
+                    tmpDirFd.get(),
+                    fn,
+                    O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC | O_EXCL | O_NOFOLLOW,
+                    0666
+                )};
                 if (!passAsFileFd) {
                     throw SysError("opening `passAsFile` file in the sandbox '%1%'", p);
                 }
@@ -925,7 +922,7 @@ void LocalDerivationGoal::initTmpDir() {
 void LocalDerivationGoal::setupConfiguredCertificateAuthority()
 {
     if (settings.caFile != "") {
-        if (pathAccessible(settings.caFile)) {
+        if (pathAccessible(settings.caFile, true)) {
             auto prefix = useChroot ?
 #if __linux__
                                     chrootRootDir
@@ -1083,8 +1080,9 @@ try {
 void LocalDerivationGoal::chownToBuilder(const Path & path)
 {
     if (!buildUser) return;
-    if (chown(path.c_str(), buildUser->getUID(), buildUser->getGID()) == -1)
+    if (sys::chown(path, buildUser->getUID(), buildUser->getGID()) == -1) {
         throw SysError("cannot change ownership of '%1%'", path);
+    }
 }
 
 void LocalDerivationGoal::chownToBuilder(const AutoCloseFD & fd)
@@ -1173,8 +1171,9 @@ void LocalDerivationGoal::runChild()
 
             /* Bind-mount chroot directory to itself, to treat it as a
                different filesystem from /, as needed for pivot_root. */
-            if (mount(chrootRootDir.c_str(), chrootRootDir.c_str(), 0, MS_BIND, 0) == -1)
+            if (sys::mount(chrootRootDir, chrootRootDir, "", MS_BIND, 0) == -1) {
                 throw SysError("unable to bind mount '%1%'", chrootRootDir);
+            }
 
             /* Bind-mount the sandbox's Nix store onto itself so that
                we can mark it as a "shared" subtree, allowing bind
@@ -1186,11 +1185,13 @@ void LocalDerivationGoal::runChild()
                to fail with EINVAL. Don't know why. */
             Path chrootStoreDir = chrootRootDir + worker.store.config().storeDir;
 
-            if (mount(chrootStoreDir.c_str(), chrootStoreDir.c_str(), 0, MS_BIND, 0) == -1)
+            if (sys::mount(chrootStoreDir, chrootStoreDir, "", MS_BIND, 0) == -1) {
                 throw SysError("unable to bind mount the Nix store", chrootStoreDir);
+            }
 
-            if (mount(0, chrootStoreDir.c_str(), 0, MS_SHARED, 0) == -1)
+            if (sys::mount("", chrootStoreDir, "", MS_SHARED, 0) == -1) {
                 throw SysError("unable to make '%s' shared", chrootStoreDir);
+            }
 
             /* Set up a nearly empty /dev, unless the user asked to
                bind-mount the host /dev. */
@@ -1295,21 +1296,31 @@ void LocalDerivationGoal::runChild()
 
             /* Bind a new instance of procfs on /proc. */
             createDirs(chrootRootDir + "/proc");
-            if (mount("none", (chrootRootDir + "/proc").c_str(), "proc", 0, 0) == -1)
+            if (sys::mount("none", chrootRootDir + "/proc", "proc", 0, 0) == -1) {
                 throw SysError("mounting /proc");
+            }
 
             /* Mount sysfs on /sys. */
             if (buildUser && buildUser->getUIDCount() != 1) {
                 createDirs(chrootRootDir + "/sys");
-                if (mount("none", (chrootRootDir + "/sys").c_str(), "sysfs", 0, 0) == -1)
+                if (sys::mount("none", chrootRootDir + "/sys", "sysfs", 0, 0) == -1) {
                     throw SysError("mounting /sys");
+                }
             }
 
             /* Mount a new tmpfs on /dev/shm to ensure that whatever
                the builder puts in /dev/shm is cleaned up automatically. */
-            if (pathExists("/dev/shm") && mount("none", (chrootRootDir + "/dev/shm").c_str(), "tmpfs", 0,
-                    fmt("size=%s", settings.sandboxShmSize).c_str()) == -1)
+            if (pathExists("/dev/shm")
+                && sys::mount(
+                       "none",
+                       chrootRootDir + "/dev/shm",
+                       "tmpfs",
+                       0,
+                       fmt("size=%s", settings.sandboxShmSize).c_str()
+                   ) == -1)
+            {
                 throw SysError("mounting /dev/shm");
+            }
 
             /* Mount a new devpts on /dev/pts.  Note that this
                requires the kernel to be compiled with
@@ -1319,7 +1330,10 @@ void LocalDerivationGoal::runChild()
                 !pathExists(chrootRootDir + "/dev/ptmx")
                 && !pathsInChroot.count("/dev/pts"))
             {
-                if (mount("none", (chrootRootDir + "/dev/pts").c_str(), "devpts", 0, "newinstance,mode=0620") == 0)
+                if (sys::mount(
+                        "none", (chrootRootDir + "/dev/pts"), "devpts", 0, "newinstance,mode=0620"
+                    )
+                    == 0)
                 {
                     createSymlink("/dev/pts/ptmx", chrootRootDir + "/dev/ptmx");
 
@@ -1364,8 +1378,9 @@ void LocalDerivationGoal::runChild()
                 throw SysError("unsharing cgroup namespace");
 
             /* Do the chroot(). */
-            if (chdir(chrootRootDir.c_str()) == -1)
+            if (sys::chdir(chrootRootDir) == -1) {
                 throw SysError("cannot change directory to '%1%'", chrootRootDir);
+            }
 
             if (mkdir("real-root", 0) == -1)
                 throw SysError("cannot create real-root directory");
@@ -1424,8 +1439,9 @@ void LocalDerivationGoal::runChild()
         }
 #endif
 
-        if (chdir(tmpDirInSandbox.c_str()) == -1)
+        if (sys::chdir(tmpDirInSandbox) == -1) {
             throw SysError("changing into '%1%'", tmpDir);
+        }
 
         /* Close all other file descriptors. */
         closeExtraFDs();
@@ -1658,7 +1674,7 @@ void LocalDerivationGoal::runChild()
 
 void LocalDerivationGoal::execBuilder(std::string builder, Strings args, Strings envStrs)
 {
-    execve(builder.c_str(), stringsToCharPtrs(args).data(), stringsToCharPtrs(envStrs).data());
+    sys::execve(builder, args, envStrs);
 }
 
 
@@ -1738,7 +1754,7 @@ try {
             continue;
         }
 
-        auto optSt = maybeLstat(actualPath.c_str());
+        auto optSt = maybeLstat(actualPath);
         if (!optSt)
             throw BuildError(
                 "builder for '%s' failed to produce output path for output '%s' at '%s'",
@@ -1809,15 +1825,18 @@ try {
             );
         }
 
+        outputGraph[scratchOutputs.at(name)] = StorePathSet{};
         std::visit(
             overloaded{/* Since we'll use the already installed versions of these, we
                                    can treat them as leaves and ignore any references they
                                    have. */
-                       [&](const AlreadyRegistered &) {
-                           outputGraph[scratchOutputs.at(name)] = StorePathSet{};
-                       },
+                       [&](const AlreadyRegistered &) {},
                        [&](const PerhapsNeedToRegister & refs) {
-                           outputGraph[scratchOutputs.at(name)] = refs.refs;
+                           for (auto & ref : refs.refs) {
+                               if (inverseOutputMap.find(ref) != inverseOutputMap.end()) {
+                                   outputGraph[scratchOutputs.at(name)].insert(ref);
+                               }
+                           }
                        }
             },
             *orifu
@@ -1828,10 +1847,8 @@ try {
         topoSort(outputsToSort, {[&](const std::string & name) {
                      StringSet dependencies;
                      for (auto & path : outputGraph.at(scratchOutputs.at(name))) {
-                         auto outputName = inverseOutputMap.find(path);
-                         if (outputName != inverseOutputMap.end()) {
-                             dependencies.insert(outputName->second);
-                         }
+                         auto outputName = inverseOutputMap.at(path);
+                         dependencies.insert(outputName);
                      }
                      return dependencies;
                  }});
@@ -2192,7 +2209,8 @@ try {
                 debug("unreferenced input: '%1%'", worker.store.printStorePath(i));
         }
 
-        localStore.optimisePath(actualPath, NoRepair); // FIXME: combine with scanForReferences()
+        // FIXME: combine with scanForReferences()
+        TRY_AWAIT(localStore.optimisePath(actualPath, NoRepair));
         worker.markContentsGood(newInfo.path);
 
         newInfo.deriver = drvPath;
@@ -2216,9 +2234,11 @@ try {
             msg << HintFmt("derivation '%s' may not be deterministic: outputs differ", drvPath.to_string());
             for (auto [oldPath, newPath]: nondeterministic) {
                 if (newPath) {
-                    msg << HintFmt("\n  output differs: output '%s' differs from '%s'", oldPath.c_str(), *newPath);
+                    msg << HintFmt(
+                        "\n  output differs: output '%s' differs from '%s'", oldPath, *newPath
+                    );
                 } else {
-                    msg << HintFmt("\n  output '%s' differs", oldPath.c_str());
+                    msg << HintFmt("\n  output '%s' differs", oldPath);
                 }
             }
             throw NotDeterministic(msg.str());
@@ -2569,36 +2589,39 @@ try {
 static void makeVisible(int parentFd, const char * entry, uid_t user, gid_t group)
 {
     struct stat st;
+    // NOLINTNEXTLINE(lix-unsafe-c-calls): entry is a dentry name
     if (fstatat(parentFd, entry, &st, AT_SYMLINK_NOFOLLOW)) {
         throw SysError("fstat(%s)", guessOrInventPathFromFD(parentFd));
     }
     if (S_ISDIR(st.st_mode)) {
-        int dirfd = openat(parentFd, entry, O_RDONLY | O_DIRECTORY | O_NOFOLLOW);
-        if (dirfd < 0) {
+        auto dirfd = sys::openat(parentFd, entry, O_RDONLY | O_DIRECTORY | O_NOFOLLOW);
+        if (!dirfd) {
             throw SysError("openat(%s/%s)", guessOrInventPathFromFD(parentFd), entry);
         }
-        AutoCloseDir dir(fdopendir(dirfd));
+        AutoCloseDir dir(fdopendir(dirfd.get()));
         if (!dir) {
-            close(dirfd);
             throw SysError("fdopendir(%s/%s)", guessOrInventPathFromFD(parentFd), entry);
         }
+        dirfd.release();
 
         struct dirent * dirent;
         while (errno = 0, dirent = readdir(dir.get())) {
             if (strcmp(dirent->d_name, ".") == 0 || strcmp(dirent->d_name, "..") == 0) {
                 continue;
             }
-            makeVisible(dirfd, dirent->d_name, user, group);
+            makeVisible(::dirfd(dir.get()), dirent->d_name, user, group);
         }
     }
 
     // ignore permissions errors for symlinks. linux can't chmod them.
     // clear special permission bits while we're here, just to be safe
-    if (fchmodat(parentFd, entry, st.st_mode & 0777, AT_SYMLINK_NOFOLLOW) && !S_ISLNK(st.st_mode)) {
+    if (sys::fchmodat(parentFd, entry, st.st_mode & 0777, AT_SYMLINK_NOFOLLOW)
+        && !S_ISLNK(st.st_mode))
+    {
         throw SysError("fchmod(%s)", guessOrInventPathFromFD(parentFd));
     }
     if (user != uid_t(-1) && group != gid_t(-1)
-        && fchownat(parentFd, entry, user, group, AT_SYMLINK_NOFOLLOW))
+        && sys::fchownat(parentFd, entry, user, group, AT_SYMLINK_NOFOLLOW))
     {
         throw SysError("fchown(%s)", guessOrInventPathFromFD(parentFd));
     }
@@ -2621,7 +2644,7 @@ void LocalDerivationGoal::finalizeTmpDir(bool force, bool duringDestruction)
             } catch (SysError & e) {
                 printError("error making '%s' accessible: %s", tmpDir, e.what());
             }
-            chmod(tmpDirRoot.c_str(), 0755);
+            (void) sys::chmod(tmpDirRoot, 0755);
         }
         else if (duringDestruction)
             deletePathUninterruptible(tmpDirRoot);
@@ -2647,5 +2670,88 @@ StorePath LocalDerivationGoal::makeFallbackPath(const StorePath & path)
         Hash(HashType::SHA256), path.name());
 }
 
+kj::Promise<Result<Goal::WorkResult>> LocalDerivationGoal::handleRawChild() noexcept
+try {
+    if (auto error = TRY_AWAIT(handleRawChildStream())) {
+        co_return std::move(*error);
+    }
+    co_return TRY_AWAIT(buildDone());
+} catch (...) {
+    co_return result::current_exception();
+}
 
+kj::Promise<Result<std::optional<Goal::WorkResult>>>
+LocalDerivationGoal::handleRawChildStream() noexcept
+try {
+    AsyncFdIoStream in(AsyncFdIoStream::shared_fd{}, builderOutPTY.get());
+
+    std::map<ActivityId, Activity> builderActivities;
+    LogLineSplitter splitter;
+
+    auto act = logger->startActivity(
+        lvlInfo,
+        actBuild,
+        buildDescription(),
+        Logger::Fields{worker.store.printStorePath(drvPath), "", 1, 1}
+    );
+
+    auto flushLine = [&](const std::string & line) {
+        if (const auto state =
+                handleJSONLogMessage(line, act, builderActivities, "the derivation builder"))
+        {
+            return *state;
+        } else {
+            logTail.push_back(line);
+            if (logTail.size() > settings.logLines) {
+                logTail.pop_front();
+            }
+
+            return act.result(resBuildLogLine, line);
+        }
+    };
+
+    auto buf = kj::heapArray<char>(4096);
+    while (true) {
+        std::string_view data;
+        try {
+            if (const auto got = TRY_AWAIT(in.read(buf.begin(), buf.size()))) {
+                data = {buf.begin(), *got};
+            } else {
+                co_return std::nullopt;
+            }
+        } catch (SysError & e) {
+            // the builder output stream may be a pty fd, and closing one pty
+            // endpoint sends EIO to the other endpoint. this is a good exit.
+            if (e.errNo == EIO) {
+                data = {};
+            } else {
+                throw;
+            }
+        }
+        lastChildActivity = AIO().provider.getTimer().now();
+
+        if (data.empty()) {
+            if (auto left = splitter.finish(); !left.empty()) {
+                if (flushLine(left) == Logger::BufferState::NeedsFlush) {
+                    TRY_AWAIT(act.getLogger().flush());
+                }
+            }
+            co_return std::nullopt;
+        }
+
+        if (logSink) {
+            (*logSink)(data);
+        }
+
+        while (!data.empty()) {
+            if (auto line = splitter.feed(data)) {
+                if (flushLine(*line) == Logger::BufferState::NeedsFlush) {
+                    TRY_AWAIT(act.getLogger().flush());
+                }
+            }
+        }
+    }
+} catch (...) {
+    co_return result::current_exception();
+}
 }

lix/libstore/build/local-derivation-goal.hh

@@ -324,6 +324,9 @@ protected:
      */
     virtual Pid startChild(std::function<void()> openSlave);
 
+    kj::Promise<Result<WorkResult>> handleRawChild() noexcept;
+    kj::Promise<Result<std::optional<WorkResult>>> handleRawChildStream() noexcept;
+
     /**
      * Set up the system call filtering required for the sandbox.
      * This currently only has an effect on Linux.

lix/libstore/build/sandbox-defaults.sb

@@ -26,6 +26,20 @@ R""(
 ; Allow getpwuid.
 (allow mach-lookup (global-name "com.apple.system.opendirectoryd.libinfo"))
 
+; Disallow messing with the top‐level build directory.
+(deny file-write-owner file-write-flags file-write-xattr file-write-mount
+      file-write-unmount
+      (literal (param "_NIX_BUILD_TOP")))
+; Nixpkgs does `chmod -R` on `$NIX_BUILD_TOP/$sourceRoot` by default,
+; which results in it trying to set the mode of `$NIX_BUILD_TOP` when
+; derivations set `sourceRoot = ".";`. Thankfully, the GNU `chmod(1)`
+; treats `ENOTSUP` as a non‐fatal, non‐reported error in this case, and
+; continues to descend into the directory tree.
+;
+; See: <https://gitweb.git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f=src/chmod.c;hb=refs/tags/v9.7#l312>
+(deny file-write-mode (with errno ENOTSUP)
+      (literal (param "_NIX_BUILD_TOP")))
+
 ; Access to /tmp and the build directory.
 ; The network-outbound/network-inbound ones are for unix domain sockets, which
 ; we allow access to in TMPDIR (but if we allow them more broadly, you could in