P1: address review (fail-open hardening)

Release produced with releng/create_release.xsh

Change-Id: I24bd93272db7b333f509f8d77820dbf5db5339e3

doc/manual/change-authors.yml

@@ -70,6 +70,9 @@ detroyejr:
   display_name: Jonathan De Troye
   github: detroyejr
 
+edef:
+  github: edef1c
+
 edolstra:
   display_name: Eelco Dolstra
   github: edolstra
@@ -197,6 +200,9 @@ roberth:
   display_name: Robert Hensing
   github: roberth
 
+sandydoo:
+  github: sandydoo
+
 seppel3210:
   github: Seppel3210
 

doc/manual/src/release-notes/rl-2.93.md

@@ -1,4 +1,242 @@
 # Lix 2.93 "Bici Bici" (2025-05-09)
+# Lix 2.93.4 (2026-05-04)
+## Fixes
+
+- `build-dir` no longer defaults to `temp-dir` [cl/3453](https://gerrit.lix.systems/c/lix/+/3453)
+
+  The directory in which temporary build directories are created no longer defaults
+  to the value of the `temp-dir` setting to avoid builders making their directories
+  world-accessible. This behavior has been used to escape the build sandbox and can
+  cause build impurities even when not used maliciously. We now default to `builds`
+  in `NIX_STATE_DIR` (which is `/nix/var/nix/b` in the default configuration).
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+- Fix develop shells for derivations with escape codes [fj#991](https://git.lix.systems/lix-project/lix/issues/991) [cl/4154](https://gerrit.lix.systems/c/lix/+/4154) [cl/4155](https://gerrit.lix.systems/c/lix/+/4155)
+
+  ASCII control characters (including `\e`, used for ANSI escape codes) in derivation variables are now correctly escaped for `nix develop` and `nix print-dev-env`, instead of erroring.
+
+  Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
+
+- Fix nix develop for derivations that rejects dependencies with structured attrs [fj#997](https://git.lix.systems/lix-project/lix/issues/997) [cl/4182](https://gerrit.lix.systems/c/lix/+/4182) [cl/4214](https://gerrit.lix.systems/c/lix/+/4214)
+
+  For the sake of concision, we refer to `disallowedReferences` in what follows,
+  but all output checks were equally fixed:
+  `{dis,}allowed{References,Requisites}`.
+
+  Derivations can define *output checks* to reject unwanted dependencies, such as
+  interpreters like `bash` or compilers like `gcc`. This can be done in two ways:
+
+  * **Legacy style**: `disallowedReferences = [ ... ]` in the environment.
+  * **Structured attrs**: `outputChecks.<output>.disallowedReferences = [ ... ]`,
+    typically used in `__json`.
+
+  Only the structured form supports derivations with multiple outputs.
+
+  `nix develop` internally rewrites derivations to create development shells. It
+  relied on the legacy `disallowedReferences`, and failed to honor the structured
+  variant. This led to broken shells in cases where `bashInteractive` was
+  explicitly disallowed using structured output checks, e.g. `nix develop
+  nixpkgs#systemd` after the "bash-less NixOS" changes.
+
+  This fix teaches `nix develop` to respect structured output checks, restoring
+  support for such derivations.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
+
+- `nix-shell` default shell directory is not `/tmp` anymore for `$NIX_BUILD_TOP` [fj#940](https://git.lix.systems/lix-project/lix/issues/940)
+
+  Previously, Lix `nix-shell`s could exit non-zero status when `stdenv`'s `dumpVars` phase failed to write to `$NIX_BUILD_TOP/env-vars`, despite `dumpVars` being intended as a debugging aid.
+
+  This happens when `TMPDIR` is not set and defaults therefore to `/tmp`, resulting in a `/tmp/env-vars` global file that every `nix-shell` wants to write.
+
+  We fix this issue by reusing a pre-created, unique, and writable location, as the build top directory, avoiding shell exiting from write failures silently.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
+
+- Fix unsigned overflow leading to out-of-band write in the NAR parser [cl/5537](https://gerrit.lix.systems/c/lix/+/5537)
+
+  The NAR parser contained an unsigned integer overflow that could be used by an
+  attacker to write arbitrary data to an unknown memory location and possibly
+  achieve code execution. A successful attack on the system-wide Lix daemon
+  could lead to privilege escalation to root. Any process that involves NAR
+  serialization could trigger this issue, including (but not limited to)
+
+    - local user interaction, whether the users are trusted or untrusted
+    - malicious substituters sending malformed NARs
+    - remote builders sending malformed build results
+    - remote daemons sending malformed inputs when requesting remote builds
+
+  Successful attacks using this bug require ASLR weakening of some sort, whether
+  by architecture constraints (e.g. on 32 bit systems, where little randomization
+  is possible) or system configuration (e.g. low ASLR entropy when loading
+  libraries), and millions of attempts. Local attacks can be mounted in less than
+  an hour. Remote builds typically require a fresh SSH connection for each build
+  and are thus less susceptible. Only one attempt can be made by substituters for
+  every build using substituters, they are thus not a likely vector for attacks.
+
+  At the time of writing, MITRE has not assigned this a CVE yet.
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae), [Raito Bezarius](https://git.lix.systems/raito), [edef](https://github.com/edef1c), and [sandydoo](https://github.com/sandydoo) for this.
+
+
+
+
+# Lix 2.93.3 (2025-07-22)
+## Improvements
+
+- `--keep-failed` chowns the build directory to the user that request the build [cl/3678](https://gerrit.lix.systems/c/lix/+/3678)
+
+  Running a build with `--keep-failed` now chowns the temporary directory from the
+  builder user and group to the user that request the build if the build came from
+  a local user connected to the daemon. This makes inspecting failed derivations a
+  lot easier. On Linux the build directory made visible to the user will not be in
+  the same path as it was in the sandbox and continuing builds will usually break.
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+
+
+
+# Lix 2.93.2 (2025-06-30)
+## Fixes
+
+- Revert CVE-2025-52992 failed mitigation [fj#883](https://git.lix.systems/lix-project/lix/issues/883) [fj#887](https://git.lix.systems/lix-project/lix/issues/887) [cl/3444](https://gerrit.lix.systems/c/lix/+/3444) [cl/3528](https://gerrit.lix.systems/c/lix/+/3528)
+
+  Following the initial mitigation of **CVE-2025-52992** in `cl/3444`, we
+  received reports of **unexpected deletion of in-use store paths**.
+
+  Upon investigation, we found that the patch did **not correctly cancel all
+  automatic deleters**, resulting in potentially critical path loss during normal
+  operation.
+
+  Given the severity and time-sensitive nature of the situation ([see incident
+  report](https://lix.systems/blog/2025-06-27-lix-critical-bug/)), we evaluated
+  possible options to repair the behavior safely. However, we concluded that a
+  rushed fix would either
+
+  * **Overdelete**, i.e. breaking running systems, or,
+  * **Underdelete**, effectively **reopening CVE-2025-52992** while leaving
+    orphaned paths behind.
+
+  As **CVE-2025-52992 has no known exploit vector**, and correctness is critical
+  in the Lix project, we have **fully reverted the previous mitigations**.
+
+  The affected patches (`cl/3444`) have been rolled back for the time being.
+
+  Moving forward, the Lix team will rework this code path in a **long-term,
+  correctness-first fix** on the main branch. We will explore backporting it to
+  stable channels once its safety is assured.
+
+  We are deeply sorry for the stability incident and the Lix team remain
+  available for assisting you in recovering your systems.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+- Fallback to safe temp dir when build-dir is unwritable [fj#876](https://git.lix.systems/lix-project/lix/issues/876) [cl/3501](https://gerrit.lix.systems/c/lix/+/3501)
+
+  Non-daemon builds started failing with a permission error after introducing the `build-dir` option:
+
+  ```
+  $ nix build --store ~/scratch nixpkgs#hello --rebuild
+  error: creating directory '/nix/var/nix/builds/nix-build-hello-2.12.2.drv-0': Permission denied
+  ```
+
+  This happens because:
+
+  1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
+  2. The user lacks permissions for that path.
+
+  We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
+
+  To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
+
+  **Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+
+
+
+# Lix 2.93.1 (2025-06-23)
+## Breaking Changes
+
+- Fixed output derivations can be run using `pasta` network isolation [fj#285](https://git.lix.systems/lix-project/lix/issues/285) [cl/3442](https://gerrit.lix.systems/c/lix/+/3442)
+
+  Fixed output derivations traditionally run in the host network namespace.
+  On Linux this allows such derivations to communicate with other sandboxes
+  or the host using the abstract Unix domains socket namespace; this hasn't
+  been unproblematic in the past and has been used in two distinct exploits
+  to break out of the sandbox. For this reason fixed output derivations can
+  now run in a network namespace (provided by [`pasta`]), restricted to TCP
+  and UDP communication with the rest of the world. When enabled this could
+  be a breaking change and we classify it as such, even though we don't yet
+  enable or require such isolation by default. We may enforce this in later
+  releases of Lix once we have sufficient confidence that breakage is rare.
+
+  [`pasta`]: https://passt.top/
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [puck](https://git.lix.systems/puck) for this.
+
+
+## Fixes
+
+- Always clean up scratch paths after derivations failed to build [cl/3444](https://gerrit.lix.systems/c/lix/+/3444)
+
+  Previously, scratch paths created during builds were not always cleaned up if
+  the derivation failed, potentially leaving behind unnecessary temporary files
+  or directories in the Nix store.
+
+  This fix ensures that such paths are consistently removed after a failed build,
+  improving Nix store hygiene, hardening Lix against mis-reuse of failed builds
+  scratch paths.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+- `build-dir` no longer defaults to `temp-dir` [cl/3443](https://gerrit.lix.systems/c/lix/+/3443)
+
+  The directory in which temporary build directories are created no longer defaults
+  to the value of the `temp-dir` setting to avoid builders making their directories
+  world-accessible. This behavior has been used to escape the build sandbox and can
+  cause build impurities even when not used maliciously. We now default to `builds`
+  in `NIX_STATE_DIR` (which is `/nix/var/nix/builds` in the default configuration).
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+- Remove reliance on Bash for remote stores via SSH [fj#830](https://git.lix.systems/lix-project/lix/issues/830) [fj#805](https://git.lix.systems/lix-project/lix/issues/805) [fj#304](https://git.lix.systems/lix-project/lix/issues/304) [cl/3159](https://gerrit.lix.systems/c/lix/+/3159)
+
+  The pre-flight `echo started` handshake -- added years ago to catch race conditions -- has been removed.
+
+  After removal of connection sharing in Lix 2.93, it required a Bash-compatible shell and a standard `echo`, so it failed on:
+
+    * builders protected by `ForceCommand` wrappers (e.g. `nix-remote-build`),
+    * BusyBox / initrd images with no Bash,
+    * hosts using non-POSIX shells such as Nushell.
+
+  The race the probe once addressed was tied to SSH connection-sharing -- since connection-sharing code has already been removed, the probe is now pointless.
+
+  Real connection or protocol errors are now left to SSH/Nix to report directly.
+
+  This is technically a breaking change if you had scripts that relied on the literal "started" which needs to be updated to rely on other signals, e.g., exit codes.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
+
+
+## Miscellany
+
+- Deprecation of CA derivations, dynamic derivations, and impure derivations [fj#815](https://git.lix.systems/lix-project/lix/issues/815)
+
+  Content-addressed derivations are now deprecated and slated for removal in Lix 2.94.
+  We're doing this because the CA derivation system has been a known cause of problems
+  and inconsistencies, is unmaintained, habitually makes improving the store code very
+  difficult (or blocks such improvements outright), and is beset by a number of design
+  flaws that in our opinion cannot be fixed without a full reimplementation from zero.
+  Dynamic derivations and impure derivations are built on the CA derivation framework,
+  and owing to this they too are deprecated and slated for removal in another release.
+
+
+
+
 
 
 # Lix 2.93.0 (2025-05-09)

docker.nix

@@ -62,8 +62,7 @@ let
     ++ autoLayered
     ++ extraPkgs;
 
-  users =
+  users = {
-    {
 
     root = {
       uid = 0;
@@ -193,13 +192,11 @@ let
             in
             ''
               {
-                ${
+                ${lib.concatStringsSep "\n" (
-                  lib.concatStringsSep "\n" (
                   builtins.map (output: ''
                     ${output} = { outPath = "${lib.getOutput output drv}"; };
                   '') outputs
-                  )
+                )}
-                }
                 outputs = [ ${lib.concatStringsSep " " (builtins.map (x: "\"${x}\"") outputs)} ];
                 name = "${drv.name}";
                 outPath = "${drv}";
@@ -361,8 +358,10 @@ let
         "org.opencontainers.image.source" = "https://git.lix.systems/lix-project/lix";
         "org.opencontainers.image.vendor" = "Lix project";
         "org.opencontainers.image.version" = pkgs.nix.version;
-        "org.opencontainers.image.description" = "Minimal Lix container image, with some batteries included.";
+        "org.opencontainers.image.description" =
-      } // lib.optionalAttrs (lixRevision != null) { "org.opencontainers.image.revision" = lixRevision; };
+          "Minimal Lix container image, with some batteries included.";
+      }
+      // lib.optionalAttrs (lixRevision != null) { "org.opencontainers.image.revision" = lixRevision; };
     };
 
     meta = {

flake.lock

@@ -16,6 +16,22 @@
         "type": "github"
       }
     },
+    "lowdown-src": {
+      "flake": false,
+      "locked": {
+        "lastModified": 1633514407,
+        "narHash": "sha256-Dw32tiMjdK9t3ETl5fzGrutQTzh2rufgZV4A/BbxuD4=",
+        "owner": "kristapsdz",
+        "repo": "lowdown",
+        "rev": "d2c2b44ff6c27b936ec27358a2653caaef8f73b8",
+        "type": "github"
+      },
+      "original": {
+        "owner": "kristapsdz",
+        "repo": "lowdown",
+        "type": "github"
+      }
+    },
     "nix2container": {
       "flake": false,
       "locked": {
@@ -32,18 +48,44 @@
         "type": "github"
       }
     },
-    "nixpkgs": {
+    "nix_2_18": {
+      "inputs": {
+        "flake-compat": [
+          "flake-compat"
+        ],
+        "lowdown-src": "lowdown-src",
+        "nixpkgs": "nixpkgs",
+        "nixpkgs-regression": [
+          "nixpkgs-regression"
+        ]
+      },
       "locked": {
-        "lastModified": 1733348545,
+        "lastModified": 1730375271,
-        "narHash": "sha256-b4JrUmqT0vFNx42aEN9LTWOHomkTKL/ayLopflVf81U=",
+        "narHash": "sha256-RrOFlDGmRXcVRV2p2HqHGqvzGNyWoD0Dado/BNlJ1SI=",
         "owner": "NixOS",
-        "repo": "nixpkgs",
+        "repo": "nix",
-        "rev": "9ecb50d2fae8680be74c08bb0a995c5383747f89",
+        "rev": "0f665ff6779454f2117dcc32e44380cda7f45523",
         "type": "github"
       },
       "original": {
         "owner": "NixOS",
-        "ref": "nixos-24.11-small",
+        "ref": "2.18.9",
+        "repo": "nix",
+        "type": "github"
+      }
+    },
+    "nixpkgs": {
+      "locked": {
+        "lastModified": 1705033721,
+        "narHash": "sha256-K5eJHmL1/kev6WuqyqqbS1cdNnSidIZ3jeqJ7GbrYnQ=",
+        "owner": "NixOS",
+        "repo": "nixpkgs",
+        "rev": "a1982c92d8980a0114372973cbdfe0a307f1bdea",
+        "type": "github"
+      },
+      "original": {
+        "owner": "NixOS",
+        "ref": "nixos-23.05-small",
         "repo": "nixpkgs",
         "type": "github"
       }
@@ -64,6 +106,22 @@
         "type": "github"
       }
     },
+    "nixpkgs_2": {
+      "locked": {
+        "lastModified": 1757198069,
+        "narHash": "sha256-m3VUcOD4rTs8J7S+3dOjWMrAjw6RcITC3XYQ98zhEFs=",
+        "owner": "NixOS",
+        "repo": "nixpkgs",
+        "rev": "0747026fc57ecb9c28901c7f7a2b5dc40e8af43c",
+        "type": "github"
+      },
+      "original": {
+        "owner": "NixOS",
+        "ref": "nixos-25.05-small",
+        "repo": "nixpkgs",
+        "type": "github"
+      }
+    },
     "pre-commit-hooks": {
       "flake": false,
       "locked": {
@@ -84,7 +142,8 @@
       "inputs": {
         "flake-compat": "flake-compat",
         "nix2container": "nix2container",
-        "nixpkgs": "nixpkgs",
+        "nix_2_18": "nix_2_18",
+        "nixpkgs": "nixpkgs_2",
         "nixpkgs-regression": "nixpkgs-regression",
         "pre-commit-hooks": "pre-commit-hooks"
       }

flake.nix

@@ -2,8 +2,19 @@
   description = "Lix: A modern, delicious implementation of the Nix package manager";
 
   inputs = {
-    nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.11-small";
+    nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.05-small";
     nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
+
+    # Required because Nix 2.18 is not in Nixpkgs ≥ 25.05 anymore.
+    nix_2_18 = {
+      url = "github:NixOS/nix/2.18.9";
+      # NOTE(Raito): this is not possible because patches on libseccomp does not apply anymore on this Nix.
+      # Let's keep the latest known nixpkgs useable with Nix 2.18 for our tests.
+      # inputs.nixpkgs.follows = "nixpkgs";
+      inputs.nixpkgs-regression.follows = "nixpkgs-regression";
+      inputs.flake-compat.follows = "flake-compat";
+    };
+
     pre-commit-hooks = {
       url = "github:cachix/git-hooks.nix";
       flake = false;
@@ -25,6 +36,7 @@
       nixpkgs-regression,
       pre-commit-hooks,
       nix2container,
+      nix_2_18,
       flake-compat,
     }:
 
@@ -163,6 +175,16 @@
         {
           nixStable = prev.nix;
 
+          nixVersions = prev.nixVersions // {
+            nix_2_3 = prev.nixVersions.nix_2_3.overrideAttrs (old: {
+              meta = old.meta // {
+                knownVulnerabilities = [ ];
+              };
+            });
+            # Nix 2.18 has been removed from Nixpkgs ≥ 25.05, so we need to reintroduce it ourselves for our tests.
+            nix_2_18 = nix_2_18.outputs.packages.${currentStdenv.hostPlatform.system}.default;
+          };
+
           # Forward from the previous stage as we don’t want it to pick the lowdown override
           nixUnstable = prev.nixUnstable;
 
@@ -197,6 +219,9 @@
             inherit versionSuffix officialRelease;
             stdenv = currentStdenv;
             busybox-sandbox-shell = final.busybox-sandbox-shell or final.default-busybox-sandbox-shell;
+            # See below
+            lowdown = final.lowdown_3_0;
+            lowdown-unsandboxed = final.lowdown_3_0.override { enableDarwinSandbox = false; };
           };
 
           lix-clang-tidy = final.callPackage ./subprojects/lix-clang-tidy { };
@@ -221,6 +246,30 @@
           boehmgc-nix = final.nix.passthru.boehmgc-nix;
           # And same thing for our build-release-notes package.
           build-release-notes = final.nix.passthru.build-release-notes;
+
+          # As soon as Nixpkgs updates to >= 3.0.0, change to lowdown_2_0!
+          # We don't change the default version in order to not change the hash
+          # of Nix/Lix from upstream Nixpkgs.
+          lowdown_3_0 =
+            assert lib.versionOlder prev.lowdown.version "3.0.0";
+            prev.lowdown.overrideAttrs (
+              finalAttrs: prevAttrs: {
+                version = "3.0.1";
+                src = final.fetchurl {
+                  url = "https://kristaps.bsd.lv/lowdown/snapshots/lowdown-${finalAttrs.version}.tar.gz";
+                  sha512 = "fe68e1b7ff23f3992398356d7aa9a330dfd7b72e22bea9a91eeef74182b209ecea0c9f3e2b2216e1a07b2358da2b746238ec9cbbdeebdd3551cef14dd2d79f46";
+                };
+
+                # no longer compiles with GNU make
+                nativeBuildInputs = prevAttrs.nativeBuildInputs ++ [ final.bmake ];
+                # dylib fixups on darwin are no longer necessary
+                postInstall = "";
+                # doesn't work on darwin due to disallowed nested sandboxes
+                doInstallCheck = prevAttrs.doInstallCheck && !(final.stdenv.hostPlatform.isDarwin);
+                doCheck = prevAttrs.doCheck && !(final.stdenv.hostPlatform.isDarwin);
+              }
+            );
+
         };
     in
     {
@@ -235,6 +284,15 @@
         # Binary package for various platforms.
         build = forAllSystems (system: self.packages.${system}.nix);
 
+        # Ensure support for lowdown < 3.0 doesn't regress for NixOS 25.11
+        build-lowdown_2_0.aarch64-linux = lib.genAttrs [ "aarch64-linux" ] (
+          system:
+          self.packages.${system}.nix.override {
+            lowdown = nixpkgsFor.${system}.native.lowdown;
+            lowdown-unsandboxed = nixpkgsFor.${system}.native.lowdown-unsandboxed;
+          }
+        );
+
         devShell = forAllSystems (system: {
           default = self.devShells.${system}.default;
           clang = self.devShells.${system}.native-clangStdenvPackages;
@@ -381,12 +439,15 @@
               in
               pkgs.symlinkJoin {
                 name = "nixpkgs-lib-tests";
-                paths =
+                paths = [
-                  [ testWithNix ]
+                  testWithNix
-                  # NOTE: nixpkgs 24.11 is being ... *creative*, and requires this dance to override
+                ]
+                # NOTE: nixpkgs 25.05 is being ... *creative*, and requires this dance to override
                 # the evaluator used for the test. it will break again in the future, don't worry.
                 ++ lib.optionals pkgs.stdenv.isLinux [
-                    (pkgs.callPackage "${nixpkgs}/ci/eval" { nixVersions.nix_2_24 = nix; }).attrpathsSuperset
+                  ((pkgs.callPackage "${nixpkgs}/ci/eval" { inherit nix; }).attrpathsSuperset {
+                    evalSystem = system;
+                  })
                 ];
               }
             );

lix/legacy/build-remote.cc

@@ -1,8 +1,21 @@
 #include <algorithm>
 #include <chrono>
 #include <set>
+#include <map>
 #include <memory>
+#include <optional>
 #include <tuple>
+#include <fstream>
+#include <sstream>
+#include <cstring>
+#include <cerrno>
+#include <sys/socket.h>
+#include <netdb.h>
+#include <netinet/in.h>
+#include <poll.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <nlohmann/json.hpp>
 #if __APPLE__
 #include <sys/time.h>
 #endif
@@ -53,6 +66,220 @@ static bool allSupportedLocally(Store & store, const std::set<std::string>& requ
     return true;
 }
 
+/* --------------------------------------------------------------------------
+ * P1: load- and memory-aware adaptive remote-build selection.
+ *
+ * All state below is populated ONCE from out-of-band, env-driven config
+ * (never from the derivation). Every helper FAILS OPEN: if config is unset,
+ * the metrics socket is unreachable/slow/malformed, or the storeUri is
+ * unknown, the helpers behave exactly like unpatched Lix
+ * (machineHasRoom -> true, liveLoadPenalty -> 0).
+ * ------------------------------------------------------------------------ */
+
+// A drv that does not match the heavy-crate table is treated as "light":
+// we have NO confident signal that it is memory-heavy, so adaptiveEstPeakRSS
+// returns nullopt and machineHasRoom never filters on account of it. (There is
+// deliberately no numeric light default - an unmatched drv must always permit,
+// so keying it on a free-RAM threshold would wrongly filter light drvs.)
+
+// name-substring -> estimated peak RSS in MiB (from LIX_ADAPTIVE_RSS_TABLE).
+static std::map<std::string, uint64_t> adaptiveRssTable;
+// machine storeUri -> "host:port" metrics endpoint (from LIX_ADAPTIVE_METRICS_MAP).
+static std::map<std::string, std::string> adaptiveMetricsMap;
+
+struct AdaptiveProbe {
+    bool ok = false;
+    uint64_t memAvailKb = 0;
+    double psiMem = 0, psiIo = 0, psiCpu = 0, load1 = 0, nproc = 0;
+};
+
+// In-process TTL cache keyed by storeUri, so selection probes each machine
+// at most once every ~2s regardless of how many drvs stream through.
+static std::map<std::string, std::pair<std::chrono::steady_clock::time_point, AdaptiveProbe>> adaptiveProbeCache;
+
+/* Parse the two env-driven config sources once. Any error leaves the tables
+ * empty, which degrades to unpatched behavior. */
+static void adaptiveLoadConfig()
+{
+    // LIX_ADAPTIVE_RSS_TABLE is a PATH to a JSON object {substring: MiB}.
+    try {
+        if (auto p = getEnv("LIX_ADAPTIVE_RSS_TABLE")) {
+            std::ifstream f(*p);
+            if (f) {
+                nlohmann::json j;
+                f >> j;
+                if (j.is_object())
+                    for (auto & [k, v] : j.items())
+                        // Per-entry guard: one bad value skips only that entry,
+                        // it does not discard the whole (otherwise valid) table.
+                        try {
+                            if (v.is_number_unsigned() || (v.is_number_integer() && v.get<int64_t>() >= 0))
+                                adaptiveRssTable[k] = v.get<uint64_t>();
+                        } catch (...) { continue; }
+            }
+        }
+    } catch (...) { adaptiveRssTable.clear(); }
+
+    // LIX_ADAPTIVE_METRICS_MAP is an inline JSON object {storeUri: "host:port"}.
+    try {
+        if (auto m = getEnv("LIX_ADAPTIVE_METRICS_MAP")) {
+            auto j = nlohmann::json::parse(*m);
+            if (j.is_object())
+                for (auto & [k, v] : j.items())
+                    // Per-entry guard: skip one bad value, keep the rest.
+                    try {
+                        if (v.is_string())
+                            adaptiveMetricsMap[k] = v.get<std::string>();
+                    } catch (...) { continue; }
+        }
+    } catch (...) { adaptiveMetricsMap.clear(); }
+}
+
+/* Estimated peak RSS (MiB) for a drv, or nullopt when the drv does not match
+ * the heavy-crate table. nullopt == "no confident heavy signal". Keyed on the
+ * store-path NAME, which is available before readDerivation and never mutates
+ * the drv. */
+static std::optional<uint64_t> adaptiveEstPeakRSS(const StorePath & drvPath)
+{
+    if (adaptiveRssTable.empty()) return std::nullopt;
+    std::string_view name = drvPath.name();
+    std::optional<uint64_t> best;
+    for (auto & [sub, mib] : adaptiveRssTable)
+        if (!sub.empty() && name.find(sub) != std::string_view::npos)
+            best = std::max(best.value_or(0), mib);
+    return best;
+}
+
+/* TCP-connect the metrics endpoint and read one line:
+ *   "MemAvail_kB psi_mem psi_io psi_cpu load1 nproc"
+ * A single ~500ms wall-clock deadline bounds the WHOLE probe (resolve +
+ * connect + read) so selection NEVER hangs, regardless of a slow or
+ * byte-dribbling peer. The endpoint MUST be a numeric IP:port - resolution is
+ * pinned to AI_NUMERICHOST|AI_NUMERICSERV so getaddrinfo never does network
+ * I/O (a hostname simply fails fast -> fail-open). Any failure returns an
+ * AdaptiveProbe with ok=false. */
+static AdaptiveProbe adaptiveProbeEndpoint(const std::string & hostport)
+{
+    AdaptiveProbe r;
+    auto colon = hostport.rfind(':');
+    if (colon == std::string::npos || colon == 0 || colon + 1 >= hostport.size())
+        return r;
+    std::string host = hostport.substr(0, colon);
+    std::string port = hostport.substr(colon + 1);
+
+    // Single wall-clock budget for the entire probe.
+    auto deadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(500);
+    auto remainingMs = [&]() -> int {
+        auto d = std::chrono::duration_cast<std::chrono::milliseconds>(
+            deadline - std::chrono::steady_clock::now()).count();
+        return d <= 0 ? 0 : (int) d;
+    };
+
+    struct addrinfo hints;
+    memset(&hints, 0, sizeof hints);
+    hints.ai_family = AF_UNSPEC;
+    hints.ai_socktype = SOCK_STREAM;
+    // Numeric-only: no DNS, no resolver blocking. Non-IP endpoint -> fail-open.
+    hints.ai_flags = AI_NUMERICHOST | AI_NUMERICSERV;
+    struct addrinfo * res = nullptr;
+    if (getaddrinfo(host.c_str(), port.c_str(), &hints, &res) != 0 || !res)
+        return r;
+
+    int fd = socket(res->ai_family, res->ai_socktype | SOCK_NONBLOCK, res->ai_protocol);
+    if (fd < 0) { freeaddrinfo(res); return r; }
+
+    int cr = connect(fd, res->ai_addr, res->ai_addrlen);
+    if (cr < 0 && errno == EINPROGRESS) {
+        struct pollfd pfd;
+        pfd.fd = fd;
+        pfd.events = POLLOUT;
+        if (poll(&pfd, 1, remainingMs()) <= 0) { close(fd); freeaddrinfo(res); return r; }
+        int soerr = 0;
+        socklen_t sl = sizeof soerr;
+        if (getsockopt(fd, SOL_SOCKET, SO_ERROR, &soerr, &sl) < 0 || soerr != 0) {
+            close(fd); freeaddrinfo(res); return r;
+        }
+    } else if (cr < 0) {
+        close(fd); freeaddrinfo(res); return r;
+    }
+    freeaddrinfo(res);
+
+    /* Read one short line. Keep the socket non-blocking and gate every recv on
+       poll(POLLIN) against the shared deadline, so the total read time is
+       bounded even if the peer drips one byte at a time. A valid reply is tiny,
+       so also cap the number of reads. */
+    std::string line;
+    char buf[512];
+    for (int iter = 0; iter < 16 && line.size() < 4096; ++iter) {
+        int rem = remainingMs();
+        if (rem == 0) break;
+        struct pollfd pfd;
+        pfd.fd = fd;
+        pfd.events = POLLIN;
+        int pr = poll(&pfd, 1, rem);
+        if (pr <= 0) break;                 // timeout or error -> fail-open
+        if (!(pfd.revents & POLLIN)) break; // POLLHUP/POLLERR with no data
+        ssize_t n = recv(fd, buf, sizeof buf, 0);
+        if (n < 0) {
+            if (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR) continue;
+            break;
+        }
+        if (n == 0) break;                  // peer closed
+        line.append(buf, n);
+        if (line.find('\n') != std::string::npos) break;
+    }
+    close(fd);
+
+    std::istringstream ss(line);
+    AdaptiveProbe tmp;
+    if (ss >> tmp.memAvailKb >> tmp.psiMem >> tmp.psiIo >> tmp.psiCpu >> tmp.load1 >> tmp.nproc) {
+        tmp.ok = true;
+        return tmp;
+    }
+    return r;
+}
+
+/* Cached probe for a machine. Unknown storeUri -> ok=false (fail-open). */
+static AdaptiveProbe adaptiveProbe(const Machine & m)
+{
+    auto now = std::chrono::steady_clock::now();
+    auto it = adaptiveProbeCache.find(m.storeUri);
+    if (it != adaptiveProbeCache.end() && now - it->second.first < std::chrono::seconds(2))
+        return it->second.second;
+
+    AdaptiveProbe r;
+    auto mit = adaptiveMetricsMap.find(m.storeUri);
+    if (mit != adaptiveMetricsMap.end())
+        r = adaptiveProbeEndpoint(mit->second);
+
+    adaptiveProbeCache[m.storeUri] = { now, r };
+    return r;
+}
+
+/* OOM guard. Returns TRUE (permit as a candidate) UNLESS we have a confident
+ * signal that the drv is heavy AND the machine's free RAM is below the drv's
+ * estimated peak RSS. No env, dead socket, or unknown machine -> permit. */
+static bool machineHasRoom(const Machine & m, const StorePath & drvPath)
+{
+    auto est = adaptiveEstPeakRSS(drvPath);
+    if (!est) return true;              // no confident heavy signal
+    auto p = adaptiveProbe(m);
+    if (!p.ok) return true;             // no live signal -> fail open
+    uint64_t freeMib = p.memAvailKb / 1024;
+    return freeMib >= *est;
+}
+
+/* Extra ranking cost from live pressure on a machine; 0 when no signal. */
+static double liveLoadPenalty(const Machine & m)
+{
+    auto p = adaptiveProbe(m);
+    if (!p.ok) return 0.0;
+    double penalty = 0.0;
+    penalty += p.psiIo / 10.0;                       // io-PSI (0..100) -> up to 10
+    if (p.nproc > 0) penalty += p.load1 / p.nproc;   // load normalized by cores
+    return penalty;
+}
+
 static int main_build_remote(AsyncIoRoot & aio, std::string programName, Strings argv)
 {
     {
@@ -108,6 +335,9 @@ static int main_build_remote(AsyncIoRoot & aio, std::string programName, Strings
         std::optional<StorePath> drvPath;
         std::string storeUri;
 
+        /* P1: parse out-of-band adaptive config once (fail-open on any error). */
+        adaptiveLoadConfig();
+
         while (true) {
 
             try {
@@ -140,14 +370,15 @@ static int main_build_remote(AsyncIoRoot & aio, std::string programName, Strings
                 bool rightType = false;
 
                 Machine * bestMachine = nullptr;
-                uint64_t bestLoad = 0;
+                double bestCost = 0;
                 for (auto & m : machines) {
                     debug("considering building on remote machine '%s'", m.storeUri);
 
                     if (m.enabled &&
                         m.systemSupported(neededSystem) &&
                         m.allSupported(requiredFeatures) &&
-                        m.mandatoryMet(requiredFeatures))
+                        m.mandatoryMet(requiredFeatures) &&
+                        machineHasRoom(m, *drvPath))
                     {
                         rightType = true;
                         AutoCloseFD free;
@@ -165,22 +396,21 @@ static int main_build_remote(AsyncIoRoot & aio, std::string programName, Strings
                         if (!free) {
                             continue;
                         }
+                        /* P1: ranking cost folds in live pressure (0 when no
+                           signal, so this reduces to load / speedFactor). */
+                        double cost = (double(load) + liveLoadPenalty(m)) / m.speedFactor;
                         bool best = false;
                         if (!bestSlotLock) {
                             best = true;
-                        } else if (load / m.speedFactor < bestLoad / bestMachine->speedFactor) {
+                        } else if (cost < bestCost) {
                             best = true;
-                        } else if (load / m.speedFactor == bestLoad / bestMachine->speedFactor) {
+                        } else if (cost == bestCost) {
                             if (m.speedFactor > bestMachine->speedFactor) {
                                 best = true;
-                            } else if (m.speedFactor == bestMachine->speedFactor) {
-                                if (load < bestLoad) {
-                                    best = true;
-                                }
                             }
                         }
                         if (best) {
-                            bestLoad = load;
+                            bestCost = cost;
                             bestSlotLock = std::move(free);
                             bestMachine = &m;
                         }

lix/legacy/nix-build.cc

@@ -187,7 +187,8 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
     if (packages && fromArgs)
         throw UsageError("'-p' and '-E' are mutually exclusive");
 
-    AutoDelete tmpDir(createTempDir("", myName));
+    AutoDelete tmpDir(createTempDir(myName));
+    AutoDelete buildTopTmpDir(createTempSubdir(tmpDir, "build-top"));
     if (outLink.empty())
         outLink = (Path) tmpDir + "/result";
 
@@ -431,7 +432,8 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
         }
 
         // Don't use defaultTempDir() here! We want to preserve the user's TMPDIR for the shell
-        env["NIX_BUILD_TOP"] = env["TMPDIR"] = env["TEMPDIR"] = env["TMP"] = env["TEMP"] = getEnvNonEmpty("TMPDIR").value_or("/tmp");
+        env["NIX_BUILD_TOP"] = env["TMPDIR"] = env["TEMPDIR"] = env["TMP"] = env["TEMP"] =
+            getEnvNonEmpty("TMPDIR").value_or(buildTopTmpDir);
         env["NIX_STORE"] = store->config().storeDir;
         env["NIX_BUILD_CORES"] = std::to_string(settings.buildCores);
 

lix/libcmd/markdown.cc

@@ -11,15 +11,34 @@ namespace nix {
 std::string renderMarkdownToTerminal(std::string_view markdown)
 {
     int windowWidth = getWindowSize().second;
+    size_t lowdown_cols = std::max(windowWidth - 5, 60);
 
-    struct lowdown_opts opts {
+    struct lowdown_opts opts{
         .type = LOWDOWN_TERM,
+#ifdef LOWDOWN_SEPARATE_TERM_OPTS
+        .term =
+            {
+                .cols = lowdown_cols,
+                .width = 0,
+                .hmargin = 0,
+                .hpadding = 4,
+                .vmargin = 0,
+                .centre = 0,
+            },
+        // maxdepth needs to be part of the ifdefs to match declaration order
         .maxdepth = 20,
-        .cols = (size_t) std::max(windowWidth - 5, 60),
+#else
+        .maxdepth = 20,
+        .cols = lowdown_cols,
         .hmargin = 0,
         .vmargin = 0,
+#endif /* LOWDOWN_SEPARATE_TERM_OPTS */
         .feat = LOWDOWN_COMMONMARK | LOWDOWN_FENCED | LOWDOWN_DEFLIST | LOWDOWN_TABLES,
+#ifdef LOWDOWN_CONSOLIDATED_OFLAGS
+        .oflags = LOWDOWN_NOLINK,
+#else
         .oflags = LOWDOWN_TERM_NOLINK,
+#endif /* LOWDOWN_CONSOLIDATED_OFLAGS */
     };
     if (!shouldANSI()) {
         opts.oflags |= LOWDOWN_TERM_NOANSI;

lix/libcmd/repl.cc

@@ -254,7 +254,7 @@ void runNix(Path program, const Strings & args)
         .program = settings.nixBinDir+ "/" + program,
         .args = args,
         .environment = subprocessEnv,
-    }).wait();
+    }).waitAndCheck();
 
     return;
 }
@@ -672,7 +672,7 @@ ProcessLineResult NixRepl::processLine(std::string line)
 
         // runProgram redirects stdout to a StringSink,
         // using runProgram2 to allow editors to display their UI
-        runProgram2(RunOptions { .program = editor, .searchPath = true, .args = args }).wait();
+        runProgram2(RunOptions { .program = editor, .searchPath = true, .args = args }).waitAndCheck();
 
         // Reload right after exiting the editor if path is not in store
         // Store is immutable, so there could be no changes, so there's no need to reload

lix/libexpr/nixexpr.hh

@@ -312,7 +312,7 @@ struct ExprAttrs
         AttrDef() { };
 
         template<typename T>
-        const T & chooseByKind(const T & plain, const T & inherited, const T & inheritedFrom) const
+        T chooseByKind(const T & plain, const T & inherited, const T & inheritedFrom) const
         {
             switch (kind) {
             case Kind::Plain:

lix/libexpr/parser/grammar.hh

@@ -686,10 +686,12 @@ struct nothing : p::nothing<Rule> {
     static_assert(!std::is_base_of_v<semantic, Rule>);
 };
 
-
-
 template<typename Self, typename OpCtx, typename AttrPathT, typename ExprT>
-struct operator_semantics {
+struct operator_semantics
+{
+private:
+    operator_semantics() = default;
+    friend Self;
 public:
     struct has_attr : grammar::v1::op::has_attr {
         AttrPathT path;
@@ -775,5 +777,4 @@ public:
         return popExpr();
     }
 };
-
 }

lix/libexpr/primops/fromTOML.cc

@@ -6,61 +6,133 @@
 
 namespace nix {
 
-void prim_fromTOML(EvalState & state, Value * * args, Value & val)
+#if HAVE_TOML11_4
+
+/**
+ * This is what toml11 < 4.0 did when choosing the subsecond precision.
+ * TOML 1.0.0 spec doesn't define how sub-millisecond ranges should be handled and calls it
+ * implementation defined behavior. For a lack of a better choice we stick with what older versions
+ * of toml11 did [1].
+ *
+ * [1]:
+ * https://github.com/ToruNiina/toml11/blob/dcfe39a783a94e8d52c885e5883a6fbb21529019/toml/datetime.hpp#L282
+ */
+static size_t normalizeSubsecondPrecision(toml::local_time lt)
 {
-    auto toml = state.forceStringNoCtx(*args[0], noPos, "while evaluating the argument passed to builtins.fromTOML");
+    auto millis = lt.millisecond;
+    auto micros = lt.microsecond;
+    auto nanos = lt.nanosecond;
+    if (millis != 0 || micros != 0 || nanos != 0) {
+        if (micros != 0 || nanos != 0) {
+            if (nanos != 0) {
+                return 9;
+            }
+            return 6;
+        }
+        return 3;
+    }
+    return 0;
+}
+
+/**
+ * Normalize date/time formats to serialize to the same strings as versions prior to toml11 4.0.
+ *
+ * Several things to consider:
+ *
+ * 1. Sub-millisecond range is represented the same way as in toml11 versions prior to 4.0.
+ * Precision is rounded towards the next multiple of 3 or capped at 9 digits.
+ * 2. Seconds must be specified. This may become optional in (yet unreleased) TOML 1.1.0, but 1.0.0
+ * defined local time in terms of RFC3339 [1].
+ * 3. date-time separator (`t`, `T` or space ` `) is canonicalized to an upper T. This is compliant
+ * with RFC3339 [1] 5.6: > Applications that generate this format SHOULD use upper case letters.
+ *
+ * [1]: https://datatracker.ietf.org/doc/html/rfc3339#section-5.6
+ */
+static void normalizeDatetimeFormat(toml::value & t)
+{
+    if (t.is_local_datetime()) {
+        auto & ldt = t.as_local_datetime();
+        t.as_local_datetime_fmt() = {
+            .delimiter = toml::datetime_delimiter_kind::upper_T,
+            // https://datatracker.ietf.org/doc/html/rfc3339#section-5.6
+            .has_seconds = true, // Mandated by TOML 1.0.0
+            .subsecond_precision = normalizeSubsecondPrecision(ldt.time),
+        };
+        return;
+    }
+
+    if (t.is_offset_datetime()) {
+        auto & odt = t.as_offset_datetime();
+        t.as_offset_datetime_fmt() = {
+            .delimiter = toml::datetime_delimiter_kind::upper_T,
+            // https://datatracker.ietf.org/doc/html/rfc3339#section-5.6
+            .has_seconds = true, // Mandated by TOML 1.0.0
+            .subsecond_precision = normalizeSubsecondPrecision(odt.time),
+        };
+        return;
+    }
+
+    if (t.is_local_time()) {
+        auto & lt = t.as_local_time();
+        t.as_local_time_fmt() = {
+            .has_seconds = true, // Mandated by TOML 1.0.0
+            .subsecond_precision = normalizeSubsecondPrecision(lt),
+        };
+        return;
+    }
+}
+
+#endif
+
+void prim_fromTOML(EvalState & state, Value ** args, Value & val)
+{
+    auto toml = state.forceStringNoCtx(
+        *args[0], noPos, "while evaluating the argument passed to builtins.fromTOML"
+    );
 
     std::istringstream tomlStream(std::string{toml});
 
-    std::function<void(Value &, toml::value)> visit;
+    auto visit = [&](this const auto & self, Value & v, toml::value t) -> void {
-
+        switch (t.type()) {
-    visit = [&](Value & v, toml::value t) {
+        case toml::value_t::table: {
-
-        switch(t.type())
-        {
-            case toml::value_t::table:
-                {
             auto table = toml::get<toml::table>(t);
+            auto attrs = state.ctx.buildBindings(table.size());
 
-                    size_t size = 0;
+            for (auto & elem : table) {
-                    for (auto & i : table) { (void) i; size++; }
+                self(attrs.alloc(elem.first), elem.second);
-
+            }
-                    auto attrs = state.ctx.buildBindings(size);
-
-                    for(auto & elem : table)
-                        visit(attrs.alloc(elem.first), elem.second);
 
             v.mkAttrs(attrs);
-                }
+        } break;
-                break;;
+        case toml::value_t::array: {
-            case toml::value_t::array:
-                {
             auto array = toml::get<std::vector<toml::value>>(t);
 
             size_t size = array.size();
             v = state.ctx.mem.newList(size);
-                    for (size_t i = 0; i < size; ++i)
+            for (size_t i = 0; i < size; ++i) {
-                        visit(*(v.listElems()[i] = state.ctx.mem.allocValue()), array[i]);
+                self(*(v.listElems()[i] = state.ctx.mem.allocValue()), array[i]);
             }
-                break;;
+        } break;
         case toml::value_t::boolean:
             v.mkBool(toml::get<bool>(t));
-                break;;
+            break;
         case toml::value_t::integer:
             v.mkInt(toml::get<int64_t>(t));
-                break;;
+            break;
         case toml::value_t::floating:
             v.mkFloat(toml::get<NixFloat>(t));
-                break;;
+            break;
         case toml::value_t::string:
             v.mkString(toml::get<std::string>(t));
-                break;;
+            break;
         case toml::value_t::local_datetime:
         case toml::value_t::offset_datetime:
         case toml::value_t::local_date:
-            case toml::value_t::local_time:
+        case toml::value_t::local_time: {
-                {
             if (experimentalFeatureSettings.isEnabled(Xp::ParseTomlTimestamps)) {
+#if HAVE_TOML11_4
+                normalizeDatetimeFormat(t);
+#endif
                 auto attrs = state.ctx.buildBindings(2);
                 attrs.alloc("_type").mkString("timestamp");
                 std::ostringstream s;
@@ -71,17 +143,27 @@ void prim_fromTOML(EvalState & state, Value * * args, Value & val)
                 // NOLINTNEXTLINE(lix-foreign-exceptions)
                 throw std::runtime_error("Dates and times are not supported");
             }
-                }
+        } break;
-                break;;
         case toml::value_t::empty:
             v.mkNull();
-                break;;
+            break;
-
         }
     };
 
     try {
-        visit(val, toml::parse(tomlStream, "fromTOML" /* the "filename" */));
+        visit(
+            val,
+            toml::parse(
+                tomlStream,
+                "fromTOML" /* the "filename" */
+#if HAVE_TOML11_4
+                ,
+                toml::spec::v(
+                    1, 0, 0
+                ) // Be explicit that we are parsing TOML 1.0.0 without extensions
+#endif
+            )
+        );
     } catch (std::exception & e) { // NOLINT(lix-foreign-exceptions) // TODO: toml::syntax_error
         state.ctx.errors.make<EvalError>("while parsing TOML: %s", e.what()).debugThrow();
     }

lix/libfetchers/git.cc

@@ -22,6 +22,7 @@
 #include <string.h>
 #include <sys/time.h>
 #include <sys/wait.h>
+#include <unistd.h>
 
 using namespace std::string_literals;
 
@@ -168,7 +169,7 @@ WorkdirInfo getWorkdirInfo(const Input & input, const Path & workdir)
         .program = "git",
         .args = { "-C", workdir, "--git-dir", gitDir, "rev-parse", "--verify", "--no-revs", "HEAD^{commit}" },
         .environment = env,
-        .mergeStderrToStdout = true
+        .redirections = {{.from = STDERR_FILENO, .to = STDOUT_FILENO}},
     });
     auto exitCode = WEXITSTATUS(result.first);
     auto errorMessage = result.second;
@@ -701,7 +702,7 @@ struct GitInputScheme : InputScheme
         auto result = runProgram(RunOptions {
             .program = "git",
             .args = { "-C", repoDir, "--git-dir", gitDir, "cat-file", "commit", input.getRev()->gitRev() },
-            .mergeStderrToStdout = true
+            .redirections = {{.from = STDERR_FILENO, .to = STDOUT_FILENO}},
         });
         if (WEXITSTATUS(result.first) == 128
             && result.second.find("bad file") != std::string::npos)
@@ -766,7 +767,7 @@ struct GitInputScheme : InputScheme
                 .args = { "-C", repoDir, "--git-dir", gitDir, "archive", input.getRev()->gitRev() },
                 .captureStdout = true,
             });
-            Finally const _wait([&] { proc.wait(); });
+            Finally const _wait([&] { proc.waitAndCheck(); });
 
             unpackTarfile(*proc.getStdout(), tmpDir);
         }

lix/libstore/binary-cache-store.cc

@@ -190,6 +190,7 @@ try {
 
         auto * buildIdDir = std::get_if<nar_index::Directory>(&narIndex);
         for (auto subdir : { "lib", "debug", ".build-id" }) {
+            if (!buildIdDir) break;
             // get returns nullptr subdir does not exist, and std::get_if propagates it.
             buildIdDir = std::get_if<nar_index::Directory>(get(buildIdDir->contents, subdir));
         }

lix/libstore/build/derivation-goal.cc

@@ -885,11 +885,7 @@ void replaceValidPath(const Path & storePath, const Path & tmpPath)
        we're repairing (say) Glibc, we end up with a broken system. */
     Path oldPath;
     if (pathExists(storePath)) {
-        do {
+        oldPath = makeTempSiblingPath(storePath);
-            oldPath = makeTempPath(storePath, ".old");
-            // store paths are often directories so we can't just unlink() it
-            // let's make sure the path doesn't exist before we try to use it
-        } while (pathExists(oldPath));
         movePath(storePath, oldPath);
     }
 
@@ -1011,11 +1007,11 @@ void runPostBuildHook(
         .program = settings.postBuildHook,
         .environment = hookEnvironment,
         .captureStdout = true,
-        .mergeStderrToStdout = true,
+        .redirections = {{.from = STDERR_FILENO, .to = STDOUT_FILENO}},
     });
     Finally const _wait([&] {
         try {
-            proc.wait();
+            proc.waitAndCheck();
         } catch (nix::Error & e) {
             e.addTrace(nullptr,
                 "while running the post-build-hook %s for derivation %s",

lix/libstore/build/local-derivation-goal.cc

@@ -13,6 +13,8 @@
 #include "lix/libutil/archive.hh"
 #include "lix/libstore/daemon.hh"
 #include "lix/libutil/regex.hh"
+#include "lix/libutil/file-descriptor.hh"
+#include "lix/libutil/file-system.hh"
 #include "lix/libutil/result.hh"
 #include "lix/libutil/topo-sort.hh"
 #include "lix/libutil/json.hh"
@@ -25,13 +27,16 @@
 #include "lix/libutil/mount.hh"
 #include "lix/libutil/strings.hh"
 #include "lix/libutil/thread-name.hh"
+#include "platform/linux.hh"
 
 #include <cstddef>
+#include <dirent.h>
 #include <exception>
 #include <regex>
 #include <queue>
 
 #include <stdexcept>
+#include <sys/stat.h>
 #include <sys/un.h>
 #include <fcntl.h>
 #include <termios.h>
@@ -110,7 +115,11 @@ LocalDerivationGoal::~LocalDerivationGoal() noexcept(false)
     /* Careful: we should never ever throw an exception from a
        destructor. */
     try { killChild(); } catch (...) { ignoreExceptionInDestructor(); }
-    try { deleteTmpDir(false, true); } catch (...) { ignoreExceptionInDestructor(); }
+    try {
+        finalizeTmpDir(false, true);
+    } catch (...) {
+        ignoreExceptionInDestructor();
+    }
 }
 
 
@@ -360,13 +369,14 @@ bool LocalDerivationGoal::cleanupDecideWhetherDiskFull()
         if (statvfs(localStore.config().realStoreDir.get().c_str(), &st) == 0 &&
             (uint64_t) st.f_bavail * st.f_bsize < required)
             diskFull = true;
-        if (statvfs(tmpDir.c_str(), &st) == 0 &&
+        if (statvfs(tmpDirRoot.c_str(), &st) == 0 && (uint64_t) st.f_bavail * st.f_bsize < required)
-            (uint64_t) st.f_bavail * st.f_bsize < required)
+        {
             diskFull = true;
         }
+    }
 #endif
 
-    deleteTmpDir(false);
+    finalizeTmpDir(false);
 
     /* Move paths out of the chroot for easier debugging of
        build failures. */
@@ -385,7 +395,7 @@ bool LocalDerivationGoal::cleanupDecideWhetherDiskFull()
 
 void LocalDerivationGoal::cleanupPostOutputsRegisteredModeCheck()
 {
-    deleteTmpDir(true);
+    finalizeTmpDir(true);
 }
 
 
@@ -480,17 +490,77 @@ try {
         });
     }
 
+    try {
+        auto buildDir = worker.buildDirOverride.value_or(settings.buildDir.get());
+
+        createDirs(buildDir);
+
         /* Create a temporary directory where the build will take
            place. */
-    tmpDir = createTempDir(
+        tmpDirRoot = createTempSubdir(buildDir, std::nullopt, 0700);
-        settings.buildDir.get().value_or(""),
+    } catch (SysError & e) {
-        "nix-build-" + std::string(drvPath.name()),
+        /*
-        false,
+         * Fallback to the global tmpdir and create a safe space there
-        false,
+         * only if it's a permission error.
-        0700
+         */
-    );
+        if (e.errNo != EACCES) {
+            throw;
+        }
 
-    chownToBuilder(tmpDir);
+        auto nixBuildsTmp = createTempDir(fmt("nix-builds-%s", geteuid()), 0700);
+        warn(
+            "Failed to use the system-wide build directory '%s', falling back to a temporary "
+            "directory inside '%s'",
+            settings.buildDir.get(),
+            nixBuildsTmp
+        );
+        tmpDirRoot = createTempSubdir(nixBuildsTmp, std::nullopt, 0700);
+        worker.buildDirOverride = nixBuildsTmp;
+    }
+    /* The TOCTOU between the previous mkdir call and this open call is unavoidable due to
+     * POSIX semantics.*/
+    tmpDirRootFd = AutoCloseFD{open(tmpDirRoot.c_str(), O_RDONLY | O_NOFOLLOW | O_DIRECTORY)};
+    if (!tmpDirRootFd) {
+        throw SysError("failed to open the build temporary directory descriptor '%1%'", tmpDirRoot);
+    }
+
+#if __APPLE__
+    // The Darwin sandbox ensures that builds cannot change the
+    // permissions of their own build directory. Unsandboxed builds
+    // disable this, but have no isolation by design anyway. The
+    // minimal sandbox (applied even when `sandbox = false`, though not
+    // when `_NIX_TEST_NO_SANDBOX` is set) prevents the creation of
+    // `set{u,g}id` files regardless.
+    tmpDir = tmpDirRoot;
+    tmpDirFd = std::move(tmpDirRootFd);
+#else
+    // place the actual build directory in a subdirectory of tmpDirRoot. if
+    // we do not do this a build can `chown 777` its build directory and so
+    // make it accessible to everyone in the system, breaking isolation. we
+    // also need the intermediate level to be inaccessible to others. build
+    // processes must be able to at least traverse to the directory though,
+    // without being able to chmod. this means either mode 0750 or 0710. we
+    // use 0710 just to be extra safe; if we ever add more directories they
+    // will not be enumerable to other processes in the builder user group.
+    tmpDir = tmpDirRoot + "/b";
+    if (mkdirat(tmpDirRootFd.get(), "b", 0700)) {
+        throw SysError("failed to create the build temporary directory '%1%'", tmpDir);
+    }
+    tmpDirFd = AutoCloseFD{openat(tmpDirRootFd.get(), "b", O_RDONLY | O_NOFOLLOW | O_DIRECTORY)};
+    if (!tmpDirFd)
+        throw SysError("failed to open the build temporary directory descriptor '%1%'", tmpDir);
+
+    if (buildUser) {
+        if (fchown(tmpDirRootFd.get(), -1, buildUser->getGID()) == -1) {
+            throw SysError("cannot change ownership of '%1%'", tmpDirRoot);
+        }
+        if (fchmod(tmpDirRootFd.get(), 0710) == -1) {
+            throw SysError("cannot change mode of '%1%'", tmpDirRoot);
+        }
+    }
+#endif
+
+    chownToBuilder(tmpDirFd);
 
     for (auto & [outputName, status] : initialOutputs) {
         /* Set scratch path we'll actually use during the build.
@@ -858,8 +928,13 @@ void LocalDerivationGoal::initTmpDir() {
                 auto hash = hashString(HashType::SHA256, i.first);
                 std::string fn = ".attr-" + hash.to_string(Base::Base32, false);
                 Path p = tmpDir + "/" + fn;
-                writeFile(p, rewriteStrings(i.second, inputRewrites));
+                /* TODO(jade): we should have BorrowedFD instead of OwnedFD. */
-                chownToBuilder(p);
+                AutoCloseFD passAsFileFd{openat(tmpDirFd.get(), fn.c_str(), O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC | O_EXCL | O_NOFOLLOW, 0666)};
+                if (!passAsFileFd) {
+                    throw SysError("opening `passAsFile` file in the sandbox '%1%'", p);
+                }
+                writeFile(passAsFileFd, rewriteStrings(i.second, inputRewrites));
+                chownToBuilder(passAsFileFd);
                 env[i.first + "Path"] = tmpDirInSandbox + "/" + fn;
             }
         }
@@ -975,6 +1050,13 @@ void LocalDerivationGoal::chownToBuilder(const Path & path)
         throw SysError("cannot change ownership of '%1%'", path);
 }
 
+void LocalDerivationGoal::chownToBuilder(const AutoCloseFD & fd)
+{
+    if (!buildUser) return;
+    if (fchown(fd.get(), buildUser->getUID(), buildUser->getGID()) == -1)
+        throw SysError("cannot change ownership of file '%1%'", fd.guessOrInventPath());
+}
+
 
 void LocalDerivationGoal::runChild()
 {
@@ -1105,7 +1187,7 @@ void LocalDerivationGoal::runChild()
                 /* N.B. it is realistic that these paths might not exist. It
                    happens when testing Nix building fixed-output derivations
                    within a pure derivation. */
-                for (auto & path : { "/etc/resolv.conf", "/etc/services", "/etc/hosts" })
+                for (auto & path : { "/etc/services", "/etc/hosts" })
                     if (pathExists(path)) {
                         // Copy the actual file, not the symlink, because we don't know where
                         // the symlink is pointing, and we don't want to chase down the entire
@@ -1126,6 +1208,11 @@ void LocalDerivationGoal::runChild()
                         copyFile(path, chrootRootDir + path, { .followSymlinks = true });
                     }
 
+                if (pathExists("/etc/resolv.conf")) {
+                    const auto resolvConf = rewriteResolvConf(readFile("/etc/resolv.conf"));
+                    writeFile(chrootRootDir + "/etc/resolv.conf", resolvConf);
+                }
+
                 if (settings.caFile != "" && pathExists(settings.caFile)) {
                     // For the same reasons as above, copy the CA certificates file too.
                     // It should be even less likely to change during the build than resolv.conf.
@@ -1253,6 +1340,36 @@ void LocalDerivationGoal::runChild()
             if (setuid(sandboxUid()) == -1)
                 throw SysError("setuid failed");
 
+            if (runPasta) {
+                // wait for the pasta interface to appear. pasta can't signal us when
+                // it's done setting up the namespace, so we have to wait for a while
+                AutoCloseFD fd(socket(PF_INET, SOCK_DGRAM, IPPROTO_IP));
+                if (!fd) throw SysError("cannot open IP socket");
+
+                struct ifreq ifr;
+                strcpy(ifr.ifr_name, LinuxLocalDerivationGoal::PASTA_NS_IFNAME);
+                // wait two minutes for the interface to appear. if it does not do so
+                // we are either grossly overloaded, or pasta startup failed somehow.
+                static constexpr int SINGLE_WAIT_US = 1000;
+                static constexpr int TOTAL_WAIT_US = 120'000'000;
+                for (unsigned tries = 0; ; tries++) {
+                    if (tries > TOTAL_WAIT_US / SINGLE_WAIT_US) {
+                        throw Error(
+                            "sandbox network setup timed out, please check daemon logs for "
+                            "possible error output."
+                        );
+                    } else if (ioctl(fd.get(), SIOCGIFFLAGS, &ifr) == 0) {
+                        if ((ifr.ifr_ifru.ifru_flags & IFF_UP) != 0) {
+                            break;
+                        }
+                    } else if (errno == ENODEV) {
+                        usleep(SINGLE_WAIT_US);
+                    } else {
+                        throw SysError("cannot get loopback interface flags");
+                    }
+                }
+            }
+
             setUser = false;
         }
 #endif
@@ -1424,6 +1541,8 @@ void LocalDerivationGoal::runChild()
 
         if (getEnv("_NIX_TEST_NO_SANDBOX") != "1") {
             Strings sandboxArgs;
+            sandboxArgs.push_back("_NIX_BUILD_TOP");
+            sandboxArgs.push_back(tmpDir);
             sandboxArgs.push_back("_GLOBAL_TMP_DIR");
             sandboxArgs.push_back(globalTmpDir);
             if (allowLocalNetworking) {
@@ -2297,21 +2416,78 @@ try {
     co_return result::current_exception();
 }
 
-
+// make `entry` in `parentFd` visible to the given user and group, preserving
-void LocalDerivationGoal::deleteTmpDir(bool force, bool duringDestruction)
+// inode modes as much as possible. if the builder sets the mode of any inode
+// to not be readable by the owner we keep this; not doing so could interfere
+// with error analysis. if the builder used multiple uids or gids we will not
+// keep them around and instead collapse them all onto the uid/gid given here
+// to not leave around inodes owned by unassigned uids/gids in the system. we
+// also clear setuid/setgid/sticky bits just to be safe even though a builder
+// should not be able to set them to begin, otherwise we may leave setuid/gid
+// executables in the tree even with user/group set to -1/-1. there have been
+// enough bugs of this kind in the past to warrant some extra attention here.
+static void makeVisible(int parentFd, const char * entry, uid_t user, gid_t group)
 {
-    if (tmpDir != "") {
+    struct stat st;
+    if (fstatat(parentFd, entry, &st, AT_SYMLINK_NOFOLLOW)) {
+        throw SysError("fstat(%s)", guessOrInventPathFromFD(parentFd));
+    }
+    if (S_ISDIR(st.st_mode)) {
+        int dirfd = openat(parentFd, entry, O_RDONLY | O_DIRECTORY | O_NOFOLLOW);
+        if (dirfd < 0) {
+            throw SysError("openat(%s/%s)", guessOrInventPathFromFD(parentFd), entry);
+        }
+        AutoCloseDir dir(fdopendir(dirfd));
+        if (!dir) {
+            close(dirfd);
+            throw SysError("fdopendir(%s/%s)", guessOrInventPathFromFD(parentFd), entry);
+        }
+
+        struct dirent * dirent;
+        while (errno = 0, dirent = readdir(dir.get())) {
+            if (strcmp(dirent->d_name, ".") == 0 || strcmp(dirent->d_name, "..") == 0) {
+                continue;
+            }
+            makeVisible(dirfd, dirent->d_name, user, group);
+        }
+    }
+
+    // ignore permissions errors for symlinks. linux can't chmod them.
+    // clear special permission bits while we're here, just to be safe
+    if (fchmodat(parentFd, entry, st.st_mode & 0777, AT_SYMLINK_NOFOLLOW) && !S_ISLNK(st.st_mode)) {
+        throw SysError("fchmod(%s)", guessOrInventPathFromFD(parentFd));
+    }
+    if (user != uid_t(-1) && group != gid_t(-1)
+        && fchownat(parentFd, entry, user, group, AT_SYMLINK_NOFOLLOW))
+    {
+        throw SysError("fchown(%s)", guessOrInventPathFromFD(parentFd));
+    }
+}
+
+void LocalDerivationGoal::finalizeTmpDir(bool force, bool duringDestruction)
+{
+    if (tmpDirRoot != "") {
         /* Don't keep temporary directories for builtins because they
            might have privileged stuff (like a copy of netrc). */
         if (settings.keepFailed && !force && !drv->isBuiltin()) {
-            printError("note: keeping build directory '%s'", tmpDir);
+            printError("note: keeping build directory '%s'", tmpDirRoot);
-            chmod(tmpDir.c_str(), 0755);
+            try {
+                // always make visible, but don't always chown. if we run as
+                // root we may not want to chown things to root:root so much
+                auto creds = worker.store.associatedCredentials();
+                makeVisible(
+                    tmpDirFd.get(), ".", creds ? creds->user : -1, creds ? creds->group : -1
+                );
+            } catch (SysError & e) {
+                printError("error making '%s' accessible: %s", tmpDir, e.what());
+            }
+            chmod(tmpDirRoot.c_str(), 0755);
         }
         else if (duringDestruction)
-            deletePathUninterruptible(tmpDir);
+            deletePathUninterruptible(tmpDirRoot);
         else
-            deletePath(tmpDir);
+            deletePath(tmpDirRoot);
-        tmpDir = "";
+        tmpDirRoot = "";
     }
 }
 

lix/libstore/build/local-derivation-goal.hh

@@ -29,7 +29,12 @@ struct LocalDerivationGoal : public DerivationGoal
     /**
      * The temporary directory.
      */
-    Path tmpDir;
+    Path tmpDirRoot, tmpDir;
+
+    /**
+     * The temporary directory file descriptor
+     */
+    AutoCloseFD tmpDirRootFd, tmpDirFd;
 
     /**
      * The path of the temporary directory in the sandbox.
@@ -191,10 +196,18 @@ struct LocalDerivationGoal : public DerivationGoal
     kj::Promise<Result<void>> writeStructuredAttrs();
 
     /**
-     * Make a file owned by the builder.
+     * Make a file owned by the builder addressed by its path.
+     *
+     * SAFETY: this function is prone to TOCTOU as it receives a path and not a descriptor.
+     * It's only safe to call in a child of a directory only visible to the owner.
      */
     void chownToBuilder(const Path & path);
 
+    /**
+     * Make a file owned by the builder addressed by its file descriptor.
+     */
+    void chownToBuilder(const AutoCloseFD & fd);
+
     int getChildStatus() override;
 
     /**
@@ -233,9 +246,12 @@ struct LocalDerivationGoal : public DerivationGoal
     void cleanupPostOutputsRegisteredModeNonCheck() override;
 
     /**
-     * Delete the temporary directory, if we have one.
+     * Delete the temporary directory or make it visible to the user requesting
+     * this build, if a temporary directory was created at all. Temporary files
+     * of derivations using builtin builders are deleted even for `keep-failed`
+     * builds as otherwise we may expose secrets (e.g. from the system .netrc).
      */
-    void deleteTmpDir(bool force, bool duringDestruction = false);
+    void finalizeTmpDir(bool force, bool duringDestruction = false);
 
     /**
      * Forcibly kill the child process, if any.
@@ -269,6 +285,12 @@ struct LocalDerivationGoal : public DerivationGoal
 protected:
     using DerivationGoal::DerivationGoal;
 
+    /**
+     * Whether to run pasta for network-endowed derivations. Running pasta
+     * currently requires actively waiting for its net-ns setup to finish.
+     */
+    bool runPasta = false;
+
     /**
      * Setup dependencies outside the sandbox.
      * Called in the parent nix process.
@@ -278,6 +300,15 @@ protected:
         throw Error("sandboxing builds is not supported on this platform");
     };
 
+    /**
+     * Rewrite resolv.conf for use in the sandbox. Used in the linux platform
+     * to replace nameservers * when using pasta for fixed output derivations.
+     */
+    virtual std::string rewriteResolvConf(std::string fromHost)
+    {
+        return fromHost;
+    }
+
     /**
      * Create a new process that runs `openSlave` and `runChild`
      * On some platforms this process is created with sandboxing flags.

lix/libstore/build/sandbox-defaults.sb

@@ -26,12 +26,28 @@ R""(
 ; Allow getpwuid.
 (allow mach-lookup (global-name "com.apple.system.opendirectoryd.libinfo"))
 
-; Access to /tmp.
+; Disallow messing with the top‐level build directory.
+(deny file-write-owner file-write-flags file-write-xattr file-write-mount
+      file-write-unmount
+      (literal (param "_NIX_BUILD_TOP")))
+; Nixpkgs does `chmod -R` on `$NIX_BUILD_TOP/$sourceRoot` by default,
+; which results in it trying to set the mode of `$NIX_BUILD_TOP` when
+; derivations set `sourceRoot = ".";`. Thankfully, the GNU `chmod(1)`
+; treats `ENOTSUP` as a non‐fatal, non‐reported error in this case, and
+; continues to descend into the directory tree.
+;
+; See: <https://gitweb.git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f=src/chmod.c;hb=refs/tags/v9.7#l312>
+(deny file-write-mode (with errno ENOTSUP)
+      (literal (param "_NIX_BUILD_TOP")))
+
+; Access to /tmp and the build directory.
 ; The network-outbound/network-inbound ones are for unix domain sockets, which
 ; we allow access to in TMPDIR (but if we allow them more broadly, you could in
 ; theory escape the sandbox)
 (allow file* process-exec network-outbound network-inbound
-       (literal "/tmp") (subpath TMPDIR))
+       (literal "/tmp")
+       (subpath TMPDIR)
+       (subpath (param "_NIX_BUILD_TOP")))
 
 ; Some packages like to read the system version.
 (allow file-read*

lix/libstore/build/worker.hh

@@ -195,6 +195,7 @@ public:
     Store & store;
     Store & evalStore;
     AsyncSemaphore substitutions, localBuilds;
+    std::optional<Path> buildDirOverride;
 
 private:
     kj::TaskSet children;

lix/libstore/crypto.cc

@@ -45,8 +45,15 @@ std::string SecretKey::signDetached(std::string_view data) const
 {
     unsigned char sig[crypto_sign_BYTES];
     unsigned long long sigLen;
-    crypto_sign_detached(sig, &sigLen, charptr_cast<const unsigned char *>(data.data()), data.size(),
+    crypto_sign_detached(
-        charptr_cast<const unsigned char *>(key.data()));
+        sig,
+        &sigLen,
+        // the following is not a string function so no null termination issues are possible here.
+        // NOLINTNEXTLINE(bugprone-suspicious-stringview-data-usage)
+        charptr_cast<const unsigned char *>(data.data()),
+        data.size(),
+        charptr_cast<const unsigned char *>(key.data())
+    );
     return name + ":" + base64Encode(std::string(reinterpret_cast<char *>(sig), sigLen));
 }
 

lix/libstore/gc.cc

@@ -96,7 +96,7 @@ void LocalStore::createTempRootsFile()
 
     /* Create the temporary roots file for this process. */
     while (true) {
-        auto tmp = makeTempPath(fnTempRoots, ".tmp");
+        auto tmp = makeTempPath(fnTempRoots);
         AutoCloseFD fd{open(tmp.c_str(), O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC, 0600)};
         if (!fd && errno != EEXIST) {
             throw SysError("opening lock file '%1%'", tmp);
@@ -966,9 +966,9 @@ try {
 
     {
         auto state(_gcState.lock());
+        state->gcWaiters.push_back(std::move(pfp.fulfiller));
 
         if (state->gcRunning) {
-            state->gcWaiters.push_back(std::move(pfp.fulfiller));
             debug("waiting for auto-GC to finish");
             goto sync;
         }

lix/libstore/globals.cc

@@ -87,6 +87,9 @@ Settings::Settings()
 #if defined(__linux__) && defined(SANDBOX_SHELL)
     sandboxPaths.setDefault(tokenizeString<StringSet>("/bin/sh=" SANDBOX_SHELL));
 #endif
+#if defined(__linux__) && defined(PASTA_PATH)
+    pastaPath.setDefault(PASTA_PATH);
+#endif
 
     /* chroot-like behavior from Apple's sandbox */
 #if __APPLE__
@@ -243,7 +246,11 @@ StringSet Settings::getDefaultExtraPlatforms()
     // x86_64 in aarch64 environments or vice versa since they can
     // always exec with their own binary preferences.
     if (std::string{SYSTEM} == "aarch64-darwin" &&
-        runProgram(RunOptions {.program = "arch", .args = {"-arch", "x86_64", "/usr/bin/true"}, .mergeStderrToStdout = true}).first == 0)
+        runProgram(RunOptions {
+            .program = "arch",
+            .args = {"-arch", "x86_64", "/usr/bin/true"},
+            .redirections = {{.from = STDERR_FILENO, .to = STDOUT_FILENO}}
+        }).first == 0)
         extraPlatforms.insert("x86_64-darwin");
 #endif
 

lix/libstore/local-binary-cache-store.cc

@@ -63,11 +63,10 @@ protected:
         const std::string & mimeType) override
     {
         auto path2 = binaryCacheDir + "/" + path;
-        static std::atomic<int> counter{0};
+        Path tmp = makeTempPath(path2);
-        Path tmp = fmt("%s.tmp.%d.%d", path2, getpid(), ++counter);
         AutoDelete del(tmp, false);
         StreamToSourceAdapter source(istream);
-        writeFile(tmp, source);
+        writeFileExcl(tmp, source);
         renameFile(tmp, path2);
         del.cancel();
     }

lix/libstore/local-store.cc

@@ -173,7 +173,7 @@ static void migrateCASchema(SQLite& db, Path schemaPath, AutoCloseFD& lockFd, Ne
             txn.commit();
         }
 
-        writeFile(schemaPath, fmt("%d", nixCASchemaVersion), 0666, true);
+        writeFileAndSync(schemaPath, fmt("%d", nixCASchemaVersion), 0666);
         lockFile(lockFd.get(), ltRead);
     }
 }
@@ -315,7 +315,7 @@ void LocalStore::initDB(DBState & state)
     else if (curSchema == 0) { /* new store */
         curSchema = nixSchemaVersion;
         openDB(state, true);
-        writeFile(schemaPath, fmt("%1%", nixSchemaVersion), 0666, true);
+        writeFileAndSync(schemaPath, fmt("%1%", nixSchemaVersion), 0666);
     }
 
     else if (curSchema < nixSchemaVersion) {
@@ -368,7 +368,7 @@ void LocalStore::initDB(DBState & state)
             txn.commit();
         }
 
-        writeFile(schemaPath, fmt("%1%", nixSchemaVersion), 0666, true);
+        writeFileAndSync(schemaPath, fmt("%1%", nixSchemaVersion), 0666);
 
         lockFile(globalLock.get(), ltRead, always_progresses);
     }
@@ -1629,7 +1629,7 @@ std::pair<Path, AutoCloseFD> LocalStore::createTempDirInStore()
         /* There is a slight possibility that `tmpDir' gets deleted by
            the GC between createTempDir() and when we acquire a lock on it.
            We'll repeat until 'tmpDir' exists and we've locked it. */
-        tmpDirFn = createTempDir(config_.realStoreDir, "tmp");
+        tmpDirFn = createTempSubdir(config_.realStoreDir, "tmp");
         tmpDirFd = AutoCloseFD{open(tmpDirFn.c_str(), O_RDONLY | O_DIRECTORY)};
         if (tmpDirFd.get() < 0) {
             continue;
@@ -2043,9 +2043,9 @@ try {
 
     createDirs(dirOf(logPath));
 
-    auto tmpFile = fmt("%s.tmp.%d", logPath, getpid());
+    auto tmpFile = makeTempSiblingPath(logPath);
 
-    writeFile(tmpFile, compress("bzip2", log));
+    writeFileExcl(tmpFile, compress("bzip2", log));
 
     renameFile(tmpFile, logPath);
     co_return result::success();

lix/libstore/local-store.hh

@@ -135,6 +135,8 @@ private:
 
     Sync<GCState> _gcState;
 
+    std::optional<AssociatedCredentials> association;
+
 public:
 
     const Path dbDir;
@@ -148,6 +150,16 @@ public:
     LocalStoreConfig & config() override { return config_; }
     const LocalStoreConfig & config() const override { return config_; }
 
+    std::optional<AssociatedCredentials> associatedCredentials() const override
+    {
+        return association;
+    }
+
+    void associateWithCredentials(uid_t user, gid_t group)
+    {
+        association = {user, group};
+    }
+
 private:
 
     const PublicKeys & getPublicKeys();

lix/libstore/meson.build

@@ -82,6 +82,7 @@ libstore_setting_definitions = files(
   'settings/narinfo-cache-negative-ttl.md',
   'settings/narinfo-cache-positive-ttl.md',
   'settings/netrc-file.md',
+  'settings/pasta-path.md',
   'settings/plugin-files.md',
   'settings/post-build-hook.md',
   'settings/pre-build-hook.md',
@@ -332,6 +333,12 @@ elif busybox.found()
   }
 endif
 
+if pasta.found()
+  cpp_str_defines += {
+    'PASTA_PATH': pasta.full_path(),
+  }
+endif
+
 cpp_args = []
 
 foreach name, value : cpp_str_defines

lix/libstore/optimise-store.cc

@@ -217,8 +217,7 @@ void LocalStore::optimisePath_(Activity * act, OptimiseStats & stats,
        its timestamp back to 0. */
     MakeReadOnly makeReadOnly(mustToggle ? dirOfPath : "");
 
-    Path tempLink = makeTempPath(config().realStoreDir, "/.tmp-link");
+    Path tempLink = makeTempPath(config().realStoreDir + "/");
-    unlink(tempLink.c_str()); // just in case; ignore errors
 
     if (link(linkPath.c_str(), tempLink.c_str()) == -1) {
         if (errno == EMLINK) {

lix/libstore/platform/linux.cc

@@ -1,16 +1,25 @@
 #include "lix/libstore/build/worker.hh"
 #include "lix/libutil/cgroup.hh"
+#include "lix/libutil/file-descriptor.hh"
+#include "lix/libutil/file-system.hh"
 #include "lix/libutil/finally.hh"
 #include "lix/libstore/gc-store.hh"
+#include "lix/libutil/processes.hh"
 #include "lix/libutil/signals.hh"
 #include "lix/libstore/platform/linux.hh"
 #include "lix/libutil/regex.hh"
 #include "lix/libutil/strings.hh"
 
+#include <csignal>
+#include <cstdlib>
 #include <grp.h>
 #include <regex>
 #include <sys/prctl.h>
 
+#if __linux__
+#include <linux/capability.h>
+#endif
+
 #if HAVE_SECCOMP
 #include <linux/filter.h>
 #include <sys/syscall.h>
@@ -61,6 +70,14 @@ static void readFileRoots(const char * path, UncheckedRoots & roots)
     }
 }
 
+LinuxLocalDerivationGoal::~LinuxLocalDerivationGoal()
+{
+    // pasta being left around mostly happens when builds are aborted
+    if (pastaPid) {
+        pastaPid.kill();
+    }
+}
+
 void LinuxLocalStore::findPlatformRoots(UncheckedRoots & unchecked)
 {
     auto procDir = AutoCloseDir{opendir("/proc")};
@@ -836,6 +853,26 @@ void LinuxLocalDerivationGoal::prepareSandbox()
     }
 }
 
+std::string LinuxLocalDerivationGoal::rewriteResolvConf(std::string fromHost)
+{
+    if (!runPasta) {
+        return fromHost;
+    }
+
+    static constexpr auto flags = std::regex::ECMAScript | std::regex::multiline;
+    static auto lineRegex = regex::parse("^nameserver\\s.*$", flags);
+    static auto v4Regex = regex::parse("^nameserver\\s+\\d{1,3}\\.", flags);
+    static auto v6Regex = regex::parse("^nameserver.*:", flags);
+    std::string nsInSandbox = "\n";
+    if (std::regex_search(fromHost, v4Regex)) {
+        nsInSandbox += fmt("nameserver %s\n", PASTA_HOST_IPV4);
+    }
+    if (std::regex_search(fromHost, v6Regex)) {
+        nsInSandbox += fmt("nameserver %s\n", PASTA_HOST_IPV6);
+    }
+    return std::regex_replace(fromHost, lineRegex, "") + nsInSandbox;
+}
+
 Pid LinuxLocalDerivationGoal::startChild(std::function<void()> openSlave)
 {
 #if HAVE_SECCOMP
@@ -863,9 +900,11 @@ Pid LinuxLocalDerivationGoal::startChild(std::function<void()> openSlave)
 
        - The private network namespace ensures that the builder
          cannot talk to the outside world (or vice versa).  It
-         only has a private loopback interface. (Fixed-output
+         only has a private loopback interface. If a copy of
-         derivations are not run in a private network namespace
+         `pasta` is available, Fixed-output derivations are run
-         to allow functions like fetchurl to work.)
+         inside a private network namespace with internet
+         access, otherwise they are run in the host's network
+         namespace, to allow functions like fetchurl to work.
 
        - The IPC namespace prevents the builder from communicating
          with outside processes using SysV IPC mechanisms (shared
@@ -886,6 +925,10 @@ Pid LinuxLocalDerivationGoal::startChild(std::function<void()> openSlave)
     if (derivationType->isSandboxed())
         privateNetwork = true;
 
+    // don't launch pasta unless we have a tun device. in a build sandbox we
+    // commonly do not, and trying to run pasta anyway naturally won't work.
+    runPasta = !privateNetwork && settings.pastaPath != "" && pathExists("/dev/net/tun");
+
     userNamespaceSync.create();
 
     Pipe sendPid;
@@ -910,7 +953,9 @@ Pid LinuxLocalDerivationGoal::startChild(std::function<void()> openSlave)
 
         ProcessOptions options;
         options.cloneFlags = CLONE_NEWPID | CLONE_NEWNS | CLONE_NEWIPC | CLONE_NEWUTS | CLONE_PARENT | SIGCHLD;
-        if (privateNetwork)
+        // we always want to create a new network namespace for pasta, even when
+        // we can't actually run it. not doing so hides bugs and impairs purity.
+        if (settings.pastaPath != "" || privateNetwork)
             options.cloneFlags |= CLONE_NEWNET;
         if (usingUserNamespace)
             options.cloneFlags |= CLONE_NEWUSER;
@@ -980,6 +1025,67 @@ Pid LinuxLocalDerivationGoal::startChild(std::function<void()> openSlave)
     /* Signal the builder that we've updated its user namespace. */
     writeFull(userNamespaceSync.writeSide.get(), "1");
 
+    if (runPasta) {
+        // Bring up pasta, for handling FOD networking. We don't let it daemonize
+        // itself for process managements reasons and kill it manually when done.
+
+        // TODO add a new sandbox mode flag to disable all or parts of this?
+        Strings args = {
+            // clang-format off
+            "--quiet",
+            "--foreground",
+            "--config-net",
+            "--gateway", PASTA_HOST_IPV4,
+            "--address", PASTA_CHILD_IPV4, "--netmask", PASTA_IPV4_NETMASK,
+            "--dns-forward", PASTA_HOST_IPV4,
+            "--gateway", PASTA_HOST_IPV6,
+            "--address", PASTA_CHILD_IPV6,
+            "--dns-forward", PASTA_HOST_IPV6,
+            "--ns-ifname", PASTA_NS_IFNAME,
+            "--no-netns-quit",
+            "--netns", "/proc/self/fd/0",
+            // clang-format on
+        };
+
+        AutoCloseFD netns(open(fmt("/proc/%i/ns/net", pid.get()).c_str(), O_RDONLY | O_CLOEXEC));
+        if (!netns) {
+            throw SysError("failed to open netns");
+        }
+
+        AutoCloseFD userns;
+        if (usingUserNamespace) {
+            userns =
+                AutoCloseFD(open(fmt("/proc/%i/ns/user", pid.get()).c_str(), O_RDONLY | O_CLOEXEC));
+            if (!userns) {
+                throw SysError("failed to open userns");
+            }
+            args.push_back("--userns");
+            args.push_back("/proc/self/fd/1");
+        }
+
+        // FIXME ideally we want a notification when pasta exits, but we cannot do
+        // this at present. without such support we need to busy-wait for pasta to
+        // set up the namespace completely and time out after a while for the case
+        // of pasta launch failures. pasta logs go to syslog only for now as well.
+        pastaPid = runProgram2({
+            .program = settings.pastaPath,
+            .args = args,
+            .uid = useBuildUsers() ? std::optional(buildUser->getUID()) : std::nullopt,
+            .gid = useBuildUsers() ? std::optional(buildUser->getGID()) : std::nullopt,
+            // TODO these redirections are crimes. pasta closes all non-stdio file
+            // descriptors very early and lacks fd arguments for the namespaces we
+            // want it to join. we cannot have pasta join the namespaces via pids;
+            // doing so requires capabilities which pasta *also* drops very early.
+            .redirections = {
+                {.from = 0, .to = netns.get()},
+                {.from = 1, .to = userns ? userns.get() : 1},
+            },
+            .caps = getuid() == 0
+                ? std::set<long>{CAP_SYS_ADMIN, CAP_NET_BIND_SERVICE}
+                : std::set<long>{},
+        });
+    }
+
     return pid;
 }
 
@@ -997,5 +1103,24 @@ void LinuxLocalDerivationGoal::killSandbox(bool getStats)
            This avoids processes unrelated to the build being killed, thus avoiding: https://git.lix.systems/lix-project/lix/issues/667 */
         LocalDerivationGoal::killSandbox(getStats);
     }
+
+    if (pastaPid) {
+        // FIXME we really want to send SIGTERM instead and wait for pasta to exit,
+        // but we do not have the infra for that right now. we send SIGKILL instead
+        // and treat exiting with that as a successful exit code until such a time.
+        // this is not likely to cause problems since pasta runs as the build user,
+        // but not inside the build sandbox. if it's killed it's either due to some
+        // external influence (in which case the sandboxed child will probably fail
+        // due to network errors, if it used the network at all) or some bug in lix
+        if (auto status = pastaPid.kill(); !WIFSIGNALED(status) || WTERMSIG(status) != SIGKILL) {
+            if (WIFSIGNALED(status)) {
+                throw Error("pasta killed by signal %i", WTERMSIG(status));
+            } else if (WIFEXITED(status)) {
+                throw Error("pasta exited with code %i", WEXITSTATUS(status));
+            } else {
+                throw Error("pasta exited with status %i", status);
+            }
+        }
+    }
 }
 }

lix/libstore/platform/linux.hh

@@ -4,6 +4,7 @@
 #include "lix/libstore/build/local-derivation-goal.hh"
 #include "lix/libstore/gc-store.hh"
 #include "lix/libstore/local-store.hh"
+#include "lix/libutil/processes.hh"
 
 namespace nix {
 
@@ -33,7 +34,23 @@ class LinuxLocalDerivationGoal : public LocalDerivationGoal
 public:
     using LocalDerivationGoal::LocalDerivationGoal;
 
+    ~LinuxLocalDerivationGoal();
+
+    // NOTE these are all C strings because macos doesn't have constexpr std::string
+    // constructors, and std::string_view is a pain to turn into std::strings again.
+    static constexpr const char * PASTA_NS_IFNAME = "eth0";
+    static constexpr const char * PASTA_HOST_IPV4 = "169.254.1.1";
+    static constexpr const char * PASTA_CHILD_IPV4 = "169.254.1.2";
+    static constexpr const char * PASTA_IPV4_NETMASK = "16";
+    // randomly chosen 6to4 prefix, mapping the same ipv4ll as above.
+    // even if this id is used on the daemon host there should not be
+    // any collisions since ipv4ll should never be addressed by ipv6.
+    static constexpr const char * PASTA_HOST_IPV6 = "64:ff9b:1:4b8e:472e:a5c8:a9fe:0101";
+    static constexpr const char * PASTA_CHILD_IPV6 = "64:ff9b:1:4b8e:472e:a5c8:a9fe:0102";
+
 private:
+    RunningProgram pastaPid;
+
     /**
      * Create and populate chroot
      */
@@ -62,6 +79,7 @@ private:
         return true;
     }
 
+    std::string rewriteResolvConf(std::string fromHost) override;
 };
 
 }

lix/libstore/remote-store.cc

@@ -147,6 +147,7 @@ void RemoteStore::setOptions(Connection & conn)
     overrides.erase(experimentalFeatureSettings.experimentalFeatures.name);
     overrides.erase(settings.pluginFiles.name);
     overrides.erase(settings.storeUri.name); // the daemon *is* the store
+    overrides.erase(settings.tarballTtl.name); // eval-time only, implictly set by flake cli
     conn.to << overrides.size();
     for (auto & i : overrides)
         conn.to << i.first << i.second.value;

lix/libstore/settings/build-dir.md

@@ -1,14 +1,24 @@
 ---
 name: build-dir
 internalName: buildDir
-settingType: PathsSetting<std::optional<Path>>
+settingType: PathsSetting<Path>
-default: null
+defaultText: "`«nixStateDir»/b`"
+defaultExpr: nixStateDir + "/b"
 ---
 The directory on the host, in which derivations' temporary build directories are created.
 
-If not set, Nix will use the [`temp-dir`](#conf-temp-dir) setting if set, otherwise the system temporary directory indicated by the `TMPDIR` environment variable.
+If not set, Lix will use the `b` subdirectory of its configured state directory.
-Note that builds are often performed by the Nix daemon, so its `TMPDIR` is used, and not that of the Nix command line interface.
+Lix will create this directory automatically with suitable permissions if it does not
+exist, otherwise its permissions must allow all users to traverse the directory (i.e.
+it must have `o+x` set, in unix parlance) for non-sandboxed builds to work correctly.
 
 This is also the location where [`--keep-failed`](@docroot@/command-ref/opt-common.md#opt-keep-failed) leaves its files.
 
 If Nix runs without sandbox, or if the platform does not support sandboxing with bind mounts (e.g. macOS), then the [`builder`](@docroot@/language/derivations.md#attr-builder)'s environment will contain this directory, instead of the virtual location [`sandbox-build-dir`](#conf-sandbox-build-dir).
+
+> Important:
+>
+> `build-dir` must not be set to a world-writable directory. Placing temporary build
+> directories in a world-writable place allows other users to access or modify build
+> data that is currently in use. This alone is merely an impurity, but combined with
+> another factor this has allowed malicious derivations to escape the build sandbox.

lix/libstore/settings/pasta-path.md

@@ -0,0 +1,10 @@
+---
+name: pasta-path
+internalName: pastaPath
+type: Path
+default: ""
+---
+If set to an absolute path, enables fully sandboxing fixed-output
+derivations, by using `pasta` to pass network traffic between the
+private network namespace. This allows for greater levels of isolation
+of builds to the host.

lix/libstore/sqlite.cc

@@ -179,8 +179,13 @@ SQLiteStmt::Use::~Use()
 SQLiteStmt::Use & SQLiteStmt::Use::operator () (std::string_view value, bool notNull)
 {
     if (notNull) {
-        if (sqlite3_bind_text(stmt.stmt.get(), curArg++, value.data(), -1, SQLITE_TRANSIENT) != SQLITE_OK)
+        if (sqlite3_bind_text(
+                stmt.stmt.get(), curArg++, value.data(), value.length(), SQLITE_TRANSIENT
+            )
+            != SQLITE_OK)
+        {
             SQLiteError::throw_(stmt.db, "binding argument");
+        }
     } else
         bind();
     return *this;

lix/libstore/ssh.cc

@@ -5,6 +5,7 @@
 #include "lix/libutil/logging.hh"
 #include "lix/libutil/strings.hh"
 #include "lix/libstore/temporary-dir.hh"
+#include <unistd.h>
 
 namespace nix {
 
@@ -21,7 +22,7 @@ SSH::SSH(const std::string & host, const std::optional<uint16_t> port, const std
         throw Error("invalid SSH host name '%s'", host);
 
     auto state(state_.lock());
-    state->tmpDir = std::make_unique<AutoDelete>(createTempDir("", "nix", true, true, 0700));
+    state->tmpDir = std::make_unique<AutoDelete>(createTempDir("nix", 0700));
 }
 
 void SSH::addCommonSSHOpts(Strings & args)
@@ -80,10 +81,11 @@ std::unique_ptr<SSH::Connection> SSH::startCommand(const std::string & command)
         // reasonably POSIX-y semantics for the things we're about
         // to do next.
         if (fakeSSH) {
-            args = { "bash" };
+            args = { "bash", "-c", command };
         } else {
-            args = { "ssh", host.c_str(), "-x", "-T", "-oRemoteCommand=bash" };
+            args = { "ssh", host.c_str(), "-x", "-T" };
             addCommonSSHOpts(args);
+            args.push_back(command);
         }
 
         execvp(args.begin()->c_str(), stringsToCharPtrs(args).data());
@@ -96,38 +98,6 @@ std::unique_ptr<SSH::Connection> SSH::startCommand(const std::string & command)
     in.readSide.reset();
     out.writeSide.reset();
 
-    // Once we hand off to nix-store (on the remote) and the caller (on the client),
-    // we lose the ability to catch SSH failing, due to Historical Architectural Decisions.
-    //
-    // We want to catch at least _some_ errors and alert the user in case of
-    // an obvious misconfiguration, so run a very simple command first
-    // to make sure things are at least somewhat operational.
-    //
-    // The exact semantics of
-    // - not having a shell prompt get in the way when non-interactive
-    // - echo doing the reasonable thing
-    // Are exactly why we specifically forced bash (via ssh RemoteCommand) earlier.
-    // We do *not* use /bin/sh because that may be busybox and busybox breaks here.
-    //
-    // FIXME: make any of this shit make sense
-    {
-        writeLine(in.writeSide.get(), "echo started");
-
-        std::string reply;
-        try {
-            reply = readLine(out.readSide.get());
-        } catch (EndOfFile & e) { }
-
-        if (reply != "started") {
-            warn("SSH to '%s' failed, stdout first line: '%s'", host, reply);
-            throw Error("failed to start SSH connection to '%s'", host);
-        }
-    }
-
-    // Now that we're reasonably confident we have something vaguely resembling
-    // a connection, hand off to the command.
-    writeLine(in.writeSide.get(), fmt("exec %s", command));
-
     conn->out = std::move(out.readSide);
     conn->in = std::move(in.writeSide);
 

lix/libstore/store-api.hh

@@ -234,6 +234,22 @@ protected:
     Store(const StoreConfig & config);
 
 public:
+    struct AssociatedCredentials
+    {
+        uid_t user;
+        gid_t group;
+    };
+
+    /**
+     * Credentials of the context using this store if this store is proxied
+     * to somewhere else and the peer context is known. Only the daemon can
+     * set this to values that make any sense, using unix peer credentials.
+     */
+    virtual std::optional<AssociatedCredentials> associatedCredentials() const
+    {
+        return {};
+    }
+
     /**
      * Perform any necessary effectful operation to make the store up and
      * running

lix/libstore/temporary-dir.cc

@@ -5,10 +5,9 @@
 
 namespace nix {
 
-Path createTempDir(const Path & tmpRoot, const Path & prefix,
+Path createTempDir(const std::optional<Path> & prefix, mode_t mode)
-    bool includePid, bool useGlobalCounter, mode_t mode)
 {
-    return createTempSubdir(tmpRoot.empty() ? defaultTempDir() : tmpRoot, prefix, includePid, useGlobalCounter, mode);
+    return createTempSubdir(defaultTempDir(), prefix, mode);
 }
 
 std::pair<AutoCloseFD, Path> createTempFile(const Path & prefix)

lix/libstore/temporary-dir.hh

@@ -8,8 +8,7 @@ namespace nix {
 /**
  * Create a temporary directory.
  */
-Path createTempDir(const Path & tmpRoot = "", const Path & prefix = "nix",
+Path createTempDir(const std::optional<Path> & prefix = "nix", mode_t mode = 0755);
-    bool includePid = true, bool useGlobalCounter = true, mode_t mode = 0755);
 
 /**
  * Create a temporary file, returning a file handle and its path.

lix/libutil/archive.cc

@@ -370,13 +370,19 @@ struct Parser
 #define READ_STRING_LIMITED(limit)                                                                 \
     ({                                                                                             \
         size_t len = FETCH_INT(size_t);                                                            \
+        if (len > (limit)) {                                                                       \
+            throw SerialisationError(                                                              \
+                "found malformed string tag. input may be a compressed NAR, which cannot be read " \
+                "directly"                                                                         \
+            );                                                                                     \
+        }                                                                                          \
         co_yield WantBytes{len + (8 - len % 8) % 8};                                               \
         StringSource src(std::string_view(buffer.data(), buffer.size()));                          \
         auto str = readString(src, (limit));                                                       \
         buffer.clear();                                                                            \
         std::move(str);                                                                            \
     })
-#define READ_STRING() READ_STRING_LIMITED(std::numeric_limits<size_t>::max())
+#define READ_STRING() READ_STRING_LIMITED(1048576)
 #define READ_PADDING(size)                                                    \
     do {                                                                      \
         if ((size) % 8) {                                                     \
@@ -632,7 +638,7 @@ struct AsyncCopier : AsyncInputStream
                 co_yield Fragment{want->n, false};
             } else if (auto f = std::get_if<Parser::FileHeader>(&*i)) {
                 co_yield Fragment{f->size, true};
-            } else if (auto sl = std::get_if<Parser::Symlink>(&*i)) {
+            } else if (auto _ = std::get_if<Parser::Symlink>(&*i)) {
                 // nothing to do
             } else if (auto dir = std::get_if<Parser::Directory>(&*i)) {
                 while (auto e = dir->content.next()) {

lix/libutil/compression.cc

@@ -259,6 +259,7 @@ struct BrotliCompressionSink : ChunkedCompressionSink
 
     void writeInternal(std::string_view data) override
     {
+        // NOLINTNEXTLINE(bugprone-suspicious-stringview-data-usage)
         auto next_in = charptr_cast<const uint8_t *>(data.data());
         size_t avail_in = data.size();
         uint8_t * next_out = outbuf;

lix/libutil/config.cc

@@ -10,6 +10,7 @@
 #include "lix/libutil/strings.hh"
 
 #include "lix/libutil/config-impl.hh"
+#include <mutex>
 
 namespace nix {
 
@@ -343,11 +344,31 @@ template<> std::string BaseSetting<StringSet>::to_string() const
 
 template<> ExperimentalFeatures BaseSetting<ExperimentalFeatures>::parse(const std::string & str, const ApplyConfigOptions & options) const
 {
+    auto warnDeprecated294 = [](std::once_flag & flag, std::string_view thing) {
+        std::call_once(flag, [&] {
+            warn(
+                "The %s experimental feature is deprecated and will be removed in Lix 2.94. "
+                "See https://git.lix.systems/lix-project/lix/issues/815 for more details.",
+                thing
+            );
+        });
+    };
+
     ExperimentalFeatures res{};
     for (auto & s : tokenizeString<StringSet>(str)) {
-        if (auto thisXpFeature = parseExperimentalFeature(s); thisXpFeature)
+        if (auto thisXpFeature = parseExperimentalFeature(s); thisXpFeature) {
+            if (*thisXpFeature == Xp::CaDerivations) {
+                static std::once_flag warned;
+                warnDeprecated294(warned, s);
+            } else if (*thisXpFeature == Xp::DynamicDerivations) {
+                static std::once_flag warned;
+                warnDeprecated294(warned, s);
+            } else if (*thisXpFeature == Xp::ImpureDerivations) {
+                static std::once_flag warned;
+                warnDeprecated294(warned, s);
+            }
             res = res | thisXpFeature.value();
-        else
+        } else
             warn("unknown experimental feature '%s'", s);
     }
     return res;

lix/libutil/file-descriptor.cc

@@ -155,6 +155,29 @@ int AutoCloseFD::get() const
     return fd;
 }
 
+std::string guessOrInventPathFromFD(int fd)
+{
+    assert(fd >= 0);
+    /* On Linux, there's no F_GETPATH available.
+     * But we can read /proc/ */
+#if __linux__
+    try {
+        return readLink(fmt("/proc/self/fd/%1%", fd).c_str());
+    } catch (...) {
+    }
+#elif defined (HAVE_F_GETPATH) && HAVE_F_GETPATH
+    std::string fdName(PATH_MAX, '\0');
+    if (fcntl(fd, F_GETPATH, fdName.data()) != -1) {
+        fdName.resize(strlen(fdName.c_str()));
+        return fdName;
+    }
+#else
+#error "No implementation for retrieving file descriptors path."
+#endif
+
+    return fmt("<fd %i>", fd);
+}
+
 
 void AutoCloseFD::close()
 {

lix/libutil/file-descriptor.hh

@@ -36,6 +36,15 @@ void writeFull(int fd, std::string_view s, bool allowInterrupts = true);
  */
 std::string drainFD(int fd, bool block = true, const size_t reserveSize=0);
 
+
+/*
+ * Will attempt to guess *A* path associated that might lead to the same file as used by this
+ * file descriptor.
+ *
+ * The returned string should NEVER be used as a valid path.
+ */
+std::string guessOrInventPathFromFD(int fd);
+
 Generator<Bytes> drainFDSource(int fd, bool block = true);
 
 class AutoCloseFD
@@ -50,6 +59,15 @@ public:
     AutoCloseFD& operator =(const AutoCloseFD & fd) = delete;
     AutoCloseFD& operator =(AutoCloseFD&& fd) noexcept(false);
     int get() const;
+
+    /*
+     * Will attempt to guess *A* path associated that might lead to the same file as used by this
+     * file descriptor.
+     *
+     * The returned string should NEVER be used as a valid path.
+     */
+    std::string guessOrInventPath() const { return guessOrInventPathFromFD(fd); }
+
     explicit operator bool() const;
     int release();
     void close();

lix/libutil/file-system.cc

@@ -3,6 +3,8 @@
 #include <filesystem>
 #include <atomic>
 #include <random>
+#include <ranges>
+#include <sys/xattr.h>
 
 #include "lix/libutil/environment-variables.hh"
 #include "lix/libutil/file-descriptor.hh"
@@ -359,26 +361,6 @@ Generator<Bytes> readFileSource(const Path & path)
 }
 
 
-void writeFile(const Path & path, std::string_view s, mode_t mode, bool sync)
-{
-    AutoCloseFD fd{open(path.c_str(), O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC, mode)};
-    if (!fd)
-        throw SysError("opening file '%1%'", path);
-    try {
-        writeFull(fd.get(), s);
-    } catch (Error & e) {
-        e.addTrace({}, "writing file '%1%'", path);
-        throw;
-    }
-    if (sync)
-        fd.fsync();
-    // Explicitly close to make sure exceptions are propagated.
-    fd.close();
-    if (sync)
-        syncParent(path);
-}
-
-
 static AutoCloseFD openForWrite(const Path & path, mode_t mode)
 {
     AutoCloseFD fd{open(path.c_str(), O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC, mode)};
@@ -387,6 +369,59 @@ static AutoCloseFD openForWrite(const Path & path, mode_t mode)
     return fd;
 }
 
+static AutoCloseFD openForWriteExcl(const Path & path, mode_t mode)
+{
+    AutoCloseFD fd{open(path.c_str(), O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC | O_EXCL, mode)};
+    if (!fd)
+        throw SysError("opening file '%1%'", path);
+    return fd;
+}
+
+void writeFile(const Path & path, std::string_view s, mode_t mode, bool allowInterrupts)
+{
+    AutoCloseFD fd = openForWrite(path, mode);
+
+    writeFile(fd, s, mode);
+
+    /* Close explicitly to propagate the exceptions. */
+    fd.close();
+}
+
+void writeFileExcl(const Path & path, std::string_view s, mode_t mode, bool allowInterrupts)
+{
+    AutoCloseFD fd = openForWriteExcl(path, mode);
+
+    writeFile(fd, s, mode, allowInterrupts);
+
+    // Close explicitly to propagate the exceptions.
+    fd.close();
+}
+
+void writeFile(AutoCloseFD & fd, std::string_view s, mode_t mode, bool allowInterrupts)
+{
+    assert(fd);
+    try {
+        writeFull(fd.get(), s);
+    } catch (Error & e) {
+        e.addTrace({}, "writing file '%1%'", fd.guessOrInventPath());
+        throw;
+    }
+}
+
+void writeFileAndSync(const Path & path, std::string_view s, mode_t mode)
+{
+    {
+        AutoCloseFD fd = openForWrite(path, mode);
+
+        writeFile(fd, s, mode);
+        fd.fsync();
+        /* Close explicitly to ensure that exceptions are propagated. */
+        fd.close();
+    }
+
+    syncParent(path);
+}
+
 static void closeForWrite(const Path & path, AutoCloseFD & fd, bool sync)
 {
     if (sync)
@@ -397,7 +432,7 @@ static void closeForWrite(const Path & path, AutoCloseFD & fd, bool sync)
         syncParent(path);
 }
 
-void writeFile(const Path & path, Source & source, mode_t mode, bool sync)
+void writeFile(const Path & path, Source & source, mode_t mode)
 {
     AutoCloseFD fd = openForWrite(path, mode);
 
@@ -414,11 +449,30 @@ void writeFile(const Path & path, Source & source, mode_t mode, bool sync)
         e.addTrace({}, "writing file '%1%'", path);
         throw;
     }
-    closeForWrite(path, fd, sync);
+    closeForWrite(path, fd, false);
 }
 
-kj::Promise<Result<void>>
+void writeFileExcl(const Path & path, Source & source, mode_t mode)
-writeFile(const Path & path, AsyncInputStream & source, mode_t mode, bool sync)
+{
+    AutoCloseFD fd = openForWriteExcl(path, mode);
+
+    std::vector<char> buf(64 * 1024);
+
+    try {
+        while (true) {
+            try {
+                auto n = source.read(buf.data(), buf.size());
+                writeFull(fd.get(), {buf.data(), n});
+            } catch (EndOfFile &) { break; }
+        }
+    } catch (Error & e) {
+        e.addTrace({}, "writing file '%1%'", path);
+        throw;
+    }
+    closeForWrite(path, fd, false);
+}
+
+kj::Promise<Result<void>> writeFile(const Path & path, AsyncInputStream & source, mode_t mode)
 try {
     AutoCloseFD fd = openForWrite(path, mode);
 
@@ -436,7 +490,7 @@ try {
         e.addTrace({}, "writing file '%1%'", path);
         throw;
     }
-    closeForWrite(path, fd, sync);
+    closeForWrite(path, fd, false);
     co_return result::success();
 } catch (...) {
     co_return result::current_exception();
@@ -450,18 +504,29 @@ void syncParent(const Path & path)
     fd.fsync();
 }
 
-static void _deletePath(int parentfd, const Path & path, uint64_t & bytesFreed, bool interruptible)
+/* TODO(horrors): a better structure that links all parent fds for the traversal root
+ * should be considered for this code
+ */
+static void _deletePath(int parentfd, const std::string & name, uint64_t & bytesFreed, bool interruptible)
 {
+    /* This ensures that `name` is an immediate child of `parentfd`. */
+    assert(!name.empty() && name.find('/') == std::string::npos && "`name` is an immediate child to `parentfd`");
+
     if (interruptible) {
         checkInterrupt();
     }
 
-    std::string name(baseNameOf(path));
+    /* FIXME(horrors): there's a minor TOCTOU here.
+     * we fstatat the inode nofollow, check if this is a directory
+     * and then open it.
+     * a better alternative is open it as O_PATH as a namefd.
+     * if it's a directory, it can be openat with the namefd.
+     */
 
     struct stat st;
     if (fstatat(parentfd, name.c_str(), &st, AT_SYMLINK_NOFOLLOW) == -1) {
         if (errno == ENOENT) return;
-        throw SysError("getting status of '%1%'", path);
+        throw SysError("getting status of '%1%' in directory '%2%'", name, guessOrInventPathFromFD(parentfd));
     }
 
     if (!S_ISDIR(st.st_mode)) {
@@ -492,24 +557,25 @@ static void _deletePath(int parentfd, const Path & path, uint64_t & bytesFreed,
         /* Make the directory accessible. */
         const auto PERM_MASK = S_IRUSR | S_IWUSR | S_IXUSR;
         if ((st.st_mode & PERM_MASK) != PERM_MASK) {
-            if (fchmodat(parentfd, name.c_str(), st.st_mode | PERM_MASK, 0) == -1)
+            if (fchmodat(parentfd, name.c_str(), st.st_mode | PERM_MASK, 0) == -1) {
-                throw SysError("chmod '%1%'", path);
+                throw SysError("chmod '%1%' in directory '%2%'", name, guessOrInventPathFromFD(parentfd));
+            }
         }
 
-        int fd = openat(parentfd, path.c_str(), O_RDONLY);
+        int fd = openat(parentfd, name.c_str(), O_RDONLY | O_DIRECTORY | O_NOFOLLOW);
         if (fd == -1)
-            throw SysError("opening directory '%1%'", path);
+            throw SysError("opening directory '%1%' in directory '%2%'", name, guessOrInventPathFromFD(parentfd));
         AutoCloseDir dir(fdopendir(fd));
         if (!dir)
-            throw SysError("opening directory '%1%'", path);
+            throw SysError("opening directory '%1%' in directory '%2%'", name, guessOrInventPathFromFD(parentfd));
-        for (auto & i : readDirectory(dir.get(), path, interruptible))
+        for (auto & i : readDirectory(dir.get(), name, interruptible))
-            _deletePath(dirfd(dir.get()), path + "/" + i.name, bytesFreed, interruptible);
+            _deletePath(dirfd(dir.get()), i.name, bytesFreed, interruptible);
     }
 
     int flags = S_ISDIR(st.st_mode) ? AT_REMOVEDIR : 0;
     if (unlinkat(parentfd, name.c_str(), flags) == -1) {
         if (errno == ENOENT) return;
-        throw SysError("cannot unlink '%1%'", path);
+        throw SysError("cannot unlink '%1%' in directory '%2%'", name, guessOrInventPathFromFD(parentfd));
     }
 }
 
@@ -525,7 +591,7 @@ static void _deletePath(const Path & path, uint64_t & bytesFreed, bool interrupt
         throw SysError("opening directory '%1%'", path);
     }
 
-    _deletePath(dirfd.get(), path, bytesFreed, interruptible);
+    _deletePath(dirfd.get(), std::string(baseNameOf(path)), bytesFreed, interruptible);
 }
 
 
@@ -611,26 +677,11 @@ void AutoDelete::reset(const Path & p, bool recursive) {
 
 //////////////////////////////////////////////////////////////////////
 
-static Path tempName(PathView parent, const Path & prefix, bool includePid,
+Path createTempSubdir(const Path & parent, const std::optional<Path> & prefix,
-    std::atomic<unsigned int> & counter)
+    mode_t mode)
 {
-    auto tmpRoot = canonPath(parent, true);
-    if (includePid)
-        return fmt("%1%/%2%-%3%-%4%", tmpRoot, prefix, getpid(), counter++);
-    else
-        return fmt("%1%/%2%-%3%", tmpRoot, prefix, counter++);
-}
-
-Path createTempSubdir(const Path & parent, const Path & prefix,
-    bool includePid, bool useGlobalCounter, mode_t mode)
-{
-    static std::atomic<unsigned int> globalCounter = 0;
-    std::atomic<unsigned int> localCounter = 0;
-    auto & counter(useGlobalCounter ? globalCounter : localCounter);
-
-    while (1) {
     checkInterrupt();
-        Path tmpDir = tempName(parent, prefix, includePid, counter);
+    Path tmpDir = makeTempPath(canonPath(parent, true) + "/", prefix);
     if (mkdir(tmpDir.c_str(), mode) == 0) {
 #if __FreeBSD__
         /* Explicitly set the group of the directory.  This is to
@@ -641,21 +692,32 @@ Path createTempSubdir(const Path & parent, const Path & prefix,
            will be owned by "wheel"; but if the user is not in
            "wheel", then "tar" will fail to unpack archives that
            have the setgid bit set on directories. */
-            if (chown(tmpDir.c_str(), (uid_t) -1, getegid()) != 0)
+        if (chown(tmpDir.c_str(), (uid_t) -1, getegid()) != 0) {
             throw SysError("setting group of directory '%1%'", tmpDir);
+        }
 #endif
         return tmpDir;
     }
-        if (errno != EEXIST)
     throw SysError("creating directory '%1%'", tmpDir);
+}
+
+Path makeTempPath(const Path & root, const std::optional<Path> & prefix)
+{
+    static thread_local std::random_device generator{};
+    std::uniform_int_distribution<uint64_t> uniform_dist{};
+    const uint64_t entropy[2] = {uniform_dist(generator), uniform_dist(generator)};
+    auto unique = base32Encode(std::as_bytes(std::span(entropy)));
+
+    if (prefix) {
+        return fmt("%s%s-%s", root, *prefix, unique);
+    } else {
+        return root + unique;
     }
 }
 
-Path makeTempPath(const Path & root, const Path & suffix)
+Path makeTempSiblingPath(const Path & path)
 {
-    // start the counter at a random value to minimize issues with preexisting temp paths
+    return makeTempPath(fs::path(path).remove_filename());
-    static std::atomic_uint_fast32_t counter(std::random_device{}());
-    return fmt("%1%%2%-%3%-%4%", root, suffix, getpid(), counter.fetch_add(1, std::memory_order_relaxed));
 }
 
 void createSymlink(const Path & target, const Path & link)
@@ -666,20 +728,9 @@ void createSymlink(const Path & target, const Path & link)
 
 void replaceSymlink(const Path & target, const Path & link)
 {
-    for (unsigned int n = 0; true; n++) {
+    Path tmp = canonPath(makeTempSiblingPath(link));
-        Path tmp = canonPath(fmt("%s/.%d_%s", dirOf(link), n, baseNameOf(link)));
-
-        try {
     createSymlink(target, tmp);
-        } catch (SysError & e) {
-            if (e.errNo == EEXIST) continue;
-            throw;
-        }
-
     renameFile(tmp, link);
-
-        break;
-    }
 }
 
 void setWriteTime(const fs::path & p, const struct stat & st)

lix/libutil/file-system.hh

@@ -189,11 +189,25 @@ Generator<Bytes> readFileSource(const Path & path);
 /**
  * Write a string to a file.
  */
-void writeFile(const Path & path, std::string_view s, mode_t mode = 0666, bool sync = false);
+void writeFile(
+    const Path & path, std::string_view s, mode_t mode = 0666, bool allowInterrupts = true
+);
+/** Write a string to an exclusively-opened file. */
+void writeFileExcl(
+    const Path & path, std::string_view s, mode_t mode = 0666, bool allowInterrupts = true
+);
+void writeFileUninterruptible(const Path & path, std::string_view s, mode_t mode = 0666);
+void writeFile(const Path & path, Source & source, mode_t mode = 0666);
+void writeFileExcl(const Path & path, Source & source, mode_t mode = 0666);
 
-void writeFile(const Path & path, Source & source, mode_t mode = 0666, bool sync = false);
+void writeFile(AutoCloseFD & fd, std::string_view s, mode_t mode = 0666, bool allowInterrupts = true);
 kj::Promise<Result<void>>
-writeFile(const Path & path, AsyncInputStream & source, mode_t mode = 0666, bool sync = false);
+writeFile(const Path & path, AsyncInputStream & source, mode_t mode = 0666);
+
+/**
+ * Write a string to a file and flush the file and its parents direcotry to disk.
+ */
+void writeFileAndSync(const Path & path, std::string_view s, mode_t mode = 0666);
 
 /**
  * Flush a file's parent directory to disk
@@ -294,16 +308,21 @@ typedef std::unique_ptr<DIR, DIRDeleter> AutoCloseDir;
 /**
  * Create a temporary directory in a given parent directory.
  */
-Path createTempSubdir(const Path & parent, const Path & prefix = "nix",
+Path createTempSubdir(const Path & parent, const std::optional<Path> & prefix = "nix",
-    bool includePid = true, bool useGlobalCounter = true, mode_t mode = 0755);
+    mode_t mode = 0755);
 
 /**
- * Return temporary path constructed by appending a suffix to a root path.
+ * Return temporary path constructed by appending to a root path.
  *
- * The constructed path looks like `<root><suffix>-<pid>-<unique>`. To create a
+ * The constructed path looks like `<root>[<prefix>-]<unique>`. To create a
- * path nested in a directory, provide a suffix starting with `/`.
+ * path nested in a directory, provide a root ending with `/`.
  */
-Path makeTempPath(const Path & root, const Path & suffix = ".tmp");
+Path makeTempPath(const Path & root, const std::optional<Path> & prefix = ".tmp");
+
+/**
+ * Return temporary path in the same directory as a given path.
+ */
+Path makeTempSiblingPath(const Path & path);
 
 /**
  * Used in various places.

lix/libutil/hash.cc

@@ -80,33 +80,6 @@ static std::string printHash16(const Hash & hash)
 }
 
 
-// omitted: E O U T
-const std::string base32Chars = "0123456789abcdfghijklmnpqrsvwxyz";
-
-
-static std::string printHash32(const Hash & hash)
-{
-    assert(hash.hashSize);
-    size_t len = hash.base32Len();
-    assert(len);
-
-    std::string s;
-    s.reserve(len);
-
-    for (int n = (int) len - 1; n >= 0; n--) {
-        unsigned int b = n * 5;
-        unsigned int i = b / 8;
-        unsigned int j = b % 8;
-        unsigned char c =
-            (hash.hash[i] >> j)
-            | (i >= hash.hashSize - 1 ? 0 : hash.hash[i + 1] << (8 - j));
-        s.push_back(base32Chars[c & 0x1f]);
-    }
-
-    return s;
-}
-
-
 std::string printHash16or32(const Hash & hash)
 {
     return hash.to_string(hash.type == HashType::MD5 ? Base::Base16 : Base::Base32, false);
@@ -125,7 +98,7 @@ std::string Hash::to_string(Base base, bool includeType) const
         s += printHash16(*this);
         break;
     case Base::Base32:
-        s += printHash32(*this);
+        s += base32EncodeStr(std::string_view(charptr_cast<const char *>(hash), hashSize));
         break;
     case Base::Base64:
     case Base::SRI:
@@ -225,26 +198,8 @@ Hash::Hash(std::string_view rest, HashType type, bool isSRI)
     }
 
     else if (!isSRI && rest.size() == base32Len()) {
-
+        auto d = base32Decode(rest);
-        for (unsigned int n = 0; n < rest.size(); ++n) {
+        memcpy(hash, d.data(), hashSize);
-            char c = rest[rest.size() - n - 1];
-            size_t digit;
-            for (digit = 0; digit < base32Chars.size(); ++digit) /* !!! slow */
-                if (base32Chars[digit] == c) break;
-            if (digit >= 32)
-                throw BadHash("invalid base-32 hash '%s'", rest);
-            unsigned int b = n * 5;
-            unsigned int i = b / 8;
-            unsigned int j = b % 8;
-            hash[i] |= digit << j;
-
-            if (i < hashSize - 1) {
-                hash[i + 1] |= digit >> (8 - j);
-            } else {
-                if (digit >> (8 - j))
-                    throw BadHash("invalid base-32 hash '%s'", rest);
-            }
-        }
     }
 
     else if (isSRI || rest.size() == base64Len()) {

lix/libutil/hash.hh

@@ -32,8 +32,6 @@ const int sha512HashSize = 64;
 
 extern std::set<std::string> hashTypes;
 
-extern const std::string base32Chars;
-
 enum class Base : int { Base64, Base32, Base16, SRI };
 
 

lix/libutil/json-utils.cc

@@ -38,6 +38,6 @@ const JSON & ensureType(
             JSON(expectedType).type_name(),
             value.type_name());
 
-    return value;
+    return value; // NOLINT(bugprone-return-const-ref-from-parameter)
 }
 }

lix/libutil/logging.hh

@@ -284,9 +284,9 @@ extern Verbosity verbosity;
  */
 #define printMsgUsing(loggerParam, level, args...)              \
     do {                                                        \
-        auto __lvl = level; \
+        auto _lix_logger_print_lvl = level;                     \
-        if (__lvl <= nix::verbosity) { \
+        if (_lix_logger_print_lvl <= nix::verbosity) {          \
-            loggerParam->log(__lvl, fmt(args)); \
+            loggerParam->log(_lix_logger_print_lvl, fmt(args)); \
         }                                                       \
     } while (0)
 #define printMsg(level, args...) printMsgUsing(logger, level, args)

lix/libutil/meson.build

@@ -306,6 +306,7 @@ libutil = library(
     openssl,
     nlohmann_json,
     kj,
+    libatomic,
   ],
   include_directories : [ '../..' ],
   cpp_pch : cpp_pch,
@@ -343,6 +344,7 @@ liblixutil = declare_dependency(
     # lix-base pkg-config externally)
     kj,
     libarchive,
+    libatomic,
   ],
   link_with : libutil
 )

lix/libutil/processes.cc

@@ -22,6 +22,7 @@
 #endif
 
 #ifdef __linux__
+# include <linux/capability.h>
 # include <sys/prctl.h>
 # include <sys/mman.h>
 #endif
@@ -249,7 +250,7 @@ std::pair<int, std::string> runProgram(RunOptions && options)
 
     try {
         auto proc = runProgram2(options);
-        Finally const _wait([&] { proc.wait(); });
+        Finally const _wait([&] { proc.waitAndCheck(); });
         stdout = proc.getStdout()->drain();
     } catch (ExecError & e) {
         status = e.status;
@@ -277,7 +278,22 @@ RunningProgram::~RunningProgram()
     }
 }
 
-void RunningProgram::wait()
+std::tuple<pid_t, std::unique_ptr<Source>, int> RunningProgram::release()
+{
+    return {pid.release(), std::move(stdoutSource), stdout_.release()};
+}
+
+int RunningProgram::kill()
+{
+    return pid.kill();
+}
+
+int RunningProgram::wait()
+{
+    return pid.wait();
+}
+
+void RunningProgram::waitAndCheck()
 {
     if (std::uncaught_exceptions() == 0) {
         int status = pid.wait();
@@ -315,12 +331,21 @@ RunningProgram runProgram2(const RunOptions & options)
             replaceEnv(*options.environment);
         if (options.captureStdout && dup2(out.writeSide.get(), STDOUT_FILENO) == -1)
             throw SysError("dupping stdout");
-        if (options.mergeStderrToStdout)
+        for (auto redirection : options.redirections) {
-            if (dup2(STDOUT_FILENO, STDERR_FILENO) == -1)
+            if (dup2(redirection.to, redirection.from) == -1) {
-                throw SysError("cannot dup stdout into stderr");
+                throw SysError("dupping fd %i to %i", redirection.from, redirection.to);
+            }
+        }
 
         if (options.chdir && chdir((*options.chdir).c_str()) == -1)
             throw SysError("chdir failed");
+
+#if __linux__
+        if (!options.caps.empty() && prctl(PR_SET_KEEPCAPS, 1) < 0) {
+            throw SysError("setting keep-caps failed");
+        }
+#endif
+
         if (options.gid && setgid(*options.gid) == -1)
             throw SysError("setgid failed");
         /* Drop all other groups if we're setgid. */
@@ -329,6 +354,45 @@ RunningProgram runProgram2(const RunOptions & options)
         if (options.uid && setuid(*options.uid) == -1)
             throw SysError("setuid failed");
 
+#if __linux__
+        if (!options.caps.empty()) {
+            if (prctl(PR_SET_KEEPCAPS, 0)) {
+                throw SysError("clearing keep-caps failed");
+            }
+
+            // we do the capability dance like this to avoid a dependency
+            // on libcap, which has a rather large build closure and many
+            // more features that we need for now. maybe some other time.
+            static constexpr uint32_t LINUX_CAPABILITY_VERSION_3 = 0x20080522;
+            static constexpr uint32_t LINUX_CAPABILITY_U32S_3 = 2;
+            struct user_cap_header_struct
+            {
+                uint32_t version;
+                int pid;
+            } hdr = {LINUX_CAPABILITY_VERSION_3, 0};
+            struct user_cap_data_struct
+            {
+                uint32_t effective;
+                uint32_t permitted;
+                uint32_t inheritable;
+            } data[LINUX_CAPABILITY_U32S_3] = {};
+            for (auto cap : options.caps) {
+                assert(cap / 32 < LINUX_CAPABILITY_U32S_3);
+                data[cap / 32].permitted |= 1 << (cap % 32);
+                data[cap / 32].inheritable |= 1 << (cap % 32);
+            }
+            if (syscall(SYS_capset, &hdr, data)) {
+                throw SysError("couldn't set capabilities");
+            }
+
+            for (auto cap : options.caps) {
+                if (prctl(PR_CAP_AMBIENT, PR_CAP_AMBIENT_RAISE, cap, 0, 0) < 0) {
+                    throw SysError("couldn't set ambient caps");
+                }
+            }
+        }
+#endif
+
         Strings args_(options.args);
         args_.push_front(options.program);
 

lix/libutil/processes.hh

@@ -76,6 +76,11 @@ std::string runProgram(Path program, bool searchPath = false,
 
 struct RunOptions
 {
+    struct Redirection
+    {
+        int from, to;
+    };
+
     Path program;
     bool searchPath = true;
     Strings args = {};
@@ -84,8 +89,11 @@ struct RunOptions
     std::optional<Path> chdir = {};
     std::optional<std::map<std::string, std::string>> environment = {};
     bool captureStdout = false;
-    bool mergeStderrToStdout = false;
     bool isInteractive = false;
+    std::vector<Redirection> redirections;
+#if __linux__
+    std::set<long> caps;
+#endif
 };
 
 struct [[nodiscard("you must call RunningProgram::wait()")]] RunningProgram
@@ -102,9 +110,23 @@ private:
 
 public:
     RunningProgram() = default;
+    RunningProgram(RunningProgram &&) = default;
+    RunningProgram & operator=(RunningProgram &&) = default;
     ~RunningProgram();
 
-    void wait();
+    explicit operator bool() const { return bool(pid); }
+
+    std::tuple<pid_t, std::unique_ptr<Source>, int> release();
+
+    int kill();
+    [[nodiscard]]
+    int wait();
+    void waitAndCheck();
+
+    std::optional<int> getStdoutFD() const
+    {
+        return stdout_ ? std::optional(stdout_.get()) : std::nullopt;
+    }
 
     Source * getStdout() const { return stdoutSource.get(); };
 };

lix/libutil/references.cc

@@ -1,4 +1,5 @@
 #include "lix/libutil/references.hh"
+#include "lix/libutil/strings.hh"
 #include "lix/libutil/hash.hh"
 #include "lix/libutil/logging.hh"
 
@@ -53,7 +54,7 @@ void RefScanSink::operator () (std::string_view data)
        previous fragment and the start of the current fragment. */
     auto s = tail;
     auto tailLen = std::min(data.size(), refLength);
-    s.append(data.data(), tailLen);
+    s.append(data.data(), tailLen); // NOLINT(bugprone-suspicious-stringview-data-usage)
     search(s, hashes, seen);
 
     search(data, hashes, seen);

lix/libutil/strings.cc

@@ -1,6 +1,7 @@
 #include "lix/libutil/strings.hh"
 #include "lix/libutil/references.hh"
 #include <boost/lexical_cast.hpp>
+#include <ranges>
 #include <stdint.h>
 
 namespace nix {
@@ -128,8 +129,41 @@ std::string shellEscape(const std::string_view s)
     std::string r;
     r.reserve(s.size() + 2);
     r += "'";
-    for (auto & i : s)
+    for (auto & i : s) {
-        if (i == '\'') r += "'\\''"; else r += i;
+        if (i == '\'') {
+            // End the single quote, add a single backslash-escaped single quote,
+            // then start a single quote again.
+            // i.e., `I didn't know` becomes `'I didn'\''t know'`.
+            r += "'\\''";
+        } else {
+            r += i;
+        }
+    }
+
+    r += '\'';
+    return r;
+}
+
+std::string bashEscape(const std::string_view s)
+{
+    std::string r;
+    r.reserve(s.size() + 2);
+    r += "'";
+    for (auto & i : s) {
+        if (!std::isprint(i)) {
+            // Close the single quote, start an "ANSI-C Quote" ($'foo'), add `\xXX`,
+            // close the ANSI-C Quote, and finally start a normal single quote again.
+            r += fmt("'$'\\x%02x''", static_cast<unsigned int>(static_cast<unsigned char>(i)));
+        } else if (i == '\'') {
+            // End the single quote, add a single backslash-escaped single quote,
+            // then start a single quote again.
+            // i.e., `I didn't know` becomes `'I didn'\''t know'`.
+            r += "'\\''";
+        } else {
+            r += i;
+        }
+    }
+
     r += '\'';
     return r;
 }
@@ -195,6 +229,82 @@ std::string base64Decode(std::string_view s)
     return res;
 }
 
+// omitted: E O U T
+const std::string base32Chars = "0123456789abcdfghijklmnpqrsvwxyz";
+
+std::string base32EncodeStr(std::string_view s)
+{
+    std::span<std::byte const> sp = std::as_bytes(std::span(s));
+    return base32Encode(sp);
+}
+
+std::string base32Encode(std::span<std::byte const> bytes)
+{
+    // log2(32) == 5.
+    constexpr int B32_BITS_PER_DIGIT = 5;
+
+    // We need to do arithmetic.
+    auto const s = std::views::transform(bytes, std::to_integer<std::uint32_t>);
+
+    if (s.empty()) {
+        return "";
+    }
+
+    ssize_t len = (s.size() * CHAR_BIT - 1) / B32_BITS_PER_DIGIT + 1;
+
+    std::string res;
+    res.reserve(len);
+
+    for (ssize_t const n : std::views::iota(0, len) | std::views::reverse) {
+        unsigned int b = n * B32_BITS_PER_DIGIT;
+        unsigned int i = b / CHAR_BIT;
+        unsigned int j = b % CHAR_BIT;
+
+        auto const curChar = s[i];
+        auto const second = i >= s.size() - 1 ? 0 : s[i + 1] << (CHAR_BIT - j);
+        auto const c = (curChar >> j) | second;
+
+        res.push_back(base32Chars[c & 0x1f]);
+    }
+
+    return res;
+}
+
+std::string base32Decode(std::string_view s)
+{
+    if (s.empty()) {
+        return "";
+    }
+
+    std::string res(((s.size() - 1) * 5) / 8 + 1, 0);
+
+    for (unsigned int n = 0; n < s.size(); ++n) {
+        char c = s[s.size() - n - 1];
+        size_t digit;
+        for (digit = 0; digit < base32Chars.size(); ++digit) /* !!! slow */ {
+            if (base32Chars[digit] == c) {
+                break;
+            }
+        }
+        if (digit >= 32) {
+            throw Error("invalid character in base-32 string '%s'", s);
+        }
+        unsigned int b = n * 5;
+        unsigned int i = b / 8;
+        unsigned int j = b % 8;
+        res[i] |= digit << j;
+
+        if (i < res.size() - 1) {
+            res[i + 1] |= digit >> (8 - j);
+        } else {
+            if (digit >> (8 - j)) {
+                throw Error("invalid base-32 string '%s'", s);
+            }
+        }
+    }
+
+    return res;
+}
 
 std::string stripIndentation(std::string_view s)
 {

lix/libutil/strings.hh

@@ -191,12 +191,24 @@ std::string toLower(const std::string & s);
  */
 std::string shellEscape(const std::string_view s);
 
+/**
+ * Same as shellEscape, but also escapes nonprinting characters using $'ANSI C quotes'.
+ */
+std::string bashEscape(const std::string_view s);
+
 /**
  * Base64 encoding/decoding.
  */
 std::string base64Encode(std::string_view s);
 std::string base64Decode(std::string_view s);
 
+/**
+ * Base32 encoding/decoding.
+ */
+extern const std::string base32Chars;
+std::string base32EncodeStr(std::string_view s);
+std::string base32Encode(std::span<std::byte const> const s);
+std::string base32Decode(std::string_view s);
 
 /**
  * Remove common leading whitespace from the lines in the string

lix/libutil/types.hh

@@ -68,13 +68,19 @@ typename T::mapped_type * get(T & map, const typename T::key_type & key)
 /**
  * Get a value for the specified key from an associate container, or a default value if the key isn't present.
  */
-template <class T>
+template<class T>
-const typename T::mapped_type & getOr(T & map,
+const typename T::mapped_type & getOr(
+    T & map [[clang::lifetimebound]],
     const typename T::key_type & key,
-    const typename T::mapped_type & defaultValue)
+    const typename T::mapped_type & defaultValue [[clang::lifetimebound]]
+)
 {
     auto i = map.find(key);
-    if (i == map.end()) return defaultValue;
+    if (i == map.end()) {
+        /* FIXME(Raito): `[[clang::lifetimebound]]` has no effect on `defaultValue` warning. */
+        // NOLINTNEXTLINE(bugprone-return-const-ref-from-parameter)
+        return defaultValue;
+    }
     return i->second;
 }
 

lix/meson.build

@@ -1,7 +1,17 @@
+# Cursed, but I don't think there's another way to get this environment variable.
+lix_suffix = run_command('bash', '-c', 'echo -n "$VERSION_SUFFIX"', check : true).stdout().strip()
+lix_version_parts = meson.project_version().split('.')
+lix_major = lix_version_parts[0]
+lix_minor = lix_version_parts[1]
+lix_patch = lix_version_parts[2].replace(lix_suffix, '')
+
 config_h = configure_file(
   configuration : {
     'PACKAGE_NAME': '"' + meson.project_name() + '"',
     'PACKAGE_VERSION': '"' + meson.project_version() + '"',
+    'LIX_MAJOR': lix_major,
+    'LIX_MINOR': lix_minor,
+    'LIX_PATCH': lix_patch,
     'PACKAGE_TARNAME': '"' + meson.project_name() + '"',
     'PACKAGE_STRING': '"' + meson.project_name() + ' ' + meson.project_version() + '"',
     'HAVE_STRUCT_DIRENT_D_TYPE': 1, # FIXME: actually check this for solaris

lix/nix/daemon.cc

@@ -364,11 +364,17 @@ static void daemonLoopImpl(std::optional<TrustedFlag> forceTrustClientOpt)
                     strncpy(savedArgv[1], processName.c_str(), strlen(savedArgv[1]));
                 }
 
+                auto store = aio.blockOn(openUncachedStore());
+                if (auto local = dynamic_cast<LocalStore *>(&*store); local && peer.uidKnown && peer.gidKnown) {
+                    local->associateWithCredentials(peer.uid, peer.gid);
+                }
+
                 //  Handle the connection.
                 FdSource from(remote.get());
                 FdSink to(remote.get());
+
                 processConnection(
-                    aio, aio.blockOn(openUncachedStore()), from, to, trusted
+                    aio, store, from, to, trusted
                 );
 
                 exit(0);

lix/nix/develop.cc

@@ -6,6 +6,7 @@
 #include "lix/libstore/store-api.hh"
 #include "lix/libstore/outputs-spec.hh"
 #include "lix/libstore/derivations.hh"
+#include "lix/libstore/parsed-derivations.hh"
 #include "lix/libutil/async.hh"
 #include "lix/libutil/json.hh"
 #include "run.hh"
@@ -141,20 +142,20 @@ struct BuildEnvironment
         for (auto & [name, value] : vars) {
             if (!ignoreVars.count(name)) {
                 if (auto str = std::get_if<String>(&value)) {
-                    out << fmt("%s=%s\n", name, shellEscape(str->value));
+                    out << fmt("%s=%s\n", name, bashEscape(str->value));
                     if (str->exported)
                         out << fmt("export %s\n", name);
                 }
                 else if (auto arr = std::get_if<Array>(&value)) {
                     out << "declare -a " << name << "=(";
                     for (auto & s : *arr)
-                        out << shellEscape(s) << " ";
+                        out << bashEscape(s) << " ";
                     out << ")\n";
                 }
                 else if (auto arr = std::get_if<Associative>(&value)) {
                     out << "declare -A " << name << "=(";
                     for (auto & [n, v] : *arr)
-                        out << "[" << shellEscape(n) << "]=" << shellEscape(v) << " ";
+                        out << "[" << bashEscape(n) << "]=" << bashEscape(v) << " ";
                     out << ")\n";
                 }
             }
@@ -214,6 +215,8 @@ const static std::string getEnvSh =
 static kj::Promise<Result<StorePath>> getDerivationEnvironment(ref<Store> store, ref<Store> evalStore, const StorePath & drvPath)
 try {
     auto drv = TRY_AWAIT(evalStore->derivationFromPath(drvPath));
+    ParsedDerivation parsedDrv(drvPath, drv);
+    JSON updatedStructuredAttrs;
 
     auto builder = baseNameOf(drv.builder);
     if (builder != "bash")
@@ -230,6 +233,20 @@ try {
     drv.env.erase("disallowedRequisites");
     drv.env.erase("name");
 
+    /* Remove output checks in structured attrs. */
+    if (auto structuredAttrs = parsedDrv.getStructuredAttrs()) {
+        drv.env.erase("__json");
+        updatedStructuredAttrs = *structuredAttrs;
+        updatedStructuredAttrs.erase("allowedReferences");
+        updatedStructuredAttrs.erase("allowedRequisites");
+        updatedStructuredAttrs.erase("disallowedReferences");
+        updatedStructuredAttrs.erase("disallowedRequisites");
+        updatedStructuredAttrs.erase("maxSize");
+        updatedStructuredAttrs.erase("maxClosureSize");
+        updatedStructuredAttrs.erase("outputChecks");
+        drv.env.emplace("__json", updatedStructuredAttrs.dump());
+    }
+
     /* Rehash and write the derivation. FIXME: would be nice to use
        'buildDerivation', but that's privileged. */
     drv.name += "-env";
@@ -561,7 +578,7 @@ struct CmdDevelop : Common, MixEnvironment
 
         auto [rcFileFd, rcFilePath] = createTempFile("nix-shell");
 
-        AutoDelete tmpDir(createTempDir("", "nix-develop"), true);
+        AutoDelete tmpDir(createTempDir("nix-develop"), true);
 
         auto script = makeRcScript(*state, store, buildEnvironment, (Path) tmpDir);
 
@@ -582,21 +599,24 @@ struct CmdDevelop : Common, MixEnvironment
         else if (!command.empty()) {
             std::vector<std::string> args;
             for (auto s : command)
-                args.push_back(shellEscape(s));
+                args.push_back(bashEscape(s));
             script += fmt("exec %s\n", concatStringsSep(" ", args));
         }
 
         else {
             script = "[ -n \"$PS1\" ] && [ -e ~/.bashrc ] && source ~/.bashrc;\n" + script;
             if (developSettings.bashPrompt != "")
-                script += fmt("[ -n \"$PS1\" ] && PS1=%s;\n",
+                script +=
-                    shellEscape(developSettings.bashPrompt.get()));
+                    fmt("[ -n \"$PS1\" ] && PS1=%s;\n",
+                        bashEscape(developSettings.bashPrompt.get()));
             if (developSettings.bashPromptPrefix != "")
-                script += fmt("[ -n \"$PS1\" ] && PS1=%s\"$PS1\";\n",
+                script +=
-                    shellEscape(developSettings.bashPromptPrefix.get()));
+                    fmt("[ -n \"$PS1\" ] && PS1=%s\"$PS1\";\n",
+                        bashEscape(developSettings.bashPromptPrefix.get()));
             if (developSettings.bashPromptSuffix != "")
-                script += fmt("[ -n \"$PS1\" ] && PS1+=%s;\n",
+                script +=
-                    shellEscape(developSettings.bashPromptSuffix.get()));
+                    fmt("[ -n \"$PS1\" ] && PS1+=%s;\n",
+                        bashEscape(developSettings.bashPromptSuffix.get()));
         }
 
         writeFull(rcFileFd.get(), script);
@@ -698,7 +718,7 @@ struct CmdPrintDevEnv : Common, MixJSON
         if (json) {
             logger->writeToStdout(buildEnvironment.toJSON());
         } else {
-            AutoDelete tmpDir(createTempDir("", "nix-dev-env"), true);
+            AutoDelete tmpDir(createTempDir("nix-dev-env"), true);
             logger->writeToStdout(makeRcScript(*state, store, buildEnvironment, tmpDir));
         }
     }

lix/nix/get-env.sh

@@ -12,6 +12,31 @@ fi
 __vars="$(declare -p)"
 __functions="$(declare -F)"
 
+# Literal control characters (ASCII 0-31) aren't valid JSON.
+__escapeCtrl() {
+    local escaped="$1"
+
+    # I don't know if NUL bytes are at ALL possible in here,
+    # but covering them is free.
+    local i=0
+    # NOTE: safe input `i` to arithmetic expansion.
+    while [[ "$i" -le 32 ]]; do
+        # Convert the decimal ASCII value to its actual string.
+        local asHex; printf -v asHex "%02x" "$i"
+        local asStr; printf -v asStr "%b" "\x$asHex"
+
+        # Format it to \uXXXX.
+        # All control characters fit within four hex digits.
+        local asUni; printf -v asUni '\\u%04x' "$i"
+
+        escaped="${escaped//"$asStr"/"$asUni"}"
+
+        i="$((i + 1))"
+    done
+
+    printf "%s" "$escaped"
+}
+
 __dumpEnv() {
     printf '{\n'
 
@@ -125,6 +150,7 @@ __escapeString() {
     __s="${__s//$'\n'/\\n}"
     __s="${__s//$'\r'/\\r}"
     __s="${__s//$'\t'/\\t}"
+    __s="$(__escapeCtrl "$__s")"
     printf '"%s"' "$__s"
 }
 

lix/nix/meson.build

@@ -177,6 +177,7 @@ nix = executable(
     boehm,
     nlohmann_json,
     kj,
+    libatomic,
   ],
   cpp_pch : cpp_pch,
   install : true,

meson.build

@@ -265,6 +265,11 @@ configdata += {
   'HAVE_SECCOMP': seccomp.found().to_int(),
 }
 
+# fcntl(F_GETPATH) returns the path of an fd on macOS and BSDs
+configdata += {
+  'HAVE_F_GETPATH': cxx.has_header_symbol('fcntl.h', 'F_GETPATH').to_int(),
+}
+
 libarchive = dependency('libarchive', required : true, include_type : 'system')
 
 brotli = [
@@ -335,6 +340,16 @@ editline = dependency('libeditline', 'editline', version : '>=1.14', required :
 
 lowdown = dependency('lowdown', version : '>=0.9.0', required : true, include_type : 'system')
 
+# TODO(sterni): drop the corresponding #ifdef after NixOS 25.05 is EOL which still distributes lowdown < 1.4.0
+if lowdown.version().version_compare('>= 1.4.0')
+  add_project_arguments('-DLOWDOWN_SEPARATE_TERM_OPTS', language: 'cpp')
+endif
+
+# TODO(sterni): drop the corresponding #ifdef after NixOS 25.11 is EOL which still distributes lowdown < 3.0.0
+if lowdown.version().version_compare('>= 3.0.0')
+  add_project_arguments('-DLOWDOWN_CONSOLIDATED_OFLAGS', language: 'cpp')
+endif
+
 # HACK(Qyriad): rapidcheck's pkg-config doesn't include the libs lol
 # Note: technically we 'check' for rapidcheck twice, for the internal-api-docs handling above,
 # but Meson will cache the result of the first one, and the required : arguments are different.
@@ -349,6 +364,9 @@ gtest = [
 ]
 
 toml11 = dependency('toml11', version : '>=3.7.0', required : true, method : 'cmake', include_type : 'system')
+configdata += {
+  'HAVE_TOML11_4': toml11.version().version_compare('>= 4.0.0').to_int(),
+}
 
 pegtl = dependency(
   'pegtl',
@@ -365,6 +383,26 @@ if is_freebsd
   libprocstat = declare_dependency(link_args : [ '-lprocstat' ])
 endif
 
+libatomic_test_program = '''
+#include <atomic>
+int main() {
+  std::atomic<uint8_t> w1;
+  std::atomic<uint16_t> w2;
+  std::atomic<uint32_t> w4;
+  std::atomic<uint64_t> w8;
+  return ++w1 + ++w2 + ++w4 + ++w8;
+}
+'''
+
+libatomic = cxx.find_library('atomic', required : false)
+
+# Some platforms like 32-Bit PowerPC need libatomic because they're lacking 64-Bit hardware atomic instructions
+# and compilers don't handle this automatically (yet).
+# See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81358 and https://clang.llvm.org/docs/Toolchain.html#atomics-library
+if not cxx.links(libatomic_test_program, name : 'test if simple atomic program links')
+  libatomic = cxx.find_library('atomic', required : true)
+endif
+
 #
 # Build-time tools
 #
@@ -446,6 +484,13 @@ configdata += {
   'HAVE_DTRACE': dtrace_feature.enabled().to_int(),
 }
 
+pasta_path = get_option('pasta-path')
+# we can't check the pasta version because passt misuses stdio (it calls _exit()
+# after printing the version, which will never print the version unless run from
+# a terminal). pasta isn't mandatory yet due to high fetcher breakage potential.
+# we *will* enable it in our own packaging, but distributions are not forced to.
+pasta = find_program(pasta_path, required : false, native : false)
+
 lsof = find_program('lsof', native : true)
 
 # This is how Nix does generated headers...

meson.options

@@ -24,6 +24,10 @@ option('sandbox-shell', type : 'string', value : 'busybox',
   description : 'path to a statically-linked shell to use as /bin/sh in sandboxes (usually busybox)',
 )
 
+option('pasta-path', type : 'string', value : 'pasta',
+  description : 'path to the location of pasta (provided by passt)',
+)
+
 option('enable-tests', type : 'boolean', value : true,
   description : 'whether to enable tests or not (requires rapidcheck and gtest)',
 )

misc/capnproto.nix

@@ -46,7 +46,8 @@ stdenv.mkDerivation rec {
   propagatedBuildInputs = [
     openssl
     zlib
-  ] ++ lib.optional (stdenv.cc.isClang && stdenv.targetPlatform.isStatic) empty-libgcc_eh;
+  ]
+  ++ lib.optional (stdenv.cc.isClang && stdenv.targetPlatform.isStatic) empty-libgcc_eh;
 
   # FIXME: separate the binaries from the stuff that user systems actually use
   # This runs into a terrible UX issue in Lix and I just don't want to debug it

misc/passt.nix

@@ -0,0 +1,64 @@
+{
+  lib,
+  stdenv,
+  buildPackages,
+  fetchurl,
+  getconf,
+  gitUpdater,
+  testers,
+}:
+
+stdenv.mkDerivation (finalAttrs: {
+  pname = "passt";
+  version = "2025_02_17.a1e48a0";
+
+  src = fetchurl {
+    url = "https://passt.top/passt/snapshot/passt-${finalAttrs.version}.tar.gz";
+    hash = "sha256-/FUXxeYv3Lb0DiXmbS2PUzfLL5ZwHJ42tiuH7YnlljE=";
+  };
+
+  postPatch = ''
+    substituteInPlace Makefile --replace-fail \
+      'PAGE_SIZE=$(shell getconf PAGE_SIZE)' \
+      "PAGE_SIZE=$(${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe getconf} PAGE_SIZE)"
+  '';
+
+  makeFlags = [
+    "prefix=${placeholder "out"}"
+    "VERSION=${finalAttrs.version}"
+  ];
+
+  passthru = {
+    tests.version = testers.testVersion {
+      package = finalAttrs.finalPackage;
+    };
+
+    updateScript = gitUpdater {
+      url = "https://passt.top/passt";
+    };
+  };
+
+  meta = with lib; {
+    homepage = "https://passt.top/passt/about/";
+    description = "Plug A Simple Socket Transport";
+    longDescription = ''
+      passt implements a translation layer between a Layer-2 network interface
+      and native Layer-4 sockets (TCP, UDP, ICMP/ICMPv6 echo) on a host.
+      It doesn't require any capabilities or privileges, and it can be used as
+      a simple replacement for Slirp.
+
+      pasta (same binary as passt, different command) offers equivalent
+      functionality, for network namespaces: traffic is forwarded using a tap
+      interface inside the namespace, without the need to create further
+      interfaces on the host, hence not requiring any capabilities or
+      privileges.
+    '';
+    license = [
+      licenses.bsd3 # and
+      licenses.gpl2Plus
+    ];
+    platforms = platforms.linux;
+    maintainers = with maintainers; [ _8aed ];
+    mainProgram = "passt";
+  };
+})

misc/systemd/nix-daemon.conf.in

@@ -1 +1,4 @@
 d @localstatedir@/nix/daemon-socket 0755 root root -  -
+d @localstatedir@/nix/b             0755 root root 7d -
+# TODO: Remove this after (at least) August 2026.
+R! @localstatedir@/nix/builds       -    -    -    -  -

package.nix

@@ -45,6 +45,8 @@
   ninja,
   ncurses,
   openssl,
+  # FIXME: we need passt 2024_12_11.09478d5 or newer, i.e. nixos 25.05 or later
+  passt-lix ? __forDefaults.passt-lix,
   pegtl,
   pkg-config,
   python3,
@@ -116,6 +118,8 @@
     # needs derivation patching to add debuginfo and coroutine library support
     # !! must build this with clang as it is affected by the gcc coroutine bugs
     capnproto-lix = callPackage ./misc/capnproto.nix { inherit stdenv; };
+
+    passt-lix = callPackage ./misc/passt.nix { };
   },
 }:
 
@@ -131,10 +135,10 @@ let
 
   # This could be the dtrace for macOS, etc, but I have no idea if it is
   # packaged or if it works.
-  dtrace-generator = lib.optional withDtrace systemtap-lix;
+  dtrace-generator = if withDtrace then systemtap-lix else null;
 
   # This is for sys/sdt.h
-  dtrace-headers = lib.optional withDtrace libsystemtap;
+  dtrace-headers = if withDtrace then libsystemtap else null;
 
   aws-sdk-cpp-nix =
     if aws-sdk-cpp == null then
@@ -205,6 +209,8 @@ assert (lintInsteadOfBuild -> lix-clang-tidy != null);
 stdenv.mkDerivation (finalAttrs: {
   inherit pname version;
 
+  __structuredAttrs = true;
+
   src = fileset.toSource {
     root = ./.;
     fileset = fileset.intersection baseFiles (
@@ -225,10 +231,9 @@ stdenv.mkDerivation (finalAttrs: {
     );
   };
 
-  VERSION_SUFFIX = versionSuffix;
+  outputs = [
-
+    "out"
-  outputs =
+  ]
-    [ "out" ]
   ++ lib.optionals (!finalAttrs.dontBuild) [
     "dev"
     "doc"
@@ -247,6 +252,7 @@ stdenv.mkDerivation (finalAttrs: {
       # which don't actually get added to PATH. And buildInputs is correct over
       # nativeBuildInputs since this should be a busybox executable on the host.
       "-Dsandbox-shell=${lib.getExe' busybox-sandbox-shell "busybox"}"
+      "-Dpasta-path=${lib.getExe' passt-lix "pasta"}"
     ]
     ++ lib.optional hostPlatform.isStatic "-Denable-embedded-sandbox-shell=true"
     ++ lib.optional (finalAttrs.dontBuild && !lintInsteadOfBuild) "-Denable-build=false"
@@ -263,13 +269,14 @@ stdenv.mkDerivation (finalAttrs: {
       (lib.mesonBool "werror" werror)
     ]
     ++ lib.optional (hostPlatform != buildPlatform) "--cross-file=${mesonCrossFile}"
+    # Temporary workaround for https://git.lix.systems/lix-project/lix/issues/832
+    ++ lib.optional (hostPlatform.isDarwin) "-Db_lto=false"
     ++ sanitizeOpts;
 
   # We only include CMake so that Meson can locate toml11, which only ships CMake dependency metadata.
   dontUseCmakeConfigure = true;
 
-  nativeBuildInputs =
+  nativeBuildInputs = [
-    [
     lixPythonForBuild
     meson
     ninja
@@ -309,8 +316,7 @@ stdenv.mkDerivation (finalAttrs: {
     llvmPackages.clang-unwrapped
   ];
 
-  buildInputs =
+  buildInputs = [
-    [
     curl
     bzip2
     xz
@@ -330,6 +336,7 @@ stdenv.mkDerivation (finalAttrs: {
   ++ lib.optionals hostPlatform.isLinux [
     libseccomp
     busybox-sandbox-shell
+    passt-lix
   ]
   ++ lib.optionals (
     stdenv.hostPlatform.isDarwin && lib.versionOlder stdenv.hostPlatform.darwinSdkVersion "11.0"
@@ -356,9 +363,9 @@ stdenv.mkDerivation (finalAttrs: {
     lixPythonForBuild
   ];
 
+  env = {
     # Needed for Meson to find Boost.
     # https://github.com/NixOS/nixpkgs/issues/86131.
-  env = {
     BOOST_INCLUDEDIR = "${lib.getDev boost}/include";
     BOOST_LIBRARYDIR = "${lib.getLib boost}/lib";
 
@@ -366,6 +373,8 @@ stdenv.mkDerivation (finalAttrs: {
     # Turns out the Nix-generated Cargo dependencies are named the same as they
     # would be in a Cargo registry cache.
     MESON_PACKAGE_CACHE_DIR = finalAttrs.cargoDeps;
+
+    VERSION_SUFFIX = versionSuffix;
   };
 
   cargoDeps = rustPlatform.importCargoLock { lockFile = ./Cargo.lock; };

perl/default.nix

@@ -35,8 +35,7 @@ perl.pkgs.toPerlModule (
       ninja
     ];
 
-    buildInputs =
+    buildInputs = [
-      [
       nix
       curl
       bzip2

releng/create_release.xsh

@@ -40,7 +40,7 @@ def setup_creds(env: RelengEnvironment):
     key = keys.get_ephemeral_key(env)
     $AWS_SECRET_ACCESS_KEY = key.secret_key
     $AWS_ACCESS_KEY_ID = key.id
-    $AWS_DEFAULT_REGION = 'garage'
+    $AWS_DEFAULT_REGION = env.s3_region
     $AWS_ENDPOINT_URL = env.s3_endpoint
 
 

releng/environment.py

@@ -5,11 +5,12 @@ import functools
 import subprocess
 import dataclasses
 
-S3_HOST = 's3.lix.systems'
+S3_HOST = 's3-admin.afnix.fr'
+S3_USER = 'lix-releng'
 
 DEFAULT_STORE_URI_BITS = {
-    'region': 'garage',
+    'region': 'global',
-    'endpoint': 's3.lix.systems',
+    'endpoint': 's3.afnix.fr',
     'want-mass-query': 'true',
     'write-nar-listing': 'true',
     'ls-compression': 'zstd',
@@ -54,7 +55,9 @@ class RelengEnvironment:
     git_repo: Callable[[], str]
     git_repo_is_gerrit: bool
     s3_endpoint: str
+    s3_region: str
     s3_ssh_host: str | None
+    s3_ssh_user: str | None
 
     docker_targets: list[DockerTarget]
 
@@ -86,17 +89,19 @@ LOCAL = RelengEnvironment(
     git_repo_is_gerrit=False,
     docker_targets=[],
     s3_endpoint = 'http://localhost:3900',
+    s3_region = 'garage',
     s3_ssh_host = None,
+    s3_ssh_user = None,
 )
 
 
 STAGING = RelengEnvironment(
     name='staging',
     colour=functools.partial(sgr, GREEN),
-    docs_bucket='s3://staging-docs',
+    docs_bucket='s3://docs.staging.lix.systems',
-    cache_bucket='s3://staging-cache',
+    cache_bucket='s3://cache.staging.lix.systems',
     cache_store_overlay={'secret-key': 'staging.key'},
-    releases_bucket='s3://staging-releases',
+    releases_bucket='s3://releases.staging.lix.systems',
     git_repo=lambda: 'ssh://git@git.lix.systems/lix-project/lix-releng-staging',
     git_repo_is_gerrit=False,
     docker_targets=[
@@ -106,8 +111,10 @@ STAGING = RelengEnvironment(
         DockerTarget('ghcr.io/lix-project/lix-releng-staging',
                      tags=['{version}', '{major}']),
     ],
-    s3_endpoint = 'https://s3.lix.systems',
+    s3_endpoint = 'https://s3.afnix.fr',
+    s3_region = 'garage',
     s3_ssh_host = S3_HOST,
+    s3_ssh_user = S3_USER,
 )
 
 GERRIT_REMOTE_RE = re.compile(r'^ssh://(\w+@)?gerrit.lix.systems:2022/lix$')
@@ -127,13 +134,13 @@ def guess_gerrit_remote():
 PROD = RelengEnvironment(
     name='production',
     colour=functools.partial(sgr, RED),
-    docs_bucket='s3://docs',
+    docs_bucket='s3://docs.lix.systems',
-    cache_bucket='s3://cache',
+    cache_bucket='s3://cache.lix.systems',
     # FIXME: we should decrypt this with age into a tempdir in the future, but
     # the issue is how to deal with the recipients file. For now, we should
     # just delete it after doing a release.
     cache_store_overlay={'secret-key': 'prod.key'},
-    releases_bucket='s3://releases',
+    releases_bucket='s3://releases.lix.systems',
     git_repo=guess_gerrit_remote,
     git_repo_is_gerrit=True,
     docker_targets=[
@@ -142,8 +149,10 @@ PROD = RelengEnvironment(
                      tags=['{version}', '{major}']),
         DockerTarget('ghcr.io/lix-project/lix', tags=['{version}', '{major}']),
     ],
-    s3_endpoint = 'https://s3.lix.systems',
+    s3_endpoint = 'https://s3.afnix.fr',
+    s3_region = 'global',
     s3_ssh_host = S3_HOST,
+    s3_ssh_user = S3_USER,
 )
 
 ENVIRONMENTS = {

releng/garage-ephemeral-key/garage-ephemeral-key.py

@@ -1,12 +1,12 @@
 # SPDX-FileCopyrightText: 2024 Jade Lovelace
+# SPDX-FileCopyrightText: 2026 Yureka Lilian <yureka@cyberchaos.dev>
 # SPDX-License-Identifier: MIT
 import argparse
 import json
 import sys
 import datetime
-import dataclasses
 import re
-from typing import Any, Literal, Optional
+from typing import Any
 import requests
 import os
 import logging
@@ -14,27 +14,34 @@ import logging
 log = logging.getLogger(__name__)
 log.setLevel(logging.INFO)
 
-fmt = logging.Formatter('{asctime} {levelname} {name}: {message}',
+fmt = logging.Formatter(
-                        datefmt='%b %d %H:%M:%S',
+    "{asctime} {levelname} {name}: {message}",
-                        style='{')
+    datefmt="%b %d %H:%M:%S",
+    style="{",
+)
 
 if not any(isinstance(h, logging.StreamHandler) for h in log.handlers):
     hand = logging.StreamHandler()
     hand.setFormatter(fmt)
     log.addHandler(hand)
 
-API_BASE = os.environ.get('GARAGE_ADMIN_API_BASE', 'http://localhost:3903')
+API_BASE = os.environ.get("GARAGE_ADMIN_API_BASE", "http://localhost:3903")
-API_KEY = os.environ['GARAGE_ADMIN_TOKEN']
+API_KEY = os.environ["GARAGE_ADMIN_TOKEN"]
+
+BUCKET_REGEX_STR = os.environ.get("BUCKET_REGEX", ".*")
+BUCKET_REGEX = re.compile(BUCKET_REGEX_STR)
 
 
 def api(method, endpoint: str, resp_json=True, **kwargs) -> Any:
-    log.info('http %s %s', method, endpoint)
+    log.info("http %s %s", method, endpoint)
-    if not endpoint.startswith('https'):
+    if not endpoint.startswith("https"):
         endpoint = API_BASE + endpoint
-    resp = requests.request(method,
+    resp = requests.request(
+        method,
         endpoint,
-                            headers={'Authorization': f'Bearer {API_KEY}'},
+        headers={"Authorization": f"Bearer {API_KEY}"},
-                            **kwargs)
+        **kwargs,
+    )
     resp.raise_for_status()
     if resp_json:
         return resp.json()
@@ -42,97 +49,64 @@ def api(method, endpoint: str, resp_json=True, **kwargs) -> Any:
         return resp
 
 
-@dataclasses.dataclass
+def get_bucket_id(bucket_name: str) -> str:
-class Key:
+    resp: dict = api(
-    name: str
+        "GET", "/v2/GetBucketInfo", params={"globalAlias": bucket_name}
-    id: str
+    )
-    secret_key: Optional[str] = None
+    return resp["id"]
 
 
-@dataclasses.dataclass
+DATEFMT = "%Y%m%d%H%M%S"
-class Bucket:
-    id: str
-
-
-def keys() -> list[Key]:
-    data: list[dict] = api('GET', '/v1/key?list')
-    return [Key(name=k['name'], id=k['id']) for k in data]
-
-
-def delete_key(key: Key):
-    api('DELETE', '/v1/key', resp_json=False, params={'id': key.id})
-
-
-def create_key(name: str) -> Key:
-    resp: dict = api('POST', '/v1/key', json={'name': name})
-    return Key(name=resp['name'],
-               id=resp['accessKeyId'],
-               secret_key=resp['secretAccessKey'])
-
-
-AccessType = Literal['read'] | Literal['write'] | Literal['owner']
-
-
-def get_bucket(bucket_name: str) -> Bucket:
-    resp: dict = api('GET', '/v1/bucket', params={'globalAlias': bucket_name})
-    return Bucket(resp['id'])
-
-
-def grant(bucket: Bucket, access_types: list[AccessType], key: Key):
-    access_types_dict = {k: True for k in access_types}
-    api('POST',
-        '/v1/bucket/allow',
-        json={
-            'bucketId': bucket.id,
-            'accessKeyId': key.id,
-            'permissions': access_types_dict,
-        })
-
-
-KEY_RE = re.compile(r'^.*ephemeral-(\d{14})$')
-DATEFMT = '%Y%m%d%H%M%S'
-
-
-def expired_keys(older_than: datetime.datetime) -> list[Key]:
-    ret = []
-    for key in keys():
-        if m := KEY_RE.match(key.name):
-            date = datetime.datetime.strptime(m.group(1), DATEFMT)
-            date = date.astimezone(datetime.UTC)
-            print(date)
-            if date < older_than:
-                ret.append(key)
-    return ret
 
 
 def do_new(args):
-    buckets = [get_bucket(b) for b in args.buckets]
+    for b in args.buckets:
+        if not BUCKET_REGEX.match(b):
+            print(f"Bucket {b} not in allowed buckeds '{BUCKET_REGEX_STR}'")
+            exit(1)
+    bucket_ids = [get_bucket_id(b) for b in args.buckets]
 
-    def optional(s: str, whether) -> list[str]:
+    key_name = args.name + "-" if args.name else ""
-        if whether:
+    expiration = datetime.datetime.now(tz=datetime.UTC) + datetime.timedelta(
-            return [s]
+        seconds=args.age_secs
-        else:
+    )
-            return []
+    key_name += "ephemeral-" + expiration.strftime(DATEFMT)
 
-    access_types: list[AccessType] = optional('read', args.read) + optional(
+    key_resp: dict = api(
-        'write', args.write) + optional('owner', args.owner)  # type: ignore
+        "POST",
+        "/v2/CreateKey",
+        json={
+            "name": key_name,
+            "expiration": expiration.isoformat(),
+            "neverExpires": False,
+        },
+    )
 
-    key_name = args.name + '-' if args.name else ''
+    for b in bucket_ids:
-    key_name += "ephemeral-" + (
+        api(
-        datetime.datetime.now(tz=datetime.UTC) +
+            "POST",
-        datetime.timedelta(seconds=args.age_secs)).strftime(DATEFMT)
+            "/v2/AllowBucketKey",
+            json={
+                "accessKeyId": key_resp["accessKeyId"],
+                "bucketId": b,
+                "permissions": {
+                    "read": args.read,
+                    "write": args.write,
+                    "owner": args.owner,
+                },
+            },
+        )
 
-    k = create_key(key_name)
+    print(
-    for b in buckets:
+        json.dumps(
-        grant(b, access_types, k)
+            {
-
+                "name": key_resp["name"],
-    print(json.dumps(dataclasses.asdict(k), indent=2))
+                "id": key_resp["accessKeyId"],
-
+                "secret_key": key_resp["secretAccessKey"],
-
+            },
-def do_clean(args):
+            indent=2,
-    older_than = datetime.datetime.now(tz=datetime.UTC)
+        )
-    for key in expired_keys(older_than):
+    )
-        delete_key(key)
 
 
 def main():
@@ -148,28 +122,27 @@ def main():
 
     new = sps.add_parser("new", help="Make an ephemeral key")
     new.add_argument("--name", help="Name prefix for the key")
-    new.add_argument("--read",
+    new.add_argument(
-                     action="store_true",
+        "--read", action="store_true", help="Grant read access to buckets"
-                     help="Grant read access to buckets")
+    )
-    new.add_argument("--write",
+    new.add_argument(
-                     action="store_true",
+        "--write", action="store_true", help="Grant write access to buckets"
-                     help="Grant write access to buckets")
+    )
-    new.add_argument("--owner",
+    new.add_argument(
-                     action="store_true",
+        "--owner", action="store_true", help="Grant owner access to buckets"
-                     help="Grant owner access to buckets")
+    )
-    new.add_argument("--age-secs",
+    new.add_argument(
+        "--age-secs",
         type=int,
         required=True,
-                     help="Maximum key lifetime in seconds")
+        help="Maximum key lifetime in seconds",
-    new.add_argument("buckets", nargs='*', help="Buckets to grant access to")
+    )
+    new.add_argument("buckets", nargs="*", help="Buckets to grant access to")
     new.set_defaults(cmd=do_new)
 
-    clean = sps.add_parser("clean", help="Clean up old keys")
-    clean.set_defaults(cmd=do_clean)
-
     args = ap.parse_args()
     args.cmd(args)
 
 
-if __name__ == '__main__':
+if __name__ == "__main__":
     main()

releng/keys.py

@@ -14,7 +14,7 @@ def get_ephemeral_key(
         env.docs_bucket.removeprefix('s3://'),
     ]
     if env.s3_ssh_host is not None:
-        command = ['ssh', '-l', 'root', env.s3_ssh_host, *command]
+        command = ['ssh', f'{env.s3_ssh_user}@{env.s3_ssh_host}', *command]
     output = subprocess.check_output(command)
     d = json.loads(output.decode())
     return environment.S3Credentials(name=d['name'],

releng/local/default.nix

@@ -65,6 +65,12 @@ in
         environment.sessionVariables = {
           GARAGE_ADMIN_TOKEN = "UkLeGWEvHnXBqnueR3ISEMWpOnm40jH2tM2HnnL/0F4=";
         };
+
+        # ≥ v6.12 kernel has a system wide corruption related to 9p. wait until
+        # https://lore.kernel.org/all/w5ap2zcsatkx4dmakrkjmaexwh3mnmgc5vhavb2miaj6grrzat@7kzr5vlsrmh5/
+        # resolves. once this is resolved and the fix lands in a stable kernel
+        # in nixpkgs, this pin can be removed.
+        boot.kernelPackages = pkgs.linuxPackages_6_6;
       };
   };
   testScript = ''

tests/functional/build-remote-trustless-should-fail-0.sh

@@ -8,7 +8,6 @@ requireSandboxSupport
 [[ $busybox =~ busybox ]] || skipTest "no busybox"
 
 unset NIX_STORE_DIR
-unset NIX_STATE_DIR
 
 # We first build a dependency of the derivation we eventually want to
 # build.

tests/functional/build-remote-trustless.sh

@@ -2,7 +2,6 @@ requireSandboxSupport
 [[ $busybox =~ busybox ]] || skipTest "no busybox"
 
 unset NIX_STORE_DIR
-unset NIX_STATE_DIR
 
 remoteDir=$TEST_ROOT/remote
 

tests/functional/build-remote.sh

@@ -3,7 +3,6 @@ requireSandboxSupport
 
 # Avoid store dir being inside sandbox build-dir
 unset NIX_STORE_DIR
-unset NIX_STATE_DIR
 
 function join_by { local d=$1; shift; echo -n "$1"; shift; printf "%s" "${@/#/$d}"; }
 
@@ -81,4 +80,4 @@ out="$(nix-build 2>&1 failing.nix \
 [[ "$out" =~ .*"note: keeping build directory".* ]]
 
 build_dir="$(grep "note: keeping build" <<< "$out" | sed -E "s/^(.*)note: keeping build directory '(.*)'(.*)$/\2/")"
-[[ "foo" = $(<"$build_dir"/bar) ]]
+[[ "foo" = $(<"$build_dir"/b/bar) ]]

tests/functional/build.sh

@@ -174,3 +174,28 @@ test "$(<<<"$out" grep -E '^error:' | wc -l)" = 3
 <<<"$out" grepQuiet -E "error: 2 dependencies of derivation '.*-x4\\.drv' failed to build"
 <<<"$out" grepQuiet -vE "hash mismatch in fixed-output derivation '.*-x3\\.drv'"
 <<<"$out" grepQuiet -vE "hash mismatch in fixed-output derivation '.*-x2\\.drv'"
+
+# Ensure when if the system build dir is inaccessible, we can still build things
+BUILD_DIR=$(mktemp -d)
+chmod 0000 "$BUILD_DIR"
+nix --build-dir "$BUILD_DIR" build -E 'with import ./config.nix; mkDerivation { name = "test"; buildCommand = "echo rawr > $out"; }' --impure --no-link
+
+# ensure that the build directory parent is not world-accessible
+chmod 0755 "$BUILD_DIR"
+FIFO="$BUILD_DIR/fifo"
+mkfifo "$FIFO"
+(
+  echo > "$FIFO"
+  trap 'echo > "$FIFO"' EXIT
+  mode=$(stat -c %a $BUILD_DIR/b/*)
+  [ "$mode" = "700" -o "$mode" = "710" ]
+) &
+nix build --build-dir "$BUILD_DIR/b" -E '
+  with import ./config.nix; mkDerivation {
+    name = "test";
+    buildCommand = "cat '"$FIFO"'; cat '"$FIFO"' > $out";
+  }' \
+  --extra-sandbox-paths "$FIFO" \
+  --impure \
+  --no-link
+wait

tests/functional/check.sh

@@ -37,39 +37,16 @@ if checkBuildTempDirRemoved $TEST_ROOT/log; then false; fi
 test_custom_build_dir() {
   local customBuildDir="$TEST_ROOT/custom-build-dir"
 
-  # Nix does not create the parent directories, and perhaps it shouldn't try to
-  # decide the permissions of build-dir.
-  mkdir "$customBuildDir"
   nix-build check.nix -A failed --argstr checkBuildId $checkBuildId \
       --no-out-link --keep-failed --option build-dir "$TEST_ROOT/custom-build-dir" 2> $TEST_ROOT/log || status=$?
   [ "$status" = "100" ]
-  [[ 1 == "$(count "$customBuildDir/nix-build-"*)" ]]
+  [[ 1 == "$(count "$customBuildDir/"*)" ]]
-  local buildDir="$customBuildDir/nix-build-"*
+  local buildDir="$customBuildDir/"*
-  grep $checkBuildId $buildDir/checkBuildId
+  grep $checkBuildId $buildDir/checkBuildId \
+    || grep $checkBuildId $buildDir/b/checkBuildId
 }
 test_custom_build_dir
 
-test_custom_temp_dir() {
-  # like test_custom_build_dir(), but uses the temp-dir setting instead
-  # build-dir inherits from temp-dir when build-dir is unset
-  local customTempDir="$TEST_ROOT/custom-temp-dir"
-
-  mkdir "$customTempDir"
-  nix-build check.nix -A failed --argstr checkBuildId $checkBuildId \
-      --no-out-link --keep-failed --option temp-dir "$customTempDir" 2> $TEST_ROOT/log || status=$?
-  [ "$status" = "100" ]
-  [[ 1 == "$(count "$customTempDir/nix-build-"*)" ]]
-  local buildDir="$customTempDir/nix-build-"*
-  grep $checkBuildId $buildDir/checkBuildId
-
-  # also check a separate code path that doesn't involve build-dir
-  # nix-shell uses temp-dir for its rcfile path
-  rcpath=$(NIX_BUILD_SHELL=$SHELL nix-shell check.nix -A deterministic --option temp-dir "$customTempDir" --run 'echo $0' 2> $TEST_ROOT/log)
-  # rcpath is <temp-dir>/nix-shell-*/rc
-  [[ $rcpath = "$customTempDir"/* ]]
-}
-test_custom_temp_dir
-
 test_shell_preserves_tmpdir() {
   # ensure commands that spawn interactive shells don't overwrite TMPDIR with temp-dir
   local envTempDir=$TEST_ROOT/shell-temp-dir-env

tests/functional/gc-auto.sh

@@ -4,6 +4,13 @@ needLocalStore "“min-free” and “max-free” are daemon options"
 
 clearStore
 
+fake_free=$TEST_ROOT/fake-free
+export _NIX_TEST_FREE_SPACE_FILE=$fake_free
+echo 1100 > $fake_free
+
+# Check that auto-GC during evaluation progresses.
+timeout --signal=KILL 10s nix eval --expr 'builtins.toFile "meow" "meow"' --min-free 2000
+
 garbage1=$(nix store add-path --name garbage1 ./nar-access.sh)
 garbage2=$(nix store add-path --name garbage2 ./nar-access.sh)
 garbage3=$(nix store add-path --name garbage3 ./nar-access.sh)
@@ -11,10 +18,6 @@ garbage3=$(nix store add-path --name garbage3 ./nar-access.sh)
 ls -l $garbage3
 POSIXLY_CORRECT=1 du $garbage3
 
-fake_free=$TEST_ROOT/fake-free
-export _NIX_TEST_FREE_SPACE_FILE=$fake_free
-echo 1100 > $fake_free
-
 fifoLock=$TEST_ROOT/fifoLock
 mkfifo "$fifoLock"
 

tests/functional/lang/eval-fail-fromTOML-timestamps.nix

@@ -55,11 +55,53 @@ builtins.fromTOML ''
   odt2 = 1979-05-27T00:32:00-07:00
   odt3 = 1979-05-27T00:32:00.999999-07:00
   odt4 = 1979-05-27 07:32:00Z
+  # milliseconds
+  odt5 = 1979-05-27 07:32:00.1Z
+  odt6 = 1979-05-27 07:32:00.12Z
+  odt7 = 1979-05-27 07:32:00.123Z
+  # microseconds
+  odt8 = 1979-05-27t07:32:00.1234Z
+  odt9 = 1979-05-27t07:32:00.12345Z
+  odt10 = 1979-05-27t07:32:00.123456Z
+  # nanoseconds
+  odt11 = 1979-05-27 07:32:00.1234567Z
+  odt12 = 1979-05-27 07:32:00.12345678Z
+  odt13 = 1979-05-27 07:32:00.123456789Z
+  # no more precision after nanoseconds
+  odt14 = 1979-05-27t07:32:00.1234567891Z
+
   ldt1 = 1979-05-27T07:32:00
-  ldt2 = 1979-05-27T00:32:00.999999
+  # milliseconds
+  ldt2 = 1979-05-27T07:32:00.1
+  ldt3 = 1979-05-27T07:32:00.12
+  ldt4 = 1979-05-27T07:32:00.123
+  # microseconds
+  ldt5 = 1979-05-27t00:32:00.1234
+  ldt6 = 1979-05-27t00:32:00.12345
+  ldt7 = 1979-05-27t00:32:00.123456
+  # nanoseconds
+  ldt8 = 1979-05-27 00:32:00.1234567
+  ldt9 = 1979-05-27 00:32:00.12345678
+  ldt10 = 1979-05-27 00:32:00.123456789
+  # no more precision after nanoseconds
+  ldt11 = 1979-05-27t00:32:00.1234567891
+
   ld1 = 1979-05-27
   lt1 = 07:32:00
-  lt2 = 00:32:00.999999
+  # milliseconds
+  lt2 = 00:32:00.1
+  lt3 = 00:32:00.12
+  lt4 = 00:32:00.123
+  # microseconds
+  lt5 = 00:32:00.1234
+  lt6 = 00:32:00.12345
+  lt7 = 00:32:00.123456
+  # nanoseconds
+  lt8 = 00:32:00.1234567
+  lt9 = 00:32:00.12345678
+  lt10 = 00:32:00.123456789
+  # no more precision after nanoseconds
+  lt11 = 00:32:00.1234567891
 
   arr1 = [ 1, 2, 3 ]
   arr2 = [ "red", "yellow", "green" ]

tests/functional/lang/eval-okay-fromTOML-timestamps.exp

@@ -1 +1 @@
-{ "1234" = "value"; "127.0.0.1" = "value"; a = { b = { c = { }; }; }; arr1 = [ 1 2 3 ]; arr2 = [ "red" "yellow" "green" ]; arr3 = [ [ 1 2 ] [ 3 4 5 ] ]; arr4 = [ "all" "strings" "are the same" "type" ]; arr5 = [ [ 1 2 ] [ "a" "b" "c" ] ]; arr7 = [ 1 2 3 ]; arr8 = [ 1 2 ]; bare-key = "value"; bare_key = "value"; bin1 = 214; bool1 = true; bool2 = false; "character encoding" = "value"; d = { e = { f = { }; }; }; dog = { "tater.man" = { type = { name = "pug"; }; }; }; flt1 = 1; flt2 = 3.1415; flt3 = -0.01; flt4 = 5e+22; flt5 = 1e+06; flt6 = -0.02; flt7 = 6.626e-34; flt8 = 9.22462e+06; fruit = [ { name = "apple"; physical = { color = "red"; shape = "round"; }; variety = [ { name = "red delicious"; } { name = "granny smith"; } ]; } { name = "banana"; variety = [ { name = "plantain"; } ]; } ]; g = { h = { i = { }; }; }; hex1 = 3735928559; hex2 = 3735928559; hex3 = 3735928559; int1 = 99; int2 = 42; int3 = 0; int4 = -17; int5 = 1000; int6 = 5349221; int7 = 12345; j = { "ʞ" = { l = { }; }; }; key = "value"; key2 = "value"; ld1 = { _type = "timestamp"; value = "1979-05-27"; }; ldt1 = { _type = "timestamp"; value = "1979-05-27T07:32:00"; }; ldt2 = { _type = "timestamp"; value = "1979-05-27T00:32:00.999999"; }; lt1 = { _type = "timestamp"; value = "07:32:00"; }; lt2 = { _type = "timestamp"; value = "00:32:00.999999"; }; name = "Orange"; oct1 = 342391; oct2 = 493; odt1 = { _type = "timestamp"; value = "1979-05-27T07:32:00Z"; }; odt2 = { _type = "timestamp"; value = "1979-05-27T00:32:00-07:00"; }; odt3 = { _type = "timestamp"; value = "1979-05-27T00:32:00.999999-07:00"; }; odt4 = { _type = "timestamp"; value = "1979-05-27T07:32:00Z"; }; physical = { color = "orange"; shape = "round"; }; products = [ { name = "Hammer"; sku = 738594937; } { } { color = "gray"; name = "Nail"; sku = 284758393; } ]; "quoted \"value\"" = "value"; site = { "google.com" = true; }; str = "I'm a string. \"You can quote me\". Name\tJosé\nLocation\tSF."; table-1 = { key1 = "some string"; key2 = 123; }; table-2 = { key1 = "another string"; key2 = 456; }; x = { y = { z = { w = { animal = { type = { name = "pug"; }; }; name = { first = "Tom"; last = "Preston-Werner"; }; point = { x = 1; y = 2; }; }; }; }; }; "ʎǝʞ" = "value"; }
+{ "1234" = "value"; "127.0.0.1" = "value"; a = { b = { c = { }; }; }; arr1 = [ 1 2 3 ]; arr2 = [ "red" "yellow" "green" ]; arr3 = [ [ 1 2 ] [ 3 4 5 ] ]; arr4 = [ "all" "strings" "are the same" "type" ]; arr5 = [ [ 1 2 ] [ "a" "b" "c" ] ]; arr7 = [ 1 2 3 ]; arr8 = [ 1 2 ]; bare-key = "value"; bare_key = "value"; bin1 = 214; bool1 = true; bool2 = false; "character encoding" = "value"; d = { e = { f = { }; }; }; dog = { "tater.man" = { type = { name = "pug"; }; }; }; flt1 = 1; flt2 = 3.1415; flt3 = -0.01; flt4 = 5e+22; flt5 = 1e+06; flt6 = -0.02; flt7 = 6.626e-34; flt8 = 9.22462e+06; fruit = [ { name = "apple"; physical = { color = "red"; shape = "round"; }; variety = [ { name = "red delicious"; } { name = "granny smith"; } ]; } { name = "banana"; variety = [ { name = "plantain"; } ]; } ]; g = { h = { i = { }; }; }; hex1 = 3735928559; hex2 = 3735928559; hex3 = 3735928559; int1 = 99; int2 = 42; int3 = 0; int4 = -17; int5 = 1000; int6 = 5349221; int7 = 12345; j = { "ʞ" = { l = { }; }; }; key = "value"; key2 = "value"; ld1 = { _type = "timestamp"; value = "1979-05-27"; }; ldt1 = { _type = "timestamp"; value = "1979-05-27T07:32:00"; }; ldt10 = { _type = "timestamp"; value = "1979-05-27T00:32:00.123456789"; }; ldt11 = { _type = "timestamp"; value = "1979-05-27T00:32:00.123456789"; }; ldt2 = { _type = "timestamp"; value = "1979-05-27T07:32:00.100"; }; ldt3 = { _type = "timestamp"; value = "1979-05-27T07:32:00.120"; }; ldt4 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123"; }; ldt5 = { _type = "timestamp"; value = "1979-05-27T00:32:00.123400"; }; ldt6 = { _type = "timestamp"; value = "1979-05-27T00:32:00.123450"; }; ldt7 = { _type = "timestamp"; value = "1979-05-27T00:32:00.123456"; }; ldt8 = { _type = "timestamp"; value = "1979-05-27T00:32:00.123456700"; }; ldt9 = { _type = "timestamp"; value = "1979-05-27T00:32:00.123456780"; }; lt1 = { _type = "timestamp"; value = "07:32:00"; }; lt10 = { _type = "timestamp"; value = "00:32:00.123456789"; }; lt11 = { _type = "timestamp"; value = "00:32:00.123456789"; }; lt2 = { _type = "timestamp"; value = "00:32:00.100"; }; lt3 = { _type = "timestamp"; value = "00:32:00.120"; }; lt4 = { _type = "timestamp"; value = "00:32:00.123"; }; lt5 = { _type = "timestamp"; value = "00:32:00.123400"; }; lt6 = { _type = "timestamp"; value = "00:32:00.123450"; }; lt7 = { _type = "timestamp"; value = "00:32:00.123456"; }; lt8 = { _type = "timestamp"; value = "00:32:00.123456700"; }; lt9 = { _type = "timestamp"; value = "00:32:00.123456780"; }; name = "Orange"; oct1 = 342391; oct2 = 493; odt1 = { _type = "timestamp"; value = "1979-05-27T07:32:00Z"; }; odt10 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123456Z"; }; odt11 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123456700Z"; }; odt12 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123456780Z"; }; odt13 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123456789Z"; }; odt14 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123456789Z"; }; odt2 = { _type = "timestamp"; value = "1979-05-27T00:32:00-07:00"; }; odt3 = { _type = "timestamp"; value = "1979-05-27T00:32:00.999999-07:00"; }; odt4 = { _type = "timestamp"; value = "1979-05-27T07:32:00Z"; }; odt5 = { _type = "timestamp"; value = "1979-05-27T07:32:00.100Z"; }; odt6 = { _type = "timestamp"; value = "1979-05-27T07:32:00.120Z"; }; odt7 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123Z"; }; odt8 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123400Z"; }; odt9 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123450Z"; }; physical = { color = "orange"; shape = "round"; }; products = [ { name = "Hammer"; sku = 738594937; } { } { color = "gray"; name = "Nail"; sku = 284758393; } ]; "quoted \"value\"" = "value"; site = { "google.com" = true; }; str = "I'm a string. \"You can quote me\". Name\tJosé\nLocation\tSF."; table-1 = { key1 = "some string"; key2 = 123; }; table-2 = { key1 = "another string"; key2 = 456; }; x = { y = { z = { w = { animal = { type = { name = "pug"; }; }; name = { first = "Tom"; last = "Preston-Werner"; }; point = { x = 1; y = 2; }; }; }; }; }; "ʎǝʞ" = "value"; }

tests/functional/lang/eval-okay-fromTOML-timestamps.nix

@@ -55,11 +55,53 @@ builtins.fromTOML ''
   odt2 = 1979-05-27T00:32:00-07:00
   odt3 = 1979-05-27T00:32:00.999999-07:00
   odt4 = 1979-05-27 07:32:00Z
+  # milliseconds
+  odt5 = 1979-05-27 07:32:00.1Z
+  odt6 = 1979-05-27 07:32:00.12Z
+  odt7 = 1979-05-27 07:32:00.123Z
+  # microseconds
+  odt8 = 1979-05-27t07:32:00.1234Z
+  odt9 = 1979-05-27t07:32:00.12345Z
+  odt10 = 1979-05-27t07:32:00.123456Z
+  # nanoseconds
+  odt11 = 1979-05-27 07:32:00.1234567Z
+  odt12 = 1979-05-27 07:32:00.12345678Z
+  odt13 = 1979-05-27 07:32:00.123456789Z
+  # no more precision after nanoseconds
+  odt14 = 1979-05-27t07:32:00.1234567891Z
+
   ldt1 = 1979-05-27T07:32:00
-  ldt2 = 1979-05-27T00:32:00.999999
+  # milliseconds
+  ldt2 = 1979-05-27T07:32:00.1
+  ldt3 = 1979-05-27T07:32:00.12
+  ldt4 = 1979-05-27T07:32:00.123
+  # microseconds
+  ldt5 = 1979-05-27t00:32:00.1234
+  ldt6 = 1979-05-27t00:32:00.12345
+  ldt7 = 1979-05-27t00:32:00.123456
+  # nanoseconds
+  ldt8 = 1979-05-27 00:32:00.1234567
+  ldt9 = 1979-05-27 00:32:00.12345678
+  ldt10 = 1979-05-27 00:32:00.123456789
+  # no more precision after nanoseconds
+  ldt11 = 1979-05-27t00:32:00.1234567891
+
   ld1 = 1979-05-27
   lt1 = 07:32:00
-  lt2 = 00:32:00.999999
+  # milliseconds
+  lt2 = 00:32:00.1
+  lt3 = 00:32:00.12
+  lt4 = 00:32:00.123
+  # microseconds
+  lt5 = 00:32:00.1234
+  lt6 = 00:32:00.12345
+  lt7 = 00:32:00.123456
+  # nanoseconds
+  lt8 = 00:32:00.1234567
+  lt9 = 00:32:00.12345678
+  lt10 = 00:32:00.123456789
+  # no more precision after nanoseconds
+  lt11 = 00:32:00.1234567891
 
   arr1 = [ 1, 2, 3 ]
   arr2 = [ "red", "yellow", "green" ]

tests/functional/nix-shell/basic.sh

@@ -132,6 +132,7 @@ set -u
     [[ ${arr2[1]} = $'\n' ]]
     [[ ${arr2[2]} = $'x\ny' ]]
     [[ $(fun) = blabla ]]
+    [[ "$ASCII_ESC" = "$(printf "\e")" ]]
     [[ $PATH = $(jq -r .variables.PATH.value $TEST_ROOT/dev-env.json):$path ]]
 )
 

tests/functional/nix-shell/shell.nix

@@ -50,6 +50,7 @@ let pkgs = rec {
     VAR_FROM_NIX = "bar";
     ASCII_PERCENT = "%";
     ASCII_AT = "@";
+    ASCII_ESC = "";
     TEST_inNixShell = if inNixShell then "true" else "false";
     inherit stdenv;
     outputs = ["dev" "out"];

tests/functional/repl_characterization/data/basic_repl.test

@@ -1,12 +1,11 @@
   nix-repl> 1 + 1
   2
 
-  nix-repl> :doc builtins.head
+  nix-repl> :doc builtins.add
-  Synopsis: builtins.head list
+  Synopsis: builtins.add e1 e2
+
+      Return the sum of the numbers e1 and e2.
 
-      Return the first element of a list; abort evaluation if
-      the argument isn’t a list or is an empty list. You can
-      test whether a list is empty by comparing it with [].
 
   nix-repl> f = a: "" + a
 

tests/functional/supplementary-groups.sh

@@ -10,7 +10,6 @@ unshare --mount --map-root-user bash <<EOF
 
   # Avoid store dir being inside sandbox build-dir
   unset NIX_STORE_DIR
-  unset NIX_STATE_DIR
 
   setLocalStore () {
     export NIX_REMOTE=\$TEST_ROOT/\$1

tests/functional2/testlib/fixtures/nix.py

@@ -107,9 +107,9 @@ class NixSettings:
 
         field_may("experimental-features", self.experimental_features)
         field_may("store", self.store)
-        assert (
+        assert self.store or self.nix_store_dir, (
-            self.store or self.nix_store_dir
+            "Failing to set either nix_store_dir or store will cause accidental use of the system store."
-        ), "Failing to set either nix_store_dir or store will cause accidental use of the system store."
+        )
         return config
 
     def to_env_overlay(self) -> dict[str, str]:
@@ -215,13 +215,14 @@ class Nix:
     def nix(self, cmd: list[str], nix_exe: str = "nix", flake: bool = False) -> NixCommand:
         return self.nix_cmd([nix_exe, *cmd], flake=flake)
 
-    nix_build = partialmethod(nix, nix_exe="nix-build")
+    # Mark each of these as correct as they are not ClassVars, but we also don't want to turn off RUF045
-    nix_shell = partialmethod(nix, nix_exe="nix-shell")
+    nix_build = partialmethod(nix, nix_exe="nix-build")  # noqa: RUF045
-    nix_store = partialmethod(nix, nix_exe="nix-store")
+    nix_shell = partialmethod(nix, nix_exe="nix-shell")  # noqa: RUF045
-    nix_env = partialmethod(nix, nix_exe="nix-env")
+    nix_store = partialmethod(nix, nix_exe="nix-store")  # noqa: RUF045
-    nix_instantiate = partialmethod(nix, nix_exe="nix-instantiate")
+    nix_env = partialmethod(nix, nix_exe="nix-env")  # noqa: RUF045
-    nix_channel = partialmethod(nix, nix_exe="nix-channel")
+    nix_instantiate = partialmethod(nix, nix_exe="nix-instantiate")  # noqa: RUF045
-    nix_prefetch_url = partialmethod(nix, nix_exe="nix-prefetch-url")
+    nix_channel = partialmethod(nix, nix_exe="nix-channel")  # noqa: RUF045
+    nix_prefetch_url = partialmethod(nix, nix_exe="nix-prefetch-url")  # noqa: RUF045
 
     def eval(self, expr: str, settings: NixSettings | None = None) -> CommandResult:
         if settings is None:

tests/nixos/ca-fd-leak/default.nix

@@ -1,90 +0,0 @@
-# Nix is a sandboxed build system. But Not everything can be handled inside its
-# sandbox: Network access is normally blocked off, but to download sources, a
-# trapdoor has to exist. Nix handles this by having "Fixed-output derivations".
-# The detail here is not important, but in our case it means that the hash of
-# the output has to be known beforehand. And if you know that, you get a few
-# rights: you no longer run inside a special network namespace!
-#
-# Now, Linux has a special feature, that not many other unices do: Abstract
-# unix domain sockets! Not only that, but those are namespaced using the
-# network namespace! That means that we have a way to create sockets that are
-# available in every single fixed-output derivation, and also all processes
-# running on the host machine! Now, this wouldn't be that much of an issue, as,
-# well, the whole idea is that the output is pure, and all processes in the
-# sandbox are killed before finalizing the output. What if we didn't need those
-# processes at all? Unix domain sockets have a semi-known trick: you can pass
-# file descriptors around!
-# This makes it possible to exfiltrate a file-descriptor with write access to
-# $out outside of the sandbox. And that file-descriptor can be used to modify
-# the contents of the store path after it has been registered.
-
-{ config, ... }:
-
-let
-  pkgs = config.nodes.machine.nixpkgs.pkgs;
-
-  # Simple C program that sends a a file descriptor to `$out` to a Unix
-  # domain socket.
-  # Compiled statically so that we can easily send it to the VM and use it
-  # inside the build sandbox.
-  sender = pkgs.runCommandWith {
-    name = "sender";
-    stdenv = pkgs.pkgsStatic.stdenv;
-  } ''
-    $CC -static -o $out ${./sender.c}
-  '';
-
-  # Okay, so we have a file descriptor shipped out of the FOD now. But the
-  # Nix store is read-only, right? .. Well, yeah. But this file descriptor
-  # lives in a mount namespace where it is not! So even when this file exists
-  # in the actual Nix store, we're capable of just modifying its contents...
-  smuggler = pkgs.writeCBin "smuggler" (builtins.readFile ./smuggler.c);
-
-  # The abstract socket path used to exfiltrate the file descriptor
-  socketName = "FODSandboxExfiltrationSocket";
-in
-{
-  name = "ca-fd-leak";
-
-  nodes.machine =
-    { config, lib, pkgs, ... }:
-    { virtualisation.writableStore = true;
-      nix.settings.substituters = lib.mkForce [ ];
-      virtualisation.additionalPaths = [ pkgs.busybox-sandbox-shell sender smuggler pkgs.socat ];
-    };
-
-  testScript = { nodes }: ''
-    start_all()
-
-    machine.succeed("echo hello")
-    # Start the smuggler server
-    machine.succeed("${smuggler}/bin/smuggler ${socketName} >&2 &")
-
-    # Build the smuggled derivation.
-    # This will connect to the smuggler server and send it the file descriptor
-    machine.succeed(r"""
-      nix-build -E '
-        builtins.derivation {
-          name = "smuggled";
-          system = builtins.currentSystem;
-          # look ma, no tricks!
-          outputHashMode = "flat";
-          outputHashAlgo = "sha256";
-          outputHash = builtins.hashString "sha256" "hello, world\n";
-          builder = "${pkgs.busybox-sandbox-shell}/bin/sh";
-          args = [ "-c" "echo \"hello, world\" > $out; ''${${sender}} ${socketName}" ];
-      }'
-    """.strip())
-
-
-    # Tell the smuggler server that we're done
-    machine.execute("echo done | ${pkgs.socat}/bin/socat - ABSTRACT-CONNECT:${socketName}")
-
-    # Check that the file was not modified
-    machine.succeed(r"""
-      cat ./result
-      test "$(cat ./result)" = "hello, world"
-    """.strip())
-  '';
-
-}

tests/nixos/ca-fd-leak/sender.c

@@ -1,65 +0,0 @@
-#include <sys/socket.h>
-#include <sys/un.h>
-#include <stdlib.h>
-#include <stddef.h>
-#include <stdio.h>
-#include <unistd.h>
-#include <fcntl.h>
-#include <errno.h>
-#include <string.h>
-#include <assert.h>
-
-int main(int argc, char **argv) {
-
-    assert(argc == 2);
-
-    int sock = socket(AF_UNIX, SOCK_STREAM, 0);
-
-    // Set up a abstract domain socket path to connect to.
-    struct sockaddr_un data;
-    data.sun_family = AF_UNIX;
-    data.sun_path[0] = 0;
-    strcpy(data.sun_path + 1, argv[1]);
-
-    // Now try to connect, To ensure we work no matter what order we are
-    // executed in, just busyloop here.
-    int res = -1;
-    while (res < 0) {
-        res = connect(sock, (const struct sockaddr *)&data,
-            offsetof(struct sockaddr_un, sun_path)
-              + strlen(argv[1])
-              + 1);
-        if (res < 0 && errno != ECONNREFUSED) perror("connect");
-        if (errno != ECONNREFUSED) break;
-    }
-
-    // Write our message header.
-    struct msghdr msg = {0};
-    msg.msg_control = malloc(128);
-    msg.msg_controllen = 128;
-
-    // Write an SCM_RIGHTS message containing the output path.
-    struct cmsghdr *hdr = CMSG_FIRSTHDR(&msg);
-    hdr->cmsg_len = CMSG_LEN(sizeof(int));
-    hdr->cmsg_level = SOL_SOCKET;
-    hdr->cmsg_type = SCM_RIGHTS;
-    int fd = open(getenv("out"), O_RDWR | O_CREAT, 0640);
-    memcpy(CMSG_DATA(hdr), (void *)&fd, sizeof(int));
-
-    msg.msg_controllen = CMSG_SPACE(sizeof(int));
-
-    // Write a single null byte too.
-    msg.msg_iov = malloc(sizeof(struct iovec));
-    msg.msg_iov[0].iov_base = "";
-    msg.msg_iov[0].iov_len = 1;
-    msg.msg_iovlen = 1;
-
-    // Send it to the othher side of this connection.
-    res = sendmsg(sock, &msg, 0);
-    if (res < 0) perror("sendmsg");
-    int buf;
-
-    // Wait for the server to close the socket, implying that it has
-    // received the commmand.
-    recv(sock, (void *)&buf, sizeof(int), 0);
-}

tests/nixos/ca-fd-leak/smuggler.c

@@ -1,66 +0,0 @@
-#include <sys/socket.h>
-#include <sys/un.h>
-#include <stdlib.h>
-#include <stddef.h>
-#include <stdio.h>
-#include <unistd.h>
-#include <assert.h>
-
-int main(int argc, char **argv) {
-
-    assert(argc == 2);
-
-    int sock = socket(AF_UNIX, SOCK_STREAM, 0);
-
-    // Bind to the socket.
-    struct sockaddr_un data;
-    data.sun_family = AF_UNIX;
-    data.sun_path[0] = 0;
-    strcpy(data.sun_path + 1, argv[1]);
-    int res = bind(sock, (const struct sockaddr *)&data,
-        offsetof(struct sockaddr_un, sun_path)
-        + strlen(argv[1])
-        + 1);
-    if (res < 0) perror("bind");
-
-    res = listen(sock, 1);
-    if (res < 0) perror("listen");
-
-    int smuggling_fd = -1;
-
-    // Accept the connection a first time to receive the file descriptor.
-    fprintf(stderr, "%s\n", "Waiting for the first connection");
-    int a = accept(sock, 0, 0);
-    if (a < 0) perror("accept");
-
-    struct msghdr msg = {0};
-    msg.msg_control = malloc(128);
-    msg.msg_controllen = 128;
-
-    // Receive the file descriptor as sent by the smuggler.
-    recvmsg(a, &msg, 0);
-
-    struct cmsghdr *hdr = CMSG_FIRSTHDR(&msg);
-    while (hdr) {
-        if (hdr->cmsg_level == SOL_SOCKET
-          && hdr->cmsg_type == SCM_RIGHTS) {
-
-            // Grab the copy of the file descriptor.
-            memcpy((void *)&smuggling_fd, CMSG_DATA(hdr), sizeof(int));
-        }
-
-        hdr = CMSG_NXTHDR(&msg, hdr);
-    }
-    fprintf(stderr, "%s\n", "Got the file descriptor. Now waiting for the second connection");
-    close(a);
-
-    // Wait for a second connection, which will tell us that the build is
-    // done
-    a = accept(sock, 0, 0);
-    fprintf(stderr, "%s\n", "Got a second connection, rewriting the file");
-    // Write a new content to the file
-    if (ftruncate(smuggling_fd, 0)) perror("ftruncate");
-    char * new_content = "Pwned\n";
-    int written_bytes = write(smuggling_fd, new_content, strlen(new_content));
-    if (written_bytes != strlen(new_content)) perror("write");
-}

tests/nixos/chown-to-user.nix

@@ -0,0 +1,75 @@
+{ lib, config, pkgs, ... }:
+
+let
+  failedNormal = config: pkgs.writeText "failed.nix" ''
+    let utils = builtins.storePath ${config.system.build.extraUtils}; in
+    derivation {
+      name = "failed";
+      system = builtins.currentSystem;
+      PATH = "''${utils}/bin";
+      builder = "''${utils}/bin/sh";
+      args = [ "-c" "mkdir dir; echo test > dir/file" ];
+    }
+  '';
+
+  failedBuiltin = pkgs.writeText "failed.nix" ''
+    derivation {
+      name = "failed";
+      system = builtins.currentSystem;
+      builder = "builtin:fetchurl";
+      url = "http://localhost/foo";
+      outputHashMode = "flat";
+    }
+  '';
+in
+{
+  name = "chown-to-user";
+
+  nodes = {
+    machine = { config, lib, pkgs, ... }: {
+      virtualisation.writableStore = true;
+      virtualisation.additionalPaths = [ config.system.build.extraUtils ];
+
+      users.users.test = {
+        isNormalUser = true;
+        group = "test";
+      };
+      users.groups.test = {};
+      nix.nrBuildUsers = 1;
+    };
+  };
+
+  testScript = { nodes, ... }: ''
+    import re
+
+    machine.wait_for_unit("multi-user.target")
+
+    # builds using the daemon chown tempdirs
+    out = machine.fail("runuser -u test -- nix-build ${failedNormal nodes.machine} --keep-failed 2>&1")
+    dir = re.search("keeping build directory '(.+?)'", out)
+    assert dir
+    assert machine.succeed(f"stat -c %U:%G:%a {dir[1]}").strip() == "root:nixbld:755"
+    assert machine.succeed(f"stat -c %U:%G:%a {dir[1]}/b").strip() == "test:test:700"
+    assert machine.succeed(f"stat -c %U:%G:%a {dir[1]}/b/dir").strip() == "test:test:755"
+    assert machine.succeed(f"stat -c %U:%G:%a {dir[1]}/b/dir/file").strip() == "test:test:644"
+
+    # builds not using the daemon do not chown tempdirs
+    out = machine.fail("NIX_REMOTE=local nix-build ${failedNormal nodes.machine} --keep-failed 2>&1")
+    dir = re.search("keeping build directory '(.+?)'", out)
+    assert dir
+    assert machine.succeed(f"stat -c %U:%G:%a {dir[1]}").strip() == "root:nixbld:755"
+    assert machine.succeed(f"stat -c %U:%G:%a {dir[1]}/b").strip() == "nixbld1:nixbld:700"
+    assert machine.succeed(f"stat -c %U:%G:%a {dir[1]}/b/dir").strip() == "nixbld1:nixbld:755"
+    assert machine.succeed(f"stat -c %U:%G:%a {dir[1]}/b/dir/file").strip() == "nixbld1:nixbld:644"
+
+    # builds using builtin builders using the daemon do not keep tempdirs
+    out = machine.fail("runuser -u test -- nix-build ${failedBuiltin} --keep-failed 2>&1")
+    dir = re.search("keeping build directory '(.+?)'", out)
+    assert not dir
+
+    # builds using builtin builders not using the daemon do not keep tempdirs
+    out = machine.fail("NIX_REMOTE=local nix-build ${failedBuiltin} --keep-failed 2>&1")
+    dir = re.search("keeping build directory '(.+?)'", out)
+    assert not dir
+  '';
+}

tests/nixos/default.nix

@@ -67,6 +67,8 @@ in
     };
   });
 
+  # Let's ensure that reasonably popular shells are tested for remote building.
+
   remoteBuildsNushell = runNixOSTestFor "x86_64-linux" ({ lib, pkgs, ... }: {
     name = "remoteBuilds_nushell";
     imports = [ ./remote-builds.nix ];
@@ -75,27 +77,11 @@ in
     };
   });
 
-  remoteBuildsWeirdShell = runNixOSTestFor "x86_64-linux" ({ lib, pkgs, ... }: {
+  remoteBuildsBusybox = runNixOSTestFor "x86_64-linux" ({ lib, pkgs, ... }: {
-    name = "remoteBuilds_weird_shell";
+    name = "remoteBuilds_busybox";
     imports = [ ./remote-builds.nix ];
     builders.config = { lib, pkgs, ... }: {
-      # a pathologically weird shell that can do nothing BUT run bash
+      users.users.root.shell = pkgs.busybox;
-      users.users.root.shell = pkgs.writeTextFile {
-        name = "watsh";
-        destination = "/bin/watsh";
-        executable = true;
-
-        text = ''
-          #!/bin/sh
-          if [ "$1" = "-c" ] && [ "$2" = "bash" ]; then
-            exec bash
-          else
-            echo "Wat."
-          fi
-        '';
-
-        passthru.shellPath = "/bin/watsh";
-      };
     };
   });
 
@@ -179,8 +165,6 @@ in
     ["i686-linux" "x86_64-linux"]
     (system: runNixOSTestFor system ./setuid/setuid.nix);
 
-  ca-fd-leak = runNixOSTestFor "x86_64-linux" ./ca-fd-leak;
-
   fetch-git = runNixOSTestFor "x86_64-linux" ./fetch-git;
 
   symlinkResolvconf = runNixOSTestFor "x86_64-linux" ./symlink-resolvconf.nix;
@@ -196,4 +180,6 @@ in
   io_uring = runNixOSTestFor "x86_64-linux" ./io_uring;
 
   fetchurl = runNixOSTestFor "x86_64-linux" ./fetchurl.nix;
+
+  chown-to-user = runNixOSTestFor "x86_64-linux" ./chown-to-user.nix;
 }

tests/nixos/fetchurl.nix

@@ -52,7 +52,7 @@ in
 
       security.pki.certificateFiles = [ "${goodCert}/cert.pem" ];
 
-      networking.hosts."127.0.0.1" = [ "good" "bad" ];
+      networking.hosts."192.168.1.1" = [ "good" "bad" ];
 
       virtualisation.writableStore = true;
 
@@ -76,7 +76,7 @@ in
     # Fetching from a server with an untrusted cert should fail.
     err = machine.fail("nix build --no-substitute --expr 'import <nix/fetchurl.nix> { url = \"https://bad/index.html\"; hash = \"sha256-rsBwZF/lPuOzdjBZN2E08FjMM3JHyXit0Xi2zN+wAZ8=\"; }' 2>&1")
     print(err)
-    assert "SSL peer certificate or SSH remote key was not OK" in err
+    assert "SSL certificate problem: self-signed certificate" in err
 
     # Fetching from a server with a trusted cert should work via environment variable override.
     machine.succeed("NIX_SSL_CERT_FILE=/tmp/cafile.pem nix build --no-substitute --expr 'import <nix/fetchurl.nix> { url = \"https://bad/index.html\"; hash = \"sha256-rsBwZF/lPuOzdjBZN2E08FjMM3JHyXit0Xi2zN+wAZ8=\"; }'")

tests/nixos/nix-copy-closure.nix

@@ -45,12 +45,12 @@ in {
     server.wait_for_unit("network-online.target")
     client.wait_for_unit("network-online.target")
 
-    client.succeed("mkdir -m 700 /root/.ssh")
+    client.succeed("mkdir -m 700 /root/.ssh || [[ -d /root/.ssh ]]")
     client.copy_from_host("key", "/root/.ssh/id_ed25519")
     client.succeed("chmod 600 /root/.ssh/id_ed25519")
 
     # Install the SSH key on the server.
-    server.succeed("mkdir -m 700 /root/.ssh")
+    server.succeed("mkdir -m 700 /root/.ssh || [[ -d /root/.ssh ]]")
     server.copy_from_host("key.pub", "/root/.ssh/authorized_keys")
     server.wait_for_unit("sshd.service")
     client.succeed(f"ssh -o StrictHostKeyChecking=no {server.name} 'echo hello world' >&2")

tests/nixos/remote-builds-ssh-ng.nix

@@ -48,6 +48,7 @@ in
           imports = [ test.config.builders.config ];
           services.openssh.enable = true;
           virtualisation.writableStore = true;
+          virtualisation.additionalPaths = [ config.system.build.extraUtils ];
           nix.settings.sandbox = true;
           nix.settings.substituters = lib.mkForce [ ];
         };
@@ -98,7 +99,7 @@ in
 
       out = client.fail("nix-build ${expr nodes.client 1} 2>&1")
       assert "Host key verification failed." in out, f"No host verification error:\n{out}"
-      assert "warning: SSH to 'root@builder' failed, stdout first line: '''" in out, f"No details about which host:\n{out}"
+      assert "'ssh-ng://root@builder'" in out, f"No details about which host:\n{out}"
 
       client.succeed(f"ssh -o StrictHostKeyChecking=no {builder.name} 'echo hello world' >&2")
 

tests/unit/libutil/archive.cc

@@ -509,4 +509,27 @@ INSTANTIATE_TEST_SUITE_P(
             concat({header, make_directory({{"DE", make_file(false, "meow")}, {"de", make_file(false, "mrrp")}})})
         ))
 );
+
+TEST_F(NarTest, stringSizeLimit)
+{
+    GeneratorSource source([]() -> Generator<Bytes> {
+        const char preamble[] =
+            "\x0d\x00\x00\x00\x00\x00\x00\x00nix-archive-1\x00\x00\x00"
+            "\x01\x00\x00\x00\x00\x00\x00\x00(\x00\x00\x00\x00\x00\x00\x00"
+            "\x04\x00\x00\x00\x00\x00\x00\x00type\x00\x00\x00\x00";
+        co_yield Bytes{preamble, sizeof(preamble) - 1};
+        // the nar parser keeps all strings in a buffer with the 8 byte length prefix in front.
+        // sufficiently large strings overflowed caused the buffer size calculation to overflow
+        // and thus allowed out-of-bounds writes in the daemon and potentially privesc to root.
+        co_yield Bytes{"\xf7\xff\xff\xff\xff\xff\xff\xff", 8};
+        // overflow would happen while reading data
+        while (true) {
+            co_yield Bytes{"foo-", 4};
+        }
+    }());
+
+    auto parser = nar::parse(source);
+
+    ASSERT_THROW(parser.next(), SerialisationError);
+}
 }

tests/unit/libutil/tests.cc

@@ -3,6 +3,8 @@
 #include "lix/libutil/strings.hh"
 #include "lix/libutil/types.hh"
 #include "lix/libutil/terminal.hh"
+#include "lix/libutil/unix-domain-socket.hh"
+#include "tests/test-data.hh"
 
 #include <gtest/gtest.h>
 
@@ -207,6 +209,85 @@ namespace nix {
         ASSERT_FALSE(pathExists("/schnitzel/darmstadt/pommes"));
     }
 
+    /* ----------------------------------------------------------------------------
+     * AutoCloseFD::guessOrInventPath
+     * --------------------------------------------------------------------------*/
+    void testGuessOrInventPathPrePostDeletion(AutoCloseFD & fd, Path & path) {
+        {
+            SCOPED_TRACE(fmt("guessing path before deletion of '%1%'", path));
+            ASSERT_TRUE(fd);
+            /* We cannot predict what the platform will return here.
+             * But it cannot fail. */
+            ASSERT_TRUE(fd.guessOrInventPath().size() >= 0);
+        }
+        {
+            SCOPED_TRACE(fmt("guessing path after deletion of '%1%'", path));
+            deletePath(path);
+            /* We cannot predict what the platform will return here.
+             * But it cannot fail. */
+            ASSERT_TRUE(fd.guessOrInventPath().size() >= 0);
+        }
+    }
+    TEST(guessOrInventPath, files) {
+        Path filePath = getUnitTestDataPath("guess-or-invent/test.txt");
+        createDirs(dirOf(filePath));
+        writeFile(filePath, "some text");
+        AutoCloseFD file{open(filePath.c_str(), O_RDONLY, 0666)};
+        testGuessOrInventPathPrePostDeletion(file, filePath);
+    }
+
+    TEST(guessOrInventPath, directories) {
+        Path dirPath = getUnitTestDataPath("guess-or-invent/test-dir");
+        createDirs(dirPath);
+        AutoCloseFD directory{open(dirPath.c_str(), O_DIRECTORY, 0666)};
+        testGuessOrInventPathPrePostDeletion(directory, dirPath);
+    }
+
+#ifdef O_PATH
+    TEST(guessOrInventPath, symlinks) {
+        Path symlinkPath = getUnitTestDataPath("guess-or-invent/test-symlink");
+        Path targetPath = getUnitTestDataPath("guess-or-invent/nowhere");
+        createDirs(dirOf(symlinkPath));
+        createSymlink(targetPath, symlinkPath);
+        AutoCloseFD symlink{open(symlinkPath.c_str(), O_PATH | O_NOFOLLOW, 0666)};
+        testGuessOrInventPathPrePostDeletion(symlink, symlinkPath);
+    }
+
+    TEST(guessOrInventPath, fifos) {
+        Path fifoPath = getUnitTestDataPath("guess-or-invent/fifo");
+        createDirs(dirOf(fifoPath));
+        ASSERT_TRUE(mkfifo(fifoPath.c_str(), 0666) == 0);
+        AutoCloseFD fifo{open(fifoPath.c_str(), O_PATH | O_NOFOLLOW, 0666)};
+        testGuessOrInventPathPrePostDeletion(fifo, fifoPath);
+    }
+#endif
+
+    TEST(guessOrInventPath, pipes) {
+        int pipefd[2];
+
+        ASSERT_TRUE(pipe(pipefd) == 0);
+
+        AutoCloseFD pipe_read{pipefd[0]};
+        ASSERT_TRUE(pipe_read);
+        AutoCloseFD pipe_write{pipefd[1]};
+        ASSERT_TRUE(pipe_write);
+
+        /* We cannot predict what the platform will return here.
+         * But it cannot fail. */
+        ASSERT_TRUE(pipe_read.guessOrInventPath().size() >= 0);
+        ASSERT_TRUE(pipe_write.guessOrInventPath().size() >= 0);
+        pipe_write.close();
+        ASSERT_TRUE(pipe_read.guessOrInventPath().size() >= 0);
+        pipe_read.close();
+    }
+
+    TEST(guessOrInventPath, sockets) {
+        Path socketPath = getUnitTestDataPath("guess-or-invent/socket");
+        createDirs(dirOf(socketPath));
+        AutoCloseFD socket = createUnixDomainSocket(socketPath, 0666);
+        testGuessOrInventPathPrePostDeletion(socket, socketPath);
+    }
+
     /* ----------------------------------------------------------------------------
      * concatStringsSep
      * --------------------------------------------------------------------------*/
@@ -283,6 +364,64 @@ namespace nix {
         ASSERT_THROW(base64Decode("cXVvZCBlcm_0IGRlbW9uc3RyYW5kdW0="), Error);
     }
 
+    /* ----------------------------------------------------------------------------
+     * base32Encode
+     * --------------------------------------------------------------------------*/
+
+    TEST(base32Encode, emptyString) {
+        ASSERT_EQ(base32EncodeStr(""), "");
+    }
+
+    TEST(base32Encode, encodesAString) {
+        ASSERT_EQ(base32EncodeStr("quod erat demonstrandum"), "6sxb4drhp4x3kdrpnsrb441s62wk541j6yxbi");
+    }
+
+    TEST(base32Encode, encodeAndDecode) {
+        auto s = "quod erat demonstrandum";
+        auto encoded = base32EncodeStr(s);
+        auto decoded = base32Decode(encoded);
+
+        ASSERT_EQ(decoded, s);
+    }
+
+    TEST(base32Encode, encodeAndDecodeNonPrintable) {
+        std::string s(257, '\0');
+        std::iota(std::rbegin(s), std::rend(s), 0);
+
+        auto encoded = base32EncodeStr(s);
+        auto decoded = base32Decode(encoded);
+
+        EXPECT_EQ(decoded.length(), 257);
+        ASSERT_EQ(decoded, s);
+    }
+
+    TEST(base32Encode, handleNulChars) {
+        std::string s = "cat girls say meow even with NULs";
+        // Just throw a NUL in there somewhere.
+        s[5] = '\0';
+
+        auto encoded = base32EncodeStr(s);
+        auto decoded = base32Decode(encoded);
+
+        EXPECT_EQ(decoded, s);
+    }
+
+    /* ----------------------------------------------------------------------------
+     * base32Decode
+     * --------------------------------------------------------------------------*/
+
+    TEST(base32Decode, emptyString) {
+        ASSERT_EQ(base32Decode(""), "");
+    }
+
+    TEST(base32Decode, decodeAString) {
+        ASSERT_EQ(base32Decode("6sxb4drhp4x3kdrpnsrb441s62wk541j6yxbi"), "quod erat demonstrandum");
+    }
+
+    TEST(base32Decode, decodeThrowsOnInvalidChar) {
+        ASSERT_THROW(base32Decode("6sxb4drhp4x3kdrpnsrb441s62wk541j6yxbe"), Error);
+    }
+
     /* ----------------------------------------------------------------------------
      * getLine
      * --------------------------------------------------------------------------*/

version.json

@@ -1,5 +1,5 @@
 {
-    "version": "2.93.0",
+    "version": "2.93.4",
-    "official_release": false,
+    "official_release": true,
     "release_name": "Bici Bici"
 }