Enable nixpkgsLibTests again

The required nixpkgs bumps have now been performed. Change-Id: I6a6a6964316f8434c355405c52b0a7b7f70b4332
flake: update nixpkgs input
2025-09-08 20:26:44 +02:00 · 2025-09-08 20:26:44 +02:00 · 2025-09-08 20:26:44 +02:00 · 2025-09-08 19:18:09 +02:00 · 2025-09-08 19:18:09 +02:00 · 2025-09-08 19:17:29 +02:00
79 changed files with 1322 additions and 519 deletions
@@ -16,6 +16,8 @@ Checks:
  - -bugprone-unchecked-optional-access
    # many warnings, seems like a questionable lint
  - -bugprone-branch-clone
+    # extremely noisy before clang 19: https://github.com/llvm/llvm-project/issues/93959
+  - -bugprone-multi-level-implicit-pointer-conversion
    # all thrown exceptions must derive from std::exception
  - hicpp-exception-baseclass
    # capturing async lambdas are dangerous
@@ -33,3 +35,4 @@ Checks:

 CheckOptions:
  bugprone-reserved-identifier.AllowedIdentifiers: '__asan_default_options'
+  bugprone-unused-return-value.AllowCastToVoid: true
@@ -1,4 +1,114 @@
 # Lix 2.91 "Dragon's Breath" (2024-08-12)
+# Lix 2.91.3 (2025-06-30)
+
+## Fixes
+- Revert CVE-2025-52992 failed mitigation [fj#883](https://git.lix.systems/lix-project/lix/issues/883) [fj#887](https://git.lix.systems/lix-project/lix/issues/887) [cl/3420](https://gerrit.lix.systems/c/lix/+/3420) [cl/3524](https://gerrit.lix.systems/c/lix/+/3524) [cl/3523](https://gerrit.lix.systems/c/lix/+/3523) [cl/3522](https://gerrit.lix.systems/c/lix/+/3522)
+
+  Following the initial mitigation of **CVE-2025-52992** in `cl/3420`, we
+  received reports of **unexpected deletion of in-use store paths**.
+
+  Upon investigation, we found that the patch did **not correctly cancel all
+  automatic deleters**, resulting in potentially critical path loss during normal
+  operation.
+
+  Given the severity and time-sensitive nature of the situation ([see incident
+  report](https://lix.systems/blog/2025-06-27-lix-critical-bug/)), we evaluated
+  possible options to repair the behavior safely. However, we concluded that a
+  rushed fix would either:
+
+  * **Overdelete**, i.e. breaking running systems, or,
+  * **Underdelete**, effectively **reopening CVE-2025-52992** while leaving
+    orphaned paths behind.
+
+  As **CVE-2025-52992 has no known exploit vector**, and correctness is critical
+  in the Lix project, we have **fully reverted the previous mitigations**.
+
+  Moving forward, the Lix team will rework this code path in a **long-term,
+  correctness-first fix** on the main branch. We will explore backporting it to
+  stable channels once its safety is assured.
+
+  We are deeply sorry for the stability incident and the Lix team remain
+  available for assisting you in recovering your systems.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
+- Fallback to safe temp dir when build-dir is unwritable [fj#876](https://git.lix.systems/lix-project/lix/issues/876) [cl/3502](https://gerrit.lix.systems/c/lix/+/3502)
+
+  Non-daemon builds started failing with a permission error after introducing the `build-dir` option:
+
+  ```
+  $ nix build --store ~/scratch nixpkgs#hello --rebuild
+  error: creating directory '/nix/var/nix/builds/nix-build-hello-2.12.2.drv-0': Permission denied
+  ```
+
+  This happens because:
+
+  1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
+  2. The user lacks permissions for that path.
+
+  We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
+
+  To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
+
+  **Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+
+# Lix 2.91.2 (2025-06-23)
+
+## Breaking Changes
+- Fixed output derivations can be run using `pasta` network isolation [fj#285](https://git.lix.systems/lix-project/lix/issues/285) [cl/3418](https://gerrit.lix.systems/c/lix/+/3418)
+
+  Fixed output derivations traditionally run in the host network namespace.
+  On Linux this allows such derivations to communicate with other sandboxes
+  or the host using the abstract Unix domains socket namespace; this hasn't
+  been unproblematic in the past and has been used in two distinct exploits
+  to break out of the sandbox. For this reason fixed output derivations can
+  now run in a network namespace (provided by [`pasta`]), restricted to TCP
+  and UDP communication with the rest of the world. When enabled this could
+  be a breaking change and we classify it as such, even though we don't yet
+  enable or require such isolation by default. We may enforce this in later
+  releases of Lix once we have sufficient confidence that breakage is rare.
+
+  [`pasta`]: https://passt.top/
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [puck](https://git.lix.systems/puck) for this.
+
+## Fixes
+- Always clean up scratch paths after derivations failed to build [cl/3420](https://gerrit.lix.systems/c/lix/+/3420)
+
+  Previously, scratch paths created during builds were not always cleaned up if
+  the derivation failed, potentially leaving behind unnecessary temporary files
+  or directories in the Nix store.
+
+  This fix ensures that such paths are consistently removed after a failed build,
+  improving Nix store hygiene, hardening Lix against mis-reuse of failed builds
+  scratch paths.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
+- `build-dir` no longer defaults to `temp-dir` [cl/3419](https://gerrit.lix.systems/c/lix/+/3419)
+
+  The directory in which temporary build directories are created no longer defaults
+  to the value of the `temp-dir` setting to avoid builders making their directories
+  world-accessible. This behavior has been used to escape the build sandbox and can
+  cause build impurities even when not used maliciously. We now default to `builds`
+  in `NIX_STATE_DIR` (which is `/nix/var/nix/builds` in the default configuration).
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+
+# Lix 2.91.1 (2024-10-18)
+
+## Fixes
+- `<nix/fetchurl.nix>` now uses TLS verification [#11585](https://github.com/NixOS/nix/pull/11585)
+
+  Previously `<nix/fetchurl.nix>` did not do TLS verification. This was because the Nix sandbox in the past did not have access to TLS certificates, and Nix checks the hash of the fetched file anyway. However, this can expose authentication data from `netrc` and URLs to man-in-the-middle attackers. In addition, Nix now in some cases (such as when using impure derivations) does *not* check the hash. Therefore we have now enabled TLS verification. This means that downloads by `<nix/fetchurl.nix>` will now fail if you're fetching from a HTTPS server that does not have a valid certificate.
+
+  `<nix/fetchurl.nix>` is also known as the builtin derivation builder `builtin:fetchurl`. It's not to be confused with the evaluation-time function `builtins.fetchurl`, which was not affected by this issue.
+
+  Many thanks to [Eelco Dolstra](https://github.com/edolstra) for this.
+
+


 # Lix 2.91.0 (2024-08-12)
@@ -62,38 +62,37 @@ let
    ++ autoLayered
    ++ extraPkgs;

-  users =
-    {
+  users = {

-      root = {
-        uid = 0;
-        shell = "${pkgs.bashInteractive}/bin/bash";
-        home = "/root";
-        gid = 0;
-        groups = [ "root" ];
-        description = "System administrator";
-      };
+    root = {
+      uid = 0;
+      shell = "${pkgs.bashInteractive}/bin/bash";
+      home = "/root";
+      gid = 0;
+      groups = [ "root" ];
+      description = "System administrator";
+    };

-      nobody = {
-        uid = 65534;
-        shell = "${pkgs.shadow}/bin/nologin";
-        home = "/var/empty";
-        gid = 65534;
-        groups = [ "nobody" ];
-        description = "Unprivileged account (don't use!)";
+    nobody = {
+      uid = 65534;
+      shell = "${pkgs.shadow}/bin/nologin";
+      home = "/var/empty";
+      gid = 65534;
+      groups = [ "nobody" ];
+      description = "Unprivileged account (don't use!)";
+    };
+  }
+  // lib.listToAttrs (
+    map (n: {
+      name = "nixbld${toString n}";
+      value = {
+        uid = 30000 + n;
+        gid = 30000;
+        groups = [ "nixbld" ];
+        description = "Nix build user ${toString n}";
      };
-    }
-    // lib.listToAttrs (
-      map (n: {
-        name = "nixbld${toString n}";
-        value = {
-          uid = 30000 + n;
-          gid = 30000;
-          groups = [ "nixbld" ];
-          description = "Nix build user ${toString n}";
-        };
-      }) (lib.lists.range 1 32)
-    );
+    }) (lib.lists.range 1 32)
+  );

  groups = {
    root.gid = 0;
@@ -193,13 +192,11 @@ let
            in
            ''
              {
-                ${
-                  lib.concatStringsSep "\n" (
-                    builtins.map (output: ''
-                      ${output} = { outPath = "${lib.getOutput output drv}"; };
-                    '') outputs
-                  )
-                }
+                ${lib.concatStringsSep "\n" (
+                  builtins.map (output: ''
+                    ${output} = { outPath = "${lib.getOutput output drv}"; };
+                  '') outputs
+                )}
                outputs = [ ${lib.concatStringsSep " " (builtins.map (x: "\"${x}\"") outputs)} ];
                name = "${drv.name}";
                outPath = "${drv}";
@@ -361,8 +358,10 @@ let
        "org.opencontainers.image.source" = "https://git.lix.systems/lix-project/lix";
        "org.opencontainers.image.vendor" = "Lix project";
        "org.opencontainers.image.version" = pkgs.nix.version;
-        "org.opencontainers.image.description" = "Minimal Lix container image, with some batteries included.";
-      } // lib.optionalAttrs (lixRevision != null) { "org.opencontainers.image.revision" = lixRevision; };
+        "org.opencontainers.image.description" =
+          "Minimal Lix container image, with some batteries included.";
+      }
+      // lib.optionalAttrs (lixRevision != null) { "org.opencontainers.image.revision" = lixRevision; };
    };

    meta = {
@@ -16,14 +16,30 @@
        "type": "github"
      }
    },
+    "lowdown-src": {
+      "flake": false,
+      "locked": {
+        "lastModified": 1633514407,
+        "narHash": "sha256-Dw32tiMjdK9t3ETl5fzGrutQTzh2rufgZV4A/BbxuD4=",
+        "owner": "kristapsdz",
+        "repo": "lowdown",
+        "rev": "d2c2b44ff6c27b936ec27358a2653caaef8f73b8",
+        "type": "github"
+      },
+      "original": {
+        "owner": "kristapsdz",
+        "repo": "lowdown",
+        "type": "github"
+      }
+    },
    "nix2container": {
      "flake": false,
      "locked": {
-        "lastModified": 1720642556,
-        "narHash": "sha256-qsnqk13UmREKmRT7c8hEnz26X3GFFyIQrqx4EaRc1Is=",
+        "lastModified": 1749158376,
+        "narHash": "sha256-uirStFNxauh0lxzBowcp28X+Sq7JgsBIDnbwbAfZwf8=",
        "owner": "nlewo",
        "repo": "nix2container",
-        "rev": "3853e5caf9ad24103b13aa6e0e8bcebb47649fe4",
+        "rev": "0f8974c58755dba441df03598eefd1e1cd50e341",
        "type": "github"
      },
      "original": {
@@ -32,18 +48,44 @@
        "type": "github"
      }
    },
-    "nixpkgs": {
+    "nix_2_18": {
+      "inputs": {
+        "flake-compat": [
+          "flake-compat"
+        ],
+        "lowdown-src": "lowdown-src",
+        "nixpkgs": "nixpkgs",
+        "nixpkgs-regression": [
+          "nixpkgs-regression"
+        ]
+      },
      "locked": {
-        "lastModified": 1721931987,
-        "narHash": "sha256-1Zg8LY0T5EfXtv0Kf4M6SFnjH7Eto4VV+EKJ/YSnhiI=",
+        "lastModified": 1730375271,
+        "narHash": "sha256-RrOFlDGmRXcVRV2p2HqHGqvzGNyWoD0Dado/BNlJ1SI=",
        "owner": "NixOS",
-        "repo": "nixpkgs",
-        "rev": "e21630230c77140bc6478a21cd71e8bb73706fce",
+        "repo": "nix",
+        "rev": "0f665ff6779454f2117dcc32e44380cda7f45523",
        "type": "github"
      },
      "original": {
        "owner": "NixOS",
-        "ref": "nixos-24.05-small",
+        "ref": "2.18.9",
+        "repo": "nix",
+        "type": "github"
+      }
+    },
+    "nixpkgs": {
+      "locked": {
+        "lastModified": 1705033721,
+        "narHash": "sha256-K5eJHmL1/kev6WuqyqqbS1cdNnSidIZ3jeqJ7GbrYnQ=",
+        "owner": "NixOS",
+        "repo": "nixpkgs",
+        "rev": "a1982c92d8980a0114372973cbdfe0a307f1bdea",
+        "type": "github"
+      },
+      "original": {
+        "owner": "NixOS",
+        "ref": "nixos-23.05-small",
        "repo": "nixpkgs",
        "type": "github"
      }
@@ -64,14 +106,30 @@
        "type": "github"
      }
    },
+    "nixpkgs_2": {
+      "locked": {
+        "lastModified": 1757198069,
+        "narHash": "sha256-m3VUcOD4rTs8J7S+3dOjWMrAjw6RcITC3XYQ98zhEFs=",
+        "owner": "NixOS",
+        "repo": "nixpkgs",
+        "rev": "0747026fc57ecb9c28901c7f7a2b5dc40e8af43c",
+        "type": "github"
+      },
+      "original": {
+        "owner": "NixOS",
+        "ref": "nixos-25.05-small",
+        "repo": "nixpkgs",
+        "type": "github"
+      }
+    },
    "pre-commit-hooks": {
      "flake": false,
      "locked": {
-        "lastModified": 1721042469,
-        "narHash": "sha256-6FPUl7HVtvRHCCBQne7Ylp4p+dpP3P/OYuzjztZ4s70=",
+        "lastModified": 1733318908,
+        "narHash": "sha256-SVQVsbafSM1dJ4fpgyBqLZ+Lft+jcQuMtEL3lQWx2Sk=",
        "owner": "cachix",
        "repo": "git-hooks.nix",
-        "rev": "f451c19376071a90d8c58ab1a953c6e9840527fd",
+        "rev": "6f4e2a2112050951a314d2733a994fbab94864c6",
        "type": "github"
      },
      "original": {
@@ -84,7 +142,8 @@
      "inputs": {
        "flake-compat": "flake-compat",
        "nix2container": "nix2container",
-        "nixpkgs": "nixpkgs",
+        "nix_2_18": "nix_2_18",
+        "nixpkgs": "nixpkgs_2",
        "nixpkgs-regression": "nixpkgs-regression",
        "pre-commit-hooks": "pre-commit-hooks"
      }
@@ -2,8 +2,19 @@
  description = "Lix: A modern, delicious implementation of the Nix package manager";

  inputs = {
-    nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.05-small";
+    nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.05-small";
    nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
+
+    # Required because Nix 2.18 is not in Nixpkgs ≥ 25.05 anymore.
+    nix_2_18 = {
+      url = "github:NixOS/nix/2.18.9";
+      # NOTE(Raito): this is not possible because patches on libseccomp does not apply anymore on this Nix.
+      # Let's keep the latest known nixpkgs useable with Nix 2.18 for our tests.
+      # inputs.nixpkgs.follows = "nixpkgs";
+      inputs.nixpkgs-regression.follows = "nixpkgs-regression";
+      inputs.flake-compat.follows = "flake-compat";
+    };
+
    pre-commit-hooks = {
      url = "github:cachix/git-hooks.nix";
      flake = false;
@@ -25,6 +36,7 @@
      nixpkgs-regression,
      pre-commit-hooks,
      nix2container,
+      nix_2_18,
      flake-compat,
    }:

@@ -161,6 +173,16 @@
        {
          nixStable = prev.nix;

+          nixVersions = prev.nixVersions // {
+            nix_2_3 = prev.nixVersions.nix_2_3.overrideAttrs (old: {
+              meta = old.meta // {
+                knownVulnerabilities = [ ];
+              };
+            });
+            # Nix 2.18 has been removed from Nixpkgs ≥ 25.05, so we need to reintroduce it ourselves for our tests.
+            nix_2_18 = nix_2_18.outputs.packages.${currentStdenv.hostPlatform.system}.default;
+          };
+
          # Forward from the previous stage as we don’t want it to pick the lowdown override
          nixUnstable = prev.nixUnstable;

@@ -346,13 +368,16 @@
            in
            pkgs.symlinkJoin {
              name = "nixpkgs-lib-tests";
-              paths =
-                [ testWithNix ]
-                # FIXME: This is disabled on darwin due to a nixpkgs bug https://github.com/NixOS/nixpkgs/issues/319147
-                # After that is fixed, it should be restored to use lib/tests/release.nix as before, rather than this reimplementation.
-                ++ lib.optionals pkgs.stdenv.isLinux [
-                  (import (nixpkgs + "/pkgs/test/release") { inherit pkgs lib nix; })
-                ];
+              paths = [
+                testWithNix
+              ]
+              # NOTE: nixpkgs 25.05 is being ... *creative*, and requires this dance to override
+              # the evaluator used for the test. it will break again in the future, don't worry.
+              ++ lib.optionals pkgs.stdenv.isLinux [
+                ((pkgs.callPackage "${nixpkgs}/ci/eval" { inherit nix; }).attrpathsSuperset {
+                  evalSystem = system;
+                })
+              ];
            }
          );
        };
@@ -227,6 +227,11 @@ configdata += {
  'HAVE_SECCOMP': seccomp.found().to_int(),
 }

+# fcntl(F_GETPATH) returns the path of an fd on macOS and BSDs
+configdata += {
+  'HAVE_F_GETPATH': cxx.has_header_symbol('fcntl.h', 'F_GETPATH').to_int(),
+}
+
 libarchive = dependency('libarchive', required : true, include_type : 'system')

 brotli = [
@@ -296,6 +301,11 @@ editline = dependency('libeditline', 'editline', version : '>=1.14', required :

 lowdown = dependency('lowdown', version : '>=0.9.0', required : true, include_type : 'system')

+# TODO(sterni): drop the corresponding #ifdef after NixOS 25.05 is EOL which still distributes lowdown < 1.4.0
+if lowdown.version().version_compare('>= 1.4.0')
+  add_project_arguments('-DLOWDOWN_SEPARATE_TERM_OPTS', language: 'cpp')
+endif
+
 # HACK(Qyriad): rapidcheck's pkg-config doesn't include the libs lol
 # Note: technically we 'check' for rapidcheck twice, for the internal-api-docs handling above,
 # but Meson will cache the result of the first one, and the required : arguments are different.
@@ -371,6 +381,13 @@ endif
 # FIXME(Qyriad): the autoconf system checks that busybox has the "standalone" feature, indicating
 # that busybox sh won't run busybox applets as builtins (which would break our sandbox).

+pasta_path = get_option('pasta-path')
+# we can't check the pasta version because passt misuses stdio (it calls _exit()
+# after printing the version, which will never print the version unless run from
+# a terminal). pasta isn't mandatory yet due to high fetcher breakage potential.
+# we *will* enable it in our own packaging, but distributions are not forced to.
+pasta = find_program(pasta_path, required : false, native : false)
+
 lsof = find_program('lsof', native : true)

 # This is how Nix does generated headers...
@@ -417,7 +434,7 @@ check_funcs = [
  'strsignal',
  'sysconf',
 ]
-if target_machine.kernel() in ['linux', 'freebsd']
+if is_linux or is_freebsd
  # musl does not have close_range as of 2024-08-10
  # patch: https://www.openwall.com/lists/musl/2024/08/01/9
  check_funcs += [ 'close_range' ]
@@ -463,6 +480,7 @@ add_project_arguments(
  # It would be nice for our headers to be idempotent instead.
  '-include', 'config.h',
  '-Wno-deprecated-declarations',
+  '-Wno-missing-field-initializers',
  '-Wimplicit-fallthrough',
  '-Werror=switch',
  '-Werror=switch-enum',
@@ -24,6 +24,10 @@ option('sandbox-shell', type : 'string', value : 'busybox',
  description : 'path to a statically-linked shell to use as /bin/sh in sandboxes (usually busybox)',
 )

+option('pasta-path', type : 'string', value : 'pasta',
+  description : 'path to the location of pasta (provided by passt)',
+)
+
 option('enable-tests', type : 'boolean', value : true,
  description : 'whether to enable tests or not (requires rapidcheck and gtest)',
 )
@@ -0,0 +1,60 @@
+{
+  lib,
+  stdenv,
+  buildPackages,
+  fetchurl,
+  getconf,
+  gitUpdater,
+  testers,
+}:
+
+stdenv.mkDerivation (finalAttrs: {
+  pname = "passt";
+  version = "2025_02_17.a1e48a0";
+
+  src = fetchurl {
+    url = "https://passt.top/passt/snapshot/passt-${finalAttrs.version}.tar.gz";
+    hash = "sha256-/FUXxeYv3Lb0DiXmbS2PUzfLL5ZwHJ42tiuH7YnlljE=";
+  };
+
+  postPatch = ''
+    substituteInPlace Makefile --replace-fail \
+      'PAGE_SIZE=$(shell getconf PAGE_SIZE)' \
+      "PAGE_SIZE=$(${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe getconf} PAGE_SIZE)"
+  '';
+
+  makeFlags = [
+    "prefix=${placeholder "out"}"
+    "VERSION=${finalAttrs.version}"
+  ];
+
+  passthru = {
+    tests.version = testers.testVersion { package = finalAttrs.finalPackage; };
+
+    updateScript = gitUpdater { url = "https://passt.top/passt"; };
+  };
+
+  meta = with lib; {
+    homepage = "https://passt.top/passt/about/";
+    description = "Plug A Simple Socket Transport";
+    longDescription = ''
+      passt implements a translation layer between a Layer-2 network interface
+      and native Layer-4 sockets (TCP, UDP, ICMP/ICMPv6 echo) on a host.
+      It doesn't require any capabilities or privileges, and it can be used as
+      a simple replacement for Slirp.
+
+      pasta (same binary as passt, different command) offers equivalent
+      functionality, for network namespaces: traffic is forwarded using a tap
+      interface inside the namespace, without the need to create further
+      interfaces on the host, hence not requiring any capabilities or
+      privileges.
+    '';
+    license = [
+      licenses.bsd3 # and
+      licenses.gpl2Plus
+    ];
+    platforms = platforms.linux;
+    maintainers = with maintainers; [ _8aed ];
+    mainProgram = "passt";
+  };
+})
@@ -1 +1,2 @@
-d @localstatedir@/nix/daemon-socket 0755 root root - -
+d @localstatedir@/nix/daemon-socket 0755 root root -  -
+d @localstatedir@/nix/builds        0755 root root 7d -
@@ -30,6 +30,8 @@
  lix-clang-tidy ? null,
  llvmPackages,
  lsof,
+  # FIXME: remove default after dropping NixOS 24.05
+  lowdown-unsandboxed ? lowdown,
  lowdown,
  mdbook,
  mdbook-linkcheck,
@@ -37,6 +39,8 @@
  meson,
  ninja,
  openssl,
+  # FIXME: we need passt 2024_12_11.09478d5 or newer, i.e. nixos 25.05 or later
+  passt-lix ? __forDefaults.passt-lix,
  pegtl,
  pkg-config,
  python3,
@@ -85,6 +89,8 @@

    lix-doc = callPackage ./lix-doc/package.nix { };
    build-release-notes = callPackage ./maintainers/build-release-notes.nix { };
+
+    passt-lix = callPackage ./misc/passt.nix { };
  },
 }:
 let
@@ -173,12 +179,13 @@ stdenv.mkDerivation (finalAttrs: {

  VERSION_SUFFIX = versionSuffix;

-  outputs =
-    [ "out" ]
-    ++ lib.optionals (!finalAttrs.dontBuild) [
-      "dev"
-      "doc"
-    ];
+  outputs = [
+    "out"
+  ]
+  ++ lib.optionals (!finalAttrs.dontBuild) [
+    "dev"
+    "doc"
+  ];

  dontBuild = lintInsteadOfBuild;

@@ -193,6 +200,7 @@ stdenv.mkDerivation (finalAttrs: {
      # which don't actually get added to PATH. And buildInputs is correct over
      # nativeBuildInputs since this should be a busybox executable on the host.
      "-Dsandbox-shell=${lib.getExe' busybox-sandbox-shell "busybox"}"
+      "-Dpasta-path=${lib.getExe' passt-lix "pasta"}"
    ]
    ++ lib.optional hostPlatform.isStatic "-Denable-embedded-sandbox-shell=true"
    ++ lib.optional (finalAttrs.dontBuild && !lintInsteadOfBuild) "-Denable-build=false"
@@ -213,66 +221,65 @@ stdenv.mkDerivation (finalAttrs: {
  # We only include CMake so that Meson can locate toml11, which only ships CMake dependency metadata.
  dontUseCmakeConfigure = true;

-  nativeBuildInputs =
-    [
-      python3
-      meson
-      ninja
-      cmake
-    ]
-    ++ [
-      (lib.getBin lowdown)
-      mdbook
-      mdbook-linkcheck
-    ]
-    ++ [
-      pkg-config
+  nativeBuildInputs = [
+    python3
+    meson
+    ninja
+    cmake
+  ]
+  ++ [
+    (lib.getBin lowdown-unsandboxed)
+    mdbook
+    mdbook-linkcheck
+  ]
+  ++ [
+    pkg-config

-      # Tests
-      git
-      mercurial
-      jq
-      lsof
-    ]
-    ++ lib.optional hostPlatform.isLinux util-linuxMinimal
-    ++ lib.optional (!officialRelease && buildUnreleasedNotes) build-release-notes
-    ++ lib.optional internalApiDocs doxygen
-    ++ lib.optionals lintInsteadOfBuild [
-      # required for a wrapped clang-tidy
-      llvmPackages.clang-tools
-      # required for run-clang-tidy
-      llvmPackages.clang-unwrapped
-    ];
+    # Tests
+    git
+    mercurial
+    jq
+    lsof
+  ]
+  ++ lib.optional hostPlatform.isLinux util-linuxMinimal
+  ++ lib.optional (!officialRelease && buildUnreleasedNotes) build-release-notes
+  ++ lib.optional internalApiDocs doxygen
+  ++ lib.optionals lintInsteadOfBuild [
+    # required for a wrapped clang-tidy
+    llvmPackages.clang-tools
+    # required for run-clang-tidy
+    llvmPackages.clang-unwrapped
+  ];

-  buildInputs =
-    [
-      curl
-      bzip2
-      xz
-      brotli
-      editline-lix
-      openssl
-      sqlite
-      libarchive
-      boost
-      lowdown
-      libsodium
-      toml11
-      lix-doc
-      pegtl
-    ]
-    ++ lib.optionals hostPlatform.isLinux [
-      libseccomp
-      busybox-sandbox-shell
-    ]
-    ++ lib.optional internalApiDocs rapidcheck
-    ++ lib.optional hostPlatform.isx86_64 libcpuid
-    # There have been issues building these dependencies
-    ++ lib.optional (hostPlatform.canExecute buildPlatform) aws-sdk-cpp-nix
-    ++ lib.optionals (finalAttrs.dontBuild) maybePropagatedInputs
-    # I am so sorry. This is because checkInputs are required to pass
-    # configure, but we don't actually want to *run* the checks here.
-    ++ lib.optionals lintInsteadOfBuild finalAttrs.checkInputs;
+  buildInputs = [
+    curl
+    bzip2
+    xz
+    brotli
+    editline-lix
+    openssl
+    sqlite
+    libarchive
+    boost
+    lowdown
+    libsodium
+    toml11
+    lix-doc
+    pegtl
+  ]
+  ++ lib.optionals hostPlatform.isLinux [
+    libseccomp
+    busybox-sandbox-shell
+    passt-lix
+  ]
+  ++ lib.optional internalApiDocs rapidcheck
+  ++ lib.optional hostPlatform.isx86_64 libcpuid
+  # There have been issues building these dependencies
+  ++ lib.optional (hostPlatform.canExecute buildPlatform) aws-sdk-cpp-nix
+  ++ lib.optionals (finalAttrs.dontBuild) maybePropagatedInputs
+  # I am so sorry. This is because checkInputs are required to pass
+  # configure, but we don't actually want to *run* the checks here.
+  ++ lib.optionals lintInsteadOfBuild finalAttrs.checkInputs;

  checkInputs = [
    gtest
@@ -419,7 +426,6 @@ stdenv.mkDerivation (finalAttrs: {
        just,
        nixfmt-rfc-style,
        skopeo,
-        xonsh,

        # Lix specific packages
        pre-commit-checks,
@@ -439,7 +445,7 @@ stdenv.mkDerivation (finalAttrs: {
            p.requests
            p.xdg-base-dirs
            p.packaging
-            (p.toPythonModule xonsh.passthru.unwrapped)
+            p.xonsh
          ]
        );
        pythonEnv = python3.withPackages pythonPackages;
@@ -35,19 +35,18 @@ perl.pkgs.toPerlModule (
      ninja
    ];

-    buildInputs =
-      [
-        nix
-        curl
-        bzip2
-        xz
-        perl
-        boost
-        perlPackages.DBI
-        perlPackages.DBDSQLite
-      ]
-      ++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium
-      ++ lib.optional stdenv.isDarwin darwin.apple_sdk.frameworks.Security;
+    buildInputs = [
+      nix
+      curl
+      bzip2
+      xz
+      perl
+      boost
+      perlPackages.DBI
+      perlPackages.DBDSQLite
+    ]
+    ++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium
+    ++ lib.optional stdenv.isDarwin darwin.apple_sdk.frameworks.Security;

    # Nixpkgs' Meson hook likes to set this to "plain".
    mesonBuildType = "debugoptimized";
@@ -19,10 +19,7 @@ endif

 cxx = meson.get_compiler('cpp')

-# Really version 5.8.0, but Perl's version string is of the form
-# "This is perl 5, version 38, subversion 2", for 5.38.2, so as far
-# as Meson is concerned, the version of Perl we need is 8 or greater.
-perl = find_program('perl', version : '>=8')
+perl = find_program('perl')

 # "compiler to convert Perl XS code into C code"
 xsubpp = find_program('xsubpp')
@@ -11,13 +11,28 @@ namespace nix {
 std::string renderMarkdownToTerminal(std::string_view markdown)
 {
    int windowWidth = getWindowSize().second;
+    size_t lowdown_cols = std::max(windowWidth - 5, 60);

-    struct lowdown_opts opts {
+    struct lowdown_opts opts{
        .type = LOWDOWN_TERM,
+#ifdef LOWDOWN_SEPARATE_TERM_OPTS
+        .term =
+            {
+                .cols = lowdown_cols,
+                .width = 0,
+                .hmargin = 0,
+                .hpadding = 4,
+                .vmargin = 0,
+                .centre = 0,
+            },
+        // maxdepth needs to be part of the ifdefs to match declaration order
        .maxdepth = 20,
-        .cols = (size_t) std::max(windowWidth - 5, 60),
+#else
+        .maxdepth = 20,
+        .cols = lowdown_cols,
        .hmargin = 0,
        .vmargin = 0,
+#endif /* LOWDOWN_SEPARATE_TERM_OPTS */
        .feat = LOWDOWN_COMMONMARK | LOWDOWN_FENCED | LOWDOWN_DEFLIST | LOWDOWN_TABLES,
        .oflags = LOWDOWN_TERM_NOLINK,
    };
@@ -251,7 +251,7 @@ void runNix(Path program, const Strings & args)
        .program = settings.nixBinDir+ "/" + program,
        .args = args,
        .environment = subprocessEnv,
-    }).wait();
+    }).waitAndCheck();

    return;
 }
@@ -650,7 +650,7 @@ ProcessLineResult NixRepl::processLine(std::string line)

        // runProgram redirects stdout to a StringSink,
        // using runProgram2 to allow editors to display their UI
-        runProgram2(RunOptions { .program = editor, .searchPath = true, .args = args }).wait();
+        runProgram2(RunOptions { .program = editor, .searchPath = true, .args = args }).waitAndCheck();

        // Reload right after exiting the editor if path is not in store
        // Store is immutable, so there could be no changes, so there's no need to reload
@@ -129,7 +129,6 @@ std::pair<SourcePath, uint32_t> findPackageFilename(EvalState & state, Value & v
    try {
        auto colon = fn.rfind(':');
        if (colon == std::string::npos) fail();
-        std::string filename(fn, 0, colon);
        auto lineno = std::stoi(std::string(fn, colon + 1, std::string::npos));
        return {CanonPath(fn.substr(0, colon)), lineno};
    } catch (std::invalid_argument & e) {
@@ -212,7 +212,7 @@ struct ExprAttrs : Expr
        AttrDef() { };

        template<typename T>
-        const T & chooseByKind(const T & plain, const T & inherited, const T & inheritedFrom) const
+        T chooseByKind(const T & plain, const T & inherited, const T & inheritedFrom) const
        {
            switch (kind) {
            case Kind::Plain:
@@ -624,10 +624,13 @@ struct nothing : p::nothing<Rule> {
    static_assert(!std::is_base_of_v<semantic, Rule>);
 };

-
-
 template<typename Self, typename OpCtx, typename AttrPathT, typename ExprT>
-struct operator_semantics {
+struct operator_semantics
+{
+private:
+    operator_semantics() = default;
+    friend Self;
+public:
    struct has_attr : grammar::op::has_attr {
        AttrPathT path;
    };
@@ -250,9 +250,7 @@ std::optional<Hash> Input::getNarHash() const

 std::optional<std::string> Input::getRef() const
 {
-    if (auto s = maybeGetStrAttr(attrs, "ref"))
-        return *s;
-    return {};
+    return maybeGetStrAttr(attrs, "ref");
 }

 std::optional<Hash> Input::getRev() const
@@ -273,16 +271,12 @@ std::optional<Hash> Input::getRev() const

 std::optional<uint64_t> Input::getRevCount() const
 {
-    if (auto n = maybeGetIntAttr(attrs, "revCount"))
-        return *n;
-    return {};
+    return maybeGetIntAttr(attrs, "revCount");
 }

 std::optional<time_t> Input::getLastModified() const
 {
-    if (auto n = maybeGetIntAttr(attrs, "lastModified"))
-        return *n;
-    return {};
+    return maybeGetIntAttr(attrs, "lastModified");
 }

 ParsedURL InputScheme::toURL(const Input & input) const
@@ -17,6 +17,7 @@
 #include <string.h>
 #include <sys/time.h>
 #include <sys/wait.h>
+#include <unistd.h>

 using namespace std::string_literals;

@@ -129,7 +130,7 @@ std::optional<std::string> readHeadCached(const std::string & actualUrl)
        // This function must behave the same way, so we return the expired
        // cached ref here.
        warn("could not get HEAD ref for repository '%s'; using expired cached ref '%s'", actualUrl, *cachedRef);
-        return *cachedRef;
+        return cachedRef;
    }

    return std::nullopt;
@@ -163,7 +164,7 @@ WorkdirInfo getWorkdirInfo(const Input & input, const Path & workdir)
        .program = "git",
        .args = { "-C", workdir, "--git-dir", gitDir, "rev-parse", "--verify", "--no-revs", "HEAD^{commit}" },
        .environment = env,
-        .mergeStderrToStdout = true
+        .redirections = {{.from = STDERR_FILENO, .to = STDOUT_FILENO}},
    });
    auto exitCode = WEXITSTATUS(result.first);
    auto errorMessage = result.second;
@@ -625,7 +626,7 @@ struct GitInputScheme : InputScheme
        auto result = runProgram(RunOptions {
            .program = "git",
            .args = { "-C", repoDir, "--git-dir", gitDir, "cat-file", "commit", input.getRev()->gitRev() },
-            .mergeStderrToStdout = true
+            .redirections = {{.from = STDERR_FILENO, .to = STDOUT_FILENO}},
        });
        if (WEXITSTATUS(result.first) == 128
            && result.second.find("bad file") != std::string::npos)
@@ -690,7 +691,7 @@ struct GitInputScheme : InputScheme
                .args = { "-C", repoDir, "--git-dir", gitDir, "archive", input.getRev()->gitRev() },
                .captureStdout = true,
            });
-            Finally const _wait([&] { proc.wait(); });
+            Finally const _wait([&] { proc.waitAndCheck(); });

            unpackTarfile(*proc.getStdout(), tmpDir);
        }
@@ -917,9 +917,9 @@ void runPostBuildHook(
        .program = settings.postBuildHook,
        .environment = hookEnvironment,
        .captureStdout = true,
-        .mergeStderrToStdout = true,
+        .redirections = {{.from = STDERR_FILENO, .to = STDOUT_FILENO}},
    });
-    Finally const _wait([&] { proc.wait(); });
+    Finally const _wait([&] { proc.waitAndCheck(); });

    // FIXME just process the data, without a wrapper sink class
    proc.getStdout()->drainInto(sink);
@@ -1212,7 +1212,7 @@ HookReply DerivationGoal::tryBuildHook()
    hook->toHook.writeSide.reset();

    /* Create the log file and pipe. */
-    Path logFile = openLogFile();
+    openLogFile();

    std::set<int> fds;
    fds.insert(hook->fromHook.readSide.get());
@@ -17,6 +17,7 @@
 #include "namespaces.hh"
 #include "child.hh"
 #include "unix-domain-socket.hh"
+#include "platform/linux.hh"

 #include <regex>
 #include <queue>
@@ -331,7 +332,7 @@ bool LocalDerivationGoal::cleanupDecideWhetherDiskFull()
        if (statvfs(localStore.realStoreDir.get().c_str(), &st) == 0 &&
            (uint64_t) st.f_bavail * st.f_bsize < required)
            diskFull = true;
-        if (statvfs(tmpDir.c_str(), &st) == 0 &&
+        if (statvfs(tmpDirRoot.c_str(), &st) == 0 &&
            (uint64_t) st.f_bavail * st.f_bsize < required)
            diskFull = true;
    }
@@ -474,11 +475,87 @@ void LocalDerivationGoal::startBuilder()
            settings.thisSystem,
            concatStringsSep<StringSet>(", ", worker.store.systemFeatures));

-    /* Create a temporary directory where the build will take
-       place. */
-    tmpDir = createTempDir(settings.buildDir.get().value_or(""), "nix-build-" + std::string(drvPath.name()), false, false, 0700);
+    try {
+        auto buildDir = worker.buildDirOverride.value_or(settings.buildDir.get());

-    chownToBuilder(tmpDir);
+        createDirs(buildDir);
+
+        /* Create a temporary directory where the build will take
+           place. */
+        tmpDirRoot =
+            createTempDir(buildDir, "nix-build-" + std::string(drvPath.name()), false, false, 0700);
+    } catch (SysError & e) {
+        /*
+         * Fallback to the global tmpdir and create a safe space there
+         * only if it's a permission error.
+         */
+        if (e.errNo != EACCES) {
+            throw;
+        }
+
+        auto globalTmp = defaultTempDir();
+        createDirs(globalTmp);
+#if __APPLE__
+        /* macOS filesystem namespacing does not exist, to avoid breaking builds, we need to weaken
+         * the mode bits on the top-level directory. This avoids issues like
+         * https://github.com/NixOS/nix/pull/11031. */
+        constexpr int toplevelDirMode = 0755;
+#else
+        constexpr int toplevelDirMode = 0700;
+#endif
+        auto nixBuildsTmp =
+            createTempDir(globalTmp, fmt("nix-builds-%s", geteuid()), false, false, toplevelDirMode);
+        warn(
+            "Failed to use the system-wide build directory '%s', falling back to a temporary "
+            "directory inside '%s'",
+            settings.buildDir.get(),
+            nixBuildsTmp
+        );
+        worker.buildDirOverride = nixBuildsTmp;
+        tmpDirRoot = createTempDir(
+            nixBuildsTmp, "nix-build-" + std::string(drvPath.name()), false, false, 0700
+        );
+    }
+    /* The TOCTOU between the previous mkdir call and this open call is unavoidable due to
+     * POSIX semantics.*/
+    tmpDirRootFd = AutoCloseFD{open(tmpDirRoot.c_str(), O_RDONLY | O_NOFOLLOW | O_DIRECTORY)};
+    if (!tmpDirRootFd)
+        throw SysError("failed to open the build temporary directory descriptor '%1%'", tmpDirRoot);
+
+    // place the actual build directory in a subdirectory of tmpDirRoot. if
+    // we do not do this a build can `chown 777` its build directory and so
+    // make it accessible to everyone in the system, breaking isolation. we
+    // also need the intermediate level to be inaccessible to others. build
+    // processes must be able to at least traverse to the directory though,
+    // without being able to chmod. this means either mode 0750 or 0710. we
+    // cannot use 0710 because the libarchive we link with is compiled with
+    // an old apple sdk that does not have O_SEARCH, which makes libarchive
+    // try to open tmpDirRoot for *read* and fail because g+r is not set. a
+    // future update to nixpkgs may fix this. until then we do not lose any
+    // security by setting mode 0750 because we use only a single subdir in
+    // tmpDirRoot, so being able to list its parent doesn't break anything.
+    //
+    // use a short name to not increase the path length too much on darwin.
+    // darwin has a severe sockaddr_un path length limitation, so this does
+    // make a difference over more evocative names. we use `b` for `build`.
+    tmpDir = tmpDirRoot + "/b";
+    if (mkdirat(tmpDirRootFd.get(), "b", 0700)) {
+        throw SysError("failed to create the build temporary directory '%1%'", tmpDir);
+    }
+    tmpDirFd = AutoCloseFD{openat(tmpDirRootFd.get(), "b", O_RDONLY | O_NOFOLLOW | O_DIRECTORY)};
+    if (!tmpDirFd)
+        throw SysError("failed to open the build temporary directory descriptor '%1%'", tmpDir);
+
+    chownToBuilder(tmpDirFd);
+
+    if (buildUser) {
+        if (fchown(tmpDirRootFd.get(), -1, buildUser->getGID()) == -1) {
+            throw SysError("cannot change ownership of '%1%'", tmpDirRoot);
+        }
+        if (fchmod(tmpDirRootFd.get(), 0750) == -1) {
+            throw SysError("cannot change mode of '%1%'", tmpDirRoot);
+        }
+    }

    for (auto & [outputName, status] : initialOutputs) {
        /* Set scratch path we'll actually use during the build.
@@ -701,7 +778,7 @@ void LocalDerivationGoal::startBuilder()
        printMsg(lvlVomit, "setting builder env variable '%1%'='%2%'", i.first, i.second);

    /* Create the log file. */
-    Path logFile = openLogFile();
+    openLogFile();

    /* Create a pseudoterminal to get the output of the builder. */
    builderOut = AutoCloseFD{posix_openpt(O_RDWR | O_NOCTTY)};
@@ -820,8 +897,13 @@ void LocalDerivationGoal::initTmpDir() {
                auto hash = hashString(HashType::SHA256, i.first);
                std::string fn = ".attr-" + hash.to_string(Base::Base32, false);
                Path p = tmpDir + "/" + fn;
-                writeFile(p, rewriteStrings(i.second, inputRewrites));
-                chownToBuilder(p);
+                /* TODO(jade): we should have BorrowedFD instead of OwnedFD. */
+                AutoCloseFD passAsFileFd{openat(tmpDirFd.get(), fn.c_str(), O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC | O_EXCL | O_NOFOLLOW, 0666)};
+                if (!passAsFileFd) {
+                    throw SysError("opening `passAsFile` file in the sandbox '%1%'", p);
+                }
+                writeFile(passAsFileFd, rewriteStrings(i.second, inputRewrites));
+                chownToBuilder(passAsFileFd);
                env[i.first + "Path"] = tmpDirInSandbox + "/" + fn;
            }
        }
@@ -1348,6 +1430,13 @@ void LocalDerivationGoal::chownToBuilder(const Path & path)
        throw SysError("cannot change ownership of '%1%'", path);
 }

+void LocalDerivationGoal::chownToBuilder(const AutoCloseFD & fd)
+{
+    if (!buildUser) return;
+    if (fchown(fd.get(), buildUser->getUID(), buildUser->getGID()) == -1)
+        throw SysError("cannot change ownership of file '%1%'", fd.guessOrInventPath());
+}
+

 void LocalDerivationGoal::runChild()
 {
@@ -1364,13 +1453,20 @@ void LocalDerivationGoal::runChild()

        bool setUser = true;

-        /* Make the contents of netrc available to builtin:fetchurl
-           (which may run under a different uid and/or in a sandbox). */
+        /* Make the contents of netrc and the CA certificate bundle
+           available to builtin:fetchurl (which may run under a
+           different uid and/or in a sandbox). */
        std::string netrcData;
-        try {
-            if (drv->isBuiltin() && drv->builder == "builtin:fetchurl" && !derivationType->isSandboxed())
+        std::string caFileData;
+        if (drv->isBuiltin() && drv->builder == "builtin:fetchurl" && !derivationType->isSandboxed()) {
+            try {
                netrcData = readFile(settings.netrcFile);
-        } catch (SysError &) { }
+            } catch (SysError &) { }
+
+            try {
+                caFileData = readFile(settings.caFile);
+            } catch (SysError &) { }
+        }

 #if __linux__
        if (useChroot) {
@@ -1468,7 +1564,7 @@ void LocalDerivationGoal::runChild()
                /* N.B. it is realistic that these paths might not exist. It
                   happens when testing Nix building fixed-output derivations
                   within a pure derivation. */
-                for (auto & path : { "/etc/resolv.conf", "/etc/services", "/etc/hosts" })
+                for (auto & path : { "/etc/services", "/etc/hosts" })
                    if (pathExists(path)) {
                        // Copy the actual file, not the symlink, because we don't know where
                        // the symlink is pointing, and we don't want to chase down the entire
@@ -1489,6 +1585,11 @@ void LocalDerivationGoal::runChild()
                        copyFile(path, chrootRootDir + path, { .followSymlinks = true });
                    }

+                if (pathExists("/etc/resolv.conf")) {
+                    const auto resolvConf = rewriteResolvConf(readFile("/etc/resolv.conf"));
+                    writeFile(chrootRootDir + "/etc/resolv.conf", resolvConf);
+                }
+
                if (settings.caFile != "" && pathExists(settings.caFile)) {
                    // For the same reasons as above, copy the CA certificates file too.
                    // It should be even less likely to change during the build than resolv.conf.
@@ -1610,6 +1711,36 @@ void LocalDerivationGoal::runChild()
            if (setuid(sandboxUid()) == -1)
                throw SysError("setuid failed");

+            if (runPasta) {
+                // wait for the pasta interface to appear. pasta can't signal us when
+                // it's done setting up the namespace, so we have to wait for a while
+                AutoCloseFD fd(socket(PF_INET, SOCK_DGRAM, IPPROTO_IP));
+                if (!fd) throw SysError("cannot open IP socket");
+
+                struct ifreq ifr;
+                strcpy(ifr.ifr_name, LinuxLocalDerivationGoal::PASTA_NS_IFNAME);
+                // wait two minutes for the interface to appear. if it does not do so
+                // we are either grossly overloaded, or pasta startup failed somehow.
+                static constexpr int SINGLE_WAIT_US = 1000;
+                static constexpr int TOTAL_WAIT_US = 120'000'000;
+                for (unsigned tries = 0; ; tries++) {
+                    if (tries > TOTAL_WAIT_US / SINGLE_WAIT_US) {
+                        throw Error(
+                            "sandbox network setup timed out, please check daemon logs for "
+                            "possible error output."
+                        );
+                    } else if (ioctl(fd.get(), SIOCGIFFLAGS, &ifr) == 0) {
+                        if ((ifr.ifr_ifru.ifru_flags & IFF_UP) != 0) {
+                            break;
+                        }
+                    } else if (errno == ENODEV) {
+                        usleep(SINGLE_WAIT_US);
+                    } else {
+                        throw SysError("cannot get loopback interface flags");
+                    }
+                }
+            }
+
            setUser = false;
        }
 #endif
@@ -1772,6 +1903,8 @@ void LocalDerivationGoal::runChild()

        if (getEnv("_NIX_TEST_NO_SANDBOX") != "1") {
            Strings sandboxArgs;
+            sandboxArgs.push_back("_NIX_BUILD_TOP");
+            sandboxArgs.push_back(tmpDir);
            sandboxArgs.push_back("_GLOBAL_TMP_DIR");
            sandboxArgs.push_back(globalTmpDir);
            if (allowLocalNetworking) {
@@ -1805,7 +1938,7 @@ void LocalDerivationGoal::runChild()
                    e.second = rewriteStrings(e.second, inputRewrites);

                if (drv->builder == "builtin:fetchurl")
-                    builtinFetchurl(drv2, netrcData);
+                    builtinFetchurl(drv2, netrcData, caFileData);
                else if (drv->builder == "builtin:buildenv")
                    builtinBuildenv(drv2);
                else if (drv->builder == "builtin:unpack-channel")
@@ -2582,16 +2715,16 @@ void LocalDerivationGoal::checkOutputs(const std::map<std::string, ValidPathInfo

 void LocalDerivationGoal::deleteTmpDir(bool force)
 {
-    if (tmpDir != "") {
+    if (tmpDirRoot != "") {
        /* Don't keep temporary directories for builtins because they
           might have privileged stuff (like a copy of netrc). */
        if (settings.keepFailed && !force && !drv->isBuiltin()) {
-            printError("note: keeping build directory '%s'", tmpDir);
-            chmod(tmpDir.c_str(), 0755);
+            printError("note: keeping build directory '%s'", tmpDirRoot);
+            chmod(tmpDirRoot.c_str(), 0755);
        }
        else
-            deletePath(tmpDir);
-        tmpDir = "";
+            deletePath(tmpDirRoot);
+        tmpDirRoot = "";
    }
 }

@@ -29,7 +29,12 @@ struct LocalDerivationGoal : public DerivationGoal
    /**
     * The temporary directory.
     */
-    Path tmpDir;
+    Path tmpDirRoot, tmpDir;
+
+    /**
+     * The temporary directory file descriptor
+     */
+    AutoCloseFD tmpDirRootFd, tmpDirFd;

    /**
     * The path of the temporary directory in the sandbox.
@@ -244,10 +249,18 @@ struct LocalDerivationGoal : public DerivationGoal
    void addDependency(const StorePath & path);

    /**
-     * Make a file owned by the builder.
+     * Make a file owned by the builder addressed by its path.
+     *
+     * SAFETY: this function is prone to TOCTOU as it receives a path and not a descriptor.
+     * It's only safe to call in a child of a directory only visible to the owner.
     */
    void chownToBuilder(const Path & path);

+    /**
+     * Make a file owned by the builder addressed by its file descriptor.
+     */
+    void chownToBuilder(const AutoCloseFD & fd);
+
    int getChildStatus() override;

    /**
@@ -324,6 +337,12 @@ struct LocalDerivationGoal : public DerivationGoal
 protected:
    using DerivationGoal::DerivationGoal;

+    /**
+     * Whether to run pasta for network-endowed derivations. Running pasta
+     * currently requires actively waiting for its net-ns setup to finish.
+     */
+    bool runPasta = false;
+
    /**
     * Setup dependencies outside the sandbox.
     * Called in the parent nix process.
@@ -333,6 +352,15 @@ protected:
        throw Error("sandboxing builds is not supported on this platform");
    };

+    /**
+     * Rewrite resolv.conf for use in the sandbox. Used in the linux platform
+     * to replace nameservers * when using pasta for fixed output derivations.
+     */
+    virtual std::string rewriteResolvConf(std::string fromHost)
+    {
+        return fromHost;
+    }
+
    /**
     * Create a new process that runs `openSlave` and `runChild`
     * On some platforms this process is created with sandboxing flags.
@@ -26,12 +26,14 @@ R""(
 ; Allow getpwuid.
 (allow mach-lookup (global-name "com.apple.system.opendirectoryd.libinfo"))

-; Access to /tmp.
+; Access to /tmp and the build directory.
 ; The network-outbound/network-inbound ones are for unix domain sockets, which
 ; we allow access to in TMPDIR (but if we allow them more broadly, you could in
 ; theory escape the sandbox)
 (allow file* process-exec network-outbound network-inbound
-       (literal "/tmp") (subpath TMPDIR))
+       (literal "/tmp")
+       (subpath TMPDIR)
+       (subpath (param "_NIX_BUILD_TOP")))

 ; Some packages like to read the system version.
 (allow file-read*
@@ -161,6 +161,7 @@ public:

    Store & store;
    Store & evalStore;
+    std::optional<Path> buildDirOverride;

    struct HookState {
        std::unique_ptr<HookInstance> instance;
@@ -6,7 +6,7 @@
 namespace nix {

 // TODO: make pluggable.
-void builtinFetchurl(const BasicDerivation & drv, const std::string & netrcData);
+void builtinFetchurl(const BasicDerivation & drv, const std::string & netrcData, const std::string & caFileData);
 void builtinUnpackChannel(const BasicDerivation & drv);

 }
@@ -6,7 +6,7 @@

 namespace nix {

-void builtinFetchurl(const BasicDerivation & drv, const std::string & netrcData)
+void builtinFetchurl(const BasicDerivation & drv, const std::string & netrcData, const std::string & caFileData)
 {
    /* Make the host's netrc data available. Too bad curl requires
       this to be stored in a file. It would be nice if we could just
@@ -16,6 +16,9 @@ void builtinFetchurl(const BasicDerivation & drv, const std::string & netrcData)
        writeFile(settings.netrcFile, netrcData, 0600);
    }

+    settings.caFile = "ca-certificates.crt";
+    writeFile(settings.caFile, caFileData, 0600);
+
    auto getAttr = [&](const std::string & name) {
        auto i = drv.env.find(name);
        if (i == drv.env.end()) throw Error("attribute '%s' missing", name);
@@ -32,10 +35,7 @@ void builtinFetchurl(const BasicDerivation & drv, const std::string & netrcData)

    auto fetch = [&](const std::string & url) {

-        /* No need to do TLS verification, because we check the hash of
-           the result anyway. */
        FileTransferRequest request(url);
-        request.verifyTLS = false;

        auto raw = fileTransfer->download(std::move(request));
        auto decompressor = makeDecompressionSource(
@@ -93,7 +93,7 @@ WireFormatGenerator CommonProto::Serialise<std::optional<ContentAddress>>::write
 {
    return [](std::string s) -> WireFormatGenerator {
        co_yield s;
-    }(caOpt ? renderContentAddress(*caOpt) : "");
+    }(caOpt ? renderContentAddress(caOpt) : "");
 }

 }
@@ -45,8 +45,15 @@ std::string SecretKey::signDetached(std::string_view data) const
 {
    unsigned char sig[crypto_sign_BYTES];
    unsigned long long sigLen;
-    crypto_sign_detached(sig, &sigLen, charptr_cast<const unsigned char *>(data.data()), data.size(),
-        charptr_cast<const unsigned char *>(key.data()));
+    crypto_sign_detached(
+        sig,
+        &sigLen,
+        // the following is not a string function so no null termination issues are possible here.
+        // NOLINTNEXTLINE(bugprone-suspicious-stringview-data-usage)
+        charptr_cast<const unsigned char *>(data.data()),
+        data.size(),
+        charptr_cast<const unsigned char *>(key.data())
+    );
    return name + ":" + base64Encode(std::string(reinterpret_cast<char *>(sig), sigLen));
 }

@@ -13,14 +13,9 @@ void Store::exportPaths(const StorePathSet & paths, Sink & sink)
    auto sorted = topoSortPaths(paths);
    std::reverse(sorted.begin(), sorted.end());

-    std::string doneLabel("paths exported");
-    //logger->incExpected(doneLabel, sorted.size());
-
    for (auto & path : sorted) {
-        //Activity act(*logger, lvlInfo, "exporting path '%s'", path);
        sink << 1;
        exportPath(path, sink);
-        //logger->incProgress(doneLabel);
    }

    sink << 0;
@@ -78,6 +78,9 @@ Settings::Settings()
 #if defined(__linux__) && defined(SANDBOX_SHELL)
    sandboxPaths = tokenizeString<StringSet>("/bin/sh=" SANDBOX_SHELL);
 #endif
+#if defined(__linux__) && defined(PASTA_PATH)
+    pastaPath.setDefault(PASTA_PATH);
+#endif

    /* chroot-like behavior from Apple's sandbox */
 #if __APPLE__
@@ -233,7 +236,8 @@ StringSet Settings::getDefaultExtraPlatforms()
    // x86_64 in aarch64 environments or vice versa since they can
    // always exec with their own binary preferences.
    if (std::string{SYSTEM} == "aarch64-darwin" &&
-        runProgram(RunOptions {.program = "arch", .args = {"-arch", "x86_64", "/usr/bin/true"}, .mergeStderrToStdout = true}).first == 0)
+        runProgram(RunOptions {.program = "arch", .args = {"-arch", "x86_64", "/usr/bin/true"},
+            .redirections = {{.from = STDERR_FILENO, .to = STDOUT_FILENO}}}).first == 0)
        extraPlatforms.insert("x86_64-darwin");
 #endif

@@ -561,6 +561,16 @@ public:
        )",
        {"build-chroot-dirs", "build-sandbox-paths"}};

+#if defined(__linux__)
+    Setting<Path> pastaPath{this, "", "pasta-path",
+        R"(
+            If set to an absolute path, enables fully sandboxing fixed-output
+            derivations, by using `pasta` to pass network traffic between the
+            private network namespace. This allows for greater levels of isolation
+            of builds to the host.
+        )"};
+#endif
+
    Setting<bool> sandboxFallback{this, true, "sandbox-fallback",
        "Whether to disable sandboxing when the kernel doesn't allow it."};

@@ -604,16 +614,25 @@ public:
        )"};
 #endif

-    Setting<std::optional<Path>> buildDir{this, std::nullopt, "build-dir",
+    Setting<Path> buildDir{this, nixStateDir + "/builds", "build-dir",
        R"(
            The directory on the host, in which derivations' temporary build directories are created.

-            If not set, Nix will use the system temporary directory indicated by the `TMPDIR` environment variable.
-            Note that builds are often performed by the Nix daemon, so its `TMPDIR` is used, and not that of the Nix command line interface.
+            If not set, Lix will use the `builds` subdirectory of its configured state directory.
+            Lix will create this directory automatically with suitable permissions if it does not
+            exist, otherwise its permissions must allow all users to traverse the directory (i.e.
+            it must have `o+x` set, in unix parlance) for non-sandboxed builds to work correctly.

            This is also the location where [`--keep-failed`](@docroot@/command-ref/opt-common.md#opt-keep-failed) leaves its files.

            If Nix runs without sandbox, or if the platform does not support sandboxing with bind mounts (e.g. macOS), then the [`builder`](@docroot@/language/derivations.md#attr-builder)'s environment will contain this directory, instead of the virtual location [`sandbox-build-dir`](#conf-sandbox-build-dir).
+
+            > Important:
+            >
+            > `build-dir` must not be set to a world-writable directory. Placing temporary build
+            > directories in a world-writable place allows other users to access or modify build
+            > data that is currently in use. This alone is merely an impurity, but combined with
+            > another factor this has allowed malicious derivations to escape the build sandbox.
        )"};

    Setting<PathSet> allowedImpureHostPrefixes{this, {}, "allowed-impure-host-deps",
@@ -167,7 +167,7 @@ void migrateCASchema(SQLite& db, Path schemaPath, AutoCloseFD& lockFd)
            txn.commit();
        }

-        writeFile(schemaPath, fmt("%d", nixCASchemaVersion), 0666, true);
+        writeFileAndSync(schemaPath, fmt("%d", nixCASchemaVersion), 0666);
        lockFile(lockFd.get(), ltRead, true);
    }
 }
@@ -304,7 +304,7 @@ LocalStore::LocalStore(const Params & params)
    else if (curSchema == 0) { /* new store */
        curSchema = nixSchemaVersion;
        openDB(*state, true);
-        writeFile(schemaPath, fmt("%1%", nixSchemaVersion), 0666, true);
+        writeFileAndSync(schemaPath, fmt("%1%", nixSchemaVersion), 0666);
    }

    else if (curSchema < nixSchemaVersion) {
@@ -353,7 +353,7 @@ LocalStore::LocalStore(const Params & params)
            txn.commit();
        }

-        writeFile(schemaPath, fmt("%1%", nixSchemaVersion), 0666, true);
+        writeFileAndSync(schemaPath, fmt("%1%", nixSchemaVersion), 0666);

        lockFile(globalLock.get(), ltRead, true);
    }
@@ -197,6 +197,12 @@ if busybox.found()
  }
 endif

+if pasta.found()
+  cpp_str_defines += {
+    'PASTA_PATH': pasta.full_path(),
+  }
+endif
+
 cpp_args = []

 foreach name, value : cpp_str_defines
@@ -152,7 +152,6 @@ struct NarAccessor : public FSAccessor

    NarMember * find(const Path & path)
    {
-        Path canon = path == "" ? "" : canonPath(path);
        NarMember * current = &root;
        auto end = path.end();
        for (auto it = path.begin(); it != end; ) {
@@ -120,7 +120,7 @@ std::string NarInfo::to_string(const Store & store) const
        res += "Sig: " + sig + "\n";

    if (ca)
-        res += "CA: " + renderContentAddress(*ca) + "\n";
+        res += "CA: " + renderContentAddress(ca) + "\n";

    return res;
 }
@@ -1,15 +1,25 @@
 #include "build/worker.hh"
 #include "cgroup.hh"
+#include "file-descriptor.hh"
+#include "file-system.hh"
 #include "finally.hh"
 #include "gc-store.hh"
+#include "processes.hh"
 #include "signals.hh"
 #include "platform/linux.hh"
 #include "regex.hh"
+#include "strings.hh"

+#include <csignal>
+#include <cstdlib>
 #include <grp.h>
 #include <regex>
 #include <sys/prctl.h>

+#if __linux__
+#include <linux/capability.h>
+#endif
+
 #if HAVE_SECCOMP
 #include <linux/filter.h>
 #include <seccomp.h>
@@ -57,6 +67,14 @@ static void readFileRoots(const char * path, UncheckedRoots & roots)
    }
 }

+LinuxLocalDerivationGoal::~LinuxLocalDerivationGoal()
+{
+    // pasta being left around mostly happens when builds are aborted
+    if (pastaPid) {
+        pastaPid.kill();
+    }
+}
+
 void LinuxLocalStore::findPlatformRoots(UncheckedRoots & unchecked)
 {
    auto procDir = AutoCloseDir{opendir("/proc")};
@@ -832,6 +850,26 @@ void LinuxLocalDerivationGoal::prepareSandbox()
    }
 }

+std::string LinuxLocalDerivationGoal::rewriteResolvConf(std::string fromHost)
+{
+    if (!runPasta) {
+        return fromHost;
+    }
+
+    static constexpr auto flags = std::regex::ECMAScript | std::regex::multiline;
+    static std::regex lineRegex("^nameserver\\s.*$", flags);
+    static std::regex v4Regex("^nameserver\\s+\\d{1,3}\\.", flags);
+    static std::regex v6Regex("^nameserver.*:", flags);
+    std::string nsInSandbox = "\n";
+    if (std::regex_search(fromHost, v4Regex)) {
+        nsInSandbox += fmt("nameserver %s\n", PASTA_HOST_IPV4);
+    }
+    if (std::regex_search(fromHost, v6Regex)) {
+        nsInSandbox += fmt("nameserver %s\n", PASTA_HOST_IPV6);
+    }
+    return std::regex_replace(fromHost, lineRegex, "") + nsInSandbox;
+}
+
 Pid LinuxLocalDerivationGoal::startChild(std::function<void()> openSlave)
 {
 #if HAVE_SECCOMP
@@ -859,9 +897,11 @@ Pid LinuxLocalDerivationGoal::startChild(std::function<void()> openSlave)

       - The private network namespace ensures that the builder
         cannot talk to the outside world (or vice versa).  It
-         only has a private loopback interface. (Fixed-output
-         derivations are not run in a private network namespace
-         to allow functions like fetchurl to work.)
+         only has a private loopback interface. If a copy of
+         `pasta` is available, Fixed-output derivations are run
+         inside a private network namespace with internet
+         access, otherwise they are run in the host's network
+         namespace, to allow functions like fetchurl to work.

       - The IPC namespace prevents the builder from communicating
         with outside processes using SysV IPC mechanisms (shared
@@ -882,6 +922,10 @@ Pid LinuxLocalDerivationGoal::startChild(std::function<void()> openSlave)
    if (derivationType->isSandboxed())
        privateNetwork = true;

+    // don't launch pasta unless we have a tun device. in a build sandbox we
+    // commonly do not, and trying to run pasta anyway naturally won't work.
+    runPasta = !privateNetwork && settings.pastaPath != "" && pathExists("/dev/net/tun");
+
    userNamespaceSync.create();

    Pipe sendPid;
@@ -906,7 +950,9 @@ Pid LinuxLocalDerivationGoal::startChild(std::function<void()> openSlave)

        ProcessOptions options;
        options.cloneFlags = CLONE_NEWPID | CLONE_NEWNS | CLONE_NEWIPC | CLONE_NEWUTS | CLONE_PARENT | SIGCHLD;
-        if (privateNetwork)
+        // we always want to create a new network namespace for pasta, even when
+        // we can't actually run it. not doing so hides bugs and impairs purity.
+        if (settings.pastaPath != "" || privateNetwork)
            options.cloneFlags |= CLONE_NEWNET;
        if (usingUserNamespace)
            options.cloneFlags |= CLONE_NEWUSER;
@@ -988,6 +1034,67 @@ Pid LinuxLocalDerivationGoal::startChild(std::function<void()> openSlave)
    /* Signal the builder that we've updated its user namespace. */
    writeFull(userNamespaceSync.writeSide.get(), "1");

+    if (runPasta) {
+        // Bring up pasta, for handling FOD networking. We don't let it daemonize
+        // itself for process managements reasons and kill it manually when done.
+
+        // TODO add a new sandbox mode flag to disable all or parts of this?
+        Strings args = {
+            // clang-format off
+            "--quiet",
+            "--foreground",
+            "--config-net",
+            "--gateway", PASTA_HOST_IPV4,
+            "--address", PASTA_CHILD_IPV4, "--netmask", PASTA_IPV4_NETMASK,
+            "--dns-forward", PASTA_HOST_IPV4,
+            "--gateway", PASTA_HOST_IPV6,
+            "--address", PASTA_CHILD_IPV6,
+            "--dns-forward", PASTA_HOST_IPV6,
+            "--ns-ifname", PASTA_NS_IFNAME,
+            "--no-netns-quit",
+            "--netns", "/proc/self/fd/0",
+            // clang-format on
+        };
+
+        AutoCloseFD netns(open(fmt("/proc/%i/ns/net", pid.get()).c_str(), O_RDONLY | O_CLOEXEC));
+        if (!netns) {
+            throw SysError("failed to open netns");
+        }
+
+        AutoCloseFD userns;
+        if (usingUserNamespace) {
+            userns =
+                AutoCloseFD(open(fmt("/proc/%i/ns/user", pid.get()).c_str(), O_RDONLY | O_CLOEXEC));
+            if (!userns) {
+                throw SysError("failed to open userns");
+            }
+            args.push_back("--userns");
+            args.push_back("/proc/self/fd/1");
+        }
+
+        // FIXME ideally we want a notification when pasta exits, but we cannot do
+        // this at present. without such support we need to busy-wait for pasta to
+        // set up the namespace completely and time out after a while for the case
+        // of pasta launch failures. pasta logs go to syslog only for now as well.
+        pastaPid = runProgram2({
+            .program = settings.pastaPath,
+            .args = args,
+            .uid = useBuildUsers() ? std::optional(buildUser->getUID()) : std::nullopt,
+            .gid = useBuildUsers() ? std::optional(buildUser->getGID()) : std::nullopt,
+            // TODO these redirections are crimes. pasta closes all non-stdio file
+            // descriptors very early and lacks fd arguments for the namespaces we
+            // want it to join. we cannot have pasta join the namespaces via pids;
+            // doing so requires capabilities which pasta *also* drops very early.
+            .redirections = {
+                {.from = 0, .to = netns.get()},
+                {.from = 1, .to = userns ? userns.get() : 1},
+            },
+            .caps = getuid() == 0
+                ? std::set<long>{CAP_SYS_ADMIN, CAP_NET_BIND_SERVICE}
+                : std::set<long>{},
+        });
+    }
+
    return pid;
 }

@@ -1002,5 +1109,24 @@ void LinuxLocalDerivationGoal::killSandbox(bool getStats)
    } else {
        LocalDerivationGoal::killSandbox(getStats);
    }
+
+    if (pastaPid) {
+        // FIXME we really want to send SIGTERM instead and wait for pasta to exit,
+        // but we do not have the infra for that right now. we send SIGKILL instead
+        // and treat exiting with that as a successful exit code until such a time.
+        // this is not likely to cause problems since pasta runs as the build user,
+        // but not inside the build sandbox. if it's killed it's either due to some
+        // external influence (in which case the sandboxed child will probably fail
+        // due to network errors, if it used the network at all) or some bug in lix
+        if (auto status = pastaPid.kill(); !WIFSIGNALED(status) || WTERMSIG(status) != SIGKILL) {
+            if (WIFSIGNALED(status)) {
+                throw Error("pasta killed by signal %i", WTERMSIG(status));
+            } else if (WIFEXITED(status)) {
+                throw Error("pasta exited with code %i", WEXITSTATUS(status));
+            } else {
+                throw Error("pasta exited with status %i", status);
+            }
+        }
+    }
 }
 }
@@ -4,6 +4,7 @@
 #include "build/local-derivation-goal.hh"
 #include "gc-store.hh"
 #include "local-store.hh"
+#include "processes.hh"

 namespace nix {

@@ -41,7 +42,23 @@ class LinuxLocalDerivationGoal : public LocalDerivationGoal
 public:
    using LocalDerivationGoal::LocalDerivationGoal;

+    ~LinuxLocalDerivationGoal();
+
+    // NOTE these are all C strings because macos doesn't have constexpr std::string
+    // constructors, and std::string_view is a pain to turn into std::strings again.
+    static constexpr const char * PASTA_NS_IFNAME = "eth0";
+    static constexpr const char * PASTA_HOST_IPV4 = "169.254.1.1";
+    static constexpr const char * PASTA_CHILD_IPV4 = "169.254.1.2";
+    static constexpr const char * PASTA_IPV4_NETMASK = "16";
+    // randomly chosen 6to4 prefix, mapping the same ipv4ll as above.
+    // even if this id is used on the daemon host there should not be
+    // any collisions since ipv4ll should never be addressed by ipv6.
+    static constexpr const char * PASTA_HOST_IPV6 = "64:ff9b:1:4b8e:472e:a5c8:a9fe:0101";
+    static constexpr const char * PASTA_CHILD_IPV6 = "64:ff9b:1:4b8e:472e:a5c8:a9fe:0102";
+
 private:
+    RunningProgram pastaPid;
+
    /**
     * Create and populate chroot
     */
@@ -70,6 +87,7 @@ private:
        return true;
    }

+    std::string rewriteResolvConf(std::string fromHost) override;
 };

 }
@@ -145,7 +145,7 @@ SQLiteStmt::Use::~Use()
 SQLiteStmt::Use & SQLiteStmt::Use::operator () (std::string_view value, bool notNull)
 {
    if (notNull) {
-        if (sqlite3_bind_text(stmt, curArg++, value.data(), -1, SQLITE_TRANSIENT) != SQLITE_OK)
+        if (sqlite3_bind_text(stmt, curArg++, value.data(), value.length(), SQLITE_TRANSIENT) != SQLITE_OK)
            SQLiteError::throw_(stmt.db, "binding argument");
    } else
        bind();
@@ -4,6 +4,7 @@
 #include "finally.hh"
 #include "logging.hh"
 #include "strings.hh"
+#include <unistd.h>

 namespace nix {

@@ -49,7 +50,7 @@ bool SSHMaster::isMasterRunning() {
    Strings args = {"-O", "check", host};
    addCommonSSHOpts(args);

-    auto res = runProgram(RunOptions {.program = "ssh", .args = args, .mergeStderrToStdout = true});
+    auto res = runProgram(RunOptions {.program = "ssh", .args = args, .redirections = {{.from = STDERR_FILENO, .to = STDOUT_FILENO}}});
    return res.first == 0;
 }

@@ -1322,8 +1322,7 @@ std::optional<StorePath> Store::getBuildDerivationPath(const StorePath & path)
    if (!path.isDerivation()) {
        try {
            auto info = queryPathInfo(path);
-            if (!info->deriver) return std::nullopt;
-            return *info->deriver;
+            return info->deriver;
        } catch (InvalidPath &) {
            return std::nullopt;
        }
@@ -239,6 +239,7 @@ struct BrotliCompressionSink : ChunkedCompressionSink

    void writeInternal(std::string_view data) override
    {
+        // NOLINTNEXTLINE(bugprone-suspicious-stringview-data-usage)
        auto next_in = charptr_cast<const uint8_t *>(data.data());
        size_t avail_in = data.size();
        uint8_t * next_out = outbuf;
@@ -156,6 +156,29 @@ int AutoCloseFD::get() const
    return fd;
 }

+std::string guessOrInventPathFromFD(int fd)
+{
+    assert(fd >= 0);
+    /* On Linux, there's no F_GETPATH available.
+     * But we can read /proc/ */
+#if __linux__
+    try {
+        return readLink(fmt("/proc/self/fd/%1%", fd).c_str());
+    } catch (...) {
+    }
+#elif defined (HAVE_F_GETPATH) && HAVE_F_GETPATH
+    std::string fdName(PATH_MAX, '\0');
+    if (fcntl(fd, F_GETPATH, fdName.data()) != -1) {
+        fdName.resize(strlen(fdName.c_str()));
+        return fdName;
+    }
+#else
+#error "No implementation for retrieving file descriptors path."
+#endif
+
+    return fmt("<fd %i>", fd);
+}
+

 void AutoCloseFD::close()
 {
@@ -228,7 +251,7 @@ void closeExtraFDs()
    auto closeRange = [](unsigned int first, unsigned int last, int flags) -> int {
      // musl does not have close_range as of 2024-08-10
      // patch: https://www.openwall.com/lists/musl/2024/08/01/9
-#ifdef HAVE_CLOSE_RANGE
+#if HAVE_CLOSE_RANGE
        return close_range(first, last, flags);
 #else
        return syscall(SYS_close_range, first, last, flags);
@@ -36,6 +36,15 @@ void writeFull(int fd, std::string_view s, bool allowInterrupts = true);
 */
 std::string drainFD(int fd, bool block = true, const size_t reserveSize=0);

+
+/*
+ * Will attempt to guess *A* path associated that might lead to the same file as used by this
+ * file descriptor.
+ *
+ * The returned string should NEVER be used as a valid path.
+ */
+std::string guessOrInventPathFromFD(int fd);
+
 Generator<Bytes> drainFDSource(int fd, bool block = true);

 class AutoCloseFD
@@ -50,6 +59,15 @@ public:
    AutoCloseFD& operator =(const AutoCloseFD & fd) = delete;
    AutoCloseFD& operator =(AutoCloseFD&& fd) noexcept(false);
    int get() const;
+
+    /*
+     * Will attempt to guess *A* path associated that might lead to the same file as used by this
+     * file descriptor.
+     *
+     * The returned string should NEVER be used as a valid path.
+     */
+    std::string guessOrInventPath() const { return guessOrInventPathFromFD(fd); }
+
    explicit operator bool() const;
    int release();
    void close();
@@ -311,17 +311,47 @@ Generator<Bytes> readFileSource(const Path & path)
 }


-void writeFile(const Path & path, std::string_view s, mode_t mode, bool sync)
+void writeFile(const Path & path, std::string_view s, mode_t mode)
 {
    AutoCloseFD fd{open(path.c_str(), O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC, mode)};
    if (!fd)
        throw SysError("opening file '%1%'", path);
+
+    writeFile(fd, s, mode);
+
+    /* Close explicitly to propagate the exceptions. */
+    fd.close();
+}
+
+void writeFile(AutoCloseFD & fd, std::string_view s, mode_t mode)
+{
+    assert(fd);
    try {
        writeFull(fd.get(), s);
    } catch (Error & e) {
-        e.addTrace({}, "writing file '%1%'", path);
+        e.addTrace({}, "writing file '%1%'", fd.guessOrInventPath());
        throw;
    }
+}
+
+void writeFileAndSync(const Path & path, std::string_view s, mode_t mode)
+{
+    {
+        AutoCloseFD fd{open(path.c_str(), O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC, mode)};
+        if (!fd)
+            throw SysError("opening file '%1%'", path);
+
+        writeFile(fd, s, mode);
+        fd.fsync();
+        /* Close explicitly to ensure that exceptions are propagated. */
+        fd.close();
+    }
+
+    syncParent(path);
+}
+
+static void closeForWrite(const Path & path, AutoCloseFD & fd, bool sync)
+{
    if (sync)
        fd.fsync();
    // Explicitly close to make sure exceptions are propagated.
@@ -330,8 +360,7 @@ void writeFile(const Path & path, std::string_view s, mode_t mode, bool sync)
        syncParent(path);
 }

-
-void writeFile(const Path & path, Source & source, mode_t mode, bool sync)
+void writeFile(const Path & path, Source & source, mode_t mode)
 {
    AutoCloseFD fd{open(path.c_str(), O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC, mode)};
    if (!fd)
@@ -350,12 +379,8 @@ void writeFile(const Path & path, Source & source, mode_t mode, bool sync)
        e.addTrace({}, "writing file '%1%'", path);
        throw;
    }
-    if (sync)
-        fd.fsync();
-    // Explicitly close to make sure exceptions are propagated.
-    fd.close();
-    if (sync)
-        syncParent(path);
+
+    closeForWrite(path, fd, false);
 }

 void syncParent(const Path & path)
@@ -366,16 +391,26 @@ void syncParent(const Path & path)
    fd.fsync();
 }

-static void _deletePath(int parentfd, const Path & path, uint64_t & bytesFreed)
+/* TODO(horrors): a better structure that links all parent fds for the traversal root
+ * should be considered for this code
+ */
+static void _deletePath(int parentfd, const Path & name, uint64_t & bytesFreed)
 {
+    /* This ensures that `name` is an immediate child of `parentfd`. */
+    assert(!name.empty() && name.find('/') == std::string::npos && "`name` is an immediate child to `parentfd`");
    checkInterrupt();

-    std::string name(baseNameOf(path));
+    /* FIXME(horrors): there's a minor TOCTOU here.
+     * we fstatat the inode nofollow, check if this is a directory
+     * and then open it.
+     * a better alternative is open it as O_PATH as a namefd.
+     * if it's a directory, it can be openat with the namefd.
+     */

    struct stat st;
    if (fstatat(parentfd, name.c_str(), &st, AT_SYMLINK_NOFOLLOW) == -1) {
        if (errno == ENOENT) return;
-        throw SysError("getting status of '%1%'", path);
+        throw SysError("getting status of '%1%' in directory '%2%'", name, guessOrInventPathFromFD(parentfd));
    }

    if (!S_ISDIR(st.st_mode)) {
@@ -406,24 +441,25 @@ static void _deletePath(int parentfd, const Path & path, uint64_t & bytesFreed)
        /* Make the directory accessible. */
        const auto PERM_MASK = S_IRUSR | S_IWUSR | S_IXUSR;
        if ((st.st_mode & PERM_MASK) != PERM_MASK) {
-            if (fchmodat(parentfd, name.c_str(), st.st_mode | PERM_MASK, 0) == -1)
-                throw SysError("chmod '%1%'", path);
+            if (fchmodat(parentfd, name.c_str(), st.st_mode | PERM_MASK, 0) == -1) {
+                throw SysError("chmod '%1%' in directory '%2%'", name, guessOrInventPathFromFD(parentfd));
+            }
        }

-        int fd = openat(parentfd, path.c_str(), O_RDONLY);
+        int fd = openat(parentfd, name.c_str(), O_RDONLY | O_DIRECTORY | O_NOFOLLOW);
        if (fd == -1)
-            throw SysError("opening directory '%1%'", path);
+            throw SysError("opening directory '%1%' in directory '%2%'", name, guessOrInventPathFromFD(parentfd));
        AutoCloseDir dir(fdopendir(fd));
        if (!dir)
-            throw SysError("opening directory '%1%'", path);
-        for (auto & i : readDirectory(dir.get(), path))
-            _deletePath(dirfd(dir.get()), path + "/" + i.name, bytesFreed);
+            throw SysError("opening directory '%1%' in directory '%2%'", name, guessOrInventPathFromFD(parentfd));
+        for (auto & i : readDirectory(dir.get(), name))
+            _deletePath(dirfd(dir.get()), i.name, bytesFreed);
    }

    int flags = S_ISDIR(st.st_mode) ? AT_REMOVEDIR : 0;
    if (unlinkat(parentfd, name.c_str(), flags) == -1) {
        if (errno == ENOENT) return;
-        throw SysError("cannot unlink '%1%'", path);
+        throw SysError("cannot unlink '%1%' in directory '%2%'", name, guessOrInventPathFromFD(parentfd));
    }
 }

@@ -439,7 +475,7 @@ static void _deletePath(const Path & path, uint64_t & bytesFreed)
        throw SysError("opening directory '%1%'", path);
    }

-    _deletePath(dirfd.get(), path, bytesFreed);
+    _deletePath(dirfd.get(), std::string(baseNameOf(path)), bytesFreed);
 }


@@ -164,9 +164,16 @@ Generator<Bytes> readFileSource(const Path & path);
 /**
 * Write a string to a file.
 */
-void writeFile(const Path & path, std::string_view s, mode_t mode = 0666, bool sync = false);
+void writeFile(const Path & path, std::string_view s, mode_t mode = 0666);

-void writeFile(const Path & path, Source & source, mode_t mode = 0666, bool sync = false);
+void writeFile(const Path & path, Source & source, mode_t mode = 0666);
+
+void writeFile(AutoCloseFD & fd, std::string_view s, mode_t mode = 0666);
+
+/**
+ * Write a string to a file and flush the file and its parents direcotry to disk.
+ */
+void writeFileAndSync(const Path & path, std::string_view s, mode_t mode = 0666);

 /**
 * Flush a file's parent directory to disk
@@ -25,7 +25,7 @@ std::string hiliteMatches(
        out.append(s.substr(last_end, m.position() - last_end));
        // Merge continous matches
        ssize_t end = start + m.length();
-        while (++it != matches.end() && (*it).position() <= end) {
+        for (++it; it != matches.end() && (*it).position() <= end; ++it) {
            auto n = *it;
            ssize_t nend = start + (n.position() - start + n.length());
            if (nend > end)
@@ -38,6 +38,6 @@ const nlohmann::json & ensureType(
            nlohmann::json(expectedType).type_name(),
            value.type_name());

-    return value;
+    return value; // NOLINT(bugprone-return-const-ref-from-parameter)
 }
 }
@@ -258,12 +258,12 @@ extern Verbosity verbosity;
 * level. Note that this has to be implemented as a macro to ensure that the
 * arguments are evaluated lazily.
 */
-#define printMsgUsing(loggerParam, level, args...) \
-    do { \
-        auto __lvl = level; \
-        if (__lvl <= nix::verbosity) { \
-            loggerParam->log(__lvl, fmt(args)); \
-        } \
+#define printMsgUsing(loggerParam, level, args...)              \
+    do {                                                        \
+        auto _lix_logger_print_lvl = level;                     \
+        if (_lix_logger_print_lvl <= nix::verbosity) {          \
+            loggerParam->log(_lix_logger_print_lvl, fmt(args)); \
+        }                                                       \
    } while (0)
 #define printMsg(level, args...) printMsgUsing(logger, level, args)

@@ -21,6 +21,7 @@
 #endif

 #ifdef __linux__
+# include <linux/capability.h>
 # include <sys/prctl.h>
 # include <sys/mman.h>
 #endif
@@ -248,7 +249,7 @@ std::pair<int, std::string> runProgram(RunOptions && options)

    try {
        auto proc = runProgram2(options);
-        Finally const _wait([&] { proc.wait(); });
+        Finally const _wait([&] { proc.waitAndCheck(); });
        stdout = proc.getStdout()->drain();
    } catch (ExecError & e) {
        status = e.status;
@@ -276,7 +277,22 @@ RunningProgram::~RunningProgram()
    }
 }

-void RunningProgram::wait()
+std::tuple<pid_t, std::unique_ptr<Source>, int> RunningProgram::release()
+{
+    return {pid.release(), std::move(stdoutSource), stdout_.release()};
+}
+
+int RunningProgram::kill()
+{
+    return pid.kill();
+}
+
+int RunningProgram::wait()
+{
+    return pid.wait();
+}
+
+void RunningProgram::waitAndCheck()
 {
    if (std::uncaught_exceptions() == 0) {
        int status = pid.wait();
@@ -314,12 +330,21 @@ RunningProgram runProgram2(const RunOptions & options)
            replaceEnv(*options.environment);
        if (options.captureStdout && dup2(out.writeSide.get(), STDOUT_FILENO) == -1)
            throw SysError("dupping stdout");
-        if (options.mergeStderrToStdout)
-            if (dup2(STDOUT_FILENO, STDERR_FILENO) == -1)
-                throw SysError("cannot dup stdout into stderr");
+        for (auto redirection : options.redirections) {
+            if (dup2(redirection.to, redirection.from) == -1) {
+                throw SysError("dupping fd %i to %i", redirection.from, redirection.to);
+            }
+        }

        if (options.chdir && chdir((*options.chdir).c_str()) == -1)
            throw SysError("chdir failed");
+
+#if __linux__
+        if (!options.caps.empty() && prctl(PR_SET_KEEPCAPS, 1) < 0) {
+            throw SysError("setting keep-caps failed");
+        }
+#endif
+
        if (options.gid && setgid(*options.gid) == -1)
            throw SysError("setgid failed");
        /* Drop all other groups if we're setgid. */
@@ -328,6 +353,45 @@ RunningProgram runProgram2(const RunOptions & options)
        if (options.uid && setuid(*options.uid) == -1)
            throw SysError("setuid failed");

+#if __linux__
+        if (!options.caps.empty()) {
+            if (prctl(PR_SET_KEEPCAPS, 0)) {
+                throw SysError("clearing keep-caps failed");
+            }
+
+            // we do the capability dance like this to avoid a dependency
+            // on libcap, which has a rather large build closure and many
+            // more features that we need for now. maybe some other time.
+            static constexpr uint32_t LINUX_CAPABILITY_VERSION_3 = 0x20080522;
+            static constexpr uint32_t LINUX_CAPABILITY_U32S_3 = 2;
+            struct user_cap_header_struct
+            {
+                uint32_t version;
+                int pid;
+            } hdr = {LINUX_CAPABILITY_VERSION_3, 0};
+            struct user_cap_data_struct
+            {
+                uint32_t effective;
+                uint32_t permitted;
+                uint32_t inheritable;
+            } data[LINUX_CAPABILITY_U32S_3] = {};
+            for (auto cap : options.caps) {
+                assert(cap / 32 < LINUX_CAPABILITY_U32S_3);
+                data[cap / 32].permitted |= 1 << (cap % 32);
+                data[cap / 32].inheritable |= 1 << (cap % 32);
+            }
+            if (syscall(SYS_capset, &hdr, data)) {
+                throw SysError("couldn't set capabilities");
+            }
+
+            for (auto cap : options.caps) {
+                if (prctl(PR_CAP_AMBIENT, PR_CAP_AMBIENT_RAISE, cap, 0, 0) < 0) {
+                    throw SysError("couldn't set ambient caps");
+                }
+            }
+        }
+#endif
+
        Strings args_(options.args);
        args_.push_front(options.program);

@@ -76,6 +76,11 @@ std::string runProgram(Path program, bool searchPath = false,

 struct RunOptions
 {
+    struct Redirection
+    {
+        int from, to;
+    };
+
    Path program;
    bool searchPath = true;
    Strings args;
@@ -84,8 +89,11 @@ struct RunOptions
    std::optional<Path> chdir;
    std::optional<std::map<std::string, std::string>> environment;
    bool captureStdout = false;
-    bool mergeStderrToStdout = false;
    bool isInteractive = false;
+    std::vector<Redirection> redirections;
+#if __linux__
+    std::set<long> caps;
+#endif
 };

 struct [[nodiscard("you must call RunningProgram::wait()")]] RunningProgram
@@ -102,9 +110,23 @@ private:

 public:
    RunningProgram() = default;
+    RunningProgram(RunningProgram &&) = default;
+    RunningProgram & operator=(RunningProgram &&) = default;
    ~RunningProgram();

-    void wait();
+    explicit operator bool() const { return bool(pid); }
+
+    std::tuple<pid_t, std::unique_ptr<Source>, int> release();
+
+    int kill();
+    [[nodiscard]]
+    int wait();
+    void waitAndCheck();
+
+    std::optional<int> getStdoutFD() const
+    {
+        return stdout_ ? std::optional(stdout_.get()) : std::nullopt;
+    }

    Source * getStdout() const { return stdoutSource.get(); };
 };
@@ -53,7 +53,7 @@ void RefScanSink::operator () (std::string_view data)
       previous fragment and the start of the current fragment. */
    auto s = tail;
    auto tailLen = std::min(data.size(), refLength);
-    s.append(data.data(), tailLen);
+    s.append(data.data(), tailLen); // NOLINT(bugprone-suspicious-stringview-data-usage)
    search(s, hashes, seen);

    search(data, hashes, seen);
@@ -75,7 +75,6 @@ void Source::operator () (char * data, size_t len)

 void Source::drainInto(Sink & sink)
 {
-    std::string s;
    std::array<char, 8192> buf;
    while (true) {
        size_t n;
@@ -251,7 +250,7 @@ Error readError(Source & source)
    auto type = readString(source);
    assert(type == "Error");
    auto level = (Verbosity) readInt(source);
-    auto name = readString(source); // removed
+    readString(source); // removed (name)
    auto msg = readString(source);
    ErrorInfo info {
        .level = level,
@@ -34,7 +34,7 @@ bool shouldANSI(StandardOutputStream fileno)
 // FIXME(jade): replace with TerminalCodeEater. wowie this is evil code.
 std::string filterANSIEscapes(std::string_view s, bool filterAll, unsigned int width, bool eatTabs)
 {
-    std::string t, e;
+    std::string t;
    size_t w = 0;
    auto i = s.begin();

@@ -80,12 +80,18 @@ typename T::mapped_type * get(T & map, const typename T::key_type & key)
 * Get a value for the specified key from an associate container, or a default value if the key isn't present.
 */
 template <class T>
-const typename T::mapped_type & getOr(T & map,
+const typename T::mapped_type & getOr(
+    T & map [[clang::lifetimebound]],
    const typename T::key_type & key,
-    const typename T::mapped_type & defaultValue)
+    const typename T::mapped_type & defaultValue [[clang::lifetimebound]]
+)
 {
    auto i = map.find(key);
-    if (i == map.end()) return defaultValue;
+    if (i == map.end()) {
+        /* FIXME(Raito): `[[clang::lifetimebound]]` has no effect on `defaultValue` warning. */
+        // NOLINTNEXTLINE(bugprone-return-const-ref-from-parameter)
+        return defaultValue;
+    }
    return i->second;
 }

@@ -1313,7 +1313,7 @@ static void opSwitchGeneration(Globals & globals, Strings opFlags, Strings opArg
        throw UsageError("exactly one argument expected");

    if (auto dstGen = string2Int<GenerationNumber>(opArgs.front()))
-        switchGeneration(globals.profile, *dstGen, globals.dryRun);
+        switchGeneration(globals.profile, dstGen, globals.dryRun);
    else
        throw UsageError("expected a generation number");
 }
@@ -116,8 +116,6 @@ struct CmdBundle : InstallableCommand
            },
        });

-        auto outPathS = store->printStorePath(outPath);
-
        if (!outLink) {
            auto * attr = vRes->attrs->get(evalState->sName);
            if (!attr)
@@ -114,7 +114,7 @@ struct CmdPathInfo : StorePathsCommand, MixJSON
                    std::cout << '\t';
                    Strings ss;
                    if (info->ultimate) ss.push_back("ultimate");
-                    if (info->ca) ss.push_back("ca:" + renderContentAddress(*info->ca));
+                    if (info->ca) ss.push_back("ca:" + renderContentAddress(info->ca));
                    for (auto & sig : info->sigs) ss.push_back(sig);
                    std::cout << concatStringsSep(" ", ss);
                }
@@ -160,7 +160,6 @@ struct CmdSearch : InstallableCommand, MixJSON
                                {"description", description},
                            };
                        } else {
-                            auto name2 = hiliteMatches(name.name, nameMatches, ANSI_GREEN, "\e[0;2m");
                            if (results > 1) logger->cout("");
                            logger->cout(
                                "* %s%s",
@@ -40,13 +40,9 @@ struct CmdCopySigs : StorePathsCommand

        ThreadPool pool;

-        std::string doneLabel = "done";
        std::atomic<size_t> added{0};

-        //logger->setExpected(doneLabel, storePaths.size());
-
        auto doPath = [&](const Path & storePathS) {
-            //Activity act(*logger, lvlInfo, "getting signatures for '%s'", storePath);

            checkInterrupt();

@@ -78,8 +74,6 @@ struct CmdCopySigs : StorePathsCommand
                store->addSignatures(storePath, newSigs);
                added += newSigs.size();
            }
-
-            //logger->incProgress(doneLabel);
        };

        for (auto & storePath : storePaths)
@@ -34,6 +34,7 @@ private:
                                  OptionalFileEntryRef File,
                                  StringRef SearchPath, StringRef RelativePath,
                                  const Module *Imported,
+                                  bool ModuleImported,
                                  SrcMgr::CharacteristicKind FileType) override;
 };

@@ -46,7 +47,7 @@ void FixIncludesCallbacks::LexedFileChanged(FileID, LexedFileChangeReason,
 void FixIncludesCallbacks::InclusionDirective(
    SourceLocation, const Token &, StringRef FileName, bool IsAngled,
    CharSourceRange FilenameRange, OptionalFileEntryRef File, StringRef,
-    StringRef, const Module *, SrcMgr::CharacteristicKind) {
+    StringRef, const Module *, bool, SrcMgr::CharacteristicKind) {
  if (Ignore)
    return;

@@ -8,7 +8,6 @@ requireSandboxSupport
 [[ $busybox =~ busybox ]] || skipTest "no busybox"

 unset NIX_STORE_DIR
-unset NIX_STATE_DIR

 # We first build a dependency of the derivation we eventually want to
 # build.
@@ -2,7 +2,6 @@ requireSandboxSupport
 [[ $busybox =~ busybox ]] || skipTest "no busybox"

 unset NIX_STORE_DIR
-unset NIX_STATE_DIR

 remoteDir=$TEST_ROOT/remote

@@ -3,7 +3,6 @@ requireSandboxSupport

 # Avoid store dir being inside sandbox build-dir
 unset NIX_STORE_DIR
-unset NIX_STATE_DIR

 function join_by { local d=$1; shift; echo -n "$1"; shift; printf "%s" "${@/#/$d}"; }

@@ -81,4 +80,4 @@ out="$(nix-build 2>&1 failing.nix \
 [[ "$out" =~ .*"note: keeping build directory".* ]]

 build_dir="$(grep "note: keeping build" <<< "$out" | sed -E "s/^(.*)note: keeping build directory '(.*)'(.*)$/\2/")"
-[[ "foo" = $(<"$build_dir"/bar) ]]
+[[ "foo" = $(<"$build_dir"/b/bar) ]]
@@ -170,3 +170,28 @@ test "$(<<<"$out" grep -E '^error:' | wc -l)" = 3
 <<<"$out" grepQuiet -E "error: 2 dependencies of derivation '.*-x4\\.drv' failed to build"
 <<<"$out" grepQuiet -vE "hash mismatch in fixed-output derivation '.*-x3\\.drv'"
 <<<"$out" grepQuiet -vE "hash mismatch in fixed-output derivation '.*-x2\\.drv'"
+
+# Ensure when if the system build dir is inaccessible, we can still build things
+BUILD_DIR=$(mktemp -d)
+chmod 0000 "$BUILD_DIR"
+nix --build-dir "$BUILD_DIR" build -E 'with import ./config.nix; mkDerivation { name = "test"; buildCommand = "echo rawr > $out"; }' --impure --no-link
+
+# ensure that the build directory parent is not world-accessible
+chmod 0755 "$BUILD_DIR"
+FIFO="$BUILD_DIR/fifo"
+mkfifo "$FIFO"
+(
+  echo > "$FIFO"
+  trap 'echo > "$FIFO"' EXIT
+  mode=$(stat -c %a $BUILD_DIR/b/*)
+  [ "$mode" = "700" -o "$mode" = "710" ]
+) &
+nix build --build-dir "$BUILD_DIR/b" -E '
+  with import ./config.nix; mkDerivation {
+    name = "test";
+    buildCommand = "cat '"$FIFO"'; cat '"$FIFO"' > $out";
+  }' \
+  --extra-sandbox-paths "$FIFO" \
+  --impure \
+  --no-link
+wait
@@ -45,7 +45,7 @@ test_custom_build_dir() {
  [ "$status" = "100" ]
  [[ 1 == "$(count "$customBuildDir/nix-build-"*)" ]]
  local buildDir="$customBuildDir/nix-build-"*
-  grep $checkBuildId $buildDir/checkBuildId
+  grep $checkBuildId $buildDir/b/checkBuildId
 }
 test_custom_build_dir

@@ -1,12 +1,11 @@
  nix-repl> 1 + 1
  2

-  nix-repl> :doc builtins.head
-  Synopsis: builtins.head list
+  nix-repl> :doc builtins.add
+  Synopsis: builtins.add e1 e2
+
+      Return the sum of the numbers e1 and e2.

-      Return the first element of a list; abort evaluation if
-      the argument isn’t a list or is an empty list. You can
-      test whether a list is empty by comparing it with [].

  nix-repl> f = a: "" + a

@@ -10,7 +10,6 @@ unshare --mount --map-root-user bash <<EOF

  # Avoid store dir being inside sandbox build-dir
  unset NIX_STORE_DIR
-  unset NIX_STATE_DIR

  setLocalStore () {
    export NIX_REMOTE=\$TEST_ROOT/\$1
@@ -1,90 +0,0 @@
-# Nix is a sandboxed build system. But Not everything can be handled inside its
-# sandbox: Network access is normally blocked off, but to download sources, a
-# trapdoor has to exist. Nix handles this by having "Fixed-output derivations".
-# The detail here is not important, but in our case it means that the hash of
-# the output has to be known beforehand. And if you know that, you get a few
-# rights: you no longer run inside a special network namespace!
-#
-# Now, Linux has a special feature, that not many other unices do: Abstract
-# unix domain sockets! Not only that, but those are namespaced using the
-# network namespace! That means that we have a way to create sockets that are
-# available in every single fixed-output derivation, and also all processes
-# running on the host machine! Now, this wouldn't be that much of an issue, as,
-# well, the whole idea is that the output is pure, and all processes in the
-# sandbox are killed before finalizing the output. What if we didn't need those
-# processes at all? Unix domain sockets have a semi-known trick: you can pass
-# file descriptors around!
-# This makes it possible to exfiltrate a file-descriptor with write access to
-# $out outside of the sandbox. And that file-descriptor can be used to modify
-# the contents of the store path after it has been registered.
-
-{ config, ... }:
-
-let
-  pkgs = config.nodes.machine.nixpkgs.pkgs;
-
-  # Simple C program that sends a a file descriptor to `$out` to a Unix
-  # domain socket.
-  # Compiled statically so that we can easily send it to the VM and use it
-  # inside the build sandbox.
-  sender = pkgs.runCommandWith {
-    name = "sender";
-    stdenv = pkgs.pkgsStatic.stdenv;
-  } ''
-    $CC -static -o $out ${./sender.c}
-  '';
-
-  # Okay, so we have a file descriptor shipped out of the FOD now. But the
-  # Nix store is read-only, right? .. Well, yeah. But this file descriptor
-  # lives in a mount namespace where it is not! So even when this file exists
-  # in the actual Nix store, we're capable of just modifying its contents...
-  smuggler = pkgs.writeCBin "smuggler" (builtins.readFile ./smuggler.c);
-
-  # The abstract socket path used to exfiltrate the file descriptor
-  socketName = "FODSandboxExfiltrationSocket";
-in
-{
-  name = "ca-fd-leak";
-
-  nodes.machine =
-    { config, lib, pkgs, ... }:
-    { virtualisation.writableStore = true;
-      nix.settings.substituters = lib.mkForce [ ];
-      virtualisation.additionalPaths = [ pkgs.busybox-sandbox-shell sender smuggler pkgs.socat ];
-    };
-
-  testScript = { nodes }: ''
-    start_all()
-
-    machine.succeed("echo hello")
-    # Start the smuggler server
-    machine.succeed("${smuggler}/bin/smuggler ${socketName} >&2 &")
-
-    # Build the smuggled derivation.
-    # This will connect to the smuggler server and send it the file descriptor
-    machine.succeed(r"""
-      nix-build -E '
-        builtins.derivation {
-          name = "smuggled";
-          system = builtins.currentSystem;
-          # look ma, no tricks!
-          outputHashMode = "flat";
-          outputHashAlgo = "sha256";
-          outputHash = builtins.hashString "sha256" "hello, world\n";
-          builder = "${pkgs.busybox-sandbox-shell}/bin/sh";
-          args = [ "-c" "echo \"hello, world\" > $out; ''${${sender}} ${socketName}" ];
-      }'
-    """.strip())
-
-
-    # Tell the smuggler server that we're done
-    machine.execute("echo done | ${pkgs.socat}/bin/socat - ABSTRACT-CONNECT:${socketName}")
-
-    # Check that the file was not modified
-    machine.succeed(r"""
-      cat ./result
-      test "$(cat ./result)" = "hello, world"
-    """.strip())
-  '';
-
-}
@@ -1,65 +0,0 @@
-#include <sys/socket.h>
-#include <sys/un.h>
-#include <stdlib.h>
-#include <stddef.h>
-#include <stdio.h>
-#include <unistd.h>
-#include <fcntl.h>
-#include <errno.h>
-#include <string.h>
-#include <assert.h>
-
-int main(int argc, char **argv) {
-
-    assert(argc == 2);
-
-    int sock = socket(AF_UNIX, SOCK_STREAM, 0);
-
-    // Set up a abstract domain socket path to connect to.
-    struct sockaddr_un data;
-    data.sun_family = AF_UNIX;
-    data.sun_path[0] = 0;
-    strcpy(data.sun_path + 1, argv[1]);
-
-    // Now try to connect, To ensure we work no matter what order we are
-    // executed in, just busyloop here.
-    int res = -1;
-    while (res < 0) {
-        res = connect(sock, (const struct sockaddr *)&data,
-            offsetof(struct sockaddr_un, sun_path)
-              + strlen(argv[1])
-              + 1);
-        if (res < 0 && errno != ECONNREFUSED) perror("connect");
-        if (errno != ECONNREFUSED) break;
-    }
-
-    // Write our message header.
-    struct msghdr msg = {0};
-    msg.msg_control = malloc(128);
-    msg.msg_controllen = 128;
-
-    // Write an SCM_RIGHTS message containing the output path.
-    struct cmsghdr *hdr = CMSG_FIRSTHDR(&msg);
-    hdr->cmsg_len = CMSG_LEN(sizeof(int));
-    hdr->cmsg_level = SOL_SOCKET;
-    hdr->cmsg_type = SCM_RIGHTS;
-    int fd = open(getenv("out"), O_RDWR | O_CREAT, 0640);
-    memcpy(CMSG_DATA(hdr), (void *)&fd, sizeof(int));
-
-    msg.msg_controllen = CMSG_SPACE(sizeof(int));
-
-    // Write a single null byte too.
-    msg.msg_iov = malloc(sizeof(struct iovec));
-    msg.msg_iov[0].iov_base = "";
-    msg.msg_iov[0].iov_len = 1;
-    msg.msg_iovlen = 1;
-
-    // Send it to the othher side of this connection.
-    res = sendmsg(sock, &msg, 0);
-    if (res < 0) perror("sendmsg");
-    int buf;
-
-    // Wait for the server to close the socket, implying that it has
-    // received the commmand.
-    recv(sock, (void *)&buf, sizeof(int), 0);
-}
@@ -1,66 +0,0 @@
-#include <sys/socket.h>
-#include <sys/un.h>
-#include <stdlib.h>
-#include <stddef.h>
-#include <stdio.h>
-#include <unistd.h>
-#include <assert.h>
-
-int main(int argc, char **argv) {
-
-    assert(argc == 2);
-
-    int sock = socket(AF_UNIX, SOCK_STREAM, 0);
-
-    // Bind to the socket.
-    struct sockaddr_un data;
-    data.sun_family = AF_UNIX;
-    data.sun_path[0] = 0;
-    strcpy(data.sun_path + 1, argv[1]);
-    int res = bind(sock, (const struct sockaddr *)&data,
-        offsetof(struct sockaddr_un, sun_path)
-        + strlen(argv[1])
-        + 1);
-    if (res < 0) perror("bind");
-
-    res = listen(sock, 1);
-    if (res < 0) perror("listen");
-
-    int smuggling_fd = -1;
-
-    // Accept the connection a first time to receive the file descriptor.
-    fprintf(stderr, "%s\n", "Waiting for the first connection");
-    int a = accept(sock, 0, 0);
-    if (a < 0) perror("accept");
-
-    struct msghdr msg = {0};
-    msg.msg_control = malloc(128);
-    msg.msg_controllen = 128;
-
-    // Receive the file descriptor as sent by the smuggler.
-    recvmsg(a, &msg, 0);
-
-    struct cmsghdr *hdr = CMSG_FIRSTHDR(&msg);
-    while (hdr) {
-        if (hdr->cmsg_level == SOL_SOCKET
-          && hdr->cmsg_type == SCM_RIGHTS) {
-
-            // Grab the copy of the file descriptor.
-            memcpy((void *)&smuggling_fd, CMSG_DATA(hdr), sizeof(int));
-        }
-
-        hdr = CMSG_NXTHDR(&msg, hdr);
-    }
-    fprintf(stderr, "%s\n", "Got the file descriptor. Now waiting for the second connection");
-    close(a);
-
-    // Wait for a second connection, which will tell us that the build is
-    // done
-    a = accept(sock, 0, 0);
-    fprintf(stderr, "%s\n", "Got a second connection, rewriting the file");
-    // Write a new content to the file
-    if (ftruncate(smuggling_fd, 0)) perror("ftruncate");
-    char * new_content = "Pwned\n";
-    int written_bytes = write(smuggling_fd, new_content, strlen(new_content));
-    if (written_bytes != strlen(new_content)) perror("write");
-}
@@ -12,6 +12,8 @@ let
      defaults = {
        nixpkgs.pkgs = nixpkgsFor.${system}.native;
        nix.checkAllErrors = false;
+        # nixos-option fails to build with lix and no tests use any of the tools
+        system.disableInstallerTools = true;
      };
      _module.args.nixpkgs = nixpkgs;
      _module.args.system = system;
@@ -142,8 +144,6 @@ in
    ["i686-linux" "x86_64-linux"]
    (system: runNixOSTestFor system ./setuid/setuid.nix);

-  ca-fd-leak = runNixOSTestFor "x86_64-linux" ./ca-fd-leak;
-
  fetch-git = runNixOSTestFor "x86_64-linux" ./fetch-git;

  symlinkResolvconf = runNixOSTestFor "x86_64-linux" ./symlink-resolvconf.nix;
@@ -157,4 +157,6 @@ in
  coredumps = runNixOSTestFor "x86_64-linux" ./coredumps;

  io_uring = runNixOSTestFor "x86_64-linux" ./io_uring;
+
+  fetchurl = runNixOSTestFor "x86_64-linux" ./fetchurl.nix;
 }
@@ -0,0 +1,84 @@
+# Test whether builtin:fetchurl properly performs TLS certificate
+# checks on HTTPS servers.
+
+{ lib, config, pkgs, ... }:
+
+let
+
+  makeTlsCert = name: pkgs.runCommand name {
+    nativeBuildInputs = with pkgs; [ openssl ];
+  } ''
+    mkdir -p $out
+    openssl req -x509 \
+      -subj '/CN=${name}/' -days 49710 \
+      -addext 'subjectAltName = DNS:${name}' \
+      -keyout "$out/key.pem" -newkey ed25519 \
+      -out "$out/cert.pem" -noenc
+  '';
+
+  goodCert = makeTlsCert "good";
+  badCert = makeTlsCert "bad";
+
+in
+
+{
+  name = "fetchurl";
+
+  nodes = {
+    machine = { lib, pkgs, ... }: {
+      services.nginx = {
+        enable = true;
+
+        virtualHosts."good" = {
+          addSSL = true;
+          sslCertificate = "${goodCert}/cert.pem";
+          sslCertificateKey = "${goodCert}/key.pem";
+          root = pkgs.runCommand "nginx-root" {} ''
+            mkdir "$out"
+            echo 'hello world' > "$out/index.html"
+          '';
+        };
+
+        virtualHosts."bad" = {
+          addSSL = true;
+          sslCertificate = "${badCert}/cert.pem";
+          sslCertificateKey = "${badCert}/key.pem";
+          root = pkgs.runCommand "nginx-root" {} ''
+            mkdir "$out"
+            echo 'foobar' > "$out/index.html"
+          '';
+        };
+      };
+
+      security.pki.certificateFiles = [ "${goodCert}/cert.pem" ];
+
+      networking.hosts."192.168.1.1" = [ "good" "bad" ];
+
+      virtualisation.writableStore = true;
+
+      nix.settings.experimental-features = "nix-command";
+    };
+  };
+
+  testScript = { nodes, ... }: ''
+    machine.wait_for_unit("nginx")
+    machine.wait_for_open_port(443)
+
+    out = machine.succeed("curl https://good/index.html")
+    assert out == "hello world\n"
+
+    out = machine.succeed("cat ${badCert}/cert.pem > /tmp/cafile.pem; curl --cacert /tmp/cafile.pem https://bad/index.html")
+    assert out == "foobar\n"
+
+    # Fetching from a server with a trusted cert should work.
+    machine.succeed("nix build --no-substitute --expr 'import <nix/fetchurl.nix> { url = \"https://good/index.html\"; hash = \"sha256-qUiQTy8PR5uPgZdpSzAYSw0u0cHNKh7A+4XSmaGSpEc=\"; }'")
+
+    # Fetching from a server with an untrusted cert should fail.
+    err = machine.fail("nix build --no-substitute --expr 'import <nix/fetchurl.nix> { url = \"https://bad/index.html\"; hash = \"sha256-rsBwZF/lPuOzdjBZN2E08FjMM3JHyXit0Xi2zN+wAZ8=\"; }' 2>&1")
+    print(err)
+    assert "SSL peer certificate or SSH remote key was not OK" in err
+
+    # Fetching from a server with a trusted cert should work via environment variable override.
+    machine.succeed("NIX_SSL_CERT_FILE=/tmp/cafile.pem nix build --no-substitute --expr 'import <nix/fetchurl.nix> { url = \"https://bad/index.html\"; hash = \"sha256-rsBwZF/lPuOzdjBZN2E08FjMM3JHyXit0Xi2zN+wAZ8=\"; }'")
+  '';
+}
@@ -45,12 +45,12 @@ in {
    server.wait_for_unit("network-online.target")
    client.wait_for_unit("network-online.target")

-    client.succeed("mkdir -m 700 /root/.ssh")
+    client.succeed("mkdir -m 700 /root/.ssh || [[ -d /root/.ssh ]]")
    client.copy_from_host("key", "/root/.ssh/id_ed25519")
    client.succeed("chmod 600 /root/.ssh/id_ed25519")

    # Install the SSH key on the server.
-    server.succeed("mkdir -m 700 /root/.ssh")
+    server.succeed("mkdir -m 700 /root/.ssh || [[ -d /root/.ssh ]]")
    server.copy_from_host("key.pub", "/root/.ssh/authorized_keys")
    server.wait_for_unit("sshd.service")
    client.succeed(f"ssh -o StrictHostKeyChecking=no {server.name} 'echo hello world' >&2")
@@ -48,6 +48,7 @@ in
          imports = [ test.config.builders.config ];
          services.openssh.enable = true;
          virtualisation.writableStore = true;
+          virtualisation.additionalPaths = [ config.system.build.extraUtils ];
          nix.settings.sandbox = true;
          nix.settings.substituters = lib.mkForce [ ];
        };
@@ -0,0 +1,16 @@
+#include "test-data.hh"
+#include "strings.hh"
+
+namespace nix {
+
+Path getUnitTestData()
+{
+    return getEnv("_NIX_TEST_UNIT_DATA").value();
+}
+
+Path getUnitTestDataPath(std::string_view path)
+{
+    return absPath(getUnitTestData() + "/" + path);
+}
+
+}
@@ -0,0 +1,20 @@
+#pragma once
+
+#include "types.hh"
+#include "environment-variables.hh"
+#include "file-system.hh"
+
+namespace nix {
+
+/**
+ * The path to the unit test data directory. See the contributing guide
+ * in the manual for further details.
+ */
+Path getUnitTestData();
+
+/**
+ * Resolve a path under the unit test data directory to an absolute path.
+ */
+Path getUnitTestDataPath(std::string_view path);
+
+}
@@ -3,6 +3,8 @@
 #include "strings.hh"
 #include "types.hh"
 #include "terminal.hh"
+#include "unix-domain-socket.hh"
+#include "tests/test-data.hh"

 #include <gtest/gtest.h>

@@ -207,6 +209,85 @@ namespace nix {
        ASSERT_FALSE(pathExists("/schnitzel/darmstadt/pommes"));
    }

+    /* ----------------------------------------------------------------------------
+     * AutoCloseFD::guessOrInventPath
+     * --------------------------------------------------------------------------*/
+    void testGuessOrInventPathPrePostDeletion(AutoCloseFD & fd, Path & path) {
+        {
+            SCOPED_TRACE(fmt("guessing path before deletion of '%1%'", path));
+            ASSERT_TRUE(fd);
+            /* We cannot predict what the platform will return here.
+             * But it cannot fail. */
+            ASSERT_TRUE(fd.guessOrInventPath().size() >= 0);
+        }
+        {
+            SCOPED_TRACE(fmt("guessing path after deletion of '%1%'", path));
+            deletePath(path);
+            /* We cannot predict what the platform will return here.
+             * But it cannot fail. */
+            ASSERT_TRUE(fd.guessOrInventPath().size() >= 0);
+        }
+    }
+    TEST(guessOrInventPath, files) {
+        Path filePath = getUnitTestDataPath("guess-or-invent/test.txt");
+        createDirs(dirOf(filePath));
+        writeFile(filePath, "some text");
+        AutoCloseFD file{open(filePath.c_str(), O_RDONLY, 0666)};
+        testGuessOrInventPathPrePostDeletion(file, filePath);
+    }
+
+    TEST(guessOrInventPath, directories) {
+        Path dirPath = getUnitTestDataPath("guess-or-invent/test-dir");
+        createDirs(dirPath);
+        AutoCloseFD directory{open(dirPath.c_str(), O_DIRECTORY, 0666)};
+        testGuessOrInventPathPrePostDeletion(directory, dirPath);
+    }
+
+#ifdef O_PATH
+    TEST(guessOrInventPath, symlinks) {
+        Path symlinkPath = getUnitTestDataPath("guess-or-invent/test-symlink");
+        Path targetPath = getUnitTestDataPath("guess-or-invent/nowhere");
+        createDirs(dirOf(symlinkPath));
+        createSymlink(targetPath, symlinkPath);
+        AutoCloseFD symlink{open(symlinkPath.c_str(), O_PATH | O_NOFOLLOW, 0666)};
+        testGuessOrInventPathPrePostDeletion(symlink, symlinkPath);
+    }
+
+    TEST(guessOrInventPath, fifos) {
+        Path fifoPath = getUnitTestDataPath("guess-or-invent/fifo");
+        createDirs(dirOf(fifoPath));
+        ASSERT_TRUE(mkfifo(fifoPath.c_str(), 0666) == 0);
+        AutoCloseFD fifo{open(fifoPath.c_str(), O_PATH | O_NOFOLLOW, 0666)};
+        testGuessOrInventPathPrePostDeletion(fifo, fifoPath);
+    }
+#endif
+
+    TEST(guessOrInventPath, pipes) {
+        int pipefd[2];
+
+        ASSERT_TRUE(pipe(pipefd) == 0);
+
+        AutoCloseFD pipe_read{pipefd[0]};
+        ASSERT_TRUE(pipe_read);
+        AutoCloseFD pipe_write{pipefd[1]};
+        ASSERT_TRUE(pipe_write);
+
+        /* We cannot predict what the platform will return here.
+         * But it cannot fail. */
+        ASSERT_TRUE(pipe_read.guessOrInventPath().size() >= 0);
+        ASSERT_TRUE(pipe_write.guessOrInventPath().size() >= 0);
+        pipe_write.close();
+        ASSERT_TRUE(pipe_read.guessOrInventPath().size() >= 0);
+        pipe_read.close();
+    }
+
+    TEST(guessOrInventPath, sockets) {
+        Path socketPath = getUnitTestDataPath("guess-or-invent/socket");
+        createDirs(dirOf(socketPath));
+        AutoCloseFD socket = createUnixDomainSocket(socketPath, 0666);
+        testGuessOrInventPathPrePostDeletion(socket, socketPath);
+    }
+
    /* ----------------------------------------------------------------------------
     * concatStringsSep
     * --------------------------------------------------------------------------*/
@@ -19,6 +19,7 @@ libutil_test_support_sources = files(
  'libutil-support/tests/cli-literate-parser.cc',
  'libutil-support/tests/hash.cc',
  'libutil-support/tests/terminal-code-eater.cc',
+  'libutil-support/tests/test-data.cc',
 )
 libutil_test_support = library(
  'lixutil-test-support',
@@ -1,5 +1,5 @@
 {
-    "version": "2.91.0",
+    "version": "2.91.3",
    "official_release": false,
    "release_name": "Dragon's Breath"
 }