libstore: fallback on creating a safe space in the default tempdir

If `settings.buildDir` cannot be written to, because we are in a chroot
store, unprivileged or anything.

We can and should always gracefully fallback to a *secure* location
inside of /tmp, i.e. `/tmp/<a directory under 0700>/<our temporary
directory for build under 0700>/...`.

This does not reintroduce CVE-2025-52991 because we are creating a
directory in-between compared to creating only ONE level of directory.

Under macOS, the first level of directory has actually mode 0755 instead
of 0700 as macOS often do not possess the right primitives to chroot
inside of these directories, leading to
https://github.com/NixOS/nix/pull/11031.

Thanks to Emily for the heads-up on this type of matter.

Fixes #876.

Change-Id: Ie521202923f763225e1901ab1b9b6c6132aaf548
Signed-off-by: Raito Bezarius <raito@lix.systems>

doc/manual/rl-next/infallible-build-dirs.md

@@ -0,0 +1,25 @@
+---
+synopsis: "Fallback to safe temp dir when build-dir is unwritable"
+issues: [fj#876]
+cls: [3503]
+category: "Fixes"
+credits: ["raito", "horrors"]
+---
+
+Non-daemon builds started failing with a permission error after introducing the `build-dir` option:
+
+```
+$ nix build --store ~/scratch nixpkgs#hello --rebuild
+error: creating directory '/nix/var/nix/builds/nix-build-hello-2.12.2.drv-0': Permission denied
+```
+
+This happens because:
+
+1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
+2. The user lacks permissions for that path.
+
+We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
+
+To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
+
+**Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.

lix/libstore/build/local-derivation-goal.cc

@@ -484,17 +484,47 @@ kj::Promise<Outcome<void, Goal::WorkResult>> LocalDerivationGoal::startBuilder()
         });
     }
 
-    createDirs(settings.buildDir.get());
+    try {
+        auto buildDir = worker.buildDirOverride.value_or(settings.buildDir.get());
 
-    /* Create a temporary directory where the build will take
-       place. */
-    tmpDir = createTempDir(
-        settings.buildDir.get(),
-        "nix-build-" + std::string(drvPath.name()),
-        false,
-        false,
-        0700
-    );
+        createDirs(buildDir);
+
+        /* Create a temporary directory where the build will take
+           place. */
+        tmpDir =
+            createTempDir(buildDir, "nix-build-" + std::string(drvPath.name()), false, false, 0700);
+    } catch (SysError & e) {
+        /*
+         * Fallback to the global tmpdir and create a safe space there
+         * only if it's a permission error.
+         */
+        if (e.errNo != EACCES) {
+            throw;
+        }
+
+        auto globalTmp = defaultTempDir();
+        createDirs(globalTmp);
+#if __APPLE__
+        /* macOS filesystem namespacing does not exist, to avoid breaking builds, we need to weaken
+         * the mode bits on the top-level directory. This avoids issues like
+         * https://github.com/NixOS/nix/pull/11031. */
+        constexpr int toplevelDirMode = 0755;
+#else
+        constexpr int toplevelDirMode = 0700;
+#endif
+        auto nixBuildsTmp =
+            createTempDir(globalTmp, fmt("nix-builds-%s", geteuid()), false, false, toplevelDirMode);
+        warn(
+            "Failed to use the system-wide build directory '%s', falling back to a temporary "
+            "directory inside '%s'",
+            settings.buildDir.get(),
+            nixBuildsTmp
+        );
+        worker.buildDirOverride = nixBuildsTmp;
+        tmpDir = createTempDir(
+            nixBuildsTmp, "nix-build-" + std::string(drvPath.name()), false, false, 0700
+        );
+    }
     /* The TOCTOU between the previous mkdir call and this open call is unavoidable due to
      * POSIX semantics.*/
     tmpDirFd = AutoCloseFD{open(tmpDir.c_str(), O_RDONLY | O_NOFOLLOW | O_DIRECTORY)};

lix/libstore/build/worker.hh

@@ -193,6 +193,7 @@ public:
     Store & evalStore;
     kj::AsyncIoContext & aio;
     AsyncSemaphore substitutions, localBuilds;
+    std::optional<Path> buildDirOverride;
 
 private:
     kj::TaskSet children;

tests/functional/build.sh

@@ -174,3 +174,8 @@ test "$(<<<"$out" grep -E '^error:' | wc -l)" = 3
 <<<"$out" grepQuiet -E "error: 2 dependencies of derivation '.*-x4\\.drv' failed to build"
 <<<"$out" grepQuiet -vE "hash mismatch in fixed-output derivation '.*-x3\\.drv'"
 <<<"$out" grepQuiet -vE "hash mismatch in fixed-output derivation '.*-x2\\.drv'"
+
+# Ensure when if the system build dir is inaccessible, we can still build things
+BUILD_DIR=$(mktemp -d)
+chmod 0000 "$BUILD_DIR"
+nix --build-dir "$BUILD_DIR" build -E 'with import ./config.nix; mkDerivation { name = "test"; buildCommand = "echo rawr > $out"; }' --impure --no-link