Commit Graph
16680 Commits
Author SHA1 Message Date
eldritch horrors a91e28009f libstore: weaken tmpdir root access mode
libarchive *should* not break with 0710 on the tmpdir root on darwin,
just like it doesn't break on linux, but for some reason it does. the
restriction to 0710 can be weakened to 0750 with causing any trouble.

fixes #921

Change-Id: Ia9fc2f8eb9695fc19cefae9857368d5a4e58c8b9
2025-07-20 16:52:55 +00:00
eldritch horrorsandRaito Bezarius c698466dc7 libstore: add intermediate directory to build-dirs
this makes the actual build directories used by builders invisible and
inaccessible to other processes on the system, avoiding another vector
for outside processes to interfere with builds or pass credentials the
build sandbox should not have access to into the build sandbox anyway.

fixes #919

Change-Id: Ifaa4d8e3940cfde1406e925f75c1375d2e86d81a
(cherry picked from commit 9d5a5c4dc0)
2025-07-18 03:41:44 +02:00
Raito Bezarius f10ca0d677 release: merge release 2.92.3 back to mainline
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh

Change-Id: I90d92d46de3073797e659b16cbc4eb6671d84cf4
2025-06-30 00:59:27 +02:00
Raito Bezarius 8c5e1c8f63 release: 2.92.3 "Bombe glacée"
Release produced with releng/create_release.xsh

Change-Id: If52727313f7f9a0dd4ce6c95dccfeb33585eac0a
2.92.3
2025-06-30 00:59:27 +02:00
Raito Bezarius fe090a7b9b release: release notes for 2.92.3
Release created with releng/create_release.xsh

Change-Id: I7683080179723ab8e36206c511c1e8819250fe5f
2025-06-30 00:59:26 +02:00
Raito Bezarius 4a3983906f version: 2.92.2 -> 2.92.3
Resolves critical correctness bugs as reported in fj#883.
Following the CVE fixes.

Change-Id: Ife85fb4ca71ba920aae4b98a6dc141857e10d759
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-29 20:36:57 +00:00
EmilyandRaito Bezarius bac0d67928 libstore: fix Unix sockets in the build directory on sandboxed macOS
We’re already allowing `/tmp` anyway, so this should be harmless,
and it fixes a regression in the default configuration caused by
moving the build directories out of `temp-dir`. (For instance, that
broke the Lix `guessOrInventPath.sockets` test.)

Note that removing `/tmp` breaks quite a few builds, so although it may
be a good idea in general it would require work on the Nixpkgs side.

Fixes: 749afbbe99
Change-Id: I6a6a69645f429bc50d4cb24283feda3d3091f534
(cherry picked from commit d1db3e5fa3)
2025-06-29 20:36:57 +00:00
Raito Bezarius 22a1d233bb libstore: fallback on creating a safe space in the default tempdir
If `settings.buildDir` cannot be written to, because we are in a chroot
store, unprivileged or anything.

We can and should always gracefully fallback to a *secure* location
inside of /tmp, i.e. `/tmp/<a directory under 0700>/<our temporary
directory for build under 0700>/...`.

This does not reintroduce CVE-2025-52991 because we are creating a
directory in-between compared to creating only ONE level of directory.

Under macOS, the first level of directory has actually mode 0755 instead
of 0700 as macOS often do not possess the right primitives to chroot
inside of these directories, leading to
https://github.com/NixOS/nix/pull/11031.

Thanks to Emily for the heads-up on this type of matter.

Fixes #876.

Change-Id: Ie521202923f763225e1901ab1b9b6c6132aaf548
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-29 20:36:57 +00:00
eldritch horrors 4901a6cda9 Revert "libstore/build: automatic clean up of unsuccessfully built scratch outputs"
This reverts commit 2f32368136 as this is the root cause for the critical correctness bug.

Change-Id: I573cdfbaed8316ad93197aab621ad7e3ff966c1d
2025-06-29 20:36:57 +00:00
eldritch horrors 8cbdc36a60 Revert "libstore: fix scratch output cleanup"
This reverts commit 1d36dd7596 as this is an insufficient fix for the critical correctness bug.

Change-Id: Id48fec6e2146ccc98edb39629d9944404a17ba8c
2025-06-29 20:36:57 +00:00
eldritch horrors 56b0b39465 Revert "libstore: don't delete already valid outputs after build"
This reverts commit 99456a0c7e as this is an insufficient fix for the critical correctness bug.

Change-Id: I950b4d007f4919877df6dd9eef5d6c1e95ca270e
2025-06-29 20:15:29 +00:00
Raito Bezarius 438e1839f1 releng: go back to a non-official release
This is necessary to unblock releng.

This is suboptimal releng because we usually merge the branches back and
the releng script did not.

Change-Id: I12c9f6713adf344c1597e2d4adafaf2d3c62f537
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-27 23:17:38 +02:00
eldritch horrors 99456a0c7e libstore: don't delete already valid outputs after build
eagerly consider outputs as not needing deletion during output
registration rather than only doing so after registration. not
waiting for registration to succeed may keep store paths alive
in the file system if registration fails for some reason; that
seem preferrable to the possibility of having another instance
of this bug. since we only leave *good* outputs around there's
not much to worry about except maybe bit of wasted disk space.

fixes #883

Change-Id: I8c22c92e39b9e203f1061278f86cde19dc4474a4
2025-06-27 16:35:53 +02:00
eldritch horrors 1d36dd7596 libstore: fix scratch output cleanup
the daemon must use real store paths, not virtual store paths. using
virtual paths may inadvertently delete paths in the system nix store
when a build was run on a redirected store as root, which isn't good

Change-Id: Id048b236bda0e0ab1f3be6ccba0ddc1de2a3e941
2025-06-27 16:35:04 +02:00
Raito Bezarius 21317ce965 release: 2.92.2 "Bombe glacée"
Release produced with releng/create_release.xsh

Change-Id: I43476413ebe28aa60618f24205fdda9a3a92902c
2.92.2
2025-06-24 10:50:36 +00:00
Raito Bezarius b43a289c02 release: release notes for 2.92.2
Release created with releng/create_release.xsh

Change-Id: I9bcc8bec834ebf9b2d0ec9df00a585b610152ba8
2025-06-24 10:50:36 +00:00
Raito Bezarius 2138b0f7e9 version: 2.92.1 -> 2.92.2
Fixes CVE-2025-46415, CVE-2025-46416, CVE-2025-52991, CVE-2025-52992,
and CVE-2025-52993.

Change-Id: Ia73b43abd1cf2d0cfa185938c59d3a836a9eb160
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-24 10:50:36 +00:00
Raito Bezarius 2f32368136 libstore/build: automatic clean up of unsuccessfully built scratch outputs
When a build fails, its scratch output paths are not cleaned up.

Until recently, this was deemed not a problem but as part of the effort
to harden the Nix builds and protect these paths against being part of a
staged attack (race conditions, etc.), we automatically cleanup after
failed builds.

Fixes CVE-2025-52992.

Change-Id: I58481b1cc83826298b9d80d37fecf81f117ccb09
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-24 10:50:36 +00:00
eldritch horrorsandRaito Bezarius a713c064af libstore: don't default build-dir to temp-dir
if a build directory is accessible to other users it is possible to
smuggle data in and out of build directories. usually this ins only
a build purity problem, but in combination with other issues it can
be used to break out of a build sandbox. to prevent this we default
to using a subdirectory of nixStateDir (which is more restrictive).

Fixes CVE-2025-52991.

Change-Id: Iacfc9b50534de158618c815f9fb99d7dae1be4d0
2025-06-24 10:50:36 +00:00
cba05329ee libstore: use pasta for FODs if available
This allows using a userspace program, pasta, to handle comms between
the build sandbox, and the outside world; allowing for full isolation
including the network namespace, closing the "fixed-output derivation
talks to the host over an abstract domain socket" hole for good.

Fixes CVE-2025-46416.

Co-Authored-By: Puck Meerburg <puck@puckipedia.com>
Change-Id: Ifd499b7dbb3784600a6e842fede65fc031ff9f15
2025-06-24 10:50:36 +00:00
eldritch horrors f5f2e1537d libutil: add capability support to runProgram2
launching pasta to not run as root will ambient require capabilities.

Change-Id: I1dd2506a1fa3944a9d9062123ef8a74903c597ea
2025-06-24 10:50:36 +00:00
eldritch horrors 0ea8649445 libutil: add generic redirections runProgram2
explicit stderr redirection makes mergeStderrToStdout unnecessary also.

Change-Id: I63de929e6dc53f6c5ceb2d43c2ce288bfc04d872
2025-06-24 10:50:36 +00:00
eldritch horrors d0678a57f9 libutil: make RunningProgram more useful
make it moveable, make it killable, and add a stdout fd accessor.

Change-Id: I2387cbe8ac67b899a322cd6c7d306ef9ea7abcd0
2025-06-24 10:50:36 +00:00
Raito Bezarius 1d4ddb7e3b libutil: ensure that _deletePath does NOT use absolute paths with dirfds
When calling `_deletePath` with a parent file descriptor, `openat` is
made effective by using relative paths to the directory file descriptor.

To avoid the problem, the signature is changed to resist misuse with an
assert in the prologue of the function.

Fixes CVE-2025-46415.

Change-Id: I6b3fc766bad2afe54dc27d47d1df3873e188de96
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-24 10:50:36 +00:00
Raito Bezarius c7867e89f9 libstore: ensure that passAsFile is created in the original temp dir
This ensures that `passAsFile` data is created inside the expected
temporary build directory by `openat()` from the parent directory file
descriptor.

Fixes CVE-2025-52993.

Change-Id: Ie5273446c4a19403088d0389ae8e3f473af8879a
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-24 10:45:29 +00:00
Raito Bezarius deb5150c82 libutil: writeFile variant for file descriptors
`writeFile` lose its `sync` boolean flag to make things simpler.

A new `writeFileAndSync` function is created and all call sites are
converted to it.

Change-Id: Ib871a5283a9c047db1e4fe48a241506e4aab9192
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-23 16:50:44 +02:00
Raito Bezarius 92eb418a59 libstore: chown to builder variant for file descriptors
We use it immediately for the build temporary directory.

Change-Id: I180193c63a2b98721f5fb8e542c4e39c099bb947
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-23 16:50:44 +02:00
Raito Bezarius 7fceee3ce3 libstore: open build directory as a dirfd as well
We now keep around a proper AutoCloseFD around the temporary directory
which we plan to use for openat operations and avoiding the build
directory being swapped out while we are doing something else.

Change-Id: I18d387b0f123ebf2d20c6405cd47ebadc5505f2a
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-23 16:50:44 +02:00
Raito Bezarius 31f976dc19 libutil: guess or invent a path from file descriptors
This is useful for certain error recovery paths (no pun intended) that
does not thread through the original path name.

Change-Id: I2d800740cb4f9912e64c923120d3f977c58ccb7e
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-23 16:50:44 +02:00
jadeandLix Systems Gerrit d8db15010d Merge changes I577bdc15,I1c5f7a5c,I75f56c80 into release-2.92
* changes:
  version: 2.92.1
  release-notes: note the flake.lock fix
  libexpr/flake: user friendly error if parsing flake.lock fails
2025-03-17 22:26:40 +00:00
Jade Lovelace 253be7d2ba version: 2.92.1
Change-Id: I577bdc1549e80c0b7621624e2d390153581b3bf2
2025-03-17 12:11:24 -07:00
Jade Lovelace a34e583305 releng: workaround awscli breaking garage by changing checksum algo
CHERRY-PICK: required to release

We use https://garagehq.deuxfleurs.fr for https://docs.lix.systems,
https://releases.lix.systems, https://cache.lix.systems. It's generally
great, but AWS doesn't, erm, care, about other implementations and broke
their client library.

We already ran into
https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/824, which was
mitigated by a garage upgrade to create a new error. These bugs were
what got us:
- https://github.com/boto/boto3/issues/4392
- https://github.com/aws/aws-cli/issues/9214

Error:
upload failed: release/manual/.nojekyll to s3://docs/manual/lix/nightly/.nojekyll An error occurred (InvalidRequest) when calling the PutObject operation: Bad request: invalid checksum algorithm

The missing checksum algorithm is CRC32NVME, with a bug filed here:
https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/963

Change-Id: Ib78a89034bf0f2a6773fc505a347b2aadb775e93
(cherry picked from commit 0f4c5b3b8a)
2025-03-17 19:10:24 +00:00
Rebecca Turnerandjade a8fb008106 releng: use aws s3 sync --delete when uploading manual
CHERRY-PICK: prereq to fixing the aws flags for garage shenanigans

For the user manual, we don't delete things that are missing when doing
aws s3 sync. This doesn't seem wise. If you delete a page in the manual,
it will stay public and visible.

This adds `--delete` to the `aws s3 sync` commands to fix this issue.

Closes #396

Change-Id: I6d7fb97bcdab96c0115d6c66fea0310125207df4
(cherry picked from commit 16df34b295)
2025-03-17 19:09:10 +00:00
Jade Lovelace 32cbb66a69 release-notes: note the flake.lock fix
Change-Id: I1c5f7a5cba12da1bfb2896ffc04ec89862cb346d
2025-03-17 12:06:39 -07:00
giliceandjade 9631e9a30f libexpr/flake: user friendly error if parsing flake.lock fails
CHERRY-PICK: repeated bug reports

Fix: https://git.lix.systems/lix-project/lix/issues/559
Change-Id: I75f56c801d7f16a2e1ef8e702f16f1ef91f7b01f
(cherry picked from commit 68373f8664)
2025-03-17 19:01:23 +00:00
jadeandLix Systems Gerrit 98772c4a3b Merge "Revert "libstore: don't use curl decompression support"" into release-2.92 2025-03-17 18:17:41 +00:00
jadeandLix Systems Gerrit a7fd5c3867 Merge "libexpr: fix checkSourcePath purity regression" into release-2.92 2025-03-17 17:58:29 +00:00
jadeandLix Systems Gerrit 6f3a7bbeb0 Revert "libstore: don't use curl decompression support"
This reverts commit 0d1f794178.

Reason for revert: pennae says it makes the bug more visible, for some unknown reason. See https://git.lix.systems/lix-project/lix/issues/662

Change-Id: I91853cb6645f188a260c4c71cfff4de7bea99feb
2025-03-17 17:56:20 +00:00
eldritch horrorsandjade 0d1f794178 libstore: don't use curl decompression support
it's broken with http2 and transfer flow control. cf fj#662

BACKPORT: due to regression
Change-Id: Iaf6312bfcefa18d168faef47f57481199dd30b8d
2025-03-09 08:39:10 +00:00
Alois Wohlschlager ef5689dc1b libexpr: fix checkSourcePath purity regression
Starting with commit 0dbfa7b26e access would also
be allowed to ancestors of allowed paths. This is (ironically) a significant
purity regression, since several users of the purity checks will themselves
assume that arbitrary descent is allowed. For example, `builtins.readDir` and
`builtins.path` could now refer to the filesystem root, breaking purity
entirely in the latter case by allowing to read arbitrary files. Restore the
previous behaviour of only allowing access to explicitly allowed paths.

Change-Id: Ie64180733ab735da9873255e1ccbf95ba7c9161c
(cherry picked from commit 9d99a7c2cf)
2025-03-04 18:15:15 +01:00
Lunaphiedandjade db55ca9a2e hotfix for releng scripts, fixes #440
Change-Id: I33b29fbf3920a2d4fb53b58db477dff5bf7a1af1
2025-02-21 01:17:44 +00:00
eldritch horrorsandJade Lovelace 4cd618272a libstore: never return from lockFile without a lock
signals could cause lockFile to return without having locked the file.
the garbage collector didn't check for this, and then hilarity ensued.

Change-Id: If86d33595e8bf5510d2b032139342261dc6e07c9
(cherry picked from commit d186064c3d)
2025-02-09 12:54:48 -08:00
eldritch horrorsandJade Lovelace 73f0213500 libstore: split tryLockFile lockFile
lockFile is currently interruptible by signals like SIGCHLD. which
happen a lot in daemons. now imagine that daemon enabled automatic
garbage collection. observe that the local store does not actually
check whether its lock operations have succeeded ... get the idea?

Change-Id: Ibfd7ee786c4fee3add72d4456a7e95e73e09c73e
(cherry picked from commit 79f9c39e36)
2025-02-09 12:54:48 -08:00
eldritch horrorsandJade Lovelace 9c1db3cd8b libstore: extract unlockFile from lockFile
propagating the "unlock" lock type through this high-level api is
nonsense. it doesn't make sense to treat locking and unlocking as
similar operations; unlocking *must* not not interruptible by our
checkInterrupt machinery or it will just leave locks lying around
for a potentially very long time. unlock operations should not be
taking long enough to *want* them interrupted anyway. even on nfs
this makes very little sense because nfs waits *uninterruptibly*.

Change-Id: I10d605c8fe6c651bee64466eee1f8e20251d39f4
(cherry picked from commit aa87c8aa93)
2025-02-09 12:54:48 -08:00
Dusk Banksandeldritch horrors 5fa27057b2 libexpr: fix --debugger --ignore-try
a65e9e5828 did not inform `tryEval` that
(as far as it's concerned) `state.debug` moved to `state.errors.debug`
and changed types. this resulted in the REPL erroneously coming up, that
REPL having a non-debug state, and segfaulting after that REPL exited.

it's probably good that `state.debug` isn't mutated by `--ignore-try`
anymore.

Change-Id: I1918e93edacd626452aa423fc2eb825080738835
Fixes: a65e9e5828 ("libexpr: extract eval error creation into new type")
Signed-off-by: Dusk Banks <me@bb010g.com>
(cherry picked from commit 6a583136b7)
2025-01-31 12:36:14 +00:00
Dusk Banksandeldritch horrors d2b1af70ee doc: add bb010g to change-authors
Change-Id: Iddd3c21b2c42669cec394ac6b80f4e766e4cb81c
Signed-off-by: Dusk Banks <me@bb010g.com>
(cherry picked from commit 050cf17307)
2025-01-31 03:21:44 +00:00
Raito Bezariusandjade 50def3fa73 fix(gc): log sudden "path in use" exceptions and recover during GC gracefully
Original-Author: picnoir <picnoir@alternativebit.fr>

Inspired from
https://github.com/NixOS/nix/pull/11922/commits/ced8d311a593fcf9c3823e4e118474ac132d8e60
and adapted for Lix needs.

TL;DR: The topological sort should ensure that it is possible to delete
the path iterated upon. Nonetheless, in some cases,
`invalidatePathChecked` can still throw `PathInUse`, the exception
bubbles up and cancel the garbage collection procedure, leaving the rest
of the paths untouched. This change ensure that the error is logged for
further investigation but doesn't prevent the GC to continue when it
can.

After code review, we decided to make it a `printInfo` to inform the
user about sudden "in use" dependencies during garbage collection and
let them re-run garbage collection if they care about this.

References: https://github.com/NixOS/nix/issues/11923
References: https://git.lix.systems/lix-project/lix/issues/621
Change-Id: I5606c9afd16b5faa747b713fde2dc24016990ba3
Signed-off-by: Raito Bezarius <raito@lix.systems>
(cherry picked from commit 6a41dae49a)
2025-01-23 06:04:00 +00:00
eldritch horrorsandjade 8e2ab5532c FileTransfer: fix race condition on awaitData
There's a race condition where awaitData could early-return for data
coming from a 404 response or similar and thus not rethrow the exception
that is forthcoming, and a related race during transfer setup (which
could retry a transfer *twice* per retry round).

This would then cause substitution failures like below since the exception
isn't caught in HttpBinaryCacheStore::getFile as intended, but instead
by an exception handler downstream of `drain()` which would error out
the entire operation.

Symptom:

 » nix-build ./docs-service.nix -o "docs-service-result"
error: unable to download 'https://cache.nixos.org/7mr3fy8w66gi5inmf0jkkkl90lxy4jyg.narinfo': HTTP e
rror 404 ()

       response body:

This is kind of a hack in how it is implemented: it assumes that you
can't intentionally be receiving a large unsuccessful response since in
such a case, `awaitData` will wait for finish() to be called to throw an
exception and will never escape until the download finishes, while
continuing to buffer the entire response into memory, which could be bad
if an error response had a large payload.

That said, nobody is sending Lix 1GiB of 404, so meh I guess, and this
is how it is seemingly intended to work. That was a design flaw of the
thing before any of the Lix team got our paws on it.

I tested this by adding _exit(0) inside the expected exception catch and
then running the offending command repeatedly to see if the symptom ever
appeared again, and it did not.

Needs cherry-pick to 2.92 and a 2.92.1 release once reviewed.

Fixes: https://git.lix.systems/lix-project/lix/issues/635
Co-Authored-By: lix@jade.fyi
Change-Id: If54f6eeaad60b5ca9d5b77d4d9232da1d295e7d1
(cherry picked from commit de58cd6e80)
2025-01-22 21:06:58 +00:00
Jade Lovelace 079528098f release: merge release 2.92.0 back to mainline
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh

Change-Id: I4e4650d96de82b46c35171b3a9fc1e3a6ca8e6a1
2025-01-18 13:04:46 -08:00
Jade Lovelace 9446732466 release: 2.92.0 "Bombe glacée"
Release produced with releng/create_release.xsh

Change-Id: Ife8c3ed7dc44b6ed334e53ec260fd17a931bf55f
2.92.0
2025-01-18 13:04:46 -08:00