Compare commits

..
56 Commits
Author SHA1 Message Date
Alois Wohlschlager 7653f109c6 Enable nixpkgsLibTests again
The required nixpkgs bumps have now been performed.

Change-Id: I6a6a6964316f8434c355405c52b0a7b7f70b4332
2025-09-07 18:58:38 +02:00
Alois Wohlschlager 16894c7bc5 flake: update nixpkgs input
Without https://github.com/NixOS/nixpkgs/pull/434761 evaluation of the
`nixpkgsLibTests` will fail in CI with recent enough Lix, due to reliance on
the TOML integer saturation bug.

Reported-by: Sergei Zimmerman <sergei@zimmerman.foo>

Change-Id: I6a6a6964838009d2c525f67035f84072fdfad988
2025-09-07 18:56:03 +02:00
594efc2e56 flake/inputs: 24.11-small -> 25.05-small
We upgrade to 25.05 release, which contains the curl commit
https://github.com/curl/curl/commit/5fbd78eb2dc4afbd8884e8eed27147fc3d4318f6
done in
https://github.com/NixOS/nixpkgs/pull/396200#issuecomment-2795944006.

This fixes HTTP transfers generating arbitrary errors and possibly
failing unusually.

Users who are already depending on 25.05-small or a recent unstable
already had the fix.

Special mention to the Linux kernel who gave me the opportunity to get
on a 24 hours bisection side quest to fix the local release engineering
test.

Special thanks to everyone who had to endure me ranting.

Change-Id: I866caf65d5ea103f1fa5eccd57df8031c9eacda0
Co-authored-by: eldritch horrors <pennae@lix.systems>
Co-authored-by: helle <helle@h3l.li>
Co-authored-by: Alois Wohlschlager <alois1@gmx-topmail.de>
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-09-07 18:46:29 +02:00
K900andAlois Wohlschlager c63eb21e8d perl: drop version check
Meson started handling those differently[0], and Perl 5.8 is old enough
that anyone running an even older Perl is honestly asking for it.

[0]: https://github.com/mesonbuild/meson/commit/a3679a64eec7c312c81d657880f34f015426c7db

Change-Id: Ifba22e38a82abf4b6965dda0eed43a3064533d25
2025-09-07 18:44:40 +02:00
Alois Wohlschlager 92cf5ac91b Disable nixpkgsLibTests temporarily
Several nixpkgs bumps up to https://github.com/NixOS/nixpkgs/pull/434761 will
be required to fix them. Disable the tests temporarily to avoid having to
squash all the backports, which would lose history.

Change-Id: I6a6a6964029a3442c4691e74a52923e3f59d6448
2025-09-07 18:41:55 +02:00
sternenseemann 88302eaaf4 libcmd: add support for lowdown >= 1.4
lowdown 1.4.0 changed the lowdown_opts to include a new and separate
lowdown_opts_term which allows for configuring values specific to
-Tterm (which we're using). This version should have been called 2.0.0
according to semver, hence 2.0.0 was released later without any actual
breaking changes to sort of migitate the problem.

We need to support lowdown >= 1.3 && < 1.4 since the ship has sailed for
updating lowdown in NixOS 25.05 as well as lowdown >= 1.4 or we'll be
stuck in Nixpkgs forever. Support for < 1.4 can be dropped as soon as
NixOS 25.05 is EOL, assuming this change lands before NixOS 25.11
branch-off.

We detect the changed API based on the lowdown version from pkg-config
and define LOWDOWN_SEPARATE_TERM_OPTS based on that. The ifdef is named
according to the specific API change that impacts us, so that it's
hopefully a little simpler to maintain going forward. In the new API,
all newly configurable settings use what would have been the (implicit)
default before. Changing some of these values, especially hpadding,
could be interesting in future changes.

Compared to cl/3081, this change makes sure to initialize all new fields
of lowdown_opts_term explicitly.

It seems that, while making -Tterm more configurable, lowdown's word
wrapping behavior changed slightly which broke basic_repl.test. I've
chosen to work around this by using builtins.add as an example which has
a very short documentation string, so wrapping doesn't matter.

Change-Id: Id73be4c0e43d7eb4f56e10a261b4254402698ff8
(cherry picked from commit 858de5f47a)
2025-07-23 23:42:28 +02:00
Jade Lovelace 4601564996 fix: VERSION_SUFFIX was not getting into meson
It was a regression caused by switching to structured attrs, I think.

Fixes: https://git.lix.systems/lix-project/lix/issues/908
Change-Id: Ia62892919945a1f16a81a2e0bb585595fac46669
(cherry picked from commit ae00b12983)
2025-07-20 21:31:20 +00:00
K900andjade 4015082602 packaging: use structuredAttrs
staging-next banned !structuredAttrs && separateDebugInfo && disallowedRequisites
due to weird output interactions. Enable structuredAttrs so we can build again.

Also, fix type confusion that makes stdenv explode (https://github.com/NixOS/nixpkgs/issues/422989).

Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: Ic0c773394ee79e10d427f27750d59892d6d1f1d1
(cherry picked from commit 378b360bf8)
2025-07-20 21:31:20 +00:00
eldritch horrors a91e28009f libstore: weaken tmpdir root access mode
libarchive *should* not break with 0710 on the tmpdir root on darwin,
just like it doesn't break on linux, but for some reason it does. the
restriction to 0710 can be weakened to 0750 with causing any trouble.

fixes #921

Change-Id: Ia9fc2f8eb9695fc19cefae9857368d5a4e58c8b9
2025-07-20 16:52:55 +00:00
eldritch horrorsandRaito Bezarius c698466dc7 libstore: add intermediate directory to build-dirs
this makes the actual build directories used by builders invisible and
inaccessible to other processes on the system, avoiding another vector
for outside processes to interfere with builds or pass credentials the
build sandbox should not have access to into the build sandbox anyway.

fixes #919

Change-Id: Ifaa4d8e3940cfde1406e925f75c1375d2e86d81a
(cherry picked from commit 9d5a5c4dc0)
2025-07-18 03:41:44 +02:00
Raito Bezarius f10ca0d677 release: merge release 2.92.3 back to mainline
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh

Change-Id: I90d92d46de3073797e659b16cbc4eb6671d84cf4
2025-06-30 00:59:27 +02:00
Raito Bezarius 8c5e1c8f63 release: 2.92.3 "Bombe glacée"
Release produced with releng/create_release.xsh

Change-Id: If52727313f7f9a0dd4ce6c95dccfeb33585eac0a
2025-06-30 00:59:27 +02:00
Raito Bezarius fe090a7b9b release: release notes for 2.92.3
Release created with releng/create_release.xsh

Change-Id: I7683080179723ab8e36206c511c1e8819250fe5f
2025-06-30 00:59:26 +02:00
Raito Bezarius 4a3983906f version: 2.92.2 -> 2.92.3
Resolves critical correctness bugs as reported in fj#883.
Following the CVE fixes.

Change-Id: Ife85fb4ca71ba920aae4b98a6dc141857e10d759
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-29 20:36:57 +00:00
EmilyandRaito Bezarius bac0d67928 libstore: fix Unix sockets in the build directory on sandboxed macOS
We’re already allowing `/tmp` anyway, so this should be harmless,
and it fixes a regression in the default configuration caused by
moving the build directories out of `temp-dir`. (For instance, that
broke the Lix `guessOrInventPath.sockets` test.)

Note that removing `/tmp` breaks quite a few builds, so although it may
be a good idea in general it would require work on the Nixpkgs side.

Fixes: 749afbbe99
Change-Id: I6a6a69645f429bc50d4cb24283feda3d3091f534
(cherry picked from commit d1db3e5fa3)
2025-06-29 20:36:57 +00:00
Raito Bezarius 22a1d233bb libstore: fallback on creating a safe space in the default tempdir
If `settings.buildDir` cannot be written to, because we are in a chroot
store, unprivileged or anything.

We can and should always gracefully fallback to a *secure* location
inside of /tmp, i.e. `/tmp/<a directory under 0700>/<our temporary
directory for build under 0700>/...`.

This does not reintroduce CVE-2025-52991 because we are creating a
directory in-between compared to creating only ONE level of directory.

Under macOS, the first level of directory has actually mode 0755 instead
of 0700 as macOS often do not possess the right primitives to chroot
inside of these directories, leading to
https://github.com/NixOS/nix/pull/11031.

Thanks to Emily for the heads-up on this type of matter.

Fixes #876.

Change-Id: Ie521202923f763225e1901ab1b9b6c6132aaf548
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-29 20:36:57 +00:00
eldritch horrors 4901a6cda9 Revert "libstore/build: automatic clean up of unsuccessfully built scratch outputs"
This reverts commit 2f32368136 as this is the root cause for the critical correctness bug.

Change-Id: I573cdfbaed8316ad93197aab621ad7e3ff966c1d
2025-06-29 20:36:57 +00:00
eldritch horrors 8cbdc36a60 Revert "libstore: fix scratch output cleanup"
This reverts commit 1d36dd7596 as this is an insufficient fix for the critical correctness bug.

Change-Id: Id48fec6e2146ccc98edb39629d9944404a17ba8c
2025-06-29 20:36:57 +00:00
eldritch horrors 56b0b39465 Revert "libstore: don't delete already valid outputs after build"
This reverts commit 99456a0c7e as this is an insufficient fix for the critical correctness bug.

Change-Id: I950b4d007f4919877df6dd9eef5d6c1e95ca270e
2025-06-29 20:15:29 +00:00
Raito Bezarius 438e1839f1 releng: go back to a non-official release
This is necessary to unblock releng.

This is suboptimal releng because we usually merge the branches back and
the releng script did not.

Change-Id: I12c9f6713adf344c1597e2d4adafaf2d3c62f537
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-27 23:17:38 +02:00
eldritch horrors 99456a0c7e libstore: don't delete already valid outputs after build
eagerly consider outputs as not needing deletion during output
registration rather than only doing so after registration. not
waiting for registration to succeed may keep store paths alive
in the file system if registration fails for some reason; that
seem preferrable to the possibility of having another instance
of this bug. since we only leave *good* outputs around there's
not much to worry about except maybe bit of wasted disk space.

fixes #883

Change-Id: I8c22c92e39b9e203f1061278f86cde19dc4474a4
2025-06-27 16:35:53 +02:00
eldritch horrors 1d36dd7596 libstore: fix scratch output cleanup
the daemon must use real store paths, not virtual store paths. using
virtual paths may inadvertently delete paths in the system nix store
when a build was run on a redirected store as root, which isn't good

Change-Id: Id048b236bda0e0ab1f3be6ccba0ddc1de2a3e941
2025-06-27 16:35:04 +02:00
Raito Bezarius 21317ce965 release: 2.92.2 "Bombe glacée"
Release produced with releng/create_release.xsh

Change-Id: I43476413ebe28aa60618f24205fdda9a3a92902c
2025-06-24 10:50:36 +00:00
Raito Bezarius b43a289c02 release: release notes for 2.92.2
Release created with releng/create_release.xsh

Change-Id: I9bcc8bec834ebf9b2d0ec9df00a585b610152ba8
2025-06-24 10:50:36 +00:00
Raito Bezarius 2138b0f7e9 version: 2.92.1 -> 2.92.2
Fixes CVE-2025-46415, CVE-2025-46416, CVE-2025-52991, CVE-2025-52992,
and CVE-2025-52993.

Change-Id: Ia73b43abd1cf2d0cfa185938c59d3a836a9eb160
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-24 10:50:36 +00:00
Raito Bezarius 2f32368136 libstore/build: automatic clean up of unsuccessfully built scratch outputs
When a build fails, its scratch output paths are not cleaned up.

Until recently, this was deemed not a problem but as part of the effort
to harden the Nix builds and protect these paths against being part of a
staged attack (race conditions, etc.), we automatically cleanup after
failed builds.

Fixes CVE-2025-52992.

Change-Id: I58481b1cc83826298b9d80d37fecf81f117ccb09
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-24 10:50:36 +00:00
eldritch horrorsandRaito Bezarius a713c064af libstore: don't default build-dir to temp-dir
if a build directory is accessible to other users it is possible to
smuggle data in and out of build directories. usually this ins only
a build purity problem, but in combination with other issues it can
be used to break out of a build sandbox. to prevent this we default
to using a subdirectory of nixStateDir (which is more restrictive).

Fixes CVE-2025-52991.

Change-Id: Iacfc9b50534de158618c815f9fb99d7dae1be4d0
2025-06-24 10:50:36 +00:00
cba05329ee libstore: use pasta for FODs if available
This allows using a userspace program, pasta, to handle comms between
the build sandbox, and the outside world; allowing for full isolation
including the network namespace, closing the "fixed-output derivation
talks to the host over an abstract domain socket" hole for good.

Fixes CVE-2025-46416.

Co-Authored-By: Puck Meerburg <puck@puckipedia.com>
Change-Id: Ifd499b7dbb3784600a6e842fede65fc031ff9f15
2025-06-24 10:50:36 +00:00
eldritch horrors f5f2e1537d libutil: add capability support to runProgram2
launching pasta to not run as root will ambient require capabilities.

Change-Id: I1dd2506a1fa3944a9d9062123ef8a74903c597ea
2025-06-24 10:50:36 +00:00
eldritch horrors 0ea8649445 libutil: add generic redirections runProgram2
explicit stderr redirection makes mergeStderrToStdout unnecessary also.

Change-Id: I63de929e6dc53f6c5ceb2d43c2ce288bfc04d872
2025-06-24 10:50:36 +00:00
eldritch horrors d0678a57f9 libutil: make RunningProgram more useful
make it moveable, make it killable, and add a stdout fd accessor.

Change-Id: I2387cbe8ac67b899a322cd6c7d306ef9ea7abcd0
2025-06-24 10:50:36 +00:00
Raito Bezarius 1d4ddb7e3b libutil: ensure that _deletePath does NOT use absolute paths with dirfds
When calling `_deletePath` with a parent file descriptor, `openat` is
made effective by using relative paths to the directory file descriptor.

To avoid the problem, the signature is changed to resist misuse with an
assert in the prologue of the function.

Fixes CVE-2025-46415.

Change-Id: I6b3fc766bad2afe54dc27d47d1df3873e188de96
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-24 10:50:36 +00:00
Raito Bezarius c7867e89f9 libstore: ensure that passAsFile is created in the original temp dir
This ensures that `passAsFile` data is created inside the expected
temporary build directory by `openat()` from the parent directory file
descriptor.

Fixes CVE-2025-52993.

Change-Id: Ie5273446c4a19403088d0389ae8e3f473af8879a
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-24 10:45:29 +00:00
Raito Bezarius deb5150c82 libutil: writeFile variant for file descriptors
`writeFile` lose its `sync` boolean flag to make things simpler.

A new `writeFileAndSync` function is created and all call sites are
converted to it.

Change-Id: Ib871a5283a9c047db1e4fe48a241506e4aab9192
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-23 16:50:44 +02:00
Raito Bezarius 92eb418a59 libstore: chown to builder variant for file descriptors
We use it immediately for the build temporary directory.

Change-Id: I180193c63a2b98721f5fb8e542c4e39c099bb947
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-23 16:50:44 +02:00
Raito Bezarius 7fceee3ce3 libstore: open build directory as a dirfd as well
We now keep around a proper AutoCloseFD around the temporary directory
which we plan to use for openat operations and avoiding the build
directory being swapped out while we are doing something else.

Change-Id: I18d387b0f123ebf2d20c6405cd47ebadc5505f2a
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-23 16:50:44 +02:00
Raito Bezarius 31f976dc19 libutil: guess or invent a path from file descriptors
This is useful for certain error recovery paths (no pun intended) that
does not thread through the original path name.

Change-Id: I2d800740cb4f9912e64c923120d3f977c58ccb7e
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-23 16:50:44 +02:00
jadeandLix Systems Gerrit d8db15010d Merge changes I577bdc15,I1c5f7a5c,I75f56c80 into release-2.92
* changes:
  version: 2.92.1
  release-notes: note the flake.lock fix
  libexpr/flake: user friendly error if parsing flake.lock fails
2025-03-17 22:26:40 +00:00
Jade Lovelace 253be7d2ba version: 2.92.1
Change-Id: I577bdc1549e80c0b7621624e2d390153581b3bf2
2025-03-17 12:11:24 -07:00
Jade Lovelace a34e583305 releng: workaround awscli breaking garage by changing checksum algo
CHERRY-PICK: required to release

We use https://garagehq.deuxfleurs.fr for https://docs.lix.systems,
https://releases.lix.systems, https://cache.lix.systems. It's generally
great, but AWS doesn't, erm, care, about other implementations and broke
their client library.

We already ran into
https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/824, which was
mitigated by a garage upgrade to create a new error. These bugs were
what got us:
- https://github.com/boto/boto3/issues/4392
- https://github.com/aws/aws-cli/issues/9214

Error:
upload failed: release/manual/.nojekyll to s3://docs/manual/lix/nightly/.nojekyll An error occurred (InvalidRequest) when calling the PutObject operation: Bad request: invalid checksum algorithm

The missing checksum algorithm is CRC32NVME, with a bug filed here:
https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/963

Change-Id: Ib78a89034bf0f2a6773fc505a347b2aadb775e93
(cherry picked from commit 0f4c5b3b8a)
2025-03-17 19:10:24 +00:00
Rebecca Turnerandjade a8fb008106 releng: use aws s3 sync --delete when uploading manual
CHERRY-PICK: prereq to fixing the aws flags for garage shenanigans

For the user manual, we don't delete things that are missing when doing
aws s3 sync. This doesn't seem wise. If you delete a page in the manual,
it will stay public and visible.

This adds `--delete` to the `aws s3 sync` commands to fix this issue.

Closes #396

Change-Id: I6d7fb97bcdab96c0115d6c66fea0310125207df4
(cherry picked from commit 16df34b295)
2025-03-17 19:09:10 +00:00
Jade Lovelace 32cbb66a69 release-notes: note the flake.lock fix
Change-Id: I1c5f7a5cba12da1bfb2896ffc04ec89862cb346d
2025-03-17 12:06:39 -07:00
giliceandjade 9631e9a30f libexpr/flake: user friendly error if parsing flake.lock fails
CHERRY-PICK: repeated bug reports

Fix: https://git.lix.systems/lix-project/lix/issues/559
Change-Id: I75f56c801d7f16a2e1ef8e702f16f1ef91f7b01f
(cherry picked from commit 68373f8664)
2025-03-17 19:01:23 +00:00
jadeandLix Systems Gerrit 98772c4a3b Merge "Revert "libstore: don't use curl decompression support"" into release-2.92 2025-03-17 18:17:41 +00:00
jadeandLix Systems Gerrit a7fd5c3867 Merge "libexpr: fix checkSourcePath purity regression" into release-2.92 2025-03-17 17:58:29 +00:00
jadeandLix Systems Gerrit 6f3a7bbeb0 Revert "libstore: don't use curl decompression support"
This reverts commit 0d1f794178.

Reason for revert: pennae says it makes the bug more visible, for some unknown reason. See https://git.lix.systems/lix-project/lix/issues/662

Change-Id: I91853cb6645f188a260c4c71cfff4de7bea99feb
2025-03-17 17:56:20 +00:00
eldritch horrorsandjade 0d1f794178 libstore: don't use curl decompression support
it's broken with http2 and transfer flow control. cf fj#662

BACKPORT: due to regression
Change-Id: Iaf6312bfcefa18d168faef47f57481199dd30b8d
2025-03-09 08:39:10 +00:00
Alois Wohlschlager ef5689dc1b libexpr: fix checkSourcePath purity regression
Starting with commit 0dbfa7b26e access would also
be allowed to ancestors of allowed paths. This is (ironically) a significant
purity regression, since several users of the purity checks will themselves
assume that arbitrary descent is allowed. For example, `builtins.readDir` and
`builtins.path` could now refer to the filesystem root, breaking purity
entirely in the latter case by allowing to read arbitrary files. Restore the
previous behaviour of only allowing access to explicitly allowed paths.

Change-Id: Ie64180733ab735da9873255e1ccbf95ba7c9161c
(cherry picked from commit 9d99a7c2cf)
2025-03-04 18:15:15 +01:00
Lunaphiedandjade db55ca9a2e hotfix for releng scripts, fixes #440
Change-Id: I33b29fbf3920a2d4fb53b58db477dff5bf7a1af1
2025-02-21 01:17:44 +00:00
eldritch horrorsandJade Lovelace 4cd618272a libstore: never return from lockFile without a lock
signals could cause lockFile to return without having locked the file.
the garbage collector didn't check for this, and then hilarity ensued.

Change-Id: If86d33595e8bf5510d2b032139342261dc6e07c9
(cherry picked from commit d186064c3d)
2025-02-09 12:54:48 -08:00
eldritch horrorsandJade Lovelace 73f0213500 libstore: split tryLockFile lockFile
lockFile is currently interruptible by signals like SIGCHLD. which
happen a lot in daemons. now imagine that daemon enabled automatic
garbage collection. observe that the local store does not actually
check whether its lock operations have succeeded ... get the idea?

Change-Id: Ibfd7ee786c4fee3add72d4456a7e95e73e09c73e
(cherry picked from commit 79f9c39e36)
2025-02-09 12:54:48 -08:00
eldritch horrorsandJade Lovelace 9c1db3cd8b libstore: extract unlockFile from lockFile
propagating the "unlock" lock type through this high-level api is
nonsense. it doesn't make sense to treat locking and unlocking as
similar operations; unlocking *must* not not interruptible by our
checkInterrupt machinery or it will just leave locks lying around
for a potentially very long time. unlock operations should not be
taking long enough to *want* them interrupted anyway. even on nfs
this makes very little sense because nfs waits *uninterruptibly*.

Change-Id: I10d605c8fe6c651bee64466eee1f8e20251d39f4
(cherry picked from commit aa87c8aa93)
2025-02-09 12:54:48 -08:00
Dusk Banksandeldritch horrors 5fa27057b2 libexpr: fix --debugger --ignore-try
a65e9e5828 did not inform `tryEval` that
(as far as it's concerned) `state.debug` moved to `state.errors.debug`
and changed types. this resulted in the REPL erroneously coming up, that
REPL having a non-debug state, and segfaulting after that REPL exited.

it's probably good that `state.debug` isn't mutated by `--ignore-try`
anymore.

Change-Id: I1918e93edacd626452aa423fc2eb825080738835
Fixes: a65e9e5828 ("libexpr: extract eval error creation into new type")
Signed-off-by: Dusk Banks <me@bb010g.com>
(cherry picked from commit 6a583136b7)
2025-01-31 12:36:14 +00:00
Dusk Banksandeldritch horrors d2b1af70ee doc: add bb010g to change-authors
Change-Id: Iddd3c21b2c42669cec394ac6b80f4e766e4cb81c
Signed-off-by: Dusk Banks <me@bb010g.com>
(cherry picked from commit 050cf17307)
2025-01-31 03:21:44 +00:00
Raito Bezariusandjade 50def3fa73 fix(gc): log sudden "path in use" exceptions and recover during GC gracefully
Original-Author: picnoir <picnoir@alternativebit.fr>

Inspired from
https://github.com/NixOS/nix/pull/11922/commits/ced8d311a593fcf9c3823e4e118474ac132d8e60
and adapted for Lix needs.

TL;DR: The topological sort should ensure that it is possible to delete
the path iterated upon. Nonetheless, in some cases,
`invalidatePathChecked` can still throw `PathInUse`, the exception
bubbles up and cancel the garbage collection procedure, leaving the rest
of the paths untouched. This change ensure that the error is logged for
further investigation but doesn't prevent the GC to continue when it
can.

After code review, we decided to make it a `printInfo` to inform the
user about sudden "in use" dependencies during garbage collection and
let them re-run garbage collection if they care about this.

References: https://github.com/NixOS/nix/issues/11923
References: https://git.lix.systems/lix-project/lix/issues/621
Change-Id: I5606c9afd16b5faa747b713fde2dc24016990ba3
Signed-off-by: Raito Bezarius <raito@lix.systems>
(cherry picked from commit 6a41dae49a)
2025-01-23 06:04:00 +00:00
eldritch horrorsandjade 8e2ab5532c FileTransfer: fix race condition on awaitData
There's a race condition where awaitData could early-return for data
coming from a 404 response or similar and thus not rethrow the exception
that is forthcoming, and a related race during transfer setup (which
could retry a transfer *twice* per retry round).

This would then cause substitution failures like below since the exception
isn't caught in HttpBinaryCacheStore::getFile as intended, but instead
by an exception handler downstream of `drain()` which would error out
the entire operation.

Symptom:

 » nix-build ./docs-service.nix -o "docs-service-result"
error: unable to download 'https://cache.nixos.org/7mr3fy8w66gi5inmf0jkkkl90lxy4jyg.narinfo': HTTP e
rror 404 ()

       response body:

This is kind of a hack in how it is implemented: it assumes that you
can't intentionally be receiving a large unsuccessful response since in
such a case, `awaitData` will wait for finish() to be called to throw an
exception and will never escape until the download finishes, while
continuing to buffer the entire response into memory, which could be bad
if an error response had a large payload.

That said, nobody is sending Lix 1GiB of 404, so meh I guess, and this
is how it is seemingly intended to work. That was a design flaw of the
thing before any of the Lix team got our paws on it.

I tested this by adding _exit(0) inside the expected exception catch and
then running the offending command repeatedly to see if the symptom ever
appeared again, and it did not.

Needs cherry-pick to 2.92 and a 2.92.1 release once reviewed.

Fixes: https://git.lix.systems/lix-project/lix/issues/635
Co-Authored-By: lix@jade.fyi
Change-Id: If54f6eeaad60b5ca9d5b77d4d9232da1d295e7d1
(cherry picked from commit de58cd6e80)
2025-01-22 21:06:58 +00:00
1474 changed files with 24576 additions and 49794 deletions
-4
View File
@@ -18,10 +18,6 @@ Checks:
- -bugprone-branch-clone
# extremely noisy before clang 19: https://github.com/llvm/llvm-project/issues/93959
- -bugprone-multi-level-implicit-pointer-conversion
# we don't compile out our asserts
- -bugprone-assert-side-effect
# FIXME(jade): figure out if this warning is any good
- -bugprone-exception-escape
# all thrown exceptions must derive from std::exception
- hicpp-exception-baseclass
# capturing async lambdas are dangerous
-4
View File
@@ -33,7 +33,3 @@ max_line_length = 0
[meson.build]
indent_style = space
indent_size = 2
[*.json]
indent_style = space
indent_size = 4
-2
View File
@@ -39,5 +39,3 @@ buildtime.bin
# Python compiled files from the code generators and test suite
*.pyc
**/.idea
Generated
+3 -3
View File
@@ -1,6 +1,6 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
version = 3
[[package]]
name = "countme"
@@ -47,9 +47,9 @@ checksum = "3fdb12b2476b595f9358c5161aa467c2438859caa136dec86c26fdd2efe17b92"
[[package]]
name = "rnix"
version = "0.12.0"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6f15e00b0ab43abd70d50b6f8cd021290028f9b7fdd7cdfa6c35997173bc1ba9"
checksum = "bb35cedbeb70e0ccabef2a31bcff0aebd114f19566086300b8f42c725fc2cb5f"
dependencies = [
"rowan",
]
-26
View File
@@ -1,26 +0,0 @@
# Docs
per-file README.md=*
per-file CONTRIBUTING.md=*
# DevX
per-file justfile=*
per-file .envrc=*
per-file .gitignore=*
per-file .github=*
per-file .mailmap=*
# Build
per-file meson.build=*
per-file meson.options=*
per-file flake.nix=*
per-file flake.lock=*
per-file *.nix=*
per-file Cargo.lock=*
per-file Cargo.toml=*
per-file version.json=*
# Code style
per-file .clang-tidy=*
per-file .clang-format=*
per-file .editorconfig=*
per-file treefmt.toml=*
+1 -4
View File
@@ -17,13 +17,10 @@ For systems that **already have a Nix implementation installed**, such as NixOS
## Building And Developing
See our [Hacking guide](https://git.lix.systems/lix-project/lix/src/branch/main/doc/manual/src/contributing/hacking.md) in our manual for instruction on how to set up a development environment and build Lix from source.
See our [Hacking guide](https://git.lix.systems/lix-project/lix/src/branch/main/doc/manual/src/contributing/hacking.md) in our manual for instruction on how to to set up a development environment and build Lix from source.
## Additional Resources
- The Lix reference manual:
- [Stable](https://docs.lix.systems/manual/lix/stable/)
- [Nightly](https://docs.lix.systems/manual/lix/nightly/) (NOTE: [not automatically updated, yet](https://git.lix.systems/lix-project/lix/issues/742))
- [Our wiki](https://wiki.lix.systems)
- [Matrix - #space:lix.systems](https://matrix.to/#/#space:lix.systems)
-1
View File
@@ -1,4 +1,3 @@
bench-*.json
bench-*.md
perf-*.json
nixpkgs
-1
View File
@@ -1 +0,0 @@
*
-237
View File
@@ -1,237 +0,0 @@
#!/usr/bin/env nix-shell
#!nix-shell -i python3 -p python3 -p hyperfine -p "if stdenv.isLinux then linuxPackages.perf else null"
import argparse
import subprocess
import os
import json
import tempfile
import platform
import shlex
import textwrap
import dataclasses
flake_args = ["--extra-experimental-features", "nix-command flakes"]
cases = {
"search": lambda build: [
f"{build}/bin/nix",
*flake_args,
"search",
"--no-eval-cache",
"github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870",
"hello",
],
"rebuild": lambda build: [
f"{build}/bin/nix",
*flake_args,
"eval",
"--raw",
"--impure",
"--expr",
textwrap.dedent("""
(import <nixpkgs/nixos> {
configuration = ./bench/nixpkgs/nixos/modules/installer/cd-dvd/installation-cd-graphical-calamares-plasma6.nix;
}).config.system.build.toplevel
""").replace("\n", " "),
],
"rebuild_lh": lambda build: [
"GC_INITIAL_HEAP_SIZE=10g",
*cases['rebuild'](build),
],
"parse": lambda build: [
f"{build}/bin/nix",
*flake_args,
"eval",
"-f",
"bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix",
],
}
arg_parser = argparse.ArgumentParser()
# FIXME(jade, gilice): it is a reasonable use case to want to run a benchmark run
# on just one build. However, since we are using hyperfine in comparison
# mode, we would have to combine the JSON ourselves to support that, which
# would probably be better done by writing a benchmarking script in
# not-bash.
arg_parser.add_argument(
'builds',
nargs='+',
help="At least two build directories to compare, containing bin/nix",
)
arg_parser.add_argument(
'--cases',
type=str,
help="A comma-separated list of cases you want to run. Defaults to running all",
)
arg_parser.add_argument(
'--mode',
nargs='+',
choices=[ "walltime", "memory" ] + [ "icount" ] if platform.system() == 'Linux' else [], # perf doesn't run on Darwin
default=[ "walltime" ],
)
arg_parser.add_argument(
'--daemon',
action='store_true',
help='Run a temporary daemon for the benchmark instead of using a local store directly',
)
args = arg_parser.parse_args()
if len(args.builds) < 1:
raise ValueError("need at least one build directory to benchmark")
benchmarks: list[str] = []
if args.cases is None:
benchmarks = list(cases.keys())
else:
for case in args.cases.split(","):
if case not in cases:
raise ValueError(f"no such case: {case}")
benchmarks.append(case)
def make_full_command(build, case):
cmd = " ".join(map(shlex.quote, cases[case](build)))
if args.daemon:
return " ".join([
f"{build}/bin/nix --extra-experimental-features nix-command daemon &",
"trap 'kill %1' EXIT;",
f"NIX_REMOTE=daemon {cmd}",
])
else:
return cmd
def bench_walltime(env):
for case in benchmarks:
for build in args.builds:
subprocess.run([
"taskset", "-c", "2,3",
"chrt", "-f","50",
*[
"hyperfine", "--warmup", "2", "--runs", "10",
"--export-json", f"bench/bench-{case}-{build}.json",
"--export-markdown", f"bench/bench-{case}-{build}.md",
"--", make_full_command(build, case),
],
], env=env, check=True)
print("Benchmarks summary\n---\n")
for case in benchmarks:
results = []
for build in args.builds:
with open(f"bench/bench-{case}-{build}.json") as fd:
results.append(json.load(fd)["results"][0])
for result in results:
print(result["command"])
print("-" * min(80,len(result["command"])))
def attr_rounded(attr):
return f"{result[attr]:.3f}"
print(" mean: ", attr_rounded("mean"), "±", attr_rounded("stddev"))
print(" user:", attr_rounded("user"), "| system", attr_rounded("system"))
print(" median: ", attr_rounded("median"))
print(" range: ", attr_rounded("min") + "s.." + attr_rounded("max")+"s")
print(" relative:", f"{result["mean"]/results[0]["mean"]:.3f}")
print("\n")
def bench_icount(env):
perf_results_for: dict[str, list[tuple[str, float]]] = {}
for case in benchmarks:
for build in args.builds:
# the perf stat -j output (incorrectly) localizes numbers, which will trip up the json parser.
env["LC_ALL"]="C"
case_command = make_full_command(build, case)
commandline = [
"perf", "stat", "-o", f"bench/perf-{case}.json", "-j",
"sh", "-c", case_command,
]
print("running", case_command)
subprocess.run(commandline, env=env, check=True, stdout=subprocess.DEVNULL) # warmup run
subprocess.run(commandline, env=env, check=True, stdout=subprocess.DEVNULL)
perf_fd = open(f"bench/perf-{case}.json")
perf_data = [json.loads(x) for x in perf_fd.readlines()]
perf_fd.close()
instr = next(x for x in perf_data if x["event"] in ["instructions", "instructions:u"]) # an implementation of a find_first iterator
if case not in perf_results_for:
perf_results_for[case] = []
perf_results_for[case].append((case_command, float(instr["counter-value"])))
print("Benchmarks summary\n---\n")
for (case, entries) in perf_results_for.items():
for entry in entries:
cmd,instr = entry
print(cmd)
print("-" * min(80,len(cmd)))
print(" instructions: ", int(instr))
print(" relative instructions:", int(instr)/perf_results_for[case][0][1])
print("\n")
@dataclasses.dataclass
class MemoryStatistics:
envBytes: int
listBytes: int
setBytes: int
valueBytes: int
heapBytes: int
heapSize: int
def bench_memory(env):
path = "bench/bench-memory.json"
env = env | {
'NIX_SHOW_STATS': '1',
'NIX_SHOW_STATS_PATH': path,
}
results: dict[str, list[tuple[str, MemoryStatistics]]] = {}
for case in benchmarks:
for build in args.builds:
case_command = make_full_command(build, case)
commandline = [ "sh", "-c", case_command ]
print("running", case_command)
subprocess.run(commandline, env=env, check=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
with open(path) as fd:
stats = json.load(fd)
results.setdefault(case, []).append((case_command, MemoryStatistics(
envBytes=stats['envs']['bytes'],
listBytes=stats['list']['bytes'],
setBytes=stats['sets']['bytes'],
valueBytes=stats['values']['bytes'],
heapSize=stats['gc']['heapSize'],
heapBytes=stats['gc']['totalBytes'],
)))
print("Benchmarks summary\n---\n")
for (case, entries) in results.items():
for cmd, stats in entries:
print(cmd)
print("-" * min(80, len(cmd)))
print(f" env bytes: {stats.envBytes :15d} | {(stats.envBytes / entries[0][1].envBytes) :.3f}x")
print(f" list bytes: {stats.listBytes :15d} | {(stats.listBytes / entries[0][1].listBytes) :.3f}x")
print(f" set bytes: {stats.setBytes :15d} | {(stats.setBytes / entries[0][1].setBytes) :.3f}x")
if not entries[0][1].valueBytes:
print(f" value bytes: {0:15d}")
else:
print(f" value bytes: {stats.valueBytes:15d} | {(stats.valueBytes / entries[0][1].valueBytes):.3f}x")
print(f" heap alloc'd: {stats.heapBytes :15d} | {(stats.heapBytes / entries[0][1].heapBytes) :.3f}x")
print(f" heap size: {stats.heapSize :15d} | {(stats.heapSize / entries[0][1].heapSize) :.3f}x")
print("\n")
with tempfile.TemporaryDirectory() as tmp_dir:
subprocess.run([
"nix", "build",
"--extra-experimental-features", "nix-command flakes",
"--impure", "--expr",'(builtins.getFlake "git+file:.").inputs.nixpkgs.outPath',
"-o","bench/nixpkgs"
], check=True)
subenv = os.environ.copy()
subenv["NIX_CONF_DIR"] = "/var/empty"
subenv["NIX_REMOTE"] = tmp_dir
subenv["NIX_PATH"] = ":".join([
"nixpkgs=bench/nixpkgs",
])
subenv["NIX_DAEMON_SOCKET_PATH"] = f"{tmp_dir}/daemon"
for mode in args.mode:
if mode == "walltime":
bench_walltime(subenv)
elif mode == "memory":
bench_memory(subenv)
else:
bench_icount(subenv)
Executable
+68
View File
@@ -0,0 +1,68 @@
#!/usr/bin/env nix-shell
#!nix-shell -i bash -p bash -p hyperfine
set -euo pipefail
shopt -s inherit_errexit
scriptdir=$(cd "$(dirname -- "$0")" ; pwd -P)
cd "$scriptdir/.."
if [[ $# -lt 2 ]]; then
# FIXME(jade): it is a reasonable use case to want to run a benchmark run
# on just one build. However, since we are using hyperfine in comparison
# mode, we would have to combine the JSON ourselves to support that, which
# would probably be better done by writing a benchmarking script in
# not-bash.
echo "Fewer than two result dirs given, nothing to compare!" >&2
echo "Pass some directories (with names indicating which alternative they are) with bin/nix in them" >&2
echo "Usage: ./bench/bench.sh result-1 result-2 [result-3...]" >&2
exit 1
fi
_exit=""
trap "$_exit" EXIT
flake_args=("--extra-experimental-features" "nix-command flakes")
# XXX: yes this is very silly. flakes~!!
nix build "${flake_args[@]}" --impure --expr '(builtins.getFlake "git+file:.").inputs.nixpkgs.outPath' -o bench/nixpkgs
# We must ignore the global config, or else NIX_PATH won't work reliably.
# See https://github.com/NixOS/nix/issues/9574
export NIX_CONF_DIR='/var/empty'
export NIX_REMOTE="$(mktemp -d)"
_exit='rm -rfv "$NIX_REMOTE"; $_exit'
export NIX_PATH="nixpkgs=bench/nixpkgs:nixos-config=bench/configuration.nix"
builds=("$@")
flake_args="${flake_args[*]@Q}"
hyperfineArgs=(
--parameter-list BUILD "$(IFS=,; echo "${builds[*]}")"
--warmup 2 --runs 10
)
declare -A cases
cases=(
[search]="{BUILD}/bin/nix $flake_args search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello"
[rebuild]="{BUILD}/bin/nix $flake_args eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'"
[rebuild-lh]="GC_INITIAL_HEAP_SIZE=10g {BUILD}/bin/nix eval $flake_args --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'"
[parse]="{BUILD}/bin/nix $flake_args eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix"
)
benches=(
rebuild
rebuild-lh
search
parse
)
for k in "${benches[@]}"; do
taskset -c 2,3 \
chrt -f 50 \
hyperfine "${hyperfineArgs[@]}" --export-json="bench/bench-${k}.json" --export-markdown="bench/bench-${k}.md" "${cases[$k]}"
done
echo "Benchmarks summary (from ./bench/summarize.jq bench/bench-*.json)"
bench/summarize.jq bench/*.json
+314
View File
@@ -0,0 +1,314 @@
{
config,
pkgs,
lib,
...
}:
{
boot = {
initrd = {
availableKernelModules = [
"xhci_pci"
"ahci"
];
kernelModules = [ "dm-snapshot" ];
luks.devices = {
croot = {
device = "/dev/sdb";
allowDiscards = true;
};
};
};
kernelModules = [ "kvm-intel" ];
kernelPackages = pkgs.linuxPackages_latest;
loader = {
systemd-boot.enable = true;
efi.canTouchEfiVariables = true;
};
};
hardware = {
enableRedistributableFirmware = true;
cpu.intel.updateMicrocode = true;
graphics.enable32Bit = true;
graphics.extraPackages = with pkgs; [
vaapiIntel
intel-media-driver
intel-compute-runtime
];
};
fileSystems = {
"/" = {
device = "/dev/sda2";
fsType = "xfs";
options = [ "noatime" ];
};
"/boot" = {
device = "/dev/sda1";
fsType = "vfat";
};
"/nas" = {
device = "nas:/";
fsType = "nfs4";
options = [
"ro"
"x-systemd.automount"
];
};
};
swapDevices = [ { device = "/dev/swap"; } ];
networking = {
useDHCP = false;
hostName = "host";
wireless = {
enable = true;
interfaces = [ "eth1" ];
};
interfaces = {
eth0.useDHCP = true;
eth1.useDHCP = true;
};
wg-quick.interfaces = {
wg0 = {
address = [ "2001:db8::1" ];
privateKeyFile = "/etc/secrets/wg0.key";
peers = [
{
publicKey = "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";
endpoint = "[2001:db8::2]:61021";
allowedIPs = [ "2001::db8:1::/64" ];
}
];
};
};
firewall.allowedUDPPorts = [ 4567 ];
};
i18n = {
defaultLocale = "en_US.UTF-8";
inputMethod.enable = true;
inputMethod.type = "ibus";
};
services = {
libinput.enable = true;
xserver = {
enable = true;
xkb.layout = "us";
xkb.variant = "altgr-intl";
xkb.options = "ctrl:nocaps";
wacom.enable = true;
videoDrivers = [ "modesetting" ];
modules = [ pkgs.xf86_input_wacom ];
displayManager.sx.enable = true;
windowManager.i3.enable = true;
};
udev.extraHwdb = ''
# not like this mattered at all
# we're not running udev from here
'';
udev.extraRules = ''
# ACTION=="add", SUBSYSTEM=="input", ...
'';
};
programs = {
light.enable = true;
wireshark = {
enable = true;
package = pkgs.wireshark-qt;
};
gnupg.agent = {
enable = true;
};
};
fonts.packages = with pkgs; [
font-awesome
noto-fonts
noto-fonts-cjk-sans
noto-fonts-emoji
noto-fonts-extra
dejavu_fonts
powerline-fonts
source-code-pro
cantarell-fonts
];
users = {
mutableUsers = false;
users = {
user = {
isNormalUser = true;
group = "user";
extraGroups = [
"wheel"
"video"
"audio"
"dialout"
"users"
"kvm"
"wireshark"
];
password = "unimportant";
};
};
groups = {
user = { };
};
};
security = {
pam.loginLimits = [
{
domain = "@audio";
item = "memlock";
type = "-";
value = "unlimited";
}
{
domain = "@audio";
item = "rtprio";
type = "-";
value = "99";
}
{
domain = "@audio";
item = "nofile";
type = "soft";
value = "99999";
}
{
domain = "@audio";
item = "nofile";
type = "hard";
value = "99999";
}
];
sudo.extraRules = [
{
users = [ "user" ];
commands = [
{
command = "${pkgs.linuxPackages.cpupower}/bin/cpupower";
options = [ "NOPASSWD" ];
}
];
}
];
};
environment.systemPackages = with pkgs; [
a2jmidid
age
ardour
bemenu
blender
breeze-icons
breeze-qt5
bubblewrap
calf
claws-mail
darktable
duperemove
emacs
feh
file
firefox
fluidsynth
adwaita-icon-theme
gnuplot
graphviz
helm
i3status-rust
inkscape
jack2
jq
krita
ldns
libqalculate
libreoffice
man-pages
nix-diff
nix-index
nix-output-monitor
open-music-kontrollers.patchmatrix
pamixer
pavucontrol
pciutils
picom
pwgen
redshift
ripgrep
rlwrap
silver-searcher
soundfont-fluid
whois
wol
xclip
xdot
xdotool
xorg.xkbcomp
yt-dlp
zathura
borgbackup
linuxPackages.cpupower
mtr
kitty
xf86_input_wacom
];
environment.pathsToLink = [ "/share/soundfonts" ];
systemd.user.services.run-python = {
after = [ "network-online.target" ];
script = ''
exec ${pkgs.python3}/bin/python
'';
serviceConfig = {
CapabilityBoundingSet = [ "" ];
KeyringMode = "private";
LockPersonality = true;
MemoryDenyWriteExecute = true;
NoNewPrivileges = true;
PrivateDevices = true;
PrivateTmp = true;
PrivateUsers = true;
ProcSubset = "pid";
ProtectClock = true;
ProtectControlGroups = true;
ProtectHome = true;
ProtectHostname = true;
ProtectKernelLogs = true;
ProtectKernelModules = true;
ProtectKernelTunables = true;
ProtectProc = "invisible";
ProtectSystem = "strict";
RestrictAddressFamilies = "AF_INET AF_INET6";
RestrictNamespaces = true;
RestrictRealtime = true;
RestrictSUIDSGID = true;
SystemCallArchitectures = "native";
SystemCallFilter = [
"@system-service"
"~ @resources @privileged"
];
UMask = "077";
};
};
system.stateVersion = "23.11";
}
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env -S jq -Mrf
def round3:
. * 1000 | round | . / 1000
;
def stats($first):
[
" mean: \(.mean | round3)s ± \(.stddev | round3)s",
" user: \(.user | round3)s | system: \(.system | round3)s",
" median: \(.median | round3)s",
" range: \(.min | round3)s ... \(.max | round3)s",
" relative: \(.mean / $first.mean | round3)"
]
| join("\n")
;
def fmt($first):
"\(.command)\n" + (. | stats($first))
;
[.results | .[0] as $first | .[] | fmt($first)] | join("\n\n") | (. + "\n\n---\n")
-1
View File
@@ -1 +0,0 @@
*
-1
View File
@@ -1 +0,0 @@
*
+3 -2
View File
@@ -7,8 +7,9 @@ create-missing = false
[output.html]
additional-css = ["custom.css"]
additional-js = ["redirects.js"]
# Jumps directly into a new Gerrit CL editing the file in question.
edit-url-template = "https://gerrit.lix.systems/admin/repos/edit/repo/lix/branch/main/file/doc/manual/{path}"
# Using our GitHub mirror enables easier typo fixes since there is no easy way
# to just submit a Gerrit CL by the web for trivial stuff.
edit-url-template = "https://github.com/lix-project/lix/tree/main/doc/manual/{path}"
git-repository-url = "https://git.lix.systems/lix-project/lix"
# Folding by default would prevent things like "Ctrl+F for nix-env" from working
# trivially, but the user should be able to fold if they want to.
-99
View File
@@ -12,10 +12,6 @@
forgejo: rbt
github: 9999years
9p4:
display_name: Ersei Saggi
github: 9p4
Artturin:
github: Artturin
@@ -40,10 +36,6 @@ alois31:
forgejo: alois31
github: alois31
andrewhamon:
display_name: Andrew Hamon
github: andrewhamon
artemist:
display_name: Artemis Tosini
forgejo: artemist
@@ -53,10 +45,6 @@ bb010g:
forgejo: bb010g
github: bb010g
blitz:
display_name: Julian Stecklina
github: blitz
cole-h:
display_name: Cole Helbling
github: cole-h
@@ -66,31 +54,14 @@ delan:
forgejo: delan
github: delan
delroth:
github: delroth
detroyejr:
display_name: Jonathan De Troye
github: detroyejr
edolstra:
display_name: Eelco Dolstra
github: edolstra
emilazy:
display_name: Emily
forgejo: emilazy
github: emilazy
ericson:
display_name: John Ericson
github: ericson2314
getchoo:
display_name: Seth Flynn
forgejo: getchoo
github: getchoo
gilice:
forgejo: gilice
@@ -111,9 +82,6 @@ ian-h-chamberlain:
forgejo: ian-h-chamberlain
github: ian-h-chamberlain
infinisil:
github: infinisil
isabelroses:
forgejo: isabelroses
github: isabelroses
@@ -125,16 +93,6 @@ jade:
just1602:
forgejo: just1602
kasimeka:
display_name: ورد
forgejo: janw4ld
github: kasimeka
kfears:
display_name: KFears
forgejo: kfearsoff
github: kfearsoff
kiara:
github: KiaraGrouwstra
@@ -150,10 +108,6 @@ lheckemann:
forgejo: lheckemann
github: lheckemann
lily:
forgejo: lilyinstarlight
github: lilyinstarlight
lilyball:
forgejo: lilyball
github: lilyball
@@ -168,51 +122,24 @@ ma27:
matthewbauer:
github: matthewbauer
mic92:
github: mic92
midnightveil:
display_name: julia
forgejo: midnightveil
github: midnightveil
nan-git:
display_name: NaN-git
github: NaN-git
ncfavier:
github: ncfavier
not-my-profile:
display_name: Martin Fischer
github: not-my-profile
p-e-meunier:
display_name: Pierre-Etienne Meunier
github: P-E-Meunier
pamplemousse:
display_name: Xavier Maso
github: pamplemousse
piegames:
display_name: piegames
forgejo: piegames
github: piegamesde
poliorcetics:
display_name: Poliorcetics
github: poliorcetics
puck:
display_name: puck
forgejo: puck
github: puckipedia
quantenzitrone:
display_name: Zitrone
forgejo: quantenzitrone
quantumjump:
display_name: Quantum Jump
github: QuantumBJump
@@ -229,18 +156,6 @@ roberth:
display_name: Robert Hensing
github: roberth
seppel3210:
github: Seppel3210
teofilc:
forgejo: teofilc
github: TeofilC
thubrecht:
display_name: Tom Hubrecht
forgejo: tom-hubrecht
github: Tom-Hubrecht
thufschmitt:
display_name: Théophane Hufschmitt
github: thufschmitt
@@ -258,24 +173,10 @@ vigress8:
forgejo: vigress8
github: vigress8
vlinkz:
display_name: Victor Fuentes
forgejo: vlinkz
github: vlinkz
winter:
forgejo: winter
github: winterqt
xanderio:
github: xanderio
xokdvium:
github: xokdvium
yorickvp:
github: yorickvp
yshui:
github: yshui
+1 -1
View File
@@ -225,7 +225,7 @@ let
showCategory = cat: ''
${optionalString (cat != "") "**${cat}:**"}
${listOptions (filterAttrs (n: v: v.category == cat && !v.hidden) allOptions)}
${listOptions (filterAttrs (n: v: v.category == cat) allOptions)}
'';
listOptions = opts: concatStringsSep "\n" (attrValues (mapAttrs showOption opts));
showOption =
-1
View File
@@ -1,7 +1,6 @@
nix_env_for_docs = {
'HOME': '/dummy',
'NIX_CONF_DIR': '/dummy',
'XDG_CONFIG_HOME': '/dummy',
'NIX_SSL_CERT_FILE': '/dummy/no-ca-bundle.crt',
'NIX_STATE_DIR': '/dummy',
'NIX_CONFIG': 'cores = 0',
+1 -5
View File
@@ -20,6 +20,7 @@
- [Basic Package Management](package-management/basic-package-mgmt.md)
- [Profiles](package-management/profiles.md)
- [Garbage Collection](package-management/garbage-collection.md)
- [Garbage Collector Roots](package-management/garbage-collector-roots.md)
- [Sharing Packages Between Machines](package-management/sharing-packages.md)
- [Serving a Nix store via HTTP](package-management/binary-cache-substituter.md)
- [Copying Closures via SSH](package-management/copy-closure.md)
@@ -39,9 +40,6 @@
- [Tuning Cores and Jobs](advanced-topics/cores-vs-jobs.md)
- [Verifying Build Reproducibility](advanced-topics/diff-hook.md)
- [Using the `post-build-hook`](advanced-topics/post-build-hook.md)
- [Pasta](advanced-topics/pasta.md)
- [Known Issues](known-issues/known-issues.md)
- [Limitations around non-isolated builds](known-issues/non-isolated-build-limits.md)
- [Command Reference](command-ref/command-ref.md)
- [Common Options](command-ref/opt-common.md)
- [Common Environment Variables](command-ref/env-common.md)
@@ -200,8 +198,6 @@
- [Release Notes](release-notes/release-notes.md)
- [Upcoming release](release-notes/rl-next.md)
<!-- RELENG-AUTO-INSERTION-MARKER (see releng/release_notes.py) -->
- [Lix 2.94 (2025-11-17)](release-notes/rl-2.94.md)
- [Lix 2.93 (2025-05-09)](release-notes/rl-2.93.md)
- [Lix 2.92 (2025-01-18)](release-notes/rl-2.92.md)
- [Lix 2.91 (2024-08-12)](release-notes/rl-2.91.md)
- [Lix 2.90 (2024-07-10)](release-notes/rl-2.90.md)
@@ -75,9 +75,7 @@ by spaces. Only the first element is required. To leave a field at its
default, set it to `-`.
1. The URI of the remote store in the format
`ssh://[username@]hostname[?port=<port>]`, e.g. `ssh://nix@mac` or `ssh://mac`.
If the ssh server is not listening on port 22 (e.g. port 1337 in this case)
the URI would be `ssh://nix@mac?port=1337`
`ssh://[username@]hostname`, e.g. `ssh://nix@mac` or `ssh://mac`.
For backward compatibility, `ssh://` may be omitted. The hostname
may be an alias defined in your `~/.ssh/config`.
-19
View File
@@ -1,19 +0,0 @@
# [Pasta](https://passt.top/passt/about/): a network sandbox for fixed-output derivations
## Introduction
This section only applies to **Linux systems** as Pasta is a Linux-only measure.
Since [CVE-2025-46416](https://lix.systems/blog/2025-06-24-lix-cves/), the Lix project decided to adopt [Pasta](https://passt.top/passt/about/) for all fixed-output derivations, protecting against various attack vectors such as UNIX abstract domain sockets or more manipulation at the network layer from a malicious fixed-output derivation code.
Pasta acts as a translation layer between a layer-2 network interface and layer-4 sockets (TCP, UDP, ICMP/ICMPv6 echo) on the host. It requires no special privileges and can serve as a alternative to [SLiRP](https://en.wikipedia.org/wiki/Slirp) which was used [by Guix to mitigate the same problem](https://codeberg.org/guix/guix/commit/fb42611b8f27960304db5a1c0d33b8371dcde2a8).
## How to disable Pasta?
It's sufficient to pass `pasta-path = ""` in your `/etc/nix/nix.conf` or on the command line `--pasta-path ""` of a Lix invocation.
## Known issues surrounding Pasta
- Only the first DNS server in `/etc/resolv.conf` is considered: failover is not possible.
- [Reduced feature set compared to the Linux kernel](https://passt.top/passt/about/#features)
- [Performance overhead in multi-gigabits contexts and IMIX MTUs](https://passt.top/passt/about/#performance_1)
+3 -3
View File
@@ -22,9 +22,9 @@ The following [concept map] shows its main components (rectangles), the objects
| |
+----------|-------------------|--------------------------------+
| Nix impl.| V |
| (Lix) | +------------------------+ |
| | | command line interface |------. |
| | +------------------------+ | |
| (Lix) | +-------------------------+ |
| | | commmand line interface |------. |
| | +-------------------------+ | |
| | | | |
| evaluated by calls manages |
| | | | |
@@ -1,12 +0,0 @@
<!--
File-ish argument syntax summary.
This file gets included into pages like nix-build.md and nix-instantiate.md, and each individual page that includes
this also links to nix-build.md for the full explanation.
-->
- A normal filesystem path, like `/home/meow/nixfiles/default.nix`
- Or a directory, like `/home/meow/nixfiles`, equivalent to above
- A single lookup path, like `<nixpkgs>` or `<nixos>`
- A URL to a tarball, like `https://github.com/NixOS/nixpkgs/archive/refs/heads/release-23.11.tar.gz`
- A [flakeref](@docroot@/command-ref/new-cli/nix3-flake.md#flake-references), introduced by the prefix `flake:`, like `flake:git+https://git.lix.systems/lix-project/lix`
- A *nixpkgs* channel tarball name, introduced by the prefix `channel:`, like `channel:nixos-unstable`.
- This uses a hard-coded URL pattern and is *not* related to the subscribed channels managed by the [nix-channel](@docroot@/command-ref/nix-channel.md) command.
+7 -43
View File
@@ -4,7 +4,7 @@
# Synopsis
`nix-build` [*fileish…*]
`nix-build` [*paths…*]
[`--arg` *name* *value*]
[`--argstr` *name* *value*]
[{`--attr` | `-A`} *attrPath*]
@@ -20,55 +20,19 @@ For documentation on the latter, run `nix build --help` or see `man nix3-build`.
# Description
The `nix-build` command builds the derivations described by the Nix
expressions in each *fileish*. If the build succeeds, it places a symlink to
expressions in *paths*. If the build succeeds, it places a symlink to
the result in the current directory. The symlink is called `result`. If
there are multiple Nix expressions, or the Nix expressions evaluate to
multiple derivations, multiple sequentially numbered symlinks are
created (`result`, `result-2`, and so on).
If no *fileish* is specified, then `nix-build` will use `default.nix` in
If no *paths* are specified, then `nix-build` will use `default.nix` in
the current directory, if it exists.
## Fileish Syntax
A given *fileish* may take one of a few different forms, the first being a simple filesystem path, e.g. `nix-build /tmp/some-file.nix`.
Like the [import builtin](../language/builtins.md#builtins-import) specifying a directory is equivalent to specifying `default.nix` within that directory.
It may also be a [search path](./env-common.md#env-NIX_PATH) (also known as a lookup path) like `<nixpkgs>`, which is convenient to use with `--attr`/`-A`:
```console
$ nix-build '<nixpkgs>' -A firefox
```
(Note the quotation marks around `<nixpkgs>`, which will be necessary in most Unix shells.)
If a *fileish* starts with `http://` or `https://`, it is interpreted as the URL of a tarball which will be fetched and unpacked.
Lix will then `import` the unpacked directory, so these tarballs must include at least a single top-level directory with a file called `default.nix`
For example, you could build from a specific version of Nixpkgs with something like:
```console
$ nix-build "https://github.com/NixOS/nixpkgs/archive/refs/heads/release-23.11.tar.gz" -A firefox
```
If a path starts with `flake:`, the rest of the argument is interpreted as a [flakeref](./new-cli/nix3-flake.md#flake-references) (see `nix flake --help` or `man nix3-flake`), which requires the "flakes" experimental feature to be enabled.
Lix will fetch the flake, and then `import` its unpacked directory, so the flake must include a file called `default.nix`.
For example, the flake analogues to the above `nix-build` commands are:
```console
$ nix-build flake:nixpkgs -A firefox
$ nix-build flake:github:NixOS/nixpkgs/release-23.11 -A firefox
```
Finally, for legacy reasons, if a path starts with `channel:`, the rest of the argument is interpreted as the name of a *nixpkgs* channel tarball to fetch from `https://channels.nixos.org/$CHANNEL_NAME/nixexprs.tar.xz`.
This is a **hard coded URL** pattern and is *not* related to the subscribed channels managed by the [nix-channel](./nix-channel.md) command.
> **Note**: any of the special syntaxes may always be disambiguated by prefixing the path.
> For example: a file in the current directory literally called `<nixpkgs>` can be addressed as `./<nixpkgs>`, to escape the special interpretation.
In summary, a path argument may be one of:
{{#include ./fileish-summary.md}}
## Notes
If an element of *paths* starts with `http://` or `https://`, it is
interpreted as the URL of a tarball that will be downloaded and unpacked
to a temporary location. The tarball must include a single top-level
directory containing at least a file named `default.nix`.
`nix-build` is essentially a wrapper around
[`nix-instantiate`](nix-instantiate.md) (to translate a high-level Nix
+4 -4
View File
@@ -11,10 +11,10 @@
Channels are a mechanism for referencing remote Nix expressions and conveniently retrieving their latest version.
The moving parts of channels are:
- The official channels listed at <https://channels.nixos.org>
- The official channels listed at <https://nixos.org/channels>
- The user-specific list of [subscribed channels](#subscribed-channels)
- The [downloaded channel contents](#channels)
- The [Nix expression search path](@docroot@/command-ref/conf-file.md#conf-nix-path), set with the [`-I` option](#opt-I) or the [`NIX_PATH` environment variable](#env-NIX_PATH)
- The [Nix expression search path](@docroot@/command-ref/conf-file.md#conf-nix-path), set with the [`-I` option](#opt-i) or the [`NIX_PATH` environment variable](#env-NIX_PATH)
> **Note**
>
@@ -77,9 +77,9 @@ This command has the following operations:
Subscribe to the Nixpkgs channel and run `hello` from the GNU Hello package:
```console
$ nix-channel --add https://channels.nixos.org/nixpkgs-unstable
$ nix-channel --add https://nixos.org/channels/nixpkgs-unstable
$ nix-channel --list
nixpkgs https://channels.nixos.org/nixpkgs
nixpkgs https://nixos.org/channels/nixpkgs
$ nix-channel --update
$ nix-shell -p hello --run hello
hello
@@ -36,7 +36,7 @@ Instead, it looks in a few locations, and acts on all profiles it finds there:
>
> Not stable; subject to change
>
> Do not rely on this functionality; it just exists for migration purposes and may change in the future.
> Do not rely on this functionality; it just exists for migration purposes and is may change in the future.
> These deprecated paths remain a private implementation detail of Lix.
<!-- FIXME(Qyriad): this is inconsistent with https://git.lix.systems/lix-project/lix/issues/215, needs updating when that happens -->
+1 -1
View File
@@ -8,7 +8,7 @@
[`--option` *name* *value*]
[`--arg` *name* *value*]
[`--argstr` *name* *value*]
[{`--file` | `-f`} *fileish*]
[{`--file` | `-f`} *path*]
[{`--profile` | `-p`} *path*]
[`--system-filter` *system*]
[`--dry-run`]
@@ -26,7 +26,7 @@ This operation deletes the specified generations of the current profile.
>
> Older *and newer* generations will be deleted by this operation.
>
> One might expect this to just delete older generations than the current one, but that is only true if the current generation is also the latest.
> One might expect this to just delete older generations than the curent one, but that is only true if the current generation is also the latest.
> Because one can roll back to a previous generation, it is possible to have generations newer than the current one.
> They will also be deleted.
@@ -11,7 +11,6 @@
[`--from-profile` *path*]
[`--preserve-installed` | `-P`]
[`--remove-all` | `-r`]
[`--priority` *priority*]
# Description
@@ -60,11 +59,6 @@ a number of possible ways:
unambiguous way, which is necessary if there are multiple
derivations with the same name.
- If `--priority` *priority* is given, the priority of the derivations being
installed is set to *priority*. This can be used to override the priority of
the derivations being installed. This is useful if *args* are store paths,
which don't have any priority information.
- If *args* are [store derivations](@docroot@/glossary.md#gloss-store-derivation), then these are
[realised](@docroot@/command-ref/nix-store/realise.md), and the resulting output paths
are installed.
@@ -148,8 +142,8 @@ To copy the store path with symbolic name `gcc` from another profile:
$ nix-env --install --from-profile /nix/var/nix/profiles/foo gcc
```
To install a specific [store derivation](@docroot@/glossary.md#gloss-store-derivation)
(typically created by `nix-instantiate`):
To install a specific [store derivation] (typically created by
`nix-instantiate`):
```console
$ nix-env --install /nix/store/fibjb1bfbpm5mrsxc4mh2d8n37sxh91i-gcc-3.4.3.drv
@@ -2,16 +2,16 @@
The following options are allowed for all `nix-env` operations, but may not always have an effect.
- `--file` / `-f` *fileish*\
- `--file` / `-f` *path*\
Specifies the Nix expression (designated below as the *active Nix
expression*) used by the `--install`, `--upgrade`, and `--query
--available` operations to obtain derivations. The default is
`~/.nix-defexpr`.
*fileish* is interpreted the same as with [nix-build](../nix-build.md#fileish-syntax).
See that section for complete details (`nix-build --help`), but in summary, a path argument may be one of:
{{#include ../fileish-summary.md}}
If the argument starts with `http://` or `https://`, it is
interpreted as the URL of a tarball that will be downloaded and
unpacked to a temporary location. The tarball must include a single
top-level directory containing at least a file named `default.nix`.
- `--profile` / `-p` *path*\
Specifies the profile to be used by those operations that operate on
@@ -22,7 +22,7 @@ left untouched; this is not an error. It is also not an error if an
element of *args* matches no installed derivations.
For a description of how *args* is mapped to a set of store paths, see
[`--install`](install.md). If *args* describes multiple
[`--install`](#operation---install). If *args* describes multiple
store paths with the same symbolic name, only the one with the highest
version is installed.
+9 -31
View File
@@ -5,13 +5,13 @@
# Synopsis
`nix-instantiate`
[`--parse` | `--eval` [`--strict`] [`--raw`] [`--json`] [`--xml`] ]
[`--parse` | `--eval` [`--strict`] [`--json`] [`--xml`] ]
[`--read-write-mode`]
[`--arg` *name* *value*]
[{`--attr`| `-A`} *attrPath*]
[`--add-root` *path*]
[`--expr` | `-E`]
*fileish*
*files*
`nix-instantiate` `--find-file` *files…*
@@ -25,11 +25,8 @@ of the resulting store derivations are printed on standard output.
[store derivation]: ../glossary.md#gloss-store-derivation
If *fileish* is the character `-`, then a Nix expression will be read from standard input.
Otherwise, each *fileish* is interpreted the same as with [nix-build](./nix-build.md#fileish-syntax).
See that section for complete details (`nix-build --help`), but in summary, a path argument may be one of:
{{#include ./fileish-summary.md}}
If *files* is the character `-`, then a Nix expression will be read from
standard input.
# Options
@@ -38,14 +35,7 @@ See that section for complete details (`nix-build --help`), but in summary, a pa
- `--parse`\
Just parse the input files, and print their abstract syntax trees on
standard output. The output format of the AST depends on the current
internal representation and may change in the future.
Tooling can use the stderr and exit code of `--parse` to check any
Nix code for correctness, but should not rely on stdout without careful
versioning. Note that `--parse` also checks for unbound variables.
In cases where this is undesired, `with {};` can be prepended
to the program to transform all such parse errors into eval errors.
standard output as a Nix expression.
- `--eval`\
Just parse and evaluate the input files, and print the resulting
@@ -107,27 +97,15 @@ See that section for complete details (`nix-build --help`), but in summary, a pa
> This option can cause non-termination, because lazy data
> structures can be infinitely large.
- `--raw`\
When used with `--eval`, the result must be coercible to a string, i.e.,
something that can be converted using `${...}`.
Integers will always generate an error when output via `--raw`, regardless of
[`coerce-integers`](../contributing/experimental-features.md#xp-feature-coerce-integers) being enabled, to avoid ambiguity.
The output is printed exactly as-is, with no quotes, escaping, or trailing
newline.
- `--json`\
When used with `--eval`, print the resulting value as an JSON
representation of the resulting value rather than as a Nix expression.
The conversion behaviour, if `--strict` is passed, is the same as
[`builtins.toJSON`](../language/builtins.md#builtins-toJSON).
representation of the abstract syntax tree rather than as a Nix expression.
- `--xml`\
When used with `--eval`, print the resulting value as an XML
representation of the resulting value rather than as a Nix expression.
The schema is the same as that used by [`builtins.toXML`](../language/builtins.md#builtins-toXML).
representation of the abstract syntax tree rather than as a Nix expression.
The schema is the same as that used by the [`toXML`
built-in](../language/builtins.md).
- `--read-write-mode`\
When used with `--eval`, perform evaluation in read/write mode so
+4 -3
View File
@@ -33,9 +33,10 @@ the environment of a derivation for development.
If *path* is not given, `nix-shell` defaults to `shell.nix` if it
exists, and `default.nix` otherwise.
If *path* is given it is interpreted like a [*fileish* argument to nix-build](./nix-build.md#fileish-syntax):
{{#include ./fileish-summary.md}}
If *path* starts with `http://` or `https://`, it is interpreted as the
URL of a tarball that will be downloaded and unpacked to a temporary
location. The tarball must include a single top-level directory
containing at least a file named `default.nix`.
If the derivation defines the variable `shellHook`, it will be run
after `$stdenv/setup` has been sourced. Since this hook is not executed
@@ -4,7 +4,7 @@
# Synopsis
`nix-store` `--delete` [`--ignore-liveness`] [`--skip-live`] [`--delete-closure`] *paths…*
`nix-store` `--delete` [`--ignore-liveness`] *paths…*
# Description
@@ -18,13 +18,6 @@ With the option `--ignore-liveness`, reachability from the roots is
ignored. However, the path still wont be deleted if there are other
paths in the store that refer to it (i.e., depend on it).
This operation will raise an error if any of the paths are still live
and `--ignore-liveness` is not passed. Passing `--skip-live` will
prevent this from being considered an error.
The option `--delete-closure` will also attempt to delete any paths
that are in the given path's dependency closure.
{{#include ./opt-common.md}}
{{#include ../opt-common.md}}
@@ -93,12 +93,9 @@ symlink.
[deriver]: ../../glossary.md#gloss-deriver
- `--valid-derivers`\
Prints the set of all [derivers](../../glossary.md#gloss-deriver) that can be
used to build the store paths *paths*.
This differs from `--deriver`, which prints the deriver that actually
produced *paths*.
No deriver may be returned if is not present in the store,
eg, if *paths* were substituted from a binary cache.
Prints a set of derivation files (`.drv`) which are supposed produce
said paths when realized. Might print nothing, for example for source paths
or paths subsituted from a binary cache.
- `--graph`\
Prints the references graph of the store paths *paths* in the format
@@ -15,6 +15,7 @@ Each of *paths* is processed as follows:
1. If it is not [valid], substitute the store derivation file itself.
2. Realise its [output paths]:
- Try to fetch from [substituters] the [store objects] associated with the output paths in the store derivation's [closure].
- With [content-addressed derivations] (experimental): Determine the output paths to realise by querying content-addressed realisation entries in the [Nix database].
- For any store paths that cannot be substituted, produce the required store objects. This involves first realising all outputs of the derivation's dependencies and then running the derivation's [`builder`](@docroot@/language/derivations.md#attr-builder) executable. <!-- TODO: Link to build process page #8888 -->
- Otherwise, and if the path is not already valid: Try to fetch the associated [store objects] in the path's [closure] from [substituters].
@@ -27,6 +28,7 @@ If no substitutes are available and no store derivation is given, realisation fa
[store objects]: @docroot@/glossary.md#gloss-store-object
[closure]: @docroot@/glossary.md#gloss-closure
[substituters]: @docroot@/command-ref/conf-file.md#conf-substituters
[content-addressed derivations]: @docroot@/contributing/experimental-features.md#xp-feature-ca-derivations
[Nix database]: @docroot@/glossary.md#gloss-nix-database
The resulting paths are printed on standard output.
+1 -1
View File
@@ -85,7 +85,7 @@ Most commands in Lix accept the following command-line options:
- `multiline-with-logs`
Display the raw logs, with a progress bar and activities each in a new line at the bottom.
Displayes the raw logs, with a progress bar and activities each in a new line at the bottom.
- <span id="opt-no-build-output">[`--no-build-output`](#opt-no-build-output)</span> / `-Q`
+2 -2
View File
@@ -661,8 +661,8 @@ Verbosity levels are:
The default level that the command starts is `ERROR`. The simplest way to
increase the verbosity by stacking `-v` option (eg: `-vvv == level 3 == INFO`).
Use `--quiet` to decrease verbosity by one level.
There is one shortcut, `--debug` to run in `DEBUG` verbosity level.
There are also two shortcuts, `--debug` to run in `DEBUG` verbosity level and
`--quiet` to run in `ERROR` verbosity level.
----------
+31 -97
View File
@@ -11,19 +11,7 @@ The following instructions assume you already have some version of Nix or Lix in
[installation instructions]: ../installation/installation.md
A typical development flow for simple changes in Lix looks like:
- [Set up and build Lix](#building)
- For large changes, check in regarding design and possibly create an RFD issue on Forgejo
- Make the changes in your editor
- [Send the changes to Gerrit](#sending-to-gerrit)
- Once you have the number for the CL from Gerrit to put in the changelog, [write a changelog entry](#release-notes) and amend it into the commit
- Update the Gerrit change by submitting it with the same command as the first time
- Request and receive a code review
- Address feedback from the review
- Amend commits, send to Gerrit again
- Submit the approved change
## Building Lix in a development shell {#building}
## Building Lix in a development shell
### Setting up the development shell
@@ -51,28 +39,17 @@ $ nix-shell -A native-clangStdenvPackages
### Building from the development shell
Run a clean build and test with `just clean setup build install test`.
You can also run the unit tests and integration tests separately:
You can build and test Lix with just:
```bash
$ just setup build test-unit
$ just install test-integration
$ just setup
$ just build
$ just test --suite=check
$ just install
$ just test --suite=installcheck
```
Many justfile aliases have a `-custom` variant which pass extra arguments to `meson`.
For example, to work on both Lix and nix-eval-jobs you can run:
```
$ just setup-custom -Dnix-eval-jobs=enabled
$ # or
$ mesonFlags=-Dnix-eval-jobs=enabled just setup
```
Note that only targets which don't accept extra arguments can be used when
running multiple targets at once; `just setup build` is fine, but `just
setup-custom build` is an error. The `test` target is usually the last one to
run, so it always accepts extra arguments.
(Check and installcheck may both be done after install, allowing you to omit the --suite argument entirely, but this is the order package.nix runs them in.)
You can also build Lix manually:
@@ -141,44 +118,7 @@ To inspect the canonical source of truth on what the state of the buildsystem co
$ meson introspect
```
## Sending changes to Gerrit for review {#sending-to-gerrit}
We use Gerrit for all our code review in Lix.
Our instance is at <https://gerrit.lix.systems>.
There's much more information about how to use Gerrit in the [wiki section on Gerrit][wiki-gerrit] including how to use Jujutsu, how to use the UI and more.
The Snix project also has some Gerrit information [in their contributing docs][snix-gerrit].
[wiki-gerrit]: https://wiki.lix.systems/books/lix-contributors/chapter/gerrit
[snix-gerrit]: https://snix.dev/docs/guides/contributing/
The gist is that once you have your SSH key and git remote set up, you can send commits for review with:
```
$ git remote set-url origin ssh://YOURUSERNAME@gerrit.lix.systems:2022/lix
$ git push origin HEAD:refs/for/main
```
Then, you can request a review via the "Reply" button on the web UI.
If you click "Suggest Owners", it will try to suggest the maintainers of the area of the code change to send review requests to.
Requesting reviews from multiple people is normal.
We do our best to respond to directly sent reviews in a few days, so feel free to request another reviewer or ask on Matrix if you've not got a response for a while.
Keep in mind that Lix is a volunteer project and we have limited bandwidth, so some changes aren't feasible to shepherd through; please check in on Matrix at design time when doing large changes.
Once you get a `Code-Review+2` vote on your change, it's rebased on `main` and CI marks it `Verified+1`, you're able (and usually expected, so you can have a second chance to check it over) to hit the Submit button to merge it.
If the change appears as "Rebase Required", you need to rebase it on `main` locally or via the Gerrit UI and wait for `Verified+1` before the Submit button is made active
The `Code-Review+2` from before will stick around through trivial rebases so no need to re-request review for a mere rebase.
## Interacting with the CI, Buildkite
We use Buildkite for our CI, usually you will not have to interact directly with it other than reviewing any errors it produces, which are linked from Gerrit.
However in certain cases a CI run will fail due to transient issues not related to your code and you will need to rerun it by hand.
You can log in to the CI via [SSO](https://buildkite.com/sso/lix-project). On your job you can then hit the "Retry failed" button to rerun it, normally you will not have a repeat of the transient issue.
If the build still fails on CI issues or all builds are failing this should be reported via [Zulip on #T-infra](https://zulip.lix.systems/#narrow/channel/7-T-infra) or [Matrix on #dev](https://matrix.to/#/%23dev%3Alix.systems?via=lix.systems).
## Building Lix with `nix`
## Building Lix outside of development shells
To build a release version of Lix for the current operating system and CPU architecture:
@@ -197,11 +137,10 @@ You can also build Lix for one of the [supported platforms](#platforms).
Lix can be built for various platforms, as specified in [`flake.nix`]:
[`flake.nix`]: https://git.lix.systems/lix-project/lix/src/branch/main/flake.nix
[`flake.nix`]: https://git.lix.systems/lix-project/lix/lix/branch/main/flake.nix
- `x86_64-linux`
- `x86_64-darwin`
- `x86_64-freebsd`
- `i686-linux`
- `aarch64-linux`
- `aarch64-darwin`
@@ -278,7 +217,7 @@ Lix uses a string with the following format to identify the *system type* or *pl
It is set when Lix is compiled for the given system, and determined by [Meson's `host_machine.cpu_family()` and `host_machine.system()` values](https://mesonbuild.com/Reference-manual_builtin_host_machine.html).
For historic reasons and backward-compatibility, some CPU and OS identifiers are translated from the GNU Autotools naming convention in [`meson.build`](https://git.lix.systems/lix-project/lix/src/branch/main/meson.build) as follows:
For historic reasons and backward-compatibility, some CPU and OS identifiers are translated from the GNU Autotools naming convention in [`meson.build`](https://git.lix.systems/lix-project/lix/blob/main/meson.build) as follows:
| `host_machine.cpu_family()` | Nix |
|----------------------------|---------------------|
@@ -305,13 +244,13 @@ To build with one of those environments, you can use
$ nix build .#nix-ccacheStdenv
```
for <a id="nix-with-flakes">flake-enabled Nix</a>, or
for flake-enabled Nix, or
```console
$ nix-build --attr nix-ccacheStdenv
```
for <a id="classic-nix">classic Nix</a>.
for classic Nix.
You can use any of the other supported environments in place of `nix-ccacheStdenv`.
@@ -335,10 +274,10 @@ Configure your editor to use the `clangd` from the shell, either by running it i
> Some other editors (e.g. Emacs, Vim) need a plugin to support LSP servers in general (e.g. [lsp-mode](https://github.com/emacs-lsp/lsp-mode) for Emacs and [vim-lsp](https://github.com/prabirshrestha/vim-lsp) for vim).
> Editor-specific setup is typically opinionated, so we will not cover it here in more detail.
# Manual and documentation
## Building the manual
### Checking links in the manual
The build checks for broken internal links.
This happens late in the process, so `nix build` is not suitable for iterating.
To build the manual incrementally, run:
```console
@@ -350,20 +289,15 @@ meson compile -C build manual
[`mdbook-linkcheck`]: https://github.com/Michael-F-Bryan/mdbook-linkcheck
[URI fragments]: https://en.wikipedia.org/wiki/URI_fragment
The built manual is in `build/doc/manual/manual/index.html`.
#### `@docroot@` variable
The build checks for broken internal links.
This happens late in the process, so `nix build` is not suitable for iterating and it's recommended to use the `meson` command above instead.
`@docroot@` provides a base path for links that occur in reusable snippets or other documentation that doesn't have a base path of its own.
### `@\docroot\@` variable
If a broken link occurs in a snippet that was inserted into multiple generated files in different directories, use `@docroot@` to reference the `doc/manual/src` directory.
`@\docroot\@` provides a base path for links that occur in reusable snippets or other documentation that doesn't have a base path of its own.
If a broken link occurs in a snippet that was inserted into multiple generated files in different directories, use `@\docroot\@` to reference the `doc/manual/src` directory.
If the `@\docroot\@` literal appears in an error message from the `mdbook-linkcheck` tool, the `@\docroot\@` replacement needs to be applied to the generated source file that mentions it.
See existing `@\docroot\@` logic in `doc/manual/substitute.py`.
Regular markdown files used for the manual have a base path of their own and they can use relative paths instead of `@\docroot\@`.
If the `@docroot@` literal appears in an error message from the `mdbook-linkcheck` tool, the `@docroot@` replacement needs to be applied to the generated source file that mentions it.
See existing `@docroot@` logic in the [Makefile].
Regular markdown files used for the manual have a base path of their own and they can use relative paths instead of `@docroot@`.
## API documentation
@@ -395,7 +329,7 @@ You can build it yourself:
Metrics about the change in line/function coverage over time will be available in the future (FIXME(lix-hydra)).
## Add a release note {#release-notes}
## Add a release note
`doc/manual/rl-next` contains release notes entries for all unreleased changes.
@@ -464,15 +398,15 @@ The following properties are supported:
### Build process
Releases have a precomputed `rl-MAJOR.MINOR.md`, and no `rl-next.md`.
Development releases have a generated `rl-next.md`.
Set `buildUnreleasedNotes = true;` in `flake.nix` to build the release notes on the fly.
# Adding experimental or deprecated features, global settings, or builtins
## Adding experimental or deprecated features, global settings, or builtins
Experimental and deprecated features, global settings, and builtins are generally referenced both in the code and in the documentation.
To prevent duplication or divergence, they are defined in data files, and a script generates the necessary glue.
The data file format is similar to the release notes: it consists of a YAML metadata header, followed by the documentation in Markdown format.
## Experimental or deprecated features
### Experimental or deprecated features
Experimental and deprecated features support the following metadata properties:
* `name` (required): user-facing name of the feature, to be used in `nix.conf` options and on the command line.
@@ -482,7 +416,7 @@ Experimental and deprecated features support the following metadata properties:
Experimental feature data files should live in `lix/libutil/experimental-features`, and deprecated features in `lix/libutil/deprecated-features`.
They must be listed in the `experimental_feature_definitions` or `deprecated_feature_definitions` lists in `lix/libutil/meson.build` respectively to be considered by the build system.
## Global settings
### Global settings
Global settings support the following metadata properties:
* `name` (required): user-facing name of the setting, to be used as key in `nix.conf` and in the `--option` command line argument.
@@ -510,23 +444,23 @@ Settings are not collected in a single place in the source tree, so an appropria
Look for related setting definition files under second-level subdirectories of `lix` whose name includes `settings`.
Then add the new file there, and don't forget to register it in the appropriate `meson.build` file.
## Builtin functions
### Builtin functions
The following metadata properties are supported for builtin functions:
* `name` (required): the language-facing name (as a member of the `builtins` attribute set) of the function.
* `implementation` (optional): a C++ expression specifying the implementation of the builtin.
It must be a function of signature `void(EvalState &, PosIdx, Value * *, Value &)`.
If not specified, defaults to `prim_${name}`.
* `renameInGlobalScope` (optional): whether the definition should be "hidden" in the global scope by prefixing its name with two underscores.
* `renameInGlobalScope` (optional): whether the definiton should be "hidden" in the global scope by prefixing its name with two underscores.
If not specified, defaults to `true`.
* `args` (required): list containing the names of the arguments, as shown in the documentation.
All arguments must be listed here since the function arity is derived as the length of this list.
* `experimental_feature` (optional): the user-facing name of the experimental feature which needs to be enabled for the builtin function to be available.
* `experimental_feature` (optional): the user-facing name of the experimental feature which needs to be enabled for the bultin function to be available.
If not specified, no experimental feature is required.
New builtin function definition files must be added to `lix/libexpr/builtins` and registered in the `builtin_definitions` list in `lix/libexpr/meson.build`.
## Builtin constants
### Builtin constants
The following metadata properties are supported for builtin constants:
* `name` (required): the language-facing name (as a member of the `builtins` attribute set) of the constant.
* `type` (required): the Nix language type of the constant; the C++ type is automatically derived.
+3 -6
View File
@@ -62,12 +62,6 @@ For `installcheck` specifically, first run `just install` before running the tes
Finer-grained filtering within a test suite is also possible using the [--gtest_filter](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) command-line option to a test suite executable, or the `GTEST_FILTER` environment variable.
### Inspecting failures
The test suite emits logs in `build/meson-logs/`; the full textual failure logs are in `build/meson-logs/testlog.txt`.
If you want a much nicer experience of viewing the logs in a structured manner, use `xunit-viewer --results build/meson-logs/testlog.junit.xml --server` to view them in a web browser.
### Unit test support libraries
There are headers and code which are not just used to test the library in question, but also downstream libraries.
@@ -455,6 +449,9 @@ I grepped `lix/` for `get[eE]nv\("` to find the mentions in Lix code.
- `NIX_CLIENT_PACKAGE` - Runs the test suite against an alternate Nix client with the current daemon.
**Expected value**: something like `/nix/store/...-nix-2.18.2`
- `NIX_TESTS_CA_BY_DEFAULT` - Pass `__contentAddressed`, `outputHashMode` and `outputHashAlgo` to builds of some input-addressed derivations in the test suite.
**Expected value**: 1
- `TEST_DATA` - Not an environment variable! This is used in repl characterization tests to refer to `tests/functional/repl_characterization/data`.
More specifically, that path is replaced with the string `$TEST_DATA` in output for reproducibility.
- `TEST_HOME` (output) - Set to the temporary directory that is set as `$HOME` inside the tests, underneath `$TEST_ROOT`.
+10 -3
View File
@@ -41,6 +41,12 @@
[realise]: #gloss-realise
- [content-addressed derivation]{#gloss-content-addressed-derivation}
A derivation which has the
[`__contentAddressed`](./language/advanced-attributes.md#adv-attr-__contentAddressed)
attribute set to `true`.
- [fixed-output derivation]{#gloss-fixed-output-derivation}
A derivation which includes the
@@ -95,7 +101,7 @@
See [File System Object](@docroot@/architecture/file-system-object.md) for details.
[file system object]: #gloss-store-path
[file system object]: #gloss-file-system-object
- [store object]{#gloss-store-object}
@@ -108,13 +114,14 @@
- [input-addressed store object]{#gloss-input-addressed-store-object}
A store object produced by building a
non-[content-addressed](#gloss-content-addressed-derivation),
non-[fixed-output](#gloss-fixed-output-derivation)
derivation.
- [output-addressed store object]{#gloss-output-addressed-store-object}
A [store object] whose [store path] is determined by its contents.
This includes derivations and the outputs of [fixed-output derivations](#gloss-fixed-output-derivation).
This includes derivations, the outputs of [content-addressed derivations](#gloss-content-addressed-derivation), and the outputs of [fixed-output derivations](#gloss-fixed-output-derivation).
- [substitute]{#gloss-substitute}
@@ -242,7 +249,7 @@
links. NARs are generated and unpacked using `nix-store --dump`
and `nix-store --restore`.
- [`∅`]{#gloss-empty-set}
- [`∅`]{#gloss-emtpy-set}
The empty set symbol. In the context of profile history, this denotes a package is not present in a particular version of the profile.
@@ -60,10 +60,3 @@ Then:
```console
$ docker run -ti lix
```
# Known issues
Lix in Docker is very sensitive to **functional** DNS resolution if you are running with [Pasta protections](../advanced-topics/pasta.md) which are enabled by default since Lix 2.93.0 on most distributions.
If you notice failure to download things, double check whether your **first** DNS entry in `/etc/resolv.conf` is functional.
Lix with [Pasta protections](../advanced-topics/pasta.md) does not support failing over the next entries.
@@ -54,6 +54,11 @@ The most current alternative to this section is to read `package.nix` and see wh
obtained from the its repository
<https://github.com/troglobit/editline>.
- The `libsodium` library for verifying cryptographic signatures
of contents fetched from binary caches.
It can be obtained from the official web site
<https://libsodium.org>.
- Recent versions of Bison and Flex to build the parser. (This is
because Nix needs GLR support in Bison and reentrancy support in
Flex.) For Bison, you need version 2.6, which can be obtained from
+1 -1
View File
@@ -148,7 +148,7 @@ paths) are set.
For example, the following command gets all dependencies of the
Pan newsreader, as described by [its
Nix expression](https://github.com/NixOS/nixpkgs/blob/master/pkgs/by-name/pa/pan/package.nix):
Nix expression](https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/networking/newsreaders/pan/default.nix):
```console
$ nix-shell '<nixpkgs>' --attr pan
@@ -1 +0,0 @@
This section lists known issues around Lix.
@@ -1,21 +0,0 @@
# Limitations of non-isolated builds
## What are non-isolated builds?
In Lix, only builds done on Linux with `sandbox = true` and a functioning
`pasta-path` are isolated from the rest of the system, all other builds are
considered non-isolated to some degree.
For example, running Lix with [Pasta](@docroot@/advanced-topics/pasta.md)
disabled makes the host network visible to fixed-output derivations, reducing
isolation somewhat.
## Clean termination of non-isolated builds
Non-isolated builds may not terminate cleanly in all cases due to limitations in Lix's process management.
This occurs when a build keeps the build log file descriptor open past the end of the actual build. A common cause of this are background tasks that aren't properly terminated before the main build process exits, for example: HTTP servers run as part of a test suite.
See [issue #1018](https://git.lix.systems/lix-project/lix/issues/1018) for an example.
The only solution is to manually terminate leftover processes in your derivation, including during failure scenarios.
+12 -3
View File
@@ -209,8 +209,15 @@ Derivations can declare some infrequently used optional attributes.
- [`__contentAddressed`]{#adv-attr-__contentAddressed}
> **Warning**
> This attribute is part of a removed [experimental feature](@docroot@/contributing/experimental-features.md).
> Setting this flag *will* cause eval errors.
> This attribute is part of an [experimental feature](@docroot@/contributing/experimental-features.md).
>
> To use this attribute, you must enable the
> [`ca-derivations`](@docroot@/contributing/experimental-features.md#xp-feature-ca-derivations) experimental feature.
> For example, in [nix.conf](../command-ref/conf-file.md) you could add:
>
> ```
> extra-experimental-features = ca-derivations
> ```
If this attribute is set to `true`, then the derivation
outputs will be stored in a content-addressed location rather than the
@@ -254,7 +261,7 @@ Derivations can declare some infrequently used optional attributes.
useful for very trivial derivations (such as `writeText` in Nixpkgs)
that are cheaper to build than to substitute from a binary cache.
You may disable the effects of this attribute by enabling the
You may disable the effects of this attibute by enabling the
`always-allow-substitutes` configuration option in Lix.
> **Note**
@@ -302,6 +309,8 @@ Derivations can declare some infrequently used optional attributes.
- `maxSize` defines the maximum size of the resulting [store object](../glossary.md#gloss-store-object).
- `maxClosureSize` defines the maximum size of the output's closure.
- `ignoreSelfRefs` controls whether self-references should be considered when
checking for allowed references/requisites.
Example:
+1 -1
View File
@@ -164,7 +164,7 @@ Note that lists are only lazy in values, and they are strict in length.
An attribute set is a collection of name-value-pairs (called *attributes*) enclosed in curly brackets (`{ }`).
An attribute name can be an identifier or a [string](#type-string).
An attribute name can be an identifier or a [string](#string).
An identifier must start with a letter (`a-z`, `A-Z`) or underscore (`_`), and can otherwise contain letters (`a-z`, `A-Z`), numbers (`0-9`), underscores (`_`), apostrophes (`'`), or dashes (`-`).
> *name* = *identifier* | *string* \
@@ -41,7 +41,7 @@ install Lix. If this is not the case for some reason, you can add it
as follows:
```console
$ nix-channel --add https://channels.nixos.org/nixpkgs-unstable
$ nix-channel --add https://nixos.org/channels/nixpkgs-unstable
$ nix-channel --update
```
@@ -49,7 +49,7 @@ $ nix-channel --update
>
> On NixOS, youre automatically subscribed to a NixOS channel
> corresponding to your NixOS major release (e.g.
> <https://channels.nixos.org/nixos-21.11>). A NixOS channel is identical
> <http://nixos.org/channels/nixos-21.11>). A NixOS channel is identical
> to the Nixpkgs channel, except that it contains only Linux binaries
> and is updated only if a set of regression tests succeed.
@@ -126,7 +126,7 @@ $ nix-env --install --attr nixpkgs.subversion
```
will install the package called `subversion` from `nixpkgs` channel (which is, of course, the
[Subversion version management system](https://subversion.apache.org/)).
[Subversion version management system](http://subversion.tigris.org/)).
> **Note**
>
@@ -71,62 +71,3 @@ $ nix-collect-garbage -d
```
is a quick and easy way to clean up your system.
## Garbage Collector Roots
### Explicit roots
All store paths to which there are symlinks in the directory
`prefix/nix/var/nix/gcroots` will be used as roots by the garbage
collector. For instance, the following command makes the path
`/nix/store/d718ef...-foo` a root of the collector:
```console
$ ln -s /nix/store/d718ef...-foo /nix/var/nix/gcroots/bar
```
That is, after this command, the garbage collector will not remove
`/nix/store/d718ef...-foo` or any of its dependencies.
Subdirectories of `prefix/nix/var/nix/gcroots` are also searched for
symlinks.
Symlinks may also point to paths outside the nix store. If the
destination of the symlink is itself a symlink to a store path, it
is also considered a root. This style of GC root is called an
"indirect root", and is created by tools like `nix-build` to avoid
garbage-collecting paths that are being used on-the-fly rather than
installed in profiles.
### In-use roots
Lix will also perform a best-effort detection of paths that are in use
by running processes when scanning for garbage collection roots, to
avoid removing paths that are still needed by running processes.
Exact details vary between platforms, but the following will generally
be taken into account:
- Executables in the store that are currently running;
- Other files in the store that are mapped into a process's address space (e.g. shared libraries);
- Files in the store to which processes have open handles;
- Store paths found in processes' environment variables.
Note that this detection is susceptible to missing paths that may still be in use for multiple reasons:
- Time-of-check-to-time-of-use (TOCTTOU): new processes may appear
after Lix has enumerated the currently running processes, and will
not be taken into account;
- Access privileges: if the garbage collection is not running as the
root user (this is typically the case for single-user
installations), it will not be able to scan processes belonging to
other users;
- Other types of references: store paths may be stored in parts of the
filesystem (e.g. databases) or process memory (e.g. environment
variables changed since the start of the process) that Lix does not
scan.
For this reason, it is recommended to create explicit roots whenever
using store paths that aren't obtained from some existing explicit GC
root.
@@ -0,0 +1,18 @@
# Garbage Collector Roots
The roots of the garbage collector are all store paths to which there
are symlinks in the directory `prefix/nix/var/nix/gcroots`. For
instance, the following command makes the path
`/nix/store/d718ef...-foo` a root of the collector:
```console
$ ln -s /nix/store/d718ef...-foo /nix/var/nix/gcroots/bar
```
That is, after this command, the garbage collector will not remove
`/nix/store/d718ef...-foo` or any of its dependencies.
Subdirectories of `prefix/nix/var/nix/gcroots` are also searched for
symlinks. Symlinks to non-store paths are followed and searched for
roots, but symlinks to non-store paths *inside* the paths reached in
that way are not followed to prevent infinite recursion.
+14 -4
View File
@@ -2,8 +2,18 @@
For historical reasons, [derivations](@docroot@/glossary.md#gloss-store-derivation) are stored on-disk in [ATerm](https://homepages.cwi.nl/~daybuild/daily-books/technology/aterm-guide/aterm-guide.html) format.
Derivations are serialised in the following format:
Derivations are serialised in one of the following formats:
```
Derive(...)
```
- ```
Derive(...)
```
For all stable derivations.
- ```
DrvWithVersion(<version-string>, ...)
```
The only `version-string`s that are in use today are for [experimental features](@docroot@/contributing/experimental-features.md):
- `"xp-dyn-drv"` for the [`dynamic-derivations`](@docroot@/contributing/experimental-features.md#xp-feature-dynamic-derivations) experimental feature.
+1 -1
View File
@@ -13,7 +13,7 @@
- The `discard-references` feature has been stabilized.
This means that the
`unsafeDiscardReferences`
[unsafeDiscardReferences](@docroot@/contributing/experimental-features.md#xp-feature-discard-references)
attribute is no longer guarded by an experimental flag and can be used
freely.
+135 -2
View File
@@ -1,4 +1,137 @@
# Lix 2.92 "Bombe glacée" (2025-01-18)
# Lix 2.92.3 (2025-06-30)
## Fixes
- Revert CVE-2025-52992 failed mitigation [fj#883](https://git.lix.systems/lix-project/lix/issues/883) [fj#887](https://git.lix.systems/lix-project/lix/issues/887) [cl/3432](https://gerrit.lix.systems/c/lix/+/3432) [cl/3525](https://gerrit.lix.systems/c/lix/+/3525) [cl/3526](https://gerrit.lix.systems/c/lix/+/3526) [cl/3527](https://gerrit.lix.systems/c/lix/+/3527)
Following the initial mitigation of **CVE-2025-52992** in `cl/3432`, we
received reports of **unexpected deletion of in-use store paths**.
Upon investigation, we found that the patch did **not correctly cancel all
automatic deleters**, resulting in potentially critical path loss during normal
operation.
Given the severity and time-sensitive nature of the situation ([see incident
report](https://lix.systems/blog/2025-06-27-lix-critical-bug/)), we evaluated
possible options to repair the behavior safely. However, we concluded that a
rushed fix would either
* **Overdelete**, i.e. breaking running systems, or,
* **Underdelete**, effectively **reopening CVE-2025-52992** while leaving
orphaned paths behind.
As **CVE-2025-52992 has no known exploit vector**, and correctness is critical
in the Lix project, we have **fully reverted the previous mitigations**.
The affected patches (`cl/3432`) have been rolled back for the time being.
Moving forward, the Lix team will rework this code path in a **long-term,
correctness-first fix** on the main branch. We will explore backporting it to
stable channels once its safety is assured.
We are deeply sorry for the stability incident and the Lix team remain
available for assisting you in recovering your systems.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Fallback to safe temp dir when build-dir is unwritable [fj#876](https://git.lix.systems/lix-project/lix/issues/876) [cl/3503](https://gerrit.lix.systems/c/lix/+/3503)
Non-daemon builds started failing with a permission error after introducing the `build-dir` option:
```
$ nix build --store ~/scratch nixpkgs#hello --rebuild
error: creating directory '/nix/var/nix/builds/nix-build-hello-2.12.2.drv-0': Permission denied
```
This happens because:
1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
2. The user lacks permissions for that path.
We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
**Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
# Lix 2.92.2 (2025-06-23)
## Breaking Changes
- Fixed output derivations can be run using `pasta` network isolation [fj#285](https://git.lix.systems/lix-project/lix/issues/285) [cl/3430](https://gerrit.lix.systems/c/lix/+/3430)
Fixed output derivations traditionally run in the host network namespace.
On Linux this allows such derivations to communicate with other sandboxes
or the host using the abstract Unix domains socket namespace; this hasn't
been unproblematic in the past and has been used in two distinct exploits
to break out of the sandbox. For this reason fixed output derivations can
now run in a network namespace (provided by [`pasta`]), restricted to TCP
and UDP communication with the rest of the world. When enabled this could
be a breaking change and we classify it as such, even though we don't yet
enable or require such isolation by default. We may enforce this in later
releases of Lix once we have sufficient confidence that breakage is rare.
[`pasta`]: https://passt.top/
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [puck](https://git.lix.systems/puck) for this.
## Fixes
- Always clean up scratch paths after derivations failed to build [cl/3432](https://gerrit.lix.systems/c/lix/+/3432)
Previously, scratch paths created during builds were not always cleaned up if
the derivation failed, potentially leaving behind unnecessary temporary files
or directories in the Nix store.
This fix ensures that such paths are consistently removed after a failed build,
improving Nix store hygiene, hardening Lix against mis-reuse of failed builds
scratch paths.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- `build-dir` no longer defaults to `temp-dir` [cl/3431](https://gerrit.lix.systems/c/lix/+/3431)
The directory in which temporary build directories are created no longer defaults
to the value of the `temp-dir` setting to avoid builders making their directories
world-accessible. This behavior has been used to escape the build sandbox and can
cause build impurities even when not used maliciously. We now default to `builds`
in `NIX_STATE_DIR` (which is `/nix/var/nix/builds` in the default configuration).
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Forbid impure path accesses in pure evaluation mode again [cl/2708](https://gerrit.lix.systems/c/lix/+/2708)
Lix 2.92.0 mistakenly started allowing the access to ancestors of allowed paths in pure evaluation mode.
This made it possible to bypass the purity restrictions, for example by copying arbitrary files to the store:
```nix
builtins.path {
path = "/";
filter = …;
}
```
Restore the previous behaviour of prohibiting such impure accesses.
Many thanks to [alois31](https://git.lix.systems/alois31) for this.
- Parsing failures in flake.lock no longer crash Lix [fj#559](https://git.lix.systems/lix-project/lix/issues/559) [cl/2401](https://gerrit.lix.systems/c/lix/+/2401)
Failure to parse `flake.lock` no longer hard-crashes Lix and instead produces a nice error message.
```
error:
… while updating the lock file of flake 'git+file:///Users/jade/lix/lix2'
… while parsing the lock file at /nix/store/mm5dqh8a729yazzj82cjffxl97n5c62s-source//flake.lock
error: [json.exception.parse_error.101] parse error at line 1, column 1: syntax error while parsing value - invalid literal;
last read: '#'
```
Many thanks to [gilice](https://git.lix.systems/gilice) for this.
- Fix `--debugger --ignore-try` [cl/2440](https://gerrit.lix.systems/c/lix/+/2440)
When in debug mode (e.g. from using the `--debugger` flag), enabling [`ignore-try`](@docroot@/command-ref/conf-file.md#conf-ignore-try) once again properly disables debug REPLs within [`builtins.tryEval`](@docroot@/language/builtins.md#builtins-tryEval) calls. Previously, a debug REPL would be started as if `ignore-try` was disabled, but that REPL wouldn't actually be in debug mode, and upon exiting the REPL the evaluating process would segfault.
Many thanks to [Dusk Banks](https://git.lix.systems/bb010g) for this.
# Lix 2.92.0 (2025-01-18)
@@ -51,8 +184,8 @@
{
formatter = eachSystem (pkgs:
pkgs.writeShellScriptBin "formatter" ''
if [[ $# = 0 ]]; then set -- .; fi
exec "${pkgs.nixfmt-rfc-style}/bin/nixfmt" "$@"
if [[ $# = 0 ]]; set -- .; fi
exec "${pkgs.nixfmt-rfc-style}/bin/nixfmt "$@"
'');
};
}
-715
View File
@@ -1,715 +0,0 @@
# Lix 2.93 "Bici Bici" (2025-05-09)
# Lix 2.93.0 (2025-05-09)
## Breaking Changes
- more deprecated features
This release cycle features a new batch of deprecated (anti-)features.
You can opt in into the old behavior with `--extra-deprecated-features` or any equivalent configuration option.
- `cr-line-endings`: Current handling of CR (`\r`) or CRLF (`\r\n`) line endings in Nix is inconsistent and broken, and will lead to unexpected evaluation results with certain strings. Given that fixing the semantics might silently alter the evaluation result of derivations, the only option at the moment is to disallow them altogether. More proper support for CRLF is planned to be added back again in the future. Until then, all files must use `\n` exclusively.
- `nul-bytes`: Currently the Nix grammar allows NUL bytes (`\0`) in strings, and thus indirectly also in identifiers. Unfortunately, several core parts of the code base still work with NUL-terminated strings and cannot easily be migrated. Also note that it is still possible to introduce NUL bytes and thus problematic behavior via other means, those are tracked separately.
Many thanks to [piegames](https://git.lix.systems/piegames) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Removal of the `recursive-nix` experimental feature [fj#767](https://git.lix.systems/lix-project/lix/issues/767) [cl/2872](https://gerrit.lix.systems/c/lix/+/2872)
The `recursive-nix` experimental feature and all associated code have been removed.
`recursive-nix` enabled running Nix operations (like evaluations and builds) *inside* a derivation builder. This worked by spawning a temporary Nix daemon socket within the build environment, allowing the derivation to emit outputs that appeared in the outer store. This was primarily used to prototype **dynamic derivations** (dyndrvs), where build plans are generated on-the-fly during a build.
However, this approach introduced critical issues:
- It entrenched the legacy Nix daemon protocol as part of the derivation ABI, which is a blocker for future stabilization.
- It imposed tight coupling between sandbox setup code and knowledge of Nix internals, complicating refactoring and long-term maintenance.
- It was never intended to be the final design for dynamic derivations. The original Nix implementation team, who are leading dyndrv development, have agreed it will be replaced (likely via `varlink` or similar) before any stabilization.
- There is currently no known usage of `recursive-nix` on `lix` or elsewhere **in production**.
If you're using `recursive-nix` for something niche or experimental, we'd love to hear from you on the RFD issue.
You can still run `nix` inside a builder manually if needed — including with isolated user namespaces and fake stores — but the special daemon-handshake machinery is gone.
This removal unblocks several important internal cleanups.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Flake inputs/`builtins.fetchTree` invocations with `type = "file"` now have consistent (but different from previous versions) resulting paths [fj#750](https://git.lix.systems/lix-project/lix/issues/750) [cl/2864](https://gerrit.lix.systems/c/lix/+/2864)
Previously `fetchTree { type = "file"; url = "...", narHash = "sha256-..."; }` could return a different result depending on whether someone has run `nix store add-path --name source ...` on a path with the same `narHash` as the flake input/`fetchTree` invocation (or if such a path exists in an accessible binary cache).
In the past `type = "file"` flake inputs were, in contrast to all other flake inputs, hashed in *flat* hash mode rather than *recursive* hash mode.
The difference between the two is that *flat* mode hashes are just what you get from `sha256sum` of a single file, whereas *recursive* hashes are the SHA256 sum of a NAR (Nix ARchive, a deterministic tarball-like format) of a file tree.
Much of flakes assumes that everything is recursive-hashed including `nix flake archive`, substitution of flake inputs from binary caches, and more, which led to the substitution path code being taken if such a path is present, yielding a different store path non-deterministically.
To fix this non-deterministic evaluation bug, we needed to break derivation hash stability, so some Nix evaluations now produce different results than previous versions of Lix.
Lix now has consistent behaviour with CppNix 2.24 with respect to `file` flake inputs: they are *always* recursively hashed.
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Builders are always started in a fresh cgroup namespace [cl/1996](https://gerrit.lix.systems/c/lix/+/1996)
If you haven't enabled the experimental `cgroups` feature, Nix previously launched builder processes in new namespaces but did not create new cgroup namespaces. As a result, derivations could access and observe the parent cgroup namespace.
Although this update introduces a breaking change, it ensures that all derivations now start in a fresh cgroup namespace by default. This reduces potential impurities observable within the sandbox, improving the likelihood of reproducible builds across different environments.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- `nix-instantiate --parse` outputs json [fj#487](https://git.lix.systems/lix-project/lix/issues/487) [nix#11124](https://github.com/NixOS/nix/issues/11124) [nix#4726](https://github.com/NixOS/nix/issues/4726) [nix#3077](https://github.com/NixOS/nix/issues/3077) [cl/2190](https://gerrit.lix.systems/c/lix/+/2190)
`nix-instantiate --parse` does not print out the AST in a Nix-like format anymore.
Instead, it now prints a JSON representation of the internal expression tree.
Tooling should not rely on the stdout of `nix-instantiate --parse`.
We've done our best to ensure that the new behavior is as compatible with the old one as possible.
If you depend on the old behavior in ways that are not covered anymore or are otherwise negatively affected by this change,
then please reach out so that we can find a sustainable solution together.
Many thanks to [piegames](https://git.lix.systems/piegames) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Remove experimental repl-flake [gh#10103](https://github.com/NixOS/nix/issues/10103) [fj#557](https://git.lix.systems/lix-project/lix/issues/557) [gh#10299](https://github.com/NixOS/nix/pull/10299) [cl/2147](https://gerrit.lix.systems/c/lix/+/2147)
The `repl-flake` experimental feature flag has been removed, its functionality is now the default when `flakes` experimental feature is active. The `nix repl` command now works like the rest of the new CLI in that `nix repl {path}` now tries to load a flake at `{path}` (or fails if the `flakes` experimental feature isn't enabled).
Many thanks to [Jonathan De Troye](https://github.com/detroyejr) and [KFears](https://git.lix.systems/kfearsoff) for this.
## Features
- `lix foo` now invokes `lix-foo` from PATH [cl/2119](https://gerrit.lix.systems/c/lix/+/2119)
Lix introduces the ability to extend the Nix command line by adding custom
binaries to the `PATH`, similar to how Git integrates with other tools. This
feature allows developers and end users to enhance their workflow by
integrating additional functionalities directly into the Nix CLI.
#### Examples
For example, a user can create a custom deployment tool, `lix-deploy-tool`, and
place it in their `PATH`. This allows them to execute `lix deploy-tool`
directly from the command line, streamlining the process of deploying
applications without needing to switch contexts or use separate commands.
#### Limitations
For now, autocompletion is supported to discover new custom commands, but the
documentation will not render them. Argument autocompletion of the custom
command is not supported either.
This is also locked behind a new experimental feature called
`lix-custom-sub-commands` to enable developing all the required features.
Only the top-level `lix` command can be extended, this is an artificial
limitation for the time being until we flesh out this feature.
#### Outline
In the future, this feature may pave the way for moving the Flake subcommand
line to its own standalone binary, allowing for a more focused approach to
managing Nix Flakes while letting the community explore alternatives to
dependency management.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- `nix-env --install` now accepts a `--priority` flag [cl/2607](https://gerrit.lix.systems/c/lix/+/2607)
`nix-env --install` now has an optional `--priority` flag.
Previously, it was only possible to specify a priority by adding a
`meta.priority` attribute to a derivation. `meta` attributes only exist during
eval, so that wouldn't work for installing a store path. It was also possible
to change a priority after initial installation using `nix-env --set-flag`,
however if there is already a conflict that needs to be resolved via priorities,
this will not work.
Now, a priority can be set at install time using `--priority`, which allows for
cleanly overriding the priority at install time.
#### Example
```console
$ nix-build
$ nix-env --install --priority 100 ./result
```
Many thanks to [Andrew Hamon](https://github.com/andrewhamon) for this.
- Add support for eBPF USDT/dtrace probes inside Lix [fj#727](https://git.lix.systems/lix-project/lix/issues/727) [cl/2884](https://gerrit.lix.systems/c/lix/+/2884)
eBPF tracers like `bpftrace` and `dtrace` are a group of similar tools for debugging production systems.
User-space statically defined tracing probes (USDT) allow for defining zero or near-zero disabled-probe-effect probes, thus allowing instrumentation of hot paths in production builds.
Lix now has internal support for defining these probes and has shipped its first probe.
As of this writing it is available by default in the Linux build of Lix.
To try it out on Linux, you can use the following example command:
```
$ sudo bpftrace -l 'usdt:/path/to/liblixstore.so:*:*'
usdt:/path/to/liblixstore.so:lix_store:filetransfer__read
$ sudo bpftrace -e 'usdt:*:lix_store:filetransfer__read { printf("%s read %d\n", str(arg0), arg1); }'
Attaching 1 probe...
https://cache.nixos.org/wvpzaycmvs39h5bcsfrxkjsg48mj4h73.narinf.. read 8192
https://cache.nixos.org/wvpzaycmvs39h5bcsfrxkjsg48mj4h73.narinf.. read 8192
https://cache.nixos.org/nar/1qshsc30nlarzdig0v9b1aasdkwaxhnv0a0.. read 65536
https://cache.nixos.org/nar/1qshsc30nlarzdig0v9b1aasdkwaxhnv0a0.. read 65536
```
Note that bpftrace does not offer any way to list the arguments to USDT probes in a human readable form.
To get the probe definitions, see the `*.d` files in the Lix source code, for example, `lix/libstore/trace-probes.d`.
For more resources on eBPF/bpftrace and dtrace, see:
* The book "BPF Performance Tools" by Brendan Gregg, which discusses bpftrace at length.
* <https://ebpf.io/get-started/>
* [Illumos' dtrace book](https://illumos.org/books/dtrace/preface.html)
Many thanks to [jade](https://git.lix.systems/jade) for this.
## Improvements
- Always print `post-build-hook` logs [fj#675](https://git.lix.systems/lix-project/lix/issues/675) [cl/2801](https://gerrit.lix.systems/c/lix/+/2801)
Logs of `post-build-hook` are now printed unconditionally.
They used to be tied to whether print-build-logs is set, which made debugging them a nightmare when they fail, since the failure output would be eaten if build logs are disabled.
Most usages of `post-build-hook` are pretty quiet especially compared to build logs, so it should not be that bothersome to not be able to turn off.
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Crashes land in syslog now [cl/2640](https://gerrit.lix.systems/c/lix/+/2640)
When Lix crashes with unexpected exceptions and in some other conditions, it prints bug reporting instructions.
Previously, these only landed in stderr and not in syslog.
However, on larger Lix installations, it may be the case that Lix crashes in the client without the logs landing in the system logs, which impeded diagnosis.
Now, such crashes always land in syslog too.
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Deletion of specific paths no longer fails fast [cl/2778](https://gerrit.lix.systems/c/lix/+/2778)
`nix-store --delete` and `nix store delete` now continue deleting
paths even if some of the given paths are still live. An error is only
thrown once deletion of all the given paths has been
attempted. Previously, if some paths were deletable and others
weren't, the deletable ones would be deleted iff they preceded the
live ones in lexical sort order.
The error message for still-live paths no longer reports the paths
that could not be deleted, because there could potentially be many of
these.
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- `--skip-live` for path deletion [cl/2778](https://gerrit.lix.systems/c/lix/+/2778)
`nix-store --delete` and `nix store delete` now support a
`--skip-live` option and a `--delete-closure` option.
This makes custom garbage-collection logic a lot easier to implement
and experiment with:
- Paths known to be large can be thrown at `nix store delete` without
having to manually filter out those that are still reachable from a
root, e.g.
`nix store delete /nix/store/*mbrola-voices*`
- The `--delete-closure` option allows extending this to paths that are
not large themselves but do have a large closure size, e.g.
`nix store delete /nix/store/*nixos-system-gamingpc*`.
- Other heuristics like atime-based deletion can be applied more
easily, because `nix store delete` once again takes over the task of
working out which paths can't be deleted.
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Allow `nix store diff-closures` to output JSON [cl/2360](https://gerrit.lix.systems/c/lix/+/2360)
Add the `--json` option to the `nix store diff-closures` command to allow users to collect diff information into a machine readable format.
```bash
$ build/lix/nix/nix store diff-closures --json /run/current-system /nix/store/n1prick95pihd4lkv58nn3pzg1yivcdb-neovim-0.10.4/bin/nvim | jq | head -n 23
{
"packages": {
"02overridedns": {
"sizeDelta": -688,
"versionsAfter": [],
"versionsBefore": [
""
]
},
"50-coredump.conf": {
"sizeDelta": -1976,
"versionsAfter": [],
"versionsBefore": [
""
]
},
"Diff": {
"sizeDelta": -514864,
"versionsAfter": [],
"versionsBefore": [
"0.4.1"
]
},
```
Many thanks to [Xavier Maso](https://github.com/pamplemousse) for this.
- Show all missing and unexpected arguments in erroneous function calls [cl/2477](https://gerrit.lix.systems/c/lix/+/2477)
When calling a function that expects an attribute set, lix will now show all
missing and unexpected arguments.
e.g. with `({ a, b, c } : a + b + c) { a = 1; d = 1; }` lix will now show the error:
```
[...]
error: function 'anonymous lambda' called without required arguments 'b' and 'c' and with unexpected argument 'd'
[...]
```
Previously lix would just show `b`.
Furthermore lix will now only suggest arguments that aren't yet used.
e.g. with `({ a?1, b?1, c?1 } : a + b + c) { a = 1; d = 1; e = 1; }` lix will now show the error:
```
[...]
error: function 'anonymous lambda' called with unexpected arguments 'd' and 'e'
at «string»:1:2:
1| ({ a?1, b?1, c?1 } : a + b + c) { a = 1; d = 1; e = 1; }
| ^
Did you mean one of b or c?
```
Previously lix would also suggest `a`.
Suggestions are unfortunately still currently just for the first missing argument.
Many thanks to [Zitrone](https://git.lix.systems/quantenzitrone) for this.
- REPL improvements [cl/2319](https://gerrit.lix.systems/c/lix/+/2319) [cl/2320](https://gerrit.lix.systems/c/lix/+/2320) [cl/2321](https://gerrit.lix.systems/c/lix/+/2321)
The REPL has seen various minor improvements:
- Variable declarations have been improved, making copy-pasting code from attrsets a lot easier:
- Declarations can now optionally end with a semicolon
- Multiple declarations can be done within one command, separated by semicolon
- The `foo.bar = "baz";` syntax from attrsets is also supported, however without the attrset merging rules and with restrictions on dynamic attrs like in `let` bindings.
- Variable names now use the proper Nix grammar rules, instead of a regex that only vaguely matched legal identifiers.
- Better error messages overall
- The `:env` command to print currently available variables now also works outside of debug mode
- Adding variables to the REPL now prints a small message on success
Many thanks to [piegames](https://git.lix.systems/piegames) for this.
- Consistently use SRI hashes in hash mismatch errors [cl/2868](https://gerrit.lix.systems/c/lix/+/2868)
Previously there were a few weird cases (flake inputs, e.g., among others) where Lix would print the old Nix base-32 hash format (sha256:abcd...) rather than the newer [SRI base64 format](https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity) (sha256-AAAA...) that is used in most Lix hash mismatch errors.
This made it annoying to compare them to hashes shown by most of the modern UI surface of Lix which uses SRI.
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Allow specifying ports for remote ssh[-ng] stores [cl/2432](https://gerrit.lix.systems/c/lix/+/2432)
You can now specify which port should be used for a remote ssh store (e.g. for remote/distributed builds) through a uri parameter.
E.g., when a remote builder `foo` is listening on port `1234` instead of the default, it can be specified like this `ssh://foo?port=1234`.
Many thanks to [seppel3210](https://github.com/Seppel3210) for this.
- Implicit `__toString` now have stack trace entries [cl/3055](https://gerrit.lix.systems/c/lix/+/3055)
Coercion of attribute sets to strings via their `__toString` attribute now produce stack
frames pointing to the coercion site and the attribute definition. This makes locating a
coercion function error easier as the fault location is now more likely to be presented.
Previously:
```
nix-repl> builtins.substring 1 1 "${{ __toString = self: throw ''bar''; }}"
error:
… while calling the 'substring' builtin
at «string»:1:1:
1| builtins.substring 1 1 "${{ __toString = self: throw ''bar''; }}"
| ^
… caused by explicit throw
at «string»:1:48:
1| builtins.substring 1 1 "${{ __toString = self: throw ''bar''; }}"
| ^
error: bar
```
Now:
```
nix-repl> builtins.substring 1 1 "${{ __toString = self: throw ''bar''; }}"
error:
… while calling the 'substring' builtin
at «string»:1:1:
1| builtins.substring 1 1 "${{ __toString = self: throw ''bar''; }}"
| ^
… while converting a set to string
at «string»:1:25:
1| builtins.substring 1 1 "${{ __toString = self: throw ''bar''; }}"
| ^
… from call site
at «string»:1:29:
1| builtins.substring 1 1 "${{ __toString = self: throw ''bar''; }}"
| ^
… while calling '__toString'
at «string»:1:42:
1| builtins.substring 1 1 "${{ __toString = self: throw ''bar''; }}"
| ^
… caused by explicit throw
at «string»:1:48:
1| builtins.substring 1 1 "${{ __toString = self: throw ''bar''; }}"
| ^
error: bar
```
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
## Fixes
- Avoid unnecessarily killing processes for the build user's UID [nix#9142](https://github.com/NixOS/nix/issues/9142) [fj#667](https://git.lix.systems/lix-project/lix/issues/667)
We no longer kill all processes under the build user's UID before and after
builds on Linux with sandboxes enabled.
This avoids unrelated processes being killed. This might happen for instance,
if the user is running Lix inside a container, wherein the build users use the same UIDs as the daemon's.
Many thanks to [teofilc](https://git.lix.systems/teofilc) for this.
- Forbid impure path accesses in pure evaluation mode again [cl/2708](https://gerrit.lix.systems/c/lix/+/2708)
Lix 2.92.0 mistakenly started allowing the access to ancestors of allowed paths in pure evaluation mode.
This made it possible to bypass the purity restrictions, for example by copying arbitrary files to the store:
```nix
builtins.path {
path = "/";
filter = …;
}
```
Restore the previous behaviour of prohibiting such impure accesses.
Many thanks to [alois31](https://git.lix.systems/alois31) for this.
- Ctrl-C works correctly on macOS again [fj#729](https://git.lix.systems/lix-project/lix/issues/729) [cl/3066](https://gerrit.lix.systems/c/lix/+/3066)
Due to a kernel bug in macOS's `poll(2)` implementation where it would forget about event subscriptions, our detection of closed connections in the Lix daemon didn't work and left around lingering daemon processes.
We have rewritten that thread to use `kqueue(2)`, which is what the `poll(2)` implementation uses internally in the macOS kernel, so now Ctrl-C on clients will reliably terminate daemons once more.
This FD close monitoring has had the highest Apple bug ID references per line of code anywhere in the project, and hopefully not using poll anymore will stop us hitting bugs in poll.
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Fetch peer PID for daemon connections on macOS [fj#640](https://git.lix.systems/lix-project/lix/issues/640) [cl/2453](https://gerrit.lix.systems/c/lix/+/2453)
`nix-daemon` will now fetch the peer PID for connections on macOS, to match behavior with Linux.
Besides showing up in the log output line, If `nix-daemon` is given an argument (such as `--daemon`)
that argument will be overwritten with the peer PID for the forked process that handles the connection,
which can be used for debugging purposes.
Many thanks to [lilyball](https://git.lix.systems/lilyball) for this.
- Test group membership better on macOS [gh#5885](https://github.com/NixOS/nix/issues/5885) [cl/2566](https://gerrit.lix.systems/c/lix/+/2566)
`nix-daemon` will now test group membership better on macOS for `trusted-users` and `allowed-users`.
It not only fetches the peer gid (which fixes `@staff`) but it also asks opendirectory for group
membership checks instead of just using the group database, which means nested groups (like `@_developer`)
and groups with synthesized membership (like `@localaccounts`) will work.
Many thanks to [lilyball](https://git.lix.systems/lilyball) for this.
- `nix store delete` no longer builds paths [cl/2782](https://gerrit.lix.systems/c/lix/+/2782)
`nix store delete` no longer realises the installables
specified. Previously, `nix store delete nixpkgs#hello` would download
hello only to immediately delete it again. Now, it exits with an error
if given an installable that isn't in the store.
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Fix nix-store --delete on paths with remaining referrers [cl/2783](https://gerrit.lix.systems/c/lix/+/2783)
Nix 2.5 introduced a regression whereby `nix-store --delete` and `nix
store delete` started to fail when trying to delete a path that was
still referenced by other paths, even if the referrers were not
reachable from any GC roots. The old behaviour, where attempting to
delete a store path would also delete its referrer closure, is now
restored.
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Add a straightforward way to detect if in a Nix3 Shell [nix#6677](https://github.com/NixOS/nix/issues/6677) [nix#3862](https://github.com/NixOS/nix/issues/3862) [cl/2090](https://gerrit.lix.systems/c/lix/+/2090)
Running `nix shell` or `nix develop` will now set `IN_NIX_SHELL` to
either `pure` or `impure`, depending on whether `--ignore-environment`
is passed. `nix develop` will always be an impure environment.
Many thanks to [Ersei Saggi](https://github.com/9p4) for this.
- Fix experimental and deprecated features showing as integers in `nix config show --json` [fj#738](https://git.lix.systems/lix-project/lix/issues/738) [cl/2882](https://gerrit.lix.systems/c/lix/+/2882)
Internal changes in 2.92 caused `nix config show --json` to show deprecated and experimental features not as the list of named features 2.91 and earlier produced, but as integers. This has been fixed.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- `builtins.fetchTree` is no longer visible in `builtins` when flakes are disabled [cl/2399](https://gerrit.lix.systems/c/lix/+/2399)
`builtins.fetchTree` is the foundation of flake inputs and flake lock files, but is not fully specified in behaviour, which leads to regressions, behaviour differences with CppNix, and other unfun times.
It's gated behind the `flakes` experimental feature, but prior to now, would throw an uncatchable error at runtime when used without the `flakes` feature enabled.
Now it's like other builtins which are experimental feature gated, where it is not visible without the relevant feature enabled.
This fixes a bug in using Eelco Dolstra's version of flake-compat on Lix (and a divergence with CppNix): https://github.com/edolstra/flake-compat/issues/66
Many thanks to [jade](https://git.lix.systems/jade) for this.
- fix usage of `builtins.filterSource` and `builtins.path` with the filter argument when using chroot stores [nix#11503](https://github.com/NixOS/nix/issues/11503)
The semantics of `builtins.filterSource` (and the `filter` argument for
`builtins.path`) have been adjusted regarding how paths inside the Nix store
are handled.
Previously, when evaluating whether a path should be included, the filtering
function received the **physical path** if the source was inside the chroot store.
Now, it receives the **logical path** instead.
This ensures consistency in path handling and avoids potential
misinterpretations of paths within the evaluator, which led to various fallouts
mentioned in <https://github.com/NixOS/nixpkgs/pull/369694>.
Many thanks to [lily](https://git.lix.systems/lilyinstarlight), [alois31](https://git.lix.systems/alois31), and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Fix `--help` formatting [fj#622](https://git.lix.systems/lix-project/lix/issues/622) [cl/2776](https://gerrit.lix.systems/c/lix/+/2776)
The help printed when invoking `nix` or `nix-store` and subcommands with `--help` previously contained garbled terminal escapes. These have been removed.
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Parsing failures in flake.lock no longer crash Lix [fj#559](https://git.lix.systems/lix-project/lix/issues/559) [cl/2401](https://gerrit.lix.systems/c/lix/+/2401)
Failure to parse `flake.lock` no longer hard-crashes Lix and instead produces a nice error message.
```
error:
… while updating the lock file of flake 'git+file:///Users/jade/lix/lix2'
… while parsing the lock file at /nix/store/mm5dqh8a729yazzj82cjffxl97n5c62s-source//flake.lock
error: [json.exception.parse_error.101] parse error at line 1, column 1: syntax error while parsing value - invalid literal;
last read: '#'
```
Many thanks to [gilice](https://git.lix.systems/gilice) for this.
- Flakes follow `--eval-system` where it makes sense [fj#673](https://git.lix.systems/lix-project/lix/issues/673) [fj#692](https://git.lix.systems/lix-project/lix/issues/692) [gh#11359](https://github.com/NixOS/nix/issues/11359) [cl/2657](https://gerrit.lix.systems/c/lix/+/2657)
Most flake commands now follow `--eval-system` when choosing attributes to build/evaluate/etc.
The exceptions are commands that actually run something on the local machine:
- nix develop
- nix run
- nix upgrade-nix
- nix fmt
- nix bundle
This is not a principled approach to cross compilation or anything, flakes still impede rather than support cross compilation, but this unbreaks many remote build use cases.
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Remove some gremlins from path garbage collection [fj#621](https://git.lix.systems/lix-project/lix/issues/621) [fj#524](https://git.lix.systems/lix-project/lix/issues/524) [cl/2465](https://gerrit.lix.systems/c/lix/+/2465) [cl/2387](https://gerrit.lix.systems/c/lix/+/2387)
Path garbage collection had some known unsoundness issues where it would delete things improperly and cause desynchronization between the filesystem state and the database state.
Now Lix tolerates better if such a condition exists by not failing the entire GC if a path fails to delete.
We also fixed a bug in our file locking implementation that is one possible root cause, but may not be every root cause.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [Raito Bezarius](https://git.lix.systems/raito) for this.
- Show illegal path references in fixed-outputs derivations [fj#530](https://git.lix.systems/lix-project/lix/issues/530) [cl/2726](https://gerrit.lix.systems/c/lix/+/2726)
The error created when referencing a store path in a Fixed-Output Derivation is now more verbose, listing the offending paths.
This allows for better pinpointing where the issue might be.
An offender is the following derivation:
```nix
pkgs.stdenv.mkDerivation {
name = "illegal-fod";
dontUnpack = true;
dontBuild = true;
installPhase = ''
cp -R ${pkgs.hello} $out
'';
outputHashMode = "recursive";
outputHashAlgo = "sha256";
outputHash = pkgs.lib.fakeHash;
}
```
The previous error shown would have been:
```
error: illegal path references in fixed-output derivation '/nix/store/rpq4m1y79s2nhs1hj7k47yiyykxykiqa-illegal-fod.drv'
```
and is now:
```
error: the fixed-output derivation '/nix/store/rpq4m1y79s2nhs1hj7k47yiyykxykiqa-illegal-fod.drv' must not reference store paths but 2 such references were found:
/nix/store/1q8w6gl1ll0mwfkqc3c2yx005s6wwfrl-hello-2.12.1
/nix/store/wn7v2vhyyyi6clcyn0s9ixvl7d4d87ic-glibc-2.40-36
```
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- Show error when item from NIX_PATH cannot be downloaded
For e.g. `nix-instantiate -I https://example.com/404`, you'd only get a warning if the download failed, such as
warning: Nix search path entry 'https://example.com/404' cannot be downloaded, ignoring
Now, the full error that caused the download failure is displayed with a note that the search
path entry is ignored, e.g.
warning:
… while downloading https://example.com/404 to satisfy NIX_PATH lookup, ignoring search path entry
warning: unable to download 'https://example.com/404': HTTP error 404 ()
response body: […]
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- Fix Lix crashing on invalid json [fj#642](https://git.lix.systems/lix-project/lix/issues/642) [fj#753](https://git.lix.systems/lix-project/lix/issues/753) [fj#759](https://git.lix.systems/lix-project/lix/issues/759) [fj#769](https://git.lix.systems/lix-project/lix/issues/769) [cl/2907](https://gerrit.lix.systems/c/lix/+/2907)
Lix no longer crashes when it receives invalid JSON. Instead it'll point to the syntax error and give some context about what happened, for example
```
nix derivation add <<<"""
error:
… while parsing a derivation from stdin
error: failed to parse JSON: [json.exception.parse_error.101] parse error at line 2, column 1: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal
```
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Fix handling of `lastModified` in tarball inputs [cl/2792](https://gerrit.lix.systems/c/lix/+/2792)
Previous versions of Lix would fail with the following error, if a
[tarball flake input](@docroot@/protocols/tarball-fetcher.md) redirect
to a URL that contains a `lastModified` field:
```
error: input attribute 'lastModified' is not an integer
```
This is now fixed.
Many thanks to [xanderio](https://github.com/xanderio) and [Julian Stecklina](https://github.com/blitz) for this.
- Fix `--debugger --ignore-try` [cl/2440](https://gerrit.lix.systems/c/lix/+/2440)
When in debug mode (e.g. from using the `--debugger` flag), enabling [`ignore-try`](@docroot@/command-ref/conf-file.md#conf-ignore-try) once again properly disables debug REPLs within [`builtins.tryEval`](@docroot@/language/builtins.md#builtins-tryEval) calls. Previously, a debug REPL would be started as if `ignore-try` was disabled, but that REPL wouldn't actually be in debug mode, and upon exiting the REPL the evaluating process would segfault.
Many thanks to [Dusk Banks](https://git.lix.systems/bb010g) for this.
- Don't consider a path with a specified rev to be `locked` [cl/2064](https://gerrit.lix.systems/c/lix/+/2064)
Until now it was allowed to do e.g.
$ echo 'lalala' > testfile
$ nix eval --expr '(builtins.fetchTree { path = "/home/ma27/testfile"; rev = "0000000000000000000000000000000000000000"; type = "path"; })'
{ lastModified = 1723656303; lastModifiedDate = "20240814172503"; narHash = "sha256-hOMY06A0ohaaCLwnhpZIMoAqi/8kG2vk30NRiqi0dfc="; outPath = "/nix/store/lhfz259iipmv9ky995rml8018jvriynh-source"; rev = "0000000000000000000000000000000000000000"; shortRev = "0000000"; }
$ cat /nix/store/lhfz259iipmv9ky995rml8018jvriynh-source
lalala
because any kind of input with a `rev` specified is considered to be locked.
With this change, inputs of type `path`, `indirect` and `tarball` are no longer
considered locked with a rev, but no hash specified.
This behavior was changed in
[CppNix 2.21 as well](https://github.com/nixos/nix/commit/071dd2b3a4e6c0b2106f1b6f14ec26e153d97446) as well.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- Fix macOS sandbox profile size errors [fj#752](https://git.lix.systems/lix-project/lix/issues/752) [fj#718](https://git.lix.systems/lix-project/lix/issues/718) [cl/2861](https://gerrit.lix.systems/c/lix/+/2861)
Fixed an issue on macOS where the sandbox profile could exceed size limits when building derivations with many dependencies. The profile is now split into multiple allowed sections to stay under the interpreter's limits.
This resolves errors like
```
error: (failed with exit code 1, previous messages: sandbox initialization failed: data object length 65730 exceeds maximum (65535)|failed to configure sandbox)
error: unexpected EOF reading a line
```
Many thanks to [Pierre-Etienne Meunier](https://github.com/P-E-Meunier) and [Poliorcetics](https://github.com/poliorcetics) for this.
- Fix interference of the multiline progress bar with output [cl/2774](https://gerrit.lix.systems/c/lix/+/2774)
In some situations, the progress indicator of the multiline progress bar would interfere with persistent output.
This would result in progress bar headers being visible in place of the desired text, for example the outputs shown after a `:b` command in the repl.
The underlying ordering issue has been fixed, so that the undesired interference does not happen any more.
Many thanks to [alois31](https://git.lix.systems/alois31) for this.
- Paralellise `nix store sign` using a thread pool [fj#399](https://git.lix.systems/lix-project/lix/issues/399) [cl/2606](https://gerrit.lix.systems/c/lix/+/2606)
`nix store sign` with a large collection of provided paths (such as when using with `--all`) has historically
signed these paths serially. Taking extreme amounts of time when preforming operations such as fixing binary
caches. This has been changed. Now these signatures are performed using a thread pool like `nix store copy-sigs`.
Many thanks to [Lunaphied](https://git.lix.systems/Lunaphied) for this.
- `post-build-hook` only receives settings that are set [fj#739](https://git.lix.systems/lix-project/lix/issues/739) [cl/2800](https://gerrit.lix.systems/c/lix/+/2800)
If one is using `post-build-hook` to upload paths to a cache, it used to be broken if CppNix was used inside the script, since CppNix would fail about unsupported configuration option values in some of Lix's defaults.
This is because `post-build-hook` receives the settings of the nix daemon in the `NIX_CONFIG` environment variable.
Now Lix only emits overridden settings to `post-build-hook` invocations, which fixes this issue in the majority of cases: where the configuration is not explicitly incompatible.
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Remove lix-initiated ssh connection sharing [fj#304](https://git.lix.systems/lix-project/lix/issues/304) [fj#644](https://git.lix.systems/lix-project/lix/issues/644) [cl/3005](https://gerrit.lix.systems/c/lix/+/3005)
Lix no longer explicitly requests ssh connection sharing (ControlMaster/ControlPath SSH
options, see also ssh_config(5) man page) when connecting to remote stores. This may
impact command latency when `NIX_REMOTE` is set to a `ssh://` or `ssh-ng://` url, or if
`--store` is specified. Remote build connections did not use ssh connection sharing.
Connection sharing configuration is now inherited from user configuration at all times. It
is now advisable to configure connection sharing for remote builders for improved latency.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
## Development
- Add `nix_plugin_entry` entry point for plugins [fj#740](https://git.lix.systems/lix-project/lix/issues/740) [fj#359](https://git.lix.systems/lix-project/lix/issues/359) [gh#8699](https://github.com/NixOS/nix/pull/8699) [cl/2826](https://gerrit.lix.systems/c/lix/+/2826)
Plugins are an exceptionally rarely used feature in Lix, but they are important as a prototyping tool for code destined for Lix itself, and we want to keep supporting them as a low-maintenance-cost feature.
As part of the overall move towards getting rid of static initializers for stability and predictability reasons, we added an explicit `nix_plugin_entry` function like CppNix has, which is called immediately after plugin load, if present.
This makes control flow more explicit and allows for easily registering things that have had their static initializer registration classes removed.
Many thanks to [jade](https://git.lix.systems/jade) and [yorickvp](https://github.com/yorickvp) for this.
## Miscellany
- Set default of `connect-timeout` to `5` [cl/2799](https://gerrit.lix.systems/c/lix/+/2799)
By default, the connection timeout to substituters is now 5s instead of 300s.
That way, unavailable substituters are detected quicker.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
-958
View File
@@ -1,958 +0,0 @@
# Lix 2.94 "Açaí na tigela" (2025-11-17)
# Lix 2.94.1 (2026-03-13)
# Lix 2.94.0 (2025-11-17)
## Breaking Changes
- Remove support for daemon protocols before 2.18 [fj#510](https://git.lix.systems/lix-project/lix/issues/510) [cl/3249](https://gerrit.lix.systems/c/lix/+/3249)
Support for daemon wire protocols belonging to Nix 2.17 or older have been
removed. This impacts clients connecting to the local daemon socket or any
remote builder configured using the `ssh-ng` protocol. Builders configured
with the `ssh` protocol are still accessible from clients such as Nix 2.3.
Additionally Lix will not be able to connect to an old daemon locally, and
remote build connections to old daemons is likewise limited to `ssh` urls.
We have decided to take this step because the old protocols are very badly
tested (if at all), maintenance overhead is high, and a number of problems
with their design makes it infeasible to remain backwards compatible while
we move Lix to a more modern RPC mechanism with better versioning support.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Remove impure derivations and dynamic derivations [fj#815](https://git.lix.systems/lix-project/lix/issues/815) [cl/3210](https://gerrit.lix.systems/c/lix/+/3210)
The `impure-derivations` and `dynamic-derivations` experimental feature have
been removed.
New impure or dynamic derivations cannot be created from this point forward, and
any such pre-existing store derivations canot be read or built any more.
Derivation outputs created by building such a derivation are still valid
until garbage collected; existing store derivations can only be garbage
collected.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- First argument to `--arg`/`--argstr` must be a valid Nix identifier [fj#496](https://git.lix.systems/lix-project/lix/issues/496)
The first argument to `--arg`/`--argstr` must be a valid Nix identifier, i.e.
`nix-build --arg config.allowUnfree true` is now rejected.
This is because that invocation is a false friend since it doesn't set
`{ config = { allowUnfree = true; }; }`, but `{ "config.allowUnfree" = true; }`.
The idea is to change the behavior to the latter in the long-term. For that,
non-identifiers started giving a warning since 2.92 and are now rejected to give people
who depend on that a chance to notice and potentially weigh in on the discussion.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- New cgroup delegation model [fj#537](https://git.lix.systems/lix-project/lix/issues/537) [fj#77](https://git.lix.systems/lix-project/lix/issues/77) [cl/3230](https://gerrit.lix.systems/c/lix/+/3230)
Builds using cgroups (i.e. `use-cgroups = true` and the experimental feature
`cgroups`) now always delegate a cgroup tree to the sandbox.
Compared to the original C++ Nix project, our delegation includes the
`subtree_control` file as well, which means that the sandbox can disable
certain controllers in its own cgroup tree.
This is a breaking change because this requires the Nix daemon to run with an
already delegated cgroup tree by the service manager.
## How to setup the cgroup tree with systemd?
systemd offers knobs to perform the required setup using:
```
[Service]
Delegate=yes
DelegateSubtree=supervisor
```
These directives are now included in our systemd packaging.
## What about using Nix as root without connecting to the daemon?
Builds run as `root` without connecting to the daemon relying on the cgroup
feature are now broken, i.e.
```console
# nix-build --use-cgroups --sandbox ... # will not work
```
Consider doing instead:
```console
# systemd-run --same-dir --wait -p Delegate=yes -p DelegateSubgroup=supervisor nix-build --use-cgroups ...
```
If you need to disable cgroups temporarily, remember that you can do
`NIX_CONF='include /etc/nix/nix.conf\nuse-cgroups = false' nix-build ...` or
`nix-build --no-use-cgroups ...`.
## What about other service managers than systemd?
systemd has a [documentation](https://systemd.io/CGROUP_DELEGATION/) on how to
handle cgroup delegation from service management perspective.
If your service manager adheres to systemd semantics, e.g. writing an extended
attribute `user.delegate=1` on the delegated cgroup tree directory and moving
the `nix-daemon` process inside a cgroup tree to respect the inner process
rule, then, the feature will work as well.
## Why is the cgroup feature still experimental?
While the cgroup feature unlocks many use cases, its behavior and integration (e.g. user experience), especially at scale on build farms or in multi-tenant environments, are not yet fully matured. Theres also potential for deeper systemd integration (e.g. using slices and scopes) that has not been fully explored.
To avoid locking in an unstable interface, were keeping the experimental flag until we have validated the feature across a broader range of scenarios, including but not limited to:
* Nix as root
* Hydra-style build farms
* Forgejo CI runners
* Shared remote builders
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [eldritch horrors](https://git.lix.systems/pennae), and [lheckemann](https://git.lix.systems/lheckemann) for this.
- Enable high compress ratio zstd compression by default for binary caches uploads [fj#945](https://git.lix.systems/lix-project/lix/issues/945) [cl/4503](https://gerrit.lix.systems/c/lix/+/4503)
The default compression method for binary cache uploads has been switched from
[`xz`](https://github.com/tukaani-project/xz) to
[`zstd`](https://github.com/facebook/zstd) to address performance and usability
issues related to modern hardware and high-speed connections.
## Why?
`xz` offers compression ratios but is single-threaded in our implementation and
very slow (~10-20 Mbps in our test), preventing full utilization of 100Mbps+
connections and significantly slowing decompression for end users.
Lix is a "compress once, decompress many" application: build farms can afford
to spend more time compressing to achieve a faster download transfer for the
end user. More importantly, it matters that all end users spend the least
amount of time decompressing.
## What about compression ratios?
`zstd` cannot achieve the same peaks as `xz`, nonetheless, `zstd` compression
level has been increased to level 12 by default to balance compression ratio
and performance.
## Synthetic test case data
* **xz** (default compression level) on a 4.4GB file: ~632MB (77s)
* **zstd** (level 12) on the same file: ~775MB (18s), 18% larger but 50% faster
* **zstd** (level 14): ~773MB (37s)
* **zstd** (level 16): ~735MB (66s)
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [Raito Bezarius](https://git.lix.systems/raito) for this.
- Repl debugger uses `--ignore-try` by default [lix#666](https://git.lix.systems/lix-project/lix/issues/666) [cl/3488](https://gerrit.lix.systems/c/lix/+/3488)
Previously, using the debugger meant that exceptions thrown in `builtins.tryEval` would trigger the debugger.
However, this caught nixpkgs initialization code, which is unhelpful in the majority of cases, so we changed the default.
To get the old behaviour, use `--no-ignore-try`.
```
$ nix repl --debugger --expr 'with import <nixpkgs> {}; pkgs.hello'
Lix 2.94.0-dev-pre20250625-9a59106
Type :? for help.
error: file 'nixpkgs-overlays' was not found in the Nix search path (add it using $NIX_PATH or -I)
This exception occurred in a 'tryEval' call. Use --ignore-try to skip these.
Added 13 variables.
nix-repl>
```
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Strings may now contain NUL bytes [cl/3968](https://gerrit.lix.systems/c/lix/+/3968)
Lix now allows strings to contain NUL bytes instead of silently truncating the
string before the first such byte. Notably NUL-bearing strings were allowed as
attribute names—even though the corresponding strings were not representable!—
leading to very surprising and incorrect behavior in corner cases, for example
```
nix-repl> builtins.fromJSON ''{"a": 1, "a\u0000b": 2}''
{
a = 1;
"ab" = 2;
}
nix-repl> builtins.attrNames (builtins.fromJSON ''{"a": 1, "a\u0000b": 2}'')
[
"a"
"a"
]
```
rather than the more correct but still with the terminal eating NUL on display
```
nix-repl> builtins.fromJSON ''{"a": 1, "a\u0000b": 2}''
{
a = 1;
"ab" = 2;
}
nix-repl> builtins.attrNames (builtins.fromJSON ''{"a": 1, "a\u0000b": 2}'')
[
"a"
"ab"
]
```
We consider this a breaking change since eval results *will* change if strings
with embedded NUL bytes were used, but we also consider the old behavior to be
not intentional (seeing how inconsistent it was) but merely fallout from a old
and misguided implementation decision to be worked around, not actually fixed.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Fixed output derivations can be run using `pasta` network isolation [fj#285](https://git.lix.systems/lix-project/lix/issues/285) [cl/3452](https://gerrit.lix.systems/c/lix/+/3452)
Fixed output derivations traditionally run in the host network namespace.
On Linux this allows such derivations to communicate with other sandboxes
or the host using the abstract Unix domains socket namespace; this hasn't
been unproblematic in the past and has been used in two distinct exploits
to break out of the sandbox. For this reason fixed output derivations can
now run in a network namespace (provided by [`pasta`]), restricted to TCP
and UDP communication with the rest of the world. When enabled this could
be a breaking change and we classify it as such, even though we don't yet
enable or require such isolation by default. We may enforce this in later
releases of Lix once we have sufficient confidence that breakage is rare.
[`pasta`]: https://passt.top/
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [puck](https://git.lix.systems/puck) for this.
- Function equality semantics are more consistent, but still bad [cl/4556](https://gerrit.lix.systems/c/lix/+/4556) [cl/4244](https://gerrit.lix.systems/c/lix/+/4244)
Lix has inherited a historic misfeature from CppNix in the form of pointer
equality checks built into the `==` operator. These checks were originally
meant to optimize comparison for large sets, but they have the unfortunate
side effect of producing unexpected results when sets containing functions
are compared. **Lix 2.93 and earlier** behave as shown in the repl session
```
Lix 2.93.3
Type :? for help.
nix-repl> f = x: x
Added f.
nix-repl> f == f
false
nix-repl> let s.f = f; in s.f == s.f
false
nix-repl> # however!
{ inherit f; } == { inherit f; }
true
nix-repl> [ f ] == [ f ]
true
nix-repl> # and, in another twist:
[ f ] == map f [ f ]
false
```
Nixpkgs relies on sets containing functions being comparable, so we cannot
simply deprecate this behavior. Due to changes to the object model used by
Lix ***all* comparisons above now evaluate to `true`**. This is considered
a breaking change because eval results may differ, but we also consider it
minor because the optimization is unsound (c.f. `let l = [NaN]; in l == l`
evaluates to `true` even though floating point `NaN` is incomparable). Lix
intends to remove this optimization altogether in the future, but until we
can do that we instead make it slightly less broken to allow other, *real*
optimizations. Function equality comparison remains **undefined behavior**
and should not be relied upon in Nixlang code that intends to be portable.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- `nix eval --write-to` has been removed [fj#974](https://git.lix.systems/lix-project/lix/issues/974) [fj#227](https://git.lix.systems/lix-project/lix/issues/227) [cl/4045](https://gerrit.lix.systems/c/lix/+/4045)
`nix eval --write-to` has been removed since it was underspecified, not widely
useful, and prone to security-sensitive misbehaviors. The feature was added in
Nix 2.4 purely for internal use in the build system. According to our research
it hasn't found any use outside of some distribution packaging scripts. Please
use structured outputs formats (such as JSON) instead as they have better type
fidelity, don't conflate attributes with paths, and are useful to other tools.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Remove the `parse-toml-timestamps` experimental feature
The `parse-toml-timestamps` experimental feature has been removed.
This feature used inband signalling to mark timestamps, making it
impossible to unambiguously parse TOML documents. It also exposed
implementationdefined behaviour in the TOML specification that
changed in the toml11 parser library.
Any interface for parsing TOML timestamps suitable for future
stabilization would necessarily involve breaking changes, and there
is no evidence this experimental feature is being relied upon in the
wild, so it has been removed.
Many thanks to [Emily](https://git.lix.systems/emilazy) for this.
- Reject overflowing TOML integer literals [cl/3916](https://gerrit.lix.systems/c/lix/+/3916)
The toml11 library used by Lix was updated. The new
version aligns with the [TOML v1.0.0 specifications
requirement](https://toml.io/en/v1.0.0#integer) to reject integer
literals that cannot be losslessly parsed. This means that code like
`builtins.fromTOML "v=0x8000000000000000"` will now produce an error
rather than silently saturating the integer result.
Many thanks to [Emily](https://git.lix.systems/emilazy) for this.
- uid-range depends on cgroups [cl/3230](https://gerrit.lix.systems/c/lix/+/3230)
`uid-range` builds now depends on `cgroups`, an experimental feature.
`uid-range` builds already depended upon `auto-allocate-uids`, another experimental feature.
The rationale for doing so is that `uid-range` provides a sandbox with many
UIDs, this is useful for re-mapping them into a nested namespace, e.g. a
container.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
## Features
- Add `inputs.self.submodules` flake attribute [fj#942](https://git.lix.systems/lix-project/lix/issues/942) [cl/3839](https://gerrit.lix.systems/c/lix/+/3839)
A port of <https://github.com/NixOS/nix/pull/12421> to Lix, which:
- adds a general `inputs.self` flake attribute that retroactively applies
configurations to a flake after it's been fetched, then triggers a refetch of
the flake with the new config.
- implements `inputs.self.submodules` that allows a flake to declare its need
for submodules, which are then fetched automatically with no need to pass
`?submodules=1` anywhere.
Many thanks to [Eelco Dolstra](https://github.com/edolstra) and [ورد](https://git.lix.systems/janw4ld) for this.
- Lix supports HTTP/3 behind `--http3` [fj#1033](https://git.lix.systems/lix-project/lix/issues/1033)
Lix now supports HTTP/3 for file transfers when the linked curl version
supports it.
By default, HTTP/3 is disabled notably due to performance issues reported in
mid-2024. [More details
here](https://daniel.haxx.se/blog/2024/06/10/http-3-in-curl-mid-2024/).
As of 2025-11-14, [NixOS official cache](https://cache.nixos.org) supports
HTTP/3 via Fastly. [More info
here](https://github.com/NixOS/infra/commit/157fa70e46afbd6338a32407be461fce05c57bf8).
To enable HTTP/3:
* Use `--http3` for individual transfers.
* Add `http3 = true` in your Nix configuration for permanent activation.
To disable it, use `--no-http3`.
**Note**:
* `--no-http2 --http3` will still enable both HTTP/2 and HTTP/3.
* `--http2 --http3` will prioritize HTTP/3 and fall back to HTTP/2 (and then
HTTP/1.1).
These are current CLI limitations. In the future, we plan to replace `--httpX`
options with `--max-http-version [1,2,3]` for easier version selection in Lix
transfers.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Add hyperlinks in attr set printing [cl/3790](https://gerrit.lix.systems/c/lix/+/3790)
The attribute set printer, such as is seen in `nix repl` or in type errors, now prints hyperlinks on each attribute name to its definition site if it is known.
Example: all of the attributes shown here are hyperlinks to the exact definition site of the attribute in question:
```
$ nix eval -f '<nixpkgs>' lib.licenses.mit
{ deprecated = false; free = true; fullName = "MIT License"; redistributable = true; shortName = "mit"; spdxId = "MIT"; url = "https://spdx.org/licenses/MIT.html"; }
```
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Experimental integer coercion in interpolated strings [cl/3198](https://gerrit.lix.systems/c/lix/+/3198)
Ever tried interpolating a port number in Lix and ended up with something like this?
```nix
"http://${config.network.host}:${builtins.toString config.network.port}/"
```
You're not alone. Thousands of Lix users suffer every day from excessive `builtins.toString` syndrome. Its 2025, and we still have to cast integers to use them in strings.
To address this, Lix introduces the **`coerce-integers`** experimental feature. When enabled, interpolated integers within `"${...}"` are automatically coerced to strings. This allows writing:
```nix
"http://${config.network.host}:${config.network.port}/"
```
without additional conversion.
To enable the feature, you need to add `coerce-integers` to your set of experimental features.
### Stabilization criteria
The `coerce-integers` feature is experimental and limited strictly to string interpolation (`"${...}"`). Before stabilization, the following must hold:
1. **Interpolation-only**
Coercion must not occur outside interpolation. Expressions like `"" + 42` must continue to fail.
2. **Expectation that no explicit cast are being observed**
Cases observing explicit coercion (e.g., via `tryEval` gadget or similar) are expected not to be load-bearing in actual production code.
### Timeline for stabilization
If the feature proves safe and is widely adopted across typical usage (e.g., actual configurations in the wild turning on the flag, non-trivial out-of-tree projects using it), the experimental flag will be removed **after six months of active use or two Lix releases**, whichever is longer.
This avoids locking the feature in experimental status indefinitely, as happened with Flakes, while allowing time for validation and ecosystem integration.
### What about coercing floats or more?
Coercion beyond integers -- such as for floats or other types -- is **not planned**, even under an experimental flag. Questions like "what is the canonical string representation of a float?" involve subtle and context-dependent trade-offs. Without a robust and principled mechanism to define and audit such behavior, introducing broader coercion risks setting unintended and hard-to-reverse precedents. The scope of `coerce-integers` is intentionally narrow and will remain so.
In terms of outlook, a proposal like https://git.lix.systems/lix-project/lix/issues/835 could pave the way for a better solution.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [delroth](https://github.com/delroth), [eldritch horrors](https://git.lix.systems/pennae), and [winter](https://git.lix.systems/winter) for this.
- nix-eval-jobs: support `--no-instantiate` flag [fj#987](https://git.lix.systems/lix-project/lix/issues/987)
`nix-eval-jobs` now supports a flag called `--no-instantiate`. With this enabled,
no write operations on the eval store are performed. That means, only evaluation is
performed, but derivations (and their gcroots) aren't created.
Many thanks to [mic92](https://github.com/mic92) and [ma27](https://git.lix.systems/ma27) for this.
## Improvements
- Assess current profile generations pointers in `nix doctor` [cl/3108](https://gerrit.lix.systems/c/lix/+/3108)
Added a new check to `nix doctor` that verifies whether the current generation of
a Nix profile can be resolved. This helps users diagnose issues with broken or
misconfigured profile symlinks.
This helps determining if you have broken symlinks or misconfigured packaging.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Improved susbtituter query speed
The code used to query substituters for derivations has been rewritten slightly
to take advantage of our asynchronous runtime. Such queries run for every build
that could download from substituters and processes every derivation that isn't
yet present on the local system. Previously Lix would use `http-connections` to
limit query concurrency, even for modern caches that support HTTP/2 and have no
limit on how many queries can be run concurrently on one single connection. Lix
no longer does this, resulting in approximately 60% reduction in query time for
medium-sized closures (e.g. NixOS system closures) during testing, although the
exact number depends greatly on local network latency and generally improves as
latency increases. Unlike previously setting `http-connections` to `1` or other
low values no longer brings a massive penalty in query performance if the cache
in use by the querying system supports HTTP/2 (as e.g. `cache.nixos.org` does).
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Hitting Control-C twice always terminates Lix [cl/3574](https://gerrit.lix.systems/c/lix/+/3574)
Hitting Control-C or sending `SIGINT` to Lix now prints an informational message
if it is still running after on second, the second Control-C/`SIGINT` terminates
Lix immediately without waiting for any shutdown code to finish running. Lix did
not treat the second such event differently from first in the past; this made it
impossible to easily terminate running Lix processes that got stuck in e.g. very
expensive Nixlang code that never interacted with the store. We now terminate as
soon as the user hits Control-C again without waiting any more, to much the same
effect as putting Lix into the background and killing it immediately afterwards.
This means you can now more conveniently break out of stuck Nixlang evaluations:
```
nix-instantiate --eval --expr 'let f = n: if n == 0 then 0 else f (n - 1) + f (n - 1); in f 32'
^CStill shutting down. Press ^C again to abort all operations immediately.
^C
❌130
```
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- `--keep-failed` chowns the build directory to the user that request the build
Running a build with `--keep-failed` now chowns the temporary directory from the
builder user and group to the user that request the build if the build came from
a local user connected to the daemon. This makes inspecting failed derivations a
lot easier. On Linux the build directory made visible to the user will not be in
the same path as it was in the sandbox and continuing builds will usually break.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Better debuggability on fixed-output hash mismatches
Fixed-output derivation hash mismatch error messages will now include the path that was
produced unexpectedly, and this path will be registered as valid even if `--check`
(`nix-store`, `nix-build`) or `--rebuild` (`nix build`) was passed. This makes comparing
the expected path with the obtained path easier, and is useful for debugging when
upstreams modify previously-published releases or when changes in fixed-output
derivations' dependencies affect their output unexpectedly.
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Add --raw flag to `nix-instantiate --eval` for unescaped output [gh#12119](https://github.com/NixOS/nix/pull/12119) [cl/2886](https://gerrit.lix.systems/c/lix/+/2886)
The `nix-instantiate --eval` command now supports a `--raw` flag. When used,
the result must be coercible to a string (as with `${...}`) and is printed
verbatim, without quotes or escaping.
Many thanks to [Martin Fischer](https://github.com/not-my-profile), [infinisil](https://github.com/infinisil), and [Raito Bezarius](https://git.lix.systems/raito) for this.
- Allow `nix store ls` to read nar listings from binary cache stores. [cl/3225](https://gerrit.lix.systems/c/lix/+/3225)
The `nix store ls` command now supports reading `.ls` nar listings from binary cache stores.
If a listing is detected for the store path being queried, the nar is no longer downloaded.
These nar listings are available in binary cache stores where the `write-nar-listing` option is
enabled, such as cache.nixos.org.
Many thanks to [Victor Fuentes](https://git.lix.systems/vlinkz) for this.
- show tree with references that lead to an output cycle [fj#551](https://git.lix.systems/lix-project/lix/issues/551)
When Lix determines a cyclic dependency between several outputs of a derivation,
it now displays which files in which outputs lead to an output cycle:
```
error: cycle detected in build of '/nix/store/gc5h2whz3rylpf34n99nswvqgkjkigmy-demo.drv' in the references of output 'bar' from output 'foo'.
Shown below are the files inside the outputs leading to the cycle:
/nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar
└───lib/libfoo: …stuffbefore /nix/store/h680k7k53rjl9p15g6h7kpym33250w0y-demo-baz andafter.…
→ /nix/store/h680k7k53rjl9p15g6h7kpym33250w0y-demo-baz
└───share/snenskek: …???? /nix/store/dm24c76p9y2mrvmwgpmi64rryw6x5qmm-demo-foo ....…
→ /nix/store/dm24c76p9y2mrvmwgpmi64rryw6x5qmm-demo-foo
└───bin/alarm: …textexttext/nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar abcabcabc.…
→ /nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar
```
Please note that showing the files and its contents while displaying the cycles only works
on Linux.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- Lix now enables parallel marking in boehm-gc [fj#983](https://git.lix.systems/lix-project/lix/issues/983) [cl/3880](https://gerrit.lix.systems/c/lix/+/3880)
This brings a fairly modest performance improvement (~38% for `nixpkgs search hello`) to evaluation, especially in scenarios that necessitate larger heap sizes.
Many thanks to [Eelco Dolstra](https://github.com/edolstra) and [Seth Flynn](https://git.lix.systems/getchoo) for this.
- `disallowedRequisites` now reports chains of disallowed requisites [fj#334](https://git.lix.systems/lix-project/lix/issues/334) [fj#626](https://git.lix.systems/lix-project/lix/issues/626) [gh#10877](https://github.com/NixOS/nix/issues/10877)
When a build fails because of [`disallowedRequisites`](@docroot@/language/advanced-attributes.md#adv-attr-disallowedRequisites), the error message now includes the chain of references that led to the failure. This makes it easier to see in which derivations the chain can be broken, to resolve the problem.
Example:
```
$ nix-build -A hello
error: output '/nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2' is not allowed to refer to the following paths:
/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-glibc-2.40-66
Shown below are chains that lead to the forbidden path(s).
/nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2
└───/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-glibc-2.40-66
```
Many thanks to [ma27](https://git.lix.systems/ma27) and [Robert Hensing](https://github.com/roberth) for this.
- Stack traces now summarize involved derivations at the bottom [cl/4493](https://gerrit.lix.systems/c/lix/+/4493)
When evaluation errors and a stack trace is printed,
For example, if I add Nheko to a NixOS `environment.systemPackages` without adding `olm-3.2.16` `nixpkgs.config.permittedInsecurePackages`, then without `--show-trace`, I previously got this:
```
error:
… while calling the 'head' builtin
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/attrsets.nix:1701:13:
1700| if length values == 1 || pred here (elemAt values 1) (head values) then
1701| head values
| ^
1702| else
… while evaluating the attribute 'value'
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/modules.nix:1118:7:
1117| // {
1118| value = addErrorContext "while evaluating the option `${showOption loc}':" value;
| ^
1119| inherit (res.defsFinal') highestPrio;
(stack trace truncated; use '--show-trace' to show the full trace)
error: Package olm-3.2.16 in /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/pkgs/by-name/ol/olm/package.nix:37 is marked as insecure, refusing to evaluate.
< -snip the whole explanation about olm's CVEs- >
```
This doesn't tell me anything about where `olm-3.2.16` came from.
With `--show-trace`, there's 1155 lines to sift through, but does contain lines like "while evaluating derivation 'nheko-0.12.1'".
With this change, those lines are summarized and collected at the bottom, regardless of `--show-trace`:
```
error:
… while calling the 'head' builtin
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/attrsets.nix:1701:13:
1700| if length values == 1 || pred here (elemAt values 1) (head values) then
1701| head values
| ^
1702| else
… while evaluating the attribute 'value'
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/modules.nix:1118:7:
1117| // {
1118| value = addErrorContext "while evaluating the option `${showOption loc}':" value;
| ^
1119| inherit (res.defsFinal') highestPrio;
(stack trace truncated; use '--show-trace' to show the full trace)
error: Package olm-3.2.16 in /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/pkgs/by-name/ol/olm/package.nix:37 is marked as insecure, refusing to evaluate.
< -snip the whole explanation about olm's CVEs- >
note: trace involved the following derivations:
derivation 'etc'
derivation 'dbus-1'
derivation 'system-path'
derivation 'nheko-0.12.1'
derivation 'mtxclient-0.10.1'
```
Now we finally know that olm was evaluated because of Nheko, without sifting through *thousands* of lines of error message.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- Symbols reuses once-allocated Value to reduce garbage collected allocations [cl/3308](https://gerrit.lix.systems/c/lix/+/3308) [cl/3300](https://gerrit.lix.systems/c/lix/+/3300) [cl/3314](https://gerrit.lix.systems/c/lix/+/3314) [cl/3310](https://gerrit.lix.systems/c/lix/+/3310) [cl/3312](https://gerrit.lix.systems/c/lix/+/3312) [cl/3313](https://gerrit.lix.systems/c/lix/+/3313)
In the Lix evaluator, **symbols** represent immutable strings, like those used
for attribute names.
In evaluator design, such strings are typically [**interned**](https://en.wikipedia.org/wiki/String_interning), stored uniquely
to save memory, and Lix inherits this approach from the original C++ codebase.
However, some builtins, like `builtins.attrNames`, must return a `Value` type
that can represent any Nix value (strings, integers, lists, etc.).
Before this change, these builtins would create lists of `Value` objects by
allocating them through the garbage collector, copying the symbols string
content each time.
This allocation is unnecessary if the interned symbols themselves also hold a
`Value` representation allocated outside the garbage collector, since these
live for the full duration of evaluation.
As a result, this reduces the number of allocations, leading to:
* A significant drop in maximum [resident set memory](https://en.wikipedia.org/wiki/Resident_set_size) (RSS), with some large-scale
tests showing up to 11% (about 500 MiB) savings in large colmena deployments.
* A slight decrease in CPU usage during Nix evaluations.
This change is inspired by https://github.com/NixOS/nix/pull/13258 but the approach is different.
**Note** : [`xokdvium`](https://github.com/xokdvium) is the rightful author of https://gerrit.lix.systems/c/lix/+/3300 and the credit was missed on our end during the development process. We are deeply sorry for this mistake.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [eldritch horrors](https://git.lix.systems/pennae), [Tom Hubrecht](https://git.lix.systems/tom-hubrecht), [xokdvium](https://github.com/xokdvium), and [NaN-git](https://github.com/NaN-git) for this.
## Fixes
- `build-dir` no longer defaults to `temp-dir` [cl/3453](https://gerrit.lix.systems/c/lix/+/3453)
The directory in which temporary build directories are created no longer defaults
to the value of the `temp-dir` setting to avoid builders making their directories
world-accessible. This behavior has been used to escape the build sandbox and can
cause build impurities even when not used maliciously. We now default to `builds`
in `NIX_STATE_DIR` (which is `/nix/var/nix/b` in the default configuration).
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Global certificate authorities are copied inside the builder's environment [gh#12698](https://github.com/NixOS/nix/issues/12698) [fj#885](https://git.lix.systems/lix-project/lix/issues/885) [cl/3765](https://gerrit.lix.systems/c/lix/+/3765)
Previously, CA certificates were only installed at
`/etc/ssl/certs/ca-certificates.crt` for sandboxed builds on Linux.
This setup was insufficient in light of recent changes in `nixpkgs`, which now
enforce HTTPS usage for `fetchurl`, even for fixed-output derivations, to
mitigate confidentiality risks such as `netrc` or credentials leakage.
`nixpkgs` still make use of a special package called `cacerts` which contains a
copy of the CA certificates maintained by Nixpkgs and added as a reference for
TLS-enabled fetchers.
As a result, having a consistent and trusted certificate authority in all
builder environments is becoming more essential.
On `nix-darwin`, the `NIX_SSL_CERT_FILE` environment variable is always
explicitly defined, but it is ignored by the sandbox setup.
Simultaneously, Nix evaluates and propagates impure environment variables via
`lib.proxyImpureEnvVars`, meaning that if `NIX_SSL_CERT_FILE` is set (which
influences the default value for `ssl-cert-file`), it will be forwarded
unchanged into the builder environment.
However, on Linux, Nix also *copies* the CA file into the sandbox, creating a
discrepancy between the value of `NIX_SSL_CERT_FILE` and the actual trusted
certificate path used during the build.
This divergence caused confusion and was partially addressed by attempts to
whitelist the CA path in the Darwin sandbox (see cl/2906), but that approach
involved a non-trivial path canonicalization step and is not as general as this one.
To address this properly, we now emit a warning and override
`NIX_SSL_CERT_FILE` inside the builder, explicitly pointing it to the CA file
copied into the sandbox.
This eliminates ambiguity between `NIX_SSL_CERT_FILE`
and `ssl-cert-file`, ensuring consistent trust anchors across platforms.
This warning might become a hard error as we figure out what to do regarding
`lib.proxyImpureEnvVars` in nixpkgs.
The behavior has been verified across sandboxed and unsandboxed builds on both
Linux and Darwin.
As a consequence of this change, approximately 500KB of CA certificate data is
now unconditionally copied into the build directory for fixed-output
derivations.
While this ensures consistent trust verification without having to restart the
daemon after system upgrades, it may introduce a slight overhead in build
performance. At present, no optimizations have been implemented to avoid this
copy, but if this overhead proves noticeable in your workflows, please open an
issue so we can evaluate and possibly implement different strategies to render
trust anchors visible.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [Emily](https://git.lix.systems/emilazy) for this.
- libstore: exponential backoff for downloads [lix#932](https://git.lix.systems/lix-project/lix/issues/932) [cl/3856](https://gerrit.lix.systems/c/lix/+/3856)
The connection timeout when downloading from e.g. a binary cache is exponentially
increased per failure. The option `connect-timeout` is now an alias to `max-connect-timeout`
which is the maximum value for a timeout. The start value is controlled
by `initial-connect-timeout` which is `5` by default.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- Fix develop shells for derivations with escape codes [fj#991](https://git.lix.systems/lix-project/lix/issues/991) [cl/4154](https://gerrit.lix.systems/c/lix/+/4154) [cl/4155](https://gerrit.lix.systems/c/lix/+/4155)
ASCII control characters (including `\e`, used for ANSI escape codes) in derivation variables are now correctly escaped for `nix develop` and `nix print-dev-env`, instead of erroring.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- nix-store --delete: always remove obsolete hardlinks [cl/3188](https://gerrit.lix.systems/c/lix/+/3188)
Deleting specific paths using `nix-store --delete` or `nix store
delete` previously did not delete hard links created by `nix-store
--optimise` even if they became obsolete, unless _all_ of the given
paths were deleted successfully. Now, hard links are always cleaned
up, even if some of the given paths could not be deleted.
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Report GC statistics correctly [cl/3188](https://gerrit.lix.systems/c/lix/+/3188)
Deleting specific paths using `nix-store --delete` or `nix store delete` previously did
not report statistics correctly when some of the paths could not be deleted, even if
others were deleted:
```
$ nix store delete /nix/store/9bwryidal9q3g91cjm6xschfn4ikd82q-hello-2.12.1 --delete-closure -v
finding garbage collector roots...
deleting '/nix/store/9bwryidal9q3g91cjm6xschfn4ikd82q-hello-2.12.1'
0 store paths deleted, 0.00 MiB freed
error: Cannot delete some of the given paths because they are still alive. Paths not deleted:
k9bxzr1l92r5y6mihrkbpbr3fmc8qszx-libidn2-2.3.8
mbx9ii53lzjlrsnlrfmzpwm33ynljwdn-libunistring-1.3
rf8hcy6bldxdqc0g6q1dcka1vh47x69s-xgcc-14.2.1.20250322-libgcc
vbrdc5wgzn0w1zdp10xd2favkjn5fk7y-glibc-2.40-66
To find out why, use nix-store --query --roots and nix-store --query --referrers.
```
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Fallback to safe temp dir when build-dir is unwritable [fj#876](https://git.lix.systems/lix-project/lix/issues/876) [cl/3501](https://gerrit.lix.systems/c/lix/+/3501)
Non-daemon builds started failing with a permission error after introducing the `build-dir` option:
```
$ nix build --store ~/scratch nixpkgs#hello --rebuild
error: creating directory '/nix/var/nix/builds/nix-build-hello-2.12.2.drv-0': Permission denied
```
This happens because:
1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
2. The user lacks permissions for that path.
We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
**Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Parse overflowing JSON number literals as floatingpoint [cl/3919](https://gerrit.lix.systems/c/lix/+/3919)
Previously, `builtins.fromJSON "-9223372036854775809"` would
return a floatingpoint number, while `builtins.fromJSON
"9223372036854775808"` would cause an evaluation error. This was
introduced with the banning of integer overflow in Lix 2.91; previously
the latter would result in C++ undefined behaviour. These cases are
now treated consistently with JSONs model of a single numeric type,
and JSON number literals that do not fit in a Nixlanguage integer
will be parsed as floatingpoint numbers.
Many thanks to [Emily](https://git.lix.systems/emilazy) for this.
- Fix handling of OSC codes in terminal output [fj#160](https://git.lix.systems/lix-project/lix/issues/160) [cl/3143](https://gerrit.lix.systems/c/lix/+/3143)
OSC codes in terminal output are now handled correctly, where OSC 8 (hyperlink) is preserved any
time color codes are allowed and all other OSC codes are stripped out. This applies not only to
output from build commands but also to rendered documentation in the REPL.
Many thanks to [lilyball](https://git.lix.systems/lilyball) for this.
- Fix nix develop for derivations that rejects dependencies with structured attrs [fj#997](https://git.lix.systems/lix-project/lix/issues/997) [cl/4182](https://gerrit.lix.systems/c/lix/+/4182)
For the sake of concision, we refer to `disallowedReferences` in what follows,
but all output checks were equally fixed:
`{dis,}allowed{References,Requisites}`.
Derivations can define *output checks* to reject unwanted dependencies, such as
interpreters like `bash` or compilers like `gcc`. This can be done in two ways:
* **Legacy style**: `disallowedReferences = [ ... ]` in the environment.
* **Structured attrs**: `outputChecks.<output>.disallowedReferences = [ ... ]`,
typically used in `__json`.
Only the structured form supports derivations with multiple outputs.
`nix develop` internally rewrites derivations to create development shells. It
relied on the legacy `disallowedReferences`, and failed to honor the structured
variant. This led to broken shells in cases where `bashInteractive` was
explicitly disallowed using structured output checks, e.g. `nix develop
nixpkgs#systemd` after the "bash-less NixOS" changes.
This fix teaches `nix develop` to respect structured output checks, restoring
support for such derivations.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- nix-eval-jobs: retain NIX_PATH [cl/3859](https://gerrit.lix.systems/c/lix/+/3859)
`nix-eval-jobs` doesn't clear the `NIX_PATH` from the environment anymore. This matches the behavior
of [upstream version `2.30`](https://github.com/nix-community/nix-eval-jobs/releases/tag/v2.30.0).
Many thanks to [ma27](https://git.lix.systems/ma27) and [mic92](https://github.com/mic92) for this.
- Remove reliance on Bash for remote stores via SSH [fj#830](https://git.lix.systems/lix-project/lix/issues/830) [fj#805](https://git.lix.systems/lix-project/lix/issues/805) [fj#304](https://git.lix.systems/lix-project/lix/issues/304) [cl/3159](https://gerrit.lix.systems/c/lix/+/3159)
The pre-flight `echo started` handshake -- added years ago to catch race conditions -- has been removed.
After removal of connection sharing in Lix 2.93, it required a Bash-compatible shell and a standard `echo`, so it failed on:
* builders protected by `ForceCommand` wrappers (e.g. `nix-remote-build`),
* BusyBox / initrd images with no Bash,
* hosts using non-POSIX shells such as Nushell.
The race the probe once addressed was tied to SSH connection-sharing -- since connection-sharing code has already been removed, the probe is now pointless.
Real connection or protocol errors are now left to SSH/Nix to report directly.
This is technically a breaking change if you had scripts that relied on the literal "started" which needs to be updated to rely on other signals, e.g., exit codes.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- repl-overlays now work in the debugger for flakes [fj#777](https://git.lix.systems/lix-project/lix/issues/777) [cl/3398](https://gerrit.lix.systems/c/lix/+/3398)
Due to a bug, it was previously not possible to use the debugger on flakes with repl-overlays, or with pure evaluation in general:
```
$ nix repl --pure-eval
Lix 2.94.0-dev-pre20250617-87d99da
Type :? for help.
Loading 'repl-overlays'...
error: access to absolute path '/Users/jade/.config/nix/repl.nix' is forbidden in pure eval mode (use '--impure' to override)
```
This is now fixed.
The contents of the repl-overlays file itself (i.e. most typically the top level lambda in it) will be evaluated in impure mode.
It may be necessary to use `builtins.seq` to force the impure operations to happen first if one wants to do impure operations inside a repl-overlays file in pure evaluation mode.
Many thanks to [jade](https://git.lix.systems/jade) for this.
- `nix-shell` default shell directory is not `/tmp` anymore for `$NIX_BUILD_TOP` [fj#940](https://git.lix.systems/lix-project/lix/issues/940)
Previously, Lix `nix-shell`s could exit non-zero status when `stdenv`'s `dumpVars` phase failed to write to `$NIX_BUILD_TOP/env-vars`, despite `dumpVars` being intended as a debugging aid.
This happens when `TMPDIR` is not set and defaults therefore to `/tmp`, resulting in a `/tmp/env-vars` global file that every `nix-shell` wants to write.
We fix this issue by reusing a pre-created, unique, and writable location, as the build top directory, avoiding shell exiting from write failures silently.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- libstore/binary-cache-store: don't cache narinfo on nix copy, remove negative entry [cl/3789](https://gerrit.lix.systems/c/lix/+/3789)
When using e.g. [Snix's nar-bridge](https://snix.dev/docs/components/overview/#nar-bridge) via
an `http`-store, Lix would create cache entries with a wrong URL to the NAR when uploading
a store-path.
This caused hard build failures for Hydra.
Lix doesn't create these entries on upload anymore. Instead, it only removes negative cache entries.
The cache entry for a narinfo is now created the first time, Lix queries the cache
for the previously uploaded store-path again.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- Lix libraries can now be linked statically [fj#789](https://git.lix.systems/lix-project/lix/issues/789) [cl/3775](https://gerrit.lix.systems/c/lix/+/3775) [cl/3778](https://gerrit.lix.systems/c/lix/+/3778)
Previously the pkg-config files distributed with Lix were only suitable for dynamic linkage, causing "undefined reference to…" linker errors when trying to link statically.
Private dependency information has now been added to make static linkage work as expected without user intervention.
In addition, relevant static libraries are now prelinked to avoid strange failures due to missing static initializers.
Many thanks to [alois31](https://git.lix.systems/alois31) for this.
- add description to zsh completions [fj#910](https://git.lix.systems/lix-project/lix/issues/910) [cl/3632](https://gerrit.lix.systems/c/lix/+/3632)
Emit descriptions when completing args in zsh completions. This uses the descriptions we already
provided in NIX\_GET\_COMPLETIONS.
Many thanks to [matthewbauer](https://github.com/matthewbauer) for this.
## Miscellany
- Deprecation of CA derivations, dynamic derivations, and impure derivations [fj#815](https://git.lix.systems/lix-project/lix/issues/815)
Content-addressed derivations are now deprecated and slated for removal in Lix 2.94.
We're doing this because the CA derivation system has been a known cause of problems
and inconsistencies, is unmaintained, habitually makes improving the store code very
difficult (or blocks such improvements outright), and is beset by a number of design
flaws that in our opinion cannot be fixed without a full reimplementation from zero.
Dynamic derivations and impure derivations are built on the CA derivation framework,
and owing to this they too are deprecated and slated for removal in another release.
+5 -44
View File
@@ -1,21 +1,9 @@
#!/usr/bin/env python3
"""
Preprocesses mdbook markdown, primarily for include directives.
The include directive format is as follows:
{{#include foo/bar/baz.md}}
The content of includes will be indented as much as the directive itself.
Including a generated file (from building Lix; generally for the 'new' CLI):
{{#include @generated@/foo/bar/baz.md}}
"""
from pathlib import Path
import json
import os, os.path
import sys
import textwrap
name = 'substitute.py'
@@ -23,12 +11,6 @@ def log(*args, **kwargs):
kwargs['file'] = sys.stderr
return print(f'{name}:', *args, **kwargs)
def remove_prefix_if_present(s: str, prefix: str) -> str | None:
if s.startswith(prefix):
return s.removeprefix(prefix)
else:
return None
def do_include(content: str, relative_md_path: Path, source_root: Path, search_path: Path):
assert not relative_md_path.is_absolute(), f'{relative_md_path=} from mdbook should be relative'
@@ -38,32 +20,16 @@ def do_include(content: str, relative_md_path: Path, source_root: Path, search_p
lines = []
for l in content.splitlines(keepends=True):
if remain := remove_prefix_if_present(l.strip(), "{{#include"):
requested = remain[1:-2]
# We indent bodies of indent directives by the indent of the
# directive itself.
num_leading_indent = len(l) - len(l.lstrip())
if subpath := remove_prefix_if_present(requested, "@generated@/"):
included = search_path / Path(subpath)
if l.strip().startswith("{{#include "):
requested = l.strip()[11:][:-2]
if requested.startswith("@generated@/"):
included = search_path / Path(requested[12:])
requested = included.relative_to(search_path)
else:
included = source_root / relative_md_path.parent / requested
requested = included.resolve().relative_to(source_root)
assert included.exists(), f"{requested} not found at {included}"
lines.append(
textwrap.indent(
do_include(
included.read_text(),
requested,
source_root,
search_path,
),
" " * num_leading_indent
)
+ "\n"
)
lines.append(do_include(included.read_text(), requested, source_root, search_path) + "\n")
else:
lines.append(l)
return "".join(lines)
@@ -90,11 +56,6 @@ def recursive_replace(data, book_root, search_path):
).replace(
'@docroot@',
("../" * len(path_to_chapter.parent.parts) or "./")[:-1]
).replace(
# this replacement is to avoid corrupting the
# hacking.md manual section on docroot
'@\\docroot\\@',
'@docroot@',
),
sub_items = [
recursive_replace(sub_item, book_root, search_path)
+2 -2
View File
@@ -8,7 +8,7 @@
tag ? "latest",
bundleNixpkgs ? true,
channelName ? "nixpkgs",
channelURL ? "https://channels.nixos.org/nixpkgs-unstable",
channelURL ? "https://nixos.org/channels/nixpkgs-unstable",
extraPkgs ? [ ],
maxLayers ? 100,
nixConf ? { },
@@ -156,7 +156,7 @@ let
nixConfContents =
(lib.concatStringsSep "\n" (
lib.mapAttrsToList (
lib.mapAttrsFlatten (
n: v:
let
vStr = if builtins.isList v then lib.concatStringsSep " " v else v;
+84 -179
View File
@@ -48,7 +48,6 @@
sgr = builtins.fromJSON ''"\u001b["'';
freezePage = "https://wiki.lix.systems/books/lix-contributors/page/freezes-and-recommended-contributions";
codebaseOverview = "https://wiki.lix.systems/books/lix-contributors/page/codebase-overview";
gerritWiki = "https://wiki.lix.systems/books/lix-contributors/page/gerrit";
contribNotice = builtins.toFile "lix-contrib-notice" ''
Hey there!
@@ -66,10 +65,6 @@
and we'd like to work together with all contributors as much as possible.
Lix is a collaborative project :)
If you want to submit a patch and you never used gerrit before, please
check our gerrit wiki section:
${sgr}32m${gerritWiki}${sgr}0m
You can open an issue at https://git.lix.systems/lix-project/lix/issues
or chat with us on Matrix: #space:lix.systems.
@@ -88,7 +83,7 @@
else
"pre${
builtins.substring 0 8 (self.lastModifiedDate or self.lastModified or "19700101")
}-dev_${self.shortRev or "dirty"}";
}_${self.shortRev or "dirty"}";
linux32BitSystems = [ "i686-linux" ];
linux64BitSystems = [
@@ -100,7 +95,6 @@
"x86_64-darwin"
"aarch64-darwin"
];
nonDarwinSystems = linuxSystems;
systems = linuxSystems ++ darwinSystems;
# If you add something here, please update the list in doc/manual/src/contributing/hacking.md.
@@ -110,7 +104,8 @@
"armv7l-linux"
"riscv64-linux"
"aarch64-linux"
"x86_64-freebsd"
# FIXME: still broken in 24.05: fails to build rustc(??) due to missing -lstdc++ dep
# "x86_64-freebsd"
# FIXME: broken dev shell due to python
# "x86_64-netbsd"
];
@@ -224,55 +219,11 @@
lix-clang-tidy = final.callPackage ./subprojects/lix-clang-tidy { };
nix-eval-jobs = final.callPackage ./subprojects/nix-eval-jobs {
srcDir = ./subprojects/nix-eval-jobs;
};
# HACK: We need nix-prefetch-git for fetchCargoVendor for Rust stuff,
# so it can't use Lix, or we infrec:
# lix -> Rust stuff -> fetchCargoVendor -> nix-prefetch-git -> nix (lix)
# This will eventually become a problem upstream, but until then,
# apply some duct tape and pray.
nix-prefetch-git =
if (lib.functionArgs prev.nix-prefetch-git.override) ? "nix" then
prev.nix-prefetch-git.override { nix = prev.nix; }
else
prev.nix-prefetch-git;
# Export the patched version of boehmgc that Lix uses into the overlay
# for consumers of this flake.
boehmgc-nix = final.nix.passthru.boehmgc-nix;
# And same thing for our build-release-notes package.
build-release-notes = final.nix.passthru.build-release-notes;
lowdown_1_3 =
# If the stable channel we are using ships lowdown >= 1.4, we need
# to swap this around, take the default lowdown from the stable
# channel and add an overridden one for the legacy version.
assert lib.versionOlder prev.lowdown.version "1.4.0";
prev.lowdown;
lowdown = prev.lowdown.overrideAttrs (prevAttrs: rec {
version = "2.0.2";
src = final.fetchurl {
url = "https://kristaps.bsd.lv/lowdown/snapshots/lowdown-${version}.tar.gz";
sha512 = "2a4d0rqh8gkw4ca3gkzddp0hjpmmw74cbks8k0inhh0vizmgbn188zdv6m1kgmr019b99g7insli8js3ci1ji7y4n5nk704bswf3z3i";
};
nativeBuildInputs = prevAttrs.nativeBuildInputs ++ [ final.buildPackages.bmake ];
postInstall = lib.replaceStrings [ "lowdown.so.1" ] [ "lowdown.so.2" ] prevAttrs.postInstall;
});
capnproto = prev.capnproto.overrideAttrs (old: {
patches =
old.patches or [ ]
++ [
# backport of https://github.com/capnproto/capnproto/pull/1810
./misc/capnproto-promise-nodiscard.patch
]
++ lib.optionals (lib.versionOlder old.version "1.2.0") [
# backport of https://github.com/capnproto/capnproto/pull/2296
./misc/capnproto-monotonic-clocks-are-a-lie.patch
];
});
};
in
{
@@ -287,30 +238,6 @@
# Binary package for various platforms.
build = forAllSystems (system: self.packages.${system}.nix);
# Building Lix twice in CI is expensive, but we can catch a lot of static
# build regressions by at least making sure it evals and configures.
configure-static = lib.genAttrs linux64BitSystems (
system:
self.packages.${system}.nix-static.overrideAttrs {
dontBuild = true;
installPhase = ''
runHook preInstall
echo "configure-static complete. exiting with success"
mkdir -p "$out"
exit 0
'';
}
);
# Ensure support for lowdown < 1.4 doesn't regress
build-lowdown_1_3 = forAllSystems (
system:
self.packages.${system}.nix.override {
lowdown = nixpkgsFor.${system}.native.lowdown_1_3;
}
);
devShell = forAllSystems (system: {
default = self.devShells.${system}.default;
clang = self.devShells.${system}.native-clangStdenvPackages;
@@ -341,9 +268,6 @@
# Perl bindings for various platforms.
perlBindings = forAllSystems (system: nixpkgsFor.${system}.native.nix.passthru.perl-bindings);
# nix-eval-jobs can be built against this Lix.
nix-eval-jobs = forAllSystems (system: nixpkgsFor.${system}.native.nix-eval-jobs);
# Binary tarball for various platforms, containing a Nix store
# with the closure of 'nix' package.
binaryTarball = forAllSystems (system: nixpkgsFor.${system}.native.nix.passthru.binaryTarball);
@@ -375,104 +299,90 @@
});
# System tests.
tests =
import ./tests/nixos {
inherit
self
lib
nixpkgs
nixpkgsFor
;
}
// {
# the n-e-j test suite is unusably slow in darwin ci. disbled until anywho fixes this.
nix-eval-jobs = (lib.genAttrs nonDarwinSystems) (
system: self.packages.${system}.nix-eval-jobs.tests.nix-eval-jobs
);
tests = import ./tests/nixos { inherit lib nixpkgs nixpkgsFor; } // {
# This is x86_64-linux only, just because we have significantly
# cheaper x86_64-linux compute in CI.
# It is clangStdenv because clang's sanitizers are nicer.
asanBuild = self.packages.x86_64-linux.nix-clangStdenv.override {
# Improve caching of non-code changes by not changing the
# derivation name every single time, since this will never be seen
# by users anyway.
versionSuffix = "";
sanitize = [
"address"
"undefined"
];
# it is very hard to make *every* CI build use this option such
# that we don't wind up building Lix twice, so we do it here where
# we are already doing so.
werror = true;
};
# This is x86_64-linux only, just because we have significantly
# cheaper x86_64-linux compute in CI.
# It is clangStdenv because clang's sanitizers are nicer.
asanBuild = self.packages.x86_64-linux.nix-clangStdenv.override {
# Improve caching of non-code changes by not changing the
# derivation name every single time, since this will never be seen
# by users anyway.
# Although this might be nicer to do with pre-commit, that would
# require adding 12MB of nodejs to the dev shell, whereas building it
# in CI with Nix avoids that at a cost of slower feedback on rarely
# touched files.
jsSyntaxCheck =
let
nixpkgs = nixpkgsFor.x86_64-linux.native;
inherit (nixpkgs) pkgs;
docSources = lib.fileset.toSource {
root = ./doc;
fileset = lib.fileset.fileFilter (f: f.hasExt "js") ./doc;
};
in
pkgs.runCommand "js-syntax-check" { } ''
find ${docSources} -type f -print -exec ${pkgs.nodejs-slim}/bin/node --check '{}' ';'
touch $out
'';
# clang-tidy run against the Lix codebase using the Lix clang-tidy plugin
clang-tidy =
let
nixpkgs = nixpkgsFor.x86_64-linux.native;
inherit (nixpkgs) pkgs;
in
pkgs.callPackage ./package.nix {
# Required since we don't support gcc stdenv
stdenv = pkgs.clangStdenv;
versionSuffix = "";
sanitize = [
"address"
"undefined"
];
# it is very hard to make *every* CI build use this option such
# that we don't wind up building Lix twice, so we do it here where
# we are already doing so.
werror = true;
lintInsteadOfBuild = true;
};
# Although this might be nicer to do with pre-commit, that would
# require adding 12MB of nodejs to the dev shell, whereas building it
# in CI with Nix avoids that at a cost of slower feedback on rarely
# touched files.
jsSyntaxCheck =
let
nixpkgs = nixpkgsFor.x86_64-linux.native;
inherit (nixpkgs) pkgs;
docSources = lib.fileset.toSource {
root = ./doc;
fileset = lib.fileset.fileFilter (f: f.hasExt "js") ./doc;
};
in
pkgs.runCommand "js-syntax-check" { } ''
find ${docSources} -type f -print -exec ${pkgs.nodejs-slim}/bin/node --check '{}' ';'
touch $out
'';
# Make sure that nix-env still produces the exact same result
# on a particular version of Nixpkgs.
evalNixpkgs =
with nixpkgsFor.x86_64-linux.native;
runCommand "eval-nixos" { buildInputs = [ nix ]; } ''
type -p nix-env
# Note: we're filtering out nixos-install-tools because https://github.com/NixOS/nixpkgs/pull/153594#issuecomment-1020530593.
time nix-env --store dummy:// -f ${nixpkgs-regression} -qaP --drv-path | sort | grep -v nixos-install-tools > packages
[[ $(sha1sum < packages | cut -c1-40) = 402242fca90874112b34718b8199d844e8b03d12 ]]
mkdir $out
'';
# clang-tidy run against the Lix codebase using the Lix clang-tidy plugin
clang-tidy =
let
nixpkgs = nixpkgsFor.x86_64-linux.native;
inherit (nixpkgs) pkgs;
in
pkgs.callPackage ./package.nix {
# Required since we don't support gcc stdenv
stdenv = pkgs.clangStdenv;
versionSuffix = "";
lintInsteadOfBuild = true;
};
# Make sure that nix-env still produces the exact same result
# on a particular version of Nixpkgs.
evalNixpkgs =
with nixpkgsFor.x86_64-linux.native;
runCommand "eval-nixos" { buildInputs = [ nix ]; } ''
type -p nix-env
# Note: we're filtering out nixos-install-tools because https://github.com/NixOS/nixpkgs/pull/153594#issuecomment-1020530593.
time nix-env --store dummy:// -f ${nixpkgs-regression} -qaP --drv-path | sort | grep -v nixos-install-tools > packages
[[ $(sha1sum < packages | cut -c1-40) = 402242fca90874112b34718b8199d844e8b03d12 ]]
mkdir $out
'';
nixpkgsLibTests = forAllSystems (
system:
let
inherit (self.packages.${system}) nix;
pkgs = nixpkgsFor.${system}.native;
testWithNix = import (nixpkgs + "/lib/tests/test-with-nix.nix") { inherit pkgs lib nix; };
in
pkgs.symlinkJoin {
name = "nixpkgs-lib-tests";
paths = [
testWithNix
]
# NOTE: nixpkgs 25.05 is being ... *creative*, and requires this dance to override
# the evaluator used for the test. it will break again in the future, don't worry.
++ lib.optionals pkgs.stdenv.isLinux [
((pkgs.callPackage "${nixpkgs}/ci/eval" { inherit nix; }).attrpathsSuperset {
evalSystem = system;
})
];
}
);
};
nixpkgsLibTests = forAllSystems (
system:
let
inherit (self.packages.${system}) nix;
pkgs = nixpkgsFor.${system}.native;
testWithNix = import (nixpkgs + "/lib/tests/test-with-nix.nix") { inherit pkgs lib nix; };
in
pkgs.symlinkJoin {
name = "nixpkgs-lib-tests";
paths = [
testWithNix
]
# NOTE: nixpkgs 25.05 is being ... *creative*, and requires this dance to override
# the evaluator used for the test. it will break again in the future, don't worry.
++ lib.optionals pkgs.stdenv.isLinux [
((pkgs.callPackage "${nixpkgs}/ci/eval" { inherit nix; }).attrpathsSuperset {
evalSystem = system;
})
];
}
);
};
pre-commit = forAvailableSystems (
system:
@@ -518,10 +428,8 @@
# devShells and packages already get checked by nix flake check, so
# this is just jobs that are special
build-lowdown_1_3 = self.hydraJobs.build-lowdown_1_3.${system};
binaryTarball = self.hydraJobs.binaryTarball.${system};
perlBindings = self.hydraJobs.perlBindings.${system};
nix-eval-jobs = self.hydraJobs.nix-eval-jobs.${system};
nixpkgsLibTests = self.hydraJobs.tests.nixpkgsLibTests.${system};
rl-next = self.hydraJobs.rl-next.${system}.user;
# Will be empty attr set on i686-linux, and filtered out by forAvailableSystems.
@@ -539,14 +447,11 @@
inherit (nixpkgsFor.${system}.native) nix;
default = nix;
inherit (nixpkgsFor.${system}.native) lix-clang-tidy nix-eval-jobs;
inherit (nixpkgsFor.${system}.native) lix-clang-tidy;
}
// (
lib.optionalAttrs (builtins.elem system linux64BitSystems) {
# python doesn't work in static builds as of 2025-06-27
nix-static = nixpkgsFor.${system}.static.nix.overrideAttrs (_: {
doCheck = false;
});
nix-static = nixpkgsFor.${system}.static.nix;
dockerImage =
let
pkgs = nixpkgsFor.${system}.native;
+7 -28
View File
@@ -8,51 +8,30 @@ list:
clean:
rm -rf build
# Prepare meson for building with extra options
setup-custom *OPTIONS:
# Prepare meson for building
setup *OPTIONS:
meson setup build --prefix="$PWD/outputs/out" $mesonFlags {{ OPTIONS }}
# Prepare meson for building
setup: (setup-custom)
# Build lix with extra options
build-custom *OPTIONS:
meson compile -C build {{ OPTIONS }}
# Build lix
build: (build-custom)
build *OPTIONS:
meson compile -C build {{ OPTIONS }}
alias compile := build
# Install lix for local development with extra options
install-custom *OPTIONS: (build-custom OPTIONS)
# Install lix for local development
install *OPTIONS: (build OPTIONS)
meson install -C build
# Install lix for local development
install: (install-custom)
# Run tests (usually requires `install`) with extra options
# Run tests
test *OPTIONS:
meson test -C build --print-errorlogs {{ OPTIONS }}
# Run unit tests only
test-unit *OPTIONS: (test "--suite" "check")
# Run integration tests only
test-integration *OPTIONS: install (test "--suite" "installcheck")
# Run functional2 tests using pytest directly, allowing for additional arguments to be passed to pytest e.g. for more granular test selection
test-functional2 *OPTIONS:
cd tests && python -m pytest -v {{ OPTIONS }} functional2
alias clang-tidy := lint
# Lint with `clang-tidy`
lint:
ninja -C build clang-tidy
alias clang-tidy-fix := lint-fix
# Fix lints with `clang-tidy-fix`
lint-fix:
ninja -C build clang-tidy-fix
-10
View File
@@ -1,10 +0,0 @@
# This reproduces the Lix Approvers plus Lix groups to yield the status quo
alois1@gmx-topmail.de
jade@lix.systems
lunaphied@lunaphied.me
maximilian@mbosch.me
me@0upti.me
pennae@lix.systems
qyriad@qyriad.me
raito@lix.systems
rbt@sent.as
-64
View File
@@ -1,64 +0,0 @@
#!@python@
import argparse
import capnp
from pathlib import Path
import os
import subprocess
import sys
if lang := os.environ.get('lix_capnp_lang'):
outputs = os.environ['lix_capnp_outputs'].split()
old_cwd = os.environ['lix_capnp_old_cwd']
schema = capnp.load('@capnp_include@/capnp/schema.capnp', imports=['@capnp_include@'])
request = schema.CodeGeneratorRequest.read(sys.stdin)
subprocess.run([lang], input=request.as_builder().to_bytes()).check_returncode()
base_dir = os.getcwd()
os.chdir(old_cwd)
include = [ str(Path(p).resolve()) for p in os.environ['lix_capnp_include'].split(':') ]
if depfile := os.environ['lix_capnp_depfile']:
deps = ""
for input in request.requestedFiles:
deps += " ".join(f"{input.filename}.{o}" for o in outputs)
deps += ":"
for dep in input.imports:
if dep.name.startswith("/"):
for candidate in (Path(i + dep.name) for i in include):
if candidate.exists():
deps += " " + str(candidate)
break
else:
raise RuntimeError("not handling relative includes")
deps += "\n\n"
Path(depfile).write_text(deps)
else:
parser = argparse.ArgumentParser()
parser.add_argument('--language')
parser.add_argument('--outdir')
parser.add_argument('--src-prefix')
parser.add_argument('--depfile', default="")
parser.add_argument('-I', '--include', action='append', default=['@capnp_include@'])
parser.add_argument('inputs', nargs='+')
args = parser.parse_args()
for infile in args.inputs:
os.environ['lix_capnp_lang'] = f"capnpc-{args.language}"
os.environ['lix_capnp_include'] = ':'.join(args.include)
os.environ['lix_capnp_depfile'] = args.depfile
os.environ['lix_capnp_old_cwd'] = os.getcwd()
if args.language == "c++":
os.environ['lix_capnp_outputs'] = "c++ h"
else:
raise RuntimeError("unknown language " + args.language)
subprocess.run([
'@capnp@',
'compile',
f'-o{sys.argv[0]}:{args.outdir}',
f'--src-prefix={args.src_prefix}',
*(f"-I{i}" for i in args.include),
infile
]).check_returncode()
+235 -479
View File
@@ -1,22 +1,7 @@
#include "lix/libstore/path.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/error.hh"
#include "lix/libutil/file-descriptor.hh"
#include "lix/libutil/logging-rpc.hh"
#include "lix/libutil/logging.hh"
#include "lix/libutil/rpc.hh"
#include "lix/libutil/types-rpc.hh" // IWYU pragma: keep
#include "lix/libutil/types.hh"
#include <algorithm>
#include <capnp/rpc-twoparty.h>
#include <cstring>
#include <exception>
#include <kj/async.h>
#include <kj/time.h>
#include <chrono>
#include <set>
#include <memory>
#include <string>
#include <tuple>
#if __APPLE__
#include <sys/time.h>
@@ -32,29 +17,13 @@
#include "lix/libstore/derivations.hh"
#include "lix/libutil/strings.hh"
#include "lix/libstore/local-store.hh"
#include "lix/libstore/types-rpc.hh"
#include "lix/libcmd/legacy.hh"
#include "lix/libutil/experimental-features.hh"
#include "lix/libutil/hash.hh"
#include "build-remote.hh"
#include "lix/libstore/build/hook-instance.capnp.h"
namespace nix {
namespace {
struct Instance final : rpc::build_remote::HookInstance::Server
{
unsigned int maxBuildJobs;
bool initialized = false, used = false;
kj::Promise<void> init(InitContext context) override;
kj::Promise<void> buildImpl(BuildContext context);
kj::Promise<void> build(BuildContext context) override;
};
}
std::string escapeUri(std::string uri)
{
std::replace(uri.begin(), uri.end(), '/', '_');
@@ -84,278 +53,11 @@ static bool allSupportedLocally(Store & store, const std::set<std::string>& requ
return true;
}
static std::tuple<bool, Machine *, AutoCloseFD> selectBestMachine(
Machines & machines,
const std::string & neededSystem,
const std::set<std::string> & requiredFeatures
)
{
bool rightType = false;
Machine * bestMachine = nullptr;
AutoCloseFD bestSlotLock;
uint64_t bestLoad = 0;
for (auto & m : machines) {
debug("considering building on remote machine '%s'", m.storeUri);
if (m.enabled && m.systemSupported(neededSystem) && m.allSupported(requiredFeatures)
&& m.mandatoryMet(requiredFeatures))
{
rightType = true;
AutoCloseFD free;
uint64_t load = 0;
for (uint64_t slot = 0; slot < m.maxJobs; ++slot) {
auto slotLock = openSlotLock(m, slot);
if (tryLockFile(slotLock.get(), ltWrite)) {
if (!free) {
free = std::move(slotLock);
}
} else {
++load;
}
}
if (!free) {
continue;
}
bool best = false;
if (!bestSlotLock) {
best = true;
} else if (load / m.speedFactor < bestLoad / bestMachine->speedFactor) {
best = true;
} else if (load / m.speedFactor == bestLoad / bestMachine->speedFactor) {
if (m.speedFactor > bestMachine->speedFactor) {
best = true;
} else if (m.speedFactor == bestMachine->speedFactor) {
if (load < bestLoad) {
best = true;
}
}
}
if (best) {
bestLoad = load;
bestSlotLock = std::move(free);
bestMachine = &m;
}
}
}
return {rightType, bestMachine, std::move(bestSlotLock)};
}
static void printSelectionFailureMessage(
Verbosity level,
const std::string_view drvstr,
const Machines & machines,
const std::string & neededSystem,
const std::set<std::string> & requiredFeatures
)
{
std::string machinesFormatted;
for (auto & m : machines) {
machinesFormatted += HintFmt(
"\n([%s], %s, [%s], [%s])",
concatStringsSep<StringSet>(", ", m.systemTypes),
m.maxJobs,
concatStringsSep<StringSet>(", ", m.supportedFeatures),
concatStringsSep<StringSet>(", ", m.mandatoryFeatures)
)
.str();
}
printMsg(
level,
"Failed to find a machine for remote build!\n"
"derivation: %s\n"
"required (system, features): (%s, [%s])\n"
"%s available machines:\n"
"(systems, maxjobs, supportedFeatures, mandatoryFeatures)%s",
drvstr,
neededSystem,
concatStringsSep<StringSet>(", ", requiredFeatures),
machines.size(),
Uncolored(machinesFormatted)
);
}
namespace {
struct BuilderConnection
{
AutoCloseFD slotLock;
std::shared_ptr<Store> sshStore;
std::string storeUri;
Pipe logPipe;
// start the thread that reads ssh stderr and turns it into log items.
// this future *must* outlive sshStore, otherwise it will never finish
kj::Promise<Result<void>> startLogThread(std::string buildDescription, std::string drvPath)
try {
if (!logPipe.readSide) {
co_return result::success();
}
logPipe.writeSide.close();
// NOTE this is very similar to handleBuilderOutput in DerivationGoal, but unlike
// the derivation goal we do not need to handle EIO from a pty here. we also have
// no timeouts or limits to keep track of, which makes deduplication less useful.
auto act = logger->startActivity(
lvlInfo, actBuild, buildDescription, Logger::Fields{drvPath, storeUri, 1, 1}
);
std::map<ActivityId, Activity> activities;
auto reader = AIO().lowLevelProvider.wrapInputFd(logPipe.readSide.get());
LogLineSplitter splitter;
auto flushLine = [&](const std::string & line) {
if (const auto state =
handleJSONLogMessage(line, act, activities, "the derivation builder"))
{
return *state;
} else {
return act.result(resBuildLogLine, line);
}
};
auto buf = kj::heapArray<char>(4096);
while (true) {
const auto got = co_await reader->tryRead(buf.begin(), 1, buf.size());
if (got == 0) {
break;
}
std::string_view data{buf.begin(), got};
while (!data.empty()) {
if (auto line = splitter.feed(data)) {
if (flushLine(*line) == Logger::BufferState::NeedsFlush) {
TRY_AWAIT(act.getLogger().flush());
}
}
}
}
if (auto line = splitter.finish(); !line.empty()) {
(void) flushLine(line);
TRY_AWAIT(act.getLogger().flush());
}
co_return result::success();
} catch (...) {
co_return result::current_exception();
}
};
struct AcceptedBuild final : rpc::build_remote::HookInstance::AcceptedBuild::Server
{
ref<Store> store;
StorePath drvPath;
BuilderConnection builder;
bool used = false;
AcceptedBuild(ref<Store> store, StorePath drvPath, BuilderConnection builder)
: store(store)
, drvPath(drvPath)
, builder(std::move(builder))
{
}
kj::Promise<void> runImpl(RunContext context);
kj::Promise<void> run(RunContext context) override;
};
enum class BuildRejected { Temporarily, Permanently };
}
static kj::Promise<Result<std::variant<BuildRejected, BuilderConnection>>> connectToBuilder(
const ref<Store> & store,
const StorePath & drvPath,
Machines & machines,
const unsigned int maxBuildJobs,
const bool amWilling,
const std::string & neededSystem,
const std::set<std::string> & requiredFeatures
)
try {
AutoCloseFD bestSlotLock;
/* It would be possible to build locally after some builds clear out,
so don't show the warning now: */
bool couldBuildLocally = maxBuildJobs > 0
&& (neededSystem == settings.thisSystem
|| settings.extraPlatforms.get().count(neededSystem) > 0)
&& allSupportedLocally(*store, requiredFeatures);
/* It's possible to build this locally right now: */
bool canBuildLocally = amWilling && couldBuildLocally;
/* Error ignored here, will be caught later */
(void) sys::mkdir(currentLoad, 0777);
while (true) {
bestSlotLock.reset();
AutoCloseFD lock = openLockFile(currentLoad + "/main-lock", true);
TRY_AWAIT(lockFileAsync(lock.get(), ltWrite));
auto [rightType, bestMachine, slotLock] =
selectBestMachine(machines, neededSystem, requiredFeatures);
bestSlotLock = std::move(slotLock);
if (!bestSlotLock) {
if (rightType && !canBuildLocally) {
co_return BuildRejected::Temporarily;
} else {
printSelectionFailureMessage(
couldBuildLocally ? lvlChatty : lvlWarn,
drvPath.to_string(),
machines,
neededSystem,
requiredFeatures
);
co_return BuildRejected::Permanently;
}
}
#if __APPLE__
futimes(bestSlotLock.get(), nullptr);
#else
futimens(bestSlotLock.get(), nullptr);
#endif
lock.reset();
std::shared_ptr<Store> sshStore;
Pipe logPipe;
try {
auto act = logger->startActivity(
lvlTalkative, actUnknown, fmt("connecting to '%s'", bestMachine->storeUri)
);
std::tie(sshStore, logPipe) = TRY_AWAIT(bestMachine->openStore());
TRY_AWAIT(sshStore->connect());
co_return BuilderConnection{
std::move(bestSlotLock), sshStore, bestMachine->storeUri, std::move(logPipe)
};
} catch (std::exception & e) { // NOLINT(lix-foreign-exceptions)
std::string msg = logPipe.readSide ? chomp(drainFD(logPipe.readSide.get(), false)) : "";
printError(
"cannot build on '%s': %s%s",
bestMachine->storeUri,
e.what(),
msg.empty() ? "" : ": " + msg
);
bestMachine->enabled = false;
}
}
} catch (...) {
co_return result::current_exception();
}
static int main_build_remote(AsyncIoRoot & aio, std::string programName, Strings argv)
static int main_build_remote(std::string programName, Strings argv)
{
{
logger = makeJSONLogger(*logger);
/* Ensure we don't get any SSH passphrase or host key popups. */
unsetenv("DISPLAY");
unsetenv("SSH_ASKPASS");
@@ -368,181 +70,227 @@ static int main_build_remote(AsyncIoRoot & aio, std::string programName, Strings
verbosity = (Verbosity) std::stoll(argv.front());
auto conn = aio.kj.lowLevelProvider->wrapUnixSocketFd(1);
capnp::TwoPartyServer srv(kj::heap<Instance>());
srv.accept(*conn, 1).wait(aio.kj.waitScope);
return 0;
}
}
kj::Promise<void> Instance::init(InitContext context)
{
try {
if (initialized) {
throw Error("build hook can only be initialized once");
}
logger = rpc::log::makeRpcLoggerClient(context.getParams().getLogger());
FdSource source(STDIN_FILENO);
/* Read the parent's settings. */
for (const auto & [name, value] : rpc::to<StringMap>(context.getParams().getSettings())) {
while (readInt(source)) {
auto name = readString(source);
auto value = readString(source);
settings.set(name, value);
}
maxBuildJobs = settings.maxBuildJobs;
auto maxBuildJobs = settings.maxBuildJobs;
settings.maxBuildJobs.set("1"); // hack to make tests with local?root= work
initPlugins();
initialized = true;
auto store = openStore();
context.getResults().initResult().setGood();
} catch (...) {
RPC_FILL(context.getResults(), initResult, std::current_exception());
}
/* It would be more appropriate to use $XDG_RUNTIME_DIR, since
that gets cleared on reboot, but it wouldn't work on macOS. */
auto currentLoadName = "/current-load";
if (auto localStore = store.dynamic_pointer_cast<LocalFSStore>())
currentLoad = std::string { localStore->config().stateDir } + currentLoadName;
else
currentLoad = settings.nixStateDir + currentLoadName;
return kj::READY_NOW;
}
std::shared_ptr<Store> sshStore;
AutoCloseFD bestSlotLock;
kj::Promise<void> Instance::buildImpl(BuildContext context)
{
if (!initialized) {
throw Error("build hook not fully initialized");
}
auto machines = getMachines();
debug("got %d remote builders", machines.size());
// FIXME this does not open a daemon connection for historical reasons.
// we may create a lot of build hook instances, and having each of them
// also create a daemon instance is inefficient and wasteful. in future
// versions of the build hook (where we don't need one hook process per
// build) we should change this to using a daemon connection, ideally a
// daemon connection provided by the parent via file descriptor passing
auto store = TRY_AWAIT(openStore(settings.storeUri, {}, AllowDaemon::Disallow));
/* It would be more appropriate to use $XDG_RUNTIME_DIR, since
that gets cleared on reboot, but it wouldn't work on macOS. */
auto currentLoadName = "/current-load";
if (auto localStore = store.try_cast_shared<LocalFSStore>()) {
currentLoad = std::string{localStore->config().stateDir} + currentLoadName;
} else {
currentLoad = settings.nixStateDir + currentLoadName;
}
auto machines = getMachines();
debug("got %d remote builders", machines.size());
if (machines.empty()) {
context.getResults().initResult().initGood().setDeclinePermanently();
co_return;
}
auto amWilling = context.getParams().getAmWilling();
auto neededSystem = rpc::to<std::string>(context.getParams().getNeededSystem());
auto drvPath = from(context.getParams().getDrvPath(), *store);
auto requiredFeatures =
rpc::to<std::set<std::string>>(context.getParams().getRequiredFeatures());
auto result = TRY_AWAIT(connectToBuilder(
store, drvPath, machines, maxBuildJobs, amWilling, neededSystem, requiredFeatures
));
if (auto immediateResponse = std::get_if<BuildRejected>(&result)) {
switch (*immediateResponse) {
case BuildRejected::Temporarily:
context.getResults().initResult().initGood().setPostpone();
co_return;
case BuildRejected::Permanently:
context.getResults().initResult().initGood().setDecline();
co_return;
if (machines.empty()) {
std::cerr << "# decline-permanently\n";
return 0;
}
}
auto builder = std::get_if<BuilderConnection>(&result);
assert(builder);
std::optional<StorePath> drvPath;
std::string storeUri;
auto ac = context.getResults().initResult().initGood().initAccept();
ac.setMachine(kj::heap<AcceptedBuild>(store, drvPath, std::move(*builder)));
}
while (true) {
kj::Promise<void> Instance::build(BuildContext context)
try {
if (used) {
throw Error("build hooks can only accept a single job");
}
used = true; // lock out other rpc calls during processing
co_await buildImpl(context);
TRY_AWAIT(logger->flush());
used = context.getResults().getResult().getGood().isAccept();
} catch (...) {
RPC_FILL(context.getResults(), getResult, std::current_exception());
}
try {
auto s = readString(source);
if (s != "try") return 0;
} catch (EndOfFile &) { return 0; }
kj::Promise<void> AcceptedBuild::run(RunContext context)
{
try {
auto oldLogger = logger;
logger = rpc::log::makeRpcLoggerClient(context.getParams().getLogger());
TRY_AWAIT(oldLogger->flush());
KJ_DEFER({
delete logger;
logger = oldLogger;
});
auto amWilling = readInt(source);
auto neededSystem = readString(source);
drvPath = store->parseStorePath(readString(source));
auto requiredFeatures = readStrings<std::set<std::string>>(source);
if (used) {
throw Error("build hooks builds are single-use items");
/* It would be possible to build locally after some builds clear out,
so don't show the warning now: */
bool couldBuildLocally = maxBuildJobs > 0
&& ( neededSystem == settings.thisSystem
|| settings.extraPlatforms.get().count(neededSystem) > 0)
&& allSupportedLocally(*store, requiredFeatures);
/* It's possible to build this locally right now: */
bool canBuildLocally = amWilling && couldBuildLocally;
/* Error ignored here, will be caught later */
mkdir(currentLoad.c_str(), 0777);
while (true) {
bestSlotLock.reset();
AutoCloseFD lock = openLockFile(currentLoad + "/main-lock", true);
lockFile(lock.get(), ltWrite);
bool rightType = false;
Machine * bestMachine = nullptr;
uint64_t bestLoad = 0;
for (auto & m : machines) {
debug("considering building on remote machine '%s'", m.storeUri);
if (m.enabled &&
m.systemSupported(neededSystem) &&
m.allSupported(requiredFeatures) &&
m.mandatoryMet(requiredFeatures))
{
rightType = true;
AutoCloseFD free;
uint64_t load = 0;
for (uint64_t slot = 0; slot < m.maxJobs; ++slot) {
auto slotLock = openSlotLock(m, slot);
if (tryLockFile(slotLock.get(), ltWrite)) {
if (!free) {
free = std::move(slotLock);
}
} else {
++load;
}
}
if (!free) {
continue;
}
bool best = false;
if (!bestSlotLock) {
best = true;
} else if (load / m.speedFactor < bestLoad / bestMachine->speedFactor) {
best = true;
} else if (load / m.speedFactor == bestLoad / bestMachine->speedFactor) {
if (m.speedFactor > bestMachine->speedFactor) {
best = true;
} else if (m.speedFactor == bestMachine->speedFactor) {
if (load < bestLoad) {
best = true;
}
}
}
if (best) {
bestLoad = load;
bestSlotLock = std::move(free);
bestMachine = &m;
}
}
}
if (!bestSlotLock) {
if (rightType && !canBuildLocally)
std::cerr << "# postpone\n";
else
{
// add the template values.
std::string drvstr;
if (drvPath.has_value())
drvstr = drvPath->to_string();
else
drvstr = "<unknown>";
std::string machinesFormatted;
for (auto & m : machines) {
machinesFormatted += HintFmt(
"\n([%s], %s, [%s], [%s])",
concatStringsSep<StringSet>(", ", m.systemTypes),
m.maxJobs,
concatStringsSep<StringSet>(", ", m.supportedFeatures),
concatStringsSep<StringSet>(", ", m.mandatoryFeatures)
).str();
}
auto error = HintFmt(
"Failed to find a machine for remote build!\n"
"derivation: %s\n"
"required (system, features): (%s, [%s])\n"
"%s available machines:\n"
"(systems, maxjobs, supportedFeatures, mandatoryFeatures)%s",
drvstr,
neededSystem,
concatStringsSep<StringSet>(", ", requiredFeatures),
machines.size(),
Uncolored(machinesFormatted)
);
printMsg(couldBuildLocally ? lvlChatty : lvlWarn, error.str());
std::cerr << "# decline\n";
}
break;
}
#if __APPLE__
futimes(bestSlotLock.get(), nullptr);
#else
futimens(bestSlotLock.get(), nullptr);
#endif
lock.reset();
try {
Activity act(*logger, lvlTalkative, actUnknown, fmt("connecting to '%s'", bestMachine->storeUri));
sshStore = bestMachine->openStore();
sshStore->connect();
storeUri = bestMachine->storeUri;
} catch (std::exception & e) {
auto msg = chomp(drainFD(5, false));
printError("cannot build on '%s': %s%s",
bestMachine->storeUri, e.what(),
msg.empty() ? "" : ": " + msg);
bestMachine->enabled = false;
continue;
}
goto connected;
}
}
used = true;
co_await runImpl(context);
TRY_AWAIT(logger->flush());
} catch (...) {
RPC_FILL(context.getResults(), getResult, std::current_exception());
}
}
kj::Promise<void> AcceptedBuild::runImpl(RunContext context)
{
try {
auto logHandler = builder.startLogThread(
fmt("%s on '%s'",
rpc::to<std::string_view>(context.getParams().getDescription()),
builder.storeUri),
store->printStorePath(drvPath)
);
connected:
close(5);
auto & sshStore = builder.sshStore;
auto & storeUri = builder.storeUri;
assert(sshStore);
auto inputs = rpc::to<std::set<StorePath>>(context.getParams().getInputs(), *store);
auto wantedOutputs = rpc::to<std::set<std::string>>(context.getParams().getWantedOutputs());
std::cerr << "# accept\n" << storeUri << "\n";
auto inputs = readStrings<PathSet>(source);
auto wantedOutputs = readStrings<StringSet>(source);
auto lockFileName = currentLoad + "/" + makeLockFilename(storeUri) + ".upload-lock";
AutoCloseFD uploadLock = openLockFile(lockFileName, true);
{
auto act = logger->startActivity(
lvlTalkative, actUnknown, fmt("waiting for the upload lock to '%s'", storeUri)
);
Activity act(*logger, lvlTalkative, actUnknown, fmt("waiting for the upload lock to '%s'", storeUri));
auto result = TRY_AWAIT(
AIO().timeoutAfter(15 * kj::MINUTES, lockFileAsync(uploadLock.get(), ltWrite))
);
if (!result) {
if (!unsafeLockFileSingleThreaded(uploadLock.get(), ltWrite, std::chrono::minutes(15)))
printError("somebody is hogging the upload lock for '%s', continuing...");
}
}
auto substitute = settings.buildersUseSubstitutes ? Substitute : NoSubstitute;
{
auto act = logger->startActivity(
lvlTalkative, actUnknown, fmt("copying dependencies to '%s'", storeUri)
);
TRY_AWAIT(copyPaths(*store, *sshStore, inputs, NoRepair, NoCheckSigs, substitute));
Activity act(*logger, lvlTalkative, actUnknown, fmt("copying dependencies to '%s'", storeUri));
copyPaths(*store, *sshStore, store->parseStorePathSet(inputs), NoRepair, NoCheckSigs, substitute);
}
uploadLock.reset();
auto drv = TRY_AWAIT(store->readDerivation(drvPath));
auto drv = store->readDerivation(*drvPath);
std::optional<BuildResult> optResult;
@@ -550,7 +298,7 @@ kj::Promise<void> AcceptedBuild::runImpl(RunContext context)
// stores), we assume we are. This is necessary for backwards
// compat.
bool trustedOrLegacy = ({
std::optional trusted = TRY_AWAIT(sshStore->isTrustedClient());
std::optional trusted = sshStore->isTrustedClient();
!trusted || *trusted;
});
@@ -569,66 +317,74 @@ kj::Promise<void> AcceptedBuild::runImpl(RunContext context)
//
// 2. Changing the `inputSrcs` set changes the associated
// output ids, which break CA derivations
if (!drv.inputDrvs.empty()) {
drv.inputSrcs = inputs;
}
optResult =
TRY_AWAIT(sshStore->buildDerivation(drvPath, (const BasicDerivation &) drv));
if (!drv.inputDrvs.map.empty())
drv.inputSrcs = store->parseStorePathSet(inputs);
optResult = sshStore->buildDerivation(*drvPath, (const BasicDerivation &) drv);
auto & result = *optResult;
if (!result.success())
throw Error("build of '%s' on '%s' failed: %s", store->printStorePath(*drvPath), storeUri, result.errorMsg);
} else {
TRY_AWAIT(copyClosure(
*store, *sshStore, StorePathSet{drvPath}, NoRepair, NoCheckSigs, substitute
));
auto res = TRY_AWAIT(sshStore->buildPathsWithResults({DerivedPath::Built{
.drvPath = makeConstantStorePath(drvPath),
.outputs = OutputsSpec::All{},
}}));
copyClosure(*store, *sshStore, StorePathSet {*drvPath}, NoRepair, NoCheckSigs, substitute);
auto res = sshStore->buildPathsWithResults({
DerivedPath::Built {
.drvPath = makeConstantStorePathRef(*drvPath),
.outputs = OutputsSpec::All {},
}
});
// One path to build should produce exactly one build result
assert(res.size() == 1);
optResult = std::move(res[0]);
}
auto & result = *optResult;
if (!result.success()) {
throw Error(
"build of '%s' on '%s' failed: %s",
store->printStorePath(drvPath),
storeUri,
result.errorMsg
);
}
auto outputHashes = staticOutputHashes(*store, drv);
std::set<Realisation> missingRealisations;
StorePathSet missingPaths;
auto outputPaths = drv.outputsAndPaths(*store);
for (auto & [outputName, outputPath] : outputPaths) {
if (!TRY_AWAIT(store->isValidPath(outputPath.second))) {
missingPaths.insert(outputPath.second);
if (experimentalFeatureSettings.isEnabled(Xp::CaDerivations) && !drv.type().hasKnownOutputPaths()) {
for (auto & outputName : wantedOutputs) {
auto thisOutputHash = outputHashes.at(outputName);
auto thisOutputId = DrvOutput{ thisOutputHash, outputName };
if (!store->queryRealisation(thisOutputId)) {
debug("missing output %s", outputName);
assert(optResult);
auto & result = *optResult;
auto i = result.builtOutputs.find(outputName);
assert(i != result.builtOutputs.end());
auto & newRealisation = i->second;
missingRealisations.insert(newRealisation);
missingPaths.insert(newRealisation.outPath);
}
}
} else {
auto outputPaths = drv.outputsAndOptPaths(*store);
for (auto & [outputName, hopefullyOutputPath] : outputPaths) {
assert(hopefullyOutputPath.second);
if (!store->isValidPath(*hopefullyOutputPath.second))
missingPaths.insert(*hopefullyOutputPath.second);
}
}
if (!missingPaths.empty()) {
auto act = logger->startActivity(
lvlTalkative, actUnknown, fmt("copying outputs from '%s'", storeUri)
);
if (auto localStore = store.try_cast_shared<LocalStore>())
Activity act(*logger, lvlTalkative, actUnknown, fmt("copying outputs from '%s'", storeUri));
if (auto localStore = store.dynamic_pointer_cast<LocalStore>())
for (auto & path : missingPaths)
localStore->locksHeld.insert(store->printStorePath(path)); /* FIXME: ugly */
TRY_AWAIT(
copyPaths(*sshStore, *store, missingPaths, NoRepair, NoCheckSigs, NoSubstitute)
);
copyPaths(*sshStore, *store, missingPaths, NoRepair, NoCheckSigs, NoSubstitute);
}
// XXX: Should be done as part of `copyPaths`
for (auto & realisation : missingRealisations) {
// Should hold, because if the feature isn't enabled the set
// of missing realisations should be empty
experimentalFeatureSettings.require(Xp::CaDerivations);
store->registerDrvOutput(realisation);
}
// drop store connection, let log handler process any remaining input
builder.sshStore = nullptr;
TRY_AWAIT(logHandler);
context.getResults().initResult().setGood();
} catch (...) {
RPC_FILL(context.getResults(), initResult, std::current_exception());
return 0;
}
}
void registerLegacyBuildRemote() {
LegacyCommandRegistry::add("build-remote", main_build_remote);
void registerBuildRemote() {
LegacyCommands::add("build-remote", main_build_remote);
}
}
+1 -1
View File
@@ -3,6 +3,6 @@
namespace nix {
void registerLegacyBuildRemote();
void registerBuildRemote();
}
+3 -8
View File
@@ -1,7 +1,5 @@
#include "dotgraph.hh"
#include "lix/libstore/store-api.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/result.hh"
#include <iostream>
@@ -43,8 +41,8 @@ static std::string makeNode(std::string_view id, std::string_view label,
}
kj::Promise<Result<void>> printDotGraph(ref<Store> store, StorePathSet && roots)
try {
void printDotGraph(ref<Store> store, StorePathSet && roots)
{
StorePathSet workList(std::move(roots));
StorePathSet doneSet;
@@ -57,7 +55,7 @@ try {
cout << makeNode(std::string(path.to_string()), path.name(), "#ff0000");
for (auto & p : TRY_AWAIT(store->queryPathInfo(path))->references) {
for (auto & p : store->queryPathInfo(path)->references) {
if (p != path) {
workList.insert(p);
cout << makeEdge(std::string(p.to_string()), std::string(path.to_string()));
@@ -66,9 +64,6 @@ try {
}
cout << "}\n";
co_return result::success();
} catch (...) {
co_return result::current_exception();
}
+1 -1
View File
@@ -5,6 +5,6 @@
namespace nix {
kj::Promise<Result<void>> printDotGraph(ref<Store> store, StorePathSet && roots);
void printDotGraph(ref<Store> store, StorePathSet && roots);
}
+3 -8
View File
@@ -1,8 +1,6 @@
#include "graphml.hh"
#include "lix/libstore/store-api.hh"
#include "lix/libstore/derivations.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/result.hh"
#include <iostream>
@@ -48,8 +46,8 @@ static std::string makeNode(const ValidPathInfo & info)
}
kj::Promise<Result<void>> printGraphML(ref<Store> store, StorePathSet && roots)
try {
void printGraphML(ref<Store> store, StorePathSet && roots)
{
StorePathSet workList(std::move(roots));
StorePathSet doneSet;
std::pair<StorePathSet::iterator, bool> ret;
@@ -69,7 +67,7 @@ try {
ret = doneSet.insert(path);
if (ret.second == false) continue;
auto info = TRY_AWAIT(store->queryPathInfo(path));
auto info = store->queryPathInfo(path);
cout << makeNode(*info);
for (auto & p : info->references) {
@@ -83,9 +81,6 @@ try {
cout << "</graph>\n";
cout << "</graphml>\n";
co_return result::success();
} catch (...) {
co_return result::current_exception();
}
+1 -1
View File
@@ -5,6 +5,6 @@
namespace nix {
kj::Promise<Result<void>> printGraphML(ref<Store> store, StorePathSet && roots);
void printGraphML(ref<Store> store, StorePathSet && roots);
}
+123 -119
View File
@@ -1,27 +1,28 @@
#include <cstring>
#include <fstream>
#include <iostream>
#include <filesystem>
#include <regex>
#include <sstream>
#include <vector>
#include <map>
#include <nlohmann/json.hpp>
#include "lix/libstore/parsed-derivations.hh"
#include "lix/libstore/store-api.hh"
#include "lix/libstore/local-fs-store.hh"
#include "lix/libstore/globals.hh"
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/current-process.hh"
#include "lix/libstore/derivations.hh"
#include "lix/libmain/shared.hh"
#include "lix/libstore/path-with-outputs.hh"
#include "lix/libexpr/eval.hh"
#include "lix/libexpr/eval-inline.hh"
#include "lix/libexpr/get-drvs.hh"
#include "lix/libcmd/common-eval-args.hh"
#include "lix/libexpr/attr-path.hh"
#include "lix/libcmd/legacy.hh"
#include "lix/libutil/finally.hh"
#include "lix/libutil/processes.hh"
#include "lix/libutil/regex.hh"
#include "lix/libutil/shlex.hh"
#include "nix-build.hh"
#include "lix/libstore/temporary-dir.hh"
@@ -32,10 +33,10 @@ namespace nix {
using namespace std::string_literals;
static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings argv)
static void main_nix_build(std::string programName, Strings argv)
{
auto dryRun = false;
auto runEnv = std::regex_search(programName, regex::parse("nix-shell$"));
auto runEnv = std::regex_search(programName, std::regex("nix-shell$"));
auto pure = false;
auto fromArgs = false;
auto packages = false;
@@ -72,7 +73,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
script = argv.front();
try {
auto lines = tokenizeString<Strings>(readFile(script), "\n");
if (std::regex_search(lines.front(), regex::parse("^#!"))) {
if (std::regex_search(lines.front(), std::regex("^#!"))) {
lines.pop_front();
inShebang = true;
savedArgs = {std::next(argv.begin()), argv.end()};
@@ -80,7 +81,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
for (auto line : lines) {
line = chomp(line);
std::smatch match;
if (std::regex_match(line, match, regex::parse("^#!\\s*nix-shell\\s+(.*)$")))
if (std::regex_match(line, match, std::regex("^#!\\s*nix-shell\\s+(.*)$")))
for (const auto & word : shell_split(match[1].str()))
argv.push_back(word);
}
@@ -93,7 +94,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
using LegacyArgs::LegacyArgs;
};
MyArgs myArgs(aio, myName, [&](Strings::iterator & arg, const Strings::iterator & end) {
MyArgs myArgs(myName, [&](Strings::iterator & arg, const Strings::iterator & end) {
if (*arg == "--help") {
showManPage(myName);
}
@@ -152,14 +153,14 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
// executes it unless it contains the string "perl" or "indir",
// or (undocumented) argv[0] does not contain "perl". Exploit
// the latter by doing "exec -a".
if (std::regex_search(interpreter, regex::parse("perl")))
if (std::regex_search(interpreter, std::regex("perl")))
execArgs = "-a PERL";
std::ostringstream joined;
for (const auto & i : savedArgs)
joined << shellEscape(i) << ' ';
if (std::regex_search(interpreter, regex::parse("ruby"))) {
if (std::regex_search(interpreter, std::regex("ruby"))) {
// Hack for Ruby. Ruby also examines the shebang. It tries to
// read the shebang to understand which packages to read from. Since
// this is handled via nix-shell -p, we wrap our ruby script execution
@@ -190,17 +191,16 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
if (packages && fromArgs)
throw UsageError("'-p' and '-E' are mutually exclusive");
AutoDelete tmpDir(createTempDir(myName));
AutoDelete buildTopTmpDir(createTempSubdir(tmpDir, "build-top"));
AutoDelete tmpDir(createTempDir("", myName));
if (outLink.empty())
outLink = (Path) tmpDir + "/result";
auto store = aio.blockOn(openStore());
auto evalStore = myArgs.evalStoreUrl ? aio.blockOn(openStore(*myArgs.evalStoreUrl)) : store;
auto store = openStore();
auto evalStore = myArgs.evalStoreUrl ? openStore(*myArgs.evalStoreUrl) : store;
auto evaluator = std::make_unique<Evaluator>(aio, myArgs.searchPath, evalStore, store);
auto evaluator = std::make_unique<Evaluator>(myArgs.searchPath, evalStore, store);
evaluator->repair = myArgs.repair;
auto state = evaluator->begin(aio);
auto state = evaluator->begin();
if (myArgs.repair) buildMode = bmRepair;
auto autoArgs = myArgs.getAutoArgs(*evaluator);
@@ -228,9 +228,8 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
left = {"default.nix"};
}
if (runEnv) {
(void) sys::setenv("IN_NIX_SHELL", pure ? "pure" : "impure", 1);
}
if (runEnv)
setenv("IN_NIX_SHELL", pure ? "pure" : "impure", 1);
DrvInfos drvs;
@@ -250,15 +249,15 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
} catch (Error & e) {};
auto [path, outputNames] = parsePathWithOutputs(absolute);
if (evalStore->isStorePath(path) && path.ends_with(".drv"))
drvs.push_back(aio.blockOn(DrvInfo::create(evalStore, absolute)));
drvs.push_back(DrvInfo(evalStore, absolute));
else
/* If we're in a #! script, interpret filenames
relative to the script. */
exprs.push_back(evaluator->parseExprFromFile(
evaluator->paths.resolveExprPath(aio.blockOn(lookupFileArg(
evaluator->paths.resolveExprPath(lookupFileArg(
*evaluator,
inShebang && !packages ? absPath(i, absPath(dirOf(script))) : i
)).unwrap())
))
));
}
}
@@ -276,13 +275,11 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
return false;
}
bool add = false;
if (v.type() == nFunction) {
if (auto pattern = dynamic_cast<AttrsPattern *>(v.lambda().fun->pattern.get())) {
for (auto & i : pattern->formals) {
if (evaluator->symbols[i.name] == "inNixShell") {
add = true;
break;
}
if (v.type() == nFunction && v.lambda.fun->hasFormals()) {
for (auto & i : v.lambda.fun->formals->formals) {
if (evaluator->symbols[i.name] == "inNixShell") {
add = true;
break;
}
}
}
@@ -290,13 +287,13 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
};
for (auto & i : attrPaths) {
Value v(
findAlongAttrPath(
*state, i, takesNixShellAttr(vRoot) ? *autoArgsWithInNixShell : *autoArgs, vRoot
)
.first
);
state->forceValue(v, noPos);
Value & v(*findAlongAttrPath(
*state,
i,
takesNixShellAttr(vRoot) ? *autoArgsWithInNixShell : *autoArgs,
vRoot
).first);
state->forceValue(v, v.determinePos(noPos));
getDerivations(
*state,
v,
@@ -315,17 +312,14 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
fetch binary cache data. */
uint64_t downloadSize, narSize;
StorePathSet willBuild, willSubstitute, unknown;
aio.blockOn(store->queryMissing(paths,
willBuild, willSubstitute, unknown, downloadSize, narSize));
store->queryMissing(paths,
willBuild, willSubstitute, unknown, downloadSize, narSize);
if (settings.printMissing) {
aio.blockOn(printMissing(
ref<Store>(store), willBuild, willSubstitute, unknown, downloadSize, narSize
));
}
if (settings.printMissing)
printMissing(ref<Store>(store), willBuild, willSubstitute, unknown, downloadSize, narSize);
if (!dryRun)
aio.blockOn(store->buildPaths(paths, buildMode, evalStore));
store->buildPaths(paths, buildMode, evalStore);
};
if (runEnv) {
@@ -333,7 +327,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
throw UsageError("nix-shell requires a single derivation");
auto & drvInfo = drvs.front();
auto drv = aio.blockOn(evalStore->derivationFromPath(drvInfo.requireDrvPath(*state)));
auto drv = evalStore->derivationFromPath(drvInfo.requireDrvPath(*state));
std::vector<DerivedPath> pathsToBuild;
RealisedPath::Set pathsToCopy;
@@ -360,7 +354,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
auto bashDrv = drv->requireDrvPath(*state);
pathsToBuild.push_back(DerivedPath::Built {
.drvPath = makeConstantStorePath(bashDrv),
.drvPath = makeConstantStorePathRef(bashDrv),
.outputs = OutputsSpec::Names {"out"},
});
pathsToCopy.insert(bashDrv);
@@ -373,25 +367,31 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
}
}
auto accumDerivedPath = [&](SingleDerivedPath::Opaque inputDrv, const StringSet & inputNode) {
if (!inputNode.empty())
std::function<void(ref<SingleDerivedPath>, const DerivedPathMap<StringSet>::ChildNode &)> accumDerivedPath;
accumDerivedPath = [&](ref<SingleDerivedPath> inputDrv, const DerivedPathMap<StringSet>::ChildNode & inputNode) {
if (!inputNode.value.empty())
pathsToBuild.push_back(DerivedPath::Built {
.drvPath = inputDrv,
.outputs = OutputsSpec::Names { inputNode },
.outputs = OutputsSpec::Names { inputNode.value },
});
for (const auto & [outputName, childNode] : inputNode.childMap)
accumDerivedPath(
make_ref<SingleDerivedPath>(SingleDerivedPath::Built { inputDrv, outputName }),
childNode);
};
// Build or fetch all dependencies of the derivation.
for (const auto & [inputDrv0, inputNode] : drv.inputDrvs) {
for (const auto & [inputDrv0, inputNode] : drv.inputDrvs.map) {
// To get around lambda capturing restrictions in the
// standard.
const auto & inputDrv = inputDrv0;
if (std::all_of(envExclude.cbegin(), envExclude.cend(),
[&](const std::string & exclude) {
return !std::regex_search(store->printStorePath(inputDrv), regex::parse(exclude));
return !std::regex_search(store->printStorePath(inputDrv), std::regex(exclude));
}))
{
accumDerivedPath(makeConstantStorePath(inputDrv), inputNode);
accumDerivedPath(makeConstantStorePathRef(inputDrv), inputNode);
pathsToCopy.insert(inputDrv);
}
}
@@ -402,14 +402,17 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
buildPaths(pathsToBuild);
if (dryRun) {
return 0;
}
if (dryRun) return;
if (shellDrv) {
auto shellDrvOutputs =
aio.blockOn(store->queryDerivationOutputMap(shellDrv.value(), &*evalStore));
shell = store->printStorePath(shellDrvOutputs.at("out")) + "/bin/bash";
auto shellDrvOutputs = store->queryPartialDerivationOutputMap(shellDrv.value(), &*evalStore);
shell = store->printStorePath(shellDrvOutputs.at("out").value()) + "/bin/bash";
}
if (experimentalFeatureSettings.isEnabled(Xp::CaDerivations)) {
auto resolvedDrv = drv.tryResolve(*store);
assert(resolvedDrv && "Successfully resolved the derivation");
drv = *resolvedDrv;
}
// Set the environment.
@@ -426,45 +429,48 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
}
// Don't use defaultTempDir() here! We want to preserve the user's TMPDIR for the shell
env["NIX_BUILD_TOP"] = env["TMPDIR"] = env["TEMPDIR"] = env["TMP"] = env["TEMP"] =
getEnvNonEmpty("TMPDIR").value_or(buildTopTmpDir);
env["NIX_BUILD_TOP"] = env["TMPDIR"] = env["TEMPDIR"] = env["TMP"] = env["TEMP"] = getEnvNonEmpty("TMPDIR").value_or("/tmp");
env["NIX_STORE"] = store->config().storeDir;
env["NIX_BUILD_CORES"] = std::to_string(settings.buildCores);
auto passAsFile = tokenizeString<StringSet>(getOr(drv.env, "passAsFile", ""));
bool keepTmp = false;
int fileNr = 0;
for (auto & var : drv.env)
if (passAsFile.count(var.first)) {
keepTmp = true;
auto fn = ".attr-" + std::to_string(fileNr++);
Path p = (Path) tmpDir + "/" + fn;
writeFile(p, var.second);
env[var.first + "Path"] = p;
} else {
} else
env[var.first] = var.second;
}
std::string structuredAttrsRC;
if (env.count("__json")) {
StorePathSet inputs;
auto accumInputClosure = [&](const StorePath & inputDrv, const StringSet & inputNode) {
auto outputs =
aio.blockOn(store->queryDerivationOutputMap(inputDrv, &*evalStore));
for (auto & i : inputNode) {
std::function<void(const StorePath &, const DerivedPathMap<StringSet>::ChildNode &)> accumInputClosure;
accumInputClosure = [&](const StorePath & inputDrv, const DerivedPathMap<StringSet>::ChildNode & inputNode) {
auto outputs = store->queryPartialDerivationOutputMap(inputDrv, &*evalStore);
for (auto & i : inputNode.value) {
auto o = outputs.at(i);
aio.blockOn(store->computeFSClosure(o, inputs));
store->computeFSClosure(*o, inputs);
}
for (const auto & [outputName, childNode] : inputNode.childMap)
accumInputClosure(*outputs.at(outputName), childNode);
};
for (const auto & [inputDrv, inputNode] : drv.inputDrvs)
for (const auto & [inputDrv, inputNode] : drv.inputDrvs.map)
accumInputClosure(inputDrv, inputNode);
ParsedDerivation parsedDrv(drvInfo.requireDrvPath(*state), drv);
if (auto structAttrs = aio.blockOn(parsedDrv.prepareStructuredAttrs(*store, inputs))) {
if (auto structAttrs = parsedDrv.prepareStructuredAttrs(*store, inputs)) {
auto json = structAttrs.value();
structuredAttrsRC = writeStructuredAttrsShell(json);
@@ -476,6 +482,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
env["NIX_ATTRS_SH_FILE"] = attrsSH;
env["NIX_ATTRS_JSON_FILE"] = attrsJSON;
keepTmp = true;
}
}
@@ -485,13 +492,24 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
lose the current $PATH directories. */
auto rcfile = (Path) tmpDir + "/rc";
auto tz = getEnv("TZ");
std::string rc =
fmt("%1%"
// always clear PATH.
// when nix-shell is run impure, we rehydrate it with the `p=$PATH` above
"unset PATH;"
"dontAddDisableDepTrack=1;\n",
(pure ? "" : "[ -n \"$PS1\" ] && [ -e ~/.bashrc ] && source ~/.bashrc; p=$PATH; "));
std::string rc = fmt(
R"(_nix_shell_clean_tmpdir() { command rm -rf %1%; }; )"
"%2%"
"%3%"
// always clear PATH.
// when nix-shell is run impure, we rehydrate it with the `p=$PATH` above
"unset PATH;"
"dontAddDisableDepTrack=1;\n",
shellEscape(tmpDir),
(keepTmp
? "trap _nix_shell_clean_tmpdir EXIT; "
"exitHooks+=(_nix_shell_clean_tmpdir); "
"failureHooks+=(_nix_shell_clean_tmpdir); "
: "_nix_shell_clean_tmpdir; "),
(pure
? ""
: "[ -n \"$PS1\" ] && [ -e ~/.bashrc ] && source ~/.bashrc; p=$PATH; ")
);
rc += structuredAttrsRC;
rc += fmt(
"\n[ -e $stdenv/setup ] && source $stdenv/setup; "
@@ -521,38 +539,27 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
vomit("Sourcing nix-shell with file %s and contents:\n%s", rcfile, rc);
writeFile(rcfile, rc);
auto args = interactive ? Strings{"--rcfile", rcfile} : Strings{rcfile};
Strings envStrs;
for (auto & i : env)
envStrs.push_back(i.first + "=" + i.second);
auto args = interactive
? Strings{"bash", "--rcfile", rcfile}
: Strings{"bash", rcfile};
auto envPtrs = stringsToCharPtrs(envStrs);
environ = envPtrs.data();
auto argPtrs = stringsToCharPtrs(args);
restoreProcessContext();
// We are going to run an interactive command, do not let the logger send a line.
logger->pause();
printMsg(lvlChatty, "running shell: %s", concatMapStringsSep(" ", args, shellEscape));
execvp(shell->c_str(), argPtrs.data());
RunningProgram proc = runProgram2(
{.program = *shell,
.searchPath = true,
.args = args,
.environment = env,
.dieWithParent = true}
);
// NOTE: we wait and return the status check immediately.
// If there's interruption, we will swallow it and wait again for termination.
auto toExitStatus = [](int waitRes) {
if (WIFEXITED(waitRes)) {
return WEXITSTATUS(waitRes);
} else if (WIFSIGNALED(waitRes)) {
return 128 + WTERMSIG(waitRes);
} else {
return 255;
}
};
try {
return toExitStatus(proc.wait());
} catch (Interrupted &) {
return toExitStatus(proc.wait());
}
throw SysError("executing shell '%s'", *shell);
}
else {
@@ -571,7 +578,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
throw Error("derivation '%s' lacks an 'outputName' attribute", store->printStorePath(drvPath));
pathsToBuild.push_back(DerivedPath::Built{
.drvPath = makeConstantStorePath(drvPath),
.drvPath = makeConstantStorePathRef(drvPath),
.outputs = OutputsSpec::Names{outputName},
});
pathsToBuildOrdered.push_back({drvPath, {outputName}});
@@ -586,9 +593,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
buildPaths(pathsToBuild);
if (dryRun) {
return 0;
}
if (dryRun) return;
std::vector<StorePath> outPaths;
@@ -598,15 +603,16 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
if (counter)
drvPrefix += fmt("-%d", counter + 1);
auto builtOutputs =
aio.blockOn(store->queryDerivationOutputMap(drvPath, &*evalStore));
auto builtOutputs = store->queryPartialDerivationOutputMap(drvPath, &*evalStore);
auto outputPath = builtOutputs.at(outputName);
auto maybeOutputPath = builtOutputs.at(outputName);
assert(maybeOutputPath);
auto outputPath = *maybeOutputPath;
if (auto store2 = store.try_cast_shared<LocalFSStore>()) {
if (auto store2 = store.dynamic_pointer_cast<LocalFSStore>()) {
std::string symlink = drvPrefix;
if (outputName != "out") symlink += "-" + outputName;
aio.blockOn(store2->addPermRoot(outputPath, absPath(symlink)));
store2->addPermRoot(outputPath, absPath(symlink));
}
outPaths.push_back(outputPath);
@@ -617,13 +623,11 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
for (auto & path : outPaths)
std::cout << store->printStorePath(path) << '\n';
}
return 0;
}
void registerLegacyNixBuildAndNixShell() {
LegacyCommandRegistry::add("nix-build", main_nix_build);
LegacyCommandRegistry::add("nix-shell", main_nix_build);
void registerNixBuildAndNixShell() {
LegacyCommands::add("nix-build", main_nix_build);
LegacyCommands::add("nix-shell", main_nix_build);
}
}
+1 -1
View File
@@ -3,6 +3,6 @@
namespace nix {
void registerLegacyNixBuildAndNixShell();
void registerNixBuildAndNixShell();
}
+30 -61
View File
@@ -7,10 +7,6 @@
#include "lix/libfetchers/fetchers.hh"
#include "lix/libexpr/eval-settings.hh" // for defexpr
#include "lix/libstore/temporary-dir.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/regex.hh"
#include "lix/libutil/result.hh"
#include "lix/libutil/users.hh"
#include "nix-channel.hh"
@@ -33,10 +29,10 @@ static void readChannels()
for (const auto & line : tokenizeString<std::vector<std::string>>(channelsFile, "\n")) {
chomp(line);
if (std::regex_search(line, regex::parse("^\\s*\\#")))
if (std::regex_search(line, std::regex("^\\s*\\#")))
continue;
auto split = tokenizeString<std::vector<std::string>>(line, " ");
auto url = std::regex_replace(split[0], regex::parse("/*$"), "");
auto url = std::regex_replace(split[0], std::regex("/*$"), "");
auto name = split.size() > 1 ? split[1] : std::string(baseNameOf(url));
channels[name] = url;
}
@@ -45,7 +41,7 @@ static void readChannels()
// Writes the list of channels.
static void writeChannels()
{
auto channelsFD = sys::open(channelsList, O_WRONLY | O_CLOEXEC | O_CREAT | O_TRUNC, 0644);
auto channelsFD = AutoCloseFD{open(channelsList.c_str(), O_WRONLY | O_CLOEXEC | O_CREAT | O_TRUNC, 0644)};
if (!channelsFD)
throw SysError("opening '%1%' for writing", channelsList);
for (const auto & channel : channels)
@@ -55,9 +51,9 @@ static void writeChannels()
// Adds a channel.
static void addChannel(const std::string & url, const std::string & name)
{
if (!regex_search(url, regex::parse("^(file|http|https)://")))
if (!regex_search(url, std::regex("^(file|http|https)://")))
throw Error("invalid channel URL '%1%'", url);
if (!regex_search(name, regex::parse("^[a-zA-Z0-9_][a-zA-Z0-9_\\.-]*$")))
if (!regex_search(name, std::regex("^[a-zA-Z0-9_][a-zA-Z0-9_\\.-]*$")))
throw Error("invalid channel identifier '%1%'", name);
readChannels();
channels[name] = url;
@@ -67,28 +63,23 @@ static void addChannel(const std::string & url, const std::string & name)
static Path profile;
// Remove a channel.
static kj::Promise<Result<void>> removeChannel(const std::string & name)
try {
static void removeChannel(const std::string & name)
{
readChannels();
channels.erase(name);
writeChannels();
TRY_AWAIT(runProgram(
settings.nixBinDir + "/nix-env", true, {"--profile", profile, "--uninstall", name}
));
co_return result::success();
} catch (...) {
co_return result::current_exception();
runProgram(settings.nixBinDir + "/nix-env", true, { "--profile", profile, "--uninstall", name });
}
static Path nixDefExpr;
// Fetch Nix expressions and binary cache URLs from the subscribed channels.
static void update(AsyncIoRoot & aio, const StringSet & channelNames)
static void update(const StringSet & channelNames)
{
readChannels();
auto store = aio.blockOn(openStore());
auto store = openStore();
auto [fd, unpackChannelPath] = createTempFile();
writeFull(fd.get(),
@@ -109,7 +100,7 @@ static void update(AsyncIoRoot & aio, const StringSet & channelNames)
auto cname = name;
std::smatch match;
auto urlBase = std::string(baseNameOf(url));
if (std::regex_search(urlBase, match, regex::parse("(-\\d.*)$")))
if (std::regex_search(urlBase, match, std::regex("(-\\d.*)$")))
cname = cname + match.str(1);
std::string extraAttrs;
@@ -123,41 +114,24 @@ static void update(AsyncIoRoot & aio, const StringSet & channelNames)
// We want to download the url to a file to see if it's a tarball while also checking if we
// got redirected in the process, so that we can grab the various parts of a nix channel
// definition from a consistent location if the redirect changes mid-download.
auto result = aio.blockOn(fetchers::downloadFile(
store,
url,
std::string(baseNameOf(url)),
false
));
auto result = fetchers::downloadFile(store, url, std::string(baseNameOf(url)), false);
auto filename = store->toRealPath(result.storePath);
url = result.effectiveUrl;
bool unpacked = false;
if (std::regex_search(filename, regex::parse("\\.tar\\.(gz|bz2|xz)$"))) {
aio.blockOn(runProgram(
settings.nixBinDir + "/nix-build",
false,
{"--no-out-link",
"--expr",
"import " + unpackChannelPath + "{ name = \"" + cname + "\"; channelName = \""
+ name + "\"; src = builtins.storePath \"" + filename + "\"; }"}
));
if (std::regex_search(filename, std::regex("\\.tar\\.(gz|bz2|xz)$"))) {
runProgram(settings.nixBinDir + "/nix-build", false, { "--no-out-link", "--expr", "import " + unpackChannelPath +
"{ name = \"" + cname + "\"; channelName = \"" + name + "\"; src = builtins.storePath \"" + filename + "\"; }" });
unpacked = true;
}
if (!unpacked) {
// Download the channel tarball.
std::optional<fetchers::DownloadFileResult> exprs;
try {
exprs = aio.blockOn(fetchers::downloadFile(
store, url + "/nixexprs.tar.xz", "nixexprs.tar.xz", false
));
filename = store->toRealPath(fetchers::downloadFile(store, url + "/nixexprs.tar.xz", "nixexprs.tar.xz", false).storePath);
} catch (FileTransferError & e) {
exprs = aio.blockOn(fetchers::downloadFile(
store, url + "/nixexprs.tar.bz2", "nixexprs.tar.bz2", false
));
filename = store->toRealPath(fetchers::downloadFile(store, url + "/nixexprs.tar.bz2", "nixexprs.tar.bz2", false).storePath);
}
filename = store->toRealPath(exprs->storePath);
}
// Regardless of where it came from, add the expression representing this channel to accumulated expression
exprs.push_back("f: f { name = \"" + cname + "\"; channelName = \"" + name + "\"; src = builtins.storePath \"" + filename + "\"; " + extraAttrs + " }");
@@ -171,16 +145,15 @@ static void update(AsyncIoRoot & aio, const StringSet & channelNames)
for (auto & expr : exprs)
envArgs.push_back(std::move(expr));
envArgs.push_back("--quiet");
aio.blockOn(runProgram(settings.nixBinDir + "/nix-env", false, envArgs));
runProgram(settings.nixBinDir + "/nix-env", false, envArgs);
// Make the channels appear in nix-env.
struct stat st;
if (sys::lstat(nixDefExpr, &st) == 0) {
if (lstat(nixDefExpr.c_str(), &st) == 0) {
if (S_ISLNK(st.st_mode))
// old-skool ~/.nix-defexpr
if (sys::unlink(nixDefExpr) == -1) {
if (unlink(nixDefExpr.c_str()) == -1)
throw SysError("unlinking %1%", nixDefExpr);
}
} else if (errno != ENOENT) {
throw SysError("getting status of %1%", nixDefExpr);
}
@@ -189,7 +162,7 @@ static void update(AsyncIoRoot & aio, const StringSet & channelNames)
replaceSymlink(profile, channelLink);
}
static int main_nix_channel(AsyncIoRoot & aio, std::string programName, Strings argv)
static int main_nix_channel(std::string programName, Strings argv)
{
{
// Figure out the name of the `.nix-channels' file to use
@@ -211,7 +184,7 @@ static int main_nix_channel(AsyncIoRoot & aio, std::string programName, Strings
cRollback
} cmd = cNone;
std::vector<std::string> args;
LegacyArgs(aio, programName, [&](Strings::iterator & arg, const Strings::iterator & end) {
LegacyArgs(programName, [&](Strings::iterator & arg, const Strings::iterator & end) {
if (*arg == "--help") {
showManPage("nix-channel");
} else if (*arg == "--version") {
@@ -249,8 +222,8 @@ static int main_nix_channel(AsyncIoRoot & aio, std::string programName, Strings
name = args[1];
} else {
name = baseNameOf(url);
name = std::regex_replace(name, regex::parse("-unstable$"), "");
name = std::regex_replace(name, regex::parse("-stable$"), "");
name = std::regex_replace(name, std::regex("-unstable$"), "");
name = std::regex_replace(name, std::regex("-stable$"), "");
}
addChannel(url, name);
}
@@ -258,7 +231,7 @@ static int main_nix_channel(AsyncIoRoot & aio, std::string programName, Strings
case cRemove:
if (args.size() != 1)
throw UsageError("'--remove' requires one argument");
aio.blockOn(removeChannel(args[0]));
removeChannel(args[0]);
break;
case cList:
if (!args.empty())
@@ -268,16 +241,12 @@ static int main_nix_channel(AsyncIoRoot & aio, std::string programName, Strings
std::cout << channel.first << ' ' << channel.second << '\n';
break;
case cUpdate:
update(aio, StringSet(args.begin(), args.end()));
update(StringSet(args.begin(), args.end()));
break;
case cListGenerations:
if (!args.empty())
throw UsageError("'--list-generations' expects no arguments");
std::cout << aio.blockOn(runProgram(
settings.nixBinDir + "/nix-env",
false,
{"--profile", profile, "--list-generations"}
)) << std::flush;
std::cout << runProgram(settings.nixBinDir + "/nix-env", false, {"--profile", profile, "--list-generations"}) << std::flush;
break;
case cRollback:
if (args.size() > 1)
@@ -289,7 +258,7 @@ static int main_nix_channel(AsyncIoRoot & aio, std::string programName, Strings
} else {
envArgs.push_back("--rollback");
}
aio.blockOn(runProgram(settings.nixBinDir + "/nix-env", false, envArgs));
runProgram(settings.nixBinDir + "/nix-env", false, envArgs);
break;
}
@@ -297,8 +266,8 @@ static int main_nix_channel(AsyncIoRoot & aio, std::string programName, Strings
}
}
void registerLegacyNixChannel() {
LegacyCommandRegistry::add("nix-channel", main_nix_channel);
void registerNixChannel() {
LegacyCommands::add("nix-channel", main_nix_channel);
}
}
+1 -1
View File
@@ -3,6 +3,6 @@
namespace nix {
void registerLegacyNixChannel();
void registerNixChannel();
}
+11 -14
View File
@@ -1,4 +1,3 @@
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/file-system.hh"
#include "lix/libstore/store-api.hh"
#include "lix/libstore/store-cast.hh"
@@ -8,9 +7,9 @@
#include "lix/libstore/globals.hh"
#include "lix/libcmd/legacy.hh"
#include "lix/libutil/signals.hh"
#include "lix/libutil/types.hh"
#include "nix-collect-garbage.hh"
#include <iostream>
#include <cerrno>
namespace nix {
@@ -23,13 +22,11 @@ bool dryRun = false;
* Of course, this makes rollbacks to before this point in time
* impossible. */
static void removeOldGenerations(std::string dir, NeverAsync = {})
void removeOldGenerations(std::string dir)
{
if (sys::access(dir, R_OK) != 0) {
return;
}
if (access(dir.c_str(), R_OK) != 0) return;
bool canWrite = sys::access(dir, W_OK) == 0;
bool canWrite = access(dir.c_str(), W_OK) == 0;
for (auto & i : readDirectory(dir)) {
checkInterrupt();
@@ -59,14 +56,14 @@ static void removeOldGenerations(std::string dir, NeverAsync = {})
}
}
static int main_nix_collect_garbage(AsyncIoRoot & aio, std::string programName, Strings argv)
static int main_nix_collect_garbage(std::string programName, Strings argv)
{
{
bool removeOld = false;
GCOptions options;
LegacyArgs(aio, programName, [&](Strings::iterator & arg, const Strings::iterator & end) {
LegacyArgs(programName, [&](Strings::iterator & arg, const Strings::iterator & end) {
if (*arg == "--help")
showManPage("nix-collect-garbage");
else if (*arg == "--version")
@@ -97,16 +94,16 @@ static int main_nix_collect_garbage(AsyncIoRoot & aio, std::string programName,
} else {
options.action = GCOptions::gcReturnDead;
}
auto store = aio.blockOn(openStore());
auto store = openStore();
auto & gcStore = require<GcStore>(*store);
GCResults results;
PrintFreed freed(true, results);
aio.blockOn(gcStore.collectGarbage(options, results));
gcStore.collectGarbage(options, results);
if (dryRun) {
// Only print results for dry run; when !dryRun, paths will be printed as they're deleted.
for (auto & i : results.paths) {
printInfo("%s", Uncolored(i));
printInfo("%s", i);
}
}
@@ -114,8 +111,8 @@ static int main_nix_collect_garbage(AsyncIoRoot & aio, std::string programName,
}
}
void registerLegacyNixCollectGarbage() {
LegacyCommandRegistry::add("nix-collect-garbage", main_nix_collect_garbage);
void registerNixCollectGarbage() {
LegacyCommands::add("nix-collect-garbage", main_nix_collect_garbage);
}
}
+1 -1
View File
@@ -3,6 +3,6 @@
namespace nix {
void registerLegacyNixCollectGarbage();
void registerNixCollectGarbage();
}
+8 -8
View File
@@ -5,7 +5,7 @@
namespace nix {
static int main_nix_copy_closure(AsyncIoRoot & aio, std::string programName, Strings argv)
static int main_nix_copy_closure(std::string programName, Strings argv)
{
{
auto gzip = false;
@@ -16,14 +16,14 @@ static int main_nix_copy_closure(AsyncIoRoot & aio, std::string programName, Str
std::string sshHost;
PathSet storePaths;
LegacyArgs(aio, programName, [&](Strings::iterator & arg, const Strings::iterator & end) {
LegacyArgs(programName, [&](Strings::iterator & arg, const Strings::iterator & end) {
if (*arg == "--help")
showManPage("nix-copy-closure");
else if (*arg == "--version")
printVersion("nix-copy-closure");
else if (*arg == "--gzip" || *arg == "--bzip2" || *arg == "--xz") {
if (*arg != "--gzip")
printTaggedWarning("'%1%' is not implemented, falling back to gzip", *arg);
warn("'%1%' is not implemented, falling back to gzip", *arg);
gzip = true;
} else if (*arg == "--from")
toMode = false;
@@ -48,21 +48,21 @@ static int main_nix_copy_closure(AsyncIoRoot & aio, std::string programName, Str
throw UsageError("no host name specified");
auto remoteUri = "ssh://" + sshHost + (gzip ? "?compress=true" : "");
auto to = aio.blockOn(toMode ? openStore(remoteUri) : openStore());
auto from = aio.blockOn(toMode ? openStore() : openStore(remoteUri));
auto to = toMode ? openStore(remoteUri) : openStore();
auto from = toMode ? openStore() : openStore(remoteUri);
RealisedPath::Set storePaths2;
for (auto & path : storePaths)
storePaths2.insert(from->followLinksToStorePath(path));
aio.blockOn(copyClosure(*from, *to, storePaths2, NoRepair, NoCheckSigs, useSubstitutes));
copyClosure(*from, *to, storePaths2, NoRepair, NoCheckSigs, useSubstitutes);
return 0;
}
}
void registerLegacyNixCopyClosure() {
LegacyCommandRegistry::add("nix-copy-closure", main_nix_copy_closure);
void registerNixCopyClosure() {
LegacyCommands::add("nix-copy-closure", main_nix_copy_closure);
}
}
+1 -1
View File
@@ -3,6 +3,6 @@
namespace nix {
void registerLegacyNixCopyClosure();
void registerNixCopyClosure();
}
+73 -96
View File
@@ -1,7 +1,6 @@
#include "lix/libcmd/cmd-profiles.hh"
#include "lix/libexpr/attr-path.hh"
#include "lix/libcmd/common-eval-args.hh"
#include "lix/libexpr/value.hh"
#include "lix/libstore/derivations.hh"
#include "lix/libutil/terminal.hh"
#include "lix/libexpr/eval.hh"
@@ -29,6 +28,7 @@
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <nlohmann/json.hpp>
using std::cout;
@@ -57,7 +57,6 @@ struct InstallSourceInfo
struct Globals
{
AsyncIoRoot & aio;
InstallSourceInfo instSource;
Path profile;
std::shared_ptr<Evaluator> state;
@@ -151,12 +150,11 @@ static void getAllExprs(Evaluator & state,
continue;
}
/* Load the expression on demand. */
Value vArg;
vArg.mkString(path2.canonical().abs());
auto vArg = state.mem.allocValue();
vArg->mkString(path2.canonical().abs());
if (seen.size() == maxAttrs)
throw Error("too many Nix expressions in directory '%1%'", path);
attrs.alloc(attrName
) = {NewValueAs::app, state.mem, state.builtins.get("import"), vArg};
attrs.alloc(attrName).mkApp(&state.builtins.get("import"), vArg);
}
else if (st.type == InputAccessor::tDirectory)
/* `path2' is a directory (with no default.nix in it);
@@ -183,7 +181,7 @@ static void loadSourceExpr(EvalState & state, const SourcePath & path_, Value &
directory). */
else if (st.type == InputAccessor::tDirectory) {
auto attrs = state.ctx.buildBindings(maxAttrs);
attrs.alloc("_combineChannels") = Value::EMPTY_LIST;
attrs.alloc("_combineChannels").mkList(0);
StringSet seen;
getAllExprs(state.ctx, path, seen, attrs);
v.mkAttrs(attrs);
@@ -200,7 +198,7 @@ static void loadDerivations(EvalState & state, const SourcePath & nixExprPath,
Value vRoot;
loadSourceExpr(state, nixExprPath, vRoot);
Value v(findAlongAttrPath(state, pathPrefix, autoArgs, vRoot).first);
Value & v(*findAlongAttrPath(state, pathPrefix, autoArgs, vRoot).first);
getDerivations(state, v, pathPrefix, autoArgs, elems, true);
@@ -231,8 +229,8 @@ static std::strong_ordering comparePriorities(EvalState & state, DrvInfo & drv1,
static bool isPrebuilt(EvalState & state, DrvInfo & elem)
{
auto path = elem.queryOutPath(state);
if (state.aio.blockOn(state.ctx.store->isValidPath(path))) return true;
return state.aio.blockOn(state.ctx.store->querySubstitutablePaths({path})).count(path);
if (state.ctx.store->isValidPath(path)) return true;
return state.ctx.store->querySubstitutablePaths({path}).count(path);
}
@@ -319,9 +317,9 @@ std::vector<Match> pickNewestOnly(EvalState & state, std::vector<Match> matches)
matches.clear();
for (auto & [name, match] : newest) {
if (multiple.find(name) != multiple.end())
printTaggedWarning(
"there are multiple derivations named '%1%'; using the first one", name
);
warn(
"there are multiple derivations named '%1%'; using the first one",
name);
matches.push_back(match);
}
@@ -427,7 +425,7 @@ static void queryInstSources(EvalState & state,
Expr & eFun = state.ctx.parseExprFromString(i, CanonPath::fromCwd());
Value vFun, vTmp;
state.eval(eFun, vFun);
vTmp = {NewValueAs::app, state.ctx.mem, vFun, vArg};
vTmp.mkApp(&vFun, &vArg);
getDerivations(state, vTmp, "", *instSource.autoArgs, elems, true);
}
@@ -449,8 +447,7 @@ static void queryInstSources(EvalState & state,
if (path.isDerivation()) {
elem.setDrvPath(path);
auto outputs =
state.aio.blockOn(state.ctx.store->queryDerivationOutputMap(path));
auto outputs = state.ctx.store->queryDerivationOutputMap(path);
elem.setOutPath(outputs.at("out"));
if (name.size() >= drvExtension.size() &&
std::string(name, name.size() - drvExtension.size()) == drvExtension)
@@ -482,7 +479,7 @@ static void queryInstSources(EvalState & state,
Value vRoot;
loadSourceExpr(state, *instSource.nixExprPath, vRoot);
for (auto & i : args) {
Value v(findAlongAttrPath(state, i, *instSource.autoArgs, vRoot).first);
Value & v(*findAlongAttrPath(state, i, *instSource.autoArgs, vRoot).first);
getDerivations(state, v, "", *instSource.autoArgs, elems, true);
}
break;
@@ -497,7 +494,7 @@ static void printMissing(EvalState & state, DrvInfos & elems)
for (auto & i : elems)
if (auto drvPath = i.queryDrvPath(state))
targets.emplace_back(DerivedPath::Built{
.drvPath = makeConstantStorePath(*drvPath),
.drvPath = makeConstantStorePathRef(*drvPath),
.outputs = OutputsSpec::All { },
});
else
@@ -505,7 +502,7 @@ static void printMissing(EvalState & state, DrvInfos & elems)
.path = i.queryOutPath(state),
});
state.aio.blockOn(printMissing(state.ctx.store, targets));
printMissing(state.ctx.store, targets);
}
@@ -514,20 +511,13 @@ static bool keep(EvalState & state, DrvInfo & drv)
return drv.queryMetaBool(state, "keep", false);
}
static void setMetaFlag(EvalState & state, DrvInfo & drv,
const std::string & name, const std::string & value)
{
Value v;
v.mkString(value);
drv.setMeta(state, name, v);
}
static void installDerivations(Globals & globals,
const Strings & args, const Path & profile, std::optional<int> priority)
const Strings & args, const Path & profile)
{
debug("installing derivations");
auto state = globals.state->begin(globals.aio);
auto state = globals.state->begin();
/* Get the set of user environment elements to be installed. */
DrvInfos newElems, newElemsTmp;
@@ -549,11 +539,6 @@ static void installDerivations(Globals & globals,
newNames.insert(DrvName(i.queryName(*state)).name);
}
if (priority) {
for (auto & drv : newElems) {
setMetaFlag(*state, drv, "priority", std::to_string((priority.value())));
}
}
while (true) {
auto lockToken = optimisticLockProfile(profile);
@@ -591,7 +576,6 @@ static void installDerivations(Globals & globals,
static void opInstall(Globals & globals, Strings opFlags, Strings opArgs)
{
std::optional<int> priority;
for (Strings::iterator i = opFlags.begin(); i != opFlags.end(); ) {
auto arg = *i++;
if (parseInstallSourceOptions(globals, i, opFlags, arg)) ;
@@ -599,17 +583,10 @@ static void opInstall(Globals & globals, Strings opFlags, Strings opArgs)
globals.preserveInstalled = true;
else if (arg == "--remove-all" || arg == "-r")
globals.removeAll = true;
else if (arg == "--priority") {
if (i == opFlags.end())
throw UsageError("'%1%' requires an argument", arg);
priority = string2Int<int>(*i++);
if (!priority)
throw UsageError("'--priority' requires an integer argument");
}
else throw UsageError("unknown flag '%1%'", arg);
}
installDerivations(globals, opArgs, globals.profile, priority);
installDerivations(globals, opArgs, globals.profile);
}
@@ -621,7 +598,7 @@ static void upgradeDerivations(Globals & globals,
{
debug("upgrading derivations");
auto state = globals.state->begin(globals.aio);
auto state = globals.state->begin();
/* Upgrade works as follows: we take all currently installed
derivations, and for any derivation matching any selector, look
@@ -688,12 +665,8 @@ static void upgradeDerivations(Globals & globals,
{
const char * action = compareVersions(drvName.version, bestVersion) <= 0
? "upgrading" : "downgrading";
printInfo(
"%1% '%2%' to '%3%'",
Uncolored(action),
i.queryName(*state),
bestElem->queryName(*state)
);
printInfo("%1% '%2%' to '%3%'",
action, i.queryName(*state), bestElem->queryName(*state));
newElems.push_back(*bestElem);
} else newElems.push_back(i);
@@ -730,6 +703,15 @@ static void opUpgrade(Globals & globals, Strings opFlags, Strings opArgs)
}
static void setMetaFlag(EvalState & state, DrvInfo & drv,
const std::string & name, const std::string & value)
{
auto v = state.ctx.mem.allocValue();
v->mkString(value);
drv.setMeta(state, name, v);
}
static void opSetFlag(Globals & globals, Strings opFlags, Strings opArgs)
{
if (opFlags.size() > 0)
@@ -742,7 +724,7 @@ static void opSetFlag(Globals & globals, Strings opFlags, Strings opArgs)
std::string flagValue = *arg++;
DrvNames selectors = drvNamesFromArgs(Strings(arg, opArgs.end()));
auto state = globals.state->begin(globals.aio);
auto state = globals.state->begin();
while (true) {
std::string lockToken = optimisticLockProfile(globals.profile);
@@ -772,9 +754,9 @@ static void opSetFlag(Globals & globals, Strings opFlags, Strings opArgs)
static void opSet(Globals & globals, Strings opFlags, Strings opArgs)
{
auto state = globals.state->begin(globals.aio);
auto state = globals.state->begin();
auto store2 = globals.state->store.try_cast_shared<LocalFSStore>();
auto store2 = globals.state->store.dynamic_pointer_cast<LocalFSStore>();
if (!store2) throw Error("--set is not supported for this Nix store");
for (Strings::iterator i = opFlags.begin(); i != opFlags.end(); ) {
@@ -798,24 +780,22 @@ static void opSet(Globals & globals, Strings opFlags, Strings opArgs)
std::vector<DerivedPath> paths {
drvPath
? (DerivedPath) (DerivedPath::Built {
.drvPath = makeConstantStorePath(*drvPath),
.drvPath = makeConstantStorePathRef(*drvPath),
.outputs = OutputsSpec::All { },
})
: (DerivedPath) (DerivedPath::Opaque {
.path = drv.queryOutPath(*state),
}),
};
globals.aio.blockOn(printMissing(globals.state->store, paths));
printMissing(globals.state->store, paths);
if (globals.dryRun) return;
globals.aio.blockOn(
globals.state->store->buildPaths(paths, globals.state->repair ? bmRepair : bmNormal)
);
globals.state->store->buildPaths(paths, globals.state->repair ? bmRepair : bmNormal);
debug("switching to new user environment");
Path generation = globals.aio.blockOn(createGeneration(
Path generation = createGeneration(
*store2,
globals.profile,
drv.queryOutPath(*state)));
drv.queryOutPath(*state));
switchLink(globals.profile, generation);
}
@@ -823,7 +803,7 @@ static void opSet(Globals & globals, Strings opFlags, Strings opArgs)
static void uninstallDerivations(Globals & globals, Strings & selectors,
Path & profile)
{
auto state = globals.state->begin(globals.aio);
auto state = globals.state->begin();
while (true) {
auto lockToken = optimisticLockProfile(profile);
@@ -851,7 +831,7 @@ static void uninstallDerivations(Globals & globals, Strings & selectors,
);
}
if (split == workingElems.end())
printTaggedWarning("selector '%s' matched no installed derivations", selector);
warn("selector '%s' matched no installed derivations", selector);
for (auto removedElem = split; removedElem != workingElems.end(); removedElem++) {
printInfo("uninstalling '%s'", removedElem->queryName(*state));
}
@@ -966,14 +946,15 @@ static VersionDiff compareVersionAgainstSet(
static void queryJSON(EvalState & state, Globals & globals, std::vector<DrvInfo> & elems, bool printOutPath, bool printDrvPath, bool printMeta)
{
JSON topObj = JSON::object();
using nlohmann::json;
json topObj = json::object();
for (auto & i : elems) {
try {
if (i.hasFailed()) continue;
auto drvName = DrvName(i.queryName(state));
JSON &pkgObj = topObj[i.attrPath];
json &pkgObj = topObj[i.attrPath];
pkgObj = {
{"name", drvName.fullName},
{"pname", drvName.name},
@@ -984,8 +965,8 @@ static void queryJSON(EvalState & state, Globals & globals, std::vector<DrvInfo>
{
DrvInfo::Outputs outputs = i.queryOutputs(state, printOutPath);
JSON &outputObj = pkgObj["outputs"];
outputObj = JSON::object();
json &outputObj = pkgObj["outputs"];
outputObj = json::object();
for (auto & j : outputs) {
if (j.second)
outputObj[j.first] = globals.state->store->printStorePath(*j.second);
@@ -1000,8 +981,8 @@ static void queryJSON(EvalState & state, Globals & globals, std::vector<DrvInfo>
}
if (printMeta) {
JSON &metaObj = pkgObj["meta"];
metaObj = JSON::object();
json &metaObj = pkgObj["meta"];
metaObj = json::object();
StringSet metaNames = i.queryMetaNames(state);
for (auto & j : metaNames) {
Value * v = i.queryMeta(state, j);
@@ -1028,7 +1009,7 @@ static void queryJSON(EvalState & state, Globals & globals, std::vector<DrvInfo>
static void opQuery(Globals & globals, Strings opFlags, Strings opArgs)
{
auto & store { *globals.state->store };
auto state = globals.state->begin(globals.aio);
auto state = globals.state->begin();
Strings remaining;
std::string attrPath;
@@ -1118,8 +1099,8 @@ static void opQuery(Globals & globals, Strings opFlags, Strings opArgs)
printMsg(lvlTalkative, "skipping derivation named '%s' which gives an assertion failure", i.queryName(*state));
i.setFailed();
}
validPaths = globals.aio.blockOn(store.queryValidPaths(paths));
substitutablePaths = globals.aio.blockOn(store.querySubstitutablePaths(paths));
validPaths = store.queryValidPaths(paths);
substitutablePaths = store.querySubstitutablePaths(paths);
}
@@ -1273,43 +1254,39 @@ static void opQuery(Globals & globals, Strings opFlags, Strings opArgs)
else {
if (v->type() == nString) {
attrs2["type"] = "string";
attrs2["value"] = v->str();
attrs2["value"] = v->string.s;
xml.writeEmptyElement("meta", attrs2);
} else if (v->type() == nInt) {
attrs2["type"] = "int";
attrs2["value"] = fmt("%1%", v->integer());
attrs2["value"] = fmt("%1%", v->integer);
xml.writeEmptyElement("meta", attrs2);
} else if (v->type() == nFloat) {
attrs2["type"] = "float";
attrs2["value"] = fmt("%1%", v->fpoint());
attrs2["value"] = fmt("%1%", v->fpoint);
xml.writeEmptyElement("meta", attrs2);
} else if (v->type() == nBool) {
attrs2["type"] = "bool";
attrs2["value"] = v->boolean() ? "true" : "false";
attrs2["value"] = v->boolean ? "true" : "false";
xml.writeEmptyElement("meta", attrs2);
} else if (v->type() == nList) {
attrs2["type"] = "strings";
XMLOpenElement m(xml, "meta", attrs2);
for (auto & elem : v->listItems()) {
if (elem.type() != nString) {
continue;
}
for (auto elem : v->listItems()) {
if (elem->type() != nString) continue;
XMLAttrs attrs3;
attrs3["value"] = elem.str();
attrs3["value"] = elem->string.s;
xml.writeEmptyElement("string", attrs3);
}
} else if (v->type() == nAttrs) {
attrs2["type"] = "strings";
XMLOpenElement m(xml, "meta", attrs2);
Bindings & attrs = *v->attrs();
Bindings & attrs = *v->attrs;
for (auto &i : attrs) {
const Attr & a(*attrs.get(i.name));
if (a.value.type() != nString) {
continue;
}
Attr & a(*attrs.find(i.name));
if(a.value->type() != nString) continue;
XMLAttrs attrs3;
attrs3["type"] = globals.state->symbols[i.name];
attrs3["value"] = a.value.str();
attrs3["value"] = a.value->string.s;
xml.writeEmptyElement("string", attrs3);
}
}
@@ -1379,7 +1356,8 @@ static void opListGenerations(Globals & globals, Strings opFlags, Strings opArgs
if (opArgs.size() != 0)
throw UsageError("no arguments expected");
PathLock lock = lockProfile(globals.profile);
PathLocks lock;
lockProfile(lock, globals.profile);
auto [gens, curGen] = findGenerations(globals.profile);
@@ -1434,7 +1412,7 @@ static void opVersion(Globals & globals, Strings opFlags, Strings opArgs)
}
static int main_nix_env(AsyncIoRoot & aio, std::string programName, Strings argv)
static int main_nix_env(std::string programName, Strings argv)
{
{
Strings opFlags, opArgs;
@@ -1443,7 +1421,7 @@ static int main_nix_env(AsyncIoRoot & aio, std::string programName, Strings argv
bool showHelp = false;
std::string file;
Globals globals{aio, {}, {}, {}, {}, {}, {}, {}, {}};
Globals globals;
globals.instSource.type = srcUnknown;
globals.instSource.systemFilter = "*";
@@ -1473,7 +1451,7 @@ static int main_nix_env(AsyncIoRoot & aio, std::string programName, Strings argv
using LegacyArgs::LegacyArgs;
};
MyArgs myArgs(aio, programName, [&](Strings::iterator & arg, const Strings::iterator & end) {
MyArgs myArgs(programName, [&](Strings::iterator & arg, const Strings::iterator & end) {
Operation oldOp = op;
if (*arg == "--help")
@@ -1542,8 +1520,7 @@ static int main_nix_env(AsyncIoRoot & aio, std::string programName, Strings argv
opFlags.push_back(*arg);
/* FIXME: hacky */
if (*arg == "--from-profile" ||
(op == opQuery && (*arg == "--attr" || *arg == "-A")) ||
(op == opInstall && (*arg == "--priority")))
(op == opQuery && (*arg == "--attr" || *arg == "-A")))
opFlags.push_back(getArg(*arg, arg, end));
}
else
@@ -1560,14 +1537,14 @@ static int main_nix_env(AsyncIoRoot & aio, std::string programName, Strings argv
if (showHelp) showManPage("nix-env" + opName);
if (!op) throw UsageError("no operation specified");
auto store = aio.blockOn(openStore());
auto store = openStore();
globals.state = std::make_shared<Evaluator>(aio, myArgs.searchPath, store);
globals.state = std::make_shared<Evaluator>(myArgs.searchPath, store);
globals.state->repair = myArgs.repair;
globals.instSource.nixExprPath = std::make_shared<SourcePath>(
file != ""
? aio.blockOn(lookupFileArg(*globals.state, file)).unwrap()
? lookupFileArg(*globals.state, file)
: CanonPath(nixExprPath));
globals.instSource.autoArgs = myArgs.getAutoArgs(*globals.state);
@@ -1586,8 +1563,8 @@ static int main_nix_env(AsyncIoRoot & aio, std::string programName, Strings argv
}
}
void registerLegacyNixEnv() {
LegacyCommandRegistry::add("nix-env", main_nix_env);
void registerNixEnv() {
LegacyCommands::add("nix-env", main_nix_env);
}
}
+1 -1
View File
@@ -3,6 +3,6 @@
namespace nix {
void registerLegacyNixEnv();
void registerNixEnv();
}
+23 -28
View File
@@ -2,6 +2,7 @@
#include "lix/libexpr/print-ambiguous.hh"
#include "lix/libmain/shared.hh"
#include "lix/libexpr/eval.hh"
#include "lix/libexpr/eval-inline.hh"
#include "lix/libexpr/get-drvs.hh"
#include "lix/libexpr/attr-path.hh"
#include "lix/libexpr/value-to-xml.hh"
@@ -12,6 +13,7 @@
#include "lix/libcmd/legacy.hh"
#include "nix-instantiate.hh"
#include <map>
#include <iostream>
@@ -22,14 +24,14 @@ static Path gcRoot;
static int rootNr = 0;
enum OutputKind { okPlain, okRaw, okXML, okJSON };
enum OutputKind { okPlain, okXML, okJSON };
void processExpr(EvalState & state, const Strings & attrPaths,
bool parseOnly, bool strict, Bindings & autoArgs,
bool evalOnly, OutputKind output, bool location, Expr & e)
{
if (parseOnly) {
std::cout << e.toJSON(state.ctx.symbols).dump(2);
e.show(state.ctx.symbols, std::cout);
std::cout << "\n";
return;
}
@@ -38,8 +40,8 @@ void processExpr(EvalState & state, const Strings & attrPaths,
state.eval(e, vRoot);
for (auto & i : attrPaths) {
Value v(findAlongAttrPath(state, i, autoArgs, vRoot).first);
state.forceValue(v, noPos);
Value & v(*findAlongAttrPath(state, i, autoArgs, vRoot).first);
state.forceValue(v, v.determinePos(noPos));
NixStringContext context;
if (evalOnly) {
@@ -47,15 +49,11 @@ void processExpr(EvalState & state, const Strings & attrPaths,
if (autoArgs.empty())
vRes = v;
else
state.autoCallFunction(autoArgs, v, vRes, noPos);
if (output == okRaw)
std::cout << *state.coerceToString(noPos, vRes, context, "while generating the nix-instantiate output", StringCoercionMode::Strict);
// We intentionally don't output a newline here. The default PS1 for Bash in NixOS starts with a newline
// and other interactive shells like Zsh are smart enough to print a missing newline before the prompt.
else if (output == okXML)
state.autoCallFunction(autoArgs, v, vRes);
if (output == okXML)
printValueAsXML(state, strict, location, vRes, std::cout, context, noPos);
else if (output == okJSON) {
printValueAsJSON(state, strict, vRes, noPos, std::cout, context);
printValueAsJSON(state, strict, vRes, v.determinePos(noPos), std::cout, context);
std::cout << std::endl;
} else {
if (strict) state.forceValueDeep(vRes);
@@ -80,9 +78,9 @@ void processExpr(EvalState & state, const Strings & attrPaths,
else {
Path rootName = absPath(gcRoot);
if (++rootNr > 1) rootName += "-" + std::to_string(rootNr);
auto store2 = state.ctx.store.try_cast_shared<LocalFSStore>();
auto store2 = state.ctx.store.dynamic_pointer_cast<LocalFSStore>();
if (store2)
drvPathS = state.aio.blockOn(store2->addPermRoot(drvPath, rootName));
drvPathS = store2->addPermRoot(drvPath, rootName);
}
std::cout << fmt("%s%s\n", drvPathS, (outputName != "out" ? "!" + outputName : ""));
}
@@ -91,7 +89,7 @@ void processExpr(EvalState & state, const Strings & attrPaths,
}
static int main_nix_instantiate(AsyncIoRoot & aio, std::string programName, Strings argv)
static int main_nix_instantiate(std::string programName, Strings argv)
{
{
Strings files;
@@ -111,7 +109,7 @@ static int main_nix_instantiate(AsyncIoRoot & aio, std::string programName, Stri
using LegacyArgs::LegacyArgs;
};
MyArgs myArgs(aio, programName, [&](Strings::iterator & arg, const Strings::iterator & end) {
MyArgs myArgs(programName, [&](Strings::iterator & arg, const Strings::iterator & end) {
if (*arg == "--help")
showManPage("nix-instantiate");
else if (*arg == "--version")
@@ -134,8 +132,6 @@ static int main_nix_instantiate(AsyncIoRoot & aio, std::string programName, Stri
gcRoot = getArg(*arg, arg, end);
else if (*arg == "--indirect")
;
else if (*arg == "--raw")
outputKind = okRaw;
else if (*arg == "--xml")
outputKind = okXML;
else if (*arg == "--json")
@@ -158,11 +154,11 @@ static int main_nix_instantiate(AsyncIoRoot & aio, std::string programName, Stri
if (evalOnly && !wantsReadWrite)
settings.readOnlyMode = true;
auto store = aio.blockOn(openStore());
auto evalStore = myArgs.evalStoreUrl ? aio.blockOn(openStore(*myArgs.evalStoreUrl)) : store;
auto store = openStore();
auto evalStore = myArgs.evalStoreUrl ? openStore(*myArgs.evalStoreUrl) : store;
auto evaluator = std::make_unique<Evaluator>(aio, myArgs.searchPath, evalStore, store);
auto state = evaluator->begin(aio);
auto evaluator = std::make_unique<Evaluator>(myArgs.searchPath, evalStore, store);
auto state = evaluator->begin();
evaluator->repair = myArgs.repair;
Bindings & autoArgs = *myArgs.getAutoArgs(*evaluator);
@@ -171,7 +167,7 @@ static int main_nix_instantiate(AsyncIoRoot & aio, std::string programName, Stri
if (findFile) {
for (auto & i : files) {
auto p = aio.blockOn(evaluator->paths.findFile(i)).unwrap();
auto p = evaluator->paths.findFile(i);
std::cout << p.canonical().abs() << std::endl;
}
return 0;
@@ -185,10 +181,9 @@ static int main_nix_instantiate(AsyncIoRoot & aio, std::string programName, Stri
files.push_back("./default.nix");
for (auto & i : files) {
Expr & e = fromArgs ? evaluator->parseExprFromString(i, CanonPath::fromCwd())
: evaluator->parseExprFromFile(evaluator->paths.resolveExprPath(
aio.blockOn(lookupFileArg(*evaluator, i)).unwrap()
));
Expr & e = fromArgs
? evaluator->parseExprFromString(i, CanonPath::fromCwd())
: evaluator->parseExprFromFile(evaluator->paths.resolveExprPath(lookupFileArg(*evaluator, i)));
processExpr(*state, attrPaths, parseOnly, strict, autoArgs,
evalOnly, outputKind, xmlOutputSourceLocation, e);
}
@@ -199,8 +194,8 @@ static int main_nix_instantiate(AsyncIoRoot & aio, std::string programName, Stri
}
}
void registerLegacyNixInstantiate() {
LegacyCommandRegistry::add("nix-instantiate", main_nix_instantiate);
void registerNixInstantiate() {
LegacyCommands::add("nix-instantiate", main_nix_instantiate);
}
}
+1 -1
View File
@@ -3,6 +3,6 @@
namespace nix {
void registerLegacyNixInstantiate();
void registerNixInstantiate();
}
+180 -319
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -3,6 +3,6 @@
namespace nix {
void registerLegacyNixStore();
void registerNixStore();
}
+28 -27
View File
@@ -1,11 +1,12 @@
#include "user-env.hh"
#include "lix/libexpr/value.hh"
#include "lix/libstore/derivations.hh"
#include "lix/libstore/store-api.hh"
#include "lix/libstore/path-with-outputs.hh"
#include "lix/libstore/local-fs-store.hh"
#include "lix/libstore/globals.hh"
#include "lix/libmain/shared.hh"
#include "lix/libexpr/eval.hh"
#include "lix/libexpr/eval-inline.hh"
#include "lix/libstore/profiles.hh"
#include "lix/libexpr/print-ambiguous.hh"
@@ -27,14 +28,13 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
drvsToBuild.push_back({*drvPath});
debug("building user environment dependencies");
state.aio.blockOn(state.ctx.store->buildPaths(
state.ctx.store->buildPaths(
toDerivedPaths(drvsToBuild),
state.ctx.repair ? bmRepair : bmNormal));
state.ctx.repair ? bmRepair : bmNormal);
/* Construct the whole top level derivation. */
StorePathSet references;
auto manifest = state.ctx.mem.newList(elems.size());
Value vManifest{NewValueAs::list, manifest};
Value manifest = state.ctx.mem.newList(elems.size());
size_t n = 0;
for (auto & i : elems) {
/* Create a pseudo-derivation containing the name, system,
@@ -57,18 +57,17 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
// Copy each output meant for installation.
auto & vOutputs = attrs.alloc(state.ctx.s.outputs);
auto outputsList = state.ctx.mem.newList(outputs.size());
vOutputs = {NewValueAs::list, outputsList};
vOutputs = state.ctx.mem.newList(outputs.size());
for (const auto & [m, j] : enumerate(outputs)) {
outputsList->elems[m].mkString(j.first);
(vOutputs.listElems()[m] = state.ctx.mem.allocValue())->mkString(j.first);
auto outputAttrs = state.ctx.buildBindings(2);
outputAttrs.alloc(state.ctx.s.outPath).mkString(state.ctx.store->printStorePath(*j.second));
attrs.alloc(j.first).mkAttrs(outputAttrs);
/* This is only necessary when installing store paths, e.g.,
`nix-env -i /nix/store/abcd...-foo'. */
state.aio.blockOn(state.ctx.store->addTempRoot(*j.second));
state.aio.blockOn(state.ctx.store->ensurePath(*j.second));
state.ctx.store->addTempRoot(*j.second);
state.ctx.store->ensurePath(*j.second);
references.insert(*j.second);
}
@@ -78,12 +77,12 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
for (auto & j : metaNames) {
Value * v = i.queryMeta(state, j);
if (!v) continue;
meta.insert(state.ctx.symbols.create(j), *v);
meta.insert(state.ctx.symbols.create(j), v);
}
attrs.alloc(state.ctx.s.meta).mkAttrs(meta);
manifest->elems[n++].mkAttrs(attrs);
(manifest.listElems()[n++] = state.ctx.mem.allocValue())->mkAttrs(attrs);
if (drvPath) references.insert(*drvPath);
}
@@ -92,9 +91,9 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
the store; we need it for future modifications of the
environment. */
std::ostringstream str;
printAmbiguous(vManifest, state.ctx.symbols, str, nullptr, std::numeric_limits<int>::max());
auto manifestFile = state.aio.blockOn(state.ctx.store->addTextToStore("env-manifest.nix",
str.str(), references));
printAmbiguous(manifest, state.ctx.symbols, str, nullptr, std::numeric_limits<int>::max());
auto manifestFile = state.ctx.store->addTextToStore("env-manifest.nix",
str.str(), references);
/* Get the environment builder expression. */
Value envBuilder;
@@ -106,34 +105,36 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
builder with the manifest as argument. */
auto attrs = state.ctx.buildBindings(3);
state.ctx.paths.mkStorePathString(manifestFile, attrs.alloc("manifest"));
attrs.insert(state.ctx.symbols.create("derivations"), vManifest);
attrs.insert(state.ctx.symbols.create("derivations"), &manifest);
Value args;
args.mkAttrs(attrs);
Value topLevel{NewValueAs::app, state.ctx.mem, envBuilder, args};
Value topLevel;
topLevel.mkApp(&envBuilder, &args);
/* Evaluate it. */
debug("evaluating user environment builder");
state.forceValue(topLevel, noPos);
state.forceValue(topLevel, topLevel.determinePos(noPos));
NixStringContext context;
const Attr & aDrvPath(*topLevel.attrs()->get(state.ctx.s.drvPath));
auto topLevelDrv = state.coerceToStorePath(aDrvPath.pos, aDrvPath.value, context, "");
const Attr & aOutPath(*topLevel.attrs()->get(state.ctx.s.outPath));
auto topLevelOut = state.coerceToStorePath(aOutPath.pos, aOutPath.value, context, "");
Attr & aDrvPath(*topLevel.attrs->find(state.ctx.s.drvPath));
auto topLevelDrv = state.coerceToStorePath(aDrvPath.pos, *aDrvPath.value, context, "");
Attr & aOutPath(*topLevel.attrs->find(state.ctx.s.outPath));
auto topLevelOut = state.coerceToStorePath(aOutPath.pos, *aOutPath.value, context, "");
/* Realise the resulting store expression. */
debug("building user environment");
std::vector<StorePathWithOutputs> topLevelDrvs;
topLevelDrvs.push_back({topLevelDrv});
state.aio.blockOn(state.ctx.store->buildPaths(
state.ctx.store->buildPaths(
toDerivedPaths(topLevelDrvs),
state.ctx.repair ? bmRepair : bmNormal));
state.ctx.repair ? bmRepair : bmNormal);
/* Switch the current user environment to the output path. */
auto store2 = state.ctx.store.try_cast_shared<LocalFSStore>();
auto store2 = state.ctx.store.dynamic_pointer_cast<LocalFSStore>();
if (store2) {
PathLock lock = lockProfile(profile);
PathLocks lock;
lockProfile(lock, profile);
Path lockTokenCur = optimisticLockProfile(profile);
if (lockToken != lockTokenCur) {
@@ -142,7 +143,7 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
}
debug("switching to new user environment");
Path generation = state.aio.blockOn(createGeneration(*store2, profile, topLevelOut));
Path generation = createGeneration(*store2, profile, topLevelOut);
switchLink(profile, generation);
}
+95 -41
View File
@@ -1,11 +1,10 @@
#include "lix/libcmd/built-path.hh"
#include "lix/libstore/derivations.hh"
#include "lix/libstore/store-api.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/json.hh"
#include "lix/libutil/result.hh"
#include <kj/async.h>
#include <nlohmann/json.hpp>
#include <optional>
namespace nix {
@@ -13,9 +12,9 @@ namespace nix {
bool MY_TYPE ::operator COMPARATOR (const MY_TYPE & other) const \
{ \
const MY_TYPE* me = this; \
auto fields1 = std::tie(me->drvPath, me->FIELD); \
auto fields1 = std::tie(*me->drvPath, me->FIELD); \
me = &other; \
auto fields2 = std::tie(me->drvPath, me->FIELD); \
auto fields2 = std::tie(*me->drvPath, me->FIELD); \
return fields1 COMPARATOR fields2; \
}
#define CMP(CHILD_TYPE, MY_TYPE, FIELD) \
@@ -23,6 +22,10 @@ namespace nix {
CMP_ONE(CHILD_TYPE, MY_TYPE, FIELD, !=) \
CMP_ONE(CHILD_TYPE, MY_TYPE, FIELD, <)
#define FIELD_TYPE std::pair<std::string, StorePath>
CMP(SingleBuiltPath, SingleBuiltPathBuilt, output)
#undef FIELD_TYPE
#define FIELD_TYPE std::map<std::string, StorePath>
CMP(SingleBuiltPath, BuiltPathBuilt, outputs)
#undef FIELD_TYPE
@@ -30,6 +33,16 @@ CMP(SingleBuiltPath, BuiltPathBuilt, outputs)
#undef CMP
#undef CMP_ONE
StorePath SingleBuiltPath::outPath() const
{
return std::visit(
overloaded{
[](const SingleBuiltPath::Opaque & p) { return p.path; },
[](const SingleBuiltPath::Built & b) { return b.output.second; },
}, raw()
);
}
StorePathSet BuiltPath::outPaths() const
{
return std::visit(
@@ -45,51 +58,92 @@ StorePathSet BuiltPath::outPaths() const
);
}
kj::Promise<Result<JSON>> BuiltPath::Built::toJSON(const Store & store) const
try {
JSON res;
res["drvPath"] = TRY_AWAIT(drvPath.toJSON(store));
SingleDerivedPath::Built SingleBuiltPath::Built::discardOutputPath() const
{
return SingleDerivedPath::Built {
.drvPath = make_ref<SingleDerivedPath>(drvPath->discardOutputPath()),
.output = output.first,
};
}
SingleDerivedPath SingleBuiltPath::discardOutputPath() const
{
return std::visit(
overloaded{
[](const SingleBuiltPath::Opaque & p) -> SingleDerivedPath {
return p;
},
[](const SingleBuiltPath::Built & b) -> SingleDerivedPath {
return b.discardOutputPath();
},
}, raw()
);
}
nlohmann::json BuiltPath::Built::toJSON(const Store & store) const
{
nlohmann::json res;
res["drvPath"] = drvPath->toJSON(store);
for (const auto & [outputName, outputPath] : outputs) {
res["outputs"][outputName] = store.printStorePath(outputPath);
}
co_return res;
} catch (...) {
co_return result::current_exception();
return res;
}
nlohmann::json SingleBuiltPath::Built::toJSON(const Store & store) const
{
nlohmann::json res;
res["drvPath"] = drvPath->toJSON(store);
auto & [outputName, outputPath] = output;
res["output"] = outputName;
res["outputPath"] = store.printStorePath(outputPath);
return res;
}
kj::Promise<Result<RealisedPath::Set>> BuiltPath::toRealisedPaths(Store & store) const
try {
nlohmann::json SingleBuiltPath::toJSON(const Store & store) const
{
return std::visit([&](const auto & buildable) {
return buildable.toJSON(store);
}, raw());
}
nlohmann::json BuiltPath::toJSON(const Store & store) const
{
return std::visit([&](const auto & buildable) {
return buildable.toJSON(store);
}, raw());
}
RealisedPath::Set BuiltPath::toRealisedPaths(Store & store) const
{
RealisedPath::Set res;
auto handlers = overloaded{
// NOLINTNEXTLINE(cppcoreguidelines-avoid-capturing-lambda-coroutines)
[&](const BuiltPath::Opaque & p) -> kj::Promise<Result<void>> {
try {
res.insert(p.path);
return {result::success()};
} catch (...) {
return {result::current_exception()};
}
},
// NOLINTNEXTLINE(cppcoreguidelines-avoid-capturing-lambda-coroutines)
[&](const BuiltPath::Built & p) -> kj::Promise<Result<void>> {
try {
auto drvHashes = TRY_AWAIT(
staticOutputHashes(store, TRY_AWAIT(store.readDerivation(p.drvPath.path)))
);
std::visit(
overloaded{
[&](const BuiltPath::Opaque & p) { res.insert(p.path); },
[&](const BuiltPath::Built & p) {
auto drvHashes =
staticOutputHashes(store, store.readDerivation(p.drvPath->outPath()));
for (auto& [outputName, outputPath] : p.outputs) {
res.insert(outputPath);
if (experimentalFeatureSettings.isEnabled(
Xp::CaDerivations)) {
auto drvOutput = get(drvHashes, outputName);
if (!drvOutput)
throw Error(
"the derivation '%s' has unrealised output '%s' (derived-path.cc/toRealisedPaths)",
store.printStorePath(p.drvPath->outPath()), outputName);
auto thisRealisation = store.queryRealisation(
DrvOutput{*drvOutput, outputName});
assert(thisRealisation); // Weve built it, so we must
// have the realisation
res.insert(*thisRealisation);
} else {
res.insert(outputPath);
}
}
co_return result::success();
} catch (...) {
co_return result::current_exception();
}
},
},
};
TRY_AWAIT(std::visit(handlers, raw()));
co_return res;
} catch (...) {
co_return result::current_exception();
raw());
return res;
}
}
+53 -5
View File
@@ -2,21 +2,67 @@
///@file
#include "lix/libstore/derived-path.hh"
#include "lix/libstore/realisation.hh"
#include "lix/libutil/result.hh"
#include <kj/async.h>
namespace nix {
struct SingleBuiltPath;
struct SingleBuiltPathBuilt {
ref<SingleBuiltPath> drvPath;
std::pair<std::string, StorePath> output;
SingleDerivedPathBuilt discardOutputPath() const;
std::string to_string(const Store & store) const;
static SingleBuiltPathBuilt parse(const Store & store, std::string_view, std::string_view);
nlohmann::json toJSON(const Store & store) const;
DECLARE_CMP(SingleBuiltPathBuilt);
};
namespace built_path::detail {
using SingleBuiltPathRaw = std::variant<
DerivedPathOpaque,
SingleBuiltPathBuilt
>;
}
struct SingleBuiltPath : built_path::detail::SingleBuiltPathRaw {
using Raw = built_path::detail::SingleBuiltPathRaw;
using Raw::Raw;
using Opaque = DerivedPathOpaque;
using Built = SingleBuiltPathBuilt;
inline const Raw & raw() const {
return static_cast<const Raw &>(*this);
}
StorePath outPath() const;
SingleDerivedPath discardOutputPath() const;
static SingleBuiltPath parse(const Store & store, std::string_view);
nlohmann::json toJSON(const Store & store) const;
};
static inline ref<SingleBuiltPath> staticDrv(StorePath drvPath)
{
return make_ref<SingleBuiltPath>(SingleBuiltPath::Opaque { drvPath });
}
/**
* A built derived path with hints in the form of optional concrete output paths.
*
* See 'BuiltPath' for more an explanation.
*/
struct BuiltPathBuilt {
DerivedPathOpaque drvPath;
ref<SingleBuiltPath> drvPath;
std::map<std::string, StorePath> outputs;
kj::Promise<Result<JSON>> toJSON(const Store & store) const;
std::string to_string(const Store & store) const;
static BuiltPathBuilt parse(const Store & store, std::string_view, std::string_view);
nlohmann::json toJSON(const Store & store) const;
DECLARE_CMP(BuiltPathBuilt);
};
@@ -44,7 +90,9 @@ struct BuiltPath : built_path::detail::BuiltPathRaw {
}
StorePathSet outPaths() const;
kj::Promise<Result<RealisedPath::Set>> toRealisedPaths(Store & store) const;
RealisedPath::Set toRealisedPaths(Store & store) const;
nlohmann::json toJSON(const Store & store) const;
};
typedef std::vector<BuiltPath> BuiltPaths;
+11 -14
View File
@@ -3,7 +3,6 @@
#include "lix/libcmd/cmd-profiles.hh"
#include "lix/libcmd/built-path.hh"
#include "lix/libstore/builtins/buildenv.hh"
#include "lix/libutil/async-io.hh"
#include "lix/libutil/logging.hh"
#include "lix/libstore/names.hh"
#include "lix/libstore/store-api.hh"
@@ -111,7 +110,7 @@ ProfileManifest::ProfileManifest(EvalState & state, const Path & profile)
auto manifestPath = profile + "/manifest.json";
if (pathExists(manifestPath)) {
auto json = json::parse(readFile(manifestPath), "a profile manifest");
auto json = nlohmann::json::parse(readFile(manifestPath));
auto version = json.value("version", 0);
std::string sUrl;
@@ -198,15 +197,15 @@ void ProfileManifest::addElement(ProfileElement element)
addElement(finalName, std::move(element));
}
JSON ProfileManifest::toJSON(Store & store) const
nlohmann::json ProfileManifest::toJSON(Store & store) const
{
auto es = JSON::object();
auto es = nlohmann::json::object();
for (auto & [name, element] : elements) {
auto paths = JSON::array();
auto paths = nlohmann::json::array();
for (auto & path : element.storePaths) {
paths.push_back(store.printStorePath(path));
}
JSON obj;
nlohmann::json obj;
obj["storePaths"] = paths;
obj["active"] = element.active;
obj["priority"] = element.priority;
@@ -218,14 +217,14 @@ JSON ProfileManifest::toJSON(Store & store) const
}
es[name] = obj;
}
JSON json;
nlohmann::json json;
json["version"] = 3;
json["elements"] = es;
return json;
}
kj::Promise<Result<StorePath>> ProfileManifest::build(ref<Store> store)
try {
StorePath ProfileManifest::build(ref<Store> store)
{
auto tempDir = createTempDir();
StorePathSet references;
@@ -267,12 +266,10 @@ try {
};
info.narSize = sink.s.size();
AsyncStringInputStream source(sink.s);
TRY_AWAIT(store->addToStore(info, source));
StringSource source(sink.s);
store->addToStore(info, source);
co_return std::move(info.path);
} catch (...) {
co_return result::current_exception();
return std::move(info.path);
}
void ProfileManifest::printDiff(
+6 -3
View File
@@ -6,11 +6,14 @@
#include "lix/libexpr/flake/flakeref.hh"
#include "lix/libexpr/get-drvs.hh"
#include "lix/libutil/types.hh"
#include "lix/libutil/json-fwd.hh"
#include "lix/libutil/url.hh"
#include "lix/libutil/url-name.hh"
#include <string>
#include <set>
#include <nlohmann/json.hpp>
namespace nix
{
@@ -59,9 +62,9 @@ struct ProfileManifest
ProfileManifest(EvalState & state, const Path & profile);
JSON toJSON(Store & store) const;
nlohmann::json toJSON(Store & store) const;
kj::Promise<Result<StorePath>> build(ref<Store> store);
StorePath build(ref<Store> store);
void addElement(std::string_view nameCandidate, ProfileElement element);
void addElement(ProfileElement element);
+31 -49
View File
@@ -2,37 +2,29 @@
#include "lix/libstore/store-api.hh"
#include "lix/libstore/local-fs-store.hh"
#include "lix/libstore/derivations.hh"
#include "lix/libexpr/nixexpr.hh"
#include "lix/libstore/profiles.hh"
#include "lix/libcmd/repl.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/error.hh"
#include <nlohmann/json.hpp>
extern char * * environ __attribute__((weak));
namespace nix {
CommandRegistry::CommandMap * CommandRegistry::commands = nullptr;
RegisterCommand::Commands * RegisterCommand::commands = nullptr;
nix::CommandMap CommandRegistry::getCommandsFor(const std::vector<std::string> & prefix)
nix::Commands RegisterCommand::getCommandsFor(const std::vector<std::string> & prefix)
{
if (!CommandRegistry::commands) {
CommandRegistry::commands = new CommandMap;
}
nix::CommandMap res;
for (auto & [name, command] : *CommandRegistry::commands) {
nix::Commands res;
for (auto & [name, command] : *RegisterCommand::commands)
if (name.size() == prefix.size() + 1) {
bool equal = true;
for (size_t i = 0; i < prefix.size(); ++i) {
if (name[i] != prefix[i]) {
equal = false;
}
}
if (equal) {
for (size_t i = 0; i < prefix.size(); ++i)
if (name[i] != prefix[i]) equal = false;
if (equal)
res.insert_or_assign(name[prefix.size()], command);
}
}
}
return res;
}
@@ -42,15 +34,14 @@ StoreCommand::StoreCommand()
ref<Store> StoreCommand::getStore()
{
if (!_store) {
_store = createStore(aio());
}
return *_store;
if (!_store)
_store = createStore();
return ref<Store>(_store);
}
ref<Store> StoreCommand::createStore(AsyncIoRoot & in)
ref<Store> StoreCommand::createStore()
{
return in.blockOn(openStore());
return openStore();
}
void StoreCommand::run()
@@ -75,9 +66,9 @@ CopyCommand::CopyCommand()
});
}
ref<Store> CopyCommand::createStore(AsyncIoRoot & in)
ref<Store> CopyCommand::createStore()
{
return srcUri.empty() ? StoreCommand::createStore(in) : in.blockOn(openStore(srcUri));
return srcUri.empty() ? StoreCommand::createStore() : openStore(srcUri);
}
ref<Store> CopyCommand::getDstStore()
@@ -85,7 +76,7 @@ ref<Store> CopyCommand::getDstStore()
if (srcUri.empty() && dstUri.empty())
throw UsageError("you must pass '--from' and/or '--to'");
return aio().blockOn(dstUri.empty() ? openStore() : openStore(dstUri));
return dstUri.empty() ? openStore() : openStore(dstUri);
}
EvalCommand::EvalCommand()
@@ -107,21 +98,21 @@ EvalCommand::~EvalCommand()
ref<Store> EvalCommand::getEvalStore()
{
if (!evalStore)
evalStore = evalStoreUrl ? aio().blockOn(openStore(*evalStoreUrl)) : getStore();
return *evalStore;
evalStore = evalStoreUrl ? openStore(*evalStoreUrl) : getStore();
return ref<Store>(evalStore);
}
ref<eval_cache::CachingEvaluator> EvalCommand::getEvaluator()
{
if (!evalState) {
evalState = std::allocate_shared<eval_cache::CachingEvaluator>(
TraceableAllocator<EvalState>(), aio(), searchPath, getEvalStore(), getStore(),
TraceableAllocator<EvalState>(), searchPath, getEvalStore(), getStore(),
startReplOnEvalErrors ? AbstractNixRepl::runSimple : nullptr
);
evalState->repair = repair;
}
return ref<eval_cache::CachingEvaluator>::unsafeFromPtr(evalState);
return ref<eval_cache::CachingEvaluator>(evalState);
}
MixOperateOnOptions::MixOperateOnOptions()
@@ -168,17 +159,10 @@ void BuiltPathsCommand::run(ref<Store> store, Installables && installables)
if (installables.size())
throw UsageError("'--all' does not expect arguments");
// XXX: Only uses opaque paths, ignores all the realisations
for (auto & p : aio().blockOn(store->queryAllValidPaths()))
for (auto & p : store->queryAllValidPaths())
paths.emplace_back(BuiltPath::Opaque{p});
} else {
paths = Installable::toBuiltPaths(
*getEvaluator()->begin(aio()),
getEvalStore(),
store,
realiseMode,
operateOn,
installables
);
paths = Installable::toBuiltPaths(*getEvaluator()->begin(), getEvalStore(), store, realiseMode, operateOn, installables);
if (recursive) {
// XXX: This only computes the store path closure, ignoring
// intermediate realisations
@@ -187,7 +171,7 @@ void BuiltPathsCommand::run(ref<Store> store, Installables && installables)
auto rootFromThis = root.outPaths();
pathsRoots.insert(rootFromThis.begin(), rootFromThis.end());
}
aio().blockOn(store->computeFSClosure(pathsRoots, pathsClosure));
store->computeFSClosure(pathsRoots, pathsClosure);
for (auto & path : pathsClosure)
paths.emplace_back(BuiltPath::Opaque{path});
}
@@ -208,7 +192,7 @@ void StorePathsCommand::run(ref<Store> store, BuiltPaths && paths)
for (auto & p : builtPath.outPaths())
storePaths.insert(p);
auto sorted = aio().blockOn(store->topoSortPaths(storePaths));
auto sorted = store->topoSortPaths(storePaths);
std::reverse(sorted.begin(), sorted.end());
run(store, std::move(sorted));
@@ -236,11 +220,11 @@ MixProfile::MixProfile()
void MixProfile::updateProfile(const StorePath & storePath)
{
if (!profile) return;
auto store = getStore().try_cast_shared<LocalFSStore>();
auto store = getStore().dynamic_pointer_cast<LocalFSStore>();
if (!store) throw Error("'--profile' is not supported for this Nix store");
auto profile2 = absPath(*profile);
switchLink(profile2,
aio().blockOn(createGeneration(*store, profile2, storePath)));
createGeneration(*store, profile2, storePath));
}
void MixProfile::updateProfile(const BuiltPaths & buildables)
@@ -305,10 +289,8 @@ void MixEnvironment::setEnviron() {
throw UsageError("--unset does not make sense with --ignore-environment");
for (const auto & var : keep) {
auto val = sys::getenv(var);
if (val) {
stringsEnv.emplace_back(fmt("%s=%s", var, val));
}
auto val = getenv(var.c_str());
if (val) stringsEnv.emplace_back(fmt("%s=%s", var.c_str(), val));
}
vectorEnv = stringsToCharPtrs(stringsEnv);
@@ -318,7 +300,7 @@ void MixEnvironment::setEnviron() {
throw UsageError("--keep does not make sense without --ignore-environment");
for (const auto & var : unset)
(void) sys::unsetenv(var);
unsetenv(var.c_str());
}
}
+17 -50
View File
@@ -7,7 +7,6 @@
#include "lix/libcmd/common-eval-args.hh"
#include "lix/libstore/path.hh"
#include "lix/libexpr/flake/lockfile.hh"
#include "lix/libutil/async.hh"
#include <optional>
@@ -39,14 +38,14 @@ struct StoreCommand : virtual Command
StoreCommand();
void run() override;
ref<Store> getStore();
virtual ref<Store> createStore(AsyncIoRoot & in);
virtual ref<Store> createStore();
/**
* Main entry point, with a `Store` provided
*/
virtual void run(ref<Store>) = 0;
private:
std::optional<ref<Store>> _store;
std::shared_ptr<Store> _store;
};
/**
@@ -59,7 +58,7 @@ struct CopyCommand : virtual StoreCommand
CopyCommand();
ref<Store> createStore(AsyncIoRoot & in) override;
ref<Store> createStore() override;
ref<Store> getDstStore();
};
@@ -81,7 +80,7 @@ struct EvalCommand : virtual StoreCommand, MixEvalArgs
virtual ref<eval_cache::CachingEvaluator> getEvaluator();
private:
std::optional<ref<Store>> evalStore;
std::shared_ptr<Store> evalStore;
std::shared_ptr<eval_cache::CachingEvaluator> evalState;
};
@@ -262,60 +261,31 @@ struct StorePathCommand : public StorePathsCommand
/**
* A helper class for registering \ref Command commands globally.
*/
struct CommandRegistry
struct RegisterCommand
{
using CommandMap = std::map<
std::vector<std::string>,
std::function<ref<Command>(AsyncIoRoot & aio)>
>;
static CommandMap * commands;
typedef std::map<std::vector<std::string>, std::function<ref<Command>()>> Commands;
static Commands * commands;
static void add(std::vector<std::string> && name,
std::function<ref<Command>(AsyncIoRoot & aio)> command)
RegisterCommand(std::vector<std::string> && name,
std::function<ref<Command>()> command)
{
if (!commands) {
commands = new CommandMap;
}
if (!commands) commands = new Commands;
commands->emplace(name, command);
}
static nix::CommandMap getCommandsFor(const std::vector<std::string> & prefix);
};
template<typename Base>
class MixAio : public Base
{
private:
AsyncIoRoot & aio_;
public:
template<typename... Args>
MixAio(AsyncIoRoot & aio, Args &&... args)
: Base(std::forward<Args>(args)...)
, aio_(aio)
{
}
AsyncIoRoot & aio() override
{
return aio_;
}
static nix::Commands getCommandsFor(const std::vector<std::string> & prefix);
};
template<class T>
static void registerCommand(const std::string & name)
static RegisterCommand registerCommand(const std::string & name)
{
CommandRegistry::add({name}, [](AsyncIoRoot & aio) {
return make_ref<MixAio<T>>(aio);
});
return RegisterCommand({name}, [](){ return make_ref<T>(); });
}
template<class T>
static void registerCommand2(std::vector<std::string> && name)
static RegisterCommand registerCommand2(std::vector<std::string> && name)
{
CommandRegistry::add(std::move(name), [](AsyncIoRoot & aio) {
return make_ref<MixAio<T>>(aio);
});
return RegisterCommand(std::move(name), [](){ return make_ref<T>(); });
}
struct MixProfile : virtual StoreCommand
@@ -360,9 +330,7 @@ void completeFlakeInputPath(
const std::vector<FlakeRef> & flakeRefs,
std::string_view prefix);
void completeFlakeRef(
AsyncIoRoot & aio, AddCompletions & completions, ref<Store> store, std::string_view prefix
);
void completeFlakeRef(AddCompletions & completions, ref<Store> store, std::string_view prefix);
void completeFlakeRefWithFragment(
AddCompletions & completions,
@@ -373,11 +341,10 @@ void completeFlakeRefWithFragment(
const Strings & defaultFlakeAttrPaths,
std::string_view prefix);
kj::Promise<Result<void>> printClosureDiff(
void printClosureDiff(
ref<Store> store,
const StorePath & beforePath,
const StorePath & afterPath,
bool json,
std::string_view indent);
}
+34 -47
View File
@@ -8,45 +8,37 @@
#include "lix/libexpr/flake/flakeref.hh"
#include "lix/libstore/store-api.hh"
#include "lix/libcmd/command.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/error.hh"
#include "lix/libutil/regex.hh"
#include <regex>
namespace nix {
static std::regex const identifierRegex = regex::parse("^[A-Za-z_][A-Za-z0-9_'-]*$");
static void checkValidNixIdentifier(const std::string & name)
static std::regex const identifierRegex("^[A-Za-z_][A-Za-z0-9_'-]*$");
static void warnInvalidNixIdentifier(const std::string & name)
{
std::smatch match;
if (!std::regex_match(name, match, identifierRegex)) {
throw UsageError(
"This invocation specifies a value for argument '%s' "
"which isn't a valid Nix identifier. "
"The project is dropping support for this so that it's possible to make e.g. "
"'%s' evaluating to '%s' in the future. "
"If you depend on this behavior, please reach out in "
"<https://git.lix.systems/lix-project/lix/issues/496> so we can discuss your use-case.",
name,
"--arg config.allowUnfree true",
"{ config.allowUnfree = true; }"
);
warn("This Nix invocation specifies a value for argument '%s' which isn't a valid \
Nix identifier. The project is considering to drop support for this \
or to require quotes around args that aren't valid Nix identifiers. \
If you depend on this behvior, please reach out in \
https://git.lix.systems/lix-project/lix/issues/496 so we can discuss \
your use-case.", name);
}
}
MixEvalArgs::MixEvalArgs()
{
addFlag(
{.longName = "arg",
.description = "Pass the value *expr* as the argument *name* to Nix functions.",
.category = category,
.labels = {"name", "expr"},
.handler = {[&](std::string name, std::string expr) {
checkValidNixIdentifier(name);
autoArgs[name] = 'E' + expr;
}}}
);
addFlag({
.longName = "arg",
.description = "Pass the value *expr* as the argument *name* to Nix functions.",
.category = category,
.labels = {"name", "expr"},
.handler = {[&](std::string name, std::string expr) {
warnInvalidNixIdentifier(name);
autoArgs[name] = 'E' + expr;
}}
});
addFlag({
.longName = "argstr",
@@ -54,7 +46,7 @@ MixEvalArgs::MixEvalArgs()
.category = category,
.labels = {"name", "string"},
.handler = {[&](std::string name, std::string s) {
checkValidNixIdentifier(name);
warnInvalidNixIdentifier(name);
autoArgs[name] = 'S' + s;
}},
});
@@ -113,7 +105,7 @@ MixEvalArgs::MixEvalArgs()
```
-I nixpkgs=channel:nixos-21.05
-I nixpkgs=https://channels.nixos.org/nixos-21.05/nixexprs.tar.xz
-I nixpkgs=https://nixos.org/channels/nixos-21.05/nixexprs.tar.xz
```
You can also fetch source trees using [flake URLs](./nix3-flake.md#url-like-syntax) and add them to the
@@ -162,7 +154,7 @@ MixEvalArgs::MixEvalArgs()
fetchers::overrideRegistry(from.input, to.input, extraAttrs);
}},
.completer = {[&](AddCompletions & completions, size_t, std::string_view prefix) {
completeFlakeRef(aio(), completions, aio().blockOn(openStore()), prefix);
completeFlakeRef(completions, openStore(), prefix);
}}
});
@@ -183,47 +175,42 @@ Bindings * MixEvalArgs::getAutoArgs(Evaluator & state)
{
auto res = state.buildBindings(autoArgs.size());
for (auto & i : autoArgs) {
Value v;
auto v = state.mem.allocValue();
if (i.second[0] == 'E')
state.evalLazily(
state.parseExprFromString(i.second.substr(1), CanonPath::fromCwd()), v
state.parseExprFromString(i.second.substr(1), CanonPath::fromCwd()), *v
);
else
v.mkString(((std::string_view) i.second).substr(1));
v->mkString(((std::string_view) i.second).substr(1));
res.insert(state.symbols.create(i.first), v);
}
return res.finish();
}
kj::Promise<Result<EvalPaths::PathResult<SourcePath, ThrownError>>>
lookupFileArg(Evaluator & state, std::string_view fileArg)
try {
SourcePath lookupFileArg(Evaluator & state, std::string_view fileArg)
{
if (EvalSettings::isPseudoUrl(fileArg)) {
auto const url = EvalSettings::resolvePseudoUrl(fileArg);
auto const downloaded = TRY_AWAIT(fetchers::downloadTarball(
auto const downloaded = fetchers::downloadTarball(
state.store,
url,
/* name */ "source",
/* locked */ false
));
);
StorePath const storePath = downloaded.tree.storePath;
co_return SourcePath(CanonPath(state.store->toRealPath(storePath)));
return CanonPath(state.store->toRealPath(storePath));
} else if (fileArg.starts_with("flake:")) {
experimentalFeatureSettings.require(Xp::Flakes);
static constexpr size_t FLAKE_LEN = std::string_view("flake:").size();
auto flakeRef = parseFlakeRef(std::string(fileArg.substr(FLAKE_LEN)), {}, true, false);
auto storePath = TRY_AWAIT(TRY_AWAIT(flakeRef.resolve(state.store)).fetchTree(state.store))
.first.storePath;
co_return SourcePath(CanonPath(state.store->toRealPath(storePath)));
} else if (fileArg.size() > 2 && fileArg.at(0) == '<' && fileArg.at(fileArg.size() - 1) == '>')
{
auto storePath = flakeRef.resolve(state.store).fetchTree(state.store).first.storePath;
return CanonPath(state.store->toRealPath(storePath));
} else if (fileArg.size() > 2 && fileArg.at(0) == '<' && fileArg.at(fileArg.size() - 1) == '>') {
Path p(fileArg.substr(1, fileArg.size() - 2));
co_return TRY_AWAIT(state.paths.findFile(p));
return state.paths.findFile(p);
} else {
co_return SourcePath(CanonPath::fromCwd(fileArg));
return CanonPath::fromCwd(fileArg);
}
} catch (...) {
co_return result::current_exception();
}
}
+1 -3
View File
@@ -1,7 +1,6 @@
#pragma once
///@file
#include "lix/libexpr/eval-error.hh"
#include "lix/libexpr/eval.hh"
#include "lix/libutil/args.hh"
#include "lix/libmain/common-args.hh"
@@ -50,7 +49,6 @@ private:
*
* @exception nix::ThrownError for failed search path lookup. Probably others.
*/
kj::Promise<Result<EvalPaths::PathResult<SourcePath, ThrownError>>>
lookupFileArg(Evaluator & state, std::string_view fileArg);
SourcePath lookupFileArg(Evaluator & state, std::string_view fileArg);
}
+12 -12
View File
@@ -7,15 +7,16 @@
#include "lix/libexpr/get-drvs.hh"
#include "lix/libexpr/flake/flake.hh"
#include <nlohmann/json.hpp>
namespace nix {
InstallableAttrPath::InstallableAttrPath(
ref<eval_cache::CachingEvaluator> state,
SourceExprCommand & cmd,
Value & v,
Value * v,
const std::string & attrPath,
ExtendedOutputsSpec extendedOutputsSpec
)
ExtendedOutputsSpec extendedOutputsSpec)
: InstallableValue(state)
, cmd(cmd)
, v(allocRootValue(v))
@@ -23,10 +24,10 @@ InstallableAttrPath::InstallableAttrPath(
, extendedOutputsSpec(std::move(extendedOutputsSpec))
{ }
std::pair<Value, PosIdx> InstallableAttrPath::toValue(EvalState & state)
std::pair<Value *, PosIdx> InstallableAttrPath::toValue(EvalState & state)
{
auto [vRes, pos] = findAlongAttrPath(state, attrPath, *cmd.getAutoArgs(*evaluator), *v);
state.forceValue(vRes, pos);
auto [vRes, pos] = findAlongAttrPath(state, attrPath, *cmd.getAutoArgs(*evaluator), **v);
state.forceValue(*vRes, pos);
return {vRes, pos};
}
@@ -35,7 +36,7 @@ DerivedPathsWithInfo InstallableAttrPath::toDerivedPaths(EvalState & state)
auto [v, pos] = toValue(state);
if (std::optional derivedPathWithInfo = trySinglePathToDerivedPaths(
state, v, pos, fmt("while evaluating the attribute '%s'", attrPath)
state, *v, pos, fmt("while evaluating the attribute '%s'", attrPath)
))
{
return { *derivedPathWithInfo };
@@ -44,7 +45,7 @@ DerivedPathsWithInfo InstallableAttrPath::toDerivedPaths(EvalState & state)
Bindings & autoArgs = *cmd.getAutoArgs(*evaluator);
DrvInfos drvInfos;
getDerivations(state, v, "", autoArgs, drvInfos, false);
getDerivations(state, *v, "", autoArgs, drvInfos, false);
// Backward compatibility hack: group results by drvPath. This
// helps keep .all output together.
@@ -77,7 +78,7 @@ DerivedPathsWithInfo InstallableAttrPath::toDerivedPaths(EvalState & state)
for (auto & [drvPath, outputs] : byDrvPath)
res.push_back({
.path = DerivedPath::Built {
.drvPath = makeConstantStorePath(drvPath),
.drvPath = makeConstantStorePathRef(drvPath),
.outputs = outputs,
},
.info = make_ref<ExtraPathInfoValue>(ExtraPathInfoValue::Value {
@@ -93,10 +94,9 @@ DerivedPathsWithInfo InstallableAttrPath::toDerivedPaths(EvalState & state)
InstallableAttrPath InstallableAttrPath::parse(
ref<eval_cache::CachingEvaluator> state,
SourceExprCommand & cmd,
Value & v,
Value * v,
std::string_view prefix,
ExtendedOutputsSpec extendedOutputsSpec
)
ExtendedOutputsSpec extendedOutputsSpec)
{
return {
state, cmd, v,
+7 -7
View File
@@ -8,6 +8,8 @@
#include "lix/libcmd/common-eval-args.hh"
#include "lix/libexpr/eval.hh"
#include <nlohmann/json.hpp>
namespace nix {
class InstallableAttrPath : public InstallableValue
@@ -20,14 +22,13 @@ class InstallableAttrPath : public InstallableValue
InstallableAttrPath(
ref<eval_cache::CachingEvaluator> state,
SourceExprCommand & cmd,
Value & v,
Value * v,
const std::string & attrPath,
ExtendedOutputsSpec extendedOutputsSpec
);
ExtendedOutputsSpec extendedOutputsSpec);
std::string what() const override { return attrPath; };
std::pair<Value, PosIdx> toValue(EvalState & state) override;
std::pair<Value *, PosIdx> toValue(EvalState & state) override;
DerivedPathsWithInfo toDerivedPaths(EvalState & state) override;
@@ -36,10 +37,9 @@ public:
static InstallableAttrPath parse(
ref<eval_cache::CachingEvaluator> state,
SourceExprCommand & cmd,
Value & v,
Value * v,
std::string_view prefix,
ExtendedOutputsSpec extendedOutputsSpec
);
ExtendedOutputsSpec extendedOutputsSpec);
};
}
+28 -19
View File
@@ -26,27 +26,36 @@ InstallableDerivedPath InstallableDerivedPath::parse(
std::string_view prefix,
ExtendedOutputsSpec extendedOutputsSpec)
{
auto derivedPath = std::visit(
overloaded{
// If the user did not use ^, we treat the output more
// liberally: we accept a symlink chain or an actual
// store path.
[&](const ExtendedOutputsSpec::Default &) -> DerivedPath {
return DerivedPath::Opaque{
.path = store->followLinksToStorePath(prefix),
auto derivedPath = std::visit(overloaded {
// If the user did not use ^, we treat the output more
// liberally: we accept a symlink chain or an actual
// store path.
[&](const ExtendedOutputsSpec::Default &) -> DerivedPath {
auto storePath = store->followLinksToStorePath(prefix);
// Remove this prior to stabilizing the new CLI.
if (storePath.isDerivation()) {
auto oldDerivedPath = DerivedPath::Built {
.drvPath = makeConstantStorePathRef(storePath),
.outputs = OutputsSpec::All { },
};
},
// If the user did use ^, we just do exactly what is written.
[&](const ExtendedOutputsSpec::Explicit & outputSpec) -> DerivedPath {
auto drv = DerivedPathOpaque::parse(*store, prefix);
return DerivedPath::Built{
.drvPath = std::move(drv),
.outputs = outputSpec,
};
},
warn(
"The interpretation of store paths arguments ending in `.drv` recently changed. If this command is now failing try again with '%s'",
oldDerivedPath.to_string(*store));
};
return DerivedPath::Opaque {
.path = std::move(storePath),
};
},
extendedOutputsSpec.raw
);
// If the user did use ^, we just do exactly what is written.
[&](const ExtendedOutputsSpec::Explicit & outputSpec) -> DerivedPath {
auto drv = make_ref<SingleDerivedPath>(SingleDerivedPath::parse(*store, prefix));
drvRequireExperiment(*drv);
return DerivedPath::Built {
.drvPath = std::move(drv),
.outputs = outputSpec,
};
},
}, extendedOutputsSpec.raw);
return InstallableDerivedPath {
store,
std::move(derivedPath),
+17 -5
View File
@@ -1,11 +1,24 @@
#include "lix/libstore/globals.hh"
#include "lix/libcmd/installable-flake.hh"
#include "lix/libcmd/installable-derived-path.hh"
#include "lix/libstore/outputs-spec.hh"
#include "lix/libcmd/command.hh"
#include "lix/libexpr/attr-path.hh"
#include "lix/libcmd/common-eval-args.hh"
#include "lix/libstore/derivations.hh"
#include "lix/libexpr/eval-inline.hh"
#include "lix/libexpr/eval.hh"
#include "lix/libexpr/get-drvs.hh"
#include "lix/libstore/store-api.hh"
#include "lix/libmain/shared.hh"
#include "lix/libexpr/flake/flake.hh"
#include "lix/libexpr/eval-cache.hh"
#include "lix/libutil/url.hh"
#include "lix/libfetchers/registry.hh"
#include "lix/libstore/build-result.hh"
#include <regex>
#include <queue>
#include <nlohmann/json.hpp>
@@ -60,8 +73,7 @@ InstallableFlake::InstallableFlake(
DerivedPathsWithInfo InstallableFlake::toDerivedPaths(EvalState & state)
{
auto act =
logger->startActivity(lvlTalkative, actUnknown, fmt("evaluating derivation '%s'", what()));
Activity act(*logger, lvlTalkative, actUnknown, fmt("evaluating derivation '%s'", what()));
auto attr = getCursor(state);
@@ -99,7 +111,7 @@ DerivedPathsWithInfo InstallableFlake::toDerivedPaths(EvalState & state)
return {{
.path = DerivedPath::Built {
.drvPath = makeConstantStorePath(std::move(drvPath)),
.drvPath = makeConstantStorePathRef(std::move(drvPath)),
.outputs = std::visit(overloaded {
[&](const ExtendedOutputsSpec::Default & d) -> OutputsSpec {
std::set<std::string> outputsToInstall;
@@ -137,9 +149,9 @@ DerivedPathsWithInfo InstallableFlake::toDerivedPaths(EvalState & state)
}};
}
std::pair<Value, PosIdx> InstallableFlake::toValue(EvalState & state)
std::pair<Value *, PosIdx> InstallableFlake::toValue(EvalState & state)
{
return {getCursor(state)->forceValue(state), noPos};
return {&getCursor(state)->forceValue(state), noPos};
}
std::vector<ref<eval_cache::AttrCursor>>
+1 -1
View File
@@ -55,7 +55,7 @@ struct InstallableFlake : InstallableValue
DerivedPathsWithInfo toDerivedPaths(EvalState & state) override;
std::pair<Value, PosIdx> toValue(EvalState & state) override;
std::pair<Value *, PosIdx> toValue(EvalState & state) override;
/**
* Get a cursor to every attrpath in getActualAttrPaths() that
+5 -10
View File
@@ -1,7 +1,6 @@
#include "lix/libcmd/installable-value.hh"
#include "lix/libexpr/eval-cache.hh"
#include "lix/libfetchers/fetch-to-store.hh"
#include "lix/libutil/archive.hh"
namespace nix {
@@ -9,9 +8,8 @@ std::vector<ref<eval_cache::AttrCursor>>
InstallableValue::getCursors(EvalState & state)
{
auto evalCache =
std::make_shared<nix::eval_cache::EvalCache>(std::nullopt, [&](EvalState & state) {
return toValue(state).first;
});
std::make_shared<nix::eval_cache::EvalCache>(std::nullopt,
[&](EvalState & state) { return toValue(state).first; });
return {evalCache->getRoot()};
}
@@ -38,10 +36,10 @@ InstallableValue & InstallableValue::require(Installable & installable)
ref<InstallableValue> InstallableValue::require(ref<Installable> installable)
{
auto castedInstallable = installable.try_cast<InstallableValue>();
auto castedInstallable = installable.dynamic_pointer_cast<InstallableValue>();
if (!castedInstallable)
throw nonValueInstallable(*installable);
return *castedInstallable;
return ref { castedInstallable };
}
std::optional<DerivedPathWithInfo> InstallableValue::trySinglePathToDerivedPaths(
@@ -49,10 +47,7 @@ std::optional<DerivedPathWithInfo> InstallableValue::trySinglePathToDerivedPaths
)
{
if (v.type() == nPath) {
auto storePath = state.aio.blockOn(fetchToStoreRecursive(
*evaluator->store,
*prepareDump(state.ctx.paths.checkSourcePath(v.path()).canonical().abs())
));
auto storePath = fetchToStore(*evaluator->store, state.ctx.paths.checkSourcePath(v.path()));
return {{
.path = DerivedPath::Opaque {
.path = std::move(storePath),
+1 -1
View File
@@ -77,7 +77,7 @@ struct InstallableValue : Installable
virtual ~InstallableValue() { }
virtual std::pair<Value, PosIdx> toValue(EvalState & state) = 0;
virtual std::pair<Value *, PosIdx> toValue(EvalState & state) = 0;
/**
* Get a cursor to each value this Installable could refer to.
+74 -55
View File
@@ -3,7 +3,6 @@
#include "lix/libcmd/installable-derived-path.hh"
#include "lix/libcmd/installable-attr-path.hh"
#include "lix/libcmd/installable-flake.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/logging.hh"
#include "lix/libstore/outputs-spec.hh"
#include "lix/libcmd/command.hh"
@@ -20,6 +19,8 @@
#include "lix/libstore/build-result.hh"
#include <nlohmann/json.hpp>
namespace nix {
void completeFlakeInputPath(
@@ -61,7 +62,7 @@ MixFlakeOptions::MixFlakeOptions()
.category = category,
.handler = {[&]() {
lockFlags.useRegistries = false;
printTaggedWarning("'--no-registries' is deprecated; use '--no-use-registries'");
warn("'--no-registries' is deprecated; use '--no-use-registries'");
}}
});
@@ -85,11 +86,9 @@ MixFlakeOptions::MixFlakeOptions()
}},
.completer = {[&](AddCompletions & completions, size_t n, std::string_view prefix) {
if (n == 0) {
completeFlakeInputPath(
completions, *getEvaluator()->begin(aio()), getFlakeRefsForCompletion(), prefix
);
completeFlakeInputPath(completions, *getEvaluator()->begin(), getFlakeRefsForCompletion(), prefix);
} else if (n == 1) {
completeFlakeRef(aio(), completions, getEvaluator()->store, prefix);
completeFlakeRef(completions, getEvaluator()->store, prefix);
}
}}
});
@@ -122,7 +121,7 @@ MixFlakeOptions::MixFlakeOptions()
.category = category,
.labels = {"flake-url"},
.handler = {[&](std::string flakeRef) {
auto evalState = getEvaluator()->begin(aio());
auto evalState = getEvaluator()->begin();
auto flake = flake::lockFlake(
*evalState,
parseFlakeRef(flakeRef, absPath(".")),
@@ -138,7 +137,7 @@ MixFlakeOptions::MixFlakeOptions()
}
}},
.completer = {[&](AddCompletions & completions, size_t, std::string_view prefix) {
completeFlakeRef(aio(), completions, getEvaluator()->store, prefix);
completeFlakeRef(completions, getEvaluator()->store, prefix);
}}
});
}
@@ -183,8 +182,8 @@ MixReadOnlyOption::MixReadOnlyOption()
Strings SourceExprCommand::getDefaultFlakeAttrPaths()
{
return {
"packages." + evalSettings.getCurrentSystem() + ".default",
"defaultPackage." + evalSettings.getCurrentSystem()
"packages." + settings.thisSystem.get() + ".default",
"defaultPackage." + settings.thisSystem.get()
};
}
@@ -193,17 +192,17 @@ Strings SourceExprCommand::getDefaultFlakeAttrPathPrefixes()
return {
// As a convenience, look for the attribute in
// 'outputs.packages'.
"packages." + evalSettings.getCurrentSystem() + ".",
"packages." + settings.thisSystem.get() + ".",
// As a temporary hack until Nixpkgs is properly converted
// to provide a clean 'packages' set, look in 'legacyPackages'.
"legacyPackages." + evalSettings.getCurrentSystem() + "."
"legacyPackages." + settings.thisSystem.get() + "."
};
}
Args::CompleterClosure SourceExprCommand::getCompleteInstallable()
{
return [this](AddCompletions & completions, size_t, std::string_view prefix) {
completeInstallable(*getEvaluator()->begin(aio()), completions, prefix);
completeInstallable(*getEvaluator()->begin(), completions, prefix);
};
}
@@ -215,9 +214,9 @@ void SourceExprCommand::completeInstallable(EvalState & state, AddCompletions &
auto evaluator = getEvaluator();
Expr & e = evaluator->parseExprFromFile(state.ctx.paths.resolveExprPath(
state.aio.blockOn(lookupFileArg(*evaluator, *file)).unwrap()
));
Expr & e = evaluator->parseExprFromFile(
state.ctx.paths.resolveExprPath(lookupFileArg(*evaluator, *file))
);
Value root;
state.eval(e, root);
@@ -235,14 +234,15 @@ void SourceExprCommand::completeInstallable(EvalState & state, AddCompletions &
prefix_ = "";
}
auto [v1, pos] = findAlongAttrPath(state, prefix_, *autoArgs, root);
auto [v, pos] = findAlongAttrPath(state, prefix_, *autoArgs, root);
Value &v1(*v);
state.forceValue(v1, pos);
Value v2;
state.autoCallFunction(*autoArgs, v1, v2, pos);
state.autoCallFunction(*autoArgs, v1, v2);
if (v2.type() == nAttrs) {
for (auto & i : *v2.attrs()) {
std::string name{evaluator->symbols[i.name]};
for (auto & i : *v2.attrs) {
std::string name = evaluator->symbols[i.name];
if (name.find(searchWord) == 0) {
if (prefix_ == "")
completions.add(name);
@@ -280,7 +280,7 @@ void completeFlakeRefWithFragment(
try {
auto hash = prefix.find('#');
if (hash == std::string::npos) {
completeFlakeRef(evalState.aio, completions, evaluator->store, prefix);
completeFlakeRef(completions, evaluator->store, prefix);
} else {
completions.setType(AddCompletions::Type::Attrs);
@@ -343,13 +343,11 @@ void completeFlakeRefWithFragment(
}
}
} catch (Error & e) {
printTaggedWarning("%1%", Uncolored(e.msg()));
warn(e.msg());
}
}
void completeFlakeRef(
AsyncIoRoot & aio, AddCompletions & completions, ref<Store> store, std::string_view prefix
)
void completeFlakeRef(AddCompletions & completions, ref<Store> store, std::string_view prefix)
{
if (!experimentalFeatureSettings.isEnabled(Xp::Flakes))
return;
@@ -360,7 +358,7 @@ void completeFlakeRef(
Args::completeDir(completions, 0, prefix);
/* Look for registry entries that match the prefix. */
for (auto & registry : aio.blockOn(fetchers::getRegistries(store))) {
for (auto & registry : fetchers::getRegistries(store)) {
for (auto & entry : registry->entries) {
auto from = entry.from.to_string();
if (!prefix.starts_with("flake:") && from.starts_with("flake:")) {
@@ -383,18 +381,16 @@ DerivedPathWithInfo Installable::toDerivedPath(EvalState & state)
return std::move(buildables[0]);
}
static kj::Promise<Result<StorePath>> getDeriver(
static StorePath getDeriver(
ref<Store> store,
const Installable & i,
const StorePath & drvPath)
try {
auto derivers = TRY_AWAIT(store->queryValidDerivers(drvPath));
{
auto derivers = store->queryValidDerivers(drvPath);
if (derivers.empty())
throw Error("'%s' does not have a known deriver", i.what());
// FIXME: use all derivers?
co_return *derivers.begin();
} catch (...) {
co_return result::current_exception();
return *derivers.begin();
}
ref<eval_cache::EvalCache> openEvalCache(
@@ -411,12 +407,12 @@ ref<eval_cache::EvalCache> openEvalCache(
if (getEnv("NIX_ALLOW_EVAL").value_or("1") == "0")
throw Error("not everything is cached, but evaluation is not allowed");
Value vFlake;
flake::callFlake(state, *lockedFlake, vFlake);
auto vFlake = state.ctx.mem.allocValue();
flake::callFlake(state, *lockedFlake, *vFlake);
state.forceAttrs(vFlake, noPos, "while parsing cached flake data");
state.forceAttrs(*vFlake, noPos, "while parsing cached flake data");
auto aOutputs = vFlake.attrs()->get(state.ctx.symbols.create("outputs"));
auto aOutputs = vFlake->attrs->get(state.ctx.symbols.create("outputs"));
assert(aOutputs);
return aOutputs->value;
@@ -449,24 +445,25 @@ Installables SourceExprCommand::parseInstallables(
throw UsageError("'--file' and '--expr' are exclusive");
auto evaluator = getEvaluator();
Value vFile;
auto vFile = evaluator->mem.allocValue();
if (file == "-") {
auto & e = evaluator->parseStdin();
state.eval(e, vFile);
state.eval(e, *vFile);
}
else if (file)
state.evalFile(state.aio.blockOn(lookupFileArg(*evaluator, *file)).unwrap(), vFile);
state.evalFile(lookupFileArg(*evaluator, *file), *vFile);
else {
auto & e = evaluator->parseExprFromString(*expr, CanonPath::fromCwd());
state.eval(e, vFile);
state.eval(e, *vFile);
}
for (auto & s : ss) {
auto [prefix, extendedOutputsSpec] = ExtendedOutputsSpec::parse(s);
result.push_back(make_ref<InstallableAttrPath>(InstallableAttrPath::parse(
evaluator, *this, vFile, std::move(prefix), std::move(extendedOutputsSpec)
)));
result.push_back(
make_ref<InstallableAttrPath>(
InstallableAttrPath::parse(
evaluator, *this, vFile, std::move(prefix), std::move(extendedOutputsSpec))));
}
} else {
@@ -522,6 +519,30 @@ ref<Installable> SourceExprCommand::parseInstallable(
return installables.front();
}
static SingleBuiltPath getBuiltPath(ref<Store> evalStore, ref<Store> store, const SingleDerivedPath & b)
{
return std::visit(
overloaded{
[&](const SingleDerivedPath::Opaque & bo) -> SingleBuiltPath {
return SingleBuiltPath::Opaque { bo.path };
},
[&](const SingleDerivedPath::Built & bfd) -> SingleBuiltPath {
auto drvPath = getBuiltPath(evalStore, store, *bfd.drvPath);
// Resolving this instead of `bfd` will yield the same result, but avoid duplicative work.
SingleDerivedPath::Built truncatedBfd {
.drvPath = makeConstantStorePathRef(drvPath.outPath()),
.output = bfd.output,
};
auto outputPath = resolveDerivedPath(*store, truncatedBfd, &*evalStore);
return SingleBuiltPath::Built {
.drvPath = make_ref<SingleBuiltPath>(std::move(drvPath)),
.output = { bfd.output, outputPath },
};
},
},
b.raw());
}
std::vector<BuiltPathWithResult> Installable::build(
EvalState & state,
ref<Store> evalStore,
@@ -600,17 +621,16 @@ std::vector<std::pair<ref<Installable>, BuiltPathWithResult>> Installable::build
case Realise::Nothing:
case Realise::Derivation:
state.aio.blockOn(printMissing(store, pathsToBuild, lvlError));
printMissing(store, pathsToBuild, lvlError);
for (auto & path : pathsToBuild) {
for (auto & aux : backmap[path]) {
std::visit(overloaded {
[&](const DerivedPath::Built & bfd) {
auto outputs =
state.aio.blockOn(resolveDerivedPath(*store, bfd, &*evalStore));
auto outputs = resolveDerivedPath(*store, bfd, &*evalStore);
res.push_back({aux.installable, {
.path = BuiltPath::Built {
.drvPath = bfd.drvPath,
.drvPath = make_ref<SingleBuiltPath>(getBuiltPath(evalStore, store, *bfd.drvPath)),
.outputs = outputs,
},
.info = aux.info}});
@@ -628,10 +648,9 @@ std::vector<std::pair<ref<Installable>, BuiltPathWithResult>> Installable::build
case Realise::Outputs: {
if (settings.printMissing)
state.aio.blockOn(printMissing(store, pathsToBuild, lvlInfo));
printMissing(store, pathsToBuild, lvlInfo);
auto buildResults =
state.aio.blockOn(store->buildPathsWithResults(pathsToBuild, bMode, evalStore));
auto buildResults = store->buildPathsWithResults(pathsToBuild, bMode, evalStore);
throwBuildErrors(buildResults, *store);
for (auto & buildResult : buildResults) {
for (auto & aux : backmap[buildResult.path]) {
@@ -642,7 +661,7 @@ std::vector<std::pair<ref<Installable>, BuiltPathWithResult>> Installable::build
outputs.emplace(outputName, realisation.outPath);
res.push_back({aux.installable, {
.path = BuiltPath::Built {
.drvPath = bfd.drvPath,
.drvPath = make_ref<SingleBuiltPath>(getBuiltPath(evalStore, store, *bfd.drvPath)),
.outputs = outputs,
},
.info = aux.info,
@@ -753,11 +772,11 @@ StorePathSet Installable::toDerivations(
bo.path.isDerivation()
? bo.path
: useDeriver
? state.aio.blockOn(getDeriver(store, *i, bo.path))
? getDeriver(store, *i, bo.path)
: throw Error("argument '%s' did not evaluate to a derivation", i->what()));
},
[&](const DerivedPath::Built & bfd) {
drvPaths.insert(bfd.drvPath.path);
drvPaths.insert(resolveDerivedPath(*store, *bfd.drvPath));
},
}, b.path.raw());
@@ -823,7 +842,7 @@ std::vector<FlakeRef> InstallableCommand::getFlakeRefsForCompletion()
void InstallablesCommand::run(ref<Store> store, std::vector<std::string> && rawInstallables)
{
auto installables = parseInstallables(*getEvaluator()->begin(aio()), store, rawInstallables);
auto installables = parseInstallables(*getEvaluator()->begin(), store, rawInstallables);
run(store, std::move(installables));
}
@@ -840,7 +859,7 @@ InstallableCommand::InstallableCommand()
void InstallableCommand::run(ref<Store> store)
{
auto installable = parseInstallable(*getEvaluator()->begin(aio()), store, _installable);
auto installable = parseInstallable(*getEvaluator()->begin(), store, _installable);
run(store, std::move(installable));
}
+1
View File
@@ -3,6 +3,7 @@
#include "lix/libexpr/eval.hh"
#include "lix/libstore/path.hh"
#include "lix/libstore/outputs-spec.hh"
#include "lix/libstore/derived-path.hh"
#include "lix/libcmd/built-path.hh"
#include "lix/libstore/store-api.hh"

Some files were not shown because too many files have changed in this diff Show More