guiwf/lix
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

merge into: guiwf/lix:main
Branches Tags
guiwf/lix:main guiwf/lix:adaptive-routing guiwf/lix:release-2.95 guiwf/lix:ci-config guiwf/lix:release-2.93 guiwf/lix:release-2.94 guiwf/lix:sb/jade/coverage guiwf/lix:sb/raitobezarius/cgroups-observability guiwf/lix:sb/jade/wip-tars guiwf/lix:sb/raito/flakes-out-of-core guiwf/lix:sb/raito/eval-explorations guiwf/lix:sb/pennae/ci-config guiwf/lix:release-2.91 guiwf/lix:release-2.92 guiwf/lix:sb/raito/darwin-kqueue-disablement guiwf/lix:sb/raito/hydra-merge guiwf/lix:sb/raito/phantom-referrers-gc guiwf/lix:repl-overlay-example guiwf/lix:reuse-compliance guiwf/lix:release-2.90 guiwf/lix:sb/pennae/io-rewrite guiwf/lix:git-series/activity-cleanup guiwf/lix:sb/tom-hubrecht/primops.cc-v2 guiwf/lix:sb/bacchanalia/cache-strlen%topic=substr-perf guiwf/lix:sb/qyriad/beta-1 guiwf/lix:sb/qyriad/2.90-beta.0 guiwf/lix:sb/rbt/test-branch guiwf/lix:sb/rbt/justfile guiwf/lix:sb/rbt/repl-overlays guiwf/lix:sb/qyriad/maint-meson guiwf/lix:sb/qyriad/maint/meson guiwf/lix:sb/puck/meson-c-api guiwf/lix:sb/jade/burn-it-all guiwf/lix:dependabot/github_actions/zeebe-io/backport-action-2.4.1 guiwf/lix:dependabot/github_actions/cachix/install-nix-action-25 guiwf/lix:dependabot/github_actions/actions/labeler-5 guiwf/lix:dependabot/github_actions/cachix/cachix-action-14
guiwf/lix:2.95.3 guiwf/lix:2.93.4 guiwf/lix:2.94.2 guiwf/lix:2.95.2 guiwf/lix:2.95.1 guiwf/lix:2.94.1 guiwf/lix:2.95.0 guiwf/lix:2.94.0 guiwf/lix:2.93.3 guiwf/lix:2.92.3 guiwf/lix:2.93.2 guiwf/lix:2.91.3 guiwf/lix:2.92.2 guiwf/lix:2.91.2 guiwf/lix:2.93.1 guiwf/lix:2.93.0 guiwf/lix:2.92.0 guiwf/lix:2.91.1 guiwf/lix:2.91.0 guiwf/lix:2.90.0 guiwf/lix:2.90.0-rc1 guiwf/lix:2.90-beta.1 guiwf/lix:2.90-beta.0 guiwf/lix:1.11.1 guiwf/lix:1.11 guiwf/lix:1.10 guiwf/lix:1.9 guiwf/lix:1.8 guiwf/lix:1.7 guiwf/lix:1.6.1 guiwf/lix:1.6 guiwf/lix:1.5.3 guiwf/lix:1.5.2 guiwf/lix:1.5.1 guiwf/lix:1.5 guiwf/lix:1.4 guiwf/lix:1.3 guiwf/lix:1.2 guiwf/lix:1.1 guiwf/lix:1.0
..
pull from: guiwf/lix:release-2.93
Branches Tags
guiwf/lix:adaptive-routing guiwf/lix:main guiwf/lix:release-2.95 guiwf/lix:ci-config guiwf/lix:release-2.93 guiwf/lix:release-2.94 guiwf/lix:sb/jade/coverage guiwf/lix:sb/raitobezarius/cgroups-observability guiwf/lix:sb/jade/wip-tars guiwf/lix:sb/raito/flakes-out-of-core guiwf/lix:sb/raito/eval-explorations guiwf/lix:sb/pennae/ci-config guiwf/lix:release-2.91 guiwf/lix:release-2.92 guiwf/lix:sb/raito/darwin-kqueue-disablement guiwf/lix:sb/raito/hydra-merge guiwf/lix:sb/raito/phantom-referrers-gc guiwf/lix:repl-overlay-example guiwf/lix:reuse-compliance guiwf/lix:release-2.90 guiwf/lix:sb/pennae/io-rewrite guiwf/lix:git-series/activity-cleanup guiwf/lix:sb/tom-hubrecht/primops.cc-v2 guiwf/lix:sb/bacchanalia/cache-strlen%topic=substr-perf guiwf/lix:sb/qyriad/beta-1 guiwf/lix:sb/qyriad/2.90-beta.0 guiwf/lix:sb/rbt/test-branch guiwf/lix:sb/rbt/justfile guiwf/lix:sb/rbt/repl-overlays guiwf/lix:sb/qyriad/maint-meson guiwf/lix:sb/qyriad/maint/meson guiwf/lix:sb/puck/meson-c-api guiwf/lix:sb/jade/burn-it-all guiwf/lix:dependabot/github_actions/zeebe-io/backport-action-2.4.1 guiwf/lix:dependabot/github_actions/cachix/install-nix-action-25 guiwf/lix:dependabot/github_actions/actions/labeler-5 guiwf/lix:dependabot/github_actions/cachix/cachix-action-14
guiwf/lix:2.95.3 guiwf/lix:2.93.4 guiwf/lix:2.94.2 guiwf/lix:2.95.2 guiwf/lix:2.95.1 guiwf/lix:2.94.1 guiwf/lix:2.95.0 guiwf/lix:2.94.0 guiwf/lix:2.93.3 guiwf/lix:2.92.3 guiwf/lix:2.93.2 guiwf/lix:2.91.3 guiwf/lix:2.92.2 guiwf/lix:2.91.2 guiwf/lix:2.93.1 guiwf/lix:2.93.0 guiwf/lix:2.92.0 guiwf/lix:2.91.1 guiwf/lix:2.91.0 guiwf/lix:2.90.0 guiwf/lix:2.90.0-rc1 guiwf/lix:2.90-beta.1 guiwf/lix:2.90-beta.0 guiwf/lix:1.11.1 guiwf/lix:1.11 guiwf/lix:1.10 guiwf/lix:1.9 guiwf/lix:1.8 guiwf/lix:1.7 guiwf/lix:1.6.1 guiwf/lix:1.6 guiwf/lix:1.5.3 guiwf/lix:1.5.2 guiwf/lix:1.5.1 guiwf/lix:1.5 guiwf/lix:1.4 guiwf/lix:1.3 guiwf/lix:1.2 guiwf/lix:1.1 guiwf/lix:1.0
76 Commits
main .. release-2.93
Author SHA1 Message Date
Raito Bezarius c2dbb0f5bf release: merge release 2.93.4 back to mainline 
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh

Change-Id: I34efc0973f445d42e2afaacf2b7fb65c5566bec9
 2026-05-04 19:15:52 +02:00
Raito Bezarius b67ee8e801 release: 2.93.4 "Bici Bici" 
Release produced with releng/create_release.xsh

Change-Id: I24bd93272db7b333f509f8d77820dbf5db5339e3
 2026-05-04 19:15:52 +02:00
Raito Bezarius b1469316cf release: release notes for 2.93.4 
Release created with releng/create_release.xsh

Change-Id: I38620bb46a4fccaf3e07a7505c749110a50a5689
 2026-05-04 19:15:40 +02:00
Raito Bezarius ac2abb6aa4 version: bump to 2.93.4 
Change-Id: Ia75252169c88be3a022ecb6efd4b5f9ca89c6fb0
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2026-05-04 19:15:30 +02:00
eldritch horrorsandRaito Bezarius 0eb56266a0 libutil: fix nar parser buffer overflow 
string data shares a buffer with the binary string length field. size
calculations for string read buffers always include the length field;
sufficiently large length fields can cause these calculations to wrap.
a malicious nar could use this for OOB writes in the daemon (as root).

since we use strings only as tags for archive members and for symlinks
with their OS-dependent length limits we can simply limit string size.
1 MiB should be sufficient for all symlinks, and tags are always tiny.

Change-Id: I89fb05f73c1dbeda45d91244aba4cd526a3d83e1
 2026-05-04 19:06:27 +02:00
Raito Bezarius 1410c6ac7d releng/keys: update the way to receive the ephemeral key 
I don't understand how `ssh -l lix-releng` is supposed to work if it
doesn't say which host to target.

Change-Id: I791f3f3f49ecd5884c9e86b5d3b617fc139e031f
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2026-05-04 19:06:27 +02:00
Raito Bezarius 4746f2e4d5 releng/environment: update staging parameters 
These parameters are now created on https://s3.afnix.fr.

Change-Id: I96b6fd913429ee46d04c412cb141edd288665ced
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2026-05-04 19:06:27 +02:00
YurekaandRaito Bezarius dcb715e773 releng: Adapt for AFNix S3 
Change-Id: I29dbd62dcc70595ba3f2ac2a466a5c26a28aea99
(cherry picked from commit 0c63036c7d)
 2026-05-04 19:06:27 +02:00
Florian KlinkandRaito Bezarius 61b44f783c libcstore: Fix null deref in writeDebugInfo for non-directory NARs 
When index-debug-info is enabled and the store path being copied is a
regular file (not a directory), std::get_if<nar_index::Directory>
returns nullptr since the NAR root is a File variant. The loop then
immediately dereferences buildIdDir->contents on the null pointer,
causing a segfault.

Add a null check at the top of the loop to break early when the NAR
root is not a directory.

Change-Id: I3a6e792b84cc12c837ecaddf4fee889e1bcb6397
(cherry picked from commit 6c7ccc2588)
 2026-05-04 16:33:27 +00:00
sterni 94cbf73aa2 libcmd: add support for lowdown >= 3.0.0 
lowdown 3.0.0 merged some flags into one to save on bits and did not add
any aliases for backward compatibility.

As with the changes for lowdown >= 1.4, we define a preprocessor flag to
gate the changes on and add a job to CI to ensure that lowdown < 3.0
keeps working (which is used by NixOS 25.11).

Unfortunately, we need to jump through some hoops to prevent nix and lix
from upstream Nixpkgs from being rebuilt due to a changed lowdown. Since
both implementation's packaging in Nixpkgs has their own package set /
fix point now, we can't simply inherit them from `prev` since they will
always be (re-)computed from the `final` fix point. As a consequence,
we need to expose our changed lowdown version at a non-default attribute
or break the builds of Nixpkgs derivations we test against.

Change-Id: I20a3e2fdaa05906f032ff66911c42867557fdd11
(cherry picked from commit e839708839aa132c8694abfdf83409a619f72c5a)
 2026-03-30 18:38:50 +02:00
Qyriad fbe811e94e libutil: include LIX_MAJOR, LIX_MINOR, and LIX_PATCH macros 
Fixes #1059.

Backport of I7d8a4648890fce7ff15695876c9b9d3a6a6a6964 to 2.93 branch.

Change-Id: Ia6b8d89007974c8cf873fa1f536ce8416a6a6964
 2026-01-15 12:40:52 +01:00
Qyriad 53dc27f752 nix3-develop: correctly escape ASCII control characters for JSON 
Fixes #991.
(cherry picked from commit 138c7161be)

Change-Id: Iafc7d9603fbc3615393d32d9630f0e8fe548950b
 2025-11-18 20:38:27 +01:00
Qyriad ced467fe49 libutil: add bashEscape, which escapes non-printing characters with $'' 
(cherry picked from commit ee91eec5cf)

Change-Id: I7e5c88ebe27c0a283982f8ac25f0fb0c6a6a6964
 2025-11-18 20:38:01 +01:00
EmilyandQyriad d76581dbcb libstore: skip the nested build directory on Darwin 
This is unnecessary because Darwin builds already cannot create
`set{u,g}id` files due to the minimal sandbox policy, because we can
forbid messing with the top‐level build directory directly in the
sandbox rules, and because Darwin builds can trivially avoid isolation
through temporary directories right now anyway.

This does regress the build directory isolation for builds with
`sandbox = false`, but I can’t imagine that mattering given the
above. The sandbox change prepares us for a world where we close
off shared temporary directories for `sandbox = true` builds and try
harder to achieve proper isolation on Darwin, but probably doesn’t
have a meaningful security impact one way or another for now.

With this change, we get down to 41 byte build directory paths on
Darwin, a ~2–3½ byte improvement over the old status quo. We can
also restore the 0710 permissions on Linux.

Change-Id: I6a6a6964a681c0365241fe7234831db656b76799
 2025-11-18 11:58:03 +01:00
EmilyandQyriad b5971baa4f libstore: default to /nix/var/nix/b for build-dir 
The minimum build directory length on Darwin with default settings
when we were still using `/tmp` was 22 bytes. Deriving build directory
names from my local store, the median and mode were 43, the mean was
around 44½, and the maximum was 127.

The switch to `/nix/var/nix/builds` over `/tmp` added a 15 byte
penalty, and the additional `/b` directory added another 2.

Now that we use opaque build directory names, the length is 48, so
we’re still at a ~3½–5 byte penalty over the previous status
quo. This change brings us down to 43, matching the previous median.

Note that these calculations do not take into account the fact that
`/tmp` is a symbolic link to `/private/tmp` on Darwin. Anything
that was canonicalizing paths would have had an additional 8 byte
penalty in the previous status quo that is not applicable here,
so we may already be ahead even without this change. If the more
opaque directory name here is undesirable, then that factor could
potentially help us squeeze by without. Alternatively, in combination
with dropping the `/b` on Darwin we could use `/nix/var/nix/bld` or
similar, but I feel that the paths in general are sufficiently opaque
that it should be okay to go with the shorter option here. Given that
some projects already had to reduce filename lengths to avoid this
limit even before the recent changes, I think it is best to try and
improve on the previous status quo.

Note that `/nix/var/nix/builds` will unfortunately not be cleaned up
on Darwin. However, we don’t clean up the directories inside it on
Darwin anyway, so hopefully that’s okay for now?

Closes: #913
Change-Id: I6a6a6964bffce7194bcddcaefb4c4a37569c7df5
 2025-11-18 11:55:23 +01:00
EmilyandQyriad 02aefad372 libstore: don’t include derivation names in build directories 
They have variable size, which is bad for #913.

Change-Id: I6a6a6964870e984c66277c7556ff3c2bc34ddca1
 2025-11-18 11:52:06 +01:00
EmilyandQyriad ba71ad6236 tests: remove obsolete code to create custom build directories 
We now do in fact do this and decide this.

Change-Id: I6a6a696493b288ed76d809122bda259dc0225846
 2025-11-18 11:52:06 +01:00
EmilyandQyriad 24348f9bca libstore: make temporary path prefixes optional 
This is not the same thing as passing an empty string, because it
avoids the `-` separator.

Change-Id: I6a6a696451667cbf500914e2dfbca2a4646ff20b
 2025-11-18 11:52:06 +01:00
EmilyandQyriad 5b0bc2e5b4 libstore: simplify createTempDir interface 
We always use the default temporary directory, because
`createUniqueDir` has an interface nice enough to use directly for
the few bespoke uses.

Change-Id: I6a6a696450b7c0a0bd76655632fb14d7c5e38199
 2025-11-18 11:52:06 +01:00
EmilyandQyriad 1fa9c4d55f libutil: use makeTempPath in createTempSubdir 
This makes the paths more nondeterministic, but more reliably unique,
and lets us remove the retry loop.

Note that this adds random entropy to the build directory visible
inside derivations on Darwin and unsandboxed Linux. It was already
non‐deterministic in the presence of concurrent builds and similar,
but now we can reliably expect it to be different every time. On the
whole I think that’s a good thing, as it is impossible to ensure
a single consistent build directory and derivation outputs should
not depend on it.

Package reproducibility isn’t great on Darwin to begin with,
though, and the reproducibility bugs this will turn up in packages
will be more urgent to fix than when the build directory was mostly
consistent. A quick survey of my local store shows that many C, C++,
and Rust binaries contain build directory references, likely due to
use of `__FILE__` and its equivalents; non‐binary offenders include:

* Install logs included in the Rust and Cargo bootstrap compilers
* Example errors in the Rust documentation referencing build paths
* Configuration information installed with CPython itself
* Python 2 metadata from resholve’s closure
* Cython metadata
* Generated headers in Facebook libraries referencing source paths
* Generated CMake files in Facebook libraries referencing source paths

I haven’t built that much in this store since the last GC, so this is
probably only a small sample of the problems across the tree. These are
all instances of <https://reproducible-builds.org/docs/build-path/>,
though, and should probably just be treated as general reproducibility
bugs outside of contexts like the Linux sandbox where we can normalize
them away entirely.

I have implemented away build directory paths for C/C++, applied some
additional fixes for non‐`__FILE__`‐related issues in binaries
from ATF and LLVM, and fixed the derivation bug causing the CPython
3 issue, and will work on upstreaming these changes. Rust is working
on the problem upstream, with some temporary workarounds we can
potentially apply in Nixpkgs for now. The rest will require some
distributed effort.

Change-Id: I6a6a69645b4915c56c0fdef904aa81684e4136c6
 2025-11-18 11:47:10 +01:00
EmilyandQyriad 9f87a43076 libstore: simplify fallback build directory logic 
This does change the behaviour when the global temporary directory
does not exist, but other uses of the global temporary directory are
already broken in that circumstance, and it should be fixed centrally
if the use case is considered desirable. The logic was not present
before the recent churn around build directories – it was added now
that Lix is taking ownership of the build directory in the store –
so this should not be a meaningful regression.

Change-Id: I6a6a6964e345ea6803226c5ad759e836de7cb0ed
 2025-11-18 11:38:20 +01:00
EmilyandQyriad 84912edd66 libstore: use makeTemp{,Sibling}Path more 
Change-Id: I6a6a6964c885be6dea0a69ee3162fbf4b812471f
 2025-11-18 11:33:31 +01:00
EmilyandQyriad a9ac3d0173 libstore: simplify makeTemp{,Sibling}Path callers 
There is now no risk of race conditions on a system with a functioning
entropy source, and the bespoke prefixes are either redundant to the
default or unnecessary.

Change-Id: I6a6a69641211c6bb979ea48ad30aecb1a53d03f0
 2025-11-18 11:22:48 +01:00
EmilyandQyriad 1df3d8c79d libutil: use OS‐provided entropy for temporary filenames 
Relax the constraints on keeping the exact same filename format to
provide a more robust source of entropy with a simpler interface
(as previously suggested by eldritch horrors). Using 128 bits of
OS‐provided entropy ensures global uniqueness and allows us to
skip any thought of gracefully handling the case where these files
already exist.

My microbenchmark that repeatedly constructed paths like this and
printed them out showed that this takes about 1.23× the time of
the previous implementation, both taking on the order of a couple
microseconds for one iteration. Since everything that uses it is doing
things more expensive than printing to standard output, the actual
performance delta is likely to be lost in the noise. If it somehow
becomes a bottleneck, it can be optimized without sacrificing the
guarantees by reading from the system RNG only to seed a thread‐local
CSPRNG like [ChaCha8Rand], but I think that’s very unlikely.

We also tweak the recommended way of creating a temporary file inside
a directory in anticipation of later changes, and rename the `suffix`
parameter to `prefix` (it’s a prefix to the random characters and
a suffix to the root, but this way is more consistent).

[ChaCha8Rand]: https://c2sp.org/chacha8rand

Change-Id: I6a6a69648502c746d13d8c3bd2768cbbf1b90466
 2025-11-18 11:08:31 +01:00
EmilyandQyriad fc22163c57 libutil: extract Base32 helpers from Hash 
base32Encode now takes std::span<std::byte>, with a base32EncodeStr
convenience wrapper which takes std::string_view.

Co-authored-by: Qyriad <qyriad@qyriad.me>

Change-Id: I6a6a6964f799dc84ecbfb55c7ca03a064cff71d9
 2025-11-18 11:08:31 +01:00
EmilyandQyriad 8a27e3d657 libstore: use makeTempSiblingPath in replaceValidPath 
Change-Id: I6a6a69641a3b4e6fdd076faac44dc314e6cc057e
 2025-11-18 11:08:31 +01:00
EmilyandQyriad 1cc3989c8e libutil: add makeTempSiblingPath helper 
The prospective callers of this should probably be doing something
smarter or more abstracted to begin with, but this is useful as an
incremental improvement for call sites with existing `makeTempPath`
logic in the face of filename length limits.

Change-Id: I6a6a69644292f5bbf984a1df90192e06c6022b53
 2025-11-18 11:08:10 +01:00
Raito Bezarius 75c0314204 nix3/develop: support structured attrs-based output checks 
nix develop should ignore output checks in general.

This was done only for the old way of specifying output checks, the
structured attrs way requires rewriting the JSON and removing the output
checks pieces.

We take a brutal approach of removing as many as possible including
non-recommended ways of doing it.

Fixes #997.

Change-Id: Iaf83029016c71b5171e56e15d4eadc1a60a8be98
Signed-off-by: Raito Bezarius <raito@lix.systems>
(cherry picked from commit 992c3ae981)
 2025-10-08 16:25:03 +00:00
Raito Bezarius 9bfef6a06c legacy/nix-build: create various temporary directories into a known tempdir 
Fixes fj#940.

When running `nix-shell`, the `$NIX_BUILD_TOP` environment variable is
set to `$TMPDIR` or `/tmp`.

nixpkgs stdenv uses $NIX_BUILD_TOP to create `$NIX_BUILD_TOP/env-vars`
which contains all the environment variables set by stdenv. This is used
for debugging purposes in combination with `--keep-failed` to reload the
bash environment of a derivation.

`$TMPDIR` is often unset, therefore, `/tmp/env-vars` was constantly
being created. On a multi-user system or, when you run Lix as root, you
might create a `/tmp/env-vars` with different permission bits.

As a result, `nix-shell` can cease to function because that file will
fail creation for an unprivileged user for example.

fj#940 rightfully remark that the code is not consistent between
nix3-develop and nix-shell and it should be reworked.

Change-Id: Iddf15945385d8bd497b2800b37fee5e1f97689b7
Signed-off-by: Raito Bezarius <raito@lix.systems>
(cherry picked from commit feab75bde0)
 2025-10-07 13:16:25 +00:00
Marie Ramlow b7c2f17e91 meson: link against libatomic if required 
Some platforms like 32-Bit PowerPC need linking against libatomic.
Try to compile and link a very simple snippet of code which uses atomics
and make libatomic required if it fails.

Change-Id: I6a6a696471e1d352fb161c537ba9023b97c2d31e
(cherry picked from commit 95448347ea)
 2025-09-13 21:32:01 +02:00
Sergei ZimmermanandEmily b6d5670bcf libexpr: Canonicalize TOML timestamps for toml11 > 4.0 
This addresses several changes from toml11 4.0 bump in
nixpkgs [1].

1. Added more regression tests for timestamp formats.
   Special attention needs to be paid to the precision
   of the subsecond range for local-time. Prior versions select the closest
   (upwards) multiple of 3 with a hard cap of 9 digits.

2. Normalize local datetime and offset datetime to always
   use the uppercase separator `T`. This is actually the issue
   surfaced in [2]. This canonicalization is basically a requirement
   by (a certain reading) of rfc3339 section 5.6 [3].

3. If using toml11 >= 4.0 also keep the old behavior wrt
   to the number of digits used for subsecond part of the local-time.

[1]: https://www.github.com/NixOS/nixpkgs/pull/331649
[2]: https://www.github.com/NixOS/nix/issues/11441
[3]: https://datatracker.ietf.org/doc/html/rfc3339

(cherry picked from commit dc769d72cb8ad22a0f89768682b5499a9d2b3d8b)
Upstream-PR: https://github.com/NixOS/nix/pull/13741
Change-Id: Iac4fbe5108be79be585e9670fa42dfd11f3c5e89
(cherry picked from commit 2898b9e7dc)
 2025-09-12 17:06:38 +01:00
Sergei ZimmermanandEmily 176b834464 libexpr: Use table.size() instead of unnecessary loop 
(cherry picked from commit d8fc55a46e0c09241131097dbf1d6fa09e0a9808)
Upstream-PR: https://github.com/NixOS/nix/pull/13741
Change-Id: I8a11e21ae3bff3a885e13fbab74e1deb162a34cf
(cherry picked from commit 19d9a87c2f)
 2025-09-12 17:06:38 +01:00
Sergei ZimmermanandEmily e29a1ccf0a libexpr: Use recursive lambda instead of std::function 
There's no reason to use a std::function for recursive lambdas
since there are polymorphic lambdas.

(cherry picked from commit a80a5c4dba0d944fab8f5ed57a343869ae96bf16)
Upstream-PR: https://github.com/NixOS/nix/pull/13741
Change-Id: I593bd04597e2ae000374ca1eca4d8928e986c0b5
(cherry picked from commit 5badc1bc8a)
 2025-09-12 17:06:38 +01:00
Sergei ZimmermanandEmily ad52cbde2f libexpr: Remove extra trailing semicolons (NFC) 
This looks really weird after the reformat.

(cherry picked from commit df4e55ffc13c413e270af134227115a20a2341ba)
Upstream-PR: https://github.com/NixOS/nix/pull/13741
Change-Id: I8de92d58620cc4545a31d8b7d533d2f1e9f4f233
(cherry picked from commit 2ca5670ec3)
 2025-09-12 17:06:38 +01:00
Sergei ZimmermanandEmily 699d3a63a6 tests/functional/lang: Add more tests for TOML timestamps 
Current test suite doesn't cover the subsecond formatting at
all and toml11 is quite finicky with that. We should at the very
least test its behavior to avoid silent breakages on updates.

(cherry picked from commit 7ed0229d1abd4414144c7af396842462ce6fc1eb)
Upstream-PR: https://github.com/NixOS/nix/pull/13741
Change-Id: I6a6a696433b168072d6ad2585dce8a3c10ccbc39
(cherry picked from commit b2e48aac5c)
 2025-09-12 17:06:38 +01:00
Emily 96a39dc464 libexpr: format fromTOML source 
Otherwise the next diff becomes very messy.

Change-Id: I6a6a6964d96543ade130d491f413ebd9fe2b7ff1
(cherry picked from commit c586596a9f)
 2025-09-12 17:06:38 +01:00
Alois Wohlschlager c8dc916356 flake: update nixpkgs input 
Without https://github.com/NixOS/nixpkgs/pull/434761 evaluation of the
`nixpkgsLibTests` will fail in CI with recent enough Lix, due to reliance on
the TOML integer saturation bug.

Reported-by: Sergei Zimmerman <sergei@zimmerman.foo>

Change-Id: I6a6a6964838009d2c525f67035f84072fdfad988
 2025-09-07 19:24:17 +02:00
sternenseemann 1a4393d0aa libcmd: add support for lowdown >= 1.4 
lowdown 1.4.0 changed the lowdown_opts to include a new and separate
lowdown_opts_term which allows for configuring values specific to
-Tterm (which we're using). This version should have been called 2.0.0
according to semver, hence 2.0.0 was released later without any actual
breaking changes to sort of migitate the problem.

We need to support lowdown >= 1.3 && < 1.4 since the ship has sailed for
updating lowdown in NixOS 25.05 as well as lowdown >= 1.4 or we'll be
stuck in Nixpkgs forever. Support for < 1.4 can be dropped as soon as
NixOS 25.05 is EOL, assuming this change lands before NixOS 25.11
branch-off.

We detect the changed API based on the lowdown version from pkg-config
and define LOWDOWN_SEPARATE_TERM_OPTS based on that. The ifdef is named
according to the specific API change that impacts us, so that it's
hopefully a little simpler to maintain going forward. In the new API,
all newly configurable settings use what would have been the (implicit)
default before. Changing some of these values, especially hpadding,
could be interesting in future changes.

Compared to cl/3081, this change makes sure to initialize all new fields
of lowdown_opts_term explicitly.

It seems that, while making -Tterm more configurable, lowdown's word
wrapping behavior changed slightly which broke basic_repl.test. I've
chosen to work around this by using builtins.add as an example which has
a very short documentation string, so wrapping doesn't matter.

Change-Id: Id73be4c0e43d7eb4f56e10a261b4254402698ff8
(cherry picked from commit 858de5f47a)
 2025-07-23 23:42:04 +02:00
Jade Lovelace 7ac20fc47c release: merge release 2.93.3 back to mainline 
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh

Change-Id: Ie9fa603173d7aab8a4ca8a6a0594158779701405
 2025-07-22 15:27:09 -07:00
Jade Lovelace e101400359 release: 2.93.3 "Bici Bici" 
Release produced with releng/create_release.xsh

Change-Id: I49a2c0c8bd79e864809b64d4c8d2b0049d570c02
 2025-07-22 15:27:08 -07:00
Jade Lovelace 54fdb1edd8 release: release notes for 2.93.3 
Release created with releng/create_release.xsh

Change-Id: Iaea203835f892efb783995543abcb1ea7c520a4a
 2025-07-22 15:26:56 -07:00
Jade Lovelaceandjade dc6d5962a5 version: 2.93.3 
Change-Id: I87df39a21f700eb973627ad0d39b532187901322
 2025-07-20 20:21:53 +00:00
Jade Lovelace 927facd35d fix: VERSION_SUFFIX was not getting into meson 
It was a regression caused by switching to structured attrs, I think.

Fixes: https://git.lix.systems/lix-project/lix/issues/908
Change-Id: Ia62892919945a1f16a81a2e0bb585595fac46669
(cherry picked from commit ae00b12983)
 2025-07-20 20:21:28 +00:00
K900andjade ba5b1cd1cc packaging: use structuredAttrs 
staging-next banned !structuredAttrs && separateDebugInfo && disallowedRequisites
due to weird output interactions. Enable structuredAttrs so we can build again.

Also, fix type confusion that makes stdenv explode (https://github.com/NixOS/nixpkgs/issues/422989).

Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: Ic0c773394ee79e10d427f27750d59892d6d1f1d1
(cherry picked from commit 378b360bf8)
 2025-07-20 20:21:28 +00:00
eldritch horrors a6201a64e5 libstore: weaken tmpdir root access mode 
libarchive *should* not break with 0710 on the tmpdir root on darwin,
just like it doesn't break on linux, but for some reason it does. the
restriction to 0710 can be weakened to 0750 with causing any trouble.

fixes #921

Change-Id: Ia9fc2f8eb9695fc19cefae9857368d5a4e58c8b9
 2025-07-20 16:52:29 +00:00
eldritch horrorsandRaito Bezarius 65c0ede1e9 libstore: chown build dirs with --keep-failed 
although we only chown if the build was requested by a local daemon
user. daemonless invocations will not chown as they do not have to.
remote builds *can* chown to the remote builder user, but that does
not seem to happen (for some reason keep-failed is not propagated).

Change-Id: Ic0ead406b38b4ca0556fec42d84888efa25123bf
(cherry picked from commit ae3b8e58c3)
 2025-07-18 14:12:50 +02:00
eldritch horrorsandRaito Bezarius 18e56efd9c libstore: add intermediate directory to build-dirs 
this makes the actual build directories used by builders invisible and
inaccessible to other processes on the system, avoiding another vector
for outside processes to interfere with builds or pass credentials the
build sandbox should not have access to into the build sandbox anyway.

fixes #919

Change-Id: Ifaa4d8e3940cfde1406e925f75c1375d2e86d81a
(cherry picked from commit 9d5a5c4dc0)
 2025-07-17 09:43:01 +00:00
Raito Bezarius f3a7bbe5f8 release: merge release 2.93.2 back to mainline 
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh

Change-Id: Ia72a7fd2461f07398c3eb0f49e7448300688dfe9
 2025-06-30 00:21:44 +02:00
Raito Bezarius 1d7368585e release: 2.93.2 "Bici Bici" 
Release produced with releng/create_release.xsh

Change-Id: I9b3c2bafcd124f53fbe91058f8015e229063ea02
 2025-06-30 00:21:44 +02:00
Raito Bezarius 016d019340 release: release notes for 2.93.2 
Release created with releng/create_release.xsh

Change-Id: I643d70eaf19440325b2f66ec5f976f7ed4362949
 2025-06-30 00:21:41 +02:00
Raito Bezarius f6ad1bfefb version: 2.93.1 -> 2.93.2 
Resolves critical correctness bugs following CVE fixes.

Change-Id: Iaa9b59feab438744e71d3c03ecf4f165699bfea5
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2025-06-29 20:33:35 +00:00
EmilyandRaito Bezarius ff16735ca5 libstore: fix Unix sockets in the build directory on sandboxed macOS 
We’re already allowing `/tmp` anyway, so this should be harmless,
and it fixes a regression in the default configuration caused by
moving the build directories out of `temp-dir`. (For instance, that
broke the Lix `guessOrInventPath.sockets` test.)

Note that removing `/tmp` breaks quite a few builds, so although it may
be a good idea in general it would require work on the Nixpkgs side.

Fixes: 749afbbe99
Change-Id: I6a6a69645f429bc50d4cb24283feda3d3091f534
(cherry picked from commit d1db3e5fa3)
 2025-06-29 20:33:35 +00:00
Raito Bezarius 85d1465b93 libstore: fallback on creating a safe space in the default tempdir 
If `settings.buildDir` cannot be written to, because we are in a chroot
store, unprivileged or anything.

We can and should always gracefully fallback to a *secure* location
inside of /tmp, i.e. `/tmp/<a directory under 0700>/<our temporary
directory for build under 0700>/...`.

This does not reintroduce CVE-2025-52991 because we are creating a
directory in-between compared to creating only ONE level of directory.

Under macOS, the first level of directory has actually mode 0755 instead
of 0700 as macOS often do not possess the right primitives to chroot
inside of these directories, leading to
https://github.com/NixOS/nix/pull/11031.

Thanks to Emily for the heads-up on this type of matter.

Fixes #876.

Change-Id: Ie521202923f763225e1901ab1b9b6c6132aaf548
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2025-06-29 20:33:35 +00:00
eldritch horrors 5a0ab5af09 Revert "libstore/build: automatic clean up of unsuccessfully built scratch outputs" 
This reverts commit f85c84db37 as this is the root cause for the critical correctness bug.

Change-Id: If71516db54138201039473485fb3cf7b5f49ccb0
 2025-06-29 20:19:30 +00:00
Raito Bezarius 9d40ddb627 releng: move back to a non-official release 
Required to make the releng scripts work.

I know this is not optimal and we should have a proper merge commit from
releng/2.93.1 appearing here, but this is fine.

Change-Id: I11f8ccb8d2a5b124cd057d948aa80dd8be3a7ffd
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2025-06-27 23:30:32 +02:00
Raito Bezarius 38b358ce27 release: 2.93.1 "Bici Bici" 
Release produced with releng/create_release.xsh

Change-Id: I64c89d0fba1e228136e50738e7a61d53306d14e6
 2025-06-24 10:50:03 +00:00
Raito Bezarius 24edb364b2 release: release notes for 2.93.1 
Release created with releng/create_release.xsh

Change-Id: I2d80bc68b7dd184ccf449de747ac46de6ac8786a
 2025-06-24 10:50:03 +00:00
Raito Bezarius 7e8c005d44 version: 2.93.0 -> 2.93.1 
* Announce the deprecation of ca-derivations and various other features
as planned initially.
* Fixes papercuts in 2.93.0 (SSH connections).
* Fixes the curl download bug for non-Nixpkgs users.
* Fixes CVE-2025-46415, CVE-2025-46416, CVE-2025-52991, CVE-2025-52992,
and CVE-2025-52993.

Change-Id: I8f700396a5ac57d2a1832833f83c22645c73697d
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2025-06-24 10:50:03 +00:00
Raito Bezarius f85c84db37 libstore/build: automatic clean up of unsuccessfully built scratch outputs 
When a build fails, its scratch output paths are not cleaned up.

Until recently, this was deemed not a problem but as part of the effort
to harden the Nix builds and protect these paths against being part of a
staged attack (race conditions, etc.), we automatically cleanup after
failed builds.

Fixes CVE-2025-52992.

Change-Id: I58481b1cc83826298b9d80d37fecf81f117ccb09
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2025-06-24 10:50:03 +00:00
eldritch horrorsandRaito Bezarius 469cb4218d libstore: don't default build-dir to temp-dir 
if a build directory is accessible to other users it is possible to
smuggle data in and out of build directories. usually this ins only
a build purity problem, but in combination with other issues it can
be used to break out of a build sandbox. to prevent this we default
to using a subdirectory of nixStateDir (which is more restrictive).

Fixes CVE-2025-52991.

Change-Id: Iacfc9b50534de158618c815f9fb99d7dae1be4d0
 2025-06-24 10:50:03 +00:00
eldritch horrorsRaito BezariusPuck Meerburg
 		959f6cb084 libstore: use pasta for FODs if available 
This allows using a userspace program, pasta, to handle comms between
the build sandbox, and the outside world; allowing for full isolation
including the network namespace, closing the "fixed-output derivation
talks to the host over an abstract domain socket" hole for good.

Fixes CVE-2025-46416.

Co-Authored-By: Puck Meerburg <puck@puckipedia.com>
Change-Id: Ifd499b7dbb3784600a6e842fede65fc031ff9f15
 2025-06-24 10:50:03 +00:00
eldritch horrorsandRaito Bezarius c773df3b58 libutil: add capability support to runProgram2 
launching pasta to not run as root will ambient require capabilities.

Change-Id: I1dd2506a1fa3944a9d9062123ef8a74903c597ea
 2025-06-24 10:50:03 +00:00
eldritch horrorsandRaito Bezarius 8ceda6db13 libutil: add generic redirections runProgram2 
explicit stderr redirection makes mergeStderrToStdout unnecessary also.

Change-Id: I63de929e6dc53f6c5ceb2d43c2ce288bfc04d872
 2025-06-24 10:50:03 +00:00
eldritch horrorsandRaito Bezarius 58b113d623 libutil: make RunningProgram more useful 
make it moveable, make it killable, and add a stdout fd accessor.

Change-Id: I2387cbe8ac67b899a322cd6c7d306ef9ea7abcd0
 2025-06-24 10:50:03 +00:00
Raito Bezarius 0df9344b28 libutil: ensure that _deletePath does NOT use absolute paths with dirfds 
When calling `_deletePath` with a parent file descriptor, `openat` is
made effective by using relative paths to the directory file descriptor.

To avoid the problem, the signature is changed to resist misuse with an
assert in the prologue of the function.

Fixes CVE-2025-46415.

Change-Id: I6b3fc766bad2afe54dc27d47d1df3873e188de96
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2025-06-24 10:50:03 +00:00
Raito Bezarius c085f5160a libstore: ensure that passAsFile is created in the original temp dir 
This ensures that `passAsFile` data is created inside the expected
temporary build directory by `openat()` from the parent directory file
descriptor.

Fixes CVE-2025-52993.

Change-Id: Ie5273446c4a19403088d0389ae8e3f473af8879a
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2025-06-24 10:46:12 +00:00
Raito Bezarius 77daadb029 libutil: writeFile variant for file descriptors 
`writeFile` lose its `sync` boolean flag to make things simpler.

A new `writeFileAndSync` function is created and all call sites are
converted to it.

Change-Id: Ib871a5283a9c047db1e4fe48a241506e4aab9192
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2025-06-23 17:25:59 +02:00
Raito Bezarius 3f02ca5c35 libstore: chown to builder variant for file descriptors 
We use it immediately for the build temporary directory.

Change-Id: I180193c63a2b98721f5fb8e542c4e39c099bb947
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2025-06-23 16:53:12 +02:00
Raito Bezarius 1a4cb13411 libstore: open build directory as a dirfd as well 
We now keep around a proper AutoCloseFD around the temporary directory
which we plan to use for openat operations and avoiding the build
directory being swapped out while we are doing something else.

Change-Id: I18d387b0f123ebf2d20c6405cd47ebadc5505f2a
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2025-06-23 16:53:12 +02:00
Raito Bezarius e9f0354f7a libutil: guess or invent a path from file descriptors 
This is useful for certain error recovery paths (no pun intended) that
does not thread through the original path name.

Change-Id: I2d800740cb4f9912e64c923120d3f977c58ccb7e
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2025-06-23 16:53:12 +02:00
Raito Bezarius fbd6a014ec flake/inputs: 24.11-small -> 25.05-small 
We upgrade to 25.05 release, which contains the curl commit
https://github.com/curl/curl/commit/5fbd78eb2dc4afbd8884e8eed27147fc3d4318f6
done in
https://github.com/NixOS/nixpkgs/pull/396200#issuecomment-2795944006.

This fixes HTTP transfers generating arbitrary errors and possibly
failing unusually.

Users who are already depending on 25.05-small or a recent unstable
already had the fix.

Special mention to the Linux kernel who gave me the opportunity to get
on a 24 hours bisection side quest to fix the local release engineering
test.

Special thanks to everyone who had to endure me ranting.

Change-Id: I866caf65d5ea103f1fa5eccd57df8031c9eacda0
Co-authored-by: eldritch horrors <pennae@lix.systems>
Co-authored-by: helle <helle@h3l.li>
Signed-off-by: Raito Bezarius <raito@lix.systems>
(cherry picked from commit 1e34c37477)
 2025-06-19 17:09:59 +02:00
Linus Heckemann 2387104452 build: disable LTO on Darwin 
Due to https://git.lix.systems/lix-project/lix/issues/832 , Lix 2.93.0
fails to build on Darwin without overrides. Until the root cause has
been determined and fixed, build without LTO.

Change-Id: I4db5eb294d8f19e5a366b1e19efa5a327b3e2e78
(cherry picked from commit da94e860dd)
 2025-06-03 19:08:20 +00:00
eldritch horrors d84f13b73f nix: don't send tarballTtl to the daemon 
it's an eval-time only setting, the daemon doesn't use it anywhere. this
is a hack, but until we have a much better settings system we are stuck.

fixes #680

Change-Id: I532088b0279f13da0a0a65c2bd2e5f9d1dfb39da
(cherry picked from commit 5917db84aa)
 2025-06-03 18:49:43 +00:00
eldritch horrors 37a570bd40 deprecate CA, dynamic, and impure derivations 
ca derivations are what we're really after, but dynamic derivations
must also go because they depend on ca derivations. we can't easily
implement dynamic derivations any other way, so we remove them too.
impure derivations build on the content-addressed infrastructure in
ways we cannot easily detangle, so they too must go for time being.

see #815

Change-Id: If61371736dfd89cc71a1b2ae5a005757c3cb9484
(cherry picked from commit d8e2f53d07)
 2025-06-03 18:20:54 +00:00
Alois Wohlschlagerandeldritch horrors e62b7236e8 libstore/gc: fix auto-GC blocking indefinitely during evaluation 
The first auto-GC request would not be registered as a waiter due to a logic
error. As a result, if that request was synchronous (as happens during
evaluation) it would be stuck forever waiting on a promise that will never be
fulfilled.
Register also the first request properly so that it is notified and unblocked
again when the GC has finished. Also add a test verifying that auto-GC
triggering during evaluation will not get stuck.

Fixes: https://git.lix.systems/lix-project/lix/issues/844
Change-Id: I157afdc737415261e48d6d01d46c586a2927a1ad
(cherry picked from commit 4505bfac8e)
 2025-06-03 12:09:32 +00:00
Raito Bezarius 33eaaf02fd libstore/ssh: remove echo started check 
The pre-flight `echo started` check over SSH was originally added in
577ebeaefb. As it is usual with these old
commits, understanding why is there a need for something is difficult.

The closest thing would be
> Fix a race starting the SSH master. We now wait synchronously for
> the SSH master to finish starting. This prevents the SSH clients
> from starting their own connections.

But, we removed SSH connection sharing, so this does not apply anymore.

Nonetheless, we believed this check was meant as a way to catch obvious
misconfigurations or SSH failures early, before handing off to
`nix-store`. However, this approach was not fruitful: it assumes the
remote has a `bash`-compatible shell, `echo` behaves in a standard way,
and no `ForceCommand` interferes—all of which are unreliable assumptions
in practice.

While the intent was to provide slightly better diagnostics (e.g. in
case of SSH hanging or returning an interactive shell), in practice it
does not meaningfully catch or improve real failure cases. The
underlying protocol or engine can and should handle those errors more
robustly anyway.

In contrast, this check *does* break several legitimate workflows,
including:

* remote builders using `ForceCommand` wrappers (e.g.
`nix-remote-build`-style setups), see
<https://discourse.nixos.org/t/wrapper-to-restrict-builder-access-through-ssh-worth-upstreaming/25834/15>,

* SSHing into minimal environments lacking `bash` (e.g. initrd,
busybox-based systems),

* configurations that don’t default to POSIX-like shells, e.g., nushell
enthusiasts.

As such, we’re removing this code. Protocol mismatch errors and SSH
failures can be rethought and handled more structurally elsewhere in the
engine.

Change-Id: I187f6881375d42ef83987a13a350c97964bbdb30
Signed-off-by: Raito Bezarius <raito@lix.systems>
(cherry picked from commit 0dd8bf6c1c)
 2025-05-18 19:51:41 +00:00
2114 changed files with 34443 additions and 64959 deletions
Expand all files Collapse all files
.cargo/config.toml
-14
View File
@@ -1,14 +0,0 @@
[target.'cfg(true)']
rustflags = [
# rustc will pass `-nodefaultlibs` without this, but we need the C++ standard library.
'-Cdefault-linker-libraries=yes',
]
[target.'cfg(target_env = "musl")']
rustflags = [
'-Cdefault-linker-libraries=yes',
# musl, at least in Nixpkgs, is not compiled with -fPIE.
# XXX: nevermind? as of Nixpkgs 26.05??
# Oh gods do we need to gate this??
#'-Crelocation-model=static',
]
.clang-format
+3 -4
View File
@@ -4,10 +4,9 @@ AccessModifierOffset: -4
AlignAfterOpenBracket: BlockIndent AlignAfterOpenBracket: BlockIndent
AlignEscapedNewlines: Left AlignEscapedNewlines: Left
AlignOperands: DontAlign AlignOperands: DontAlign
AlignTrailingComments: false AllowShortBlocksOnASingleLine: Always
AllowShortBlocksOnASingleLine: Empty
AllowShortFunctionsOnASingleLine: Empty AllowShortFunctionsOnASingleLine: Empty
AllowShortIfStatementsOnASingleLine: Never AllowShortIfStatementsOnASingleLine: WithoutElse
AlwaysBreakBeforeMultilineStrings: true AlwaysBreakBeforeMultilineStrings: true
AlwaysBreakTemplateDeclarations: Yes AlwaysBreakTemplateDeclarations: Yes
BinPackArguments: false BinPackArguments: false
@@ -36,7 +35,7 @@ BreakAfterAttributes: Always
BreakBeforeBinaryOperators: NonAssignment BreakBeforeBinaryOperators: NonAssignment
BreakBeforeBraces: Custom BreakBeforeBraces: Custom
BreakConstructorInitializers: BeforeComma BreakConstructorInitializers: BeforeComma
ColumnLimit: 110 ColumnLimit: 100
EmptyLineAfterAccessModifier: Leave EmptyLineAfterAccessModifier: Leave
EmptyLineBeforeAccessModifier: Leave EmptyLineBeforeAccessModifier: Leave
FixNamespaceComments: false FixNamespaceComments: false
.clang-tidy
+7 -4
View File
@@ -8,16 +8,18 @@ Checks:
- -bugprone-narrowing-conversions - -bugprone-narrowing-conversions
# kind of nonsense # kind of nonsense
- -bugprone-easily-swappable-parameters - -bugprone-easily-swappable-parameters
# too many warnings for now
- -bugprone-implicit-widening-of-multiplication-result
# Lix's exception handling is Questionable # Lix's exception handling is Questionable
- -bugprone-empty-catch - -bugprone-empty-catch
# many warnings # many warnings
- -bugprone-unchecked-optional-access - -bugprone-unchecked-optional-access
# many warnings, seems like a questionable lint # many warnings, seems like a questionable lint
- -bugprone-branch-clone - -bugprone-branch-clone
# extremely noisy before clang 19: https://github.com/llvm/llvm-project/issues/93959
- -bugprone-multi-level-implicit-pointer-conversion
# we don't compile out our asserts # we don't compile out our asserts
- -bugprone-assert-side-effect - -bugprone-assert-side-effect
# FIXME(jade): figure out if this warning is any good
- -bugprone-exception-escape
# all thrown exceptions must derive from std::exception # all thrown exceptions must derive from std::exception
- hicpp-exception-baseclass - hicpp-exception-baseclass
# capturing async lambdas are dangerous # capturing async lambdas are dangerous
@@ -25,6 +27,9 @@ Checks:
# crimes must be appropriately declared as crimes # crimes must be appropriately declared as crimes
- cppcoreguidelines-pro-type-cstyle-cast - cppcoreguidelines-pro-type-cstyle-cast
- lix-* - lix-*
# This can not yet be applied to Lix itself since we need to do source
# reorganization so that lix/ include paths work.
- -lix-fixincludes
# This lint is included as an example, but the lib function it replaces is # This lint is included as an example, but the lib function it replaces is
# already gone. # already gone.
- -lix-hasprefixsuffix - -lix-hasprefixsuffix
@@ -33,5 +38,3 @@ Checks:
CheckOptions: CheckOptions:
bugprone-reserved-identifier.AllowedIdentifiers: '__asan_default_options' bugprone-reserved-identifier.AllowedIdentifiers: '__asan_default_options'
bugprone-unused-return-value.AllowCastToVoid: true bugprone-unused-return-value.AllowCastToVoid: true
ExtraArgs: ["-Werror=unnecessary-virtual-specifier"]
.editorconfig
-4
View File
@@ -33,7 +33,3 @@ max_line_length = 0
[meson.build] [meson.build]
indent_style = space indent_style = space
indent_size = 2 indent_size = 2
[*.json]
indent_style = space
indent_size = 4
.gitignore
+1 -7
View File
@@ -1,5 +1,4 @@
/build outputs/
/outputs
# GNU Global # GNU Global
GPATH GPATH
@@ -40,8 +39,3 @@ buildtime.bin
# Python compiled files from the code generators and test suite # Python compiled files from the code generators and test suite
*.pyc *.pyc
**/.idea
# Yeah, I've got no clue.
/subprojects/.wraplock
.mailmap
-3
View File
@@ -1,5 +1,2 @@
Fiona Behrens <me@kloenk.dev> Fiona Behrens <me@kloenk.dev>
Fiona Behrens <me@kloenk.dev> <me@kloenk.de> Fiona Behrens <me@kloenk.dev> <me@kloenk.de>
rootile <lix@rootile.de>
rootile <lix@rootile.de> <commentator2.0@crystal-cavern.systems>
rootile <lix@rootile.de> <lix@crystal-cavern.systems>
Cargo.lock
Generated
+5 -883
View File
@@ -1,255 +1,6 @@
# This file is automatically @generated by Cargo. # This file is automatically @generated by Cargo.
# It is not intended for manual editing. # It is not intended for manual editing.
version = 4 version = 3
[[package]]
name = "aho-corasick"
version = "1.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ddd31a130427c27518df266943a5308ed92d4b226cc639f5a8f1002816174301"
dependencies = [
"memchr",
]
[[package]]
name = "allocator-api2"
version = "0.2.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
[[package]]
name = "anstream"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "824a212faf96e9acacdbd09febd34438f8f711fb84e09a8916013cd7815ca28d"
dependencies = [
"anstyle",
"anstyle-parse",
"anstyle-query",
"anstyle-wincon",
"colorchoice",
"is_terminal_polyfill",
"utf8parse",
]
[[package]]
name = "anstyle"
version = "1.0.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000"
[[package]]
name = "anstyle-parse"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52ce7f38b242319f7cabaa6813055467063ecdc9d355bbb4ce0c68908cd8130e"
dependencies = [
"utf8parse",
]
[[package]]
name = "anstyle-query"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc"
dependencies = [
"windows-sys",
]
[[package]]
name = "anstyle-wincon"
version = "3.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d"
dependencies = [
"anstyle",
"once_cell_polyfill",
"windows-sys",
]
[[package]]
name = "ar_archive_writer"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4087686b4b0a3427190bae57a1d9a478dbb2d40c5dc1bd6e2b6d797913bdd348"
dependencies = [
"object",
]
[[package]]
name = "ariadne"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72fe02fc62033df9ba41cba57ee19acf5e742511a140c7dbc3a873e19a19a1bd"
dependencies = [
"unicode-width 0.1.14",
"yansi",
]
[[package]]
name = "askama"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b79091df18a97caea757e28cd2d5fda49c6cd4bd01ddffd7ff01ace0c0ad2c28"
dependencies = [
"askama_derive",
"askama_escape",
"humansize",
"num-traits",
"percent-encoding",
]
[[package]]
name = "askama_derive"
version = "0.12.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "19fe8d6cb13c4714962c072ea496f3392015f0989b1a2847bb4b2d9effd71d83"
dependencies = [
"askama_parser",
"basic-toml",
"mime",
"mime_guess",
"proc-macro2",
"quote",
"serde",
"syn",
]
[[package]]
name = "askama_escape"
version = "0.10.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "619743e34b5ba4e9703bba34deac3427c72507c7159f5fd030aea8cac0cfe341"
[[package]]
name = "askama_parser"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "acb1161c6b64d1c3d83108213c2a2533a342ac225aabd0bda218278c2ddb00c0"
dependencies = [
"nom",
]
[[package]]
name = "autocfg"
version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
[[package]]
name = "basic-toml"
version = "0.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba62675e8242a4c4e806d12f11d136e626e6c8361d6b829310732241652a178a"
dependencies = [
"serde",
]
[[package]]
name = "bitflags"
version = "2.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4388bee8683e3d04af747c73422af53102d2bd24d9eadb6cbc100baef4b43f8"
[[package]]
name = "block-buffer"
version = "0.10.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71"
dependencies = [
"generic-array",
]
[[package]]
name = "cc"
version = "1.2.65"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e228eec9be7c17ccb640b59b36a5cd805ea2a564a4c5e162c2f659fea30d3b96"
dependencies = [
"find-msvc-tools",
"shlex",
]
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "cfg_aliases"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724"
[[package]]
name = "chumsky"
version = "1.0.0-alpha.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0e82d74e6c83060ec269fe9e0d408d6de4a1645d525f9a0bbbb841ba4efd91ac"
dependencies = [
"hashbrown 0.15.5",
"regex-automata 0.3.9",
"serde",
"stacker",
"unicode-ident",
"unicode-segmentation",
]
[[package]]
name = "clap"
version = "4.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ddb117e43bbf7dacf0a4190fef4d345b9bad68dfc649cb349e7d17d28428e51"
dependencies = [
"clap_builder",
"clap_derive",
]
[[package]]
name = "clap_builder"
version = "4.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "714a53001bf66416adb0e2ef5ac857140e7dc3a0c48fb28b2f10762fc4b5069f"
dependencies = [
"anstream",
"anstyle",
"clap_lex",
"strsim",
]
[[package]]
name = "clap_derive"
version = "4.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2ce8604710f6733aa641a2b3731eaa1e8b3d9973d5e3565da11800813f997a9"
dependencies = [
"heck",
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "clap_lex"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9"
[[package]]
name = "clipboard-win"
version = "5.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bde03770d3df201d4fb868f2c9c59e66a3e4e2bd06692a0fe701e7103c7e84d4"
dependencies = [
"error-code",
]
[[package]]
name = "colorchoice"
version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
[[package]] [[package]]
name = "countme" name = "countme"
@@ -257,65 +8,12 @@ version = "3.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7704b5fdd17b18ae31c4c1da5a2e0305a2bf17b5249300a9ee9ed7b72114c636" checksum = "7704b5fdd17b18ae31c4c1da5a2e0305a2bf17b5249300a9ee9ed7b72114c636"
[[package]]
name = "cpufeatures"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280"
dependencies = [
"libc",
]
[[package]]
name = "crypto-common"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
dependencies = [
"generic-array",
"typenum",
]
[[package]]
name = "digest"
version = "0.10.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292"
dependencies = [
"block-buffer",
"crypto-common",
]
[[package]] [[package]]
name = "dissimilar" name = "dissimilar"
version = "1.0.9" version = "1.0.9"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59f8e79d1fbf76bdfbde321e902714bf6c49df88a7dda6fc682fc2979226962d" checksum = "59f8e79d1fbf76bdfbde321e902714bf6c49df88a7dda6fc682fc2979226962d"
[[package]]
name = "either"
version = "1.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "91622ff5e7162018101f2fea40d6ebf4a78bbe5a49736a2020649edf9693679e"
[[package]]
name = "endian-type"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "869b0adbda23651a9c5c0c3d270aac9fcb52e8622a8f2b17e57802d7791962f2"
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "error-code"
version = "3.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dea2df4cf52843e0452895c455a1a2cfbb842a1e7329671acf418fdc53ed4c59"
[[package]] [[package]]
name = "expect-test" name = "expect-test"
version = "1.5.0" version = "1.5.0"
@@ -326,139 +24,12 @@ dependencies = [
"once_cell", "once_cell",
] ]
[[package]]
name = "find-msvc-tools"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582"
[[package]]
name = "foldhash"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2"
[[package]]
name = "generic-array"
version = "0.14.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a"
dependencies = [
"typenum",
"version_check",
]
[[package]] [[package]]
name = "hashbrown" name = "hashbrown"
version = "0.14.5" version = "0.14.5"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1" checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1"
[[package]]
name = "hashbrown"
version = "0.15.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1"
dependencies = [
"allocator-api2",
"equivalent",
"foldhash",
]
[[package]]
name = "hashbrown"
version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
[[package]]
name = "heck"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "hex"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70"
[[package]]
name = "home"
version = "0.5.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cc627f471c528ff0c4a49e1d5e60450c8f6461dd6d10ba9dcd3a61d3dff7728d"
dependencies = [
"windows-sys",
]
[[package]]
name = "humansize"
version = "2.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6cb51c9a029ddc91b07a787f1d86b53ccfa49b0e86688c946ebe8d3555685dd7"
dependencies = [
"libm",
]
[[package]]
name = "indexmap"
version = "2.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d466e9454f08e4a911e14806c24e16fba1b4c121d1ea474396f396069cf949d9"
dependencies = [
"equivalent",
"hashbrown 0.17.1",
]
[[package]]
name = "is_terminal_polyfill"
version = "1.70.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695"
[[package]]
name = "itertools"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b1c173a5686ce8bfa551b3563d0c2170bf24ca44da99c7ca4bfdab5418c3fe57"
dependencies = [
"either",
]
[[package]]
name = "libc"
version = "0.2.186"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66"
[[package]]
name = "libm"
version = "0.2.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6d2cec3eae94f9f509c767b45932f1ada8350c4bdb85af2fcab4a3c14807981"
[[package]]
name = "licxxbridge"
version = "0.0.0"
dependencies = [
"clap",
"zngur",
]
[[package]]
name = "lix"
version = "0.0.0"
dependencies = [
"lix-doc",
"pkg-config",
"regex",
"rootcause",
"rustyline",
"rustyline-derive",
"zngur",
]
[[package]] [[package]]
name = "lix-doc" name = "lix-doc"
version = "0.0.1" version = "0.0.1"
@@ -468,228 +39,21 @@ dependencies = [
"rowan", "rowan",
] ]
[[package]]
name = "log"
version = "0.4.33"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "memchr"
version = "2.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "88904434abc2901f197fe8cc55f0445e7ded921dba5911dad2e2b39b48e663c4"
[[package]]
name = "mime"
version = "0.3.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6877bb514081ee2a7ff5ef9de3281f14a4dd4bceac4c09388074a6b5df8a139a"
[[package]]
name = "mime_guess"
version = "2.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f7c44f8e672c00fe5308fa235f821cb4198414e1c77935c1ab6948d3fd78550e"
dependencies = [
"mime",
"unicase",
]
[[package]]
name = "minimal-lexical"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a"
[[package]]
name = "nibble_vec"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "77a5d83df9f36fe23f0c3648c6bbb8b0298bb5f1939c8f2704431371f4b84d43"
dependencies = [
"smallvec",
]
[[package]]
name = "nix"
version = "0.31.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf20d2fde8ff38632c426f1165ed7436270b44f199fc55284c38276f9db47c3d"
dependencies = [
"bitflags",
"cfg-if",
"cfg_aliases",
"libc",
]
[[package]]
name = "nom"
version = "7.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a"
dependencies = [
"memchr",
"minimal-lexical",
]
[[package]]
name = "num-traits"
version = "0.2.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841"
dependencies = [
"autocfg",
]
[[package]]
name = "object"
version = "0.37.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ff76201f031d8863c38aa7f905eca4f53abbfa15f609db4277d44cd8938f33fe"
dependencies = [
"memchr",
]
[[package]] [[package]]
name = "once_cell" name = "once_cell"
version = "1.19.0" version = "1.19.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3fdb12b2476b595f9358c5161aa467c2438859caa136dec86c26fdd2efe17b92" checksum = "3fdb12b2476b595f9358c5161aa467c2438859caa136dec86c26fdd2efe17b92"
[[package]]
name = "once_cell_polyfill"
version = "1.70.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
[[package]]
name = "percent-encoding"
version = "2.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pkg-config"
version = "0.3.33"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "19f132c84eca552bf34cab8ec81f1c1dcc229b811638f9d283dceabe58c5569e"
[[package]]
name = "proc-macro2"
version = "1.0.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934"
dependencies = [
"unicode-ident",
]
[[package]]
name = "psm"
version = "0.1.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "645dbe486e346d9b5de3ef16ede18c26e6c70ad97418f4874b8b1889d6e761ea"
dependencies = [
"ar_archive_writer",
"cc",
]
[[package]]
name = "quote"
version = "1.0.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
dependencies = [
"proc-macro2",
]
[[package]]
name = "radix_trie"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3b4431027dcd37fc2a73ef740b5f233aa805897935b8bce0195e41bbf9a3289a"
dependencies = [
"endian-type",
"nibble_vec",
]
[[package]]
name = "regex"
version = "1.12.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f1292b7759ae1cb9ec195452d1390a074f0cd8541ab7a5a8c31cd6db45d4a6ba"
dependencies = [
"aho-corasick",
"memchr",
"regex-automata 0.4.14",
"regex-syntax 0.8.11",
]
[[package]]
name = "regex-automata"
version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59b23e92ee4318893fa3fe3e6fb365258efbfe6ac6ab30f090cdcbb7aa37efa9"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax 0.7.5",
]
[[package]]
name = "regex-automata"
version = "0.4.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e1dd4122fc1595e8162618945476892eefca7b88c52820e74af6262213cae8f"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax 0.8.11",
]
[[package]]
name = "regex-syntax"
version = "0.7.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dbb5fb1acd8a1a18b3dd5be62d25485eb770e05afb408a9627d14d451bae12da"
[[package]]
name = "regex-syntax"
version = "0.8.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
[[package]] [[package]]
name = "rnix" name = "rnix"
version = "0.12.0" version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6f15e00b0ab43abd70d50b6f8cd021290028f9b7fdd7cdfa6c35997173bc1ba9" checksum = "bb35cedbeb70e0ccabef2a31bcff0aebd114f19566086300b8f42c725fc2cb5f"
dependencies = [ dependencies = [
"rowan", "rowan",
] ]
[[package]]
name = "rootcause"
version = "0.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b660d9968fae12f4e691f2b2be5d9a3a6de875300c682e8d2cb89a618dd60875"
dependencies = [
"hashbrown 0.17.1",
"indexmap",
"rootcause-internals",
"rustc-hash 2.1.3",
"triomphe",
]
[[package]]
name = "rootcause-internals"
version = "0.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0184f6fcff3b58b7c963aee6e3cc915c04331aa6eef974f79d7d44d21e246c24"
dependencies = [
"triomphe",
]
[[package]] [[package]]
name = "rowan" name = "rowan"
version = "0.15.16" version = "0.15.16"
@@ -697,8 +61,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0a542b0253fa46e632d27a1dc5cf7b930de4df8659dc6e720b647fc72147ae3d" checksum = "0a542b0253fa46e632d27a1dc5cf7b930de4df8659dc6e720b647fc72147ae3d"
dependencies = [ dependencies = [
"countme", "countme",
"hashbrown 0.14.5", "hashbrown",
"rustc-hash 1.1.0", "rustc-hash",
"text-size", "text-size",
] ]
@@ -708,250 +72,8 @@ version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2" checksum = "08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2"
[[package]]
name = "rustc-hash"
version = "2.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6b1e7f9a428571be2dc5bc0505c13fb6bf936822b894ec87abf8a08a4e51742d"
[[package]]
name = "rustyline"
version = "18.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "53f6a737db68eb1a8ccff86b584b2fc13eca6a7bb6f78ebc7c529547e3ab9684"
dependencies = [
"bitflags",
"cfg-if",
"clipboard-win",
"home",
"libc",
"log",
"memchr",
"nix",
"radix_trie",
"unicode-segmentation",
"unicode-width 0.2.2",
"utf8parse",
"windows-sys",
]
[[package]]
name = "rustyline-derive"
version = "0.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "64e5587417a3c4e16a4415e8d7d07f80998ed835ade621d19dfbe9fbe3205b0f"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "serde"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde_core"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "sha2"
version = "0.10.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283"
dependencies = [
"cfg-if",
"cpufeatures",
"digest",
]
[[package]]
name = "shlex"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
[[package]]
name = "smallvec"
version = "1.15.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ed6a63f02c8539c91a8685a86f4099661ba3da017932f6ebbea6de3f0fa7c90"
[[package]]
name = "stacker"
version = "0.1.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "640c8cdd92b6b12f5bcb1803ca3bbf5ab96e5e6b6b96b9ab77dabe9e880b3190"
dependencies = [
"cc",
"cfg-if",
"libc",
"psm",
"windows-sys",
]
[[package]]
name = "strsim"
version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f"
[[package]]
name = "syn"
version = "2.0.118"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1b9ae57f904213ebb649ce6895b8a66c66f0203b9319718f69a5612a065b1422"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]] [[package]]
name = "text-size" name = "text-size"
version = "1.1.1" version = "1.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f18aa187839b2bdb1ad2fa35ead8c4c2976b64e4363c386d45ac0f7ee85c9233" checksum = "f18aa187839b2bdb1ad2fa35ead8c4c2976b64e4363c386d45ac0f7ee85c9233"
[[package]]
name = "triomphe"
version = "0.1.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b40688ea6389c8171614b25491f71d4a27946e0c7ce2da1c6de27e25abf1a0ae"
[[package]]
name = "typenum"
version = "1.20.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20"
[[package]]
name = "unicase"
version = "2.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dbc4bc3a9f746d862c45cb89d705aa10f187bb96c76001afab07a0d35ce60142"
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "unicode-segmentation"
version = "1.13.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c6f5d3c3b1bf09027a88a6bc961fc00497d651009560b5463668dc81b0fa87a8"
[[package]]
name = "unicode-width"
version = "0.1.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7dd6e30e90baa6f72411720665d41d89b9a3d039dc45b8faea1ddd07f617f6af"
[[package]]
name = "unicode-width"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4ac048d71ede7ee76d585517add45da530660ef4390e49b098733c6e897f254"
[[package]]
name = "utf8parse"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
[[package]]
name = "version_check"
version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
[[package]]
name = "windows-link"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-sys"
version = "0.61.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
dependencies = [
"windows-link",
]
[[package]]
name = "yansi"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09041cd90cf85f7f8b2df60c646f853b7f535ce68f85244eb6731cf89fa498ec"
[[package]]
name = "zngur"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7fc912d12934b4d04aabc52c14db6fc88ae8aa5a47902f27e8759c2de254723f"
dependencies = [
"zngur-generator",
]
[[package]]
name = "zngur-def"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f27e49a62db537cea43a6c122ded7eda99c4ac0ca1d7cfd74f2bf6a6c88712d"
dependencies = [
"indexmap",
"itertools",
]
[[package]]
name = "zngur-generator"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5ffad8c6994c477023aba7613bdf17c470003bbd191846d3096e84ffecf86d4c"
dependencies = [
"askama",
"hex",
"indexmap",
"itertools",
"sha2",
"zngur-def",
"zngur-parser",
]
[[package]]
name = "zngur-parser"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8d66c1b85ca6eab9576df5de31758cb1f9046b4ce47b25f45dabcb0fc7ef8789"
dependencies = [
"ariadne",
"chumsky",
"itertools",
"zngur-def",
]
Cargo.toml
+1 -23
View File
@@ -1,28 +1,6 @@
[workspace] [workspace]
resolver = "2" resolver = "2"
members = [ members = ["lix/lix-doc"]
"lix/lix-doc",
"lix/lix-rs",
"tools/licxxbridge",
]
[workspace.package] [workspace.package]
edition = "2021" edition = "2021"
[workspace.dependencies]
clap = "4"
regex = "1.12.4"
rootcause = "0.13.0"
rustyline = "18"
rustyline-derive = "0.12"
syn = "2.0"
zngur = "0.10"
pkg-config = "0.3.33"
[profile.dev]
opt-level = 1
[profile.release]
debug = "full"
debug-assertions = true
overflow-checks = true
OWNERS
-26
View File
@@ -1,26 +0,0 @@
# Docs
per-file README.md=*
per-file CONTRIBUTING.md=*
# DevX
per-file justfile=*
per-file .envrc=*
per-file .gitignore=*
per-file .github=*
per-file .mailmap=*
# Build
per-file meson.build=*
per-file meson.options=*
per-file flake.nix=*
per-file flake.lock=*
per-file *.nix=*
per-file Cargo.lock=*
per-file Cargo.toml=*
per-file version.json=*
# Code style
per-file .clang-tidy=*
per-file .clang-format=*
per-file .editorconfig=*
per-file treefmt.toml=*
bench/OWNERS
-1
View File
@@ -1 +0,0 @@
*
bench/bench.py
+36 -158
View File
@@ -7,45 +7,15 @@ import os
import json import json
import tempfile import tempfile
import platform import platform
import shlex
import textwrap
import dataclasses
from pathlib import Path
flake_args = ["--extra-experimental-features", "nix-command flakes"] flake_args = ["--extra-experimental-features","'nix-command flakes'"]
# hyperfine has its own variable substitution, so we use that and pass build="{BUILD}" here.
# perf doesn't have variable substitution, so we call these with build being the actual build directory.
cases = { cases = {
"search": lambda build: [ "search": lambda build: [f"{build}/bin/nix", *flake_args, "search", "--no-eval-cache", "github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870", "hello"],
f"{build}/bin/nix", "rebuild": lambda build: [f"{build}/bin/nix", *flake_args, "eval", "--raw", "--impure", "--expr", "'with import <nixpkgs/nixos> {}; system'"],
*flake_args, "rebuild_lh": lambda build: ["GC_INITIAL_HEAP_SIZE=10g", f"{build}/bin/nix", *flake_args, "eval", "--raw", "--impure", "--expr", "'with import <nixpkgs/nixos> {}; system'"],
"search", "parse": lambda build: [f"{build}/bin/nix", *flake_args, "eval", "-f", "bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix"],
"--no-eval-cache",
f"path:{Path('./bench/nixpkgs/').readlink()}",
"hello",
],
"rebuild": lambda build: [
f"{build}/bin/nix",
*flake_args,
"eval",
"--raw",
"--impure",
"--expr",
textwrap.dedent("""
(import <nixpkgs/nixos> {
configuration = ./bench/nixpkgs/nixos/modules/installer/cd-dvd/installation-cd-graphical-calamares-plasma6.nix;
}).config.system.build.toplevel
""").replace("\n", " "),
],
"rebuild_lh": lambda build: [
"GC_INITIAL_HEAP_SIZE=10g",
*cases['rebuild'](build),
],
"parse": lambda build: [
f"{build}/bin/nix",
*flake_args,
"eval",
"-f",
"bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix",
],
} }
arg_parser = argparse.ArgumentParser() arg_parser = argparse.ArgumentParser()
@@ -54,81 +24,46 @@ arg_parser = argparse.ArgumentParser()
# mode, we would have to combine the JSON ourselves to support that, which # mode, we would have to combine the JSON ourselves to support that, which
# would probably be better done by writing a benchmarking script in # would probably be better done by writing a benchmarking script in
# not-bash. # not-bash.
arg_parser.add_argument( arg_parser.add_argument('builds', nargs='+', help="At least two build directories to compare, containing bin/nix")
'builds', arg_parser.add_argument('--cases', type=str, help="A comma-separated list of cases you want to run. Defaults to running all")
nargs='+', available_modes = [ "walltime" ] + [ "icount" ] if platform.system() == 'Linux' else [] # perf doesn't run on Darwin
help="At least two build directories to compare, containing bin/nix", arg_parser.add_argument('--mode', choices=available_modes, default="walltime")
)
arg_parser.add_argument(
'--cases',
type=str,
help="A comma-separated list of cases you want to run. Defaults to running all",
)
arg_parser.add_argument(
'--mode',
nargs='+',
choices=[ "walltime", "memory" ] + [ "icount" ] if platform.system() == 'Linux' else [], # perf doesn't run on Darwin
default=[ "walltime" ],
)
arg_parser.add_argument(
'--daemon',
action='store_true',
help='Run a temporary daemon for the benchmark instead of using a local store directly',
)
args = arg_parser.parse_args() args = arg_parser.parse_args()
if len(args.builds) < 1: if len(args.builds) < 2:
raise ValueError("need at least one build directory to benchmark") raise ValueError("need at least two build directories to compare")
benchmarks: list[str] = [] benchmarks: list[str] = []
if args.cases is None: if args.cases is None:
benchmarks = list(cases.keys()) benchmarks = list(cases.keys())
else: else:
for case in args.cases.split(","): for case in args.cases.split(","):
if case not in cases: if case not in cases: raise ValueError(f"no such case: {case}")
raise ValueError(f"no such case: {case}")
benchmarks.append(case) benchmarks.append(case)
def make_full_command(build, case):
cmd = " ".join(map(shlex.quote, cases[case](build)))
if args.daemon:
return " ".join([
f"{build}/bin/nix --extra-experimental-features nix-command daemon &",
"trap 'kill %1' EXIT;",
f"NIX_REMOTE=daemon {cmd}",
])
else:
return cmd
def bench_walltime(env): def bench_walltime(env):
hyperfine_args = ["--parameter-list", "BUILD", ','.join(args.builds), "--warmup", "2", "--runs", "10"]
for case in benchmarks: for case in benchmarks:
for build in args.builds: case_command = cases[case]("{BUILD}") # see the comment on cases
subprocess.run([ subprocess.run([
"taskset", "-c", "2,3", "taskset", "-c", "2,3",
"chrt", "-f","50", "chrt", "-f","50",
*[ "hyperfine", *hyperfine_args, "--export-json", f"bench/bench-{case}.json", "--export-markdown", f"bench/bench-{case}.md", "--", " ".join(case_command)
"hyperfine", "--warmup", "2", "--runs", "10", ], env=env, check=True)
"--export-json", f"bench/bench-{case}-{build}.json",
"--export-markdown", f"bench/bench-{case}-{build}.md",
"--", make_full_command(build, case),
],
], env=env, check=True)
print("Benchmarks summary\n---\n") print("Benchmarks summary\n---\n")
for case in benchmarks: for case in benchmarks:
results = [] fd = open(f"bench/bench-{case}.json")
for build in args.builds: result_json = json.load(fd)
with open(f"bench/bench-{case}-{build}.json") as fd: fd.close()
results.append(json.load(fd)["results"][0]) for result in result_json["results"]:
for result in results:
print(result["command"]) print(result["command"])
print("-" * min(80,len(result["command"]))) print("-" * min(80,len(result["command"])))
def attr_rounded(attr): attr_rounded = lambda attr: f"{result[attr]:.3f}"
return f"{result[attr]:.3f}"
print(" mean: ", attr_rounded("mean"), "±", attr_rounded("stddev")) print(" mean: ", attr_rounded("mean"), "±", attr_rounded("stddev"))
print(" user:", attr_rounded("user"), "| system", attr_rounded("system")) print(" user:", attr_rounded("user"), "| system", attr_rounded("system"))
print(" median: ", attr_rounded("median")) print(" median: ", attr_rounded("median"))
print(" range: ", attr_rounded("min") + "s.." + attr_rounded("max")+"s") print(" range: ", attr_rounded("min") + "s.." + attr_rounded("max")+"s")
print(" relative:", f"{result["mean"]/results[0]["mean"]:.3f}") print(" relative:", f"{result["mean"]/result_json["results"][0]["mean"]:.3f}")
print("\n") print("\n")
@@ -136,14 +71,12 @@ def bench_icount(env):
perf_results_for: dict[str, list[tuple[str, float]]] = {} perf_results_for: dict[str, list[tuple[str, float]]] = {}
for case in benchmarks: for case in benchmarks:
for build in args.builds: for build in args.builds:
case_command = cases[case](build)
# the perf stat -j output (incorrectly) localizes numbers, which will trip up the json parser. # the perf stat -j output (incorrectly) localizes numbers, which will trip up the json parser.
env["LC_ALL"]="C" env["LC_ALL"]="C"
case_command = make_full_command(build, case)
commandline = [ commandline = [
"perf", "stat", "-o", f"bench/perf-{case}.json", "-j", "perf", "stat", "-o", f"bench/perf-{case}.json", "-j", "sh", "-c", " ".join(case_command)
"sh", "-c", case_command,
] ]
print("running", case_command)
subprocess.run(commandline, env=env, check=True, stdout=subprocess.DEVNULL) # warmup run subprocess.run(commandline, env=env, check=True, stdout=subprocess.DEVNULL) # warmup run
subprocess.run(commandline, env=env, check=True, stdout=subprocess.DEVNULL) subprocess.run(commandline, env=env, check=True, stdout=subprocess.DEVNULL)
perf_fd = open(f"bench/perf-{case}.json") perf_fd = open(f"bench/perf-{case}.json")
@@ -151,9 +84,8 @@ def bench_icount(env):
perf_fd.close() perf_fd.close()
instr = next(x for x in perf_data if x["event"] in ["instructions", "instructions:u"]) # an implementation of a find_first iterator instr = next(x for x in perf_data if x["event"] in ["instructions", "instructions:u"]) # an implementation of a find_first iterator
if case not in perf_results_for: if case not in perf_results_for: perf_results_for[case] = []
perf_results_for[case] = [] perf_results_for[case].append((" ".join(case_command), float(instr["counter-value"])))
perf_results_for[case].append((case_command, float(instr["counter-value"])))
print("Benchmarks summary\n---\n") print("Benchmarks summary\n---\n")
for (case, entries) in perf_results_for.items(): for (case, entries) in perf_results_for.items():
@@ -165,54 +97,6 @@ def bench_icount(env):
print(" relative instructions:", int(instr)/perf_results_for[case][0][1]) print(" relative instructions:", int(instr)/perf_results_for[case][0][1])
print("\n") print("\n")
@dataclasses.dataclass
class MemoryStatistics:
envBytes: int
listBytes: int
setBytes: int
valueBytes: int
heapBytes: int
heapSize: int
def bench_memory(env):
path = "bench/bench-memory.json"
env = env | {
'NIX_SHOW_STATS': '1',
'NIX_SHOW_STATS_PATH': path,
}
results: dict[str, list[tuple[str, MemoryStatistics]]] = {}
for case in benchmarks:
for build in args.builds:
case_command = make_full_command(build, case)
commandline = [ "sh", "-c", case_command ]
print("running", case_command)
subprocess.run(commandline, env=env, check=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
with open(path) as fd:
stats = json.load(fd)
results.setdefault(case, []).append((case_command, MemoryStatistics(
envBytes=stats['envs']['bytes'],
listBytes=stats['list']['bytes'],
setBytes=stats['sets']['bytes'],
valueBytes=stats['values']['bytes'],
heapSize=stats['gc']['heapSize'],
heapBytes=stats['gc']['totalBytes'],
)))
print("Benchmarks summary\n---\n")
for (case, entries) in results.items():
for cmd, stats in entries:
print(cmd)
print("-" * min(80, len(cmd)))
print(f" env bytes: {stats.envBytes :15d} | {(stats.envBytes / entries[0][1].envBytes) :.3f}x")
print(f" list bytes: {stats.listBytes :15d} | {(stats.listBytes / entries[0][1].listBytes) :.3f}x")
print(f" set bytes: {stats.setBytes :15d} | {(stats.setBytes / entries[0][1].setBytes) :.3f}x")
if not entries[0][1].valueBytes:
print(f" value bytes: {0:15d}")
else:
print(f" value bytes: {stats.valueBytes:15d} | {(stats.valueBytes / entries[0][1].valueBytes):.3f}x")
print(f" heap alloc'd: {stats.heapBytes :15d} | {(stats.heapBytes / entries[0][1].heapBytes) :.3f}x")
print(f" heap size: {stats.heapSize :15d} | {(stats.heapSize / entries[0][1].heapSize) :.3f}x")
print("\n")
with tempfile.TemporaryDirectory() as tmp_dir: with tempfile.TemporaryDirectory() as tmp_dir:
subprocess.run([ subprocess.run([
@@ -224,15 +108,9 @@ with tempfile.TemporaryDirectory() as tmp_dir:
subenv = os.environ.copy() subenv = os.environ.copy()
subenv["NIX_CONF_DIR"] = "/var/empty" subenv["NIX_CONF_DIR"] = "/var/empty"
subenv["NIX_REMOTE"] = tmp_dir subenv["NIX_REMOTE"] = tmp_dir
subenv["NIX_PATH"] = ":".join([ subenv["NIX_PATH"] = "nixpkgs=bench/nixpkgs:nixos-config=bench/configuration.nix"
"nixpkgs=bench/nixpkgs",
])
subenv["NIX_DAEMON_SOCKET_PATH"] = f"{tmp_dir}/daemon"
for mode in args.mode: if args.mode == "walltime":
if mode == "walltime": bench_walltime(subenv)
bench_walltime(subenv) else:
elif mode == "memory": bench_icount(subenv)
bench_memory(subenv)
else:
bench_icount(subenv)
bench/configuration.nix
+314
View File
@@ -0,0 +1,314 @@
{
config,
pkgs,
lib,
...
}:
{
boot = {
initrd = {
availableKernelModules = [
"xhci_pci"
"ahci"
];
kernelModules = [ "dm-snapshot" ];
luks.devices = {
croot = {
device = "/dev/sdb";
allowDiscards = true;
};
};
};
kernelModules = [ "kvm-intel" ];
kernelPackages = pkgs.linuxPackages_latest;
loader = {
systemd-boot.enable = true;
efi.canTouchEfiVariables = true;
};
};
hardware = {
enableRedistributableFirmware = true;
cpu.intel.updateMicrocode = true;
graphics.enable32Bit = true;
graphics.extraPackages = with pkgs; [
vaapiIntel
intel-media-driver
intel-compute-runtime
];
};
fileSystems = {
"/" = {
device = "/dev/sda2";
fsType = "xfs";
options = [ "noatime" ];
};
"/boot" = {
device = "/dev/sda1";
fsType = "vfat";
};
"/nas" = {
device = "nas:/";
fsType = "nfs4";
options = [
"ro"
"x-systemd.automount"
];
};
};
swapDevices = [ { device = "/dev/swap"; } ];
networking = {
useDHCP = false;
hostName = "host";
wireless = {
enable = true;
interfaces = [ "eth1" ];
};
interfaces = {
eth0.useDHCP = true;
eth1.useDHCP = true;
};
wg-quick.interfaces = {
wg0 = {
address = [ "2001:db8::1" ];
privateKeyFile = "/etc/secrets/wg0.key";
peers = [
{
publicKey = "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";
endpoint = "[2001:db8::2]:61021";
allowedIPs = [ "2001::db8:1::/64" ];
}
];
};
};
firewall.allowedUDPPorts = [ 4567 ];
};
i18n = {
defaultLocale = "en_US.UTF-8";
inputMethod.enable = true;
inputMethod.type = "ibus";
};
services = {
libinput.enable = true;
xserver = {
enable = true;
xkb.layout = "us";
xkb.variant = "altgr-intl";
xkb.options = "ctrl:nocaps";
wacom.enable = true;
videoDrivers = [ "modesetting" ];
modules = [ pkgs.xf86_input_wacom ];
displayManager.sx.enable = true;
windowManager.i3.enable = true;
};
udev.extraHwdb = ''
# not like this mattered at all
# we're not running udev from here
'';
udev.extraRules = ''
# ACTION=="add", SUBSYSTEM=="input", ...
'';
};
programs = {
light.enable = true;
wireshark = {
enable = true;
package = pkgs.wireshark-qt;
};
gnupg.agent = {
enable = true;
};
};
fonts.packages = with pkgs; [
font-awesome
noto-fonts
noto-fonts-cjk-sans
noto-fonts-emoji
noto-fonts-extra
dejavu_fonts
powerline-fonts
source-code-pro
cantarell-fonts
];
users = {
mutableUsers = false;
users = {
user = {
isNormalUser = true;
group = "user";
extraGroups = [
"wheel"
"video"
"audio"
"dialout"
"users"
"kvm"
"wireshark"
];
password = "unimportant";
};
};
groups = {
user = { };
};
};
security = {
pam.loginLimits = [
{
domain = "@audio";
item = "memlock";
type = "-";
value = "unlimited";
}
{
domain = "@audio";
item = "rtprio";
type = "-";
value = "99";
}
{
domain = "@audio";
item = "nofile";
type = "soft";
value = "99999";
}
{
domain = "@audio";
item = "nofile";
type = "hard";
value = "99999";
}
];
sudo.extraRules = [
{
users = [ "user" ];
commands = [
{
command = "${pkgs.linuxPackages.cpupower}/bin/cpupower";
options = [ "NOPASSWD" ];
}
];
}
];
};
environment.systemPackages = with pkgs; [
a2jmidid
age
ardour
bemenu
blender
breeze-icons
breeze-qt5
bubblewrap
calf
claws-mail
darktable
duperemove
emacs
feh
file
firefox
fluidsynth
adwaita-icon-theme
gnuplot
graphviz
helm
i3status-rust
inkscape
jack2
jq
krita
ldns
libqalculate
libreoffice
man-pages
nix-diff
nix-index
nix-output-monitor
open-music-kontrollers.patchmatrix
pamixer
pavucontrol
pciutils
picom
pwgen
redshift
ripgrep
rlwrap
silver-searcher
soundfont-fluid
whois
wol
xclip
xdot
xdotool
xorg.xkbcomp
yt-dlp
zathura
borgbackup
linuxPackages.cpupower
mtr
kitty
xf86_input_wacom
];
environment.pathsToLink = [ "/share/soundfonts" ];
systemd.user.services.run-python = {
after = [ "network-online.target" ];
script = ''
exec ${pkgs.python3}/bin/python
'';
serviceConfig = {
CapabilityBoundingSet = [ "" ];
KeyringMode = "private";
LockPersonality = true;
MemoryDenyWriteExecute = true;
NoNewPrivileges = true;
PrivateDevices = true;
PrivateTmp = true;
PrivateUsers = true;
ProcSubset = "pid";
ProtectClock = true;
ProtectControlGroups = true;
ProtectHome = true;
ProtectHostname = true;
ProtectKernelLogs = true;
ProtectKernelModules = true;
ProtectKernelTunables = true;
ProtectProc = "invisible";
ProtectSystem = "strict";
RestrictAddressFamilies = "AF_INET AF_INET6";
RestrictNamespaces = true;
RestrictRealtime = true;
RestrictSUIDSGID = true;
SystemCallArchitectures = "native";
SystemCallFilter = [
"@system-service"
"~ @resources @privileged"
];
UMask = "077";
};
};
system.stateVersion = "23.11";
}
contrib/OWNERS
-1
View File
@@ -1 +0,0 @@
*
contrib/plugins/meson.build
-13
View File
@@ -1,13 +0,0 @@
plugin_mtls_store = shared_module(
'plugin_mtls_store',
'plugin_mtls_store.cc',
# don't link liblix* into plugins (host process provides them at runtime).
# Explicitly link curl so it binds to Nix-store libcurl, not /usr/lib/libcurl.
dependencies : [
liblix.partial_dependency(includes : true, compile_args : true),
curl,
],
install : false,
build_by_default : true,
link_args : plugin_link_args,
)
contrib/plugins/mtls-http-binary-cache-store.md
-14
View File
@@ -1,14 +0,0 @@
R"(
**Store URL format**: `https+mtls://...`
This store allows a binary cache to be accessed via HTTPS with mutual TLS (client certificate authentication).
Both parameters are required:
- `tls-certificate`, a path to the TLS client certificate
- `tls-private-key`, a path to the TLS private key backing the client certificate
If you don't need mTLS, use `https://` instead.
)"
contrib/plugins/plugin_mtls_store.cc
-102
View File
@@ -1,102 +0,0 @@
#include "lix/libstore/store-api.hh"
#include "lix/libutil/config.hh"
#include "lix/libstore/http-binary-cache-store.hh"
#include <stdlib.h>
#include <curl/curl.h>
namespace nix {
struct mTLSBinaryCacheStoreConfig : HttpBinaryCacheStoreConfig
{
using HttpBinaryCacheStoreConfig::HttpBinaryCacheStoreConfig;
const std::string name() override
{
return "mTLS HTTP Binary Cache Store";
}
std::string doc() override
{
return
#include "mtls-http-binary-cache-store.md"
;
}
PathsSetting<nix::Path> tlsCertificate{
this,
"",
"tls-certificate",
"Path of the TLS client certificate in PEM format as expected by CURLOPT_SSLCERT"
};
PathsSetting<nix::Path> tlsKey{
this,
"",
"tls-private-key",
"Path of the TLS client certificate private key in PEM format as expected by CURLOPT_SSLKEY"
};
};
struct mTLSBinaryCacheStoreImpl : public HttpBinaryCacheStore
{
struct Keyring
{
nix::Path tlsCertificate;
nix::Path tlsKey;
};
mTLSBinaryCacheStoreConfig config_;
std::shared_ptr<Keyring> keyring;
mTLSBinaryCacheStoreConfig & config() override
{
return config_;
}
const mTLSBinaryCacheStoreConfig & config() const override
{
return config_;
}
mTLSBinaryCacheStoreImpl(
const std::string & uriScheme, const Path & _cacheUri, mTLSBinaryCacheStoreConfig config
)
: Store(config)
, HttpBinaryCacheStore("https", _cacheUri, config)
, config_(std::move(config))
, keyring(std::make_shared<Keyring>(config_.tlsCertificate.get(), config_.tlsKey.get()))
{
}
FileTransferOptions makeOptions(Headers && headers = {}) override
{
auto options = HttpBinaryCacheStore::makeOptions(std::move(headers));
auto baseExtraSetup = std::move(options.extraSetup);
auto keyring = this->keyring;
options.extraSetup = [keyring, baseExtraSetup{std::move(baseExtraSetup)}](CURL * req) {
if (baseExtraSetup) {
baseExtraSetup(req);
}
const bool haveCert = !keyring->tlsCertificate.empty();
const bool haveKey = !keyring->tlsKey.empty();
if (!(haveCert && haveKey)) {
throw Error("https+mtls requires both tls-certificate and tls-private-key");
}
curl_easy_setopt(req, CURLOPT_SSLCERT, keyring->tlsCertificate.c_str());
curl_easy_setopt(req, CURLOPT_SSLKEY, keyring->tlsKey.c_str());
};
return options;
}
static std::set<std::string> uriSchemes()
{
return {"https+mtls"};
}
};
}
extern "C" void nix_plugin_entry()
{
nix::StoreImplementations::add<nix::mTLSBinaryCacheStoreImpl, nix::mTLSBinaryCacheStoreConfig>();
}
default.nix
+9 -15
View File
@@ -1,15 +1,9 @@
let (import (
lockFile = builtins.fromJSON (builtins.readFile ./flake.lock); let
flake-compat-node = lockFile.nodes.${lockFile.nodes.root.inputs.flake-compat}; lock = builtins.fromJSON (builtins.readFile ./flake.lock);
flake-compat = builtins.fetchTarball { in
inherit (flake-compat-node.locked) url; fetchTarball {
sha256 = flake-compat-node.locked.narHash; url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
}; sha256 = lock.nodes.flake-compat.locked.narHash;
}
flake = ( ) { src = ./.; }).defaultNix
import flake-compat {
src = ./.;
}
);
in
flake.defaultNix
doc/OWNERS
-1
View File
@@ -1 +0,0 @@
*
doc/manual/anchors.jq
+3 -2
View File
@@ -24,7 +24,8 @@ def map_contents_recursively(transformer):
def process_command: def process_command:
.[0] as $context | .[0] as $context |
.[1] as $body | .[1] as $body |
$body | .items |= map(map_contents_recursively(if $context.renderer == "html" then transform_anchors_html else transform_anchors_strip end)) $body + {
; sections: $body.sections | map(map_contents_recursively(if $context.renderer == "html" then transform_anchors_html else transform_anchors_strip end)),
};
process_command process_command
doc/manual/book.toml
+3 -3
View File
@@ -22,16 +22,16 @@ fold.level = 30
# not want to disable the links preprocessor entirely though because that requires # not want to disable the links preprocessor entirely though because that requires
# disabling *all* built-in preprocessors and selectively reenabling those we want. # disabling *all* built-in preprocessors and selectively reenabling those we want.
[preprocessor.substitute] [preprocessor.substitute]
command = "python3 substitute.py" command = "python3 doc/manual/substitute.py"
before = ["anchors", "links"] before = ["anchors", "links"]
[preprocessor.anchors] [preprocessor.anchors]
renderers = ["html"] renderers = ["html"]
command = "jq --from-file anchors.jq" command = "jq --from-file doc/manual/anchors.jq"
[output.markdown] [output.markdown]
[output.linkcheck2] [output.linkcheck]
# no Internet during the build (in the sandbox) # no Internet during the build (in the sandbox)
follow-web-links = false follow-web-links = false
doc/manual/change-authors.yml
-94
View File
@@ -48,11 +48,6 @@ artemist:
display_name: Artemis Tosini display_name: Artemis Tosini
forgejo: artemist forgejo: artemist
astreaprtcl:
display_name: Astreaprtcl
forgejo: astreaprtcl
github: astreaprtcl
bb010g: bb010g:
display_name: Dusk Banks display_name: Dusk Banks
forgejo: bb010g forgejo: bb010g
@@ -62,10 +57,6 @@ blitz:
display_name: Julian Stecklina display_name: Julian Stecklina
github: blitz github: blitz
blokyk:
display_name: blokyk
github: blokyk
cole-h: cole-h:
display_name: Cole Helbling display_name: Cole Helbling
github: cole-h github: cole-h
@@ -75,9 +66,6 @@ delan:
forgejo: delan forgejo: delan
github: delan github: delan
delroth:
github: delroth
detroyejr: detroyejr:
display_name: Jonathan De Troye display_name: Jonathan De Troye
github: detroyejr github: detroyejr
@@ -89,20 +77,10 @@ edolstra:
display_name: Eelco Dolstra display_name: Eelco Dolstra
github: edolstra github: edolstra
emilazy:
display_name: Emily
forgejo: emilazy
github: emilazy
ericson: ericson:
display_name: John Ericson display_name: John Ericson
github: ericson2314 github: ericson2314
getchoo:
display_name: Seth Flynn
forgejo: getchoo
github: getchoo
gilice: gilice:
forgejo: gilice forgejo: gilice
@@ -111,9 +89,6 @@ goldstein:
forgejo: goldstein forgejo: goldstein
github: GoldsteinE github: GoldsteinE
gustavderdrache:
github: gustavderdrache
horrors: horrors:
display_name: eldritch horrors display_name: eldritch horrors
forgejo: pennae forgejo: pennae
@@ -126,9 +101,6 @@ ian-h-chamberlain:
forgejo: ian-h-chamberlain forgejo: ian-h-chamberlain
github: ian-h-chamberlain github: ian-h-chamberlain
infinisil:
github: infinisil
isabelroses: isabelroses:
forgejo: isabelroses forgejo: isabelroses
github: isabelroses github: isabelroses
@@ -140,19 +112,6 @@ jade:
just1602: just1602:
forgejo: just1602 forgejo: just1602
k900:
display_name: K900
forgejo: K900
github: K900
kasimeka:
display_name: ورد
forgejo: janw4ld
github: kasimeka
keysmashes:
github: keysmashes
kfears: kfears:
display_name: KFears display_name: KFears
forgejo: kfearsoff forgejo: kfearsoff
@@ -191,33 +150,14 @@ ma27:
matthewbauer: matthewbauer:
github: matthewbauer github: matthewbauer
mic92:
github: mic92
midnightveil: midnightveil:
display_name: julia display_name: julia
forgejo: midnightveil forgejo: midnightveil
github: midnightveil github: midnightveil
milibopp:
display_name: Emilia Bopp
forgejo: milibopp
github: milibopp
nan-git:
display_name: NaN-git
github: NaN-git
ncfavier: ncfavier:
github: ncfavier github: ncfavier
nkk0:
github: nkk0
not-my-profile:
display_name: Martin Fischer
github: not-my-profile
p-e-meunier: p-e-meunier:
display_name: Pierre-Etienne Meunier display_name: Pierre-Etienne Meunier
github: P-E-Meunier github: P-E-Meunier
@@ -256,32 +196,16 @@ raito:
forgejo: raito forgejo: raito
github: RaitoBezarius github: RaitoBezarius
rkjnsn:
display_name: Erik Jensen
forgejo: rkjnsn
github: rkjnsn
roberth: roberth:
display_name: Robert Hensing display_name: Robert Hensing
github: roberth github: roberth
rootile:
display_name: rootile (Rutile)
forgejo: rootile
sandydoo: sandydoo:
github: sandydoo github: sandydoo
seppel3210: seppel3210:
github: Seppel3210 github: Seppel3210
sterni:
forgejo: sterni
github: sternenseemann
stevalkr:
github: stevalkr
teofilc: teofilc:
forgejo: teofilc forgejo: teofilc
github: TeofilC github: TeofilC
@@ -308,14 +232,6 @@ vigress8:
forgejo: vigress8 forgejo: vigress8
github: vigress8 github: vigress8
vlaci:
github: vlaci
vlinkz:
display_name: Victor Fuentes
forgejo: vlinkz
github: vlinkz
winter: winter:
forgejo: winter forgejo: winter
github: winterqt github: winterqt
@@ -323,21 +239,11 @@ winter:
xanderio: xanderio:
github: xanderio github: xanderio
xokdvium:
github: xokdvium
xyenon:
forgejo: xyenon
github: xyenon
yorickvp: yorickvp:
github: yorickvp github: yorickvp
yshui: yshui:
github: yshui github: yshui
ysndr:
github: ysndr
zimbatm: zimbatm:
github: zimbatm github: zimbatm
doc/manual/generate-manpage.nix
+2 -2
View File
@@ -69,7 +69,7 @@ let
let let
result = squash '' result = squash ''
- ${ - ${
if inlineHTML then ''<span id="conf-${name}">[`${name}`](#conf-${name})</span>'' else "`${name}`" if inlineHTML then ''<span id="conf-${name}">[`${name}`](#conf-${name})</span>'' else ''`${name}`''
} }
${indent " " body} ${indent " " body}
@@ -225,7 +225,7 @@ let
showCategory = cat: '' showCategory = cat: ''
${optionalString (cat != "") "**${cat}:**"} ${optionalString (cat != "") "**${cat}:**"}
${listOptions (filterAttrs (n: v: v.category == cat && !v.hidden) allOptions)} ${listOptions (filterAttrs (n: v: v.category == cat) allOptions)}
''; '';
listOptions = opts: concatStringsSep "\n" (attrValues (mapAttrs showOption opts)); listOptions = opts: concatStringsSep "\n" (attrValues (mapAttrs showOption opts));
showOption = showOption =
doc/manual/meson.build
+4 -9
View File
@@ -41,13 +41,9 @@ manual = custom_target(
'-euo', 'pipefail', '-euo', 'pipefail',
'-c', '-c',
''' '''
@0@ @INPUT0@ @3@ > @DEPFILE@ @0@ @INPUT0@ @CURRENT_SOURCE_DIR@ > @DEPFILE@
cd @SOURCE_ROOT@
# Needs to be in lix/doc/manual for e.g. substitute.py @1@ build doc/manual -d @2@ | { grep -Fv "because fragment resolution isn't implemented" || :; }
pushd @3@
@1@ build . -d @2@
popd
rm -rf @2@/manual rm -rf @2@/manual
mv @2@/html @2@/manual mv @2@/html @2@/manual
find @2@/manual -iname meson.build -delete find @2@/manual -iname meson.build -delete
@@ -55,7 +51,6 @@ manual = custom_target(
python.full_path(), python.full_path(),
mdbook.full_path(), mdbook.full_path(),
meson.current_build_dir(), meson.current_build_dir(),
meson.current_source_dir()
), ),
], ],
input : [ input : [
@@ -86,7 +81,7 @@ manual = custom_target(
depfile : 'manual.d', depfile : 'manual.d',
env : { env : {
'RUST_LOG': 'info', 'RUST_LOG': 'info',
'MANUAL_SUBSTITUTE_SEARCH': meson.current_build_dir() / 'src', 'MDBOOK_SUBSTITUTE_SEARCH': meson.current_build_dir() / 'src',
}, },
) )
manual_md = manual[1] manual_md = manual[1]
doc/manual/rl-next/arg-behavior.md
-10
View File
@@ -1,10 +0,0 @@
---
synopsis: "allow setting nested attributes via `--arg`/`--argstr`"
cls: [5338]
category: "Features"
credits: [ma27]
issues: [fj#496]
---
Passing `--arg config.allowUnfree true` to e.g. `nix-build` now results in `config` with value
`{ allowUnfree = true; }` passed to the expression.
doc/manual/rl-next/check-netrc-or-ca-file.md
-9
View File
@@ -1,9 +0,0 @@
---
synopsis: "check for missing ca-file or netrc-file if one is specified"
cls: [5646]
category: "Improvements"
credits: [astreaprtcl]
issues: [fj#1106]
---
If the settings `ssl-cert-file` or `netrc-file` have been set by the user, check if those files actually exist and fail if they are missing.
doc/manual/rl-next/empty-attr-paths.md
-10
View File
@@ -1,10 +0,0 @@
---
synopsis: "libexpr: allow empty attr-names in parseAttrPath if they are quoted"
cls: [5375]
category: "Miscellany"
credits: [ma27]
---
Empty strings are now allowed in attribute paths as consumed by e.g. `nix-build`.
I.e. `nix-build -A 'foo."".bar'` works now.
The quotes are necessary, i.e. `nix-build -A foo..bar` will throw an error.
doc/manual/rl-next/fetch-empty-hash.md
-10
View File
@@ -1,10 +0,0 @@
---
synopsis: "don't treat tarball fetches with empty or zero hash as locked"
cls: []
category: "Fixes"
credits: [horrors]
issues: [fj#1233]
---
Lix no longer treats tarball fetches with empty or zero hashes as locked.
All such fetches are now also affected by `tarball-ttl` as a consequence.
doc/manual/rl-next/floor-ceil-corrupt-integers.md
-13
View File
@@ -1,13 +0,0 @@
---
synopsis: builtins.floor/builtins.ceil handle out-of-range inputs correctly
issues: [nix#12899]
cls: [3923]
prs: [nix#13013]
category: "Breaking Changes"
credits: [jade, nan-git, rootile]
---
Previously, `builtins.floor` and `builtins.ceil` always cast the input into a floating point value before running the operation and casting the floating point result back into an integer.
No checks were made for precision loss in either coercing integer inputs or converting the output to an integer (and in fact in the latter case, invoked undefined behaviour).
Now, Lix checks for precision loss on integer input (to avoid a silent eval semantics change if we were to simply pass it through as-is) and on integer output.
If your code fails to evaluate after this change, use `--extra-deprecated-features floor-ceil-corrupt-integers`.
doc/manual/rl-next/libexpr-fix-break.md
-14
View File
@@ -1,14 +0,0 @@
---
synopsis: "builtins.break doesn't break expression anymore"
issues: [1165]
cls: [5422]
category: "Fixes"
credits: [blokyk]
---
Wrapping an expression in `builtins.break` used to break some builtins like
`map` and the `is*` functions, which could modify the execution path of code
inadvertently, made debugging nix harder than it already is, and in some cases
even crashed the interpreter. Now, using `break` should be completely
transparent to whatever function receives it as an input, preventing the
above-mentioned issues.
doc/manual/rl-next/libexpr-fix-stdout.md
-9
View File
@@ -1,9 +0,0 @@
---
synopsis: "flake config warnings are now printed to stderr"
issues: [1155]
cls: [5379]
category: "Fixes"
credits: [lheckemann]
---
The settings listed in a flake-config confirmation prompt are now printed to stderr rather than stdout, which allows `nix print-dev-env` to emit valid bash again even in the presence of untrusted settings.
doc/manual/rl-next/libfetchers-lock-cache-input.md
-21
View File
@@ -1,21 +0,0 @@
---
synopsis: "Use a lock when fetching inputs"
issues: [1122]
cls: [5438]
category: "Fixes"
credits: [lheckemann]
---
Up to now, attempting to fetch the same git input from multiple processes
concurrently when the input is not yet cached presented multiple issues:
- If the input was not already present, it would unnecessarily be fetched
multiple times;
- Access to the fetcher cache database was contentious, and could lead to
evaluation or flake locking failing unnecessary because the fetcher cache
was locked.
We now acquire a lock on a path based on a hash of the input specification
before accessing the fetcher db, reducing contention significantly, and
preventing more than one process from fetching the same path at the same time.
doc/manual/rl-next/mimalloc.md
-16
View File
@@ -1,16 +0,0 @@
---
synopsis: "Use mimalloc for faster evaluation"
cls: [5645]
category: Features
credits: [getchoo, lovesegfault]
---
Lix now links with [mimalloc](https://github.com/microsoft/mimalloc),
replacing the system's default `malloc()` for all non-GC allocations.
This yields a **5–12% wall-clock improvement** on evaluation workloads,
ranging from `nix-instantiate hello` to `nix-env -qa` and full NixOS
configurations.
The allocator can be disabled at build time with `-Dmimalloc=disabled`,
or by passing the `useMimalloc = false` override to the `lix` package.
doc/manual/rl-next/minimum-lowdown-1.4.md
-10
View File
@@ -1,10 +0,0 @@
---
synopsis: "Lix now requires lowdown 1.4.0 or later"
issues: []
cls: [5374]
category: Packaging
credits: [sterni]
---
Support for linking against `lowdown < 1.4.0` has been removed from Lix since
all supported Nixpkgs channels distribute lowdown 2.0.4 or later.
doc/manual/rl-next/nej-apply-flag.md
-12
View File
@@ -1,12 +0,0 @@
---
synopsis: "nix-eval-jobs support `--apply` flag"
cls: [5748]
category: "Features"
credits: [isabelroses,mic92,ysndr]
issues: [fj#1214]
---
`nix-eval-jobs` now supports the `--apply` flag. With this you can apply the
provided function to the each derivation, the result of this function will then
be serialized as a JSON value and stored inside `"extraValue"` key of the json
line output.
doc/manual/rl-next/nix-copy-closure-include-outputs.md
-10
View File
@@ -1,10 +0,0 @@
---
synopsis: "Fix `nix-copy-closure --include-outputs`"
issues: [gh#5105]
cls: [5588]
category: "Fixes"
credits: [rkjnsn]
---
The `--include-outputs` flag for `nix-copy-closure` now works as intended.
Previously, the option was accepted but silently ignored.
doc/manual/rl-next/nix-doctor.md
-14
View File
@@ -1,14 +0,0 @@
---
synopsis: "Improve nix doctor"
cls: [5316, 5317, 5318, 5319, 5320, 5768, 5829]
category: Features
credits: [rootile, raito]
---
The `nix doctor` diagnosics interface now provides a lot more useful information including, but not limited to:
- General system information (OS, Hardware etc)
- Nix Information like Sandbox, Version, Store, State and other directories
- Flake registry
- Search path Information
- Nixpkgs provenance
- Remote builder configuration (including remote connection)
- fix crash when having relative Paths in PATH
doc/manual/rl-next/nix-path-shadow-error.md
-11
View File
@@ -1,11 +0,0 @@
---
synopsis: "Shadowing internal files through the Nix search path is now an error"
issues: [998]
cls: [4632, 5370]
category: "Breaking Changes"
credits: [thubrecht, jade, horrors]
---
As Lix uses the path `<nix/fetchurl.nix>` for bootstrapping purposes, the ability to shadow it by adding `nix=/some/path` (or `/other/path` that contains a `nix` directory) to the search path is not desirable.
Lix 2.95 deprecated this behavior with a warning, Lix 2.96 now turns it into a hard error if the `nix-path-shadow` deprecated feature isn't enabled. This deprecated feature is slated to be removed in Lix 2.98.
doc/manual/rl-next/no-max-connections.md
-11
View File
@@ -1,11 +0,0 @@
---
synopsis: "Remove `max-connections` store parameters for `ssh://` and `ssh-ng://` stores"
cls: []
category: Miscellany
credits: [horrors]
---
The `max-connections` parameter was undocumented, untested, and (in the case of `ssh`) even ignored
entirely for remote builds. During a survey of public nixos configurations we have found *two* uses
of `max-connections` for `ssh-ng`, and none at all for `ssh`. Since it is so rarely used but brings
significant internal complexity that hinders improvements we have decided to remove these features.
doc/manual/rl-next/repl-relative-st.md
-18
View File
@@ -1,18 +0,0 @@
---
synopsis: "Allow moving between stack frames relative to current debugger frame"
issues: [1156]
cls: [5411]
category: "Improvements"
credits: [blokyk]
---
Debugging functional programs often involve switching between a bunch of stack
frames to get the full context of what's happening and who's calling who.
Before this change, going up or down the stack in the nix debugger with `:st`
meant remembering the absolute index of each stack frame, instead of their
positions relative to one another; this got tiring *fast*.
Now, you can prepend `:st`'s argument with a + or - sign to indicate you want to
move relative to the current stack frame. For example, typing `:st +3` when you
were on frame `10` will go frame `13`; vice-versa, typing `:st -4` on frame `6`
will go to frame `2`.
doc/manual/rl-next/repl-reverse-bt.md
-17
View File
@@ -1,17 +0,0 @@
---
synopsis: "Print REPL backtraces in more convenient order"
issues: []
cls: [5491]
category: "Improvements"
credits: [blokyk]
---
When using the debugger, stack traces printed with the `:bt` command were
previously printed in reverse order compared to most other situations where they
appeared: the current stack frame would be printed at the very top, with the
most outer frame at the bottom, meaning that you'd have to scroll up to get a
sense of where you are.
With this change, the stack frames are printed such that the most relevant ones
are immediatly visible at the bottom, just like other traces in lix (e.g.
ones caused by errors).
doc/manual/rl-next/repl-st-errors.md
-14
View File
@@ -1,14 +0,0 @@
---
synopsis: "invalid arguments to :st now print an error"
cls: [5386]
category: "Improvements"
credits: [blokyk]
---
When using the debugger, the `:st` command used to traverse the call stack would
silently fail and put the debugger in an invalid state if the argument given to
it wasn't a valid stack frame index.
This change adds an error message warning the user if the given index wasn't a
valid frame (telling them the range of valid indices), as well as if it wasn't
even a valid integer to begin with.
doc/manual/rl-next/rustyline.md
-12
View File
@@ -1,12 +0,0 @@
---
synopsis: "REPL now uses rustyline"
cls: [5703]
category: "Improvements"
credits: [horrors]
issues: []
---
The REPL now uses [rustyline](https://github.com/kkawakam/rustyline) for input processing instead
of editline. This comes with some improvements to REPL behavior: wrapping lines no longer confuse
the line editor, unicode is fully supported, pasting multiline expressions is noew possible, even
undo commands are now available! We plan to improve the REPL further using these newfound powers.
doc/manual/rl-next/structuredattrs-url-guesswork.md
-11
View File
@@ -1,11 +0,0 @@
---
synopsis: "Hash mismatch diagnostics now work with `structuredAttrs`"
issues: [fj#1175]
cls: [5441]
category: Fixes
credits: [keysmashes]
---
Nixpkgs fetchers like `fetchurl` now use `structuredAttrs`, which broke the
hash mismatch diagnostics added in Lix 2.91. This has been fixed and the likely
URL is now shown again.
doc/manual/rl-next/trivial-flakes.md
-18
View File
@@ -1,18 +0,0 @@
---
synopsis: "Changes to `flake.nix` validation"
cls: [5523]
category: "Breaking Changes"
credits: [piegames, Qyriad, horrors]
issues: [gh#4945]
---
Flakes try to keep their inputs and metadata "simple", to make sure no unbounded computation may happen when calling e.g. `nix flake show`.
Those checks were haphazard, a maintenance burden, and also easily circumventable.
Lix has now replaced all the old checks by a simple rule: **No function calls outside of `outputs`.**
This is easier to reason about than the previous set of inconsistent rules, and crucially now also allows syntax features that users felt like they *should* have worked in the past, like let bindings.
However, some warts still remain for now: Some syntax constructs like `-1` internally desugar to `__sub 0 1`, which is a function call and thus remains forbidden.
This will be rectified as soon as the deprecation period of the respective anti-features has been completed.
This change is **breaking** in the sense that flakes which are written with the newly allowed language features will not evaluate with an older Lix version which still uses the old, more restrictive checks.
Crucially, this also affects **all transitive dependants** of such Flakes.
doc/manual/rl-next/unsigned-overflow-vuln.md
-28
View File
@@ -1,28 +0,0 @@
---
synopsis: "Fix unsigned overflow leading to out-of-band write in the NAR parser"
cls: [5554]
category: "Fixes"
credits: [horrors, raito, edef, sandydoo]
issues: []
---
The NAR parser contained an unsigned integer overflow that could be used by an
attacker to write arbitrary data to an unknown memory location and possibly
achieve code execution. A successful attack on the system-wide Lix daemon
could lead to privilege escalation to root. Any process that involves NAR
serialization could trigger this issue, including (but not limited to)
- local user interaction, whether the users are trusted or untrusted
- malicious substituters sending malformed NARs
- remote builders sending malformed build results
- remote daemons sending malformed inputs when requesting remote builds
Successful attacks using this bug require ASLR weakening of some sort, whether
by architecture constraints (e.g. on 32 bit systems, where little randomization
is possible) or system configuration (e.g. low ASLR entropy when loading
libraries), and millions of attempts. Local attacks can be mounted in less than
an hour. Remote builds typically require a fresh SSH connection for each build
and are thus less susceptible. Only one attempt can be made by substituters for
every build using substituters, they are thus not a likely vector for attacks.
At the time of writing, MITRE has not assigned this a CVE yet.
doc/manual/rl-next/user-traces.md
-23
View File
@@ -1,23 +0,0 @@
---
synopsis: "Always print frames from `addErrorContext` in error traces"
cls: [5847]
category: "Improvements"
credits: [blokyk]
issues: []
---
The [`builtins.addErrorContext`](@docroot@/language/builtins.md#builtins-addErrorContext)
function allows an author to add artificial stack frames with custom messages to
help end-users understand the context of an error and the path the code took to
get there, without having to read and understand the original source code. A
particularly notable user of this is the Nixpkgs module system, which adds
custom frames detailing what option it's evaluating or which definition it's
looking at.
However, previously, these frames would end up treated just as any other,
meaning they would most often not be visible without `--show-trace`; yet, using
`--show-trace`, they would be drowned out in the noise of the hundreds of other
frames, rendering them just as unusable.
With this change, these frames are now unconditionally shown, even without
`--show-trace`, which makes basic error traces much more informative.
doc/manual/src/SUMMARY.md
+1 -5
View File
@@ -20,6 +20,7 @@
- [Basic Package Management](package-management/basic-package-mgmt.md) - [Basic Package Management](package-management/basic-package-mgmt.md)
- [Profiles](package-management/profiles.md) - [Profiles](package-management/profiles.md)
- [Garbage Collection](package-management/garbage-collection.md) - [Garbage Collection](package-management/garbage-collection.md)
- [Garbage Collector Roots](package-management/garbage-collector-roots.md)
- [Sharing Packages Between Machines](package-management/sharing-packages.md) - [Sharing Packages Between Machines](package-management/sharing-packages.md)
- [Serving a Nix store via HTTP](package-management/binary-cache-substituter.md) - [Serving a Nix store via HTTP](package-management/binary-cache-substituter.md)
- [Copying Closures via SSH](package-management/copy-closure.md) - [Copying Closures via SSH](package-management/copy-closure.md)
@@ -39,9 +40,6 @@
- [Tuning Cores and Jobs](advanced-topics/cores-vs-jobs.md) - [Tuning Cores and Jobs](advanced-topics/cores-vs-jobs.md)
- [Verifying Build Reproducibility](advanced-topics/diff-hook.md) - [Verifying Build Reproducibility](advanced-topics/diff-hook.md)
- [Using the `post-build-hook`](advanced-topics/post-build-hook.md) - [Using the `post-build-hook`](advanced-topics/post-build-hook.md)
- [Pasta](advanced-topics/pasta.md)
- [Known Issues](known-issues/known-issues.md)
- [Limitations around non-isolated builds](known-issues/non-isolated-build-limits.md)
- [Command Reference](command-ref/command-ref.md) - [Command Reference](command-ref/command-ref.md)
- [Common Options](command-ref/opt-common.md) - [Common Options](command-ref/opt-common.md)
- [Common Environment Variables](command-ref/env-common.md) - [Common Environment Variables](command-ref/env-common.md)
@@ -200,8 +198,6 @@
- [Release Notes](release-notes/release-notes.md) - [Release Notes](release-notes/release-notes.md)
- [Upcoming release](release-notes/rl-next.md) - [Upcoming release](release-notes/rl-next.md)
<!-- RELENG-AUTO-INSERTION-MARKER (see releng/release_notes.py) --> <!-- RELENG-AUTO-INSERTION-MARKER (see releng/release_notes.py) -->
- [Lix 2.95 (2026-03-13)](release-notes/rl-2.95.md)
- [Lix 2.94 (2025-11-17)](release-notes/rl-2.94.md)
- [Lix 2.93 (2025-05-09)](release-notes/rl-2.93.md) - [Lix 2.93 (2025-05-09)](release-notes/rl-2.93.md)
- [Lix 2.92 (2025-01-18)](release-notes/rl-2.92.md) - [Lix 2.92 (2025-01-18)](release-notes/rl-2.92.md)
- [Lix 2.91 (2024-08-12)](release-notes/rl-2.91.md) - [Lix 2.91 (2024-08-12)](release-notes/rl-2.91.md)
doc/manual/src/advanced-topics/distributed-builds.md
+96 -184
View File
@@ -41,17 +41,106 @@ contains Nix.
> If you are building via the Lix daemon (default on Linux and macOS), it is the Lix daemon user account (that is, `root`) that should have SSH access to a user (not necessarily `root`) on the remote machine. > If you are building via the Lix daemon (default on Linux and macOS), it is the Lix daemon user account (that is, `root`) that should have SSH access to a user (not necessarily `root`) on the remote machine.
> >
> Furthermore, `root` needs to have the public host keys for the remote system in its `.ssh/known_hosts`. > Furthermore, `root` needs to have the public host keys for the remote system in its `.ssh/known_hosts`.
> To add them to `known_hosts` for root, do `ssh-keyscan HOST | sudo tee -a ~root/.ssh/known_hosts`. > To add them to `known_hosts` for root, do `ssh-keyscan USER@HOST | sudo tee -a ~root/.ssh/known_hosts`.
> >
> If you can’t or don’t want to configure `root` to be able to access the remote machine, you can use a private Nix store instead by passing e.g. `--store ~/my-nix` when running a Nix command from the local machine. > If you can’t or don’t want to configure `root` to be able to access the remote machine, you can use a private Nix store instead by passing e.g. `--store ~/my-nix` when running a Nix command from the local machine.
## Configuration
The list of remote machines can be specified on the command line or in The list of remote machines can be specified on the command line or in
the Lix configuration file. The former is convenient for testing. the Lix configuration file. The former is convenient for testing. For
Additionally, there are two supported formats to configure remote builders: example, the following command allows you to build a derivation for
The legacy, "space"-separated format and starting with Lix 2.95.0, a TOML. `x86_64-darwin` on a Linux machine:
```console
$ uname
Linux
$ nix build --impure \
--expr '(with import <nixpkgs> { system = "x86_64-darwin"; }; runCommand "foo" {} "uname > $out")' \
--builders 'ssh://mac x86_64-darwin'
[1/0/1 built, 0.0 MiB DL] building foo on ssh://mac
$ cat ./result
Darwin
```
It is possible to specify multiple builders separated by a semicolon or
a newline, e.g.
```console
--builders 'ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd'
```
Each machine specification consists of the following elements, separated
by spaces. Only the first element is required. To leave a field at its
default, set it to `-`.
1. The URI of the remote store in the format
`ssh://[username@]hostname[?port=<port>]`, e.g. `ssh://nix@mac` or `ssh://mac`.
If the ssh server is not listening on port 22 (e.g. port 1337 in this case)
the URI would be `ssh://nix@mac?port=1337`
For backward compatibility, `ssh://` may be omitted. The hostname
may be an alias defined in your `~/.ssh/config`.
2. A comma-separated list of Nix platform type identifiers, such as
`x86_64-darwin`. It is possible for a machine to support multiple
platform types, e.g., `i686-linux,x86_64-linux`. If omitted, this
defaults to the local platform type.
3. The SSH identity file to be used to log in to the remote machine. If
omitted, SSH will use its regular identities.
4. The maximum number of builds that Lix will execute in parallel on
the machine. Typically this should be equal to the number of CPU
cores. For instance, the machine `itchy` in the example will execute
up to 8 builds in parallel.
5. The “speed factor”, indicating the relative speed of the machine. If
there are multiple machines of the right type, Lix will prefer the
fastest, taking load into account.
6. A comma-separated list of *supported features*. If a derivation has
the `requiredSystemFeatures` attribute, then Lix will only perform
the derivation on a machine that has the specified features. For
instance, the attribute
```nix
requiredSystemFeatures = [ "kvm" ];
```
will cause the build to be performed on a machine that has the `kvm`
feature.
7. A comma-separated list of *mandatory features*. A machine will only
be used to build a derivation if all of the machine’s mandatory
features appear in the derivation’s `requiredSystemFeatures`
attribute.
8. The (base64-encoded) public host key of the remote machine. If omitted, SSH
will use its regular known-hosts file. Specifically, the field is calculated
via `base64 -w0 /etc/ssh/ssh_host_ed25519_key.pub`.
For example, the machine specification
nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm
nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2
nix@poochie.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 1 2 kvm benchmark
specifies several machines that can perform `i686-linux` builds.
However, `poochie` will only do builds that have the attribute
```nix
requiredSystemFeatures = [ "benchmark" ];
```
or
```nix
requiredSystemFeatures = [ "benchmark" "kvm" ];
```
`itchy` cannot do builds that require `kvm`, but `scratchy` does support
such builds. For regular builds, `itchy` will be preferred over
`scratchy` because it has a higher speed factor.
Remote builders can also be configured in `nix.conf`, e.g. Remote builders can also be configured in `nix.conf`, e.g.
@@ -70,180 +159,3 @@ option `builders-use-substitutes` in your local `nix.conf`.
To build only on remote builders and disable building on the local To build only on remote builders and disable building on the local
machine, you can use the option `--max-jobs 0`. machine, you can use the option `--max-jobs 0`.
---
Each machine specification consists of the following attributes.
How those are combined within the configuration file differs for the formats, and will be explained further down.
1. `uri` (**required**)
The URI of the remote store in the format
`ssh[-ng]://[username@]hostname[?port=<port>]`, e.g. `ssh://nix@mac` or `ssh://mac`.
If the ssh server is not listening on port 22 (e.g. port 1337 in this case)
the URI would be `ssh[-ng]://nix@mac?port=1337`. The hostname
may be an alias defined in your `~/.ssh/config`.
2. `system-types` (**optional**)
A list of Nix platform type identifiers, such as
`x86_64-darwin`. It is possible for a machine to support multiple
platform types, e.g., `i686-linux` and `x86_64-linux`.
Defaults to the local platform type
3. `ssh-key` (**optional**)
The SSH identity file to be used to log in to the remote machine.
Defaults to SSHs regular identities.
4. `jobs` (**optional**)
The maximum number of builds that Lix will execute in parallel on
the machine. Typically, this should be equal to the number of CPU
cores divided by the cores within the target machines configuration, i.e. `jobs * cores ~= cpu cores`
Defaults to 1; must be a positive integer.
5. `speed-factor`
The “speed factor”, indicating the relative speed of the machine. If
there are multiple machines of the right type, Lix will prefer the
fastest, taking load into account.
Defaults to 1; must be a positive float.
6. `supported-features` (**optional**)
A list of *supported features*. If a derivation has
the `requiredSystemFeatures` attribute, then Lix will only schedule
the derivation on a machine that has the specified features. For
example, the attribute
```nix
requiredSystemFeatures = [ "kvm" ];
```
will cause the build to be performed on a machine that has the `kvm`
feature.
Defaults to an empty list.
7. `mandatory-features` (**optional**)
A list of *mandatory features*. A machine will only
be used to build a derivation if all the machine’s mandatory
features appear in the derivation’s `requiredSystemFeatures`
attribute.
Defaults to an empty list.
8. `ssh-public-host-key` (**optional**)
The public host key of the remote machine.
Defaults to basic ssh behavior (checking contents of the known-hosts file)
### Using a TOML configuration
Each machine is configured as an attribute within the map called `machines`.
The attributes name is the machines name.
Attributes can be in any order.
For example:
```toml
version = 1
[machines.andesite]
uri = "ssh://lix@andesite.lix.systems" # toml also allows for comments
system-types = ["i686-linux"]
jobs = 8
speed-factor = 1.0
supported-features = ["kvm"]
ssh-key = "/home/deepslate/.ssh/id_ed25519"
[machines.diorite]
uri = "ssh://lix@diorite.lix.systems"
system-types = ["i686-linux"]
jobs = 8
speed-factor = 2.0
ssh-key = "/home/deepslate/.ssh/id_ed25519"
[machines.granite]
uri = "ssh://lix@granite.lix.systems"
system-types = ["i686-linux"]
jobs = 1
speed-factor = 2.0
supported-features = ["kvm", "benchmark"]
ssh-key = "/home/deepslate/.ssh/id_ed25519"
[machines.legacy]
uri = "ssh://nix@nix-15-11.nixos.org"
enable = false
```
> **Note**
>
> If the version tag is omitted (e.g. in the CLI), it defaults to the latest version.
> It is strongly recommended to always provide a version tag for configuration within files to avoid breakage.
For testing purposes, one can also define a builder ad hoc on the CLI as follows:
`--builders 'machines.andesite = {uri = "ssh://lix@andesite.lix.systems", jobs = 8}'`
#### Special handling of fields
- `enable` (**optional**)
If set to false, the declared machine will not be loaded.
This allows one to statically disable machines.
Defaults to true
### Using the legacy format
> **Warning**
>
> This format is frozen and new features / configuration options will not be backported to this format.
It is possible to specify multiple builders separated by a semicolon or
a newline, e.g.
```console
--builders 'ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd'
```
Every machine specification consists of the elements listed in the section above, seperated by any amount of spaces or tabs.
The Attributes need to be provided **in order** and without names.
To leave a field at its default, set it to `-`.
Lists are colon seperated, without additional spaces.
```
lix@andesite.lix.systems i686-linux /home/deepslate/.ssh/id_ed25519 8 1 kvm
lix@diorite.lix.systems i686-linux /home/deepslate/.ssh/id_ed25519 8 2
lix@granite.lix.systems i686-linux /home/deepslate/.ssh/id_ed25519 1 2 kvm benchmark
```
#### Special handling of fields
- `uri`: Due to backward compatibility, the `ssh://` may be omitted for the store-uri.
- `ssh-public-host-key`: The key must be provided encoded in base64. Specifically calculated via `base64 -w0 /etc/ssh/ssh_host_ed25519_key.pub`
### Format detection
At first, the given configuration is being parsed syntactically as a toml.
If parsing fails and the given configuration contains a `"` the error is presented to the user, as those characters are necessary for TOML, but disallowed for the legacy format.
Otherwise, parsing is retried using the legacy format.
If non-syntactic errors are detected within the toml, the exception will always be shown to the user directly.
## Builder selection
The configuration(s) above specify several machines that can perform `i686-linux` builds.
However, `granite` will only do builds that have the attribute
```nix
requiredSystemFeatures = [ "benchmark" ];
```
or
```nix
requiredSystemFeatures = [ "benchmark" "kvm" ];
```
`diorite` cannot do builds that require `kvm`, but `andesite` does support
such builds. For regular builds, `diorite` will be preferred over
`andesite` because it has a higher speed factor.
doc/manual/src/advanced-topics/pasta.md
-19
View File
@@ -1,19 +0,0 @@
# [Pasta](https://passt.top/passt/about/): a network sandbox for fixed-output derivations
## Introduction
This section only applies to **Linux systems** as Pasta is a Linux-only measure.
Since [CVE-2025-46416](https://lix.systems/blog/2025-06-24-lix-cves/), the Lix project decided to adopt [Pasta](https://passt.top/passt/about/) for all fixed-output derivations, protecting against various attack vectors such as UNIX abstract domain sockets or more manipulation at the network layer from a malicious fixed-output derivation code.
Pasta acts as a translation layer between a layer-2 network interface and layer-4 sockets (TCP, UDP, ICMP/ICMPv6 echo) on the host. It requires no special privileges and can serve as a alternative to [SLiRP](https://en.wikipedia.org/wiki/Slirp) which was used [by Guix to mitigate the same problem](https://codeberg.org/guix/guix/commit/fb42611b8f27960304db5a1c0d33b8371dcde2a8).
## How to disable Pasta?
It's sufficient to pass `pasta-path = ""` in your `/etc/nix/nix.conf` or on the command line `--pasta-path ""` of a Lix invocation.
## Known issues surrounding Pasta
- Only the first DNS server in `/etc/resolv.conf` is considered: failover is not possible.
- [Reduced feature set compared to the Linux kernel](https://passt.top/passt/about/#features)
- [Performance overhead in multi-gigabits contexts and IMIX MTUs](https://passt.top/passt/about/#performance_1)
doc/manual/src/command-ref/nix-build.md
+1 -1
View File
@@ -58,7 +58,7 @@ $ nix-build flake:nixpkgs -A firefox
$ nix-build flake:github:NixOS/nixpkgs/release-23.11 -A firefox $ nix-build flake:github:NixOS/nixpkgs/release-23.11 -A firefox
``` ```
Finally, for legacy reasons, if a path starts with `channel:`, the rest of the argument is interpreted as the name of a *nixpkgs* channel tarball to fetch from `https://channels.nixos.org/$CHANNEL_NAME/nixexprs.tar.xz`. Finally, for legacy reasons, if a path starts with `channel:`, the rest of the argument is interpreted as the name of a *nixpkgs* channel tarball to fetch from `https://nixos.org/channels/$CHANNEL_NAME/nixexprs.tar.xz`.
This is a **hard coded URL** pattern and is *not* related to the subscribed channels managed by the [nix-channel](./nix-channel.md) command. This is a **hard coded URL** pattern and is *not* related to the subscribed channels managed by the [nix-channel](./nix-channel.md) command.
> **Note**: any of the special syntaxes may always be disambiguated by prefixing the path. > **Note**: any of the special syntaxes may always be disambiguated by prefixing the path.
doc/manual/src/command-ref/nix-channel.md
+3 -3
View File
@@ -11,7 +11,7 @@
Channels are a mechanism for referencing remote Nix expressions and conveniently retrieving their latest version. Channels are a mechanism for referencing remote Nix expressions and conveniently retrieving their latest version.
The moving parts of channels are: The moving parts of channels are:
- The official channels listed at <https://channels.nixos.org> - The official channels listed at <https://nixos.org/channels>
- The user-specific list of [subscribed channels](#subscribed-channels) - The user-specific list of [subscribed channels](#subscribed-channels)
- The [downloaded channel contents](#channels) - The [downloaded channel contents](#channels)
- The [Nix expression search path](@docroot@/command-ref/conf-file.md#conf-nix-path), set with the [`-I` option](#opt-I) or the [`NIX_PATH` environment variable](#env-NIX_PATH) - The [Nix expression search path](@docroot@/command-ref/conf-file.md#conf-nix-path), set with the [`-I` option](#opt-I) or the [`NIX_PATH` environment variable](#env-NIX_PATH)
@@ -77,9 +77,9 @@ This command has the following operations:
Subscribe to the Nixpkgs channel and run `hello` from the GNU Hello package: Subscribe to the Nixpkgs channel and run `hello` from the GNU Hello package:
```console ```console
$ nix-channel --add https://channels.nixos.org/nixpkgs-unstable $ nix-channel --add https://nixos.org/channels/nixpkgs-unstable
$ nix-channel --list $ nix-channel --list
nixpkgs https://channels.nixos.org/nixpkgs nixpkgs https://nixos.org/channels/nixpkgs
$ nix-channel --update $ nix-channel --update
$ nix-shell -p hello --run hello $ nix-shell -p hello --run hello
hello hello
doc/manual/src/command-ref/nix-env/install.md
+2 -2
View File
@@ -148,8 +148,8 @@ To copy the store path with symbolic name `gcc` from another profile:
$ nix-env --install --from-profile /nix/var/nix/profiles/foo gcc $ nix-env --install --from-profile /nix/var/nix/profiles/foo gcc
``` ```
To install a specific [store derivation](@docroot@/glossary.md#gloss-store-derivation) To install a specific [store derivation] (typically created by
(typically created by `nix-instantiate`): `nix-instantiate`):
```console ```console
$ nix-env --install /nix/store/fibjb1bfbpm5mrsxc4mh2d8n37sxh91i-gcc-3.4.3.drv $ nix-env --install /nix/store/fibjb1bfbpm5mrsxc4mh2d8n37sxh91i-gcc-3.4.3.drv
doc/manual/src/command-ref/nix-instantiate.md
+5 -17
View File
@@ -5,7 +5,7 @@
# Synopsis # Synopsis
`nix-instantiate` `nix-instantiate`
[`--parse` | `--eval` [`--strict`] [`--raw`] [`--json`] [`--xml`] ] [`--parse` | `--eval` [`--strict`] [`--json`] [`--xml`] ]
[`--read-write-mode`] [`--read-write-mode`]
[`--arg` *name* *value*] [`--arg` *name* *value*]
[{`--attr`| `-A`} *attrPath*] [{`--attr`| `-A`} *attrPath*]
@@ -107,27 +107,15 @@ See that section for complete details (`nix-build --help`), but in summary, a pa
> This option can cause non-termination, because lazy data > This option can cause non-termination, because lazy data
> structures can be infinitely large. > structures can be infinitely large.
- `--raw`\
When used with `--eval`, the result must be coercible to a string, i.e.,
something that can be converted using `${...}`.
Integers will always generate an error when output via `--raw`, regardless of
[`coerce-integers`](../contributing/experimental-features.md#xp-feature-coerce-integers) being enabled, to avoid ambiguity.
The output is printed exactly as-is, with no quotes, escaping, or trailing
newline.
- `--json`\ - `--json`\
When used with `--eval`, print the resulting value as an JSON When used with `--eval`, print the resulting value as an JSON
representation of the resulting value rather than as a Nix expression. representation of the abstract syntax tree rather than as a Nix expression.
The conversion behaviour, if `--strict` is passed, is the same as
[`builtins.toJSON`](../language/builtins.md#builtins-toJSON).
- `--xml`\ - `--xml`\
When used with `--eval`, print the resulting value as an XML When used with `--eval`, print the resulting value as an XML
representation of the resulting value rather than as a Nix expression. representation of the abstract syntax tree rather than as a Nix expression.
The schema is the same as that used by [`builtins.toXML`](../language/builtins.md#builtins-toXML). The schema is the same as that used by the [`toXML`
built-in](../language/builtins.md).
- `--read-write-mode`\ - `--read-write-mode`\
When used with `--eval`, perform evaluation in read/write mode so When used with `--eval`, perform evaluation in read/write mode so
doc/manual/src/command-ref/nix-store/realise.md
+2
View File
@@ -15,6 +15,7 @@ Each of *paths* is processed as follows:
1. If it is not [valid], substitute the store derivation file itself. 1. If it is not [valid], substitute the store derivation file itself.
2. Realise its [output paths]: 2. Realise its [output paths]:
- Try to fetch from [substituters] the [store objects] associated with the output paths in the store derivation's [closure]. - Try to fetch from [substituters] the [store objects] associated with the output paths in the store derivation's [closure].
- With [content-addressed derivations] (experimental): Determine the output paths to realise by querying content-addressed realisation entries in the [Nix database].
- For any store paths that cannot be substituted, produce the required store objects. This involves first realising all outputs of the derivation's dependencies and then running the derivation's [`builder`](@docroot@/language/derivations.md#attr-builder) executable. <!-- TODO: Link to build process page #8888 --> - For any store paths that cannot be substituted, produce the required store objects. This involves first realising all outputs of the derivation's dependencies and then running the derivation's [`builder`](@docroot@/language/derivations.md#attr-builder) executable. <!-- TODO: Link to build process page #8888 -->
- Otherwise, and if the path is not already valid: Try to fetch the associated [store objects] in the path's [closure] from [substituters]. - Otherwise, and if the path is not already valid: Try to fetch the associated [store objects] in the path's [closure] from [substituters].
@@ -27,6 +28,7 @@ If no substitutes are available and no store derivation is given, realisation fa
[store objects]: @docroot@/glossary.md#gloss-store-object [store objects]: @docroot@/glossary.md#gloss-store-object
[closure]: @docroot@/glossary.md#gloss-closure [closure]: @docroot@/glossary.md#gloss-closure
[substituters]: @docroot@/command-ref/conf-file.md#conf-substituters [substituters]: @docroot@/command-ref/conf-file.md#conf-substituters
[content-addressed derivations]: @docroot@/contributing/experimental-features.md#xp-feature-ca-derivations
[Nix database]: @docroot@/glossary.md#gloss-nix-database [Nix database]: @docroot@/glossary.md#gloss-nix-database
The resulting paths are printed on standard output. The resulting paths are printed on standard output.
doc/manual/src/command-ref/opt-common.md
-6
View File
@@ -177,12 +177,6 @@ Most commands in Lix accept the following command-line options:
You can override this using `--arg`, e.g., `nix-env --install --attr pkgname --arg system \"i686-freebsd\"`. You can override this using `--arg`, e.g., `nix-env --install --attr pkgname --arg system \"i686-freebsd\"`.
(Note that since the argument is a Nix string literal, you have to escape the quotes.) (Note that since the argument is a Nix string literal, you have to escape the quotes.)
Additionally, dots are interpreted as attribute-path separators.
I.e. `nix-instantiate '<nixpkgs>' -A hello-unfree --arg config.allowUnfree true` will result in an argument `config` with value `{ allowUnfree = true; }` being passed to `<nixpkgs>`.
Please note that merging of different arguments is rejected.
I.e. `--arg config '{ cudaSupport = true; }' --arg config.allowUnfree true` will not work whereas `--arg config.cudaSupport true --arg config.allowUnfree true` is accepted.
- <span id="opt-argstr">[`--argstr`](#opt-argstr)</span> *name* *value* - <span id="opt-argstr">[`--argstr`](#opt-argstr)</span> *name* *value*
This option is like `--arg`, only the value is not a Nix expression but a string. This option is like `--arg`, only the value is not a Nix expression but a string.
doc/manual/src/contributing/cli-guideline.md
+2 -2
View File
@@ -661,8 +661,8 @@ Verbosity levels are:
The default level that the command starts is `ERROR`. The simplest way to The default level that the command starts is `ERROR`. The simplest way to
increase the verbosity by stacking `-v` option (eg: `-vvv == level 3 == INFO`). increase the verbosity by stacking `-v` option (eg: `-vvv == level 3 == INFO`).
Use `--quiet` to decrease verbosity by one level. There are also two shortcuts, `--debug` to run in `DEBUG` verbosity level and
There is one shortcut, `--debug` to run in `DEBUG` verbosity level. `--quiet` to run in `ERROR` verbosity level.
---------- ----------
doc/manual/src/contributing/deprecated-features.md
+1 -1
View File
@@ -19,7 +19,7 @@ This description is not normative, but a feature removal may roughly happen like
1. Add a warning when the feature is being used. 1. Add a warning when the feature is being used.
2. Disable the feature by default, putting it behind a deprecated feature flag. 2. Disable the feature by default, putting it behind a deprecated feature flag.
- If disabling the feature started out as an opt-in experimental feature, turn that experimental flag into a no-op or remove it entirely. - If disabling the feature started out as an opt-in experimental feature, turn that experimental flag into a no-op or remove it entirely.
For example, `--extra-experimental-features no-url-literals` becomes `--extra-deprecated-features url-literals`. For example, `--extra-experimental-features=no-url-literals` becomes `--extra-deprecated-features=url-literals`.
3. Decide on a time frame for how long that feature will still be supported for backwards compatibility, and clearly communicate that in the error messages. 3. Decide on a time frame for how long that feature will still be supported for backwards compatibility, and clearly communicate that in the error messages.
- Sometimes, automatic migration to alternatives is possible, and such should be provided if possible - Sometimes, automatic migration to alternatives is possible, and such should be provided if possible
- At least one NixOS release cycle should be the minimum - At least one NixOS release cycle should be the minimum
doc/manual/src/contributing/hacking.md
+31 -136
View File
@@ -11,19 +11,7 @@ The following instructions assume you already have some version of Nix or Lix in
[installation instructions]: ../installation/installation.md [installation instructions]: ../installation/installation.md
A typical development flow for simple changes in Lix looks like: ## Building Lix in a development shell
- [Set up and build Lix](#building)
- For large changes, check in regarding design and possibly create an RFD issue on Forgejo
- Make the changes in your editor
- [Send the changes to Gerrit](#sending-to-gerrit)
- Once you have the number for the CL from Gerrit to put in the changelog, [write a changelog entry](#release-notes) and amend it into the commit
- Update the Gerrit change by submitting it with the same command as the first time
- Request and receive a code review
- Address feedback from the review
- Amend commits, send to Gerrit again
- Submit the approved change
## Building Lix in a development shell {#building}
### Setting up the development shell ### Setting up the development shell
@@ -51,64 +39,28 @@ $ nix-shell -A native-clangStdenvPackages
### Building from the development shell ### Building from the development shell
We have a [justfile](https://just.systems/) for extra convenient building. Run a clean build and test with `just clean build install test`.
It defaults to using `./build` as the build directory, and `$out` (`./outputs/out`) as the install directory.
For most cases, you can clean-build, install, and run the tests with:
```bash
$ just setup --wipe && just test
```
> **Note**
>
> The `--wipe` argument to `meson setup` conveniently works whether you have an existing build directory or not.
>
> However, it is *mostly*, but not *exactly* equivalent to deleting the build directory first.
> In particular, previously specified `-D` build options are **preserved** with `--wipe` (for some reason).
> For example, if you fetch and checkout a new version of Lix, and that new version *removes* a Meson build option from `./meson.options`, *and* a previous invocation in that build directory explicitly set that option, then `meson setup --wipe build` will error, complaining about the unknown option.
> For these cases, `just clean` will give you a well-and-truly-this-time-for-real clean build.
Because the integration tests require installation to work, `just test` automatically also calls `just install`, and Meson helpfully will automatically build any targets that need building when trying to install them.
You can override the build directory or install directory by setting the justfile [variables](https://just.systems/man/en/setting-variables-from-the-command-line.html) `outdir` and `builddir` on the command-line:
```bash
$ just builddir=build-before-bisect outdir=out-before-bisect setup
$ just builddir=build-before-bisect test
```
You'll have to set `builddir` for every target, but `outdir` only needs to be set for `setup`.
You can also run the unit tests and integration tests separately: You can also run the unit tests and integration tests separately:
```bash ```bash
$ just setup $ just setup build test-unit
$ just test-unit $ just install test-integration
$ just test-integration
``` ```
Most justfile targets forward all further arguments to the underlying Meson invocation. Many targets have a `-custom` variant which pass extra arguments to `meson`.
For example, to work on both Lix and nix-eval-jobs you can run: For example, to work on both Lix and nix-eval-jobs you can run:
```bash ```
$ just setup -Dnix-eval-jobs=enabled $ just setup-custom -Dnix-eval-jobs=enabled
$ # or
$ mesonFlags=-Dnix-eval-jobs=enabled just setup
``` ```
Note that only targets which *don't* accept extra arguments can have other targets following them. Note that only targets which don't accept extra arguments can be used when
`just clean setup` is equivalent to `just clean && just setup`, but `just build test` runs the `build` target with the argument `test`. running multiple targets at once; `just setup build` is fine, but `just
This means that if you want to, for example, build with lower parallelism, and then test, you will have to do something like this: setup-custom build` is an error. The `test` target is usually the last one to
run, so it always accepts extra arguments.
```bash
$ just build -j4
$ just test
```
Finally, the rewrite of the integration test suite, functional2, also has its own justfile target which allows passing extra arguments to pytest.
For example, to collect and list all functional2 tests without running them, you can pass pytest's `--collect-only` argument:
```bash
$ just test-functional2 --collect-only
```
You can also build Lix manually: You can also build Lix manually:
@@ -177,59 +129,7 @@ To inspect the canonical source of truth on what the state of the buildsystem co
$ meson introspect $ meson introspect
``` ```
#### LLD ## Building Lix outside of development shells
The development shell on Linux uses LLD by default for faster link times.
This is set using `mesonFlags`, so to override it, you can simplify re-specify the linker to Meson:
```bash
$ just setup -Dc_link_args=-fuse-ld=ld -Dcpp_link_args=-fuse-ld=ld
```
While using LLD, you may find it helpful to use ThinLTO for even further improvements to link times for incremental builds:
```bash
$ just setup -Db_lto=true -Db_lto_mode=thin -Db_thinlto_cache=true
```
## Sending changes to Gerrit for review {#sending-to-gerrit}
We use Gerrit for all our code review in Lix.
Our instance is at <https://gerrit.lix.systems>.
There's much more information about how to use Gerrit in the [wiki section on Gerrit][wiki-gerrit] including how to use Jujutsu, how to use the UI and more.
The Snix project also has some Gerrit information [in their contributing docs][snix-gerrit].
[wiki-gerrit]: https://wiki.lix.systems/books/contributing/chapter/intro-to-gerrit
[snix-gerrit]: https://snix.dev/docs/guides/contributing/
The gist is that once you have your SSH key and git remote set up, you can send commits for review with:
```
$ git remote set-url origin ssh://YOURUSERNAME@gerrit.lix.systems:2022/lix
$ git push origin HEAD:refs/for/main
```
Then, you can request a review via the "Reply" button on the web UI.
If you click "Suggest Owners", it will try to suggest the maintainers of the area of the code change to send review requests to.
Requesting reviews from multiple people is normal.
We do our best to respond to directly sent reviews in a few days, so feel free to request another reviewer or ask on Matrix if you've not got a response for a while.
Keep in mind that Lix is a volunteer project and we have limited bandwidth, so some changes aren't feasible to shepherd through; please check in on Matrix at design time when doing large changes.
Once you get a `Code-Review+2` vote on your change, it's rebased on `main` and CI marks it `Verified+1`, you're able (and usually expected, so you can have a second chance to check it over) to hit the Submit button to merge it.
If the change appears as "Rebase Required", you need to rebase it on `main` locally or via the Gerrit UI and wait for `Verified+1` before the Submit button is made active
The `Code-Review+2` from before will stick around through trivial rebases so no need to re-request review for a mere rebase.
## Interacting with the CI, Buildkite
We use Buildkite for our CI, usually you will not have to interact directly with it other than reviewing any errors it produces, which are linked from Gerrit.
However in certain cases a CI run will fail due to transient issues not related to your code and you will need to rerun it by hand.
You can log in to the CI via [SSO](https://buildkite.com/sso/afnix). On your job you can then hit the "Retry failed" button to rerun it, normally you will not have a repeat of the transient issue.
If the build still fails on CI issues or all builds are failing this should be reported via [Zulip on #T-infra](https://zulip.lix.systems/#narrow/channel/7-T-infra) or [Matrix on #dev](https://matrix.to/#/%23dev%3Alix.systems?via=lix.systems).
## Building Lix with `nix`
To build a release version of Lix for the current operating system and CPU architecture: To build a release version of Lix for the current operating system and CPU architecture:
@@ -386,10 +286,10 @@ Configure your editor to use the `clangd` from the shell, either by running it i
> Some other editors (e.g. Emacs, Vim) need a plugin to support LSP servers in general (e.g. [lsp-mode](https://github.com/emacs-lsp/lsp-mode) for Emacs and [vim-lsp](https://github.com/prabirshrestha/vim-lsp) for vim). > Some other editors (e.g. Emacs, Vim) need a plugin to support LSP servers in general (e.g. [lsp-mode](https://github.com/emacs-lsp/lsp-mode) for Emacs and [vim-lsp](https://github.com/prabirshrestha/vim-lsp) for vim).
> Editor-specific setup is typically opinionated, so we will not cover it here in more detail. > Editor-specific setup is typically opinionated, so we will not cover it here in more detail.
# Manual and documentation ### Checking links in the manual
## Building the manual
The build checks for broken internal links.
This happens late in the process, so `nix build` is not suitable for iterating.
To build the manual incrementally, run: To build the manual incrementally, run:
```console ```console
@@ -401,20 +301,15 @@ meson compile -C build manual
[`mdbook-linkcheck`]: https://github.com/Michael-F-Bryan/mdbook-linkcheck [`mdbook-linkcheck`]: https://github.com/Michael-F-Bryan/mdbook-linkcheck
[URI fragments]: https://en.wikipedia.org/wiki/URI_fragment [URI fragments]: https://en.wikipedia.org/wiki/URI_fragment
The built manual is in `build/doc/manual/manual/index.html`. #### `@docroot@` variable
The build checks for broken internal links. `@docroot@` provides a base path for links that occur in reusable snippets or other documentation that doesn't have a base path of its own.
This happens late in the process, so `nix build` is not suitable for iterating and it's recommended to use the `meson` command above instead.
### `@\docroot\@` variable If a broken link occurs in a snippet that was inserted into multiple generated files in different directories, use `@docroot@` to reference the `doc/manual/src` directory.
`@\docroot\@` provides a base path for links that occur in reusable snippets or other documentation that doesn't have a base path of its own. If the `@docroot@` literal appears in an error message from the `mdbook-linkcheck` tool, the `@docroot@` replacement needs to be applied to the generated source file that mentions it.
See existing `@docroot@` logic in the [Makefile].
If a broken link occurs in a snippet that was inserted into multiple generated files in different directories, use `@\docroot\@` to reference the `doc/manual/src` directory. Regular markdown files used for the manual have a base path of their own and they can use relative paths instead of `@docroot@`.
If the `@\docroot\@` literal appears in an error message from the `mdbook-linkcheck` tool, the `@\docroot\@` replacement needs to be applied to the generated source file that mentions it.
See existing `@\docroot\@` logic in `doc/manual/substitute.py`.
Regular markdown files used for the manual have a base path of their own and they can use relative paths instead of `@\docroot\@`.
## API documentation ## API documentation
@@ -446,7 +341,7 @@ You can build it yourself:
Metrics about the change in line/function coverage over time will be available in the future (FIXME(lix-hydra)). Metrics about the change in line/function coverage over time will be available in the future (FIXME(lix-hydra)).
## Add a release note {#release-notes} ## Add a release note
`doc/manual/rl-next` contains release notes entries for all unreleased changes. `doc/manual/rl-next` contains release notes entries for all unreleased changes.
@@ -515,15 +410,15 @@ The following properties are supported:
### Build process ### Build process
Releases have a precomputed `rl-MAJOR.MINOR.md`, and no `rl-next.md`. Releases have a precomputed `rl-MAJOR.MINOR.md`, and no `rl-next.md`.
Development releases have a generated `rl-next.md`. Set `buildUnreleasedNotes = true;` in `flake.nix` to build the release notes on the fly.
# Adding experimental or deprecated features, global settings, or builtins ## Adding experimental or deprecated features, global settings, or builtins
Experimental and deprecated features, global settings, and builtins are generally referenced both in the code and in the documentation. Experimental and deprecated features, global settings, and builtins are generally referenced both in the code and in the documentation.
To prevent duplication or divergence, they are defined in data files, and a script generates the necessary glue. To prevent duplication or divergence, they are defined in data files, and a script generates the necessary glue.
The data file format is similar to the release notes: it consists of a YAML metadata header, followed by the documentation in Markdown format. The data file format is similar to the release notes: it consists of a YAML metadata header, followed by the documentation in Markdown format.
## Experimental or deprecated features ### Experimental or deprecated features
Experimental and deprecated features support the following metadata properties: Experimental and deprecated features support the following metadata properties:
* `name` (required): user-facing name of the feature, to be used in `nix.conf` options and on the command line. * `name` (required): user-facing name of the feature, to be used in `nix.conf` options and on the command line.
@@ -533,7 +428,7 @@ Experimental and deprecated features support the following metadata properties:
Experimental feature data files should live in `lix/libutil/experimental-features`, and deprecated features in `lix/libutil/deprecated-features`. Experimental feature data files should live in `lix/libutil/experimental-features`, and deprecated features in `lix/libutil/deprecated-features`.
They must be listed in the `experimental_feature_definitions` or `deprecated_feature_definitions` lists in `lix/libutil/meson.build` respectively to be considered by the build system. They must be listed in the `experimental_feature_definitions` or `deprecated_feature_definitions` lists in `lix/libutil/meson.build` respectively to be considered by the build system.
## Global settings ### Global settings
Global settings support the following metadata properties: Global settings support the following metadata properties:
* `name` (required): user-facing name of the setting, to be used as key in `nix.conf` and in the `--option` command line argument. * `name` (required): user-facing name of the setting, to be used as key in `nix.conf` and in the `--option` command line argument.
@@ -561,12 +456,12 @@ Settings are not collected in a single place in the source tree, so an appropria
Look for related setting definition files under second-level subdirectories of `lix` whose name includes `settings`. Look for related setting definition files under second-level subdirectories of `lix` whose name includes `settings`.
Then add the new file there, and don't forget to register it in the appropriate `meson.build` file. Then add the new file there, and don't forget to register it in the appropriate `meson.build` file.
## Builtin functions ### Builtin functions
The following metadata properties are supported for builtin functions: The following metadata properties are supported for builtin functions:
* `name` (required): the language-facing name (as a member of the `builtins` attribute set) of the function. * `name` (required): the language-facing name (as a member of the `builtins` attribute set) of the function.
* `implementation` (optional): a C++ expression specifying the implementation of the builtin. * `implementation` (optional): a C++ expression specifying the implementation of the builtin.
It must be a function of signature `Value(EvalState &, PosIdx, Value * *)`. It must be a function of signature `void(EvalState &, PosIdx, Value * *, Value &)`.
If not specified, defaults to `prim_${name}`. If not specified, defaults to `prim_${name}`.
* `renameInGlobalScope` (optional): whether the definition should be "hidden" in the global scope by prefixing its name with two underscores. * `renameInGlobalScope` (optional): whether the definition should be "hidden" in the global scope by prefixing its name with two underscores.
If not specified, defaults to `true`. If not specified, defaults to `true`.
@@ -577,7 +472,7 @@ The following metadata properties are supported for builtin functions:
New builtin function definition files must be added to `lix/libexpr/builtins` and registered in the `builtin_definitions` list in `lix/libexpr/meson.build`. New builtin function definition files must be added to `lix/libexpr/builtins` and registered in the `builtin_definitions` list in `lix/libexpr/meson.build`.
## Builtin constants ### Builtin constants
The following metadata properties are supported for builtin constants: The following metadata properties are supported for builtin constants:
* `name` (required): the language-facing name (as a member of the `builtins` attribute set) of the constant. * `name` (required): the language-facing name (as a member of the `builtins` attribute set) of the constant.
* `type` (required): the Nix language type of the constant; the C++ type is automatically derived. * `type` (required): the Nix language type of the constant; the C++ type is automatically derived.
doc/manual/src/contributing/testing.md
+5 -10
View File
@@ -62,12 +62,6 @@ For `installcheck` specifically, first run `just install` before running the tes
Finer-grained filtering within a test suite is also possible using the [--gtest_filter](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) command-line option to a test suite executable, or the `GTEST_FILTER` environment variable. Finer-grained filtering within a test suite is also possible using the [--gtest_filter](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) command-line option to a test suite executable, or the `GTEST_FILTER` environment variable.
### Inspecting failures
The test suite emits logs in `build/meson-logs/`; the full textual failure logs are in `build/meson-logs/testlog.txt`.
If you want a much nicer experience of viewing the logs in a structured manner, use `xunit-viewer --results build/meson-logs/testlog.junit.xml --server` to view them in a web browser.
### Unit test support libraries ### Unit test support libraries
There are headers and code which are not just used to test the library in question, but also downstream libraries. There are headers and code which are not just used to test the library in question, but also downstream libraries.
@@ -383,10 +377,7 @@ I grepped `lix/` for `get[eE]nv\("` to find the mentions in Lix code.
Overrides compile-time configuration of various locations used by Lix. See `lix/libstore/globals.cc`. Overrides compile-time configuration of various locations used by Lix. See `lix/libstore/globals.cc`.
**Expected value**: a directory **Expected value**: a directory
- `LIX_DAEMON_SOCKET_DIR` (optional) - Overrides the daemon socket directory from `$NIX_STATE_DIR/daemon-socket`. - `NIX_DAEMON_SOCKET_PATH` (optional) - Overrides the daemon socket path from `$NIX_STATE_DIR/daemon-socket/socket`.
**Expected value**: a directory
- `NIX_DAEMON_SOCKET_PATH` (optional) - Overrides the daemon socket path from `$NIX_STATE_DIR/daemon-socket/socket`. Ignored if `LIX_DAEMON_SOCKET_DIR` is set.
**Expected value**: path to a socket **Expected value**: path to a socket
- `NIX_LOG_FD` (output) - An FD number for logs in `internal-json` format to be sent to. - `NIX_LOG_FD` (output) - An FD number for logs in `internal-json` format to be sent to.
@@ -404,6 +395,7 @@ I grepped `lix/` for `get[eE]nv\("` to find the mentions in Lix code.
**Expected value**: the path to an executable shell **Expected value**: the path to an executable shell
- `PRINT_PATH` - Undocumented. Used by `nix-prefetch-url` as an alternative form of `--print-path`. Why??? - `PRINT_PATH` - Undocumented. Used by `nix-prefetch-url` as an alternative form of `--print-path`. Why???
- `_NIX_IN_TEST` - If present with any value, makes `fetchClosure` accept file URLs in addition to HTTP ones. Why is this not `_NIX_FORCE_HTTP`??
Not used anywhere else. Not used anywhere else.
- `NIX_ALLOW_EVAL` - Used by eval-cache tests to block evaluation if set to `0`. - `NIX_ALLOW_EVAL` - Used by eval-cache tests to block evaluation if set to `0`.
@@ -457,6 +449,9 @@ I grepped `lix/` for `get[eE]nv\("` to find the mentions in Lix code.
- `NIX_CLIENT_PACKAGE` - Runs the test suite against an alternate Nix client with the current daemon. - `NIX_CLIENT_PACKAGE` - Runs the test suite against an alternate Nix client with the current daemon.
**Expected value**: something like `/nix/store/...-nix-2.18.2` **Expected value**: something like `/nix/store/...-nix-2.18.2`
- `NIX_TESTS_CA_BY_DEFAULT` - Pass `__contentAddressed`, `outputHashMode` and `outputHashAlgo` to builds of some input-addressed derivations in the test suite.
**Expected value**: 1
- `TEST_DATA` - Not an environment variable! This is used in repl characterization tests to refer to `tests/functional/repl_characterization/data`. - `TEST_DATA` - Not an environment variable! This is used in repl characterization tests to refer to `tests/functional/repl_characterization/data`.
More specifically, that path is replaced with the string `$TEST_DATA` in output for reproducibility. More specifically, that path is replaced with the string `$TEST_DATA` in output for reproducibility.
- `TEST_HOME` (output) - Set to the temporary directory that is set as `$HOME` inside the tests, underneath `$TEST_ROOT`. - `TEST_HOME` (output) - Set to the temporary directory that is set as `$HOME` inside the tests, underneath `$TEST_ROOT`.
doc/manual/src/glossary.md
+9 -2
View File
@@ -41,6 +41,12 @@
[realise]: #gloss-realise [realise]: #gloss-realise
- [content-addressed derivation]{#gloss-content-addressed-derivation}
A derivation which has the
[`__contentAddressed`](./language/advanced-attributes.md#adv-attr-__contentAddressed)
attribute set to `true`.
- [fixed-output derivation]{#gloss-fixed-output-derivation} - [fixed-output derivation]{#gloss-fixed-output-derivation}
A derivation which includes the A derivation which includes the
@@ -89,7 +95,7 @@
[store path]: #gloss-store-path [store path]: #gloss-store-path
- [file system object]{#gloss-file-system-object} - [file system object]{#gloss-store-object}
The Nix data model for representing simplified file system data. The Nix data model for representing simplified file system data.
@@ -108,13 +114,14 @@
- [input-addressed store object]{#gloss-input-addressed-store-object} - [input-addressed store object]{#gloss-input-addressed-store-object}
A store object produced by building a A store object produced by building a
non-[content-addressed](#gloss-content-addressed-derivation),
non-[fixed-output](#gloss-fixed-output-derivation) non-[fixed-output](#gloss-fixed-output-derivation)
derivation. derivation.
- [output-addressed store object]{#gloss-output-addressed-store-object} - [output-addressed store object]{#gloss-output-addressed-store-object}
A [store object] whose [store path] is determined by its contents. A [store object] whose [store path] is determined by its contents.
This includes derivations and the outputs of [fixed-output derivations](#gloss-fixed-output-derivation). This includes derivations, the outputs of [content-addressed derivations](#gloss-content-addressed-derivation), and the outputs of [fixed-output derivations](#gloss-fixed-output-derivation).
- [substitute]{#gloss-substitute} - [substitute]{#gloss-substitute}
doc/manual/src/installation/installing-docker.md
-7
View File
@@ -60,10 +60,3 @@ Then:
```console ```console
$ docker run -ti lix $ docker run -ti lix
``` ```
# Known issues
Lix in Docker is very sensitive to **functional** DNS resolution if you are running with [Pasta protections](../advanced-topics/pasta.md) which are enabled by default since Lix 2.93.0 on most distributions.
If you notice failure to download things, double check whether your **first** DNS entry in `/etc/resolv.conf` is functional.
Lix with [Pasta protections](../advanced-topics/pasta.md) does not support failing over the next entries.
doc/manual/src/installation/prerequisites-source.md
+9
View File
@@ -50,6 +50,15 @@ The most current alternative to this section is to read `package.nix` and see wh
- The `boost` library of version 1.66.0 or higher. It can be obtained - The `boost` library of version 1.66.0 or higher. It can be obtained
from the official web site <https://www.boost.org/>. from the official web site <https://www.boost.org/>.
- The `editline` library of version 1.14.0 or higher. It can be
obtained from the its repository
<https://github.com/troglobit/editline>.
- The `libsodium` library for verifying cryptographic signatures
of contents fetched from binary caches.
It can be obtained from the official web site
<https://libsodium.org>.
- Recent versions of Bison and Flex to build the parser. (This is - Recent versions of Bison and Flex to build the parser. (This is
because Nix needs GLR support in Bison and reentrancy support in because Nix needs GLR support in Bison and reentrancy support in
Flex.) For Bison, you need version 2.6, which can be obtained from Flex.) For Bison, you need version 2.6, which can be obtained from
doc/manual/src/known-issues/known-issues.md
-1
View File
@@ -1 +0,0 @@
This section lists known issues around Lix.
doc/manual/src/known-issues/non-isolated-build-limits.md
-21
View File
@@ -1,21 +0,0 @@
# Limitations of non-isolated builds
## What are non-isolated builds?
In Lix, only builds done on Linux with `sandbox = true` and a functioning
`pasta-path` are isolated from the rest of the system, all other builds are
considered non-isolated to some degree.
For example, running Lix with [Pasta](@docroot@/advanced-topics/pasta.md)
disabled makes the host network visible to fixed-output derivations, reducing
isolation somewhat.
## Clean termination of non-isolated builds
Non-isolated builds may not terminate cleanly in all cases due to limitations in Lix's process management.
This occurs when a build keeps the build log file descriptor open past the end of the actual build. A common cause of this are background tasks that aren't properly terminated before the main build process exits, for example: HTTP servers run as part of a test suite.
See [issue #1018](https://git.lix.systems/lix-project/lix/issues/1018) for an example.
The only solution is to manually terminate leftover processes in your derivation, including during failure scenarios.
doc/manual/src/language/advanced-attributes.md
+11 -2
View File
@@ -209,8 +209,15 @@ Derivations can declare some infrequently used optional attributes.
- [`__contentAddressed`]{#adv-attr-__contentAddressed} - [`__contentAddressed`]{#adv-attr-__contentAddressed}
> **Warning** > **Warning**
> This attribute is part of a removed [experimental feature](@docroot@/contributing/experimental-features.md). > This attribute is part of an [experimental feature](@docroot@/contributing/experimental-features.md).
> Setting this flag *will* cause eval errors. >
> To use this attribute, you must enable the
> [`ca-derivations`](@docroot@/contributing/experimental-features.md#xp-feature-ca-derivations) experimental feature.
> For example, in [nix.conf](../command-ref/conf-file.md) you could add:
>
> ```
> extra-experimental-features = ca-derivations
> ```
If this attribute is set to `true`, then the derivation If this attribute is set to `true`, then the derivation
outputs will be stored in a content-addressed location rather than the outputs will be stored in a content-addressed location rather than the
@@ -302,6 +309,8 @@ Derivations can declare some infrequently used optional attributes.
- `maxSize` defines the maximum size of the resulting [store object](../glossary.md#gloss-store-object). - `maxSize` defines the maximum size of the resulting [store object](../glossary.md#gloss-store-object).
- `maxClosureSize` defines the maximum size of the output's closure. - `maxClosureSize` defines the maximum size of the output's closure.
- `ignoreSelfRefs` controls whether self-references should be considered when
checking for allowed references/requisites.
Example: Example:
doc/manual/src/language/derivations.md
-6
View File
@@ -17,12 +17,6 @@ the attributes of which specify the inputs of the build.
string. This is used as a symbolic name for the package by string. This is used as a symbolic name for the package by
`nix-env`, and it is appended to the output paths of the derivation. `nix-env`, and it is appended to the output paths of the derivation.
> **Note**
>
> Names can only contain alphanumerical characters (0-9, a-z, A-Z)
> as well as `+`, `-`, `.`, `_`, `?` and `=`. Names must be neither
> `.` nor `..`, and must not start with `.-` or `..-`.
- There must be an attribute named [`builder`]{#attr-builder} that identifies the - There must be an attribute named [`builder`]{#attr-builder} that identifies the
program that is executed to perform the build. It can be either a program that is executed to perform the build. It can be either a
derivation or a source (a local file reference, e.g., derivation or a source (a local file reference, e.g.,
doc/manual/src/language/values.md
+1 -1
View File
@@ -164,7 +164,7 @@ Note that lists are only lazy in values, and they are strict in length.
An attribute set is a collection of name-value-pairs (called *attributes*) enclosed in curly brackets (`{ }`). An attribute set is a collection of name-value-pairs (called *attributes*) enclosed in curly brackets (`{ }`).
An attribute name can be an identifier or a [double-quoted string](#type-string). An attribute name can be an identifier or a [string](#type-string).
An identifier must start with a letter (`a-z`, `A-Z`) or underscore (`_`), and can otherwise contain letters (`a-z`, `A-Z`), numbers (`0-9`), underscores (`_`), apostrophes (`'`), or dashes (`-`). An identifier must start with a letter (`a-z`, `A-Z`) or underscore (`_`), and can otherwise contain letters (`a-z`, `A-Z`), numbers (`0-9`), underscores (`_`), apostrophes (`'`), or dashes (`-`).
> *name* = *identifier* | *string* \ > *name* = *identifier* | *string* \
doc/manual/src/package-management/basic-package-mgmt.md
+3 -3
View File
@@ -5,7 +5,7 @@
FIXME(Lix): This section does not document the most common modern practices in terms of avoiding channels, pinning, declarative software installation (see flakey-profile or home-manager or NixOS), or using flakes, etc. FIXME(Lix): This section does not document the most common modern practices in terms of avoiding channels, pinning, declarative software installation (see flakey-profile or home-manager or NixOS), or using flakes, etc.
It is, however, likely correct at a technical level. It is, however, likely correct at a technical level.
For more information on modern practices, see the [resources](https://wiki.lix.systems/books/lix-users/page/nix-resources) page on the Lix site. For more information on modern practices, see the [resources](https://lix.systems/resources) page on the Lix site.
</div> </div>
@@ -41,7 +41,7 @@ install Lix. If this is not the case for some reason, you can add it
as follows: as follows:
```console ```console
$ nix-channel --add https://channels.nixos.org/nixpkgs-unstable $ nix-channel --add https://nixos.org/channels/nixpkgs-unstable
$ nix-channel --update $ nix-channel --update
``` ```
@@ -49,7 +49,7 @@ $ nix-channel --update
> >
> On NixOS, you’re automatically subscribed to a NixOS channel > On NixOS, you’re automatically subscribed to a NixOS channel
> corresponding to your NixOS major release (e.g. > corresponding to your NixOS major release (e.g.
> <https://channels.nixos.org/nixos-21.11>). A NixOS channel is identical > <http://nixos.org/channels/nixos-21.11>). A NixOS channel is identical
> to the Nixpkgs channel, except that it contains only Linux binaries > to the Nixpkgs channel, except that it contains only Linux binaries
> and is updated only if a set of regression tests succeed. > and is updated only if a set of regression tests succeed.
doc/manual/src/package-management/garbage-collection.md
-59
View File
@@ -71,62 +71,3 @@ $ nix-collect-garbage -d
``` ```
is a quick and easy way to clean up your system. is a quick and easy way to clean up your system.
## Garbage Collector Roots
### Explicit roots
All store paths to which there are symlinks in the directory
`prefix/nix/var/nix/gcroots` will be used as roots by the garbage
collector. For instance, the following command makes the path
`/nix/store/d718ef...-foo` a root of the collector:
```console
$ ln -s /nix/store/d718ef...-foo /nix/var/nix/gcroots/bar
```
That is, after this command, the garbage collector will not remove
`/nix/store/d718ef...-foo` or any of its dependencies.
Subdirectories of `prefix/nix/var/nix/gcroots` are also searched for
symlinks.
Symlinks may also point to paths outside the nix store. If the
destination of the symlink is itself a symlink to a store path, it
is also considered a root. This style of GC root is called an
"indirect root", and is created by tools like `nix-build` to avoid
garbage-collecting paths that are being used on-the-fly rather than
installed in profiles.
### In-use roots
Lix will also perform a best-effort detection of paths that are in use
by running processes when scanning for garbage collection roots, to
avoid removing paths that are still needed by running processes.
Exact details vary between platforms, but the following will generally
be taken into account:
- Executables in the store that are currently running;
- Other files in the store that are mapped into a process's address space (e.g. shared libraries);
- Files in the store to which processes have open handles;
- Store paths found in processes' environment variables.
Note that this detection is susceptible to missing paths that may still be in use for multiple reasons:
- Time-of-check-to-time-of-use (TOCTTOU): new processes may appear
after Lix has enumerated the currently running processes, and will
not be taken into account;
- Access privileges: if the garbage collection is not running as the
root user (this is typically the case for single-user
installations), it will not be able to scan processes belonging to
other users;
- Other types of references: store paths may be stored in parts of the
filesystem (e.g. databases) or process memory (e.g. environment
variables changed since the start of the process) that Lix does not
scan.
For this reason, it is recommended to create explicit roots whenever
using store paths that aren't obtained from some existing explicit GC
root.
doc/manual/src/package-management/garbage-collector-roots.md
+18
View File
@@ -0,0 +1,18 @@
# Garbage Collector Roots
The roots of the garbage collector are all store paths to which there
are symlinks in the directory `prefix/nix/var/nix/gcroots`. For
instance, the following command makes the path
`/nix/store/d718ef...-foo` a root of the collector:
```console
$ ln -s /nix/store/d718ef...-foo /nix/var/nix/gcroots/bar
```
That is, after this command, the garbage collector will not remove
`/nix/store/d718ef...-foo` or any of its dependencies.
Subdirectories of `prefix/nix/var/nix/gcroots` are also searched for
symlinks. Symlinks to non-store paths are followed and searched for
roots, but symlinks to non-store paths *inside* the paths reached in
that way are not followed to prevent infinite recursion.
doc/manual/src/protocols/derivation-aterm.md
+14 -4
View File
@@ -2,8 +2,18 @@
For historical reasons, [derivations](@docroot@/glossary.md#gloss-store-derivation) are stored on-disk in [ATerm](https://homepages.cwi.nl/~daybuild/daily-books/technology/aterm-guide/aterm-guide.html) format. For historical reasons, [derivations](@docroot@/glossary.md#gloss-store-derivation) are stored on-disk in [ATerm](https://homepages.cwi.nl/~daybuild/daily-books/technology/aterm-guide/aterm-guide.html) format.
Derivations are serialised in the following format: Derivations are serialised in one of the following formats:
``` - ```
Derive(...) Derive(...)
``` ```
For all stable derivations.
- ```
DrvWithVersion(<version-string>, ...)
```
The only `version-string`s that are in use today are for [experimental features](@docroot@/contributing/experimental-features.md):
- `"xp-dyn-drv"` for the [`dynamic-derivations`](@docroot@/contributing/experimental-features.md#xp-feature-dynamic-derivations) experimental feature.
doc/manual/src/quick-start.md
+1 -1
View File
@@ -5,7 +5,7 @@
FIXME(Lix): This chapter is quite outdated with respect to recommended practices in 2024 and needs updating. FIXME(Lix): This chapter is quite outdated with respect to recommended practices in 2024 and needs updating.
The commands in here will work, however, and the installation section is up to date. The commands in here will work, however, and the installation section is up to date.
For more updated guidance, see the links on <https://wiki.lix.systems/books/lix-users/page/nix-resources> For more updated guidance, see the links on <https://lix.systems/resources/>
</div> </div>
doc/manual/src/release-notes/rl-2.93.md
+238
View File
@@ -1,4 +1,242 @@
# Lix 2.93 "Bici Bici" (2025-05-09) # Lix 2.93 "Bici Bici" (2025-05-09)
# Lix 2.93.4 (2026-05-04)
## Fixes
- `build-dir` no longer defaults to `temp-dir` [cl/3453](https://gerrit.lix.systems/c/lix/+/3453)
The directory in which temporary build directories are created no longer defaults
to the value of the `temp-dir` setting to avoid builders making their directories
world-accessible. This behavior has been used to escape the build sandbox and can
cause build impurities even when not used maliciously. We now default to `builds`
in `NIX_STATE_DIR` (which is `/nix/var/nix/b` in the default configuration).
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Fix develop shells for derivations with escape codes [fj#991](https://git.lix.systems/lix-project/lix/issues/991) [cl/4154](https://gerrit.lix.systems/c/lix/+/4154) [cl/4155](https://gerrit.lix.systems/c/lix/+/4155)
ASCII control characters (including `\e`, used for ANSI escape codes) in derivation variables are now correctly escaped for `nix develop` and `nix print-dev-env`, instead of erroring.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- Fix nix develop for derivations that rejects dependencies with structured attrs [fj#997](https://git.lix.systems/lix-project/lix/issues/997) [cl/4182](https://gerrit.lix.systems/c/lix/+/4182) [cl/4214](https://gerrit.lix.systems/c/lix/+/4214)
For the sake of concision, we refer to `disallowedReferences` in what follows,
but all output checks were equally fixed:
`{dis,}allowed{References,Requisites}`.
Derivations can define *output checks* to reject unwanted dependencies, such as
interpreters like `bash` or compilers like `gcc`. This can be done in two ways:
* **Legacy style**: `disallowedReferences = [ ... ]` in the environment.
* **Structured attrs**: `outputChecks.<output>.disallowedReferences = [ ... ]`,
typically used in `__json`.
Only the structured form supports derivations with multiple outputs.
`nix develop` internally rewrites derivations to create development shells. It
relied on the legacy `disallowedReferences`, and failed to honor the structured
variant. This led to broken shells in cases where `bashInteractive` was
explicitly disallowed using structured output checks, e.g. `nix develop
nixpkgs#systemd` after the "bash-less NixOS" changes.
This fix teaches `nix develop` to respect structured output checks, restoring
support for such derivations.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- `nix-shell` default shell directory is not `/tmp` anymore for `$NIX_BUILD_TOP` [fj#940](https://git.lix.systems/lix-project/lix/issues/940)
Previously, Lix `nix-shell`s could exit non-zero status when `stdenv`'s `dumpVars` phase failed to write to `$NIX_BUILD_TOP/env-vars`, despite `dumpVars` being intended as a debugging aid.
This happens when `TMPDIR` is not set and defaults therefore to `/tmp`, resulting in a `/tmp/env-vars` global file that every `nix-shell` wants to write.
We fix this issue by reusing a pre-created, unique, and writable location, as the build top directory, avoiding shell exiting from write failures silently.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Fix unsigned overflow leading to out-of-band write in the NAR parser [cl/5537](https://gerrit.lix.systems/c/lix/+/5537)
The NAR parser contained an unsigned integer overflow that could be used by an
attacker to write arbitrary data to an unknown memory location and possibly
achieve code execution. A successful attack on the system-wide Lix daemon
could lead to privilege escalation to root. Any process that involves NAR
serialization could trigger this issue, including (but not limited to)
- local user interaction, whether the users are trusted or untrusted
- malicious substituters sending malformed NARs
- remote builders sending malformed build results
- remote daemons sending malformed inputs when requesting remote builds
Successful attacks using this bug require ASLR weakening of some sort, whether
by architecture constraints (e.g. on 32 bit systems, where little randomization
is possible) or system configuration (e.g. low ASLR entropy when loading
libraries), and millions of attempts. Local attacks can be mounted in less than
an hour. Remote builds typically require a fresh SSH connection for each build
and are thus less susceptible. Only one attempt can be made by substituters for
every build using substituters, they are thus not a likely vector for attacks.
At the time of writing, MITRE has not assigned this a CVE yet.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae), [Raito Bezarius](https://git.lix.systems/raito), [edef](https://github.com/edef1c), and [sandydoo](https://github.com/sandydoo) for this.
# Lix 2.93.3 (2025-07-22)
## Improvements
- `--keep-failed` chowns the build directory to the user that request the build [cl/3678](https://gerrit.lix.systems/c/lix/+/3678)
Running a build with `--keep-failed` now chowns the temporary directory from the
builder user and group to the user that request the build if the build came from
a local user connected to the daemon. This makes inspecting failed derivations a
lot easier. On Linux the build directory made visible to the user will not be in
the same path as it was in the sandbox and continuing builds will usually break.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
# Lix 2.93.2 (2025-06-30)
## Fixes
- Revert CVE-2025-52992 failed mitigation [fj#883](https://git.lix.systems/lix-project/lix/issues/883) [fj#887](https://git.lix.systems/lix-project/lix/issues/887) [cl/3444](https://gerrit.lix.systems/c/lix/+/3444) [cl/3528](https://gerrit.lix.systems/c/lix/+/3528)
Following the initial mitigation of **CVE-2025-52992** in `cl/3444`, we
received reports of **unexpected deletion of in-use store paths**.
Upon investigation, we found that the patch did **not correctly cancel all
automatic deleters**, resulting in potentially critical path loss during normal
operation.
Given the severity and time-sensitive nature of the situation ([see incident
report](https://lix.systems/blog/2025-06-27-lix-critical-bug/)), we evaluated
possible options to repair the behavior safely. However, we concluded that a
rushed fix would either
* **Overdelete**, i.e. breaking running systems, or,
* **Underdelete**, effectively **reopening CVE-2025-52992** while leaving
orphaned paths behind.
As **CVE-2025-52992 has no known exploit vector**, and correctness is critical
in the Lix project, we have **fully reverted the previous mitigations**.
The affected patches (`cl/3444`) have been rolled back for the time being.
Moving forward, the Lix team will rework this code path in a **long-term,
correctness-first fix** on the main branch. We will explore backporting it to
stable channels once its safety is assured.
We are deeply sorry for the stability incident and the Lix team remain
available for assisting you in recovering your systems.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Fallback to safe temp dir when build-dir is unwritable [fj#876](https://git.lix.systems/lix-project/lix/issues/876) [cl/3501](https://gerrit.lix.systems/c/lix/+/3501)
Non-daemon builds started failing with a permission error after introducing the `build-dir` option:
```
$ nix build --store ~/scratch nixpkgs#hello --rebuild
error: creating directory '/nix/var/nix/builds/nix-build-hello-2.12.2.drv-0': Permission denied
```
This happens because:
1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
2. The user lacks permissions for that path.
We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
**Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
# Lix 2.93.1 (2025-06-23)
## Breaking Changes
- Fixed output derivations can be run using `pasta` network isolation [fj#285](https://git.lix.systems/lix-project/lix/issues/285) [cl/3442](https://gerrit.lix.systems/c/lix/+/3442)
Fixed output derivations traditionally run in the host network namespace.
On Linux this allows such derivations to communicate with other sandboxes
or the host using the abstract Unix domains socket namespace; this hasn't
been unproblematic in the past and has been used in two distinct exploits
to break out of the sandbox. For this reason fixed output derivations can
now run in a network namespace (provided by [`pasta`]), restricted to TCP
and UDP communication with the rest of the world. When enabled this could
be a breaking change and we classify it as such, even though we don't yet
enable or require such isolation by default. We may enforce this in later
releases of Lix once we have sufficient confidence that breakage is rare.
[`pasta`]: https://passt.top/
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [puck](https://git.lix.systems/puck) for this.
## Fixes
- Always clean up scratch paths after derivations failed to build [cl/3444](https://gerrit.lix.systems/c/lix/+/3444)
Previously, scratch paths created during builds were not always cleaned up if
the derivation failed, potentially leaving behind unnecessary temporary files
or directories in the Nix store.
This fix ensures that such paths are consistently removed after a failed build,
improving Nix store hygiene, hardening Lix against mis-reuse of failed builds
scratch paths.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- `build-dir` no longer defaults to `temp-dir` [cl/3443](https://gerrit.lix.systems/c/lix/+/3443)
The directory in which temporary build directories are created no longer defaults
to the value of the `temp-dir` setting to avoid builders making their directories
world-accessible. This behavior has been used to escape the build sandbox and can
cause build impurities even when not used maliciously. We now default to `builds`
in `NIX_STATE_DIR` (which is `/nix/var/nix/builds` in the default configuration).
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Remove reliance on Bash for remote stores via SSH [fj#830](https://git.lix.systems/lix-project/lix/issues/830) [fj#805](https://git.lix.systems/lix-project/lix/issues/805) [fj#304](https://git.lix.systems/lix-project/lix/issues/304) [cl/3159](https://gerrit.lix.systems/c/lix/+/3159)
The pre-flight `echo started` handshake -- added years ago to catch race conditions -- has been removed.
After removal of connection sharing in Lix 2.93, it required a Bash-compatible shell and a standard `echo`, so it failed on:
* builders protected by `ForceCommand` wrappers (e.g. `nix-remote-build`),
* BusyBox / initrd images with no Bash,
* hosts using non-POSIX shells such as Nushell.
The race the probe once addressed was tied to SSH connection-sharing -- since connection-sharing code has already been removed, the probe is now pointless.
Real connection or protocol errors are now left to SSH/Nix to report directly.
This is technically a breaking change if you had scripts that relied on the literal "started" which needs to be updated to rely on other signals, e.g., exit codes.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
## Miscellany
- Deprecation of CA derivations, dynamic derivations, and impure derivations [fj#815](https://git.lix.systems/lix-project/lix/issues/815)
Content-addressed derivations are now deprecated and slated for removal in Lix 2.94.
We're doing this because the CA derivation system has been a known cause of problems
and inconsistencies, is unmaintained, habitually makes improving the store code very
difficult (or blocks such improvements outright), and is beset by a number of design
flaws that in our opinion cannot be fixed without a full reimplementation from zero.
Dynamic derivations and impure derivations are built on the CA derivation framework,
and owing to this they too are deprecated and slated for removal in another release.
# Lix 2.93.0 (2025-05-09) # Lix 2.93.0 (2025-05-09)
doc/manual/src/release-notes/rl-2.94.md
-955
View File
@@ -1,955 +0,0 @@
# Lix 2.94 "Açaí na tigela" (2025-11-17)
# Lix 2.94.0 (2025-11-17)
## Breaking Changes
- Remove support for daemon protocols before 2.18 [fj#510](https://git.lix.systems/lix-project/lix/issues/510) [cl/3249](https://gerrit.lix.systems/c/lix/+/3249)
Support for daemon wire protocols belonging to Nix 2.17 or older have been
removed. This impacts clients connecting to the local daemon socket or any
remote builder configured using the `ssh-ng` protocol. Builders configured
with the `ssh` protocol are still accessible from clients such as Nix 2.3.
Additionally Lix will not be able to connect to an old daemon locally, and
remote build connections to old daemons is likewise limited to `ssh` urls.
We have decided to take this step because the old protocols are very badly
tested (if at all), maintenance overhead is high, and a number of problems
with their design makes it infeasible to remain backwards compatible while
we move Lix to a more modern RPC mechanism with better versioning support.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Remove impure derivations and dynamic derivations [fj#815](https://git.lix.systems/lix-project/lix/issues/815) [cl/3210](https://gerrit.lix.systems/c/lix/+/3210)
The `impure-derivations` and `dynamic-derivations` experimental feature have
been removed.
New impure or dynamic derivations cannot be created from this point forward, and
any such pre-existing store derivations canot be read or built any more.
Derivation outputs created by building such a derivation are still valid
until garbage collected; existing store derivations can only be garbage
collected.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- First argument to `--arg`/`--argstr` must be a valid Nix identifier [fj#496](https://git.lix.systems/lix-project/lix/issues/496)
The first argument to `--arg`/`--argstr` must be a valid Nix identifier, i.e.
`nix-build --arg config.allowUnfree true` is now rejected.
This is because that invocation is a false friend since it doesn't set
`{ config = { allowUnfree = true; }; }`, but `{ "config.allowUnfree" = true; }`.
The idea is to change the behavior to the latter in the long-term. For that,
non-identifiers started giving a warning since 2.92 and are now rejected to give people
who depend on that a chance to notice and potentially weigh in on the discussion.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- New cgroup delegation model [fj#537](https://git.lix.systems/lix-project/lix/issues/537) [fj#77](https://git.lix.systems/lix-project/lix/issues/77) [cl/3230](https://gerrit.lix.systems/c/lix/+/3230)
Builds using cgroups (i.e. `use-cgroups = true` and the experimental feature
`cgroups`) now always delegate a cgroup tree to the sandbox.
Compared to the original C++ Nix project, our delegation includes the
`subtree_control` file as well, which means that the sandbox can disable
certain controllers in its own cgroup tree.
This is a breaking change because this requires the Nix daemon to run with an
already delegated cgroup tree by the service manager.
## How to setup the cgroup tree with systemd?
systemd offers knobs to perform the required setup using:
```
[Service]
Delegate=yes
DelegateSubtree=supervisor
```
These directives are now included in our systemd packaging.
## What about using Nix as root without connecting to the daemon?
Builds run as `root` without connecting to the daemon relying on the cgroup
feature are now broken, i.e.
```console
# nix-build --use-cgroups --sandbox ... # will not work
```
Consider doing instead:
```console
# systemd-run --same-dir --wait -p Delegate=yes -p DelegateSubgroup=supervisor nix-build --use-cgroups ...
```
If you need to disable cgroups temporarily, remember that you can do
`NIX_CONF='include /etc/nix/nix.conf\nuse-cgroups = false' nix-build ...` or
`nix-build --no-use-cgroups ...`.
## What about other service managers than systemd?
systemd has a [documentation](https://systemd.io/CGROUP_DELEGATION/) on how to
handle cgroup delegation from service management perspective.
If your service manager adheres to systemd semantics, e.g. writing an extended
attribute `user.delegate=1` on the delegated cgroup tree directory and moving
the `nix-daemon` process inside a cgroup tree to respect the inner process
rule, then, the feature will work as well.
## Why is the cgroup feature still experimental?
While the cgroup feature unlocks many use cases, its behavior and integration (e.g. user experience), especially at scale on build farms or in multi-tenant environments, are not yet fully matured. There’s also potential for deeper systemd integration (e.g. using slices and scopes) that has not been fully explored.
To avoid locking in an unstable interface, we’re keeping the experimental flag until we have validated the feature across a broader range of scenarios, including but not limited to:
* Nix as root
* Hydra-style build farms
* Forgejo CI runners
* Shared remote builders
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [eldritch horrors](https://git.lix.systems/pennae), and [lheckemann](https://git.lix.systems/lheckemann) for this.
- Enable high compress ratio zstd compression by default for binary caches uploads [fj#945](https://git.lix.systems/lix-project/lix/issues/945) [cl/4503](https://gerrit.lix.systems/c/lix/+/4503)
The default compression method for binary cache uploads has been switched from
[`xz`](https://github.com/tukaani-project/xz) to
[`zstd`](https://github.com/facebook/zstd) to address performance and usability
issues related to modern hardware and high-speed connections.
## Why?
`xz` offers compression ratios but is single-threaded in our implementation and
very slow (~10-20 Mbps in our test), preventing full utilization of 100Mbps+
connections and significantly slowing decompression for end users.
Lix is a "compress once, decompress many" application: build farms can afford
to spend more time compressing to achieve a faster download transfer for the
end user. More importantly, it matters that all end users spend the least
amount of time decompressing.
## What about compression ratios?
`zstd` cannot achieve the same peaks as `xz`, nonetheless, `zstd` compression
level has been increased to level 12 by default to balance compression ratio
and performance.
## Synthetic test case data
* **xz** (default compression level) on a 4.4GB file: ~632MB (77s)
* **zstd** (level 12) on the same file: ~775MB (18s), 18% larger but 50% faster
* **zstd** (level 14): ~773MB (37s)
* **zstd** (level 16): ~735MB (66s)
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [Raito Bezarius](https://git.lix.systems/raito) for this.
- Repl debugger uses `--ignore-try` by default [lix#666](https://git.lix.systems/lix-project/lix/issues/666) [cl/3488](https://gerrit.lix.systems/c/lix/+/3488)
Previously, using the debugger meant that exceptions thrown in `builtins.tryEval` would trigger the debugger.
However, this caught nixpkgs initialization code, which is unhelpful in the majority of cases, so we changed the default.
To get the old behaviour, use `--no-ignore-try`.
```
$ nix repl --debugger --expr 'with import <nixpkgs> {}; pkgs.hello'
Lix 2.94.0-dev-pre20250625-9a59106
Type :? for help.
error: file 'nixpkgs-overlays' was not found in the Nix search path (add it using $NIX_PATH or -I)
This exception occurred in a 'tryEval' call. Use --ignore-try to skip these.
Added 13 variables.
nix-repl>
```
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Strings may now contain NUL bytes [cl/3968](https://gerrit.lix.systems/c/lix/+/3968)
Lix now allows strings to contain NUL bytes instead of silently truncating the
string before the first such byte. Notably NUL-bearing strings were allowed as
attribute names—even though the corresponding strings were not representable!—
leading to very surprising and incorrect behavior in corner cases, for example
```
nix-repl> builtins.fromJSON ''{"a": 1, "a\u0000b": 2}''
{
a = 1;
"ab" = 2;
}
nix-repl> builtins.attrNames (builtins.fromJSON ''{"a": 1, "a\u0000b": 2}'')
[
"a"
"a"
]
```
rather than the more correct but still with the terminal eating NUL on display
```
nix-repl> builtins.fromJSON ''{"a": 1, "a\u0000b": 2}''
{
a = 1;
"ab" = 2;
}
nix-repl> builtins.attrNames (builtins.fromJSON ''{"a": 1, "a\u0000b": 2}'')
[
"a"
"ab"
]
```
We consider this a breaking change since eval results *will* change if strings
with embedded NUL bytes were used, but we also consider the old behavior to be
not intentional (seeing how inconsistent it was) but merely fallout from a old
and misguided implementation decision to be worked around, not actually fixed.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Fixed output derivations can be run using `pasta` network isolation [fj#285](https://git.lix.systems/lix-project/lix/issues/285) [cl/3452](https://gerrit.lix.systems/c/lix/+/3452)
Fixed output derivations traditionally run in the host network namespace.
On Linux this allows such derivations to communicate with other sandboxes
or the host using the abstract Unix domains socket namespace; this hasn't
been unproblematic in the past and has been used in two distinct exploits
to break out of the sandbox. For this reason fixed output derivations can
now run in a network namespace (provided by [`pasta`]), restricted to TCP
and UDP communication with the rest of the world. When enabled this could
be a breaking change and we classify it as such, even though we don't yet
enable or require such isolation by default. We may enforce this in later
releases of Lix once we have sufficient confidence that breakage is rare.
[`pasta`]: https://passt.top/
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [puck](https://git.lix.systems/puck) for this.
- Function equality semantics are more consistent, but still bad [cl/4556](https://gerrit.lix.systems/c/lix/+/4556) [cl/4244](https://gerrit.lix.systems/c/lix/+/4244)
Lix has inherited a historic misfeature from CppNix in the form of pointer
equality checks built into the `==` operator. These checks were originally
meant to optimize comparison for large sets, but they have the unfortunate
side effect of producing unexpected results when sets containing functions
are compared. **Lix 2.93 and earlier** behave as shown in the repl session
```
Lix 2.93.3
Type :? for help.
nix-repl> f = x: x
Added f.
nix-repl> f == f
false
nix-repl> let s.f = f; in s.f == s.f
false
nix-repl> # however!
{ inherit f; } == { inherit f; }
true
nix-repl> [ f ] == [ f ]
true
nix-repl> # and, in another twist:
[ f ] == map f [ f ]
false
```
Nixpkgs relies on sets containing functions being comparable, so we cannot
simply deprecate this behavior. Due to changes to the object model used by
Lix ***all* comparisons above now evaluate to `true`**. This is considered
a breaking change because eval results may differ, but we also consider it
minor because the optimization is unsound (c.f. `let l = [NaN]; in l == l`
evaluates to `true` even though floating point `NaN` is incomparable). Lix
intends to remove this optimization altogether in the future, but until we
can do that we instead make it slightly less broken to allow other, *real*
optimizations. Function equality comparison remains **undefined behavior**
and should not be relied upon in Nixlang code that intends to be portable.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- `nix eval --write-to` has been removed [fj#974](https://git.lix.systems/lix-project/lix/issues/974) [fj#227](https://git.lix.systems/lix-project/lix/issues/227) [cl/4045](https://gerrit.lix.systems/c/lix/+/4045)
`nix eval --write-to` has been removed since it was underspecified, not widely
useful, and prone to security-sensitive misbehaviors. The feature was added in
Nix 2.4 purely for internal use in the build system. According to our research
it hasn't found any use outside of some distribution packaging scripts. Please
use structured outputs formats (such as JSON) instead as they have better type
fidelity, don't conflate attributes with paths, and are useful to other tools.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Remove the `parse-toml-timestamps` experimental feature
The `parse-toml-timestamps` experimental feature has been removed.
This feature used in‐band signalling to mark timestamps, making it
impossible to unambiguously parse TOML documents. It also exposed
implementation‐defined behaviour in the TOML specification that
changed in the toml11 parser library.
Any interface for parsing TOML timestamps suitable for future
stabilization would necessarily involve breaking changes, and there
is no evidence this experimental feature is being relied upon in the
wild, so it has been removed.
Many thanks to [Emily](https://git.lix.systems/emilazy) for this.
- Reject overflowing TOML integer literals [cl/3916](https://gerrit.lix.systems/c/lix/+/3916)
The toml11 library used by Lix was updated. The new
version aligns with the [TOML v1.0.0 specification’s
requirement](https://toml.io/en/v1.0.0#integer) to reject integer
literals that cannot be losslessly parsed. This means that code like
`builtins.fromTOML "v=0x8000000000000000"` will now produce an error
rather than silently saturating the integer result.
Many thanks to [Emily](https://git.lix.systems/emilazy) for this.
- uid-range depends on cgroups [cl/3230](https://gerrit.lix.systems/c/lix/+/3230)
`uid-range` builds now depends on `cgroups`, an experimental feature.
`uid-range` builds already depended upon `auto-allocate-uids`, another experimental feature.
The rationale for doing so is that `uid-range` provides a sandbox with many
UIDs, this is useful for re-mapping them into a nested namespace, e.g. a
container.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
## Features
- Add `inputs.self.submodules` flake attribute [fj#942](https://git.lix.systems/lix-project/lix/issues/942) [cl/3839](https://gerrit.lix.systems/c/lix/+/3839)
A port of <https://github.com/NixOS/nix/pull/12421> to Lix, which:
- adds a general `inputs.self` flake attribute that retroactively applies
configurations to a flake after it's been fetched, then triggers a refetch of
the flake with the new config.
- implements `inputs.self.submodules` that allows a flake to declare its need
for submodules, which are then fetched automatically with no need to pass
`?submodules=1` anywhere.
Many thanks to [Eelco Dolstra](https://github.com/edolstra) and [ورد](https://git.lix.systems/janw4ld) for this.
- Lix supports HTTP/3 behind `--http3` [fj#1033](https://git.lix.systems/lix-project/lix/issues/1033)
Lix now supports HTTP/3 for file transfers when the linked curl version
supports it.
By default, HTTP/3 is disabled notably due to performance issues reported in
mid-2024. [More details
here](https://daniel.haxx.se/blog/2024/06/10/http-3-in-curl-mid-2024/).
As of 2025-11-14, [NixOS official cache](https://cache.nixos.org) supports
HTTP/3 via Fastly. [More info
here](https://github.com/NixOS/infra/commit/157fa70e46afbd6338a32407be461fce05c57bf8).
To enable HTTP/3:
* Use `--http3` for individual transfers.
* Add `http3 = true` in your Nix configuration for permanent activation.
To disable it, use `--no-http3`.
**Note**:
* `--no-http2 --http3` will still enable both HTTP/2 and HTTP/3.
* `--http2 --http3` will prioritize HTTP/3 and fall back to HTTP/2 (and then
HTTP/1.1).
These are current CLI limitations. In the future, we plan to replace `--httpX`
options with `--max-http-version [1,2,3]` for easier version selection in Lix
transfers.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Add hyperlinks in attr set printing [cl/3790](https://gerrit.lix.systems/c/lix/+/3790)
The attribute set printer, such as is seen in `nix repl` or in type errors, now prints hyperlinks on each attribute name to its definition site if it is known.
Example: all of the attributes shown here are hyperlinks to the exact definition site of the attribute in question:
```
$ nix eval -f '<nixpkgs>' lib.licenses.mit
{ deprecated = false; free = true; fullName = "MIT License"; redistributable = true; shortName = "mit"; spdxId = "MIT"; url = "https://spdx.org/licenses/MIT.html"; }
```
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Experimental integer coercion in interpolated strings [cl/3198](https://gerrit.lix.systems/c/lix/+/3198)
Ever tried interpolating a port number in Lix and ended up with something like this?
```nix
"http://${config.network.host}:${builtins.toString config.network.port}/"
```
You're not alone. Thousands of Lix users suffer every day from excessive `builtins.toString` syndrome. It’s 2025, and we still have to cast integers to use them in strings.
To address this, Lix introduces the **`coerce-integers`** experimental feature. When enabled, interpolated integers within `"${...}"` are automatically coerced to strings. This allows writing:
```nix
"http://${config.network.host}:${config.network.port}/"
```
without additional conversion.
To enable the feature, you need to add `coerce-integers` to your set of experimental features.
### Stabilization criteria
The `coerce-integers` feature is experimental and limited strictly to string interpolation (`"${...}"`). Before stabilization, the following must hold:
1. **Interpolation-only**
Coercion must not occur outside interpolation. Expressions like `"" + 42` must continue to fail.
2. **Expectation that no explicit cast are being observed**
Cases observing explicit coercion (e.g., via `tryEval` gadget or similar) are expected not to be load-bearing in actual production code.
### Timeline for stabilization
If the feature proves safe and is widely adopted across typical usage (e.g., actual configurations in the wild turning on the flag, non-trivial out-of-tree projects using it), the experimental flag will be removed **after six months of active use or two Lix releases**, whichever is longer.
This avoids locking the feature in experimental status indefinitely, as happened with Flakes, while allowing time for validation and ecosystem integration.
### What about coercing floats or more?
Coercion beyond integers -- such as for floats or other types -- is **not planned**, even under an experimental flag. Questions like "what is the canonical string representation of a float?" involve subtle and context-dependent trade-offs. Without a robust and principled mechanism to define and audit such behavior, introducing broader coercion risks setting unintended and hard-to-reverse precedents. The scope of `coerce-integers` is intentionally narrow and will remain so.
In terms of outlook, a proposal like https://git.lix.systems/lix-project/lix/issues/835 could pave the way for a better solution.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [delroth](https://github.com/delroth), [eldritch horrors](https://git.lix.systems/pennae), and [winter](https://git.lix.systems/winter) for this.
- nix-eval-jobs: support `--no-instantiate` flag [fj#987](https://git.lix.systems/lix-project/lix/issues/987)
`nix-eval-jobs` now supports a flag called `--no-instantiate`. With this enabled,
no write operations on the eval store are performed. That means, only evaluation is
performed, but derivations (and their gcroots) aren't created.
Many thanks to [mic92](https://github.com/mic92) and [ma27](https://git.lix.systems/ma27) for this.
## Improvements
- Assess current profile generations pointers in `nix doctor` [cl/3108](https://gerrit.lix.systems/c/lix/+/3108)
Added a new check to `nix doctor` that verifies whether the current generation of
a Nix profile can be resolved. This helps users diagnose issues with broken or
misconfigured profile symlinks.
This helps determining if you have broken symlinks or misconfigured packaging.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Improved susbtituter query speed
The code used to query substituters for derivations has been rewritten slightly
to take advantage of our asynchronous runtime. Such queries run for every build
that could download from substituters and processes every derivation that isn't
yet present on the local system. Previously Lix would use `http-connections` to
limit query concurrency, even for modern caches that support HTTP/2 and have no
limit on how many queries can be run concurrently on one single connection. Lix
no longer does this, resulting in approximately 60% reduction in query time for
medium-sized closures (e.g. NixOS system closures) during testing, although the
exact number depends greatly on local network latency and generally improves as
latency increases. Unlike previously setting `http-connections` to `1` or other
low values no longer brings a massive penalty in query performance if the cache
in use by the querying system supports HTTP/2 (as e.g. `cache.nixos.org` does).
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Hitting Control-C twice always terminates Lix [cl/3574](https://gerrit.lix.systems/c/lix/+/3574)
Hitting Control-C or sending `SIGINT` to Lix now prints an informational message
if it is still running after on second, the second Control-C/`SIGINT` terminates
Lix immediately without waiting for any shutdown code to finish running. Lix did
not treat the second such event differently from first in the past; this made it
impossible to easily terminate running Lix processes that got stuck in e.g. very
expensive Nixlang code that never interacted with the store. We now terminate as
soon as the user hits Control-C again without waiting any more, to much the same
effect as putting Lix into the background and killing it immediately afterwards.
This means you can now more conveniently break out of stuck Nixlang evaluations:
```
❯ nix-instantiate --eval --expr 'let f = n: if n == 0 then 0 else f (n - 1) + f (n - 1); in f 32'
^CStill shutting down. Press ^C again to abort all operations immediately.
^C
❌130 ❯
```
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- `--keep-failed` chowns the build directory to the user that request the build
Running a build with `--keep-failed` now chowns the temporary directory from the
builder user and group to the user that request the build if the build came from
a local user connected to the daemon. This makes inspecting failed derivations a
lot easier. On Linux the build directory made visible to the user will not be in
the same path as it was in the sandbox and continuing builds will usually break.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Better debuggability on fixed-output hash mismatches
Fixed-output derivation hash mismatch error messages will now include the path that was
produced unexpectedly, and this path will be registered as valid even if `--check`
(`nix-store`, `nix-build`) or `--rebuild` (`nix build`) was passed. This makes comparing
the expected path with the obtained path easier, and is useful for debugging when
upstreams modify previously-published releases or when changes in fixed-output
derivations' dependencies affect their output unexpectedly.
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Add --raw flag to `nix-instantiate --eval` for unescaped output [gh#12119](https://github.com/NixOS/nix/pull/12119) [cl/2886](https://gerrit.lix.systems/c/lix/+/2886)
The `nix-instantiate --eval` command now supports a `--raw` flag. When used,
the result must be coercible to a string (as with `${...}`) and is printed
verbatim, without quotes or escaping.
Many thanks to [Martin Fischer](https://github.com/not-my-profile), [infinisil](https://github.com/infinisil), and [Raito Bezarius](https://git.lix.systems/raito) for this.
- Allow `nix store ls` to read nar listings from binary cache stores. [cl/3225](https://gerrit.lix.systems/c/lix/+/3225)
The `nix store ls` command now supports reading `.ls` nar listings from binary cache stores.
If a listing is detected for the store path being queried, the nar is no longer downloaded.
These nar listings are available in binary cache stores where the `write-nar-listing` option is
enabled, such as cache.nixos.org.
Many thanks to [Victor Fuentes](https://git.lix.systems/vlinkz) for this.
- show tree with references that lead to an output cycle [fj#551](https://git.lix.systems/lix-project/lix/issues/551)
When Lix determines a cyclic dependency between several outputs of a derivation,
it now displays which files in which outputs lead to an output cycle:
```
error: cycle detected in build of '/nix/store/gc5h2whz3rylpf34n99nswvqgkjkigmy-demo.drv' in the references of output 'bar' from output 'foo'.
Shown below are the files inside the outputs leading to the cycle:
/nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar
└───lib/libfoo: …stuffbefore /nix/store/h680k7k53rjl9p15g6h7kpym33250w0y-demo-baz andafter.…
→ /nix/store/h680k7k53rjl9p15g6h7kpym33250w0y-demo-baz
└───share/snenskek: …???? /nix/store/dm24c76p9y2mrvmwgpmi64rryw6x5qmm-demo-foo ....…
→ /nix/store/dm24c76p9y2mrvmwgpmi64rryw6x5qmm-demo-foo
└───bin/alarm: …textexttext/nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar abcabcabc.…
→ /nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar
```
Please note that showing the files and its contents while displaying the cycles only works
on Linux.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- Lix now enables parallel marking in boehm-gc [fj#983](https://git.lix.systems/lix-project/lix/issues/983) [cl/3880](https://gerrit.lix.systems/c/lix/+/3880)
This brings a fairly modest performance improvement (~38% for `nixpkgs search hello`) to evaluation, especially in scenarios that necessitate larger heap sizes.
Many thanks to [Eelco Dolstra](https://github.com/edolstra) and [Seth Flynn](https://git.lix.systems/getchoo) for this.
- `disallowedRequisites` now reports chains of disallowed requisites [fj#334](https://git.lix.systems/lix-project/lix/issues/334) [fj#626](https://git.lix.systems/lix-project/lix/issues/626) [gh#10877](https://github.com/NixOS/nix/issues/10877)
When a build fails because of [`disallowedRequisites`](@docroot@/language/advanced-attributes.md#adv-attr-disallowedRequisites), the error message now includes the chain of references that led to the failure. This makes it easier to see in which derivations the chain can be broken, to resolve the problem.
Example:
```
$ nix-build -A hello
error: output '/nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2' is not allowed to refer to the following paths:
/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-glibc-2.40-66
Shown below are chains that lead to the forbidden path(s).
/nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2
└───/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-glibc-2.40-66
```
Many thanks to [ma27](https://git.lix.systems/ma27) and [Robert Hensing](https://github.com/roberth) for this.
- Stack traces now summarize involved derivations at the bottom [cl/4493](https://gerrit.lix.systems/c/lix/+/4493)
When evaluation errors and a stack trace is printed,
For example, if I add Nheko to a NixOS `environment.systemPackages` without adding `olm-3.2.16` `nixpkgs.config.permittedInsecurePackages`, then without `--show-trace`, I previously got this:
```
error:
… while calling the 'head' builtin
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/attrsets.nix:1701:13:
1700| if length values == 1 || pred here (elemAt values 1) (head values) then
1701| head values
| ^
1702| else
… while evaluating the attribute 'value'
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/modules.nix:1118:7:
1117| // {
1118| value = addErrorContext "while evaluating the option `${showOption loc}':" value;
| ^
1119| inherit (res.defsFinal') highestPrio;
(stack trace truncated; use '--show-trace' to show the full trace)
error: Package ‘olm-3.2.16’ in /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/pkgs/by-name/ol/olm/package.nix:37 is marked as insecure, refusing to evaluate.
< -snip the whole explanation about olm's CVEs- >
```
This doesn't tell me anything about where `olm-3.2.16` came from.
With `--show-trace`, there's 1 155 lines to sift through, but does contain lines like "while evaluating derivation 'nheko-0.12.1'".
With this change, those lines are summarized and collected at the bottom, regardless of `--show-trace`:
```
error:
… while calling the 'head' builtin
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/attrsets.nix:1701:13:
1700| if length values == 1 || pred here (elemAt values 1) (head values) then
1701| head values
| ^
1702| else
… while evaluating the attribute 'value'
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/modules.nix:1118:7:
1117| // {
1118| value = addErrorContext "while evaluating the option `${showOption loc}':" value;
| ^
1119| inherit (res.defsFinal') highestPrio;
(stack trace truncated; use '--show-trace' to show the full trace)
error: Package ‘olm-3.2.16’ in /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/pkgs/by-name/ol/olm/package.nix:37 is marked as insecure, refusing to evaluate.
< -snip the whole explanation about olm's CVEs- >
note: trace involved the following derivations:
derivation 'etc'
derivation 'dbus-1'
derivation 'system-path'
derivation 'nheko-0.12.1'
derivation 'mtxclient-0.10.1'
```
Now we finally know that olm was evaluated because of Nheko, without sifting through *thousands* of lines of error message.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- Symbols reuses once-allocated Value to reduce garbage collected allocations [cl/3308](https://gerrit.lix.systems/c/lix/+/3308) [cl/3300](https://gerrit.lix.systems/c/lix/+/3300) [cl/3314](https://gerrit.lix.systems/c/lix/+/3314) [cl/3310](https://gerrit.lix.systems/c/lix/+/3310) [cl/3312](https://gerrit.lix.systems/c/lix/+/3312) [cl/3313](https://gerrit.lix.systems/c/lix/+/3313)
In the Lix evaluator, **symbols** represent immutable strings, like those used
for attribute names.
In evaluator design, such strings are typically [**interned**](https://en.wikipedia.org/wiki/String_interning), stored uniquely
to save memory, and Lix inherits this approach from the original C++ codebase.
However, some builtins, like `builtins.attrNames`, must return a `Value` type
that can represent any Nix value (strings, integers, lists, etc.).
Before this change, these builtins would create lists of `Value` objects by
allocating them through the garbage collector, copying the symbol’s string
content each time.
This allocation is unnecessary if the interned symbols themselves also hold a
`Value` representation allocated outside the garbage collector, since these
live for the full duration of evaluation.
As a result, this reduces the number of allocations, leading to:
* A significant drop in maximum [resident set memory](https://en.wikipedia.org/wiki/Resident_set_size) (RSS), with some large-scale
tests showing up to 11% (about 500 MiB) savings in large colmena deployments.
* A slight decrease in CPU usage during Nix evaluations.
This change is inspired by https://github.com/NixOS/nix/pull/13258 but the approach is different.
**Note** : [`xokdvium`](https://github.com/xokdvium) is the rightful author of https://gerrit.lix.systems/c/lix/+/3300 and the credit was missed on our end during the development process. We are deeply sorry for this mistake.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [eldritch horrors](https://git.lix.systems/pennae), [Tom Hubrecht](https://git.lix.systems/tom-hubrecht), [xokdvium](https://github.com/xokdvium), and [NaN-git](https://github.com/NaN-git) for this.
## Fixes
- `build-dir` no longer defaults to `temp-dir` [cl/3453](https://gerrit.lix.systems/c/lix/+/3453)
The directory in which temporary build directories are created no longer defaults
to the value of the `temp-dir` setting to avoid builders making their directories
world-accessible. This behavior has been used to escape the build sandbox and can
cause build impurities even when not used maliciously. We now default to `builds`
in `NIX_STATE_DIR` (which is `/nix/var/nix/b` in the default configuration).
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Global certificate authorities are copied inside the builder's environment [gh#12698](https://github.com/NixOS/nix/issues/12698) [fj#885](https://git.lix.systems/lix-project/lix/issues/885) [cl/3765](https://gerrit.lix.systems/c/lix/+/3765)
Previously, CA certificates were only installed at
`/etc/ssl/certs/ca-certificates.crt` for sandboxed builds on Linux.
This setup was insufficient in light of recent changes in `nixpkgs`, which now
enforce HTTPS usage for `fetchurl`, even for fixed-output derivations, to
mitigate confidentiality risks such as `netrc` or credentials leakage.
`nixpkgs` still make use of a special package called `cacerts` which contains a
copy of the CA certificates maintained by Nixpkgs and added as a reference for
TLS-enabled fetchers.
As a result, having a consistent and trusted certificate authority in all
builder environments is becoming more essential.
On `nix-darwin`, the `NIX_SSL_CERT_FILE` environment variable is always
explicitly defined, but it is ignored by the sandbox setup.
Simultaneously, Nix evaluates and propagates impure environment variables via
`lib.proxyImpureEnvVars`, meaning that if `NIX_SSL_CERT_FILE` is set (which
influences the default value for `ssl-cert-file`), it will be forwarded
unchanged into the builder environment.
However, on Linux, Nix also *copies* the CA file into the sandbox, creating a
discrepancy between the value of `NIX_SSL_CERT_FILE` and the actual trusted
certificate path used during the build.
This divergence caused confusion and was partially addressed by attempts to
whitelist the CA path in the Darwin sandbox (see cl/2906), but that approach
involved a non-trivial path canonicalization step and is not as general as this one.
To address this properly, we now emit a warning and override
`NIX_SSL_CERT_FILE` inside the builder, explicitly pointing it to the CA file
copied into the sandbox.
This eliminates ambiguity between `NIX_SSL_CERT_FILE`
and `ssl-cert-file`, ensuring consistent trust anchors across platforms.
This warning might become a hard error as we figure out what to do regarding
`lib.proxyImpureEnvVars` in nixpkgs.
The behavior has been verified across sandboxed and unsandboxed builds on both
Linux and Darwin.
As a consequence of this change, approximately 500 KB of CA certificate data is
now unconditionally copied into the build directory for fixed-output
derivations.
While this ensures consistent trust verification without having to restart the
daemon after system upgrades, it may introduce a slight overhead in build
performance. At present, no optimizations have been implemented to avoid this
copy, but if this overhead proves noticeable in your workflows, please open an
issue so we can evaluate and possibly implement different strategies to render
trust anchors visible.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [Emily](https://git.lix.systems/emilazy) for this.
- libstore: exponential backoff for downloads [lix#932](https://git.lix.systems/lix-project/lix/issues/932) [cl/3856](https://gerrit.lix.systems/c/lix/+/3856)
The connection timeout when downloading from e.g. a binary cache is exponentially
increased per failure. The option `connect-timeout` is now an alias to `max-connect-timeout`
which is the maximum value for a timeout. The start value is controlled
by `initial-connect-timeout` which is `5` by default.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- Fix develop shells for derivations with escape codes [fj#991](https://git.lix.systems/lix-project/lix/issues/991) [cl/4154](https://gerrit.lix.systems/c/lix/+/4154) [cl/4155](https://gerrit.lix.systems/c/lix/+/4155)
ASCII control characters (including `\e`, used for ANSI escape codes) in derivation variables are now correctly escaped for `nix develop` and `nix print-dev-env`, instead of erroring.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- nix-store --delete: always remove obsolete hardlinks [cl/3188](https://gerrit.lix.systems/c/lix/+/3188)
Deleting specific paths using `nix-store --delete` or `nix store
delete` previously did not delete hard links created by `nix-store
--optimise` even if they became obsolete, unless _all_ of the given
paths were deleted successfully. Now, hard links are always cleaned
up, even if some of the given paths could not be deleted.
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Report GC statistics correctly [cl/3188](https://gerrit.lix.systems/c/lix/+/3188)
Deleting specific paths using `nix-store --delete` or `nix store delete` previously did
not report statistics correctly when some of the paths could not be deleted, even if
others were deleted:
```
$ nix store delete /nix/store/9bwryidal9q3g91cjm6xschfn4ikd82q-hello-2.12.1 --delete-closure -v
finding garbage collector roots...
deleting '/nix/store/9bwryidal9q3g91cjm6xschfn4ikd82q-hello-2.12.1'
0 store paths deleted, 0.00 MiB freed
error: Cannot delete some of the given paths because they are still alive. Paths not deleted:
k9bxzr1l92r5y6mihrkbpbr3fmc8qszx-libidn2-2.3.8
mbx9ii53lzjlrsnlrfmzpwm33ynljwdn-libunistring-1.3
rf8hcy6bldxdqc0g6q1dcka1vh47x69s-xgcc-14.2.1.20250322-libgcc
vbrdc5wgzn0w1zdp10xd2favkjn5fk7y-glibc-2.40-66
To find out why, use nix-store --query --roots and nix-store --query --referrers.
```
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Fallback to safe temp dir when build-dir is unwritable [fj#876](https://git.lix.systems/lix-project/lix/issues/876) [cl/3501](https://gerrit.lix.systems/c/lix/+/3501)
Non-daemon builds started failing with a permission error after introducing the `build-dir` option:
```
$ nix build --store ~/scratch nixpkgs#hello --rebuild
error: creating directory '/nix/var/nix/builds/nix-build-hello-2.12.2.drv-0': Permission denied
```
This happens because:
1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
2. The user lacks permissions for that path.
We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
**Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Parse overflowing JSON number literals as floating‐point [cl/3919](https://gerrit.lix.systems/c/lix/+/3919)
Previously, `builtins.fromJSON "-9223372036854775809"` would
return a floating‐point number, while `builtins.fromJSON
"9223372036854775808"` would cause an evaluation error. This was
introduced with the banning of integer overflow in Lix 2.91; previously
the latter would result in C++ undefined behaviour. These cases are
now treated consistently with JSON’s model of a single numeric type,
and JSON number literals that do not fit in a Nix‐language integer
will be parsed as floating‐point numbers.
Many thanks to [Emily](https://git.lix.systems/emilazy) for this.
- Fix handling of OSC codes in terminal output [fj#160](https://git.lix.systems/lix-project/lix/issues/160) [cl/3143](https://gerrit.lix.systems/c/lix/+/3143)
OSC codes in terminal output are now handled correctly, where OSC 8 (hyperlink) is preserved any
time color codes are allowed and all other OSC codes are stripped out. This applies not only to
output from build commands but also to rendered documentation in the REPL.
Many thanks to [lilyball](https://git.lix.systems/lilyball) for this.
- Fix nix develop for derivations that rejects dependencies with structured attrs [fj#997](https://git.lix.systems/lix-project/lix/issues/997) [cl/4182](https://gerrit.lix.systems/c/lix/+/4182)
For the sake of concision, we refer to `disallowedReferences` in what follows,
but all output checks were equally fixed:
`{dis,}allowed{References,Requisites}`.
Derivations can define *output checks* to reject unwanted dependencies, such as
interpreters like `bash` or compilers like `gcc`. This can be done in two ways:
* **Legacy style**: `disallowedReferences = [ ... ]` in the environment.
* **Structured attrs**: `outputChecks.<output>.disallowedReferences = [ ... ]`,
typically used in `__json`.
Only the structured form supports derivations with multiple outputs.
`nix develop` internally rewrites derivations to create development shells. It
relied on the legacy `disallowedReferences`, and failed to honor the structured
variant. This led to broken shells in cases where `bashInteractive` was
explicitly disallowed using structured output checks, e.g. `nix develop
nixpkgs#systemd` after the "bash-less NixOS" changes.
This fix teaches `nix develop` to respect structured output checks, restoring
support for such derivations.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- nix-eval-jobs: retain NIX_PATH [cl/3859](https://gerrit.lix.systems/c/lix/+/3859)
`nix-eval-jobs` doesn't clear the `NIX_PATH` from the environment anymore. This matches the behavior
of [upstream version `2.30`](https://github.com/nix-community/nix-eval-jobs/releases/tag/v2.30.0).
Many thanks to [ma27](https://git.lix.systems/ma27) and [mic92](https://github.com/mic92) for this.
- Remove reliance on Bash for remote stores via SSH [fj#830](https://git.lix.systems/lix-project/lix/issues/830) [fj#805](https://git.lix.systems/lix-project/lix/issues/805) [fj#304](https://git.lix.systems/lix-project/lix/issues/304) [cl/3159](https://gerrit.lix.systems/c/lix/+/3159)
The pre-flight `echo started` handshake -- added years ago to catch race conditions -- has been removed.
After removal of connection sharing in Lix 2.93, it required a Bash-compatible shell and a standard `echo`, so it failed on:
* builders protected by `ForceCommand` wrappers (e.g. `nix-remote-build`),
* BusyBox / initrd images with no Bash,
* hosts using non-POSIX shells such as Nushell.
The race the probe once addressed was tied to SSH connection-sharing -- since connection-sharing code has already been removed, the probe is now pointless.
Real connection or protocol errors are now left to SSH/Nix to report directly.
This is technically a breaking change if you had scripts that relied on the literal "started" which needs to be updated to rely on other signals, e.g., exit codes.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- repl-overlays now work in the debugger for flakes [fj#777](https://git.lix.systems/lix-project/lix/issues/777) [cl/3398](https://gerrit.lix.systems/c/lix/+/3398)
Due to a bug, it was previously not possible to use the debugger on flakes with repl-overlays, or with pure evaluation in general:
```
$ nix repl --pure-eval
Lix 2.94.0-dev-pre20250617-87d99da
Type :? for help.
Loading 'repl-overlays'...
error: access to absolute path '/Users/jade/.config/nix/repl.nix' is forbidden in pure eval mode (use '--impure' to override)
```
This is now fixed.
The contents of the repl-overlays file itself (i.e. most typically the top level lambda in it) will be evaluated in impure mode.
It may be necessary to use `builtins.seq` to force the impure operations to happen first if one wants to do impure operations inside a repl-overlays file in pure evaluation mode.
Many thanks to [jade](https://git.lix.systems/jade) for this.
- `nix-shell` default shell directory is not `/tmp` anymore for `$NIX_BUILD_TOP` [fj#940](https://git.lix.systems/lix-project/lix/issues/940)
Previously, Lix `nix-shell`s could exit non-zero status when `stdenv`'s `dumpVars` phase failed to write to `$NIX_BUILD_TOP/env-vars`, despite `dumpVars` being intended as a debugging aid.
This happens when `TMPDIR` is not set and defaults therefore to `/tmp`, resulting in a `/tmp/env-vars` global file that every `nix-shell` wants to write.
We fix this issue by reusing a pre-created, unique, and writable location, as the build top directory, avoiding shell exiting from write failures silently.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- libstore/binary-cache-store: don't cache narinfo on nix copy, remove negative entry [cl/3789](https://gerrit.lix.systems/c/lix/+/3789)
When using e.g. [Snix's nar-bridge](https://snix.dev/docs/components/overview/#nar-bridge) via
an `http`-store, Lix would create cache entries with a wrong URL to the NAR when uploading
a store-path.
This caused hard build failures for Hydra.
Lix doesn't create these entries on upload anymore. Instead, it only removes negative cache entries.
The cache entry for a narinfo is now created the first time, Lix queries the cache
for the previously uploaded store-path again.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- Lix libraries can now be linked statically [fj#789](https://git.lix.systems/lix-project/lix/issues/789) [cl/3775](https://gerrit.lix.systems/c/lix/+/3775) [cl/3778](https://gerrit.lix.systems/c/lix/+/3778)
Previously the pkg-config files distributed with Lix were only suitable for dynamic linkage, causing "undefined reference to…" linker errors when trying to link statically.
Private dependency information has now been added to make static linkage work as expected without user intervention.
In addition, relevant static libraries are now prelinked to avoid strange failures due to missing static initializers.
Many thanks to [alois31](https://git.lix.systems/alois31) for this.
- add description to zsh completions [fj#910](https://git.lix.systems/lix-project/lix/issues/910) [cl/3632](https://gerrit.lix.systems/c/lix/+/3632)
Emit descriptions when completing args in zsh completions. This uses the descriptions we already
provided in NIX\_GET\_COMPLETIONS.
Many thanks to [matthewbauer](https://github.com/matthewbauer) for this.
## Miscellany
- Deprecation of CA derivations, dynamic derivations, and impure derivations [fj#815](https://git.lix.systems/lix-project/lix/issues/815)
Content-addressed derivations are now deprecated and slated for removal in Lix 2.94.
We're doing this because the CA derivation system has been a known cause of problems
and inconsistencies, is unmaintained, habitually makes improving the store code very
difficult (or blocks such improvements outright), and is beset by a number of design
flaws that in our opinion cannot be fixed without a full reimplementation from zero.
Dynamic derivations and impure derivations are built on the CA derivation framework,
and owing to this they too are deprecated and slated for removal in another release.
doc/manual/src/release-notes/rl-2.95.md
-546
View File
@@ -1,546 +0,0 @@
# Lix 2.95 "Kakigōri" (2026-03-13)
# Lix 2.95.0 (2026-03-13)
## Breaking Changes
- Deprecate shadowing internal files through the Nix search path [lix#998](https://git.lix.systems/lix-project/lix/issues/998) [cl/4632](https://gerrit.lix.systems/c/lix/+/4632)
As Lix uses the path `<nix/fetchurl.nix>` for bootstrapping purposes, the ability to shadow it by adding `nix=/some/path` (or `/other/path` that contains a `nix` directory) to the search path is not desirable.
To alleviate potential issues, Lix now emits a warning when the Nix search path contains potential shadows for internal files, which will be changed to an error in a future release.
The warning can be disabled by enabling the deprecated feature `nix-path-shadow`.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- More deprecated features [cl/2092](https://gerrit.lix.systems/c/lix/+/2092) [cl/2310](https://gerrit.lix.systems/c/lix/+/2310) [cl/2311](https://gerrit.lix.systems/c/lix/+/2311) [cl/4638](https://gerrit.lix.systems/c/lix/+/4638) [cl/4652](https://gerrit.lix.systems/c/lix/+/4652) [cl/4764](https://gerrit.lix.systems/c/lix/+/4764)
This release cycle features a new batch of deprecated (anti-)features.
You can opt in into the old behavior with `--extra-deprecated-features` or any equivalent configuration option.
- `broken-string-indentation` indented strings (those starting with `''`) might produce unintended results due to how the whitespace stripping is done. Those cases will now warn the user.
- `broken-string-escape` "escaped" characters without a properly defined escape sequence evaluate to "themselves". This is in most cases unintended behaviour, both for writing regexes, and using legacy or uncommon escape sequences like `\f`. The user will now be warned, if those are present.
- `floating-without-zero` so far, one was able to declare a float using something like `.123`. This can cause confusion about accessing attributes. Floating point numbers must now always include the leading zero, i.e. `0.123`
- `rec-set-merges` Attribute sets like `{ foo = {}; foo.bar = 42;}` implicitly merge at parse time, however if one of them is marked as recursive but not the others then the recursive attribute may get lost (order-dependent). Therefore, merging attrs with mixed-`rec` is now forbidden.
- `rec-set-dynamic-attrs` Dynamic attributes have weird semantics in the presence of recursive attrsets (they evaluate *after* the rest of the set). This is now forbidden.
- `or-as-identifier` `or` as an identifier has always been weird since the `or` (almost-)keyword has been introduced. We are deprecating the backcompat hacks from the early days of Nix in favor of making `or` a full and proper keyword.
- `tokens-no-whitespace` Function applications without space around the arguments like `0a`, `0.00.0` or `foo"1"2` are now forbidden. The same applies to list elements. The primary reason for this deprecation is to remove foot guns around surprising tokenization rules regarding number literals, but this will also free up some syntax for other purposes (e.g. `r""` strings) for reuse at some point in the future.
- `shadow-internal-symbols` has been expanded to also forbid shadowing `null`, `true` and `false`.
- `ancient-let` deprecation has been turned into a full parser error instead of a warning.
- `rec-set-overrides` deprecation has been turned into a full parser error instead of a warning.
Many thanks to [piegames](https://git.lix.systems/piegames), [rootile (Rutile)](https://git.lix.systems/rootile), and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Move `/root/.cache/nix` to `/var/cache/nix` by default [lix#634](https://git.lix.systems/lix-project/lix/issues/634) [cl/4671](https://gerrit.lix.systems/c/lix/+/4671)
By default, Lix attempts to locate a cache directory for its operations (such
as the narinfo cache) by checking the value of `$XDG_CACHE_DIR`.
However, since the Nix daemon is a system service, using `$XDG_CACHE_DIR` is
not typical in this context.
To address this, systemd provides a better solution. Specifically, when
`CacheDirectory=` is set in the `[Service]` section of a systemd unit, it
automatically sets the `$CACHE_DIRECTORY` environment variable and systemd will
manage that cache directory for us.
Now, our systemd unit includes `CacheDirectory=nix`, which sets the
`$CACHE_DIRECTORY` and takes precedence over `$XDG_CACHE_DIR`.
If the daemon is run under user units, systemd will automatically set
`$XDG_CACHE_DIR`.
If neither of these variables is set, Lix falls back to its default behavior.
By default, Lix will try to find a cache directory for its various operations
(e.g. narinfo cache) by looking into `$XDG_CACHE_DIR`.
In summary, what was stored in `/root/.cache/nix` is now moved to
`/var/cache/nix/nix`.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Remove `fetch-closure` experimental feature [lix#1010](https://git.lix.systems/lix-project/lix/issues/1010) [cl/4595](https://gerrit.lix.systems/c/lix/+/4595)
The `fetch-closure` experimental feature has been removed.
Outside of allowing the user to import closure from binary cache,
`fetchClosure` also allowed you to do the following:
* rewrite non-CA path to CA
* reject non-CA paths at fetching time
* reject CA paths at fetching time
Some people are using those mechanism to prevent users from having to build any
package and force going via the declared cache or as a way to use ancient/old
software without paying the evaluation cost of a second nixpkgs.
Both use cases are somewhat of an antipattern in Nix semantics. If the user
cannot fetch a program directly via the substituter mechanism and fall back to
local build, this is a feature *and* a misconfiguration. If the user cannot build
certain derivations because they are too expensive, the build directives should
pass `-j0` or similar.
As for the second usecase, there's a different way to do it that also allows to
have a way to reproduce the paths that are hardcoded in that file, perform
`import (fetchurl "https://my-cache/${hashparts storepath}.drv")` rather, i.e.
an IFD to a possibly well known name. The backend can generate them on the fly
or once, and possess stable names.
Finally, as for the non-CA → CA features, Lix removed ca-derivations.
fetchClosure offers ca-derivations-like features which suffers from similar
shortcomings albeit lessened. It only follows that we should deprecate
and remove these capabilities.
Many thanks to [just1602](https://git.lix.systems/just1602) for this.
## Features
- `nix store add-path` now supports references [cl/5205](https://gerrit.lix.systems/c/lix/+/5205)
Lix supports two categories of hashes in store paths: input-addressed and output-addressed.
Currently, in Nix language, there is no way to produce output-addressed paths with references, as fixed-output derivations forbid references.
However, the Nix store actually *supports* references in output-addressed paths.
This is very useful for importing build products created outside of Lix that reference dependency store paths since such build products have no associated derivation so don't make any sense to input-address.
Previously, output-addressed paths with references could only be created by writing a custom client to the rather-baroque Nix daemon protocol; now it's available in the CLI.
Using `nix store add-path --references-list-json REFS_LIST_FILE SOME_PATH` with a JSON list of string store paths, you can now create such paths with the Lix CLI.
They may be consumed from Nix language using something like `builtins.storePath` or the following which also works in pure evaluation mode:
```nix
# Hack from https://git.lix.systems/lix-project/lix/issues/402#issuecomment-5889
path:
builtins.appendContext path {
${path} = {
path = true;
};
}
```
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Add `builtins.warn` for emitting warnings from Nix code [cl/2248](https://gerrit.lix.systems/c/lix/+/2248)
Lix now has a builtin function for emitting warnings.
Like `builtins.trace`, it takes two arguments: the message to emit, and the expression to return.
_Unlike_ `builtins.trace`, `builtins.warn` requires the first argument — the message — to be a string.
In the future we may extend `builtins.warn` to accept a more structured API.
To go along with this, we also have two new config settings:
- [`debugger-on-warn`](@docroot@/command-ref/conf-file.md#conf-debugger-on-warn), which, when used with `--debugger`, makes `builtins.warn` also function like [`builtins.break`](@docroot@/language/builtins.md#builtins-break).
- [`abort-on-warn`](@docroot@/command-ref/conf-file.md#conf-abort-on-warn), which aborts evaluation entirely after the warning is emitted.
Many thanks to [Emilia Bopp](https://git.lix.systems/milibopp) and [Qyriad](https://git.lix.systems/Qyriad) for this.
- `keep-env-derivations` is now supported for nix3 CLI (`nix profile`) [lix#1095](https://git.lix.systems/lix-project/lix/issues/1095) [cl/5332](https://gerrit.lix.systems/c/lix/+/5332)
The `keep-env-derivations` feature is now available for `nix profile`. This allows users to prevent the garbage collection of derivations used to install a profile, even when `keep-derivations = false` (set to `true` by default).
Previously, `nix-env` supported this feature, but `nix profile` **never** did. This caused issues when garbage collection removed the associated `.drv` files, which are required, for example, by vulnerability management tools (e.g. [vulnix](https://github.com/nix-community/vulnix)) for proper operation.
This issue has now been resolved.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Make `log-format` a setting [cl/4686](https://gerrit.lix.systems/c/lix/+/4686)
The [`--log-format` CLI option](@docroot@/command-ref/opt-common.md#opt-log-format) can now be set in [`nix.conf`](@docroot@/command-ref/conf-file.md#conf-log-format)!
For example, you can now persistently enable the `multiline-with-logs` log format [added in Lix 2.91](@docroot@/release-notes/rl-2.91.md) by adding the following to your `nix.conf`:
```conf
log-format = multiline-with-logs
```
Or the equivalent in a NixOS configuration:
```nix
{
nix.settings.log-format = "multiline-with-logs";
}
```
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- Allow remote builders to be configured using TOML [cl/4533](https://gerrit.lix.systems/c/lix/+/4533)
Lix now supports configuring remote builders using a TOML file instead of the old, very cursed and incomprehensible format.
This comes with not only a human-understandable file, but also with better messages and error reports on misconfiguration.
A more detailed Documentation can be found on the [distributed-builds](@docroot@/advanced-topics/distributed-builds.md) documentation page.
Many thanks to [rootile (Rutile)](https://git.lix.systems/rootile) and [Qyriad](https://git.lix.systems/Qyriad) for this.
- Emit warnings when encountering IFD with `warn-import-from-derivation` [nix#13279](https://github.com/NixOS/nix/pull/13279) [cl/3879](https://gerrit.lix.systems/c/lix/+/3879)
Instead of only being able to toggle the use of [Import from
Derivation](https://nix.dev/manual/nix/stable/language/import-from-derivation) with
`allow-import-from-derivation`, Lix is now able to warn users whenever IFD is encountered with
`warn-import-from-derivation`.
Many thanks to [Seth Flynn](https://git.lix.systems/getchoo), [gustavderdrache](https://github.com/gustavderdrache), and [Eelco Dolstra](https://github.com/edolstra) for this.
## Improvements
- Collect Flakes untrusted settings into one prompt [lix#682](https://git.lix.systems/lix-project/lix/issues/682) [cl/2921](https://gerrit.lix.systems/c/lix/+/2921)
When working with Flakes containing untrusted settings, a prompt is shown for each setting, asking whether to vet or approve it. This looks like:
```
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all)
```
In Flakes with a large number of settings to approve or reject, this process can become tedious as each option must be handled individually.
To address this, all untrusted settings are now consolidated into a single prompt: allowing for bulk acceptance permanently or not, rejection, or detailed review. For example:
### Scrutiny scenario
```console
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all) n
warning: you can set 'accept-flake-config' to 'false' to automatically reject configuration options supplied by flakes
Do you want to allow setting 'allow-dirty = false'? (yes for now/Allow always/no for now) y
Do you want to allow setting 'sandbox = false'? (yes for now/Allow always/no for now) n
```
### Reject everything scenario
```console
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all) N
Rejecting all untrusted nix.conf entries
warning: you can set 'accept-flake-config' to 'false' to automatically reject configuration options supplied by flakes
```
### Accept everything scenario
```console
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all) y
```
### Accept everything PERMANENTLY scenario
Note that accepting everything permanently will authorize these options for any
further operations.
The file containing this trust information is usually located in
`~/.local/share/nix/trusted-settings.json` and can be edited manually to revoke
this permission until Lix provides a first-class command for this manipulation.
```console
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all) A
```
Many thanks to [isabelroses](https://git.lix.systems/isabelroses), [Raito Bezarius](https://git.lix.systems/raito), and [eldritch horrors](https://git.lix.systems/pennae) for this.
- `--check` or `--rebuild` is clearer about a missing path [lix#485](https://git.lix.systems/lix-project/lix/issues/485)
Previously, when running Lix with --check or --rebuild, failures often surfaced
as an unhelpful error:
> "some outputs of '...' are not valid, so checking is not possible"
This message could mean two different things:
- The requested output paths don't exist at all, or,
- Some outputs exist but are not known to Lix
Lix cannot reliably distinguish these cases, so it treated them the same.
We've updated the error messages to clarify what Lix can determine: whether any
valid outputs (> 0) are present or whether no outputs are available.
When no valid outputs can be found, Lix will now suggest building the derivation
normally (without --check or --rebuild) before trying again.
When some valid outputs are present, Lix now reports which ones are valid,
shows the full list of known outputs, and also suggests building the derivation
normally.
In the future, Lix may automate this recovery step when it knows how to rebuild
the paths, but implementing that safely requires more extensive changes to the
codebase.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- `nix develop` no longer ignores the env variable `SSL_CERT_FILE` [cl/5042](https://gerrit.lix.systems/c/lix/+/5042)
Running `nix develop` and `nix print-dev-env` on shells that define the environment variable `SSL_CERT_FILE` now works correctly by exporting that variable inside the built shell.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- Linux sandbox launch overhead greatly reduced [cl/5030](https://gerrit.lix.systems/c/lix/+/5030) [cl/5073](https://gerrit.lix.systems/c/lix/+/5073) [cl/5074](https://gerrit.lix.systems/c/lix/+/5074)
Sandboxed builds are now much cheaper to launch on Linux, with constant management
overhead. This will mostly be noticeable when building derivation trees containing
many small derivations like nixpkgs' `writeFile` or `runCommand` with scripts that
exit quickly. In synthetic tests we have seen build times of 3000 small runCommand
drop from 80 seconds to 14 seconds, which is the most optimistic case in practice.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- mTLS store connections via a plugin [cl/3754](https://gerrit.lix.systems/c/lix/+/3754) [cl/3696](https://gerrit.lix.systems/c/lix/+/3696) [cl/3697](https://gerrit.lix.systems/c/lix/+/3697) [cl/3698](https://gerrit.lix.systems/c/lix/+/3698)
To support use cases requiring mutual TLS (mTLS) authentication when connecting
to remote Nix stores, e.g. private stores, we have introduced a **contributed**
mTLS plugin extending the Lix store interface.
This design follows an extensibility model which was brought up [by a proposal
of making Kerberos authentication possible in Lix
directly](https://gerrit.lix.systems/c/lix/+/3637).
This mTLS plugin serves as a concrete example of how store connection
mechanisms can be modularized through external plugins, without extending Lix
core. This idea can be generalized to integrate automatic certificate renewal
or advanced integrations with secrets engine or posture checks.
It enables custom TLS client certificates to be used for authenticating against
a remote store that enforces mTLS.
To use the plugin, configure Lix manually by setting in your `nix.conf`:
```
plugin-files = /a/path/to/libplugin_mtls_store.so
```
Currently, this must be done explicitly. In the future, Nixpkgs will provide a
mechanism to reference an up-to-date and curated set of plugins automatically.
Making plugins easily consumable outside of Nixpkgs (e.g., from external plugin
registries or binary distributions) remains an open question and will require
further design.
Contributed plugins come with significantly reduced **stability** and
**maintenance** guarantees compared to the Lix core. We encourage users who
depend on a given plugin to take on maintenance responsibilities and apply for
ownership within the Lix mono-repository. These plugins are subject to removal
at any time.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [eldritch horrors](https://git.lix.systems/pennae), [mic92](https://github.com/mic92), [vlaci](https://github.com/vlaci), and [nkk0](https://github.com/nkk0) for this.
- Add an indication of nix-shell nesting depth [lix#826](https://git.lix.systems/lix-project/lix/issues/826) [cl/4657](https://gerrit.lix.systems/c/lix/+/4657)
When in a nix shell (either via a `nix-shell` or a `nix develop` invocation), a variable `NIX_SHELL_LEVEL` is exported to indicate the nesting depth of nix shells.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- `nix store delete` can now unlink a GC root before deleting its closure [cl/4660](https://gerrit.lix.systems/c/lix/+/4660)
Ever build something, and then you want to delete it and whatever dependencies it downloaded?
Before you had to resolve the `result` symlink and copy it, then delete it, *then* `nix store delete --delete-closure --skip-live` on the path you copied.
Now you can just pass `--unlink` and the `result` symlink itself.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- `nix path-info` no longer lies to the user about fetching paths [lix#323](https://git.lix.systems/lix-project/lix/issues/323) [cl/4866](https://gerrit.lix.systems/c/lix/+/4866)
When running `nix path-info` with an installable that is not present in the store, Lix no longer
tells the user which paths are missing and that they will be fetched, as the documentation clearly
states that this command does not fetch missing paths.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- Derivations can now be printed in detail in `nix repl` [cl/3842](https://gerrit.lix.systems/c/lix/+/3842)
Traditionally derivations printed in the REPL would only print a formatted object
representing the path of the derivation file it refers to. This makes inspecting
the enhanced derivation attribute sets encountered from `mkDerivation` or similar
wrappers more difficult. Even the `:p`/`:print` command would not elaborate attribute sets
tagged as a derivation.
With this change you can now use `:p`/`:print` to directly inspect a derivation
by providing one as the top-level object. Derivation attribute sets will only be
printed two levels deep and internal derivation attrsets will remain in unexpanded
path form as before. `drvAttrs` will also be elided as these attributes are already
present in the top-level attribute set of the derivation. These heuristics provide
a balance between readability and functionality. When the `:p`/`:print` is omitted,
a bare derivation is printed in the path format as before.
Many thanks to [Lunaphied](https://git.lix.systems/Lunaphied) for this.
- Reject `__json` in structured attributes derivations [lix#380](https://git.lix.systems/lix-project/lix/issues/380) [cl/5286](https://gerrit.lix.systems/c/lix/+/5286)
In structured attributes derivations, `__json` is used internally to store the
JSON representation of the `env` attribute field that users can set.
Unfortunately, a user can set `__json` *and* enable structured attributes,
resulting in a broken derivation from a semantic point of view.
As no user can benefit from setting `__json` *and* enable structured attributes,
we disallow that possibility and throw an error from now on.
This is not seen as a breaking change because there's no user code that can
benefit from this behavior, hence, it's an improvement to user experience.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Shells support `$NIX_LOG_FD` now [lix#336](https://git.lix.systems/lix-project/lix/issues/336) [cl/4694](https://gerrit.lix.systems/c/lix/+/4694) [cl/4695](https://gerrit.lix.systems/c/lix/+/4695)
Lix's "debugging" shells (`nix3-develop` and `nix-shell`) now set the
`$NIX_LOG_FD` environment variable.
This means that [hook logging in
stdenv](https://github.com/NixOS/nixpkgs/pull/310387) appears while debugging
derivations via `nix3-develop` or `nix-shell`.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Supplementary groups are now supported for daemon authentication [lix#968](https://git.lix.systems/lix-project/lix/issues/968) [cl/5021](https://gerrit.lix.systems/c/lix/+/5021)
macOS, FreeBSD and Linux now support receiving supplementary groups during UNIX domain authentication to a Lix daemon.
This change is particularly beneficial for systemd units with `DynamicUser=true` that need to connect to a Lix daemon, using a `SupplementaryGroups=` allocated by systemd in the context of the process. This is desirable if you wish to harden Lix clients.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [Tom Hubrecht](https://git.lix.systems/tom-hubrecht), [alois31](https://git.lix.systems/alois31), and [eldritch horrors](https://git.lix.systems/pennae) for this.
## Fixes
- Nix shells' `$NIX_BUILD_TOP` are shorter [lix#1044](https://git.lix.systems/lix-project/lix/issues/1044) [cl/4663](https://gerrit.lix.systems/c/lix/+/4663)
Following the changes in 2.94.0 to shorten build directory paths, aimed at [resolving UNIX domain socket length issues](https://gerrit.lix.systems/c/lix/+/4168/13) and [improving nix-shell](https://git.lix.systems/lix-project/lix/issues/940), we inadvertently introduced an excessively long path for the `$NIX_BUILD_TOP` environment variable used by Nix shells (their effective temporary `/build` directory).
To fix this, we replaced the `build-top-$HASH` directory name with simply `build-top`, reducing these paths by at least 30 characters.
We also added a test to ensure that Nix shells do not introduce more than 50 extra characters relative to their base directory (e.g., `/tmp` when `$TMPDIR` is not set).
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Fix resolving of symlinks in flake paths [lix#106](https://git.lix.systems/lix-project/lix/issues/106) [lix#12286](https://git.lix.systems/lix-project/lix/pulls/12286) [cl/4783](https://gerrit.lix.systems/c/lix/+/4783)
Flake paths are now canonicalized to resolve symlinks. This ensures that when a flake is accessed via a symlink, paths are resolved relative to the target directory, not the symlink's location.
Many thanks to [stevalkr](https://github.com/stevalkr) and [xyenon](https://git.lix.systems/xyenon) for this.
- The REPL no longer considers failed loads for `:reload` [lix#50](https://git.lix.systems/lix-project/lix/issues/50) [cl/4864](https://gerrit.lix.systems/c/lix/+/4864) [cl/4865](https://gerrit.lix.systems/c/lix/+/4865) [cl/4700](https://gerrit.lix.systems/c/lix/+/4700) [cl/4889](https://gerrit.lix.systems/c/lix/+/4889)
The [REPL](@docroot@/command-ref/new-cli/nix3-repl.md) allows "loading" files, flakes, and expressions into the environment, with the commands `:load`/`:l`, `:load-flake`/`:lf`, and `:add`/`:a` respectively.
The results of those stay in the environment as-is even if their sources change, until the `:reload` command is used.
However `:reload` would re-perform *all* instances of `:l`/`:lf`/`:a`, meaning you would get things like this:
```nix
nix-repl> :l /tmp/texting.nix
error: getting status of '/tmp/texting.nix': No such file or directory
# oops, typo.
nix-repl> :l /tmp/testing.nix
# Do some stuff…
nix-repl> :reload
error: getting status of '/tmp/texting.nix': No such file or directory
```
This is pretty silly, but also *incredibly* annoying, as it would stop there and *not* reload the correct files anymore.
This effectively meant typoing any of the load commands would make `:reload` useless for the rest of the entire `nix repl` session!
This has been fixed, so now only *successful* loads count towards `:reload`.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [Qyriad](https://git.lix.systems/Qyriad) for this.
- Consistently use commit hash as rev when locking git inputs [cl/4762](https://gerrit.lix.systems/c/lix/+/4762)
Lix will now use commit hashes instead of tag object hashes in the `rev` field
when fetching git inputs by tag in `flake.lock` and `builtins.fetchTree` output.
Note that this means that Lix may change some `flake.lock` files on re-locking. Old `flake.lock` files still remain valid.
Many thanks to [goldstein](https://git.lix.systems/goldstein) for this.
## Development
- Functional lang migration [lix#856](https://git.lix.systems/lix-project/lix/issues/856) [cl/3213](https://gerrit.lix.systems/c/lix/+/3213)
We have done it! The functional/lang framework has now been fully migrated to functional2/lang.
This means: no more `just clean` and `just install` mess and whatever because one removed a test.
The lang test suite is also getting a face lift, with an improved folder structure and restructuring of many tests.
Only the first CL of the chain is provided but there's way more changes associated to this project.
Many thanks to [piegames](https://git.lix.systems/piegames) and [rootile (Rutile)](https://git.lix.systems/rootile) for this.
## Miscellany
- Warn instead of erroring when the final destination of a transfer changes in-flight [lix#1004](https://git.lix.systems/lix-project/lix/issues/1004) [cl/4641](https://gerrit.lix.systems/c/lix/+/4641)
Lix will now emit a warning during downloads where the final destination changes suddently mid-transfer instead of throwing an error.
This transfer behavior has been known to happen very rarely while fetching from some CDNs.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- `impersonate-linux-26` setting removed [cl/5047](https://gerrit.lix.systems/c/lix/+/5047)
Linux 3.0 was released 15 years ago. The `impersonate-linux-26` setting was added
14 years ago with no mention of it being necessary to build anything, only saying
that it improves determinism—which isn't accurate since impersonating Linux 2.6.x
still allows the version string to change, and the final component of the version
does still change with each Linux release. Since this setting should be no longer
necessary in modern systems and workarounds for building old code exist (by using
e.g. `setarch --uname-2.6` to wrap builds) we are removing this setting from Lix.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Default to showing build logs in the new-style (nix3) CLI [cl/4674](https://gerrit.lix.systems/c/lix/+/4674)
Lix will now show logs by default, in addition to the progress bar, when invoked through the new-style "nix3" CLI (`nix build`, etc)
Many thanks to [K900](https://git.lix.systems/K900) for this.
- Lix daemons are now fully socket-activated on systemd setups [lix#1030](https://git.lix.systems/lix-project/lix/issues/1030)
When launched by systemd, Lix no longer uses a persistent daemon process and uses systemd socket
activation instead. This is necessary to support the `cgroups` and `auto-allocate-uids` features
and may improve observability of daemon behavior with common systemd-based monitoring solutions.
The old behavior with a single persistent daemon is still available, but disabled by default. It
is not possible to enable both a persistent daemon and socket activation, starting one stops the
other automatically. Existing installations should not require any changes when they're updated.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Plugin interfaces have changed (again) [lix#359](https://git.lix.systems/lix-project/lix/issues/359) [cl/4933](https://gerrit.lix.systems/c/lix/+/4933) [cl/4934](https://gerrit.lix.systems/c/lix/+/4934)
The `RegisterPrimOp` class used to register builtins has been removed. Plugins
must now call `PluginPrimOps::add` from their `nix_plugin_entry` with the same
parameters previously passed to `RegisterRrimOp` to register any new builtins.
The `GlobalConfig::Register` helper class has also been removed. Adding config
options to the system is now done with `GlobalConfig::registerGlobalConfig`; a
plugin can add config values by calling this function from `nix_plugin_entry`.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
doc/manual/substitute.py
+4 -9
View File
@@ -70,9 +70,9 @@ def do_include(content: str, relative_md_path: Path, source_root: Path, search_p
def recursive_replace(data, book_root, search_path): def recursive_replace(data, book_root, search_path):
match data: match data:
case {'items': items}: case {'sections': sections}:
return data | dict( return data | dict(
items = [recursive_replace(item, book_root, search_path) for item in items], sections = [recursive_replace(section, book_root, search_path) for section in sections],
) )
case {'Chapter': chapter}: case {'Chapter': chapter}:
path_to_chapter = Path(chapter['path']) path_to_chapter = Path(chapter['path'])
@@ -90,11 +90,6 @@ def recursive_replace(data, book_root, search_path):
).replace( ).replace(
'@docroot@', '@docroot@',
("../" * len(path_to_chapter.parent.parts) or "./")[:-1] ("../" * len(path_to_chapter.parent.parts) or "./")[:-1]
).replace(
# this replacement is to avoid corrupting the
# hacking.md manual section on docroot
'@\\docroot\\@',
'@docroot@',
), ),
sub_items = [ sub_items = [
recursive_replace(sub_item, book_root, search_path) recursive_replace(sub_item, book_root, search_path)
@@ -119,10 +114,10 @@ def main():
context, book = json.load(sys.stdin) context, book = json.load(sys.stdin)
# book_root is the directory where book contents leave (ie, src/) # book_root is the directory where book contents leave (ie, src/)
book_root = Path(context['root']) / context['config']['book'].get('src', 'src') book_root = Path(context['root']) / context['config']['book']['src']
# includes pointing into @generated@ will look here # includes pointing into @generated@ will look here
search_path = Path(os.environ['MANUAL_SUBSTITUTE_SEARCH']) search_path = Path(os.environ['MDBOOK_SUBSTITUTE_SEARCH'])
# Find @var@ in all parts of our recursive book structure. # Find @var@ in all parts of our recursive book structure.
replaced_content = recursive_replace(book, book_root, search_path) replaced_content = recursive_replace(book, book_root, search_path)
docker.nix
+2 -2
View File
@@ -8,7 +8,7 @@
tag ? "latest", tag ? "latest",
bundleNixpkgs ? true, bundleNixpkgs ? true,
channelName ? "nixpkgs", channelName ? "nixpkgs",
channelURL ? "https://channels.nixos.org/nixpkgs-unstable", channelURL ? "https://nixos.org/channels/nixpkgs-unstable",
extraPkgs ? [ ], extraPkgs ? [ ],
maxLayers ? 100, maxLayers ? 100,
nixConf ? { }, nixConf ? { },
@@ -381,7 +381,7 @@ image
pkgs.buildPackages.runCommand "docker-image-tarball-${pkgs.nix.version}" pkgs.buildPackages.runCommand "docker-image-tarball-${pkgs.nix.version}"
{ {
nativeBuildInputs = [ pkgs.buildPackages.bubblewrap ]; nativeBuildInputs = [ pkgs.buildPackages.bubblewrap ];
meta.description = "Docker image tarball with Lix for ${pkgs.stdenv.hostPlatform.system}"; meta.description = "Docker image tarball with Lix for ${pkgs.system}";
} }
'' ''
mkdir -p $out/nix-support mkdir -p $out/nix-support
flake.lock
Generated
+19 -17
View File
@@ -3,15 +3,17 @@
"flake-compat": { "flake-compat": {
"flake": false, "flake": false,
"locked": { "locked": {
"lastModified": 1751685974, "lastModified": 1696426674,
"narHash": "sha256-NKw96t+BgHIYzHUjkTK95FqYRVKB8DHpVhefWSz/kTw=", "narHash": "sha256-kvjfFW7WAETZlt09AgDn1MrtKzP7t90Vf7vypd3OL1U=",
"rev": "549f2762aebeff29a2e5ece7a7dc0f955281a1d1", "owner": "edolstra",
"type": "tarball", "repo": "flake-compat",
"url": "https://git.lix.systems/api/v1/repos/lix-project/flake-compat/archive/549f2762aebeff29a2e5ece7a7dc0f955281a1d1.tar.gz" "rev": "0f9255e01c2351cc7d116c072cb317785dd33b33",
"type": "github"
}, },
"original": { "original": {
"type": "tarball", "owner": "edolstra",
"url": "https://git.lix.systems/lix-project/flake-compat/archive/main.tar.gz" "repo": "flake-compat",
"type": "github"
} }
}, },
"lowdown-src": { "lowdown-src": {
@@ -33,11 +35,11 @@
"nix2container": { "nix2container": {
"flake": false, "flake": false,
"locked": { "locked": {
"lastModified": 1767195068, "lastModified": 1724996935,
"narHash": "sha256-+OMnL79ZjqM/PCz2hoQ12MnXNoSSfBGnsYBOZnA9XbI=", "narHash": "sha256-njRK9vvZ1JJsP8oV2OgkBrpJhgQezI03S7gzskCcHos=",
"owner": "nlewo", "owner": "nlewo",
"repo": "nix2container", "repo": "nix2container",
"rev": "bb6801be998ba857a62c002cb77ece66b0a57298", "rev": "fa6bb0a1159f55d071ba99331355955ae30b3401",
"type": "github" "type": "github"
}, },
"original": { "original": {
@@ -106,16 +108,16 @@
}, },
"nixpkgs_2": { "nixpkgs_2": {
"locked": { "locked": {
"lastModified": 1783770249, "lastModified": 1757198069,
"narHash": "sha256-K8pGvFito5dp9T0+clr60q+bJPGEskK75aAJ39w7HBM=", "narHash": "sha256-m3VUcOD4rTs8J7S+3dOjWMrAjw6RcITC3XYQ98zhEFs=",
"owner": "NixOS", "owner": "NixOS",
"repo": "nixpkgs", "repo": "nixpkgs",
"rev": "62463162b3ce92919f19ada41a70a0d943a08da8", "rev": "0747026fc57ecb9c28901c7f7a2b5dc40e8af43c",
"type": "github" "type": "github"
}, },
"original": { "original": {
"owner": "NixOS", "owner": "NixOS",
"ref": "nixos-26.05-small", "ref": "nixos-25.05-small",
"repo": "nixpkgs", "repo": "nixpkgs",
"type": "github" "type": "github"
} }
@@ -123,11 +125,11 @@
"pre-commit-hooks": { "pre-commit-hooks": {
"flake": false, "flake": false,
"locked": { "locked": {
"lastModified": 1769939035, "lastModified": 1733318908,
"narHash": "sha256-Fok2AmefgVA0+eprw2NDwqKkPGEI5wvR+twiZagBvrg=", "narHash": "sha256-SVQVsbafSM1dJ4fpgyBqLZ+Lft+jcQuMtEL3lQWx2Sk=",
"owner": "cachix", "owner": "cachix",
"repo": "git-hooks.nix", "repo": "git-hooks.nix",
"rev": "a8ca480175326551d6c4121498316261cbb5b260", "rev": "6f4e2a2112050951a314d2733a994fbab94864c6",
"type": "github" "type": "github"
}, },
"original": { "original": {
flake.nix
+405 -61
View File
@@ -2,7 +2,7 @@
description = "Lix: A modern, delicious implementation of the Nix package manager"; description = "Lix: A modern, delicious implementation of the Nix package manager";
inputs = { inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-26.05-small"; nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.05-small";
nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2"; nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
# Required because Nix 2.18 is not in Nixpkgs ≥ 25.05 anymore. # Required because Nix 2.18 is not in Nixpkgs ≥ 25.05 anymore.
@@ -24,7 +24,7 @@
flake = false; flake = false;
}; };
flake-compat = { flake-compat = {
url = "https://git.lix.systems/lix-project/flake-compat/archive/main.tar.gz"; url = "github:edolstra/flake-compat";
flake = false; flake = false;
}; };
}; };
@@ -42,7 +42,6 @@
let let
inherit (nixpkgs) lib; inherit (nixpkgs) lib;
lixSrc = self;
# This notice gets echoed as a dev shell hook, and can be turned off with # This notice gets echoed as a dev shell hook, and can be turned off with
# `touch .nocontribmsg` # `touch .nocontribmsg`
@@ -77,35 +76,201 @@
(Run `touch .nocontribmsg` to hide this message.) (Run `touch .nocontribmsg` to hide this message.)
''; '';
scope = import ./nix-support/build/inputs.nix { versionJson = builtins.fromJSON (builtins.readFile ./version.json);
inherit officialRelease = versionJson.official_release;
lib
nixpkgs
nix_2_18
nix2container
lixSrc
nixpkgs-regression
;
};
inherit (scope) # Set to true to build the release notes for the next release.
crossSystems buildUnreleasedNotes = true;
darwinSystems
forAllStdenvs
forAllSystems
forAvailableSystems
linux64BitSystems
nixpkgsFor
overlayFor
systems
versionSuffix
;
inherit (scope.callPackage ./nix-support/build/outputs.nix { }) versionSuffix =
packages if officialRelease then
ciArtifacts ""
tests else
; "pre${
builtins.substring 0 8 (self.lastModifiedDate or self.lastModified or "19700101")
}_${self.shortRev or "dirty"}";
linux32BitSystems = [ "i686-linux" ];
linux64BitSystems = [
"x86_64-linux"
"aarch64-linux"
];
linuxSystems = linux32BitSystems ++ linux64BitSystems;
darwinSystems = [
"x86_64-darwin"
"aarch64-darwin"
];
systems = linuxSystems ++ darwinSystems;
# If you add something here, please update the list in doc/manual/src/contributing/hacking.md.
# Thanks~
crossSystems = [
"armv6l-linux"
"armv7l-linux"
"riscv64-linux"
"aarch64-linux"
"x86_64-freebsd"
# FIXME: broken dev shell due to python
# "x86_64-netbsd"
];
stdenvs = [
# see assertion in package.nix why these two are disabled
# "stdenv"
# "gccStdenv"
"clangStdenv"
"libcxxStdenv"
"ccacheStdenv"
];
forAllSystems = lib.genAttrs systems;
# Same as forAllSystems, but removes nulls, in case something is broken
# on that system.
forAvailableSystems =
f: lib.filterAttrs (name: value: value != null && value != { }) (forAllSystems f);
forAllCrossSystems = lib.genAttrs crossSystems;
forAllStdenvs =
f:
lib.listToAttrs (
map (stdenvName: {
name = "${stdenvName}Packages";
value = f stdenvName;
}) stdenvs
)
// {
# TODO delete this and reënable gcc stdenvs once gcc compiles kj coros correctly
stdenvPackages = f "clangStdenv";
};
# Memoize nixpkgs for different platforms for efficiency.
nixpkgsFor = forAllSystems (
system:
let
make-pkgs =
crossSystem: stdenv:
import nixpkgs {
localSystem = {
inherit system;
};
crossSystem = if crossSystem == null then null else { system = crossSystem; };
overlays = [ (overlayFor (p: p.${stdenv})) ];
};
stdenvs = forAllStdenvs (make-pkgs null);
native = stdenvs.stdenvPackages;
in
{
inherit stdenvs native;
static = native.pkgsStatic;
cross = forAllCrossSystems (crossSystem: make-pkgs crossSystem "clangStdenv");
}
);
overlayFor =
getStdenv: final: prev:
let
currentStdenv = getStdenv final;
in
{
nixStable = prev.nix;
nixVersions = prev.nixVersions // {
nix_2_3 = prev.nixVersions.nix_2_3.overrideAttrs (old: {
meta = old.meta // {
knownVulnerabilities = [ ];
};
});
# Nix 2.18 has been removed from Nixpkgs ≥ 25.05, so we need to reintroduce it ourselves for our tests.
nix_2_18 = nix_2_18.outputs.packages.${currentStdenv.hostPlatform.system}.default;
};
# Forward from the previous stage as we don’t want it to pick the lowdown override
nixUnstable = prev.nixUnstable;
check-headers = final.buildPackages.callPackage ./maintainers/check-headers.nix { };
check-syscalls = final.buildPackages.callPackage ./maintainers/check-syscalls.nix { };
default-busybox-sandbox-shell = final.busybox.override {
useMusl = true;
enableStatic = true;
enableMinimal = true;
extraConfig = ''
CONFIG_FEATURE_FANCY_ECHO y
CONFIG_FEATURE_SH_MATH y
CONFIG_FEATURE_SH_MATH_64 y
CONFIG_ASH y
CONFIG_ASH_OPTIMIZE_FOR_SIZE y
CONFIG_ASH_ALIAS y
CONFIG_ASH_BASH_COMPAT y
CONFIG_ASH_CMDCMD y
CONFIG_ASH_ECHO y
CONFIG_ASH_GETOPTS y
CONFIG_ASH_INTERNAL_GLOB y
CONFIG_ASH_JOB_CONTROL y
CONFIG_ASH_PRINTF y
CONFIG_ASH_TEST y
'';
};
nix = final.callPackage ./package.nix {
inherit versionSuffix officialRelease;
stdenv = currentStdenv;
busybox-sandbox-shell = final.busybox-sandbox-shell or final.default-busybox-sandbox-shell;
# See below
lowdown = final.lowdown_3_0;
lowdown-unsandboxed = final.lowdown_3_0.override { enableDarwinSandbox = false; };
};
lix-clang-tidy = final.callPackage ./subprojects/lix-clang-tidy { };
nix-eval-jobs = final.callPackage ./subprojects/nix-eval-jobs {
srcDir = ./subprojects/nix-eval-jobs;
};
# HACK: We need nix-prefetch-git for fetchCargoVendor for Rust stuff,
# so it can't use Lix, or we infrec:
# lix -> Rust stuff -> fetchCargoVendor -> nix-prefetch-git -> nix (lix)
# This will eventually become a problem upstream, but until then,
# apply some duct tape and pray.
nix-prefetch-git =
if (lib.functionArgs prev.nix-prefetch-git.override) ? "nix" then
prev.nix-prefetch-git.override { nix = prev.nix; }
else
prev.nix-prefetch-git;
# Export the patched version of boehmgc that Lix uses into the overlay
# for consumers of this flake.
boehmgc-nix = final.nix.passthru.boehmgc-nix;
# And same thing for our build-release-notes package.
build-release-notes = final.nix.passthru.build-release-notes;
# As soon as Nixpkgs updates to >= 3.0.0, change to lowdown_2_0!
# We don't change the default version in order to not change the hash
# of Nix/Lix from upstream Nixpkgs.
lowdown_3_0 =
assert lib.versionOlder prev.lowdown.version "3.0.0";
prev.lowdown.overrideAttrs (
finalAttrs: prevAttrs: {
version = "3.0.1";
src = final.fetchurl {
url = "https://kristaps.bsd.lv/lowdown/snapshots/lowdown-${finalAttrs.version}.tar.gz";
sha512 = "fe68e1b7ff23f3992398356d7aa9a330dfd7b72e22bea9a91eeef74182b209ecea0c9f3e2b2216e1a07b2358da2b746238ec9cbbdeebdd3551cef14dd2d79f46";
};
# no longer compiles with GNU make
nativeBuildInputs = prevAttrs.nativeBuildInputs ++ [ final.bmake ];
# dylib fixups on darwin are no longer necessary
postInstall = "";
# doesn't work on darwin due to disallowed nested sandboxes
doInstallCheck = prevAttrs.doInstallCheck && !(final.stdenv.hostPlatform.isDarwin);
doCheck = prevAttrs.doCheck && !(final.stdenv.hostPlatform.isDarwin);
}
);
};
in in
{ {
# for repl debugging # for repl debugging
@@ -115,13 +280,178 @@
# 'nix.perl-bindings' packages. # 'nix.perl-bindings' packages.
overlays.default = overlayFor (p: p.clangStdenv); overlays.default = overlayFor (p: p.clangStdenv);
hydraJobs = ciArtifacts // { hydraJobs = {
# Binary package for various platforms.
build = forAllSystems (system: self.packages.${system}.nix);
# Ensure support for lowdown < 3.0 doesn't regress for NixOS 25.11
build-lowdown_2_0.aarch64-linux = lib.genAttrs [ "aarch64-linux" ] (
system:
self.packages.${system}.nix.override {
lowdown = nixpkgsFor.${system}.native.lowdown;
lowdown-unsandboxed = nixpkgsFor.${system}.native.lowdown-unsandboxed;
}
);
devShell = forAllSystems (system: { devShell = forAllSystems (system: {
default = self.devShells.${system}.default; default = self.devShells.${system}.default;
clang = self.devShells.${system}.native-clangStdenvPackages; clang = self.devShells.${system}.native-clangStdenvPackages;
}); });
inherit tests; rl-next = forAllSystems (
system:
let
rl-next-check =
name: dir:
let
pkgs = nixpkgsFor.${system}.native;
in
pkgs.buildPackages.runCommand "test-${name}-release-notes" { } ''
LANG=C.UTF-8 ${lib.getExe pkgs.build-release-notes} --change-authors ${./doc/manual/change-authors.yml} ${dir} >$out
'';
in
{
user = rl-next-check "rl-next" ./doc/manual/rl-next;
}
);
# Completion tests for the Nix REPL.
repl-completion = forAllSystems (
system: nixpkgsFor.${system}.native.callPackage ./tests/repl-completion.nix { }
);
# Perl bindings for various platforms.
perlBindings = forAllSystems (system: nixpkgsFor.${system}.native.nix.passthru.perl-bindings);
# nix-eval-jobs can be built against this Lix.
nix-eval-jobs = forAllSystems (system: nixpkgsFor.${system}.native.nix-eval-jobs);
# Binary tarball for various platforms, containing a Nix store
# with the closure of 'nix' package.
binaryTarball = forAllSystems (system: nixpkgsFor.${system}.native.nix.passthru.binaryTarball);
# docker image with Lix inside
dockerImage = lib.genAttrs linux64BitSystems (system: self.packages.${system}.dockerImage);
# API docs for Nix's unstable internal C++ interfaces.
internal-api-docs =
let
nixpkgs = nixpkgsFor.x86_64-linux.native;
inherit (nixpkgs) pkgs;
nix = pkgs.callPackage ./package.nix {
inherit versionSuffix officialRelease buildUnreleasedNotes;
inherit (pkgs) build-release-notes;
# Required since we don't support gcc stdenv
stdenv = pkgs.clangStdenv;
internalApiDocs = true;
busybox-sandbox-shell = pkgs.busybox-sandbox-shell;
};
in
nix.overrideAttrs (prev: {
# This Hydra job is just for the internal API docs.
# We don't need the build artifacts here.
dontBuild = true;
doCheck = false;
doInstallCheck = false;
});
# System tests.
tests =
import ./tests/nixos {
inherit
self
lib
nixpkgs
nixpkgsFor
;
}
// {
nix-eval-jobs = forAllSystems (system: self.packages.${system}.nix-eval-jobs.tests.nix-eval-jobs);
# This is x86_64-linux only, just because we have significantly
# cheaper x86_64-linux compute in CI.
# It is clangStdenv because clang's sanitizers are nicer.
asanBuild = self.packages.x86_64-linux.nix-clangStdenv.override {
# Improve caching of non-code changes by not changing the
# derivation name every single time, since this will never be seen
# by users anyway.
versionSuffix = "";
sanitize = [
"address"
"undefined"
];
# it is very hard to make *every* CI build use this option such
# that we don't wind up building Lix twice, so we do it here where
# we are already doing so.
werror = true;
};
# Although this might be nicer to do with pre-commit, that would
# require adding 12MB of nodejs to the dev shell, whereas building it
# in CI with Nix avoids that at a cost of slower feedback on rarely
# touched files.
jsSyntaxCheck =
let
nixpkgs = nixpkgsFor.x86_64-linux.native;
inherit (nixpkgs) pkgs;
docSources = lib.fileset.toSource {
root = ./doc;
fileset = lib.fileset.fileFilter (f: f.hasExt "js") ./doc;
};
in
pkgs.runCommand "js-syntax-check" { } ''
find ${docSources} -type f -print -exec ${pkgs.nodejs-slim}/bin/node --check '{}' ';'
touch $out
'';
# clang-tidy run against the Lix codebase using the Lix clang-tidy plugin
clang-tidy =
let
nixpkgs = nixpkgsFor.x86_64-linux.native;
inherit (nixpkgs) pkgs;
in
pkgs.callPackage ./package.nix {
# Required since we don't support gcc stdenv
stdenv = pkgs.clangStdenv;
versionSuffix = "";
lintInsteadOfBuild = true;
};
# Make sure that nix-env still produces the exact same result
# on a particular version of Nixpkgs.
evalNixpkgs =
with nixpkgsFor.x86_64-linux.native;
runCommand "eval-nixos" { buildInputs = [ nix ]; } ''
type -p nix-env
# Note: we're filtering out nixos-install-tools because https://github.com/NixOS/nixpkgs/pull/153594#issuecomment-1020530593.
time nix-env --store dummy:// -f ${nixpkgs-regression} -qaP --drv-path | sort | grep -v nixos-install-tools > packages
[[ $(sha1sum < packages | cut -c1-40) = 402242fca90874112b34718b8199d844e8b03d12 ]]
mkdir $out
'';
nixpkgsLibTests = forAllSystems (
system:
let
inherit (self.packages.${system}) nix;
pkgs = nixpkgsFor.${system}.native;
testWithNix = import (nixpkgs + "/lib/tests/test-with-nix.nix") { inherit pkgs lib nix; };
in
pkgs.symlinkJoin {
name = "nixpkgs-lib-tests";
paths = [
testWithNix
]
# NOTE: nixpkgs 25.05 is being ... *creative*, and requires this dance to override
# the evaluator used for the test. it will break again in the future, don't worry.
++ lib.optionals pkgs.stdenv.isLinux [
((pkgs.callPackage "${nixpkgs}/ci/eval" { inherit nix; }).attrpathsSuperset {
evalSystem = system;
})
];
}
);
};
pre-commit = forAvailableSystems ( pre-commit = forAvailableSystems (
system: system:
@@ -181,7 +511,42 @@
} }
); );
inherit packages; packages = forAllSystems (
system:
rec {
inherit (nixpkgsFor.${system}.native) nix;
default = nix;
inherit (nixpkgsFor.${system}.native) lix-clang-tidy nix-eval-jobs;
}
// (
lib.optionalAttrs (builtins.elem system linux64BitSystems) {
nix-static = nixpkgsFor.${system}.static.nix;
dockerImage =
let
pkgs = nixpkgsFor.${system}.native;
nix2container' = import nix2container { inherit pkgs system; };
in
import ./docker.nix {
inherit pkgs;
nix2container = nix2container'.nix2container;
tag = pkgs.nix.version;
};
}
// builtins.listToAttrs (
map (crossSystem: {
name = "nix-${crossSystem}";
value = nixpkgsFor.${system}.cross.${crossSystem}.nix;
}) crossSystems
)
// builtins.listToAttrs (
map (stdenvName: {
name = "nix-${stdenvName}";
value = nixpkgsFor.${system}.stdenvs."${stdenvName}Packages".nix;
}) stdenvs
)
)
);
devShells = devShells =
let let
@@ -192,11 +557,8 @@
inherit stdenv versionSuffix; inherit stdenv versionSuffix;
busybox-sandbox-shell = pkgs.busybox-sandbox-shell or pkgs.default-busybox-sandbox; busybox-sandbox-shell = pkgs.busybox-sandbox-shell or pkgs.default-busybox-sandbox;
internalApiDocs = false; internalApiDocs = false;
includeSanitizerLibs = true;
# Use LLD in the dev shell by default for faster link times.
useLld = stdenv.hostPlatform.isLinux;
}; };
pre-commit = self.hydraJobs.pre-commit.${pkgs.stdenv.hostPlatform.system} or { }; pre-commit = self.hydraJobs.pre-commit.${pkgs.system} or { };
in in
pkgs.callPackage nix.mkDevShell { pkgs.callPackage nix.mkDevShell {
pre-commit-checks = pre-commit; pre-commit-checks = pre-commit;
@@ -214,30 +576,12 @@
in in
(makeShells "native" nixpkgsFor.${system}.native) (makeShells "native" nixpkgsFor.${system}.native)
// (makeShells "static" nixpkgsFor.${system}.static) // (makeShells "static" nixpkgsFor.${system}.static)
// (lib.listToAttrs ( // (forAllCrossSystems (
# Provide e.g., both '.#native-aarch64-linux` and `.#static-aarch64-linux`, crossSystem:
# for each cross-system. let
# "native" feels like a misnomer here since it's literally cross compiling, pkgs = nixpkgsFor.${system}.cross.${crossSystem};
# but at least it's consistent with the native/static dichotomy we've set up. in
lib.concatMap ( makeShell pkgs pkgs.clangStdenv
crossSystem:
let
pkgs = nixpkgsFor.${system}.cross.${crossSystem};
inherit (pkgs) pkgsStatic;
native = makeShell pkgs pkgs.clangStdenv;
static = makeShell pkgsStatic pkgsStatic.clangStdenv;
in
[
{
name = "native-${crossSystem}";
value = native;
}
{
name = "static-${crossSystem}";
value = static;
}
]
) crossSystems
)) ))
// { // {
default = self.devShells.${system}.native-clangStdenvPackages; default = self.devShells.${system}.native-clangStdenvPackages;
justfile
+24 -44
View File
@@ -1,67 +1,47 @@
# https://just.systems/man/en/ # https://just.systems/man/en/
#
# Take a look at ./doc/manual/src/contributing/hacking.md for a detailed
# explanation on how to use this file!
# Pin the shell to bash (anything sufficiently POSIX-y would do)
# HACK: We use https://github.com/casey/just#positional-arguments
# and `"@$"` to forward arguments to the inner commands.
# The reason we require this is that `{{ OPTIONS }}` does not escape any values,
# and thus requires one additional level of escaping when running `just` commands with e.g. spaces in them.
# just provides no good solution to this problem, so we have to rely on its forwarding of arguments and shell semantics.
set shell := ["bash", "-uc"]
outdir := x"${out:-$PWD/outputs/out}"
builddir := "build"
# List all available targets # List all available targets
list: list:
just --list just --list
# Clean build artifacts and outputs. # Clean build artifacts
clean: clean:
rm -rf {{ quote(builddir) }}/* {{ quote(builddir) }}/.* {{ quote(outdir) }}/* {{ quote(outdir) }}/.* rm -rf build
cargo clean
# Prepare meson for building. # Prepare meson for building with extra options
[positional-arguments] setup-custom *OPTIONS:
setup *OPTIONS: meson setup build --prefix="$PWD/outputs/out" $mesonFlags {{ OPTIONS }}
meson setup {{ builddir }} --reconfigure --prefix="{{outdir}}" $mesonFlags "$@"
# Prepare meson for building
setup: (setup-custom)
# Build lix with extra options # Build lix with extra options
[positional-arguments] build-custom *OPTIONS:
build *OPTIONS: meson compile -C build {{ OPTIONS }}
meson compile -C {{ builddir }} "$@"
# Build lix
build: (build-custom)
alias compile := build alias compile := build
# `meson install` will automatically build anything that needs to be built to install it. # Install lix for local development with extra options
[doc("Install Lix for local development")] install-custom *OPTIONS: (build-custom OPTIONS)
[positional-arguments] meson install -C build
install *OPTIONS:
meson install --quiet -C {{ builddir }} "$@"
# Run all tests tests (installs first). # Install lix for local development
[positional-arguments] install: (install-custom)
test *OPTIONS: (install)
meson test -C {{ builddir }} --print-errorlogs --max-lines 10000 "$@" # Run tests (usually requires `install`) with extra options
test *OPTIONS:
meson test -C build --print-errorlogs {{ OPTIONS }}
# Run unit tests only # Run unit tests only
test-unit *OPTIONS: (test "--suite" "check") test-unit *OPTIONS: (test "--suite" "check")
# Run integration tests only # Run integration tests only
test-integration *OPTIONS: (test "--suite" "installcheck" OPTIONS) test-integration *OPTIONS: install (test "--suite" "installcheck")
# Run functional2 tests using pytest directly, allowing for additional arguments to be passed to pytest e.g. for more granular test selection alias clang-tidy := lint
[positional-arguments]
test-functional2 *OPTIONS:
cd tests/functional2 && python -m pytest -v "$@"
# special target for cargo because meson cannot be convinced to not mangle cargo test output,
# and getting properly colored test output any other way also doesn't look all that possible.
[positional-arguments]
test-rs *OPTIONS:
meson test -C {{ builddir }} --interactive lix-rs-tests "$@"
# Lint with `clang-tidy` # Lint with `clang-tidy`
lint: lint:
lix/OWNERS
-10
View File
@@ -1,10 +0,0 @@
# This reproduces the Lix Approvers plus Lix groups to yield the status quo
alois1@gmx-topmail.de
jade@lix.systems
lunaphied@lunaphied.me
maximilian@mbosch.me
me@0upti.me
pennae@lix.systems
qyriad@qyriad.me
raito@lix.systems
rbt@sent.as
lix/code-generation/__init__.py
-1
View File
@@ -1 +0,0 @@
# noqa: N999 # consistency with rest of the codebase
lix/code-generation/build_builtin_constants.py
+84 -104
View File
@@ -1,113 +1,93 @@
import dataclasses from typing import List, NamedTuple
from enum import Enum
from textwrap import dedent, indent
from typing import NamedTuple
from common import cxx_literal, generate_file, load_data
from common import cxx_literal, generate_file, load_data, get_argument_parser KNOWN_KEYS = set([
'name',
'type',
'constructorArgs',
'implementation',
'impure',
'renameInGlobalScope',
])
IMPURE_NOTE = """ class BuiltinConstant(NamedTuple):
> **Note** name: str
type: str
implementation: str
impure: bool
rename_in_global_scope: bool
documentation: str
def parse(datum):
unknown_keys = set(datum.keys()) - KNOWN_KEYS
if unknown_keys:
raise Exception('unknown keys', unknown_keys)
return BuiltinConstant(
name = datum['name'],
type = datum['type'],
implementation = ('{' + ', '.join([f'NewValueAs::{datum["type"]}', *datum['constructorArgs']]) + '}') if 'constructorArgs' in datum else datum['implementation'],
impure = datum.get('impure', False),
rename_in_global_scope = datum.get('renameInGlobalScope', True),
documentation = datum.content,
)
VALUE_TYPES = {
'attrs': 'nAttrs',
'boolean': 'nBool',
'integer': 'nInt',
'list': 'nList',
'null': 'nNull',
'string': 'nString',
}
HUMAN_TYPES = {
'attrs': 'set',
'boolean': 'Boolean',
'integer': 'integer',
'list': 'list',
'null': 'null',
'string': 'string',
}
def main():
import argparse
ap = argparse.ArgumentParser()
ap.add_argument('--header', help='Path of the header to generate')
ap.add_argument('--docs', help='Path of the documentation file to generate')
ap.add_argument('defs', help='Builtin definition files', nargs='+')
args = ap.parse_args()
builtin_constants = load_data(args.defs, BuiltinConstant.parse)
generate_file(args.header, builtin_constants, lambda constant:
# `builtins` is magic and must come first
'' if constant.name == 'builtins' else constant.name,
lambda constant:
f'''{'if (!evalSettings.pureEval) ' if constant.impure else ''}{{
addConstant({cxx_literal(('__' if constant.rename_in_global_scope else '') + constant.name)}, {constant.implementation}, {{
.type = {VALUE_TYPES[constant.type]},
.doc = {cxx_literal(constant.documentation)},
.impureOnly = {cxx_literal(constant.impure)},
}});
}}
''')
generate_file(args.docs, builtin_constants, lambda constant: constant.name, lambda constant:
f'''<dt id="builtins-{constant.name}">
<a href="#builtins-{constant.name}"><code>{constant.name}</code></a> ({HUMAN_TYPES[constant.type]})
</dt>
<dd>
{constant.documentation}
''' + ('''> **Note**
> >
> Not available in [pure evaluation mode](@docroot@/command-ref/conf-file.md#conf-pure-eval). > Not available in [pure evaluation mode](@docroot@/command-ref/conf-file.md#conf-pure-eval).
""" ''' if constant.impure else '') + '''</dd>
''')
class TypeName(NamedTuple): if __name__ == '__main__':
human: str
code: str
class BuiltinType(TypeName, Enum):
attrs = TypeName("set", "nAttrs")
boolean = TypeName("boolean", "nBool")
integer = TypeName("integer", "nInt")
list = TypeName("list", "nList")
null = TypeName("null", "nNull")
string = TypeName("string", "nString")
@classmethod
def from_string(cls, t_name: str) -> "BuiltinType":
for t in cls:
if t_name == t.name:
return t
msg = f"Invalid builtin type: {t_name}"
raise ValueError(msg)
@dataclasses.dataclass
class BuiltinConstant:
name: str
documentation: str
# Fields with different name in the Post than in here
# our fields
type: BuiltinType = dataclasses.field(init=False)
# Post fields
type_str: dataclasses.InitVar[str]
constructor_args: dataclasses.InitVar[list[str] | None] = None
implementation: str = ""
impure: bool = False
rename_in_global_scope: bool = True
def __post_init__(self, type_str: str, constructor_args: list[str] | None):
self.type = BuiltinType.from_string(type_str)
if constructor_args is not None:
args = [f"NewValueAs::{type_str}"] + constructor_args
self.implementation = f"{{{','.join(args)}}}"
@property
def code(self) -> str:
cond = "if (!evalSettings.pureEval) " if self.impure else ""
return dedent(f"""
{cond} {{
addConstant(
{cxx_literal(("__" if self.rename_in_global_scope else "") + self.name)},
{self.implementation},
{{
.type = {self.type.code},
.doc = {cxx_literal(self.documentation)},
.impureOnly = {cxx_literal(self.impure)},
}}
);
}}
""")
@property
def docs(self) -> str:
indentation = " " * 3
return dedent(f"""
<dt id="builtins-{self.name}">
<a href="#builtins-{self.name}"><code>{self.name}</code></a> ({self.type.human})
</dt>
<dd>
{indent(self.documentation, indentation)}
{indent(IMPURE_NOTE, indentation) if self.impure else ""}
</dd>
""")
def main():
args = get_argument_parser().parse_args()
builtin_constants = load_data(args.defs, BuiltinConstant)
generate_file(
args.header,
builtin_constants,
lambda constant:
# `builtins` is magic and must come first
"" if constant.name == "builtins" else constant.name,
lambda b: b.code,
)
generate_file(args.docs, builtin_constants, lambda constant: constant.name, lambda b: b.docs)
if __name__ == "__main__":
main() main()
lix/code-generation/build_builtins.py
+66 -74
View File
@@ -1,90 +1,82 @@
import dataclasses from typing import List, NamedTuple, Optional
from textwrap import dedent, indent
from common import ( from build_experimental_features import ExperimentalFeature
cxx_literal, from common import cxx_literal, generate_file, load_data
generate_file,
load_data,
get_argument_parser,
get_experimental_features,
)
KNOWN_KEYS = set([
'name',
'implementation',
'renameInGlobalScope',
'args',
'experimentalFeature',
])
@dataclasses.dataclass class Builtin(NamedTuple):
class Builtin:
name: str name: str
implementation: str
rename_in_global_scope: bool
args: List[str]
experimental_feature: Optional[str]
documentation: str documentation: str
args: list[str]
experimental_feature: str | None = None
implementation: str = ""
rename_in_global_scope: bool = True
def __post_init__(self): def parse(datum):
self.implementation = self.implementation or f"prim_{self.name}" unknown_keys = set(datum.keys()) - KNOWN_KEYS
if unknown_keys:
def generate_code(self, experimental_features: dict[str, str]) -> str: raise Exception('unknown keys', unknown_keys)
xf = experimental_features[self.experimental_feature] return Builtin(
cond = ( name = datum['name'],
f"if (experimentalFeatureSettings.isEnabled({xf})) " implementation = datum['implementation'] if 'implementation' in datum else f'prim_{datum["name"]}',
if self.experimental_feature rename_in_global_scope = datum.get('renameInGlobalScope', True),
else "" args = datum['args'],
experimental_feature = datum.get('experimentalFeature', None),
documentation = datum.content,
) )
return dedent(f"""
{cond}{{
addPrimOp({{
.name = {cxx_literal(("__" if self.rename_in_global_scope else "") + self.name)},
.args = {cxx_literal(self.args)},
.arity = {len(self.args)},
.doc = {cxx_literal(self.documentation)},
.fun = {self.implementation},
.experimentalFeature = {xf},
}});
}}
""")
@property
def docs(self) -> str:
return dedent(f"""
<dt id="builtins-{self.name}">
<a href="#builtins-{self.name}"><code>{self.name} {
" ".join([f"<var>{arg}</var>" for arg in self.args])
}</code></a>
</dt>
<dd>
{indent(self.documentation, " " * 3)}
{
f"This function is only available if the [{self.experimental_feature}](@docroot@/contributing/experimental-features.md#xp-feature-{self.experimental_feature}) experimental feature is enabled."
if self.experimental_feature is not None
else ""
}
</dd>
""")
def main(): def main():
ap = get_argument_parser() import argparse
ap.add_argument(
"--experimental-features", help="Directory containing the experimental feature definitions" ap = argparse.ArgumentParser()
) ap.add_argument('--header', help='Path of the header to generate')
ap.add_argument('--docs', help='Path of the documentation file to generate')
ap.add_argument('--experimental-features', help='Directory containing the experimental feature definitions')
ap.add_argument('defs', help='Builtin definition files', nargs='+')
args = ap.parse_args() args = ap.parse_args()
builtins = load_data(args.defs, Builtin) builtins = load_data(args.defs, Builtin.parse)
experimental_features = get_experimental_features( experimental_feature_names = set([builtin.experimental_feature for (_, builtin) in builtins])
args.experimental_features, [b.experimental_feature for (_, b) in builtins] experimental_feature_names.discard(None)
) experimental_feature_files = [f'{args.experimental_features}/{name}.md' for name in experimental_feature_names]
experimental_features = load_data(experimental_feature_files, ExperimentalFeature.parse)
experimental_features = dict(map(lambda path_and_feature:
(path_and_feature[1].name, f'Xp::{path_and_feature[1].internal_name}'), experimental_features))
experimental_features[None] = 'std::nullopt'
generate_file( generate_file(args.header, builtins, lambda builtin: builtin.name, lambda builtin:
args.header, f'''{'' if builtin.experimental_feature is None else f'if (experimentalFeatureSettings.isEnabled({experimental_features[builtin.experimental_feature]})) '}{{
builtins, addPrimOp({{
lambda builtin: builtin.name, .name = {cxx_literal(('__' if builtin.rename_in_global_scope else '') + builtin.name)},
lambda b: b.generate_code(experimental_features), .args = {cxx_literal(builtin.args)},
) .arity = {len(builtin.args)},
generate_file(args.docs, builtins, lambda builtin: builtin.name, lambda b: b.docs) .doc = {cxx_literal(builtin.documentation)},
.fun = {builtin.implementation},
.experimentalFeature = {experimental_features[builtin.experimental_feature]},
}});
}}
''')
generate_file(args.docs, builtins, lambda builtin: builtin.name, lambda builtin:
f'''<dt id="builtins-{builtin.name}">
<a href="#builtins-{builtin.name}"><code>{builtin.name} {' '.join([f'<var>{arg}</var>' for arg in builtin.args])}</code></a>
</dt>
<dd>
{builtin.documentation}
if __name__ == "__main__": ''' + (f'''This function is only available if the [{builtin.experimental_feature}](@docroot@/contributing/experimental-features.md#xp-feature-{builtin.experimental_feature}) experimental feature is enabled.
''' if builtin.experimental_feature is not None else '') + '''</dd>
''')
if __name__ == '__main__':
main() main()
lix/code-generation/build_experimental_features.py
+58
View File
@@ -0,0 +1,58 @@
from typing import NamedTuple
from common import cxx_literal, generate_file, load_data
KNOWN_KEYS = set([
'name',
'internalName',
])
class ExperimentalFeature(NamedTuple):
name: str
internal_name: str
description: str
def parse(datum):
unknown_keys = set(datum.keys()) - KNOWN_KEYS
if unknown_keys:
raise ValueError('unknown keys', unknown_keys)
return ExperimentalFeature(
name = datum['name'],
internal_name = datum['internalName'],
description = datum.content,
)
def main():
import argparse
ap = argparse.ArgumentParser()
ap.add_argument('--deprecated', action='store_true', help='Generate deprecated features')
ap.add_argument('--header', help='Path of the declaration header to generate')
ap.add_argument('--impl-header', help='Path of the implementation header to generate')
ap.add_argument('--descriptions', help='Path of the description file to generate')
ap.add_argument('--shortlist', help='Path of the shortlist file to generate')
ap.add_argument('defs', help='Experimental feature definition files', nargs='+')
args = ap.parse_args()
features = load_data(args.defs, ExperimentalFeature.parse)
generate_file(args.header, features, lambda feature: feature.name, lambda feature:
f' {feature.internal_name},\n')
generate_file(args.impl_header, features, lambda feature: feature.name, lambda feature:
f''' {{
.tag = {"Dep" if args.deprecated else "Xp"}::{feature.internal_name},
.name = {cxx_literal(feature.name)},
.description = {cxx_literal(feature.description)},
}},
''')
generate_file(args.descriptions, features, lambda feature: feature.name, lambda feature:
f'''## [`{feature.name}`]{{#{"dp" if args.deprecated else "xp"}-feature-{feature.name}}}
{feature.description}
''')
generate_file(args.shortlist, features, lambda feature: feature.name, lambda feature:
f' - [`{feature.name}`](@docroot@/contributing/{"deprecated" if args.deprecated else "experimental"}-features.md#{"dp" if args.deprecated else "xp"}-feature-{feature.name})\n')
if __name__ == '__main__':
main()
lix/code-generation/build_extra_features.py
-105
View File
@@ -1,105 +0,0 @@
import dataclasses
from enum import Enum
from textwrap import dedent
from typing import ClassVar, NamedTuple
from common import cxx_literal, generate_file, load_data, get_argument_parser
class FeatureTypeNames(NamedTuple):
code_tag: str
doc_tag: str
class TimelineEvent(NamedTuple):
date: str
release: str
message: str
cls: list[int]
class FeatureType(FeatureTypeNames, Enum):
experimental = FeatureTypeNames("Xp", "xp")
deprecated = FeatureTypeNames("Dep", "dp")
@dataclasses.dataclass
class ExtraFeature:
name: str
internal_name: str
documentation: str
timeline: list[TimelineEvent] = dataclasses.field(default_factory=list)
type: ClassVar[FeatureType]
@property
def code(self) -> str:
return dedent(f"""
{{
.tag = {ExtraFeature.type.code_tag}::{self.internal_name},
.name = {cxx_literal(self.name)},
.description = {cxx_literal(self.documentation)},
}},
""")
@property
def docs(self) -> str:
timeline = (
f"""
### Timeline
{
"\n ".join(
[
f"- {event.date}, {event.release}: {event.message} [{", ".join([f'[CL {cl}](https://gerrit.lix.systems/c/lix/+/{cl})' for cl in event.cls])}]"
for event in self.timeline
]
)
}
"""
if self.timeline
else ""
)
return dedent(f"""
## [`{self.name}`]{{#{ExtraFeature.type.doc_tag}-feature-{self.name}}}
{self.documentation.replace("\n", f"\n{' ' * 3}")}
{timeline}
""")
@property
def short_docs(self) -> str:
return f" - [`{self.name}`](@docroot@/contributing/{ExtraFeature.type.name}-features.md#{ExtraFeature.type.doc_tag}-feature-{self.name})\n"
def main():
ap = get_argument_parser()
ap.add_argument("--deprecated", action="store_true", help="Generate deprecated features")
ap.add_argument("--impl-header", help="Path of the implementation header to generate")
ap.add_argument("--shortlist", help="Path of the shortlist file to generate")
args = ap.parse_args()
ExtraFeature.type = FeatureType.deprecated if args.deprecated else FeatureType.experimental
def load(**kwargs) -> ExtraFeature:
kwargs["timeline"] = [TimelineEvent(**args) for args in kwargs.get("timeline", [])]
return ExtraFeature(**kwargs)
features = load_data(args.defs, load)
generate_file(
args.header,
features,
lambda feature: feature.name,
lambda feature: f" {feature.internal_name},\n",
)
generate_file(args.impl_header, features, lambda feature: feature.name, lambda f: f.code)
generate_file(args.docs, features, lambda feature: feature.name, lambda f: f.docs)
generate_file(args.shortlist, features, lambda feature: feature.name, lambda f: f.short_docs)
if __name__ == "__main__":
main()
lix/code-generation/build_settings.py
+120 -136
View File
@@ -1,157 +1,141 @@
import dataclasses from typing import List, NamedTuple, Optional
from textwrap import dedent
from typing import Any
from common import ( from build_experimental_features import ExperimentalFeature
cxx_literal, from common import cxx_literal, generate_file, load_data
generate_file,
load_data,
get_experimental_features,
get_argument_parser,
)
KNOWN_KEYS = set([
'name',
'internalName',
'platforms',
'type',
'settingType',
'default',
'defaultExpr',
'defaultText',
'aliases',
'experimentalFeature',
'deprecated',
])
PLATFORM_WARNING = """ class Setting(NamedTuple):
> **Note** name: str
> This setting is only available on {platforms} systems. internal_name: str
description: str
platforms: Optional[List[str]]
setting_type: str
default_expr: str
default_text: str
aliases: List[str]
experimental_feature: Optional[str]
deprecated: bool
""" def parse(datum):
unknown_keys = set(datum.keys()) - KNOWN_KEYS
if unknown_keys:
raise ValueError('unknown keys', unknown_keys)
default_text = f'`{nix_conf_literal(datum["default"])}`' if 'default' in datum else datum['defaultText']
if default_text == '``':
default_text = '*empty*'
return Setting(
name = datum['name'],
internal_name = datum['internalName'],
description = datum.content,
platforms = datum.get('platforms', None),
setting_type = f'Setting<{datum["type"]}>' if 'type' in datum else datum['settingType'],
default_expr = cxx_literal(datum['default']) if 'default' in datum else datum['defaultExpr'],
default_text = default_text,
aliases = datum.get('aliases', []),
experimental_feature = datum.get('experimentalFeature', None),
deprecated = datum.get('deprecated', False),
)
XP_WARNING = """ platform_names = {
> **Warning** 'darwin': 'Darwin',
'linux': 'Linux',
}
def nix_conf_literal(v):
if v is None:
return ''
elif isinstance(v, bool) and v == False: # 0 == False
return 'false'
elif isinstance(v, bool) and v == True: # 1 == True
return 'true'
elif isinstance(v, int):
return str(v)
elif isinstance(v, str):
return v
elif isinstance(v, list):
return ' '.join([nix_conf_literal(item) for item in v])
else:
raise NotImplementedError(f'Cannot represent {repr(v)} in nix.conf')
def indent(prefix, body):
return ''.join(['\n' if line == '' else f'{prefix}{line}\n' for line in body.split('\n')])
def main():
import argparse
ap = argparse.ArgumentParser()
ap.add_argument('--kernel', help='Name of the kernel Lix will run on')
ap.add_argument('--header', help='Path of the header to generate')
ap.add_argument('--docs', help='Path of the documentation file to generate')
ap.add_argument('--experimental-features', help='Directory containing the experimental feature definitions')
ap.add_argument('defs', help='Setting definition files', nargs='+')
args = ap.parse_args()
settings = load_data(args.defs, Setting.parse)
experimental_feature_names = set([setting.experimental_feature for (_, setting) in settings])
experimental_feature_names.discard(None)
experimental_feature_files = [f'{args.experimental_features}/{name}.md' for name in experimental_feature_names]
experimental_features = load_data(experimental_feature_files, ExperimentalFeature.parse)
experimental_features = dict(map(lambda path_and_feature:
(path_and_feature[1].name, f'Xp::{path_and_feature[1].internal_name}'), experimental_features))
experimental_features[None] = 'std::nullopt'
generate_file(args.header, settings, lambda setting: setting.name, lambda setting:
f'''{setting.setting_type} {setting.internal_name} {{
this,
{setting.default_expr},
{cxx_literal(setting.name)},
{cxx_literal(setting.description)},
{cxx_literal(setting.aliases)},
true,
{experimental_features[setting.experimental_feature]},
{cxx_literal(setting.deprecated)}
}};
''' if setting.platforms is None or args.kernel in setting.platforms else '')
generate_file(args.docs, settings, lambda setting: setting.name, lambda setting:
f'''- <span id="conf-{setting.name}">[`{setting.name}`](#conf-{setting.name})</span>
{indent(" ", setting.description)}
''' + (f''' > **Note**
> This setting is only available on {', '.join([platform_names[platform] for platform in setting.platforms])} systems.
''' if setting.platforms is not None else '') + (f''' > **Warning**
> This setting is part of an > This setting is part of an
> [experimental feature](@docroot@/contributing/experimental-features.md). > [experimental feature](@docroot@/contributing/experimental-features.md).
To change this setting, you need to make sure the corresponding experimental feature, To change this setting, you need to make sure the corresponding experimental feature,
[`{feature}`](@docroot@/contributing/experimental-features.md#xp-feature-{feature}), [`{setting.experimental_feature}`](@docroot@/contributing/experimental-features.md#xp-feature-{setting.experimental_feature}),
is enabled. is enabled.
For example, include the following in [`nix.conf`](#): For example, include the following in [`nix.conf`](#):
``` ```
extra-experimental-features = {feature} extra-experimental-features = {setting.experimental_feature}
{name} = ... {setting.name} = ...
``` ```
""" ''' if setting.experimental_feature is not None else '') + (''' > **Warning**
DEPR_WARNING = """
> **Warning**
> This setting is deprecated and will be removed in a future version of Lix. > This setting is deprecated and will be removed in a future version of Lix.
""" ''' if setting.deprecated else '') + f''' **Default:** {setting.default_text}
''' + (f''' **Deprecated alias:** {', '.join([f'`{item}`' for item in setting.aliases])}
@dataclasses.dataclass ''' if setting.aliases != [] else ''))
class Setting:
name: str
internal_name: str
documentation: str
default_text: str = "" if __name__ == '__main__':
setting_type: str = ""
default_expr: str = ""
platforms: list[str] = dataclasses.field(default_factory=list)
aliases: list[str] = dataclasses.field(default_factory=list)
experimental_feature: str | None = None
deprecated: bool = False
default: dataclasses.InitVar[str | None] = None
type_str: dataclasses.InitVar[str | None] = None
def __post_init__(self, default: Any, type_str: str | None):
if default is not None: # is not None nor an empty String
self.default_text = f"`{nix_conf_literal(default)}`"
self.default_expr = self.default_expr or cxx_literal(default)
self.default_text = self.default_text or "*empty*"
if type_str is not None:
self.setting_type = f"Setting<{type_str}>"
def generate_code(self, experimental_features: dict[str | None, str]) -> str:
indentation = " " * 4
expr = (indent(indentation, self.default_expr) + indentation) if "\n" in self.default_expr else self.default_expr
return dedent(f"""
{self.setting_type} {self.internal_name} {{
this,
{expr},
{cxx_literal(self.name)},
{cxx_literal(self.documentation)},
{cxx_literal(self.aliases)},
true,
{experimental_features[self.experimental_feature]},
{cxx_literal(self.deprecated)}
}};
""")
@property
def docs(self) -> str:
indentation = " " * 3
platforms = [p.capitalize() for p in self.platforms]
aliases = [f"`{item}`" for item in self.aliases]
description = dedent(f"""
{indent(indentation, self.documentation)}
{indent(indentation, PLATFORM_WARNING.format(platforms=str(platforms)[1:-1])) if self.platforms else ""}
{indent(indentation, XP_WARNING.format(feature=self.experimental_feature, name=self.name)) if self.experimental_feature is not None else ""}
{indent(indentation, DEPR_WARNING) if self.deprecated else ""}
**Default:** {self.default_text}
{f"**Deprecated alias:** {str(aliases)[1:-1]}\n" if self.aliases else ""}
""")
return f'- <span id="conf-{self.name}">[`{self.name}`](#conf-{self.name})</span>' + indent(
" ", # indent by two space to make it part of the list point
description,
)
platform_names = {"darwin": "Darwin", "linux": "Linux"}
def nix_conf_literal(v: Any) -> str:
if v is None:
return ""
if v is False:
return "false"
if v is True:
return "true"
if isinstance(v, int):
return str(v)
if isinstance(v, str):
return v
if isinstance(v, list):
return " ".join([nix_conf_literal(item) for item in v])
msg = f"Cannot represent {v!r} in nix.conf"
raise NotImplementedError(msg)
def indent(prefix: str, body: str) -> str:
return "".join(["\n" if not line else f"{prefix}{line}\n" for line in body.split("\n")])
def main():
ap = get_argument_parser()
ap.add_argument("--kernel", help="Name of the kernel Lix will run on")
ap.add_argument(
"--experimental-features", help="Directory containing the experimental feature definitions"
)
args = ap.parse_args()
settings = load_data(args.defs, Setting)
experimental_features = get_experimental_features(
args.experimental_features, [s.experimental_feature for (_, s) in settings]
)
generate_file(
args.header,
settings,
lambda setting: setting.name,
lambda setting: setting.generate_code(experimental_features)
if not setting.platforms or args.kernel in setting.platforms
else "",
)
generate_file(args.docs, settings, lambda setting: setting.name, lambda setting: setting.docs)
if __name__ == "__main__":
main() main()
lix/code-generation/capnpc_wrapper.py
-68
View File
@@ -1,68 +0,0 @@
#!@python@
# ruff: noqa: SIM112 # ignore lowercase env variable names for capnpc as we have them in lower case as arguments
import argparse
import capnp
from pathlib import Path
import os
import subprocess
import sys
if lang := os.environ.get("lix_capnp_lang"):
outputs = os.environ["lix_capnp_outputs"].split()
old_cwd = os.environ["lix_capnp_old_cwd"]
schema = capnp.load("@capnp_include@/capnp/schema.capnp", imports=["@capnp_include@"])
request = schema.CodeGeneratorRequest.read(sys.stdin)
subprocess.run([lang], input=request.as_builder().to_bytes()).check_returncode()
base_dir = Path.cwd()
os.chdir(old_cwd)
include = [str(Path(p).resolve()) for p in os.environ["lix_capnp_include"].split(":")]
if depfile := os.environ["lix_capnp_depfile"]:
deps = ""
for input_file in request.requestedFiles:
deps += " ".join(f"{input_file.filename}.{o}" for o in outputs)
deps += ":"
for dep in input_file.imports:
if dep.name.startswith("/"):
for candidate in (Path(i + dep.name) for i in include):
if candidate.exists():
deps += " " + str(candidate)
break
else:
msg = "not handling relative includes"
raise RuntimeError(msg)
deps += "\n\n"
Path(depfile).write_text(deps)
else:
parser = argparse.ArgumentParser()
parser.add_argument("--language")
parser.add_argument("--outdir")
parser.add_argument("--src-prefix")
parser.add_argument("--depfile", default="")
parser.add_argument("-I", "--include", action="append", default=["@capnp_include@"])
parser.add_argument("inputs", nargs="+")
args = parser.parse_args()
for infile in args.inputs:
os.environ["lix_capnp_lang"] = f"capnpc-{args.language}"
os.environ["lix_capnp_include"] = ":".join(args.include)
os.environ["lix_capnp_depfile"] = args.depfile
os.environ["lix_capnp_old_cwd"] = str(Path.cwd())
if args.language == "c++":
os.environ["lix_capnp_outputs"] = "c++ h"
else:
raise RuntimeError("unknown language " + args.language)
subprocess.run(
[
"@capnp@",
"compile",
f"-o{sys.argv[0]}:{args.outdir}",
f"--src-prefix={args.src_prefix}",
*(f"-I{i}" for i in args.include),
infile,
]
).check_returncode()
lix/code-generation/common.py
+42 -95
View File
@@ -1,113 +1,60 @@
import argparse
import re
from collections.abc import Callable
from pathlib import Path
from typing import Any
import frontmatter import frontmatter
import pathlib
from collections import defaultdict
def cxx_escape_character(c):
def cxx_escape_character(c: str) -> str: if ord(c) >= 0x20 and ord(c) < 0x7f and c != '"' and c != '?' and c != '\\':
if 0x20 <= ord(c) < 0x7F and c != '"' and c != "?" and c != "\\":
return c return c
if c == "\t": elif c == '\t':
return r"\t" return r'\t'
if c == "\n": elif c == '\n':
return r"\n" return r'\n'
if c == "\r": elif c == '\r':
return r"\r" return r'\r'
if c == '"': elif c == '"':
return r"\"" return r'\"'
if c == "?": elif c == '?':
return r"\?" return r'\?'
if c == "\\": elif c == '\\':
return r"\\" return r'\\'
if ord(c) <= 0xFFFF: elif ord(c) <= 0xffff:
return str.format(r"\u{:04x}", ord(c)) return str.format(r'\u{:04x}', ord(c))
return str.format(r"\U{:08x}", ord(c)) else:
return str.format(r'\U{:08x}', ord(c))
def cxx_literal(v):
def cxx_literal(v: Any) -> str:
if v is None: if v is None:
return "std::nullopt" return 'std::nullopt'
if v is False: elif isinstance(v, bool) and v == False: # 0 == False
return "false" return 'false'
if v is True: elif isinstance(v, bool) and v == True: # 1 == True
return "true" return 'true'
if isinstance(v, int): elif isinstance(v, int):
return str(v) return str(v)
if isinstance(v, str): elif isinstance(v, str):
return "".join(['"', *(cxx_escape_character(c) for c in v), '"']) return ''.join(['"', *(cxx_escape_character(c) for c in v), '"'])
if isinstance(v, list): elif isinstance(v, list):
return f"{{{', '.join([cxx_literal(item) for item in v])}}}" return f'{{{", ".join([cxx_literal(item) for item in v])}}}'
msg = f"cannot represent {v!r} in C++" else:
raise NotImplementedError(msg) raise NotImplementedError(f'cannot represent {repr(v)} in C++')
def load_data(defs, parse_function):
def get_experimental_features(
base_path: str, human_names: list[str | None]
) -> dict[str | None, str]:
experimental_feature_files = {
f"{base_path}/{xp_name}.md" for xp_name in human_names if xp_name is not None
}
from build_extra_features import ExtraFeature # noqa: PLC0415 # Avoid cyclic import
experimental_features_data = load_data(list(experimental_feature_files), ExtraFeature)
experimental_features: dict[str | None, str] = {
xf.name: f"Xp::{xf.internal_name}" for _, xf in experimental_features_data
}
experimental_features[None] = "std::nullopt"
return experimental_features
FIELD_RENAMES = {"type": "type_str", "content": "documentation"}
def load_data[T](defs: list[str], parse_function: type[T]) -> list[tuple[str, T]]:
data = [] data = []
for path in defs: for path in defs:
try: try:
datum = { datum = frontmatter.load(path)
# convert camelCase to snake_case data.append((path, parse_function(datum)))
re.sub(r"(?<=.)([A-Z])", lambda m: f"_{m.group(1).lower()}", k): v
for k, v in frontmatter.load(path).to_dict().items()
}
for post_name, field_name in FIELD_RENAMES.items():
if post_name in datum:
datum[field_name] = datum.pop(post_name)
data.append((path, parse_function(**datum)))
except Exception as e: except Exception as e:
e.add_note(f"in {path}") e.add_note(f'in {path}')
raise raise
return data return data
def generate_file(path, data, sort_key_function, generate_function):
def generate_file[T](
path: str | None,
data: list[T],
sort_key_function: Callable[[T], str],
generate_function: Callable[[T], str],
):
if path is not None: if path is not None:
with Path(path).open("w") as out: with open(path, 'w') as out:
for path, datum in sorted( for path, datum in sorted(data, key=lambda pathAndDatum: sort_key_function(pathAndDatum[1])):
data, key=lambda path_and_datum: sort_key_function(path_and_datum[1])
):
try: try:
text = generate_function(datum) out.write(generate_function(datum))
out.write(text)
except Exception as e: except Exception as e:
e.add_note(f"in {path}") e.add_note(f'in {path}')
raise raise
def get_argument_parser() -> argparse.ArgumentParser:
ap = argparse.ArgumentParser()
ap.add_argument("--header", help="Path of the header to generate")
ap.add_argument("--docs", help="Path of the documentation file to generate")
ap.add_argument("defs", help="Builtin definition files", nargs="+")
return ap
lix/legacy/build-remote.cc
+242 -397
View File
@@ -1,23 +1,7 @@
#include "lix/libstore/path.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/error.hh"
#include "lix/libutil/file-descriptor.hh"
#include "lix/libutil/logging-rpc.hh"
#include "lix/libutil/logging.hh"
#include "lix/libutil/result.hh"
#include "lix/libutil/rpc.hh"
#include "lix/libutil/types-rpc.hh" // IWYU pragma: keep
#include "lix/libutil/types.hh"
#include <algorithm> #include <algorithm>
#include <capnp/rpc-twoparty.h> #include <chrono>
#include <cstring>
#include <exception>
#include <kj/async.h>
#include <kj/time.h>
#include <set> #include <set>
#include <memory> #include <memory>
#include <string>
#include <tuple> #include <tuple>
#if __APPLE__ #if __APPLE__
#include <sys/time.h> #include <sys/time.h>
@@ -33,29 +17,13 @@
#include "lix/libstore/derivations.hh" #include "lix/libstore/derivations.hh"
#include "lix/libutil/strings.hh" #include "lix/libutil/strings.hh"
#include "lix/libstore/local-store.hh" #include "lix/libstore/local-store.hh"
#include "lix/libstore/types-rpc.hh"
#include "lix/libcmd/legacy.hh" #include "lix/libcmd/legacy.hh"
#include "lix/libutil/experimental-features.hh" #include "lix/libutil/experimental-features.hh"
#include "lix/libutil/hash.hh" #include "lix/libutil/hash.hh"
#include "build-remote.hh" #include "build-remote.hh"
#include "lix/libstore/build/hook-instance.capnp.h"
namespace nix { namespace nix {
namespace {
struct Instance final : rpc::build_remote::HookInstance::Server
{
unsigned int maxBuildJobs;
bool initialized = false, used = false;
kj::Promise<void> init(InitContext context) override;
kj::Promise<Result<void>> buildImpl(BuildContext context);
kj::Promise<void> build(BuildContext context) override;
};
}
std::string escapeUri(std::string uri) std::string escapeUri(std::string uri)
{ {
std::replace(uri.begin(), uri.end(), '/', '_'); std::replace(uri.begin(), uri.end(), '/', '_');
@@ -70,7 +38,7 @@ static std::string makeLockFilename(const std::string & storeUri) {
// This avoids issues with the escaped URI being very long and causing // This avoids issues with the escaped URI being very long and causing
// path too long errors, while also avoiding any possibility of collision // path too long errors, while also avoiding any possibility of collision
// caused by simple truncation. // caused by simple truncation.
auto hash = hashString(HashType::SHA256, storeUri).to_string(HashFormat::Base32, false); auto hash = hashString(HashType::SHA256, storeUri).to_string(Base::Base32, false);
return escapeUri(storeUri).substr(0, 48) + "-" + hash.substr(0, 16); return escapeUri(storeUri).substr(0, 48) + "-" + hash.substr(0, 16);
} }
@@ -85,206 +53,11 @@ static bool allSupportedLocally(Store & store, const std::set<std::string>& requ
return true; return true;
} }
static std::tuple<bool, Machine *, AutoCloseFD> selectBestMachine(
Machines & machines,
const std::string & neededSystem,
const std::set<std::string> & requiredFeatures
)
{
bool rightType = false;
Machine * bestMachine = nullptr;
AutoCloseFD bestSlotLock;
uint64_t bestLoad = 0;
for (auto & m : machines) {
debug("considering building on remote machine '%s'", m.name);
if (m.enabled && m.systemSupported(neededSystem) && m.allSupported(requiredFeatures)
&& m.mandatoryMet(requiredFeatures))
{
rightType = true;
AutoCloseFD free;
uint64_t load = 0;
for (uint64_t slot = 0; slot < m.maxJobs; ++slot) {
auto slotLock = openSlotLock(m, slot);
if (tryLockFile(slotLock.get(), ltWrite)) {
if (!free) {
free = std::move(slotLock);
}
} else {
++load;
}
}
if (!free) {
continue;
}
bool best = false;
if (!bestSlotLock) {
best = true;
} else if (load / m.speedFactor < bestLoad / bestMachine->speedFactor) {
best = true;
} else if (load / m.speedFactor == bestLoad / bestMachine->speedFactor) {
if (m.speedFactor > bestMachine->speedFactor) {
best = true;
} else if (m.speedFactor == bestMachine->speedFactor) {
if (load < bestLoad) {
best = true;
}
}
}
if (best) {
bestLoad = load;
bestSlotLock = std::move(free);
bestMachine = &m;
}
}
}
return {rightType, bestMachine, std::move(bestSlotLock)};
}
static void printSelectionFailureMessage(
Verbosity level,
const std::string_view drvstr,
const Machines & machines,
const std::string & neededSystem,
const std::set<std::string> & requiredFeatures
)
{
std::string machinesFormatted;
for (auto & m : machines) {
machinesFormatted += HintFmt(
"\n([%s], %s, [%s], [%s])",
concatStringsSep<StringSet>(", ", m.systemTypes),
m.maxJobs,
concatStringsSep<StringSet>(", ", m.supportedFeatures),
concatStringsSep<StringSet>(", ", m.mandatoryFeatures)
)
.str();
}
printMsg(
level,
"Failed to find a machine for remote build!\n"
"derivation: %s\n"
"required (system, features): (%s, [%s])\n"
"%s available machines:\n"
"(systems, maxjobs, supportedFeatures, mandatoryFeatures)%s",
drvstr,
neededSystem,
concatStringsSep<StringSet>(", ", requiredFeatures),
machines.size(),
Uncolored(machinesFormatted)
);
}
namespace {
struct BuilderConnection
{
AutoCloseFD slotLock;
std::shared_ptr<Store> sshStore;
std::string storeUri;
};
struct AcceptedBuild final : rpc::build_remote::HookInstance::AcceptedBuild::Server
{
ref<Store> store;
StorePath drvPath;
BuilderConnection builder;
bool used = false;
AcceptedBuild(ref<Store> store, StorePath drvPath, BuilderConnection builder)
: store(store)
, drvPath(drvPath)
, builder(std::move(builder))
{
}
kj::Promise<Result<void>> runImpl(RunContext context);
kj::Promise<void> run(RunContext context) override;
};
enum class BuildRejected { Temporarily, Permanently };
}
static kj::Promise<Result<std::variant<BuildRejected, BuilderConnection>>> connectToBuilder(
const ref<Store> & store,
const StorePath & drvPath,
Machines & machines,
const unsigned int maxBuildJobs,
const bool amWilling,
const std::string & neededSystem,
const std::set<std::string> & requiredFeatures
)
try {
AutoCloseFD bestSlotLock;
/* It would be possible to build locally after some builds clear out,
so don't show the warning now: */
bool couldBuildLocally = maxBuildJobs > 0
&& (neededSystem == settings.thisSystem
|| settings.extraPlatforms.get().count(neededSystem) > 0)
&& allSupportedLocally(*store, requiredFeatures);
/* It's possible to build this locally right now: */
bool canBuildLocally = amWilling && couldBuildLocally;
/* Error ignored here, will be caught later */
(void) sys::mkdir(currentLoad, 0777);
while (true) {
bestSlotLock.reset();
AutoCloseFD lock = openLockFile(currentLoad + "/main-lock", true);
TRY_AWAIT(lockFileAsync(lock.get(), ltWrite));
auto [rightType, bestMachine, slotLock] =
selectBestMachine(machines, neededSystem, requiredFeatures);
bestSlotLock = std::move(slotLock);
if (!bestSlotLock) {
if (rightType && !canBuildLocally) {
co_return BuildRejected::Temporarily;
} else {
printSelectionFailureMessage(
couldBuildLocally ? lvlChatty : lvlWarn,
drvPath.to_string(),
machines,
neededSystem,
requiredFeatures
);
co_return BuildRejected::Permanently;
}
}
#if __APPLE__
futimes(bestSlotLock.get(), nullptr);
#else
futimens(bestSlotLock.get(), nullptr);
#endif
lock.reset();
std::shared_ptr<Store> sshStore;
try {
auto act =
logger->startActivity(lvlTalkative, actUnknown, fmt("connecting to '%s'", bestMachine->name));
sshStore = TRY_AWAIT(bestMachine->openStore());
co_return BuilderConnection{std::move(bestSlotLock), sshStore, bestMachine->storeUri};
} catch (std::exception & e) { // NOLINT(lix-foreign-exceptions)
printError("cannot build on '%s': %s", bestMachine->name, e.what());
bestMachine->enabled = false;
}
}
} catch (...) {
co_return result::current_exception();
}
static int main_build_remote(AsyncIoRoot & aio, std::string programName, Strings argv) static int main_build_remote(AsyncIoRoot & aio, std::string programName, Strings argv)
{ {
{ {
logger = makeJSONLogger(*logger);
/* Ensure we don't get any SSH passphrase or host key popups. */ /* Ensure we don't get any SSH passphrase or host key popups. */
unsetenv("DISPLAY"); unsetenv("DISPLAY");
unsetenv("SSH_ASKPASS"); unsetenv("SSH_ASKPASS");
@@ -295,180 +68,236 @@ static int main_build_remote(AsyncIoRoot & aio, std::string programName, Strings
if (argv.size() != 1) if (argv.size() != 1)
throw UsageError("called without required arguments"); throw UsageError("called without required arguments");
setVerbosity((Verbosity) std::stoll(argv.front())); verbosity = (Verbosity) std::stoll(argv.front());
auto conn = aio.kj.lowLevelProvider->wrapUnixSocketFd(1); FdSource source(STDIN_FILENO);
capnp::TwoPartyServer srv(kj::heap<Instance>());
srv.accept(*conn, 1).wait(aio.kj.waitScope);
return 0;
}
}
kj::Promise<void> Instance::init(InitContext context)
{
try {
if (initialized) {
throw Error("build hook can only be initialized once");
}
logger = rpc::log::makeRpcLoggerClient(context.getParams().getLogger());
/* Read the parent's settings. */ /* Read the parent's settings. */
for (const auto & [name, value] : rpc::to<StringMap>(context.getParams().getSettings())) { while (readInt(source)) {
auto name = readString(source);
auto value = readString(source);
settings.set(name, value); settings.set(name, value);
} }
maxBuildJobs = settings.maxBuildJobs; auto maxBuildJobs = settings.maxBuildJobs;
settings.maxBuildJobs.set("1"); // hack to make tests with local?root= work settings.maxBuildJobs.set("1"); // hack to make tests with local?root= work
initPlugins(); initPlugins();
initialized = true; auto store = aio.blockOn(openStore());
} catch (...) {
rpc::rethrow_as_rpc_error();
}
return kj::READY_NOW; /* It would be more appropriate to use $XDG_RUNTIME_DIR, since
} that gets cleared on reboot, but it wouldn't work on macOS. */
auto currentLoadName = "/current-load";
if (auto localStore = store.try_cast_shared<LocalFSStore>())
currentLoad = std::string { localStore->config().stateDir } + currentLoadName;
else
currentLoad = settings.nixStateDir + currentLoadName;
kj::Promise<Result<void>> Instance::buildImpl(BuildContext context) std::shared_ptr<Store> sshStore;
try { AutoCloseFD bestSlotLock;
if (!initialized) {
throw Error("build hook not fully initialized");
}
// FIXME this does not open a daemon connection for historical reasons. auto machines = getMachines();
// we may create a lot of build hook instances, and having each of them debug("got %d remote builders", machines.size());
// also create a daemon instance is inefficient and wasteful. in future
// versions of the build hook (where we don't need one hook process per
// build) we should change this to using a daemon connection, ideally a
// daemon connection provided by the parent via file descriptor passing
auto store = TRY_AWAIT(openStore(settings.storeUri, {}, AllowDaemon::Disallow));
/* It would be more appropriate to use $XDG_RUNTIME_DIR, since if (machines.empty()) {
that gets cleared on reboot, but it wouldn't work on macOS. */ std::cerr << "# decline-permanently\n";
auto currentLoadName = "/current-load"; return 0;
if (auto localStore = store.try_cast_shared<LocalFSStore>()) {
currentLoad = std::string{localStore->config().stateDir} + currentLoadName;
} else {
currentLoad = settings.nixStateDir + currentLoadName;
}
auto machines = getMachines();
debug("got %d remote builders", machines.size());
if (machines.empty()) {
context.getResults().initResult().setDeclinePermanently();
co_return result::success();
}
auto amWilling = context.getParams().getAmWilling();
auto neededSystem = rpc::to<std::string>(context.getParams().getNeededSystem());
auto drvPath = from(context.getParams().getDrvPath(), *store);
auto requiredFeatures =
rpc::to<std::set<std::string>>(context.getParams().getRequiredFeatures());
auto result = TRY_AWAIT(connectToBuilder(
store, drvPath, machines, maxBuildJobs, amWilling, neededSystem, requiredFeatures
));
if (auto immediateResponse = std::get_if<BuildRejected>(&result)) {
switch (*immediateResponse) {
case BuildRejected::Temporarily:
context.getResults().initResult().setPostpone();
co_return result::success();
case BuildRejected::Permanently:
context.getResults().initResult().setDecline();
co_return result::success();
} }
}
auto builder = std::get_if<BuilderConnection>(&result); std::optional<StorePath> drvPath;
assert(builder); std::string storeUri;
auto ac = context.getResults().initResult().initAccept(); while (true) {
ac.setMachine(kj::heap<AcceptedBuild>(store, drvPath, std::move(*builder)));
co_return result::success(); try {
} catch (...) { auto s = readString(source);
co_return result::current_exception(); if (s != "try") return 0;
} } catch (EndOfFile &) { return 0; }
kj::Promise<void> Instance::build(BuildContext context) auto amWilling = readInt(source);
try { auto neededSystem = readString(source);
if (used) { drvPath = store->parseStorePath(readString(source));
throw Error("build hooks can only accept a single job"); auto requiredFeatures = readStrings<std::set<std::string>>(source);
}
used = true; // lock out other rpc calls during processing
auto result = co_await buildImpl(context);
TRY_AWAIT(logger->flush());
used = result.has_value() && context.getResults().getResult().isAccept();
result.value();
} catch (...) {
rpc::rethrow_as_rpc_error();
}
kj::Promise<void> AcceptedBuild::run(RunContext context) /* It would be possible to build locally after some builds clear out,
{ so don't show the warning now: */
try { bool couldBuildLocally = maxBuildJobs > 0
auto oldLogger = logger; && ( neededSystem == settings.thisSystem
logger = rpc::log::makeRpcLoggerClient(context.getParams().getLogger()); || settings.extraPlatforms.get().count(neededSystem) > 0)
TRY_AWAIT(oldLogger->flush()); && allSupportedLocally(*store, requiredFeatures);
KJ_DEFER({ /* It's possible to build this locally right now: */
delete logger; bool canBuildLocally = amWilling && couldBuildLocally;
logger = oldLogger;
});
if (used) { /* Error ignored here, will be caught later */
throw Error("build hooks builds are single-use items"); mkdir(currentLoad.c_str(), 0777);
while (true) {
bestSlotLock.reset();
AutoCloseFD lock = openLockFile(currentLoad + "/main-lock", true);
lockFile(lock.get(), ltWrite);
bool rightType = false;
Machine * bestMachine = nullptr;
uint64_t bestLoad = 0;
for (auto & m : machines) {
debug("considering building on remote machine '%s'", m.storeUri);
if (m.enabled &&
m.systemSupported(neededSystem) &&
m.allSupported(requiredFeatures) &&
m.mandatoryMet(requiredFeatures))
{
rightType = true;
AutoCloseFD free;
uint64_t load = 0;
for (uint64_t slot = 0; slot < m.maxJobs; ++slot) {
auto slotLock = openSlotLock(m, slot);
if (tryLockFile(slotLock.get(), ltWrite)) {
if (!free) {
free = std::move(slotLock);
}
} else {
++load;
}
}
if (!free) {
continue;
}
bool best = false;
if (!bestSlotLock) {
best = true;
} else if (load / m.speedFactor < bestLoad / bestMachine->speedFactor) {
best = true;
} else if (load / m.speedFactor == bestLoad / bestMachine->speedFactor) {
if (m.speedFactor > bestMachine->speedFactor) {
best = true;
} else if (m.speedFactor == bestMachine->speedFactor) {
if (load < bestLoad) {
best = true;
}
}
}
if (best) {
bestLoad = load;
bestSlotLock = std::move(free);
bestMachine = &m;
}
}
}
if (!bestSlotLock) {
if (rightType && !canBuildLocally)
std::cerr << "# postpone\n";
else
{
// add the template values.
std::string drvstr;
if (drvPath.has_value())
drvstr = drvPath->to_string();
else
drvstr = "<unknown>";
std::string machinesFormatted;
for (auto & m : machines) {
machinesFormatted += HintFmt(
"\n([%s], %s, [%s], [%s])",
concatStringsSep<StringSet>(", ", m.systemTypes),
m.maxJobs,
concatStringsSep<StringSet>(", ", m.supportedFeatures),
concatStringsSep<StringSet>(", ", m.mandatoryFeatures)
).str();
}
auto error = HintFmt(
"Failed to find a machine for remote build!\n"
"derivation: %s\n"
"required (system, features): (%s, [%s])\n"
"%s available machines:\n"
"(systems, maxjobs, supportedFeatures, mandatoryFeatures)%s",
drvstr,
neededSystem,
concatStringsSep<StringSet>(", ", requiredFeatures),
machines.size(),
Uncolored(machinesFormatted)
);
printMsg(couldBuildLocally ? lvlChatty : lvlWarn, error.str());
std::cerr << "# decline\n";
}
break;
}
#if __APPLE__
futimes(bestSlotLock.get(), nullptr);
#else
futimens(bestSlotLock.get(), nullptr);
#endif
lock.reset();
try {
Activity act(*logger, lvlTalkative, actUnknown, fmt("connecting to '%s'", bestMachine->storeUri));
sshStore = aio.blockOn(bestMachine->openStore());
aio.blockOn(sshStore->connect());
storeUri = bestMachine->storeUri;
} catch (std::exception & e) { // NOLINT(lix-foreign-exceptions)
auto msg = chomp(drainFD(5, false));
printError("cannot build on '%s': %s%s",
bestMachine->storeUri, e.what(),
msg.empty() ? "" : ": " + msg);
bestMachine->enabled = false;
continue;
}
goto connected;
}
} }
used = true;
auto result = co_await runImpl(context);
TRY_AWAIT(logger->flush());
result.value();
} catch (...) {
rpc::rethrow_as_rpc_error();
}
}
kj::Promise<Result<void>> AcceptedBuild::runImpl(RunContext context) connected:
{ close(5);
try {
auto & sshStore = builder.sshStore;
auto & storeUri = builder.storeUri;
auto inputs = rpc::to<std::set<StorePath>>(context.getParams().getInputs(), *store); assert(sshStore);
auto wantedOutputs = rpc::to<std::set<std::string>>(context.getParams().getWantedOutputs());
std::cerr << "# accept\n" << storeUri << "\n";
auto inputs = readStrings<PathSet>(source);
auto wantedOutputs = readStrings<StringSet>(source);
auto lockFileName = currentLoad + "/" + makeLockFilename(storeUri) + ".upload-lock"; auto lockFileName = currentLoad + "/" + makeLockFilename(storeUri) + ".upload-lock";
AutoCloseFD uploadLock = openLockFile(lockFileName, true); AutoCloseFD uploadLock = openLockFile(lockFileName, true);
{ {
auto act = logger->startActivity( Activity act(*logger, lvlTalkative, actUnknown, fmt("waiting for the upload lock to '%s'", storeUri));
lvlTalkative, actUnknown, fmt("waiting for the upload lock to '%s'", storeUri)
);
auto result = TRY_AWAIT( if (!unsafeLockFileSingleThreaded(uploadLock.get(), ltWrite, std::chrono::minutes(15)))
AIO().timeoutAfter(15 * kj::MINUTES, lockFileAsync(uploadLock.get(), ltWrite)) printError("somebody is hogging the upload lock for '%s', continuing...");
);
if (!result) {
printError("somebody is hogging the upload lock for '%s', continuing...", storeUri);
}
} }
auto substitute = settings.buildersUseSubstitutes ? Substitute : NoSubstitute; auto substitute = settings.buildersUseSubstitutes ? Substitute : NoSubstitute;
{ {
auto act = logger->startActivity( Activity act(*logger, lvlTalkative, actUnknown, fmt("copying dependencies to '%s'", storeUri));
lvlTalkative, actUnknown, fmt("copying dependencies to '%s'", storeUri) aio.blockOn(copyPaths(
); *store,
TRY_AWAIT(copyPaths(*store, *sshStore, inputs, NoRepair, NoCheckSigs, substitute)); *sshStore,
store->parseStorePathSet(inputs),
NoRepair,
NoCheckSigs,
substitute
));
} }
uploadLock.reset(); uploadLock.reset();
auto drv = TRY_AWAIT(store->readDerivation(drvPath)); auto drv = aio.blockOn(store->readDerivation(*drvPath));
std::optional<BuildResult> optResult; std::optional<BuildResult> optResult;
@@ -476,7 +305,7 @@ kj::Promise<Result<void>> AcceptedBuild::runImpl(RunContext context)
// stores), we assume we are. This is necessary for backwards // stores), we assume we are. This is necessary for backwards
// compat. // compat.
bool trustedOrLegacy = ({ bool trustedOrLegacy = ({
std::optional trusted = TRY_AWAIT(sshStore->isTrustedClient()); std::optional trusted = aio.blockOn(sshStore->isTrustedClient());
!trusted || *trusted; !trusted || *trusted;
}); });
@@ -495,57 +324,73 @@ kj::Promise<Result<void>> AcceptedBuild::runImpl(RunContext context)
// //
// 2. Changing the `inputSrcs` set changes the associated // 2. Changing the `inputSrcs` set changes the associated
// output ids, which break CA derivations // output ids, which break CA derivations
if (!drv.inputDrvs.empty()) { if (!drv.inputDrvs.map.empty())
drv.inputSrcs = inputs; drv.inputSrcs = store->parseStorePathSet(inputs);
} optResult = aio.blockOn(sshStore->buildDerivation(*drvPath, (const BasicDerivation &) drv));
optResult = auto & result = *optResult;
TRY_AWAIT(sshStore->buildDerivation(drvPath, (const BasicDerivation &) drv)); if (!result.success())
throw Error("build of '%s' on '%s' failed: %s", store->printStorePath(*drvPath), storeUri, result.errorMsg);
} else { } else {
TRY_AWAIT(copyClosure( aio.blockOn(copyClosure(
*store, *sshStore, StorePathSet{drvPath}, NoRepair, NoCheckSigs, substitute *store, *sshStore, StorePathSet{*drvPath}, NoRepair, NoCheckSigs, substitute
)); ));
auto res = TRY_AWAIT(sshStore->buildPathsWithResults({DerivedPath::Built{ auto res = aio.blockOn(sshStore->buildPathsWithResults({
.drvPath = makeConstantStorePath(drvPath), DerivedPath::Built {
.outputs = OutputsSpec::All{}, .drvPath = makeConstantStorePathRef(*drvPath),
}})); .outputs = OutputsSpec::All {},
}
}));
// One path to build should produce exactly one build result // One path to build should produce exactly one build result
assert(res.size() == 1); assert(res.size() == 1);
optResult = std::move(res[0]); optResult = std::move(res[0]);
} }
auto & result = *optResult;
if (!result.success()) {
throw Error(
"build of '%s' on '%s' failed: %s",
store->printStorePath(drvPath),
storeUri,
result.errorMsg
);
}
auto outputHashes = aio.blockOn(staticOutputHashes(*store, drv));
std::set<Realisation> missingRealisations;
StorePathSet missingPaths; StorePathSet missingPaths;
auto outputPaths = drv.outputsAndPaths(*store); if (experimentalFeatureSettings.isEnabled(Xp::CaDerivations) && !drv.type().hasKnownOutputPaths()) {
for (auto & [outputName, outputPath] : outputPaths) { for (auto & outputName : wantedOutputs) {
if (!TRY_AWAIT(store->isValidPath(outputPath.second))) { auto thisOutputHash = outputHashes.at(outputName);
missingPaths.insert(outputPath.second); auto thisOutputId = DrvOutput{ thisOutputHash, outputName };
if (!aio.blockOn(store->queryRealisation(thisOutputId))) {
debug("missing output %s", outputName);
assert(optResult);
auto & result = *optResult;
auto i = result.builtOutputs.find(outputName);
assert(i != result.builtOutputs.end());
auto & newRealisation = i->second;
missingRealisations.insert(newRealisation);
missingPaths.insert(newRealisation.outPath);
}
}
} else {
auto outputPaths = drv.outputsAndOptPaths(*store);
for (auto & [outputName, hopefullyOutputPath] : outputPaths) {
assert(hopefullyOutputPath.second);
if (!aio.blockOn(store->isValidPath(*hopefullyOutputPath.second)))
missingPaths.insert(*hopefullyOutputPath.second);
} }
} }
if (!missingPaths.empty()) { if (!missingPaths.empty()) {
auto act = logger->startActivity( Activity act(*logger, lvlTalkative, actUnknown, fmt("copying outputs from '%s'", storeUri));
lvlTalkative, actUnknown, fmt("copying outputs from '%s'", storeUri)
);
if (auto localStore = store.try_cast_shared<LocalStore>()) if (auto localStore = store.try_cast_shared<LocalStore>())
for (auto & path : missingPaths) for (auto & path : missingPaths)
localStore->locksHeld.insert(store->printStorePath(path)); /* FIXME: ugly */ localStore->locksHeld.insert(store->printStorePath(path)); /* FIXME: ugly */
TRY_AWAIT( aio.blockOn(
copyPaths(*sshStore, *store, missingPaths, NoRepair, NoCheckSigs, NoSubstitute) copyPaths(*sshStore, *store, missingPaths, NoRepair, NoCheckSigs, NoSubstitute)
); );
} }
// XXX: Should be done as part of `copyPaths`
for (auto & realisation : missingRealisations) {
// Should hold, because if the feature isn't enabled the set
// of missing realisations should be empty
experimentalFeatureSettings.require(Xp::CaDerivations);
aio.blockOn(store->registerDrvOutput(realisation));
}
co_return result::success(); return 0;
} catch (...) {
co_return result::current_exception();
} }
} }
lix/legacy/builtin-builder.cc
-103
View File
@@ -1,103 +0,0 @@
#include "lix/libcmd/legacy.hh"
#include "lix/libstore/builtins.hh"
#include "lix/libstore/builtins/buildenv.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/error.hh"
#include "lix/libutil/file-system.hh"
#include "lix/libutil/logging.hh"
#include "lix/libutil/strings.hh"
#include "lix/libutil/types.hh"
#include <string_view>
using std::literals::operator""sv;
namespace nix {
static int main_builtin_builder(AsyncIoRoot & aio, std::string programName, Strings argv)
{
logger = makeJSONLogger(*logger);
std::map<std::string, std::string> env;
auto argvIt = argv.begin();
const auto argvEnd = argv.end();
// we do not use the argument parsing functions we have in libmain here, neither
// the legacy versions nor the newer ones. the legacy version could work, but we
// want to provide two sets of arguments separated by `--` and would need rather
// unpleasant state handling to use the legacy parser. the more modern parser is
// entirely incapable of doing this for us since it's all statically configured.
const auto getArg = [&](std::string_view desc) {
if (argvIt == argvEnd) {
throw Error("expected a value for %s", desc);
}
return *argvIt++;
};
if (auto val = string2Int<int>(getArg("verbosity"))) {
setVerbosity(verbosityFromIntClamped(*val));
} else {
throw Error("expected a verbosity argument");
}
while (argvIt != argvEnd) {
const auto arg = getArg("option");
if (arg == "--") {
break;
} else if (!arg.starts_with("--")) {
throw Error("unexpected builtin option %s", arg);
}
auto value = unescapeNul(getArg(arg));
globalConfig.set(arg.substr(2), value);
}
while (argvIt != argvEnd) {
const auto key = getArg("builder argument");
if (!key.starts_with("--")) {
throw Error("unexpected builtin builder argument %s", key);
}
env[unescapeNul(key.substr(2))] = unescapeNul(getArg(key));
}
auto getAttr = [&](const std::string & name) {
auto i = env.find(name);
if (i == env.end()) {
throw Error("attribute '%s' missing", name);
}
return i->second;
};
const auto builder = getAttr("builder");
if (builder == "builtin:fetchurl") {
const auto outputHashMode = getAttr("outputHashMode");
const auto hash = outputHashMode == "flat" ? [&] -> std::optional<Hash> {
const auto ht = parseHashTypeOpt(getAttr("outputHashAlgo"));
return newHashAllowEmpty(getAttr("outputHash"), ht);
}()
: std::nullopt;
BuiltinFetchurl{
.storePath = getAttr("out"),
.mainUrl = getAttr("url"),
.unpack = getOr(env, "unpack", "0") == "1",
.executable = getOr(env, "executable", "0") == "1",
.hash = hash,
}
.run(aio);
} else if (builder == "builtin:buildenv") {
builtinBuildenv(getAttr("out"), tokenizeString<Strings>(getAttr("derivations")), getAttr("manifest"));
} else if (builder == "builtin:unpack-channel") {
builtinUnpackChannel(getAttr("out"), getAttr("channelName"), getAttr("src"));
} else {
throw Error("unknown builtin builder %s", builder);
}
return 0;
}
void registerLegacyBuiltinBuilder()
{
LegacyCommandRegistry::add("builtin-builder", main_builtin_builder);
}
}
lix/legacy/builtin-builder.hh
-6
View File
@@ -1,6 +0,0 @@
#pragma once
///@file
namespace nix {
void registerLegacyBuiltinBuilder();
}
lix/legacy/dotgraph.cc
+10 -8
View File
@@ -4,7 +4,9 @@
#include "lix/libutil/result.hh" #include "lix/libutil/result.hh"
#include <iostream> #include <iostream>
#include <sstream>
using std::cout;
namespace nix { namespace nix {
@@ -40,31 +42,31 @@ static std::string makeNode(std::string_view id, std::string_view label,
dotQuote(id), dotQuote(label), dotQuote(colour)); dotQuote(id), dotQuote(label), dotQuote(colour));
} }
kj::Promise<Result<std::string>> formatDotGraph(ref<Store> store, StorePathSet && roots)
kj::Promise<Result<void>> printDotGraph(ref<Store> store, StorePathSet && roots)
try { try {
StorePathSet workList(std::move(roots)); StorePathSet workList(std::move(roots));
StorePathSet doneSet; StorePathSet doneSet;
std::stringstream result;
result << "digraph G {\n"; cout << "digraph G {\n";
while (!workList.empty()) { while (!workList.empty()) {
auto path = std::move(workList.extract(workList.begin()).value()); auto path = std::move(workList.extract(workList.begin()).value());
if (!doneSet.insert(path).second) continue; if (!doneSet.insert(path).second) continue;
result << makeNode(std::string(path.to_string()), path.name(), "#ff0000"); cout << makeNode(std::string(path.to_string()), path.name(), "#ff0000");
for (auto & p : TRY_AWAIT(store->queryPathInfo(path))->references) { for (auto & p : TRY_AWAIT(store->queryPathInfo(path))->references) {
if (p != path) { if (p != path) {
workList.insert(p); workList.insert(p);
result << makeEdge(std::string(p.to_string()), std::string(path.to_string())); cout << makeEdge(std::string(p.to_string()), std::string(path.to_string()));
} }
} }
} }
result << "}\n"; cout << "}\n";
co_return result.str(); co_return result::success();
} catch (...) { } catch (...) {
co_return result::current_exception(); co_return result::current_exception();
} }
lix/legacy/dotgraph.hh
+2 -1
View File
@@ -5,5 +5,6 @@
namespace nix { namespace nix {
kj::Promise<Result<std::string>> formatDotGraph(ref<Store> store, StorePathSet && roots); kj::Promise<Result<void>> printDotGraph(ref<Store> store, StorePathSet && roots);
} }
lix/legacy/graphml.cc
+18 -16
View File
@@ -5,7 +5,9 @@
#include "lix/libutil/result.hh" #include "lix/libutil/result.hh"
#include <iostream> #include <iostream>
#include <sstream>
using std::cout;
namespace nix { namespace nix {
@@ -45,21 +47,21 @@ static std::string makeNode(const ValidPathInfo & info)
(info.path.isDerivation() ? "derivation" : "output-path")); (info.path.isDerivation() ? "derivation" : "output-path"));
} }
kj::Promise<Result<std::string>> formatGraphML(ref<Store> store, StorePathSet && roots)
kj::Promise<Result<void>> printGraphML(ref<Store> store, StorePathSet && roots)
try { try {
StorePathSet workList(std::move(roots)); StorePathSet workList(std::move(roots));
StorePathSet doneSet; StorePathSet doneSet;
std::pair<StorePathSet::iterator, bool> ret; std::pair<StorePathSet::iterator, bool> ret;
std::stringstream result;
result << "<?xml version='1.0' encoding='utf-8'?>\n" cout << "<?xml version='1.0' encoding='utf-8'?>\n"
<< "<graphml xmlns='http://graphml.graphdrawing.org/xmlns'\n" << "<graphml xmlns='http://graphml.graphdrawing.org/xmlns'\n"
<< " xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'\n" << " xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'\n"
<< " xsi:schemaLocation='http://graphml.graphdrawing.org/xmlns/1.0/graphml.xsd'>\n" << " xsi:schemaLocation='http://graphml.graphdrawing.org/xmlns/1.0/graphml.xsd'>\n"
<< "<key id='narSize' for='node' attr.name='narSize' attr.type='long'/>" << "<key id='narSize' for='node' attr.name='narSize' attr.type='long'/>"
<< "<key id='name' for='node' attr.name='name' attr.type='string'/>" << "<key id='name' for='node' attr.name='name' attr.type='string'/>"
<< "<key id='type' for='node' attr.name='type' attr.type='string'/>" << "<key id='type' for='node' attr.name='type' attr.type='string'/>"
<< "<graph id='G' edgedefault='directed'>\n"; << "<graph id='G' edgedefault='directed'>\n";
while (!workList.empty()) { while (!workList.empty()) {
auto path = std::move(workList.extract(workList.begin()).value()); auto path = std::move(workList.extract(workList.begin()).value());
@@ -68,20 +70,20 @@ try {
if (ret.second == false) continue; if (ret.second == false) continue;
auto info = TRY_AWAIT(store->queryPathInfo(path)); auto info = TRY_AWAIT(store->queryPathInfo(path));
result << makeNode(*info); cout << makeNode(*info);
for (auto & p : info->references) { for (auto & p : info->references) {
if (p != path) { if (p != path) {
workList.insert(p); workList.insert(p);
result << makeEdge(path.to_string(), p.to_string()); cout << makeEdge(path.to_string(), p.to_string());
} }
} }
} }
result << "</graph>\n"; cout << "</graph>\n";
result << "</graphml>\n"; cout << "</graphml>\n";
co_return result.str(); co_return result::success();
} catch (...) { } catch (...) {
co_return result::current_exception(); co_return result::current_exception();
} }
lix/legacy/graphml.hh
+2 -1
View File
@@ -5,5 +5,6 @@
namespace nix { namespace nix {
kj::Promise<Result<std::string>> formatGraphML(ref<Store> store, StorePathSet && roots); kj::Promise<Result<void>> printGraphML(ref<Store> store, StorePathSet && roots);
} }
lix/legacy/meson.build
-2
View File
@@ -4,7 +4,6 @@ legacy_sources = files(
# `build-remote` is not really legacy (it powers all remote builds), but it's # `build-remote` is not really legacy (it powers all remote builds), but it's
# not a `nix3` command. # not a `nix3` command.
'build-remote.cc', 'build-remote.cc',
'builtin-builder.cc',
'dotgraph.cc', 'dotgraph.cc',
'graphml.cc', 'graphml.cc',
'nix-build.cc', 'nix-build.cc',
@@ -20,7 +19,6 @@ legacy_sources = files(
legacy_headers = files( legacy_headers = files(
'build-remote.hh', 'build-remote.hh',
'builtin-builder.hh',
'nix-build.hh', 'nix-build.hh',
'nix-channel.hh', 'nix-channel.hh',
'nix-collect-garbage.hh', 'nix-collect-garbage.hh',
lix/legacy/nix-build.cc
+91 -95
View File
@@ -9,7 +9,6 @@
#include "lix/libstore/store-api.hh" #include "lix/libstore/store-api.hh"
#include "lix/libstore/local-fs-store.hh" #include "lix/libstore/local-fs-store.hh"
#include "lix/libstore/globals.hh" #include "lix/libstore/globals.hh"
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/current-process.hh" #include "lix/libutil/current-process.hh"
#include "lix/libstore/derivations.hh" #include "lix/libstore/derivations.hh"
#include "lix/libmain/shared.hh" #include "lix/libmain/shared.hh"
@@ -19,13 +18,10 @@
#include "lix/libcmd/common-eval-args.hh" #include "lix/libcmd/common-eval-args.hh"
#include "lix/libexpr/attr-path.hh" #include "lix/libexpr/attr-path.hh"
#include "lix/libcmd/legacy.hh" #include "lix/libcmd/legacy.hh"
#include "lix/libutil/finally.hh"
#include "lix/libutil/processes.hh"
#include "lix/libutil/regex.hh" #include "lix/libutil/regex.hh"
#include "lix/libutil/shlex.hh" #include "lix/libutil/shlex.hh"
#include "nix-build.hh" #include "nix-build.hh"
#include "lix/libstore/temporary-dir.hh" #include "lix/libstore/temporary-dir.hh"
#include "lix/libutil/strings.hh"
extern char * * environ __attribute__((weak)); // Man what even is this extern char * * environ __attribute__((weak)); // Man what even is this
@@ -33,7 +29,7 @@ namespace nix {
using namespace std::string_literals; using namespace std::string_literals;
static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings argv) static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings argv)
{ {
auto dryRun = false; auto dryRun = false;
auto runEnv = std::regex_search(programName, regex::parse("nix-shell$")); auto runEnv = std::regex_search(programName, regex::parse("nix-shell$"));
@@ -192,11 +188,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
throw UsageError("'-p' and '-E' are mutually exclusive"); throw UsageError("'-p' and '-E' are mutually exclusive");
AutoDelete tmpDir(createTempDir(myName)); AutoDelete tmpDir(createTempDir(myName));
// NOTE: we assume there's no `build-top` directory created inside of `tmpDir` and we have AutoDelete buildTopTmpDir(createTempSubdir(tmpDir, "build-top"));
// ownership of this.
auto buildTopTmpDir = tmpDir + "/build-top";
createDirs(buildTopTmpDir);
if (outLink.empty()) if (outLink.empty())
outLink = (Path) tmpDir + "/result"; outLink = (Path) tmpDir + "/result";
@@ -213,7 +205,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
auto autoArgsWithInNixShell = autoArgs; auto autoArgsWithInNixShell = autoArgs;
if (runEnv) { if (runEnv) {
auto newArgs = evaluator->buildBindings(autoArgsWithInNixShell->size() + 1); auto newArgs = evaluator->buildBindings(autoArgsWithInNixShell->size() + 1);
newArgs.insert("inNixShell", {NewValueAs::boolean, true}); newArgs.alloc("inNixShell").mkBool(true);
for (auto & i : *autoArgs) newArgs.insert(i); for (auto & i : *autoArgs) newArgs.insert(i);
autoArgsWithInNixShell = newArgs.finish(); autoArgsWithInNixShell = newArgs.finish();
} }
@@ -233,9 +225,8 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
left = {"default.nix"}; left = {"default.nix"};
} }
if (runEnv) { if (runEnv)
(void) sys::setenv("IN_NIX_SHELL", pure ? "pure" : "impure", 1); setenv("IN_NIX_SHELL", pure ? "pure" : "impure", 1);
}
DrvInfos drvs; DrvInfos drvs;
@@ -272,7 +263,8 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
if (attrPaths.empty()) attrPaths = {""}; if (attrPaths.empty()) attrPaths = {""};
for (auto e : exprs) { for (auto e : exprs) {
Value vRoot = state->eval(e); Value vRoot;
state->eval(e, vRoot);
std::function<bool(const Value & v)> takesNixShellAttr; std::function<bool(const Value & v)> takesNixShellAttr;
takesNixShellAttr = [&](const Value & v) { takesNixShellAttr = [&](const Value & v) {
@@ -281,7 +273,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
} }
bool add = false; bool add = false;
if (v.type() == nFunction) { if (v.type() == nFunction) {
if (auto pattern = dynamic_cast<AttrsPattern *>(v.lambda().fun->pattern.get())) { if (auto pattern = dynamic_cast<AttrsPattern *>(v.lambda.fun->pattern.get())) {
for (auto & i : pattern->formals) { for (auto & i : pattern->formals) {
if (evaluator->symbols[i.name] == "inNixShell") { if (evaluator->symbols[i.name] == "inNixShell") {
add = true; add = true;
@@ -294,12 +286,12 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
}; };
for (auto & i : attrPaths) { for (auto & i : attrPaths) {
Value v( Value & v(*findAlongAttrPath(
findAlongAttrPath( *state,
*state, i, takesNixShellAttr(vRoot) ? *autoArgsWithInNixShell : *autoArgs, vRoot i,
) takesNixShellAttr(vRoot) ? *autoArgsWithInNixShell : *autoArgs,
.first vRoot
); ).first);
state->forceValue(v, noPos); state->forceValue(v, noPos);
getDerivations( getDerivations(
*state, *state,
@@ -355,7 +347,8 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
"(import <nixpkgs> {}).bashInteractive", "(import <nixpkgs> {}).bashInteractive",
CanonPath::fromCwd()); CanonPath::fromCwd());
Value v = state->eval(expr); Value v;
state->eval(expr, v);
auto drv = getDerivation(*state, v, false); auto drv = getDerivation(*state, v, false);
if (!drv) if (!drv)
@@ -363,7 +356,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
auto bashDrv = drv->requireDrvPath(*state); auto bashDrv = drv->requireDrvPath(*state);
pathsToBuild.push_back(DerivedPath::Built { pathsToBuild.push_back(DerivedPath::Built {
.drvPath = makeConstantStorePath(bashDrv), .drvPath = makeConstantStorePathRef(bashDrv),
.outputs = OutputsSpec::Names {"out"}, .outputs = OutputsSpec::Names {"out"},
}); });
pathsToCopy.insert(bashDrv); pathsToCopy.insert(bashDrv);
@@ -376,16 +369,22 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
} }
} }
auto accumDerivedPath = [&](SingleDerivedPath::Opaque inputDrv, const StringSet & inputNode) { std::function<void(ref<SingleDerivedPath>, const DerivedPathMap<StringSet>::ChildNode &)> accumDerivedPath;
if (!inputNode.empty())
accumDerivedPath = [&](ref<SingleDerivedPath> inputDrv, const DerivedPathMap<StringSet>::ChildNode & inputNode) {
if (!inputNode.value.empty())
pathsToBuild.push_back(DerivedPath::Built { pathsToBuild.push_back(DerivedPath::Built {
.drvPath = inputDrv, .drvPath = inputDrv,
.outputs = OutputsSpec::Names { inputNode }, .outputs = OutputsSpec::Names { inputNode.value },
}); });
for (const auto & [outputName, childNode] : inputNode.childMap)
accumDerivedPath(
make_ref<SingleDerivedPath>(SingleDerivedPath::Built { inputDrv, outputName }),
childNode);
}; };
// Build or fetch all dependencies of the derivation. // Build or fetch all dependencies of the derivation.
for (const auto & [inputDrv0, inputNode] : drv.inputDrvs) { for (const auto & [inputDrv0, inputNode] : drv.inputDrvs.map) {
// To get around lambda capturing restrictions in the // To get around lambda capturing restrictions in the
// standard. // standard.
const auto & inputDrv = inputDrv0; const auto & inputDrv = inputDrv0;
@@ -394,7 +393,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
return !std::regex_search(store->printStorePath(inputDrv), regex::parse(exclude)); return !std::regex_search(store->printStorePath(inputDrv), regex::parse(exclude));
})) }))
{ {
accumDerivedPath(makeConstantStorePath(inputDrv), inputNode); accumDerivedPath(makeConstantStorePathRef(inputDrv), inputNode);
pathsToCopy.insert(inputDrv); pathsToCopy.insert(inputDrv);
} }
} }
@@ -405,14 +404,18 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
buildPaths(pathsToBuild); buildPaths(pathsToBuild);
if (dryRun) { if (dryRun) return;
return 0;
}
if (shellDrv) { if (shellDrv) {
auto shellDrvOutputs = auto shellDrvOutputs =
aio.blockOn(store->queryDerivationOutputMap(shellDrv.value(), &*evalStore)); aio.blockOn(store->queryPartialDerivationOutputMap(shellDrv.value(), &*evalStore));
shell = store->printStorePath(shellDrvOutputs.at("out")) + "/bin/bash"; shell = store->printStorePath(shellDrvOutputs.at("out").value()) + "/bin/bash";
}
if (experimentalFeatureSettings.isEnabled(Xp::CaDerivations)) {
auto resolvedDrv = aio.blockOn(drv.tryResolve(*store));
assert(resolvedDrv && "Successfully resolved the derivation");
drv = *resolvedDrv;
} }
// Set the environment. // Set the environment.
@@ -428,18 +431,6 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
env["__ETC_PROFILE_SOURCED"] = "1"; env["__ETC_PROFILE_SOURCED"] = "1";
} }
// Set NIX_SHELL_LEVEL
env["NIX_SHELL_LEVEL"] = std::to_string(
getEnvNonEmpty("NIX_SHELL_LEVEL")
.and_then([](std::string lvl) { return string2Int<size_t>(lvl); })
.value_or(0)
+ 1
);
// We re-export similarly to what occurs inside of a derivation goal `NIX_LOG_FD` to stderr.
// So that stdenv hooks that logs information can be observed inside this debugging tool.
env["NIX_LOG_FD"] = "2";
// Don't use defaultTempDir() here! We want to preserve the user's TMPDIR for the shell // Don't use defaultTempDir() here! We want to preserve the user's TMPDIR for the shell
env["NIX_BUILD_TOP"] = env["TMPDIR"] = env["TEMPDIR"] = env["TMP"] = env["TEMP"] = env["NIX_BUILD_TOP"] = env["TMPDIR"] = env["TEMPDIR"] = env["TMP"] = env["TEMP"] =
getEnvNonEmpty("TMPDIR").value_or(buildTopTmpDir); getEnvNonEmpty("TMPDIR").value_or(buildTopTmpDir);
@@ -448,33 +439,38 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
auto passAsFile = tokenizeString<StringSet>(getOr(drv.env, "passAsFile", "")); auto passAsFile = tokenizeString<StringSet>(getOr(drv.env, "passAsFile", ""));
bool keepTmp = false;
int fileNr = 0; int fileNr = 0;
for (auto & var : drv.env) for (auto & var : drv.env)
if (passAsFile.count(var.first)) { if (passAsFile.count(var.first)) {
keepTmp = true;
auto fn = ".attr-" + std::to_string(fileNr++); auto fn = ".attr-" + std::to_string(fileNr++);
Path p = (Path) tmpDir + "/" + fn; Path p = (Path) tmpDir + "/" + fn;
writeFile(p, var.second); writeFile(p, var.second);
env[var.first + "Path"] = p; env[var.first + "Path"] = p;
} else { } else
env[var.first] = var.second; env[var.first] = var.second;
}
std::string structuredAttrsRC; std::string structuredAttrsRC;
if (env.count("__json")) { if (env.count("__json")) {
StorePathSet inputs; StorePathSet inputs;
auto accumInputClosure = [&](const StorePath & inputDrv, const StringSet & inputNode) { std::function<void(const StorePath &, const DerivedPathMap<StringSet>::ChildNode &)> accumInputClosure;
accumInputClosure = [&](const StorePath & inputDrv, const DerivedPathMap<StringSet>::ChildNode & inputNode) {
auto outputs = auto outputs =
aio.blockOn(store->queryDerivationOutputMap(inputDrv, &*evalStore)); aio.blockOn(store->queryPartialDerivationOutputMap(inputDrv, &*evalStore));
for (auto & i : inputNode) { for (auto & i : inputNode.value) {
auto o = outputs.at(i); auto o = outputs.at(i);
aio.blockOn(store->computeFSClosure(o, inputs)); aio.blockOn(store->computeFSClosure(*o, inputs));
} }
for (const auto & [outputName, childNode] : inputNode.childMap)
accumInputClosure(*outputs.at(outputName), childNode);
}; };
for (const auto & [inputDrv, inputNode] : drv.inputDrvs) for (const auto & [inputDrv, inputNode] : drv.inputDrvs.map)
accumInputClosure(inputDrv, inputNode); accumInputClosure(inputDrv, inputNode);
ParsedDerivation parsedDrv(drvInfo.requireDrvPath(*state), drv); ParsedDerivation parsedDrv(drvInfo.requireDrvPath(*state), drv);
@@ -491,6 +487,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
env["NIX_ATTRS_SH_FILE"] = attrsSH; env["NIX_ATTRS_SH_FILE"] = attrsSH;
env["NIX_ATTRS_JSON_FILE"] = attrsJSON; env["NIX_ATTRS_JSON_FILE"] = attrsJSON;
keepTmp = true;
} }
} }
@@ -500,13 +497,24 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
lose the current $PATH directories. */ lose the current $PATH directories. */
auto rcfile = (Path) tmpDir + "/rc"; auto rcfile = (Path) tmpDir + "/rc";
auto tz = getEnv("TZ"); auto tz = getEnv("TZ");
std::string rc = std::string rc = fmt(
fmt("%1%" R"(_nix_shell_clean_tmpdir() { command rm -rf %1%; }; )"
// always clear PATH. "%2%"
// when nix-shell is run impure, we rehydrate it with the `p=$PATH` above "%3%"
"unset PATH;" // always clear PATH.
"dontAddDisableDepTrack=1;\n", // when nix-shell is run impure, we rehydrate it with the `p=$PATH` above
(pure ? "" : "[ -n \"$PS1\" ] && [ -e ~/.bashrc ] && source ~/.bashrc; p=$PATH; ")); "unset PATH;"
"dontAddDisableDepTrack=1;\n",
shellEscape(tmpDir),
(keepTmp
? "trap _nix_shell_clean_tmpdir EXIT; "
"exitHooks+=(_nix_shell_clean_tmpdir); "
"failureHooks+=(_nix_shell_clean_tmpdir); "
: "_nix_shell_clean_tmpdir; "),
(pure
? ""
: "[ -n \"$PS1\" ] && [ -e ~/.bashrc ] && source ~/.bashrc; p=$PATH; ")
);
rc += structuredAttrsRC; rc += structuredAttrsRC;
rc += fmt( rc += fmt(
"\n[ -e $stdenv/setup ] && source $stdenv/setup; " "\n[ -e $stdenv/setup ] && source $stdenv/setup; "
@@ -536,37 +544,27 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
vomit("Sourcing nix-shell with file %s and contents:\n%s", rcfile, rc); vomit("Sourcing nix-shell with file %s and contents:\n%s", rcfile, rc);
writeFile(rcfile, rc); writeFile(rcfile, rc);
auto args = interactive ? Strings{"--rcfile", rcfile} : Strings{rcfile}; Strings envStrs;
for (auto & i : env)
envStrs.push_back(i.first + "=" + i.second);
auto args = interactive
? Strings{"bash", "--rcfile", rcfile}
: Strings{"bash", rcfile};
auto envPtrs = stringsToCharPtrs(envStrs);
environ = envPtrs.data();
auto argPtrs = stringsToCharPtrs(args);
restoreProcessContext();
// We are going to run an interactive command, do not let the logger send a line.
logger->pause(); logger->pause();
printMsg(lvlChatty, "running shell: %s", concatMapStringsSep(" ", args, shellEscape)); execvp(shell->c_str(), argPtrs.data());
RunningProgram proc = runProgram2({ throw SysError("executing shell '%s'", *shell);
.program = *shell,
.searchPath = true,
.args = args,
.environment = env,
});
// NOTE: we wait and return the status check immediately.
// If there's interruption, we will swallow it and wait again for termination.
auto toExitStatus = [](int waitRes) {
if (WIFEXITED(waitRes)) {
return WEXITSTATUS(waitRes);
} else if (WIFSIGNALED(waitRes)) {
return 128 + WTERMSIG(waitRes);
} else {
return 255;
}
};
try {
return toExitStatus(proc.wait());
} catch (Interrupted &) {
return toExitStatus(proc.wait());
}
} }
else { else {
@@ -585,7 +583,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
throw Error("derivation '%s' lacks an 'outputName' attribute", store->printStorePath(drvPath)); throw Error("derivation '%s' lacks an 'outputName' attribute", store->printStorePath(drvPath));
pathsToBuild.push_back(DerivedPath::Built{ pathsToBuild.push_back(DerivedPath::Built{
.drvPath = makeConstantStorePath(drvPath), .drvPath = makeConstantStorePathRef(drvPath),
.outputs = OutputsSpec::Names{outputName}, .outputs = OutputsSpec::Names{outputName},
}); });
pathsToBuildOrdered.push_back({drvPath, {outputName}}); pathsToBuildOrdered.push_back({drvPath, {outputName}});
@@ -600,9 +598,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
buildPaths(pathsToBuild); buildPaths(pathsToBuild);
if (dryRun) { if (dryRun) return;
return 0;
}
std::vector<StorePath> outPaths; std::vector<StorePath> outPaths;
@@ -613,9 +609,11 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
drvPrefix += fmt("-%d", counter + 1); drvPrefix += fmt("-%d", counter + 1);
auto builtOutputs = auto builtOutputs =
aio.blockOn(store->queryDerivationOutputMap(drvPath, &*evalStore)); aio.blockOn(store->queryPartialDerivationOutputMap(drvPath, &*evalStore));
auto outputPath = builtOutputs.at(outputName); auto maybeOutputPath = builtOutputs.at(outputName);
assert(maybeOutputPath);
auto outputPath = *maybeOutputPath;
if (auto store2 = store.try_cast_shared<LocalFSStore>()) { if (auto store2 = store.try_cast_shared<LocalFSStore>()) {
std::string symlink = drvPrefix; std::string symlink = drvPrefix;
@@ -631,8 +629,6 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
for (auto & path : outPaths) for (auto & path : outPaths)
std::cout << store->printStorePath(path) << '\n'; std::cout << store->printStorePath(path) << '\n';
} }
return 0;
} }
void registerLegacyNixBuildAndNixShell() { void registerLegacyNixBuildAndNixShell() {
lix/legacy/nix-channel.cc
+12 -30
View File
@@ -8,9 +8,7 @@
#include "lix/libexpr/eval-settings.hh" // for defexpr #include "lix/libexpr/eval-settings.hh" // for defexpr
#include "lix/libstore/temporary-dir.hh" #include "lix/libstore/temporary-dir.hh"
#include "lix/libutil/async.hh" #include "lix/libutil/async.hh"
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/regex.hh" #include "lix/libutil/regex.hh"
#include "lix/libutil/result.hh"
#include "lix/libutil/users.hh" #include "lix/libutil/users.hh"
#include "nix-channel.hh" #include "nix-channel.hh"
@@ -45,7 +43,7 @@ static void readChannels()
// Writes the list of channels. // Writes the list of channels.
static void writeChannels() static void writeChannels()
{ {
auto channelsFD = sys::open(channelsList, O_WRONLY | O_CLOEXEC | O_CREAT | O_TRUNC, 0644); auto channelsFD = AutoCloseFD{open(channelsList.c_str(), O_WRONLY | O_CLOEXEC | O_CREAT | O_TRUNC, 0644)};
if (!channelsFD) if (!channelsFD)
throw SysError("opening '%1%' for writing", channelsList); throw SysError("opening '%1%' for writing", channelsList);
for (const auto & channel : channels) for (const auto & channel : channels)
@@ -67,18 +65,13 @@ static void addChannel(const std::string & url, const std::string & name)
static Path profile; static Path profile;
// Remove a channel. // Remove a channel.
static kj::Promise<Result<void>> removeChannel(const std::string & name) static void removeChannel(const std::string & name)
try { {
readChannels(); readChannels();
channels.erase(name); channels.erase(name);
writeChannels(); writeChannels();
TRY_AWAIT(runProgram( runProgram(settings.nixBinDir + "/nix-env", true, { "--profile", profile, "--uninstall", name });
settings.nixBinDir + "/nix-env", true, {"--profile", profile, "--uninstall", name}
));
co_return result::success();
} catch (...) {
co_return result::current_exception();
} }
static Path nixDefExpr; static Path nixDefExpr;
@@ -134,14 +127,8 @@ static void update(AsyncIoRoot & aio, const StringSet & channelNames)
bool unpacked = false; bool unpacked = false;
if (std::regex_search(filename, regex::parse("\\.tar\\.(gz|bz2|xz)$"))) { if (std::regex_search(filename, regex::parse("\\.tar\\.(gz|bz2|xz)$"))) {
aio.blockOn(runProgram( runProgram(settings.nixBinDir + "/nix-build", false, { "--no-out-link", "--expr", "import " + unpackChannelPath +
settings.nixBinDir + "/nix-build", "{ name = \"" + cname + "\"; channelName = \"" + name + "\"; src = builtins.storePath \"" + filename + "\"; }" });
false,
{"--no-out-link",
"--expr",
"import " + unpackChannelPath + "{ name = \"" + cname + "\"; channelName = \""
+ name + "\"; src = builtins.storePath \"" + filename + "\"; }"}
));
unpacked = true; unpacked = true;
} }
@@ -171,16 +158,15 @@ static void update(AsyncIoRoot & aio, const StringSet & channelNames)
for (auto & expr : exprs) for (auto & expr : exprs)
envArgs.push_back(std::move(expr)); envArgs.push_back(std::move(expr));
envArgs.push_back("--quiet"); envArgs.push_back("--quiet");
aio.blockOn(runProgram(settings.nixBinDir + "/nix-env", false, envArgs)); runProgram(settings.nixBinDir + "/nix-env", false, envArgs);
// Make the channels appear in nix-env. // Make the channels appear in nix-env.
struct stat st; struct stat st;
if (sys::lstat(nixDefExpr, &st) == 0) { if (lstat(nixDefExpr.c_str(), &st) == 0) {
if (S_ISLNK(st.st_mode)) if (S_ISLNK(st.st_mode))
// old-skool ~/.nix-defexpr // old-skool ~/.nix-defexpr
if (sys::unlink(nixDefExpr) == -1) { if (unlink(nixDefExpr.c_str()) == -1)
throw SysError("unlinking %1%", nixDefExpr); throw SysError("unlinking %1%", nixDefExpr);
}
} else if (errno != ENOENT) { } else if (errno != ENOENT) {
throw SysError("getting status of %1%", nixDefExpr); throw SysError("getting status of %1%", nixDefExpr);
} }
@@ -258,7 +244,7 @@ static int main_nix_channel(AsyncIoRoot & aio, std::string programName, Strings
case cRemove: case cRemove:
if (args.size() != 1) if (args.size() != 1)
throw UsageError("'--remove' requires one argument"); throw UsageError("'--remove' requires one argument");
aio.blockOn(removeChannel(args[0])); removeChannel(args[0]);
break; break;
case cList: case cList:
if (!args.empty()) if (!args.empty())
@@ -273,11 +259,7 @@ static int main_nix_channel(AsyncIoRoot & aio, std::string programName, Strings
case cListGenerations: case cListGenerations:
if (!args.empty()) if (!args.empty())
throw UsageError("'--list-generations' expects no arguments"); throw UsageError("'--list-generations' expects no arguments");
std::cout << aio.blockOn(runProgram( std::cout << runProgram(settings.nixBinDir + "/nix-env", false, {"--profile", profile, "--list-generations"}) << std::flush;
settings.nixBinDir + "/nix-env",
false,
{"--profile", profile, "--list-generations"}
)) << std::flush;
break; break;
case cRollback: case cRollback:
if (args.size() > 1) if (args.size() > 1)
@@ -289,7 +271,7 @@ static int main_nix_channel(AsyncIoRoot & aio, std::string programName, Strings
} else { } else {
envArgs.push_back("--rollback"); envArgs.push_back("--rollback");
} }
aio.blockOn(runProgram(settings.nixBinDir + "/nix-env", false, envArgs)); runProgram(settings.nixBinDir + "/nix-env", false, envArgs);
break; break;
} }
lix/legacy/nix-collect-garbage.cc
+21 -13
View File
@@ -1,4 +1,3 @@
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/file-system.hh" #include "lix/libutil/file-system.hh"
#include "lix/libstore/store-api.hh" #include "lix/libstore/store-api.hh"
#include "lix/libstore/store-cast.hh" #include "lix/libstore/store-cast.hh"
@@ -25,11 +24,9 @@ bool dryRun = false;
static void removeOldGenerations(std::string dir, NeverAsync = {}) static void removeOldGenerations(std::string dir, NeverAsync = {})
{ {
if (sys::access(dir, R_OK) != 0) { if (access(dir.c_str(), R_OK) != 0) return;
return;
}
bool canWrite = sys::access(dir, W_OK) == 0; bool canWrite = access(dir.c_str(), W_OK) == 0;
for (auto & i : readDirectory(dir)) { for (auto & i : readDirectory(dir)) {
checkInterrupt(); checkInterrupt();
@@ -64,7 +61,7 @@ static int main_nix_collect_garbage(AsyncIoRoot & aio, std::string programName,
{ {
bool removeOld = false; bool removeOld = false;
GCOptions options = {.action = GCOptions::gcDeleteDead}; GCOptions options;
LegacyArgs(aio, programName, [&](Strings::iterator & arg, const Strings::iterator & end) { LegacyArgs(aio, programName, [&](Strings::iterator & arg, const Strings::iterator & end) {
if (*arg == "--help") if (*arg == "--help")
@@ -75,13 +72,12 @@ static int main_nix_collect_garbage(AsyncIoRoot & aio, std::string programName,
else if (*arg == "--delete-older-than") { else if (*arg == "--delete-older-than") {
removeOld = true; removeOld = true;
deleteOlderThan = getArg(*arg, arg, end); deleteOlderThan = getArg(*arg, arg, end);
} else if (*arg == "--dry-run") {
options.action = GCOptions::gcReturnDead;
} else if (*arg == "--max-freed") {
options.maxFreed = std::max(getIntArg<int64_t>(*arg, arg, end, true), (int64_t) 0);
} else {
return false;
} }
else if (*arg == "--dry-run") dryRun = true;
else if (*arg == "--max-freed")
options.maxFreed = std::max(getIntArg<int64_t>(*arg, arg, end, true), (int64_t) 0);
else
return false;
return true; return true;
}).parseCmdline(argv); }).parseCmdline(argv);
@@ -93,12 +89,24 @@ static int main_nix_collect_garbage(AsyncIoRoot & aio, std::string programName,
} }
// Run the actual garbage collector. // Run the actual garbage collector.
if (!dryRun) {
options.action = GCOptions::gcDeleteDead;
} else {
options.action = GCOptions::gcReturnDead;
}
auto store = aio.blockOn(openStore()); auto store = aio.blockOn(openStore());
auto & gcStore = require<GcStore>(*store); auto & gcStore = require<GcStore>(*store);
GCResults results; GCResults results;
PrintFreed freed(options.action, results); PrintFreed freed(true, results);
aio.blockOn(gcStore.collectGarbage(options, results)); aio.blockOn(gcStore.collectGarbage(options, results));
if (dryRun) {
// Only print results for dry run; when !dryRun, paths will be printed as they're deleted.
for (auto & i : results.paths) {
printInfo("%s", i);
}
}
return 0; return 0;
} }
} }

Some files were not shown because too many files have changed in this diff Show More