Compare commits

...
Author SHA1 Message Date
Jade Lovelace 533429d89a release: 2.94.1 "Açaí na tigela"
Release produced with releng/create_release.xsh

Change-Id: Ie38032fe8dfebed83f9d24762ce48f73b23149b5
2026-03-13 09:39:06 -07:00
Jade Lovelace c135090468 release: release notes for 2.94.1
Release created with releng/create_release.xsh

Change-Id: I970e8f5422e3705917b04c09d28498fd7b849d27
2026-03-13 09:38:52 -07:00
Jade Lovelace 72f7965679 2.94.1: version
Change-Id: I92eeeebfd6b966ce77833785482db989962f8c9e
2026-03-13 09:38:44 -07:00
Raito Bezarius b7be40c785 libstore/build: fix starvation during substitution
When the destructor of PathSubstitutionGoal is run, this happens in a
sync context and can cause starvation of all ongoing IO w.r.t. to other
substitutions, including our own substitution.

While there's only a decompressor thread per stream, the other side of
the IO runs on the event loop.

In order to fix this, it is sufficient to remove the thread indirection
and inline the async code.

Fixes #1126. Great thanks to horrors' patience.

Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I3eb37bc37d156f0f5528364e568fdaa2ced58011
Signed-off-by: Raito Bezarius <raito@lix.systems>
(cherry picked from commit 505d0669dc)
2026-02-11 15:39:06 +00:00
Qyriad 6350f51458 libutil: include LIX_MAJOR, LIX_MINOR, and LIX_PATCH macros
Backport of I7d8a4648890fce7ff15695876c9b9d3a6a6a6964 to 2.94 branch.

Change-Id: Ia6b8d89007974c8cf873fa1f536ce8416a6a6964
2026-01-15 11:48:26 +00:00
eldritch horrorsandRaito Bezarius c6f3f3a0d3 libexpr: fix app chain extension
during the value rewrite we accidentally broke extension of incomplete
primop application. this only shows up when binding on incomplete call
to a primop to a name, binding an incomplete call to *that* to another
name, and then finally calling the second binding with enough args for
a complete primop application. since this only shows up when calling a
primop with three or more args it took a while to surface. we have few
builtins that match this: foldl', replaceStrings, and substring. these
are not used incompletely in this manner very often, so this lingered.

fixes #1102

Change-Id: I218dffc14ae876efc86a86c7eb6c895e2405201c
2026-01-14 22:09:56 +00:00
Maximilian Bosch de4cfec46a libstore: fix reporting output cycles on drvs with references to other drvs
Closes #1064

The culprit here is that `genGraphString` is only invoked with the
store-paths associated with the outputs of the derivation, so when
filling `dependents`, the `graph_data.find(p)` call would return the end
of the iterator when doing this for references to other store-paths.

As a result, the code wrote information behind the graph data-structure
causing a corruption. For me, this resulted in a SIGSEGV most of the
time and in a few cases in an uncaught `map::at`-exception as reported
by Niklas.

This patch changes two aspects of the original implementation:

* When filling `dependents` in the node-set, use `map.at()` instead of
  `map.find()->second`. The latter doesn't make any sense and was the
  cause of corrupting memory. The `at` would've made it far easier to
  spot this in the first place.

* Filter out store-paths that don't belong to a different output of the
  derivation when creating `outputGraph`. This variable is used on two
  places, `genGraphString` and for topological sorting.

  The latter already filters out store-paths from a different drv, so
  this is happening now when creating the variable in the first place
  such that `genGraphString` never ends up with corrupt data in the
  first place. This is the actual bugfix.

Implemented a regression-test for this case to be sure.

Change-Id: Ie02144d89c32b0a776cb1ece0601d0229315ebc3
(cherry picked from commit 0a5f474a25)
2025-12-08 07:20:51 +01:00
Raito Bezarius 0873bed39d nix/upgrade-nix: disallow daemon connections for the store
Prior to I6a6a6964d2b5ad47ae5ea9eb11af9b6373ce2141 — `sudo nix
upgrade-nix` would perform direct store access.

This ensured a certain number of desireable properties for upgrading the
Lix binary itself.

We re-introduce direct store access for upgrading Lix binaries.

Fixes #1060.

Change-Id: I523c4d3023ed5fe9eff8fde9a266c56a0de47d8c
Signed-off-by: Raito Bezarius <raito@lix.systems>
(cherry picked from commit d2ca1810b1)
2025-12-06 22:20:18 +00:00
Zoe ZuserandRaito Bezarius 5dcb90548f meson: fix libstore.pc
typo of aws-cpp-sdk-transfer as aws-cpp-std-transfer prevents linking
against lix

Change-Id: Id847eab2601698696030d31fcd51288aa5f3d274
(cherry picked from commit 06f987fb0c)
2025-12-05 16:58:28 +00:00
Jade Lovelace 54118efaf6 release: merge release 2.94.0 back to mainline
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh

Change-Id: Ida5959473ec469dd5e5e977ebdc8aab6e694b0de
2025-11-17 11:42:20 -08:00
Jade Lovelace fd38f625e6 release: 2.94.0 "Açaí na tigela"
Release produced with releng/create_release.xsh

Change-Id: If2ac30e569759c9035a6fc999c631ec545d190fa
2025-11-17 11:42:19 -08:00
Jade Lovelace 49a5495cbc release: release notes for 2.94.0
Release created with releng/create_release.xsh

Change-Id: Id7c776dc0ed46b1f6f09db2c77e04166469f5710
2025-11-17 11:42:12 -08:00
Jade Lovelace b6a8e312e7 2.94: release name
Change-Id: I18f5abe00130b53053e3a04074617071c0f884a2
2025-11-17 11:41:57 -08:00
WeetHet 7d46e9db7e libstore/build: extend DNS configuration access in network sandbox
https://github.com/NixOS/nixpkgs/pull/451579 PR
enabled c-aresSupport for curl on darwin which ended up breaking
DNS resolution in FOD sandboxes:

```
nix-run> exporting https://tangled.org/@weethet.bsky.social/nix-run (rev 73d7bf6b58848fb8f42e3a69816e0847f041c689) into /nix/store/m4m951648wmipxgwrgsml9gzjwfpfhm7-nix-run-73d7bf6
nix-run> Initialized empty Git repository in /nix/store/m4m951648wmipxgwrgsml9gzjwfpfhm7-nix-run-73d7bf6/.git/
nix-run> fatal: unable to access 'https://tangled.org/@weethet.bsky.social/nix-run/': Could not resolve host: tangled.org (Could not contact DNS servers)
nix-run> fatal: unable to access 'https://tangled.org/@weethet.bsky.social/nix-run/': Could not resolve host: tangled.org (Could not contact DNS servers)
nix-run> fatal: unable to access 'https://tangled.org/@weethet.bsky.social/nix-run/': Could not resolve host: tangled.org (Could not contact DNS servers)
nix-run> Unable to checkout 73d7bf6b58848fb8f42e3a69816e0847f041c689 from https://tangled.org/@weethet.bsky.social/nix-run.
```

with these sandbox failures:
```
deny mach-lookup com.apple.SystemConfiguration.DNSConfiguration
deny file-read-metadata /private/etc/hosts
deny file-read-data /private/etc/hosts
```

We allow those so that DNS resolution work fine

Change-Id: I9102293691972feb085adf8e9b1ad915bb3a36ab
2025-11-17 19:37:08 +00:00
Raito Bezarius 7e193f962e libstore/filetransfer: support HTTP/3 transfers if the user requests it
This adds the enablement code to support HTTP/3 if the user requests it.

We leave it disabled because h3 is not onpar with h2 performance.

Change-Id: I1fd3d4c97b972dcf36bccacc6c9a8290e22b31e0
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-11-17 19:11:32 +00:00
adam fb4cc15f39 chore: update nixos channel URLs
Channels have moved from `nixos.org/channels` to `cnannels.nixos.org`.
This udates all relevant links (excluding release notes) to use the new
canonical URLs and replaces HTTP with HTTPS.

Fixes #1031.

Change-Id: I212821c44ac5e482c8e9eaa415c7d8ee17ff8341
Signed-off-by: adam <me@adamperkowski.dev>
2025-11-15 18:21:47 +00:00
Alois Wohlschlager 243836ed0a libutil: remove the no-url-literals experimental feature
URL literals were deprecated in 278fddc317,
effectively stabilizing the experimental feature. Enabling it has no effect any
more. Remove it.

Change-Id: I8fbca03b1a2be6a8cddfce644043ac0a6a6a6964
2025-11-15 15:54:39 +01:00
Raito Bezariusandeldritch horrors 5fd9c65a8f doc/manual/known-issues: init
We start this section with shortcomings of unsandboxed builds.

Fixes #1018.

Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: Ieb17e4340beab0c1197951813ae602de453a3fd9
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-11-14 21:36:48 +01:00
EmilyandQyriad 82463616f9 libstore: skip the nested build directory on Darwin
This is unnecessary because Darwin builds already cannot create
`set{u,g}id` files due to the minimal sandbox policy, because we can
forbid messing with the top‐level build directory directly in the
sandbox rules, and because Darwin builds can trivially avoid isolation
through temporary directories right now anyway.

This does regress the build directory isolation for builds with
`sandbox = false`, but I can’t imagine that mattering given the
above. The sandbox change prepares us for a world where we close
off shared temporary directories for `sandbox = true` builds and try
harder to achieve proper isolation on Darwin, but probably doesn’t
have a meaningful security impact one way or another for now.

With this change, we get down to 41 byte build directory paths on
Darwin, a ~2–3½ byte improvement over the old status quo. We can
also restore the 0710 permissions on Linux.

Change-Id: I6a6a6964a681c0365241fe7234831db656b76799
2025-11-14 14:58:15 +01:00
EmilyandQyriad e87e547a36 libstore: default to /nix/var/nix/b for build-dir
The minimum build directory length on Darwin with default settings
when we were still using `/tmp` was 22 bytes. Deriving build directory
names from my local store, the median and mode were 43, the mean was
around 44½, and the maximum was 127.

The switch to `/nix/var/nix/builds` over `/tmp` added a 15 byte
penalty, and the additional `/b` directory added another 2.

Now that we use opaque build directory names, the length is 48, so
we’re still at a ~3½–5 byte penalty over the previous status
quo. This change brings us down to 43, matching the previous median.

Note that these calculations do not take into account the fact that
`/tmp` is a symbolic link to `/private/tmp` on Darwin. Anything
that was canonicalizing paths would have had an additional 8 byte
penalty in the previous status quo that is not applicable here,
so we may already be ahead even without this change. If the more
opaque directory name here is undesirable, then that factor could
potentially help us squeeze by without. Alternatively, in combination
with dropping the `/b` on Darwin we could use `/nix/var/nix/bld` or
similar, but I feel that the paths in general are sufficiently opaque
that it should be okay to go with the shorter option here. Given that
some projects already had to reduce filename lengths to avoid this
limit even before the recent changes, I think it is best to try and
improve on the previous status quo.

Note that `/nix/var/nix/builds` will unfortunately not be cleaned up
on Darwin. However, we don’t clean up the directories inside it on
Darwin anyway, so hopefully that’s okay for now?

Closes: #913
Change-Id: I6a6a6964bffce7194bcddcaefb4c4a37569c7df5
2025-11-14 14:58:15 +01:00
EmilyandQyriad 13a37f1246 libstore: don’t include derivation names in build directories
They have variable size, which is bad for #913.

Change-Id: I6a6a6964870e984c66277c7556ff3c2bc34ddca1
2025-11-14 14:58:15 +01:00
EmilyandQyriad 1102cc180a tests: remove obsolete code to create custom build directories
We now do in fact do this and decide this.

Change-Id: I6a6a696493b288ed76d809122bda259dc0225846
2025-11-14 14:58:15 +01:00
EmilyandQyriad ea0a2c8e74 libstore: make temporary path prefixes optional
This is not the same thing as passing an empty string, because it
avoids the `-` separator.

Change-Id: I6a6a696451667cbf500914e2dfbca2a4646ff20b
2025-11-14 14:58:15 +01:00
EmilyandQyriad c5221e508f libstore: simplify createTempDir interface
We always use the default temporary directory, because
`createUniqueDir` has an interface nice enough to use directly for
the few bespoke uses.

Change-Id: I6a6a696450b7c0a0bd76655632fb14d7c5e38199
2025-11-14 14:58:15 +01:00
EmilyandQyriad b1cfb58187 libutil: use makeTempPath in createTempSubdir
This makes the paths more nondeterministic, but more reliably unique,
and lets us remove the retry loop.

Note that this adds random entropy to the build directory visible
inside derivations on Darwin and unsandboxed Linux. It was already
non‐deterministic in the presence of concurrent builds and similar,
but now we can reliably expect it to be different every time. On the
whole I think that’s a good thing, as it is impossible to ensure
a single consistent build directory and derivation outputs should
not depend on it.

Package reproducibility isn’t great on Darwin to begin with,
though, and the reproducibility bugs this will turn up in packages
will be more urgent to fix than when the build directory was mostly
consistent. A quick survey of my local store shows that many C, C++,
and Rust binaries contain build directory references, likely due to
use of `__FILE__` and its equivalents; non‐binary offenders include:

* Install logs included in the Rust and Cargo bootstrap compilers
* Example errors in the Rust documentation referencing build paths
* Configuration information installed with CPython itself
* Python 2 metadata from resholve’s closure
* Cython metadata
* Generated headers in Facebook libraries referencing source paths
* Generated CMake files in Facebook libraries referencing source paths

I haven’t built that much in this store since the last GC, so this is
probably only a small sample of the problems across the tree. These are
all instances of <https://reproducible-builds.org/docs/build-path/>,
though, and should probably just be treated as general reproducibility
bugs outside of contexts like the Linux sandbox where we can normalize
them away entirely.

I have implemented away build directory paths for C/C++, applied some
additional fixes for non‐`__FILE__`‐related issues in binaries
from ATF and LLVM, and fixed the derivation bug causing the CPython
3 issue, and will work on upstreaming these changes. Rust is working
on the problem upstream, with some temporary workarounds we can
potentially apply in Nixpkgs for now. The rest will require some
distributed effort.

Change-Id: I6a6a69645b4915c56c0fdef904aa81684e4136c6
2025-11-14 14:58:15 +01:00
EmilyandQyriad 6e7212bad7 libstore: simplify fallback build directory logic
This does change the behaviour when the global temporary directory
does not exist, but other uses of the global temporary directory are
already broken in that circumstance, and it should be fixed centrally
if the use case is considered desirable. The logic was not present
before the recent churn around build directories – it was added now
that Lix is taking ownership of the build directory in the store –
so this should not be a meaningful regression.

Change-Id: I6a6a6964e345ea6803226c5ad759e836de7cb0ed
2025-11-14 14:58:15 +01:00
EmilyandQyriad d23bf515a7 libstore: use makeTemp{,Sibling}Path more
Change-Id: I6a6a6964c885be6dea0a69ee3162fbf4b812471f
2025-11-14 14:58:15 +01:00
EmilyandQyriad 0b2fcd1bcd libstore: simplify makeTemp{,Sibling}Path callers
There is now no risk of race conditions on a system with a functioning
entropy source, and the bespoke prefixes are either redundant to the
default or unnecessary.

Change-Id: I6a6a69641211c6bb979ea48ad30aecb1a53d03f0
2025-11-14 14:58:15 +01:00
EmilyandQyriad d03b1eca57 libutil: use OS‐provided entropy for temporary filenames
Relax the constraints on keeping the exact same filename format to
provide a more robust source of entropy with a simpler interface
(as previously suggested by eldritch horrors). Using 128 bits of
OS‐provided entropy ensures global uniqueness and allows us to
skip any thought of gracefully handling the case where these files
already exist.

My microbenchmark that repeatedly constructed paths like this and
printed them out showed that this takes about 1.23× the time of
the previous implementation, both taking on the order of a couple
microseconds for one iteration. Since everything that uses it is doing
things more expensive than printing to standard output, the actual
performance delta is likely to be lost in the noise. If it somehow
becomes a bottleneck, it can be optimized without sacrificing the
guarantees by reading from the system RNG only to seed a thread‐local
CSPRNG like [ChaCha8Rand], but I think that’s very unlikely.

We also tweak the recommended way of creating a temporary file inside
a directory in anticipation of later changes, and rename the `suffix`
parameter to `prefix` (it’s a prefix to the random characters and
a suffix to the root, but this way is more consistent).

[ChaCha8Rand]: https://c2sp.org/chacha8rand

Change-Id: I6a6a69648502c746d13d8c3bd2768cbbf1b90466
2025-11-14 14:58:15 +01:00
EmilyandQyriad 9f3ba30783 libutil: extract Base32 helpers from Hash
base32Encode now takes std::span<std::byte>, with a base32EncodeStr
convenience wrapper which takes std::string_view.

Co-authored-by: Qyriad <qyriad@qyriad.me>

Change-Id: I6a6a6964f799dc84ecbfb55c7ca03a064cff71d9
2025-11-14 14:58:15 +01:00
eldritch horrors 2fa40c9de4 libstore: bring back some pointer equality nonsense
the change to shareable thunks also removed a few cases of pointer
equality checks that allowed structures containing functions to be
considered equal to other sets containing the same functions, even
if the sets themselves were pointer-equal themselves. *so* busted.

Change-Id: If87fdab658f9037ce2a654f69a9e3da6ae2f53e5
2025-11-14 02:56:53 +00:00
Qyriad 78bdc60c11 libutil: refactor Pos::getSource to Origin
Change-Id: I1501fceb9ff1e768c8f5c45b65d5638568babf63
2025-11-12 10:40:21 +01:00
Qyriad 3ad1af8a98 libexpr: add trace for if 'foo' errors in 'or foo'
Change-Id: I59365e4c1bf4aed71c008554eca0c9e6d085219d
2025-11-07 14:54:02 +01:00
Qyriad 2806613954 libexpr: refactor ExprSelect::eval
Change-Id: Ibb21dbd3c25649b3b301bc7baaa52a8aa938ebe0
2025-11-07 14:54:02 +01:00
Qyriad 2e7bfa4474 libutil: impl <=> for Pos::{Stdin,String}
Change-Id: Ieb5016a261f5a90bbf08b2afb69bb1c288d07ca7
2025-11-07 12:07:23 +01:00
Qyriad f39cbc5d60 libutil: replace Pos operator< with <=> and constexpr ==
Change-Id: I60080c416b2ee2b0efce4584cf13f37c0596d73d
2025-11-07 12:07:23 +01:00
Qyriad f00d720d83 libutil: refactor trace types and functions into their own file
Change-Id: I00fba6b382991322a083bedeaf169eb5a86f5aeb
2025-11-06 19:19:36 +01:00
eldritch horrors 1e386c3780 libstore: add usage hints to repairPath on daemons
we can't change the protocol to allow daemons to do this, and we should
not try to guess what the `auto` store uri means depending on whether a
command was run by root or not due to copious side effects and not even
being able to tell whether the `auto` store uri was given explicitly or
not. while `auto` may *technically* allow this via its naming we should
resist the urge to add a hack and fix the underlying protocols instead,
especially since repairPath should be a rare, superuser only operation.

fixes #888

Change-Id: I1b53245db226199f827a89a237a2ab9907c3f766
2025-11-06 15:10:34 +01:00
eldritch horrors 24054c1107 libutil: restore process context before subcommand exec
fixes #1028

Change-Id: Ic50b9cc0c65607cd96dd81fd770cda34b4caf9d5
2025-11-05 13:56:09 +00:00
eldritch horrors 0d24aee673 testing: drop compression from truncated nar test
some versions of zstd enable checksumming, some don't. don't use it.

fixes #1027

Change-Id: I06811072d9e56a682a792f084cf76a81c7b5b4ce
2025-11-05 00:02:10 +01:00
eldritch horrors f369c711dd libutil: prefer reporting libarchive source errors
if curl fails during a transfer and said transfer is piped through
libarchive we see very strange errors (#1009). in such cases we'll
want to prioritize reporting the source errors to allow debugging.

Change-Id: I8cc231257eca5b749471ecd38df15071170decc2
2025-11-03 23:35:42 +00:00
hexchen 57658b2146 tests/functional/daemon-trust: fallback to uid
The test would fail if the identity cannot be determined
(as is the case e.g. when using auto-allocate-uids under darwin).
Instead we use the uid when the user name is not available.

Fixes: #1025

Signed-off-by: hexchen <hexchen@lilwit.ch>
Change-Id: I12f35840b78f989b5fd09bbd44c833dc6a6a6964
2025-11-03 18:24:07 +01:00
Raito BezariusandQyriad d4f404ded3 nix/doctor: test whether the current profile generation points somewhere
This helps users to debug whether their current profile symlinks are
correctly set.

Expected outputs look like this:
```
❯ sudo ./outputs/out/bin/nix doctor
[snip]

[PASS] All profiles are gcroots.
[PASS] Client protocol matches store protocol.
[INFO] You are trusted by store uri: local
[FAIL] Error: current generation cannot be discovered for profile:
'/nix/var/nix/profiles/default'
```

```
❯ ./outputs/out/bin/nix doctor
[snip]

[PASS] All profiles are gcroots.
[PASS] Client protocol matches store protocol.
[INFO] You are trusted by store uri: daemon
[PASS] You have 28 generations for profile
'/nix/var/nix/profiles/per-user/raito/profile'
The current generation number is '290'
```

Change-Id: I50c69cbeac3291d668f4c2332803411579adc944
Signed-off-by: Raito Bezarius <raito@lix.systems>
Co-authored-by: Qyriad <qyriad@qyriad.me>
2025-11-03 14:41:47 +00:00
Qyriad a30d87eadc nix/doctor: skip inaccessible PATH elems for duplicate nix
Inaccessible-but-extant PATH elements can't be executed anyway, so we
can skip them as if they didn't exist, instead of erroring like before.

Change-Id: I9288c3ecb6768171e62a3088122d98421558eb03
2025-11-03 14:41:47 +00:00
Qyriad d15b99f489 libstore/static-init: don't crash if nixBinDir exists but isn't accessible
In d2696cdd¹, libstore was made to prioritize NIX_BIN_DIR/nix (over
/proc/self/exe) if it exists. But `pathExists()` throws an exception if
lstat() returns any error other than non-existence.

Because this check happens in libstore's Settings constructor for the
static-initialized global `nix::Settings`, this case would abort Lix
well before main().

[1]: d2696cdd1e

Change-Id: I62e44c8a33b28c234900df3e4790c1badefee358
2025-11-03 14:41:47 +00:00
Commentator2.0 acf6e1c6b7 tests/functional2/file_helper: add a way to interpolate environment variables into files
So far it was impossible to put absolute paths into files, as one
wouldn't know whre the test would be placed.
This commit adds a new Fileish variant called `EnvTemplate` which uses
the given string as a template and replaces the `@ENV_VARIABLE_NAME@`
placeholders with the according value of the environment variable.
This way one can use `@HOME@` or `@TEST_ROOT@` to build absolute paths

Change-Id: I425cb6408dceb8d7f26d136ace4ac98b1ca31ec3
2025-11-03 12:58:37 +00:00
Raito Bezarius 85db5e3bf4 libstore/store-api: rate-limit concurrent copies based on system limits
Since we have async, we are unleashed and therefore we started to cause
people to run into system limits like number of open files.

We introduce an async semaphore which uses max(25 % of max open files,
max number of cores) which should lead to 256 maximum concurrent copies.

We believe that the copy operation is the only one that can cause issues
given that the daemon already runs with obscene fd limits.

Fixes #1022.

Change-Id: Iec433d10d5c5003962ea749cf8e32bafb314f0d9
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-11-01 23:47:13 +00:00
eldritch horrors f5ae0d1eff libutil: fix EBADF when cancelling decompression streams
relying on the sink/source code is a mistake. no exceptions.

Change-Id: Ied4ffc4065ce628ed7f8397526cdb68c8ac652d8
2025-11-01 20:42:12 +00:00
Raito Bezarius df862c1655 libstore/s3: attach more information to error messages
In case of empty messages, it is good to print the raw error code.

Additionally, we print request IDs which can help users to reconcile
what happened with the service provider.

Change-Id: I4d83c011c1b7a5514e3d1b21123df38308279044
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-11-01 21:17:26 +01:00
Raito Bezarius ccf196d7f4 libstore/s3: resolve completion status via the transfer status callback
Completion status can be updated in two ways:

- progress callback
- transfer status update callback (esp. important in multipart
transfers)

It's guaranteed that one of the two will provide progress for our
promise.

This fixes #945.

Change-Id: Iac9e92df34f82fbc1facae7a411ccabf1688da52
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-11-01 21:17:26 +01:00
Raito Bezarius 91867941fa libstore/s3: sign payloads if the request demands it
This fixes problems with S3 implementations that have mandatory payload
signing, e.g. Garage v2, AWS S3 itself, etc.

The problem manifested itself when the AWS SDK threw some error codes 99
(NETWORK_CONNECTION) with no messages and retried until it finally came
to a moment where the state machinery decided to send a
`Transfer-Encoding` header in a `Content-Encoding`/`Content-Length`-set
request with signed headers (even though payload signing is disabled),
causing the server to reject the transfer and crash the copy.

I did not debug super far what went wrong in AWS SDK, but I can confirm
this change makes transfers possible to finish with Garage v2.

Change-Id: Icc9e6a9f2afb0d760cf2d1e27816decd385a1d85
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-10-31 20:01:14 +01:00
Commentator2.0 9ed75192ec tests/functional2: fix crash on empty nix.nix([]) calls on darwin
Calling `nix.nix` with an empty argument list, would crash on darwin due
to a Index Out Of Range Error, as the build="auto" functionallity would
try to access the first element of the provided arguments (second
element at the place of call) resulting in a crash

Change-Id: Ia678109808b3f75a30182114baa421cedaa8759b
2025-10-30 18:53:11 +00:00
Raito Bezarius 95f15cf94f libstore/binary-cache: default to zstd for compression
libarchive's xz offers single threaded xz compression which is very slow
and provides ~10-20Mbps compression speed in addition to maxing a core.

In exchange, it achieves optimal compression ratios among all our
compression methods.

Nonetheless, xz prevent the saturation of 1Gbps+ connections and slow
down significantly decompression for end users. As these connections and
faster hardware is becoming prevalent for cache servers and clients, we
offer to default to zstd.

Lix is a "compress once, decompress many times" application. To avoid
incurring a high penalty to end users very sensitive to compress ratio
(very slow Internet connections), we dampen the consequences of
switching to zstd by increasing the default zstd level to 12.

On one example, xz will compress a 4.4GB file to 632MB, zstd on 12 will
compress it to 775MB, that is a ~18 % increase over the optimal xz
compression. zstd took 18 seconds to produce this file.

Increasing to level 14 leads to a 773MB file while taking 37s.
Increasing to level 16 leads to 735MB file while taking 66s.

Finally, xz took 77s, so a 50 % reduction in time taken to compress in
exchange of an increase of 18 % of the compressed size.

This change will reduce issues encountered in #945 but is probably not
the root cause.

References:

- https://discourse.nixos.org/t/switch-cache-nixos-org-to-zstd-to-fix-slow-nixos-updates-nix-downloads/23961

Change-Id: I7beda2bf2c1fed146dcb797b8f85dc290c486ab2
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-10-30 15:10:10 +01:00
eldritch horrors 047aad9992 libstore: fix s3 promise fulfiller lifetimes
if a promise for an async s3 operation is cancelled is inner fulfiller
will be destroyed, but the s3 operation may still be running. once the
operation finished and attempted to fulfill the promise we'd segfault.

Change-Id: I603c75d6cc39a1aee465b3ceee06b638f7f7f596
2025-10-30 13:57:15 +01:00
Raito Bezarius 1f08eaeb55 doc/manual: provide more information about Pasta and its shortcomings
As we saw actual users running into them throughout the debugging of
#920 and #1014.

It's best to document and point end users to this so they can take
measures for themselves.

Change-Id: I7d08407f4354055bf65fc6dd7d1624c5a9304402
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-10-30 00:04:21 +01:00
Raito Bezarius 480c8e52b6 libstore/linux/pasta: always list an IPv4 & IPv6 nameserver
Fixes #1014.

Address family detection is performed by Pasta, we influence Pasta into
enabling dual stack by listing both an IPv4 & IPv6.

Change-Id: I4121e0eae7d5185c287957106adc36edf3f82a40
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-10-29 21:44:12 +00:00
Raito Bezariusandeldritch horrors 12b87538e3 lix/nix-shell: cleanup of shell directories happens in Lix itself
Instead of letting inner shell perform the cleanup, we will fork/execvp
and finish the work ourselves. Once an interruption arrives, we wait
again and return the exit status code of the inner shell.

This idea was suggested by eldritch horrors.

Fixes #1020.

Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: Id38af7ac33874aa1c4b7ca9c6123e3cbd8c3cd8c
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-10-29 20:13:23 +01:00
Raito Bezarius f6d723b22f lix/main: return the legacy command statuses
This was forgotten which should have been part of 64b9247f.

Change-Id: I3cf83d8699f2c1963ce530bbea6a511075202a79
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-10-29 18:53:00 +01:00
Raito Bezarius 64b9427f98 legacy/nix-build: return exit codes
Legacy commands "main function" will now return an exit code.
Fun fact: `main_nix_build` was the only one not returning integers
already.

Change-Id: Ia43a16c3c3fb9a670e8889aefc4ee9b6528a7df4
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-10-28 20:38:39 +01:00
Raito Bezarius 56b4ed3908 libmain/main: propagate status codes via return values for mainWrapped
This also prevent `legacy` to return nothing as it's void.

Change-Id: Ic797544a59b04b41ad9e1c46af4cad3a1cb6fdc8
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-10-28 20:38:39 +01:00
eldritch horrors 2c176afa7a libutil: don't allow interactive runProgram2
realistically only runProgram is ever called for interactive reasons,
and even those calls seem to be rather ill-advised in many cases. the
chance of multiple interactive processes interfering with each other,
whether for input or for output, must be very low to make this in any
way reasonable: if e.g. git calls ssh for multiple fetched inputs and
ssh requests passphrases for both we can otherwise not guarantee that
*any* input is routed correctly. misrouted output is merely annoying.

Change-Id: I794e3fdf0a3238cb9292003a89ac267f0de7a939
2025-10-28 11:53:33 +00:00
Qyriad 9523446901 libexpr/libutil: summarize derivations in stack traces regardless of --show-trace
Eval errors will now print a simple, no frills chain of involved
derivations at the bottom. For example, trying to evaluate
`pkgs.xonsh.override { python3 = pkgs.python2; }` has the usual

Package ‘python-2.7.18.8’ in /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/pkgs/development/interpreters/python/cpython/2.7/default.nix:398 is marked as insecure, refusing to evaluate.

message, but now also includes the following:

       note: trace involved the following derivations:
       derivation 'xonsh-0.19.9'
       derivation 'python2.7-xonsh-0.19.9'
       derivation 'python2.7-setuptools-44.0.0'

To give the user information about why the erroring derivation was
involved in the first place.

We would like more structured information in the future, but this should
still be a significant improvement.

Change-Id: Icf6da52abd0a043cfb63943bf0b0c160c21ee59e
2025-10-27 20:28:27 +01:00
Commentator2.0 73c019c57b tests/functional2/lang: allow adding requiring of global assets like config.nix
Change-Id: Ia740b91fb3d18f7d01c0d551196b71e7f3e46384
2025-10-27 17:07:47 +01:00
Qyriad 836caf1173 libutil: add concepts for constraining iterables ("ranges")
Change-Id: I5d75abb9d3713a3fab863666786aa8d38cc93dbd
2025-10-27 13:16:29 +01:00
Qyriad b8462562f2 libutil: allow moving Trace into Error
`pushTrace` now move-constructs or copy-constructs, instead of always
copy-constructing.

Change-Id: Ie5f02643e98b7a6c1adbe58b0d46d9e116eba28e
2025-10-27 13:14:50 +01:00
eldritch horrors 18efc848fe libstore: move curl-multi wrapper into own class
the wrapper is needed by transfer streams to restart a failed transfer
if desired. curlFileTransfer itself is more of a fancy handler for the
thread we're dedicating to curl io handling. the thread will stay with
the multi handle for now because quit handling needs to stay there. we
could have CurlMulti keep only a flag, but that does not help us much.

Change-Id: I99550f0bbb635b75898ca7260f08275df86050e3
2025-10-23 22:52:09 +00:00
eldritch horrors b460a7a38b libstore: move curlFileTransfer::TransferItem up
we have anonymous namespaces and no other scoping requirements, we can
move it out. this will also make it easier to move *other* things out,
such as the transport streams and all the shared multi state handling.

Change-Id: Iafa1eaeb69e096e98c4667f0531b17d5fe7c78a6
2025-10-24 00:16:15 +02:00
eldritch horrors 253ed1cc05 libstore: abort all transfers when shutting down curl thread
waiting for them to finish leaves us prone to deadlocking, e.g. if some
active transfers is blocked and hasn't been destroyed before our thread
was asked to exit. aborting transfers is the safer option here; even if
we cause use after frees we'll at least have a crash, not hang forever.

Change-Id: I333dcced810e8a85e2ded1f377962a0176798098
2025-10-22 17:36:21 +00:00
Raito Bezarius e0d2fa89d3 libstore/build: resolve symlinks beneath settings.caFile
This behavior existed before I32f8b5ce20fe9b6a911768114c92f95fc886cc07
was introduced.

It was remarked in
https://gerrit.lix.systems/c/lix/+/3765/comment/1ff0b2b7_ae67ab1b/.

Change-Id: I7a72e7c36c6043a6c2cb8cab57ad45a552d5e2b6
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-10-22 13:23:48 +00:00
Alois Wohlschlager 7a0b8834a4 Reapply "functional2: fix in the development shell"
This reverts commit 3ef7e0711b.

Change-Id: I5b76d2c2e9b0dd669818c7218493d3e76a6a6964
2025-10-22 07:41:46 +02:00
Alois Wohlschlager 0e0853cc70 functional2: handle unset or empty BUILD_TEST_SHELL properly
If BUILD_TEST_SHELL is not set or empty, None or the empty string respectively
should not make it into the PATH. Ensure this property.

Change-Id: I4ce9b0c06c407b465308b63b9cb64e7d6a6a6964
2025-10-22 07:41:46 +02:00
eldritch horrors 5346b2bc68 libstore: simplify worker child handling a bit
we no longer need the optional wrapping.

Change-Id: I742cca23753c6e0a97ada0b301bb42f442803ea8
2025-10-20 12:43:22 +00:00
eldritch horrors 31b6eb2786 libstore: convert build-hook logging to rpc
Change-Id: I0c20f89de113dce6032c93a32e9fcd43b4478f55
2025-10-20 12:43:22 +00:00
eldritch horrors 5b7ed433d6 libutil: add logging rpc definitions
this is still somewhat experimental and should be considered in flux. we
will have to nail down a logger interface once we start moving the store
protocol to rpc, but until we do that we can use build hooks to test it.

Change-Id: Id20cd346c9520f45871799c31b0af040adde56ef
2025-10-20 12:43:22 +00:00
eldritch horrors 2d2cd7ac03 libutil: always rethrow kj::CanceledException
the fiber runtime requires this for correctness.

Change-Id: I5ebc6a046596403e86cebaa95962053ff7813877
2025-10-20 12:43:22 +00:00
eldritch horrors bab44b870e libstore: handle log size limiting in only one place
extracting the size-specific log handling into a new sink deduplicates
the size-limiting code and lets us clean up the derivation goal a bit:
a number of log-related fields we kept only because previous ownership
semantics were a lot more ad-hoc can be moved into the new sink class.

Change-Id: Ie770a276597c84f9d928a9624c0f0de86adfe815
2025-10-20 12:43:22 +00:00
eldritch horrors 4b9723a23f libstore: DerivationGoal::{handleChildOutput -> wrapChildHandler}
we'll want to remove handleRawChildStream from DerivationGoal, but the
timeout handling is still important. the wrapper is thus useful, but a
more generic way to pass it a promise to do the handling is necessary.

Change-Id: I55bf041d51e4f8a8b8291deae053423d06724329
2025-10-20 12:43:22 +00:00
eldritch horrors b02a66d13e libstore: simplify hook instance handling
we never need the connection or the rpc client directly after the hook
is constructed, only the hook instance rpc object. wrapping them using
kj primitives makes this more explicit and prevents destruction errors

Change-Id: I7e0b14d7e365c2e001273e04c0e741ba4afb12f2
2025-10-20 12:43:08 +00:00
eldritch horrors 6c416ff2ca libstore: make build hook exit status reusable
this makes it possible to wait for the hook to exit in one place,
process its exit status for some purpose, and later process it in
another place for a different purpose. rpc will need this to shut
down a hook cleanly after the hook has completed its assignments.

Change-Id: I5af2d5aac4b02a0a200d720d0e6f0b5df5496aaf
2025-10-19 16:09:32 +00:00
eldritch horrors 7e171a4abe libstore: localize act, builderActivities
only LocalDerivationGoal needs these, and once the child is done
running (or is begin torn down) we can just destroy all of them.

Change-Id: Id4d9f0105aaf33d6736c435bb891d3f9af8e7894
2025-10-18 21:33:54 +00:00
eldritch horrors 0365d17d7e libutil: disallow implicit ref downcasts
wtf?

Change-Id: I3dbce7052e72525c9430efb0f5090b168db04750
2025-10-18 21:33:54 +00:00
eldritch horrors 641769414b libutil: add a timed wait to async mutex
wrapping the wait in a timeout at the call site is insufficient because
a timeout would not relock the mutex and leave the raii object invalid.

Change-Id: I0be2aaf6b3c9966070fb0f1deaab048765fd7af3
2025-10-18 21:33:54 +00:00
Commentator2.0 2541db5321 tests/functional2/lang: fix expected skip message
it seems like pytest is updating the message printed when skipping a set
due to no parametrization
which leads to breakage on staging-next.
this commit changes the expected message to allow both old and new skip
message

Change-Id: I34d11ad84d7075911d8229fbc89889bb880a27c9
2025-10-18 17:19:01 +02:00
Commentator2.0 0ac71ea1cc tests/functional2/lang: fix wrong unused file error for generic test group collection
Change-Id: I1c548fcebe45d2a6621987091e3a852b8a9b5765
2025-10-18 11:24:00 +00:00
Commentator2.0 4fcfdc66fe tests/functional2: migrate debugger.sh
Change-Id: I82c5b739db09530ff216c91608be2f70a2036e11
2025-10-18 11:23:43 +00:00
helle e8d281eac6 doc: information at the use-cgroups option on conflicts it can give
as discussed on Zulip and with Lunaphied

Change-Id: I7c6b3e7994c1e7961033e3daf509c81348927baf
2025-10-18 09:51:36 +00:00
K900 6e2edbff93 flake: only apply the monotonic clocks patch to capnp<1.2.0
1.2.0 already includes it so building on unstable fails.

Change-Id: Iefa49203c371e5bab164c06b9da6f89a89ce7cfe
2025-10-18 12:04:06 +03:00
eldritch horrors a55fe35dac apply capnp patches again
This partially reverts commit f68233ec43.

while more modern capnp in *does* have the build system changes, it
does *not* have the patches. and those patches are rather important
to us; the clock patch because our CI machiens don't behave the way
capnp expect and the nodiscard patch because it *is* a bugsquasher.

Change-Id: Ic273777f09fd8e5e7001f815db8998dae2dd7b88
2025-10-17 19:21:17 +00:00
EmilyandQyriad 6ea1d8e37e libstore: use makeTempSiblingPath in replaceValidPath
Change-Id: I6a6a69641a3b4e6fdd076faac44dc314e6cc057e
2025-10-17 18:41:58 +00:00
EmilyandQyriad 39ebb982ea libutil: add makeTempSiblingPath helper
The prospective callers of this should probably be doing something
smarter or more abstracted to begin with, but this is useful as an
incremental improvement for call sites with existing `makeTempPath`
logic in the face of filename length limits.

Change-Id: I6a6a69644292f5bbf984a1df90192e06c6022b53
2025-10-17 18:41:58 +00:00
EmilyandQyriad d9eead0eac tests: add fetchTarball test for symlink temporary directories
Change-Id: I6a6a6964dfe85b5ee4f66b8c442fe448c7c5c87c
2025-10-17 18:41:58 +00:00
Qyriad 013d5e54e9 libstore: fix non-awaited calls to optimisePath()
LocalStore::optimisePath() was made asynchronous in 38b75b736¹, but we
lost warnings for not awaiting kj::Promise in f68233ec4², since our
capnproto no longer has the [[nodiscard]] patch.

[1]: 38b75b7367
[2]: f68233ec43

Change-Id: I2363396949f40dd43eae08cc76be9ede6b488cf7
2025-10-17 18:41:43 +00:00
eldritch horrors 20ba2b49fb testing: mock ssh during nix-copy prompt test
we don't need full ssh, we only need to know something makes it onto the terminal.

Change-Id: Ic31808e10e1b876b35d0dbdeee0b5ede7b5b9883
2025-10-17 15:13:37 +02:00
eldritch horrors ab9dddfb7a libutil: wrap async kj exceptions as errors
this is important for rpc error reporting: if the peer disconnects we
want to throw a useful error that can e.g. let pending builds finish,
not kill the entire process because kj errors are non-lix exceptions.

Change-Id: I41ebe10ad7fd65d886238ce52eeca1fa243b1bb2
2025-10-17 11:33:00 +00:00
eldritch horrors 21d7cb8e1c libstore: initialize build hooks via rpc
this removes another file descriptor from the build hook. we are now
using only rpc to communicate between build hook and daemon, setting
the stage for a future world in which we don't even need build hooks

Change-Id: I706d9d3f2a420abd42fc7762c205931a45d3c8c5
2025-10-17 11:33:00 +00:00
eldritch horrors 9b886e2acf libstore: remove unused DerivationGoal::machineName
interpolation of the remote store name into activity text is now done in
the build hook, so we no longer need to send it back to the daemon. this
also reduces the dependency of the daemon on how *exactly* remote builds
are done a little bit, and maybe in time we can make these fully opaque.

Change-Id: Ie51c4f2cd107350fec4ad0a8ef83621ed86db6a2
2025-10-17 11:33:00 +00:00
eldritch horrors 711ad52776 libstore: move logSize into each log handler
there's no need for this to be an instance variable of the goals.

Change-Id: Iaa315e8b5d8649d8eb3c3fedc387c0f0c48945fc
2025-10-17 11:33:00 +00:00
eldritch horrors 645a0533eb libstore: move sandbox log handling to LocalDerivationGoal
only local builds can now have a builderOutFD, remote builds only log
via json streams. these two kinds of logs have different requirements
and each is only needed by its respective build kind. splitting these
apart thus makes sense, though ideally we'd also split DerivationGoal
into a RemoteDerivationGoal to clean up the rest, but that will wait.

Change-Id: Ib577537266d1160355ab9c44b4604ebda87a7d04
2025-10-17 11:33:00 +00:00
eldritch horrors a824877d45 libstore: don't treat legacy ssh like a sandbox
treat it like ssh-ng instead, and have the build hook do the translation
of ssh stdout to the json log message steam the derivaiton goal can use.
since the ssh-ng path also handles all store urls that aren't legacy ssh
we now have a single logging system that handles every remote build kind
equally, without requiring fd passing into the build hook. this is later
required to rpc-ify the log stream emitted by build hooks to the daemon.

Change-Id: Ifb522eb8a9745029050f16b1b3b3601a6ddac748
2025-10-17 11:33:00 +00:00
Alois Wohlschlager ac8209404c nix-eval-jobs: estimate memory usage using the heap size
When the maximum memory size is chosen too large (relative to the free memory
usage), so that the evaluation workers don't fit into the free memory, they
will start being swapped out. Further increase in memory usage will not be
reflected in the RSS, so they will not exit, and instead fill up all swap too
until the system runs out of memory.

The BDW-GC keeps track of its heap size, including any parts of it that may be
swapped out. For this reason it does not suffer from a similar problem. While
not all memory usage by the evaluation worker is accounted for by the GC, a
simple affine model for the overhead works reasonably well in practice. The
specific parameter values were determined using nixpkgs evaluation, and appear
to work quite well also for somewhat different workloads (like the `hydraJobs`
of Lix, which include a bunch of NixOS system configurations). So this is what
gets used.

When Lix is configured without the BDW-GC, of course this approach cannot be
used. In this case the old strategy is retained.

Change-Id: I6cb5f0a9b4ceda9dd14be165dda108cd6a6a6964
2025-10-16 15:22:54 +00:00
Alois Wohlschlager ed9cc5f448 nix-eval-jobs: move more worker initialization code to its file
Change-Id: Ief8b16220c153e402ecbdeabd576344c6a6a6964
2025-10-16 15:22:54 +00:00
Alois Wohlschlager e36ac58f93 nix-eval-jobs: hide hidden flags in the --help
Right now there are no hidden flags so this does not matter, but this will
change very soon.

Change-Id: I6d65ce9c32170b6d4f4a691ff9f8806a6a6a6964
2025-10-16 15:22:54 +00:00
eldritch horrors a85115bd33 libutil: add a log line splitter
we'll need this in other places too soon.

Change-Id: I8ddd57aefa4cbd1f8f7232f35e30a204ace77e58
2025-10-16 12:31:08 +00:00
eldritch horrors daff1b148f build-remote: de-optionalize an always-set parameter
Change-Id: Ifd79229cf5f608e3cfe5532ab24b2a4efb547e15
2025-10-16 11:51:27 +00:00
eldritch horrors cac395ffbc nej: fix some clang-tidy lints
these are really annoying when running `just lint` with nej enabled.

Change-Id: I8d35e77fe9905fdd6c28f95361f7431459079b96
2025-10-15 20:27:28 +02:00
eldritch horrors 59b583b46d n-e-j: disable darwin ci
it's a disaster. cf #1015

Change-Id: I4671c1ea0f1b6b2750b97cf3c42497460ef33259
2025-10-15 13:37:18 +00:00
helle 30a5f8ec15 doc: Mention Buildkite in the hacking document
Includes link to the Buildkite SSO login, which is currently not visible
anywhere else in documentation.

Change-Id: Ic3ec6412127b2ac0df7e6efc4ec6020cc9d8b34f
2025-10-15 09:21:49 +00:00
Jade LovelaceandCommentator2.0 faa0b4f291 functional2/http_server: add some example misbehaviour routes
Part of https://git.lix.systems/lix-project/lix/issues/949.

This also fixes a bug where our SO_REUSEADDR wasn't done before bind was
called and thus didn't work as intended.

Co-authored-by: Commentator2.0 <lix@crystal-cavern.systems>

Change-Id: Ie69fd594a32d7a6f538fb66d9d2cac6fe117d563
2025-10-13 18:30:51 +02:00
Jade LovelaceandCommentator2.0 556012e409 functional2: forbid chdir and setting environment
These are a footgun and are not acceptable in functional2 due to thread
safety, effects on other tests, etc.

Co-authored-by: Commentator2.0 <lix@crystal-cavern.systems>

Change-Id: I8d7285061eaa9bab27edd52f3646024c8cf605e5
2025-10-13 18:30:51 +02:00
Commentator2.0 844feb17b5 tests/functional2/lang: fix duplicate collection when both err and out are provided
When providing both a `.err.exp` and a `.out.exp` for a lang test
**not** containing a toml, the test was collected twice and hence threw
a duplicate id error.
This commit adresses this issue

Change-Id: Ia781d69e0f1db2809a30192d6cd0a98861e6cc32
2025-10-13 14:33:25 +00:00
eldritch horrors 2e3d97eb37 libstore: move currentHookLine into handleHookOutput
Change-Id: I00cdc7fae1b953f2d541bc966d15afa00c04b06a
2025-10-13 11:26:49 +00:00
eldritch horrors 26e8e3caac libutil: allow builders to create all activity types
this restriction was added in 4af2611bd1
and extended later. it makes little sense to allow *one* activity type
specifically and drop all others on the floor, especially since broken
builders can create transfer activities with ids that would be used by
other, "real" activities. the only thing achieved here is to drop logs
written in json format in the build sandbox, which does not help much.

we may revisit this restriction later during other log-related rework.

Change-Id: I8bda494083877b71a2f958470fa52380f6ec4968
2025-10-13 11:26:49 +00:00
eldritch horrors b04521e4ec libstore: move log handling fully into handleBuilderOutput
it was split up like this only because the old worker system had no
promises. since we have them now we can use them for encapsulation.

Change-Id: Idbd523eff617d4c53c14b125ab3dfce4979cdab1
2025-10-13 11:26:49 +00:00
eldritch horrors 1210a4e025 libstore: flush log lines only from handleBuilderOutput
also change the tests to ensure that we handle all lines correctly.

Change-Id: I13de286bda8eee57acd53af1ab5c081d3048b500
2025-10-13 11:26:49 +00:00
eldritch horrors f29dfb3d3c libutil: add buffer state management to loggers
currently all loggers can always accept messages and never suggest
flushing buffers. in the future this may change, and at that point
we're already fully set up for it. local loggers should never keep
asynchronous (i.e. network-backed) buffers, disk buffers are fine.
networked loggers will require buffers and periodic flushes later.

Change-Id: Ide2114f5bc17f4a1d289c92ed4f9381a1d59dacf
2025-10-13 11:26:49 +00:00
eldritch horrors 38b75b7367 libstore: make store optimization non-recursive
this keeps the call stack flat even for very deep store directories.
moving progress reporting out of the method optimizing one path also
lets us move the inevitable IO wait and associated promise overhead.

Change-Id: I5083d88b05c247c7e19a21a646bcf3f62bfa3200
2025-10-13 11:26:49 +00:00
eldritch horrors 34b1f362cd libutil: wrap activity updates in macros
we'll eventually want to yield to flush buffers from the non-sync versions.

Change-Id: I8bcbb1c4c8af39c7d7ebf5a76e1ce1dc98067e00
2025-10-12 22:37:31 +00:00
eldritch horrors b73a7f1815 libutil: clean up the Activity interface a bit
create activities from loggers themselves instead of passing the logger
as a constructor argument and allow direct construction of children, no
direct logger access needed. most call sites are not changed because we
still need to handle the "no parent" case, and the logger method can do
that more cleanly than a ternary at each site that creates an activity.
we may eventually want to create a root activity, which is cleaner too.

Change-Id: I295e056228dabb08a1316eba7973874784baa113
2025-10-12 22:37:31 +00:00
eldritch horrors 47000b658e libutil: don't trust RemoteStore activity ids
remotes can generate the same activity ids we might generate locally,
especially if the remote is accessed over the network. in that case a
pid collision is possible, and since activity ids are sequential it's
very possible to create colliding activity ids on both sides as well.

Change-Id: Id58074a41f5f7a59171b52818d1fb5a1beb4bf40
2025-10-12 22:37:31 +00:00
eldritch horrors ed034d0526 libutil: make Activity a real, move-only class
activities are scopes, and there's no reason scopes should not be able
to move around. this also means we can *create* them elsewhere without
also having to box them in some way, making parent relationships clear

Change-Id: I5df036e12ebd8270feb4dca1f23b6bee1f08e906
2025-10-12 22:37:31 +00:00
Lunaphied e9f6baedd1 libstore: add an informative message about disabling cgroups
Change-Id: I0271eff2f7a357af18ddd6ae0571d548a9adbe3e
2025-10-12 20:12:33 +02:00
Commentator2.0 d9c1c0925c unify version string between lix and nixos-module
Currenlty `lix` and `nixos-module` have slightly diverging version
numbering. This commit fixes the lix side of things, in accordance to
the format proposed in
https://git.lix.systems/lix-project/lix/issues/585#issuecomment-7336
which allows for better use of `builtins.compareVersions`

fixes #585

Change-Id: I2072b701f9fadf780e7aab67b62fbcd2ec5df76d
2025-10-11 18:47:46 +00:00
eldritch horrors b88a6e6f11 libexpr: use pascal strings for eval
this has no performance impact in any benchmarks we've run. nul bytes
are still used as implicit truncation points in many places all over:
rejecting them in all locations that treat them as a string end point
requires large changes such as using a proper path library everywhere

Change-Id: I936158bd435f6abf009a689adfbc24496262c578
2025-10-11 12:57:57 +02:00
eldritch horrors 53d172a308 libutil: remove unused CanonPath methods
Change-Id: Id593e0d03373af88e03a9da473935e5c00f86b33
2025-10-10 16:55:34 +00:00
eldritch horrors 2f16cf6014 treewide: lint syscalls for c string safety
Change-Id: I3caf476e59dcb7899ac5a3d83dfa3fb7ceaaabf0
2025-10-10 16:13:54 +00:00
eldritch horrors 3ef7e0711b Revert "functional2: fix in the development shell"
This reverts commit 42691f0d94.

Reason for revert: darwin builds time out now, apparently due to newly appearing infinite loops in the test suite

Change-Id: I81a09efd53636a287f3095ac87ed8854b6e57922
2025-10-10 12:49:45 +00:00
Commentator2.0 5e2412ea7e tests/functional2: migrate add.sh
Change-Id: I699929b61db19a4ab7bea8281df830de9fea6d04
2025-10-08 13:09:17 +02:00
Alois Wohlschlager 42691f0d94 functional2: fix in the development shell
The functional2 test suite was broken in the development shell for two mostly
separate reasons leading to "no such file or directory" errors:

* The `BUILD_TEST_SHELL` (already containing the correct path) would always be
  set from the Meson option `build_test_shell`, even to the empty string if
  that option is not set. Skip the overwrite in this case to make the
  environment variable work again.
* Sandboxed builds would fail to find their builder `/bin/sh`, since in the
  development shell busybox is not found, so no sandbox shell gets configured.
  Use the shell from `BUILD_TEST_SHELL` instead. (How this does not break the
  old functional tests remains a mystery.)

Change-Id: I6a6a696424e8caaef3f9b68e3738bfd58ea0b056
2025-10-07 20:14:48 +00:00
Raito Bezarius 41fab8a31c legacy/nix-store: do not crash on cache unsoundness
Local store make use of SQLite as a cache for output maps, they can
diverge with actual store reality.

Causing a crash at realization time.

We can just throw an error instead.

Change-Id: I943a4bddfd2461a34933e9ec079a6638ab58e5b1
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-10-07 13:35:42 +00:00
Raito Bezarius feab75bde0 legacy/nix-build: create various temporary directories into a known tempdir
Fixes fj#940.

When running `nix-shell`, the `$NIX_BUILD_TOP` environment variable is
set to `$TMPDIR` or `/tmp`.

nixpkgs stdenv uses $NIX_BUILD_TOP to create `$NIX_BUILD_TOP/env-vars`
which contains all the environment variables set by stdenv. This is used
for debugging purposes in combination with `--keep-failed` to reload the
bash environment of a derivation.

`$TMPDIR` is often unset, therefore, `/tmp/env-vars` was constantly
being created. On a multi-user system or, when you run Lix as root, you
might create a `/tmp/env-vars` with different permission bits.

As a result, `nix-shell` can cease to function because that file will
fail creation for an unprivileged user for example.

fj#940 rightfully remark that the code is not consistent between
nix3-develop and nix-shell and it should be reworked.

Change-Id: Iddf15945385d8bd497b2800b37fee5e1f97689b7
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-10-07 03:28:45 +00:00
eldritch horrorsandRaito Bezarius 7e665feae6 libexpr: make .debug_gdb_scripts aligned
This avoids mangling debug gdb scripts and breaking them resulting in no
script loading at all.

Change-Id: Ib5e7034c37a1e41d1a1c9bd8442b8b8b8758e350
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-10-06 23:00:40 +02:00
Raito Bezariusandeldritch horrors 7b6bcffe8b libexpr/genericClosure: de-ptr-ize UnsafeValueList
UnsafeValueList held pointers from Value which were not necessarily GC
allocated, causing mayhem when evaluating something with genericClosure
(texlive environments).

We get rid of storing pointers and we let comparison take places on
const references.

We keep `gc_allocator<Value>` so that GC can scan things it allocates
inside the `res` list.

Co-authored-by: eldritch horrors <pennae@lix.systems>
Reported-by: qbit
Change-Id: I4fed3a3d9e18a2ef2d751f32d81801540e196f92
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-10-06 23:00:40 +02:00
Raito Bezarius b5d31b7780 libexpr: CompareValues can take const references
Change-Id: I5ab1e9f71c2f066eb9bb39683ff00c5190c1fcd7
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-10-06 22:27:36 +02:00
eldritch horrors 14737888e0 libexpr: add a gdb script with value pretty printers
Change-Id: Ia4a6feb194c73dddf468c78287962713bef582f7
2025-10-05 16:23:05 +02:00
eldritch horrors 80654b84b6 libexpr: de-ptr-ize Value references
thunk values are shareable, and we can represent invalid/uninitialized
values with a special bit pattern that makes no sense otherwise. there
is no need to keep allocating values on the heap, instead we can treat
values like reference-counted smart pointers to heap objects, which in
turn lets us save a lot of allocations and, ultimately, gc heap space.

compared to our baseline (main of 2025-09-27) we save 15%+ memory on a
system rebuild and 17% on nix search. eval time regresses by ~3% for a
system rebuild, while nix search is 7% faster. further optimization is
probably possible (but for now this will just have to be good enough).

Change-Id: Ib6c47acdbe2fac4f76a83c2269f16f30ef66b2e1
2025-10-05 16:23:05 +02:00
eldritch horrors 28c4431c69 libexpr: reformat stuff we'll change soon
mainly to keep the next diff smaller. it'll be large enough as is.

Change-Id: Ib8a34520f03539cbf6aa2f0e66cbed05fe1225eb
2025-10-05 16:22:35 +02:00
eldritch horrors db35aa753b libexpr: de-ptr-ize many Value uses
with thunk state being shareable we no longer need to worry about value
uniqueness, only about value lifetime. this means we can liberally drop
indirections and allocations, passing references instead of pointers or
using stack memory instead of gc-managed memory for some intermediates.

Change-Id: I2d48a6fd57a376d544bd9bd2d05e5420611986d1
2025-10-05 16:22:35 +02:00
eldritch horrors 23df732ddb libexpr: remove ValueVector{,Map}
these typedefs were convenient in the past, but now they're not really.

Change-Id: I7522f7582ead545148af8e6444295d620ee0872a
2025-10-05 16:22:35 +02:00
eldritch horrors ad4c6b6eb1 libexpr: remove unused return types
Change-Id: Ief898b70f9781bd3dbe66734710f388daa2f2fed
2025-10-05 16:22:34 +02:00
eldritch horrors f4dee18693 libexpr: remove type punning in primt_attrValues
there's no measurable performance gain in real-world testing to abusing
our list value storage like this. we haven't tested how much storage we
actually need on the stack to offset most of the temporary storage cost
and used 128 as a "good enough" value instead, reserving 1 kiB of stack
space on 64 bit platforms in a leaf function. this should do for a bit.

Change-Id: Ie98519b1da3e6fe685da88d1c44ffb4580fb592a
2025-10-05 14:56:56 +02:00
eldritch horrors 8108e8a760 libexpr: tag Value::Acb
Value is already tagged. Value::Acb blocks are allocated for lambdas (so
we can fit the value tag into the three bits we have available), but the
current layout is rather wasteful for this purpose. the type bits can be
stored together with parts of pointers, which in the lambda case will be
the scope the lambda captures. the expr could also be used, but Env is a
gc-allocated item and thus guaranteed to be aligned properly for tagging

Change-Id: Ia685875387c7795bc4a00d73d1ce3cfea84e7297
2025-10-05 14:56:56 +02:00
eldritch horrors b19bbdfee1 libexpr: make thunk state shareable
this is a strong prerequisite for making values themselves copyable
without duplicating evaluation side effects. with this we can treat
`Value` the way we treated `Value *` to date and drop indirections.

Change-Id: I08f30d12697614a3ae7149615f6f1da83b13f52b
2025-10-05 14:56:56 +02:00
eldritch horrorsandRaito Bezarius 57333a0e60 libutil: preserve ForeignException type info
preserve type information of wrapped exception when constructing a
ForeignException. this requires using `std::current_exception` for
construction in all cases as passing the exception by reference to
any function erases dynamic type information: `throw` may copy the
thrown exception from its source expression; this only works as we
need it to if static and dynamic type of the expression are equal.

fixes #1000

Change-Id: I7167f298540de7c43c2566592bdb8d7a93c34d31
2025-10-05 00:40:22 +00:00
eldritch horrors dc99a5ea1f libexpr: reduce Value size to one pointer
Change-Id: Id6e4a2f68eaa4afdd006379ebcf839c4a126b819
2025-10-04 21:06:52 +02:00
eldritch horrors 0e9e9fd917 libexpr: heap-allocate "large" integers
Change-Id: Ic391f2f1bf87f044d7a688196ba9e0ad766d65aa
2025-10-04 16:31:45 +02:00
eldritch horrors 79586575c5 libexpr: heap-allocate app nodes
Change-Id: I9a39dcf0be7589cedf494757e21665c5d50e446b
2025-10-04 16:31:45 +02:00
eldritch horrors 94a866d44f libexpr: heap-allocate thunk control state
Change-Id: I20d8ab1d6f683c0a2f3b77edf9bdad147d62c8fa
2025-10-04 16:31:45 +02:00
eldritch horrors d84f3003cf libexpr: move lambdas to auxiliary storage
Change-Id: Ibe4885f17c0ba1634ed6dbca0a45f8bd4619d69b
2025-10-04 16:25:05 +02:00
eldritch horrors 1b5f4eb5eb libexpr: move primops to auxiliary storage
same as for null: we have few of them, they're statically allocated,
and they're not the largest contributor to the value population. not
storing them in Value itself frees up resources we *will* use later.

Change-Id: I521f9f243f48f56a78f7bffdf1dc1f0bc40a5c2d
2025-10-04 16:25:04 +02:00
eldritch horrors 05b80a5ba1 libexpr: move null to auxiliary storage
we only need the one object for it. there's no need to waste precious
bits of the value internal type enum for this simple singleton datum.

Change-Id: Ie314b5bf429015e518798d9d65ad8ab2bb84a38e
2025-10-04 16:20:07 +02:00
eldritch horrors 1d998a343b libexpr: move floats to auxiliary storage
floats are used very rarely, and our float support is bad enough to
strongly discourage using them on reproducibility grounds alone. we
can thus move them to more expensive storage without hurting folks.

Change-Id: I1086f612f85e294dd3fae4a2d334e09f52bbe4a8
2025-10-04 16:20:07 +02:00
eldritch horrors 4d7f468c87 libexpr: move external value refs to aux storage
external values very rarely appear during eval "normal" eval, and
creating them is pretty expensive. does *anything* even use them?

Change-Id: Id50fa3f76b7e1f551d550d99996a1ed5880b2531
2025-10-04 16:20:07 +02:00
eldritch horrors 90be3e34c9 libexpr: heap-alloc string control blocks
despite not using allocation caches this does not have a statistically
significant performance impact, with less than 1% extra memory needed.

Change-Id: Ibe51a55ba986e471f217f3724977af17880fafff
2025-10-04 16:20:07 +02:00
f1ef994f12 libexpr: enable parallel marking in boehm-gc
Using parallel marking in GC speeds up evaluation a fair bit

```
Benchmark 1: ./lix-main/bin/nix search nixpkgs hello --no-eval-cache
  Time (mean ± σ):     20.740 s ±  0.046 s    [User: 17.583 s, System: 3.062 s]
  Range (min … max):   20.678 s … 20.794 s    10 runs

Benchmark 2: ./lix-parallel-marking/bin/nix search nixpkgs hello --no-eval-cache
  Time (mean ± σ):     15.037 s ±  0.080 s    [User: 19.602 s, System: 3.125 s]
  Range (min … max):   14.960 s … 15.227 s    10 runs

Summary
  ./lix-parallel-marking/bin/nix search nixpkgs hello --no-eval-cache ran
    1.38 ± 0.01 times faster than ./lix-main/bin/nix search nixpkgs hello --no-eval-cache
```

Based-on: https://github.com/NixOS/nix/pull/13708
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Change-Id: Ibc7625f21e0ee7c8ad66203eeb3aca5d83977731
2025-10-01 23:07:28 +00:00
eldritch horrors 1c4e77387a libexpr: remove tPrimOpApp
using the same nodes as tApp is possible, and thanks to multi-arg app
nodes it can even be a bit faster than the linked lists used to date.

Change-Id: Idccb7c0b54c808e62da85d1c42ee09e6e92c4f7b
2025-09-29 17:56:05 +02:00
eldritch horrors 0b5b14ddc7 libexpr: add multi-arg app nodes
these behave like the old chains of app nodes, but they can store more
than one argument per node. for tApp values themselves this is not all
that useful, but if we could share tApp and tPrimOpApp backing storage
we could avoid creating and traversing the linked lists of values that
are currently needed to represent partially applied builtin functions.

Change-Id: I5a2a02d9733e1e0be5443459e2998d62fd3b9a5b
2025-09-29 17:56:05 +02:00
eldritch horrors 3350ab8a56 libexpr: use std::span for callFunction
Change-Id: I8c94bafabdb2416c85d9721d3d6424f52fbd45e0
2025-09-29 17:56:05 +02:00
eldritch horrors 0d69da184c libexpr: unify strings and paths
paths already are just strings with slightly magic semantics. the type
can mirror that at no perf cost, letting us drop one internalType tag.

Change-Id: I98acaa4fe3bedd28fc0841e1b81184d8dcddacc5
2025-09-29 17:56:05 +02:00
eldritch horrors d23e3a09a4 libexpr: alloc list storage as a (length, vla) type
memory overhead is minimal and performance impact not measurable. once
we've done something like this for all value types that don't fit in a
single machine word we can cut a word from Value, offsetting the cost.

Change-Id: I9813bacd7e851957ad3426aed8f74033179a4212
2025-09-29 17:55:54 +02:00
eldritch horrors 01f1852360 libexpr: don't inline small lists into values
this has no measurable performance impact thanks to the new caches.

Change-Id: Ib403a9a567161675f78e8c5d314d6340183d181d
2025-09-29 15:22:41 +02:00
eldritch horrors 739624ddcf libexpr: cache more allocation sizes
we now use a single cache set for a number of sizes from one to eight
words. this also matches small attrsets, but perf impact seems small.

Change-Id: Icf16b329b98a20fcc9fe75e6395e148f0852c798
2025-09-29 15:22:41 +02:00
eldritch horrors c7cc7d6c31 libexpr: "hide" Value union members
on its own this is not very useful, but having accessors for every value
kind is a prerequisite for doing smart things with Value than the union.
the net effect for now is only to add a few parentheses across the tree.

Change-Id: I88688ac09eb08495dad1eb221034ca540f094950
2025-09-29 15:22:41 +02:00
eldritch horrors 2dae1141d9 bench: add memory benchmark
Change-Id: I4b2aa305b452a3a0c73e37953a5c332219d43d2c
2025-09-29 15:22:41 +02:00
eldritch horrors f31e8e2b55 bench: allow benchmarking a single build
Change-Id: Ie77247f9348eaeea740b0f894819ee2383d92312
2025-09-28 13:35:37 +00:00
eldritch horrors e4ee5e2eb5 testing: disable substitution in f2
mirrors f1. much faster.

Change-Id: I38bbbd5b26220f480afe76ef9302e5c90f21507a
2025-09-28 13:19:13 +00:00
310 changed files with 9743 additions and 5424 deletions
+61 -8
View File
@@ -9,6 +9,7 @@ import tempfile
import platform
import shlex
import textwrap
import dataclasses
flake_args = ["--extra-experimental-features", "nix-command flakes"]
cases = {
@@ -64,8 +65,9 @@ arg_parser.add_argument(
)
arg_parser.add_argument(
'--mode',
choices=[ "walltime" ] + [ "icount" ] if platform.system() == 'Linux' else [], # perf doesn't run on Darwin
default="walltime",
nargs='+',
choices=[ "walltime", "memory" ] + [ "icount" ] if platform.system() == 'Linux' else [], # perf doesn't run on Darwin
default=[ "walltime" ],
)
arg_parser.add_argument(
'--daemon',
@@ -73,8 +75,8 @@ arg_parser.add_argument(
help='Run a temporary daemon for the benchmark instead of using a local store directly',
)
args = arg_parser.parse_args()
if len(args.builds) < 2:
raise ValueError("need at least two build directories to compare")
if len(args.builds) < 1:
raise ValueError("need at least one build directory to benchmark")
benchmarks: list[str] = []
if args.cases is None:
@@ -162,6 +164,54 @@ def bench_icount(env):
print(" relative instructions:", int(instr)/perf_results_for[case][0][1])
print("\n")
@dataclasses.dataclass
class MemoryStatistics:
envBytes: int
listBytes: int
setBytes: int
valueBytes: int
heapBytes: int
heapSize: int
def bench_memory(env):
path = "bench/bench-memory.json"
env = env | {
'NIX_SHOW_STATS': '1',
'NIX_SHOW_STATS_PATH': path,
}
results: dict[str, list[tuple[str, MemoryStatistics]]] = {}
for case in benchmarks:
for build in args.builds:
case_command = make_full_command(build, case)
commandline = [ "sh", "-c", case_command ]
print("running", case_command)
subprocess.run(commandline, env=env, check=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
with open(path) as fd:
stats = json.load(fd)
results.setdefault(case, []).append((case_command, MemoryStatistics(
envBytes=stats['envs']['bytes'],
listBytes=stats['list']['bytes'],
setBytes=stats['sets']['bytes'],
valueBytes=stats['values']['bytes'],
heapSize=stats['gc']['heapSize'],
heapBytes=stats['gc']['totalBytes'],
)))
print("Benchmarks summary\n---\n")
for (case, entries) in results.items():
for cmd, stats in entries:
print(cmd)
print("-" * min(80, len(cmd)))
print(f" env bytes: {stats.envBytes :15d} | {(stats.envBytes / entries[0][1].envBytes) :.3f}x")
print(f" list bytes: {stats.listBytes :15d} | {(stats.listBytes / entries[0][1].listBytes) :.3f}x")
print(f" set bytes: {stats.setBytes :15d} | {(stats.setBytes / entries[0][1].setBytes) :.3f}x")
if not entries[0][1].valueBytes:
print(f" value bytes: {0:15d}")
else:
print(f" value bytes: {stats.valueBytes:15d} | {(stats.valueBytes / entries[0][1].valueBytes):.3f}x")
print(f" heap alloc'd: {stats.heapBytes :15d} | {(stats.heapBytes / entries[0][1].heapBytes) :.3f}x")
print(f" heap size: {stats.heapSize :15d} | {(stats.heapSize / entries[0][1].heapSize) :.3f}x")
print("\n")
with tempfile.TemporaryDirectory() as tmp_dir:
subprocess.run([
@@ -178,7 +228,10 @@ with tempfile.TemporaryDirectory() as tmp_dir:
])
subenv["NIX_DAEMON_SOCKET_PATH"] = f"{tmp_dir}/daemon"
if args.mode == "walltime":
bench_walltime(subenv)
else:
bench_icount(subenv)
for mode in args.mode:
if mode == "walltime":
bench_walltime(subenv)
elif mode == "memory":
bench_memory(subenv)
else:
bench_icount(subenv)
+5
View File
@@ -86,6 +86,11 @@ ericson:
display_name: John Ericson
github: ericson2314
getchoo:
display_name: Seth Flynn
forgejo: getchoo
github: getchoo
gilice:
forgejo: gilice
-16
View File
@@ -1,16 +0,0 @@
---
synopsis: First argument to `--arg`/`--argstr` must be a valid Nix identifier
issues: [fj#496]
category: "Breaking Changes"
credits: [ma27]
---
The first argument to `--arg`/`--argstr` must be a valid Nix identifier, i.e.
`nix-build --arg config.allowUnfree true` is now rejected.
This is because that invocation is a false friend since it doesn't set
`{ config = { allowUnfree = true; }; }`, but `{ "config.allowUnfree" = true; }`.
The idea is to change the behavior to the latter in the long-term. For that,
non-identifiers started giving a warning since 2.92 and are now rejected to give people
who depend on that a chance to notice and potentially weigh in on the discussion.
-20
View File
@@ -1,20 +0,0 @@
---
synopsis: "Improved susbtituter query speed"
issues: []
cls: []
category: Improvements
credits: [horrors]
---
The code used to query substituters for derivations has been rewritten slightly
to take advantage of our asynchronous runtime. Such queries run for every build
that could download from substituters and processes every derivation that isn't
yet present on the local system. Previously Lix would use `http-connections` to
limit query concurrency, even for modern caches that support HTTP/2 and have no
limit on how many queries can be run concurrently on one single connection. Lix
no longer does this, resulting in approximately 60% reduction in query time for
medium-sized closures (e.g. NixOS system closures) during testing, although the
exact number depends greatly on local network latency and generally improves as
latency increases. Unlike previously setting `http-connections` to `1` or other
low values no longer brings a massive penalty in query performance if the cache
in use by the querying system supports HTTP/2 (as e.g. `cache.nixos.org` does).
-12
View File
@@ -1,12 +0,0 @@
---
synopsis: "`build-dir` no longer defaults to `temp-dir`"
cls: [3453]
category: "Fixes"
credits: [horrors]
---
The directory in which temporary build directories are created no longer defaults
to the value of the `temp-dir` setting to avoid builders making their directories
world-accessible. This behavior has been used to escape the build sandbox and can
cause build impurities even when not used maliciously. We now default to `builds`
in `NIX_STATE_DIR` (which is `/nix/var/nix/builds` in the default configuration).
@@ -1,60 +0,0 @@
---
synopsis: "Global certificate authorities are copied inside the builder's environment"
issues: [gh#12698, fj#885]
cls: [3765]
category: Fixes
credits: [raito, emilazy]
---
Previously, CA certificates were only installed at
`/etc/ssl/certs/ca-certificates.crt` for sandboxed builds on Linux.
This setup was insufficient in light of recent changes in `nixpkgs`, which now
enforce HTTPS usage for `fetchurl`, even for fixed-output derivations, to
mitigate confidentiality risks such as `netrc` or credentials leakage.
`nixpkgs` still make use of a special package called `cacerts` which contains a
copy of the CA certificates maintained by Nixpkgs and added as a reference for
TLS-enabled fetchers.
As a result, having a consistent and trusted certificate authority in all
builder environments is becoming more essential.
On `nix-darwin`, the `NIX_SSL_CERT_FILE` environment variable is always
explicitly defined, but it is ignored by the sandbox setup.
Simultaneously, Nix evaluates and propagates impure environment variables via
`lib.proxyImpureEnvVars`, meaning that if `NIX_SSL_CERT_FILE` is set (which
influences the default value for `ssl-cert-file`), it will be forwarded
unchanged into the builder environment.
However, on Linux, Nix also *copies* the CA file into the sandbox, creating a
discrepancy between the value of `NIX_SSL_CERT_FILE` and the actual trusted
certificate path used during the build.
This divergence caused confusion and was partially addressed by attempts to
whitelist the CA path in the Darwin sandbox (see cl/2906), but that approach
involved a non-trivial path canonicalization step and is not as general as this one.
To address this properly, we now emit a warning and override
`NIX_SSL_CERT_FILE` inside the builder, explicitly pointing it to the CA file
copied into the sandbox.
This eliminates ambiguity between `NIX_SSL_CERT_FILE`
and `ssl-cert-file`, ensuring consistent trust anchors across platforms.
This warning might become a hard error as we figure out what to do regarding
`lib.proxyImpureEnvVars` in nixpkgs.
The behavior has been verified across sandboxed and unsandboxed builds on both
Linux and Darwin.
As a consequence of this change, approximately 500KB of CA certificate data is
now unconditionally copied into the build directory for fixed-output
derivations.
While this ensures consistent trust verification without having to restart the
daemon after system upgrades, it may introduce a slight overhead in build
performance. At present, no optimizations have been implemented to avoid this
copy, but if this overhead proves noticeable in your workflows, please open an
issue so we can evaluate and possibly implement different strategies to render
trust anchors visible.
-69
View File
@@ -1,69 +0,0 @@
---
synopsis: New cgroup delegation model
issues: [fj#537, fj#77]
cls: [3230]
category: "Breaking Changes"
credits: [raito, horrors, lheckemann]
---
Builds using cgroups (i.e. `use-cgroups = true` and the experimental feature
`cgroups`) now always delegate a cgroup tree to the sandbox.
Compared to the original C++ Nix project, our delegation includes the
`subtree_control` file as well, which means that the sandbox can disable
certain controllers in its own cgroup tree.
This is a breaking change because this requires the Nix daemon to run with an
already delegated cgroup tree by the service manager.
## How to setup the cgroup tree with systemd?
systemd offers knobs to perform the required setup using:
```
[Service]
Delegate=yes
DelegateSubtree=supervisor
```
These directives are now included in our systemd packaging.
## What about using Nix as root without connecting to the daemon?
Builds run as `root` without connecting to the daemon relying on the cgroup
feature are now broken, i.e.
```console
# nix-build --use-cgroups --sandbox ... # will not work
```
Consider doing instead:
```console
# systemd-run --same-dir --wait -p Delegate=yes -p DelegateSubgroup=supervisor nix-build --use-cgroups ...
```
If you need to disable cgroups temporarily, remember that you can do
`NIX_CONF='include /etc/nix/nix.conf\nuse-cgroups = false' nix-build ...` or
`nix-build --no-use-cgroups ...`.
## What about other service managers than systemd?
systemd has a [documentation](https://systemd.io/CGROUP_DELEGATION/) on how to
handle cgroup delegation from service management perspective.
If your service manager adheres to systemd semantics, e.g. writing an extended
attribute `user.delegate=1` on the delegated cgroup tree directory and moving
the `nix-daemon` process inside a cgroup tree to respect the inner process
rule, then, the feature will work as well.
## Why is the cgroup feature still experimental?
While the cgroup feature unlocks many use cases, its behavior and integration (e.g. user experience), especially at scale on build farms or in multi-tenant environments, are not yet fully matured. Theres also potential for deeper systemd integration (e.g. using slices and scopes) that has not been fully explored.
To avoid locking in an unstable interface, were keeping the experimental flag until we have validated the feature across a broader range of scenarios, including but not limited to:
* Nix as root
* Hydra-style build farms
* Forgejo CI runners
* Shared remote builders
@@ -1,16 +0,0 @@
---
synopsis: Deprecation of CA derivations, dynamic derivations, and impure derivations
issues: [fj#815]
cls: []
significance: significant
category: Miscellany
credits: []
---
Content-addressed derivations are now deprecated and slated for removal in Lix 2.94.
We're doing this because the CA derivation system has been a known cause of problems
and inconsistencies, is unmaintained, habitually makes improving the store code very
difficult (or blocks such improvements outright), and is beset by a number of design
flaws that in our opinion cannot be fixed without a full reimplementation from zero.
Dynamic derivations and impure derivations are built on the CA derivation framework,
and owing to this they too are deprecated and slated for removal in another release.
-25
View File
@@ -1,25 +0,0 @@
---
synopsis: "Hitting Control-C twice always terminates Lix"
cls: [3574]
issues: []
category: "Improvements"
credits: [horrors]
---
Hitting Control-C or sending `SIGINT` to Lix now prints an informational message
if it is still running after on second, the second Control-C/`SIGINT` terminates
Lix immediately without waiting for any shutdown code to finish running. Lix did
not treat the second such event differently from first in the past; this made it
impossible to easily terminate running Lix processes that got stuck in e.g. very
expensive Nixlang code that never interacted with the store. We now terminate as
soon as the user hits Control-C again without waiting any more, to much the same
effect as putting Lix into the background and killing it immediately afterwards.
This means you can now more conveniently break out of stuck Nixlang evaluations:
```
nix-instantiate --eval --expr 'let f = n: if n == 0 then 0 else f (n - 1) + f (n - 1); in f 32'
^CStill shutting down. Press ^C again to abort all operations immediately.
^C
❌130
```
-12
View File
@@ -1,12 +0,0 @@
---
synopsis: "libstore: exponential backoff for downloads"
issues: [lix#932]
cls: [3856]
category: Fixes
credits: [ma27]
---
The connection timeout when downloading from e.g. a binary cache is exponentially
increased per failure. The option `connect-timeout` is now an alias to `max-connect-timeout`
which is the maximum value for a timeout. The start value is controlled
by `initial-connect-timeout` which is `5` by default.
-9
View File
@@ -1,9 +0,0 @@
---
synopsis: Fix develop shells for derivations with escape codes
issues: [fj#991]
cls: [4154, 4155]
category: Fixes
credits: [Qyriad]
---
ASCII control characters (including `\e`, used for ANSI escape codes) in derivation variables are now correctly escaped for `nix develop` and `nix print-dev-env`, instead of erroring.
-13
View File
@@ -1,13 +0,0 @@
---
synopsis: "nix-store --delete: always remove obsolete hardlinks"
issues: []
cls: [3188]
category: Fixes
credits: [lheckemann]
---
Deleting specific paths using `nix-store --delete` or `nix store
delete` previously did not delete hard links created by `nix-store
--optimise` even if they became obsolete, unless _all_ of the given
paths were deleted successfully. Now, hard links are always cleaned
up, even if some of the given paths could not be deleted.
-24
View File
@@ -1,24 +0,0 @@
---
synopsis: "Report GC statistics correctly"
issues: []
cls: [3188]
category: Fixes
credits: [lheckemann]
---
Deleting specific paths using `nix-store --delete` or `nix store delete` previously did
not report statistics correctly when some of the paths could not be deleted, even if
others were deleted:
```
$ nix store delete /nix/store/9bwryidal9q3g91cjm6xschfn4ikd82q-hello-2.12.1 --delete-closure -v
finding garbage collector roots...
deleting '/nix/store/9bwryidal9q3g91cjm6xschfn4ikd82q-hello-2.12.1'
0 store paths deleted, 0.00 MiB freed
error: Cannot delete some of the given paths because they are still alive. Paths not deleted:
k9bxzr1l92r5y6mihrkbpbr3fmc8qszx-libidn2-2.3.8
mbx9ii53lzjlrsnlrfmzpwm33ynljwdn-libunistring-1.3
rf8hcy6bldxdqc0g6q1dcka1vh47x69s-xgcc-14.2.1.20250322-libgcc
vbrdc5wgzn0w1zdp10xd2favkjn5fk7y-glibc-2.40-66
To find out why, use nix-store --query --roots and nix-store --query --referrers.
```
-16
View File
@@ -1,16 +0,0 @@
---
synopsis: Add `inputs.self.submodules` flake attribute
issues: [fj#942]
cls: [3839]
category: Features
credits: [edolstra, kasimeka]
---
A port of <https://github.com/NixOS/nix/pull/12421> to Lix, which:
- adds a general `inputs.self` flake attribute that retroactively applies
configurations to a flake after it's been fetched, then triggers a refetch of
the flake with the new config.
- implements `inputs.self.submodules` that allows a flake to declare its need
for submodules, which are then fetched automatically with no need to pass
`?submodules=1` anywhere.
@@ -1,16 +0,0 @@
---
synopsis: Add hyperlinks in attr set printing
issues: []
cls: [3790]
category: Features
credits: [jade]
---
The attribute set printer, such as is seen in `nix repl` or in type errors, now prints hyperlinks on each attribute name to its definition site if it is known.
Example: all of the attributes shown here are hyperlinks to the exact definition site of the attribute in question:
```
$ nix eval -f '<nixpkgs>' lib.licenses.mit
{ deprecated = false; free = true; fullName = "MIT License"; redistributable = true; shortName = "mit"; spdxId = "MIT"; url = "https://spdx.org/licenses/MIT.html"; }
```
-24
View File
@@ -1,24 +0,0 @@
---
synopsis: Repl debugger uses `--ignore-try` by default
issues: [lix#666]
cls: [3488]
category: Breaking Changes
credits: [jade]
---
Previously, using the debugger meant that exceptions thrown in `builtins.tryEval` would trigger the debugger.
However, this caught nixpkgs initialization code, which is unhelpful in the majority of cases, so we changed the default.
To get the old behaviour, use `--no-ignore-try`.
```
$ nix repl --debugger --expr 'with import <nixpkgs> {}; pkgs.hello'
Lix 2.94.0-dev-pre20250625-9a59106
Type :? for help.
error: file 'nixpkgs-overlays' was not found in the Nix search path (add it using $NIX_PATH or -I)
This exception occurred in a 'tryEval' call. Use --ignore-try to skip these.
Added 13 variables.
nix-repl>
```
@@ -1,25 +0,0 @@
---
synopsis: "Fallback to safe temp dir when build-dir is unwritable"
issues: [fj#876]
cls: [3501]
category: "Fixes"
credits: ["raito", "horrors"]
---
Non-daemon builds started failing with a permission error after introducing the `build-dir` option:
```
$ nix build --store ~/scratch nixpkgs#hello --rebuild
error: creating directory '/nix/var/nix/builds/nix-build-hello-2.12.2.drv-0': Permission denied
```
This happens because:
1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
2. The user lacks permissions for that path.
We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
**Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.
-47
View File
@@ -1,47 +0,0 @@
---
synopsis: Experimental integer coercion in interpolated strings
issues: []
cls: [3198]
category: "Features"
credits: [raito, delroth, horrors, winter]
---
Ever tried interpolating a port number in Lix and ended up with something like this?
```nix
"http://${config.network.host}:${builtins.toString config.network.port}/"
```
You're not alone. Thousands of Lix users suffer every day from excessive `builtins.toString` syndrome. Its 2025, and we still have to cast integers to use them in strings.
To address this, Lix introduces the **`coerce-integers`** experimental feature. When enabled, interpolated integers within `"${...}"` are automatically coerced to strings. This allows writing:
```nix
"http://${config.network.host}:${config.network.port}/"
```
without additional conversion.
To enable the feature, you need to add `coerce-integers` to your set of experimental features.
### Stabilization criteria
The `coerce-integers` feature is experimental and limited strictly to string interpolation (`"${...}"`). Before stabilization, the following must hold:
1. **Interpolation-only**
Coercion must not occur outside interpolation. Expressions like `"" + 42` must continue to fail.
2. **Expectation that no explicit cast are being observed**
Cases observing explicit coercion (e.g., via `tryEval` gadget or similar) are expected not to be load-bearing in actual production code.
### Timeline for stabilization
If the feature proves safe and is widely adopted across typical usage (e.g., actual configurations in the wild turning on the flag, non-trivial out-of-tree projects using it), the experimental flag will be removed **after six months of active use or two Lix releases**, whichever is longer.
This avoids locking the feature in experimental status indefinitely, as happened with Flakes, while allowing time for validation and ecosystem integration.
### What about coercing floats or more?
Coercion beyond integers -- such as for floats or other types -- is **not planned**, even under an experimental flag. Questions like "what is the canonical string representation of a float?" involve subtle and context-dependent trade-offs. Without a robust and principled mechanism to define and audit such behavior, introducing broader coercion risks setting unintended and hard-to-reverse precedents. The scope of `coerce-integers` is intentionally narrow and will remain so.
In terms of outlook, a proposal like https://git.lix.systems/lix-project/lix/issues/835 could pave the way for a better solution.
@@ -1,16 +0,0 @@
---
synopsis: Parse overflowing JSON number literals as floatingpoint
issues: []
cls: [3919]
category: "Fixes"
credits: [emilazy]
---
Previously, `builtins.fromJSON "-9223372036854775809"` would
return a floatingpoint number, while `builtins.fromJSON
"9223372036854775808"` would cause an evaluation error. This was
introduced with the banning of integer overflow in Lix 2.91; previously
the latter would result in C++ undefined behaviour. These cases are
now treated consistently with JSONs model of a single numeric type,
and JSON number literals that do not fit in a Nixlanguage integer
will be parsed as floatingpoint numbers.
-13
View File
@@ -1,13 +0,0 @@
---
synopsis: "`--keep-failed` chowns the build directory to the user that request the build"
issues: []
cls: []
category: Improvements
credits: [horrors]
---
Running a build with `--keep-failed` now chowns the temporary directory from the
builder user and group to the user that request the build if the build came from
a local user connected to the daemon. This makes inspecting failed derivations a
lot easier. On Linux the build directory made visible to the user will not be in
the same path as it was in the sandbox and continuing builds will usually break.
@@ -1,11 +0,0 @@
---
synopsis: Fix handling of OSC codes in terminal output
issues: [fj#160]
cls: [3143]
category: Fixes
credits: [lilyball]
---
OSC codes in terminal output are now handled correctly, where OSC 8 (hyperlink) is preserved any
time color codes are allowed and all other OSC codes are stripped out. This applies not only to
output from build commands but also to rendered documentation in the REPL.
-14
View File
@@ -1,14 +0,0 @@
---
synopsis: Better debuggability on fixed-output hash mismatches
issues: []
cls: []
category: Improvements
credits: [lheckemann]
---
Fixed-output derivation hash mismatch error messages will now include the path that was
produced unexpectedly, and this path will be registered as valid even if `--check`
(`nix-store`, `nix-build`) or `--rebuild` (`nix build`) was passed. This makes comparing
the expected path with the obtained path easier, and is useful for debugging when
upstreams modify previously-published releases or when changes in fixed-output
derivations' dependencies affect their output unexpectedly.
-10
View File
@@ -1,10 +0,0 @@
---
synopsis: "nix-eval-jobs: support `--no-instantiate` flag"
issues: [fj#987]
category: Features
credits: [mic92,ma27]
---
`nix-eval-jobs` now supports a flag called `--no-instantiate`. With this enabled,
no write operations on the eval store are performed. That means, only evaluation is
performed, but derivations (and their gcroots) aren't created.
@@ -1,29 +0,0 @@
---
synopsis: "Fix nix develop for derivations that rejects dependencies with structured attrs"
issues: [fj#997]
cls: [4182]
category: Fixes
credits: [raito]
---
For the sake of concision, we refer to `disallowedReferences` in what follows,
but all output checks were equally fixed:
`{dis,}allowed{References,Requisites}`.
Derivations can define *output checks* to reject unwanted dependencies, such as
interpreters like `bash` or compilers like `gcc`. This can be done in two ways:
* **Legacy style**: `disallowedReferences = [ ... ]` in the environment.
* **Structured attrs**: `outputChecks.<output>.disallowedReferences = [ ... ]`,
typically used in `__json`.
Only the structured form supports derivations with multiple outputs.
`nix develop` internally rewrites derivations to create development shells. It
relied on the legacy `disallowedReferences`, and failed to honor the structured
variant. This led to broken shells in cases where `bashInteractive` was
explicitly disallowed using structured output checks, e.g. `nix develop
nixpkgs#systemd` after the "bash-less NixOS" changes.
This fix teaches `nix develop` to respect structured output checks, restoring
support for such derivations.
-12
View File
@@ -1,12 +0,0 @@
---
synopsis: "Add --raw flag to `nix-instantiate --eval` for unescaped output"
issues: []
prs: [gh#12119]
cls: [2886]
category: Improvements
credits: [not-my-profile, infinisil, raito]
---
The `nix-instantiate --eval` command now supports a `--raw` flag. When used,
the result must be coercible to a string (as with `${...}`) and is printed
verbatim, without quotes or escaping.
@@ -1,12 +0,0 @@
---
synopsis: Allow `nix store ls` to read nar listings from binary cache stores.
issues: []
cls: [3225]
category: Improvements
credits: [vlinkz]
---
The `nix store ls` command now supports reading `.ls` nar listings from binary cache stores.
If a listing is detected for the store path being queried, the nar is no longer downloaded.
These nar listings are available in binary cache stores where the `write-nar-listing` option is
enabled, such as cache.nixos.org.
@@ -1,10 +0,0 @@
---
synopsis: "nix-eval-jobs: retain NIX_PATH"
issues: []
cls: [3859]
category: Fixes
credits: [ma27,mic92]
---
`nix-eval-jobs` doesn't clear the `NIX_PATH` from the environment anymore. This matches the behavior
of [upstream version `2.30`](https://github.com/nix-community/nix-eval-jobs/releases/tag/v2.30.0).
-25
View File
@@ -1,25 +0,0 @@
---
synopsis: "show tree with references that lead to an output cycle"
issues: [fj#551]
category: Improvements
credits: [ma27]
---
When Lix determines a cyclic dependency between several outputs of a derivation,
it now displays which files in which outputs lead to an output cycle:
```
error: cycle detected in build of '/nix/store/gc5h2whz3rylpf34n99nswvqgkjkigmy-demo.drv' in the references of output 'bar' from output 'foo'.
Shown below are the files inside the outputs leading to the cycle:
/nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar
└───lib/libfoo: …stuffbefore /nix/store/h680k7k53rjl9p15g6h7kpym33250w0y-demo-baz andafter.…
→ /nix/store/h680k7k53rjl9p15g6h7kpym33250w0y-demo-baz
└───share/snenskek: …???? /nix/store/dm24c76p9y2mrvmwgpmi64rryw6x5qmm-demo-foo ....…
→ /nix/store/dm24c76p9y2mrvmwgpmi64rryw6x5qmm-demo-foo
└───bin/alarm: …textexttext/nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar abcabcabc.…
→ /nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar
```
Please note that showing the files and its contents while displaying the cycles only works
on Linux.
-20
View File
@@ -1,20 +0,0 @@
---
synopsis: "Fixed output derivations can be run using `pasta` network isolation"
cls: [3452]
issues: [fj#285]
category: "Breaking Changes"
credits: [horrors, puck]
---
Fixed output derivations traditionally run in the host network namespace.
On Linux this allows such derivations to communicate with other sandboxes
or the host using the abstract Unix domains socket namespace; this hasn't
been unproblematic in the past and has been used in two distinct exploits
to break out of the sandbox. For this reason fixed output derivations can
now run in a network namespace (provided by [`pasta`]), restricted to TCP
and UDP communication with the rest of the world. When enabled this could
be a breaking change and we classify it as such, even though we don't yet
enable or require such isolation by default. We may enforce this in later
releases of Lix once we have sufficient confidence that breakage is rare.
[`pasta`]: https://passt.top/
@@ -1,21 +0,0 @@
---
synopsis: Remove reliance on Bash for remote stores via SSH
issues: [fj#830, fj#805, fj#304]
cls: [3159]
category: "Fixes"
credits: [raito]
---
The pre-flight `echo started` handshake -- added years ago to catch race conditions -- has been removed.
After removal of connection sharing in Lix 2.93, it required a Bash-compatible shell and a standard `echo`, so it failed on:
* builders protected by `ForceCommand` wrappers (e.g. `nix-remote-build`),
* BusyBox / initrd images with no Bash,
* hosts using non-POSIX shells such as Nushell.
The race the probe once addressed was tied to SSH connection-sharing -- since connection-sharing code has already been removed, the probe is now pointless.
Real connection or protocol errors are now left to SSH/Nix to report directly.
This is technically a breaking change if you had scripts that relied on the literal "started" which needs to be updated to rely on other signals, e.g., exit codes.
-20
View File
@@ -1,20 +0,0 @@
---
synopsis: Remove support for daemon protocols before 2.18
issues: [fj#510]
cls: [3249]
significance: significant
category: "Breaking Changes"
credits: [horrors]
---
Support for daemon wire protocols belonging to Nix 2.17 or older have been
removed. This impacts clients connecting to the local daemon socket or any
remote builder configured using the `ssh-ng` protocol. Builders configured
with the `ssh` protocol are still accessible from clients such as Nix 2.3.
Additionally Lix will not be able to connect to an old daemon locally, and
remote build connections to old daemons is likewise limited to `ssh` urls.
We have decided to take this step because the old protocols are very badly
tested (if at all), maintenance overhead is high, and a number of problems
with their design makes it infeasible to remain backwards compatible while
we move Lix to a more modern RPC mechanism with better versioning support.
-14
View File
@@ -1,14 +0,0 @@
---
synopsis: "`nix eval --write-to` has been removed"
cls: [4045]
issues: [fj#974, fj#227]
category: "Breaking Changes"
credits: [horrors]
---
`nix eval --write-to` has been removed since it was underspecified, not widely
useful, and prone to security-sensitive misbehaviors. The feature was added in
Nix 2.4 purely for internal use in the build system. According to our research
it hasn't found any use outside of some distribution packaging scripts. Please
use structured outputs formats (such as JSON) instead as they have better type
fidelity, don't conflate attributes with paths, and are useful to other tools.
@@ -1,17 +0,0 @@
---
synopsis: Remove impure derivations and dynamic derivations
issues: [fj#815]
cls: [3210]
significance: significant
category: "Breaking Changes"
credits: [horrors]
---
The `impure-derivations` and `dynamic-derivations` experimental feature have
been removed.
New impure or dynamic derivations cannot be created from this point forward, and
any such pre-existing store derivations canot be read or built any more.
Derivation outputs created by building such a derivation are still valid
until garbage collected; existing store derivations can only be garbage
collected.
@@ -1,17 +0,0 @@
---
synopsis: Remove the `parse-toml-timestamps` experimental feature
category: "Breaking Changes"
credits: [emilazy]
---
The `parse-toml-timestamps` experimental feature has been removed.
This feature used inband signalling to mark timestamps, making it
impossible to unambiguously parse TOML documents. It also exposed
implementationdefined behaviour in the TOML specification that
changed in the toml11 parser library.
Any interface for parsing TOML timestamps suitable for future
stabilization would necessarily involve breaking changes, and there
is no evidence this experimental feature is being relied upon in the
wild, so it has been removed.
-20
View File
@@ -1,20 +0,0 @@
---
synopsis: "repl-overlays now work in the debugger for flakes"
issues: [fj#777]
cls: [3398]
category: Fixes
credits: [jade]
---
Due to a bug, it was previously not possible to use the debugger on flakes with repl-overlays, or with pure evaluation in general:
```
$ nix repl --pure-eval
Lix 2.94.0-dev-pre20250617-87d99da
Type :? for help.
Loading 'repl-overlays'...
error: access to absolute path '/Users/jade/.config/nix/repl.nix' is forbidden in pure eval mode (use '--impure' to override)
```
This is now fixed.
The contents of the repl-overlays file itself (i.e. most typically the top level lambda in it) will be evaluated in impure mode.
It may be necessary to use `builtins.seq` to force the impure operations to happen first if one wants to do impure operations inside a repl-overlays file in pure evaluation mode.
@@ -1,19 +0,0 @@
---
synopsis: "`disallowedRequisites` now reports chains of disallowed requisites"
issues: [fj#334,fj#626,gh#10877]
category: Improvements
credits: [ma27,roberth]
---
When a build fails because of [`disallowedRequisites`](@docroot@/language/advanced-attributes.md#adv-attr-disallowedRequisites), the error message now includes the chain of references that led to the failure. This makes it easier to see in which derivations the chain can be broken, to resolve the problem.
Example:
```
$ nix-build -A hello
error: output '/nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2' is not allowed to refer to the following paths:
/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-glibc-2.40-66
Shown below are chains that lead to the forbidden path(s).
/nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2
└───/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-glibc-2.40-66
```
@@ -1,17 +0,0 @@
---
synopsis: "libstore/binary-cache-store: don't cache narinfo on nix copy, remove negative entry"
issues: []
cls: [3789]
category: Fixes
credits: [ma27]
---
When using e.g. [Snix's nar-bridge](https://snix.dev/docs/components/overview/#nar-bridge) via
an `http`-store, Lix would create cache entries with a wrong URL to the NAR when uploading
a store-path.
This caused hard build failures for Hydra.
Lix doesn't create these entries on upload anymore. Instead, it only removes negative cache entries.
The cache entry for a narinfo is now created the first time, Lix queries the cache
for the previously uploaded store-path again.
-10
View File
@@ -1,10 +0,0 @@
---
synopsis: "Lix libraries can now be linked statically"
issues: [fj#789]
cls: [3775, 3778]
category: Fixes
credits: [alois31]
---
Previously the pkg-config files distributed with Lix were only suitable for dynamic linkage, causing "undefined reference to…" linker errors when trying to link statically.
Private dependency information has now been added to make static linkage work as expected without user intervention.
In addition, relevant static libraries are now prelinked to avoid strange failures due to missing static initializers.
-34
View File
@@ -1,34 +0,0 @@
---
synopsis: Symbols reuses once-allocated Value to reduce garbage collected allocations
issues: []
cls: [3308, 3300, 3314, 3310, 3312, 3313]
category: Improvements
credits: [raito, horrors, thubrecht, xokdvium, nan-git]
---
In the Lix evaluator, **symbols** represent immutable strings, like those used
for attribute names.
In evaluator design, such strings are typically [**interned**](https://en.wikipedia.org/wiki/String_interning), stored uniquely
to save memory, and Lix inherits this approach from the original C++ codebase.
However, some builtins, like `builtins.attrNames`, must return a `Value` type
that can represent any Nix value (strings, integers, lists, etc.).
Before this change, these builtins would create lists of `Value` objects by
allocating them through the garbage collector, copying the symbols string
content each time.
This allocation is unnecessary if the interned symbols themselves also hold a
`Value` representation allocated outside the garbage collector, since these
live for the full duration of evaluation.
As a result, this reduces the number of allocations, leading to:
* A significant drop in maximum [resident set memory](https://en.wikipedia.org/wiki/Resident_set_size) (RSS), with some large-scale
tests showing up to 11% (about 500 MiB) savings in large colmena deployments.
* A slight decrease in CPU usage during Nix evaluations.
This change is inspired by https://github.com/NixOS/nix/pull/13258 but the approach is different.
**Note** : [`xokdvium`](https://github.com/xokdvium) is the rightful author of https://gerrit.lix.systems/c/lix/+/3300 and the credit was missed on our end during the development process. We are deeply sorry for this mistake.
@@ -1,14 +0,0 @@
---
synopsis: Reject overflowing TOML integer literals
issues: []
cls: [3916]
category: "Breaking Changes"
credits: [emilazy]
---
The toml11 library used by Lix was updated. The new
version aligns with the [TOML v1.0.0 specifications
requirement](https://toml.io/en/v1.0.0#integer) to reject integer
literals that cannot be losslessly parsed. This means that code like
`builtins.fromTOML "v=0x8000000000000000"` will now produce an error
rather than silently saturating the integer result.
@@ -1,15 +0,0 @@
---
synopsis: uid-range depends on cgroups
issues: []
cls: [3230]
category: "Breaking Changes"
credits: [raito, horrors]
---
`uid-range` builds now depends on `cgroups`, an experimental feature.
`uid-range` builds already depended upon `auto-allocate-uids`, another experimental feature.
The rationale for doing so is that `uid-range` provides a sandbox with many
UIDs, this is useful for re-mapping them into a nested namespace, e.g. a
container.
@@ -1,10 +0,0 @@
---
synopsis: add description to zsh completions
issues: [fj#910]
cls: [3632]
category: "Fixes"
credits: [matthewbauer]
---
Emit descriptions when completing args in zsh completions. This uses the descriptions we already
provided in NIX\_GET\_COMPLETIONS.
+4
View File
@@ -39,6 +39,9 @@
- [Tuning Cores and Jobs](advanced-topics/cores-vs-jobs.md)
- [Verifying Build Reproducibility](advanced-topics/diff-hook.md)
- [Using the `post-build-hook`](advanced-topics/post-build-hook.md)
- [Pasta](advanced-topics/pasta.md)
- [Known Issues](known-issues/known-issues.md)
- [Limitations around non-isolated builds](known-issues/non-isolated-build-limits.md)
- [Command Reference](command-ref/command-ref.md)
- [Common Options](command-ref/opt-common.md)
- [Common Environment Variables](command-ref/env-common.md)
@@ -197,6 +200,7 @@
- [Release Notes](release-notes/release-notes.md)
- [Upcoming release](release-notes/rl-next.md)
<!-- RELENG-AUTO-INSERTION-MARKER (see releng/release_notes.py) -->
- [Lix 2.94 (2025-11-17)](release-notes/rl-2.94.md)
- [Lix 2.93 (2025-05-09)](release-notes/rl-2.93.md)
- [Lix 2.92 (2025-01-18)](release-notes/rl-2.92.md)
- [Lix 2.91 (2024-08-12)](release-notes/rl-2.91.md)
+19
View File
@@ -0,0 +1,19 @@
# [Pasta](https://passt.top/passt/about/): a network sandbox for fixed-output derivations
## Introduction
This section only applies to **Linux systems** as Pasta is a Linux-only measure.
Since [CVE-2025-46416](https://lix.systems/blog/2025-06-24-lix-cves/), the Lix project decided to adopt [Pasta](https://passt.top/passt/about/) for all fixed-output derivations, protecting against various attack vectors such as UNIX abstract domain sockets or more manipulation at the network layer from a malicious fixed-output derivation code.
Pasta acts as a translation layer between a layer-2 network interface and layer-4 sockets (TCP, UDP, ICMP/ICMPv6 echo) on the host. It requires no special privileges and can serve as a alternative to [SLiRP](https://en.wikipedia.org/wiki/Slirp) which was used [by Guix to mitigate the same problem](https://codeberg.org/guix/guix/commit/fb42611b8f27960304db5a1c0d33b8371dcde2a8).
## How to disable Pasta?
It's sufficient to pass `pasta-path = ""` in your `/etc/nix/nix.conf` or on the command line `--pasta-path ""` of a Lix invocation.
## Known issues surrounding Pasta
- Only the first DNS server in `/etc/resolv.conf` is considered: failover is not possible.
- [Reduced feature set compared to the Linux kernel](https://passt.top/passt/about/#features)
- [Performance overhead in multi-gigabits contexts and IMIX MTUs](https://passt.top/passt/about/#performance_1)
+1 -1
View File
@@ -58,7 +58,7 @@ $ nix-build flake:nixpkgs -A firefox
$ nix-build flake:github:NixOS/nixpkgs/release-23.11 -A firefox
```
Finally, for legacy reasons, if a path starts with `channel:`, the rest of the argument is interpreted as the name of a *nixpkgs* channel tarball to fetch from `https://nixos.org/channels/$CHANNEL_NAME/nixexprs.tar.xz`.
Finally, for legacy reasons, if a path starts with `channel:`, the rest of the argument is interpreted as the name of a *nixpkgs* channel tarball to fetch from `https://channels.nixos.org/$CHANNEL_NAME/nixexprs.tar.xz`.
This is a **hard coded URL** pattern and is *not* related to the subscribed channels managed by the [nix-channel](./nix-channel.md) command.
> **Note**: any of the special syntaxes may always be disambiguated by prefixing the path.
+3 -3
View File
@@ -11,7 +11,7 @@
Channels are a mechanism for referencing remote Nix expressions and conveniently retrieving their latest version.
The moving parts of channels are:
- The official channels listed at <https://nixos.org/channels>
- The official channels listed at <https://channels.nixos.org>
- The user-specific list of [subscribed channels](#subscribed-channels)
- The [downloaded channel contents](#channels)
- The [Nix expression search path](@docroot@/command-ref/conf-file.md#conf-nix-path), set with the [`-I` option](#opt-I) or the [`NIX_PATH` environment variable](#env-NIX_PATH)
@@ -77,9 +77,9 @@ This command has the following operations:
Subscribe to the Nixpkgs channel and run `hello` from the GNU Hello package:
```console
$ nix-channel --add https://nixos.org/channels/nixpkgs-unstable
$ nix-channel --add https://channels.nixos.org/nixpkgs-unstable
$ nix-channel --list
nixpkgs https://nixos.org/channels/nixpkgs
nixpkgs https://channels.nixos.org/nixpkgs
$ nix-channel --update
$ nix-shell -p hello --run hello
hello
+8
View File
@@ -170,6 +170,14 @@ Once you get a `Code-Review+2` vote on your change, it's rebased on `main` and C
If the change appears as "Rebase Required", you need to rebase it on `main` locally or via the Gerrit UI and wait for `Verified+1` before the Submit button is made active
The `Code-Review+2` from before will stick around through trivial rebases so no need to re-request review for a mere rebase.
## Interacting with the CI, Buildkite
We use Buildkite for our CI, usually you will not have to interact directly with it other than reviewing any errors it produces, which are linked from Gerrit.
However in certain cases a CI run will fail due to transient issues not related to your code and you will need to rerun it by hand.
You can log in to the CI via [SSO](https://buildkite.com/sso/lix-project). On your job you can then hit the "Retry failed" button to rerun it, normally you will not have a repeat of the transient issue.
If the build still fails on CI issues or all builds are failing this should be reported via [Zulip on #T-infra](https://zulip.lix.systems/#narrow/channel/7-T-infra) or [Matrix on #dev](https://matrix.to/#/%23dev%3Alix.systems?via=lix.systems).
## Building Lix with `nix`
To build a release version of Lix for the current operating system and CPU architecture:
+6
View File
@@ -62,6 +62,12 @@ For `installcheck` specifically, first run `just install` before running the tes
Finer-grained filtering within a test suite is also possible using the [--gtest_filter](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) command-line option to a test suite executable, or the `GTEST_FILTER` environment variable.
### Inspecting failures
The test suite emits logs in `build/meson-logs/`; the full textual failure logs are in `build/meson-logs/testlog.txt`.
If you want a much nicer experience of viewing the logs in a structured manner, use `xunit-viewer --results build/meson-logs/testlog.junit.xml --server` to view them in a web browser.
### Unit test support libraries
There are headers and code which are not just used to test the library in question, but also downstream libraries.
@@ -60,3 +60,10 @@ Then:
```console
$ docker run -ti lix
```
# Known issues
Lix in Docker is very sensitive to **functional** DNS resolution if you are running with [Pasta protections](../advanced-topics/pasta.md) which are enabled by default since Lix 2.93.0 on most distributions.
If you notice failure to download things, double check whether your **first** DNS entry in `/etc/resolv.conf` is functional.
Lix with [Pasta protections](../advanced-topics/pasta.md) does not support failing over the next entries.
@@ -0,0 +1 @@
This section lists known issues around Lix.
@@ -0,0 +1,21 @@
# Limitations of non-isolated builds
## What are non-isolated builds?
In Lix, only builds done on Linux with `sandbox = true` and a functioning
`pasta-path` are isolated from the rest of the system, all other builds are
considered non-isolated to some degree.
For example, running Lix with [Pasta](@docroot@/advanced-topics/pasta.md)
disabled makes the host network visible to fixed-output derivations, reducing
isolation somewhat.
## Clean termination of non-isolated builds
Non-isolated builds may not terminate cleanly in all cases due to limitations in Lix's process management.
This occurs when a build keeps the build log file descriptor open past the end of the actual build. A common cause of this are background tasks that aren't properly terminated before the main build process exits, for example: HTTP servers run as part of a test suite.
See [issue #1018](https://git.lix.systems/lix-project/lix/issues/1018) for an example.
The only solution is to manually terminate leftover processes in your derivation, including during failure scenarios.
@@ -41,7 +41,7 @@ install Lix. If this is not the case for some reason, you can add it
as follows:
```console
$ nix-channel --add https://nixos.org/channels/nixpkgs-unstable
$ nix-channel --add https://channels.nixos.org/nixpkgs-unstable
$ nix-channel --update
```
@@ -49,7 +49,7 @@ $ nix-channel --update
>
> On NixOS, youre automatically subscribed to a NixOS channel
> corresponding to your NixOS major release (e.g.
> <http://nixos.org/channels/nixos-21.11>). A NixOS channel is identical
> <https://channels.nixos.org/nixos-21.11>). A NixOS channel is identical
> to the Nixpkgs channel, except that it contains only Linux binaries
> and is updated only if a set of regression tests succeed.
+958
View File
@@ -0,0 +1,958 @@
# Lix 2.94 "Açaí na tigela" (2025-11-17)
# Lix 2.94.1 (2026-03-13)
# Lix 2.94.0 (2025-11-17)
## Breaking Changes
- Remove support for daemon protocols before 2.18 [fj#510](https://git.lix.systems/lix-project/lix/issues/510) [cl/3249](https://gerrit.lix.systems/c/lix/+/3249)
Support for daemon wire protocols belonging to Nix 2.17 or older have been
removed. This impacts clients connecting to the local daemon socket or any
remote builder configured using the `ssh-ng` protocol. Builders configured
with the `ssh` protocol are still accessible from clients such as Nix 2.3.
Additionally Lix will not be able to connect to an old daemon locally, and
remote build connections to old daemons is likewise limited to `ssh` urls.
We have decided to take this step because the old protocols are very badly
tested (if at all), maintenance overhead is high, and a number of problems
with their design makes it infeasible to remain backwards compatible while
we move Lix to a more modern RPC mechanism with better versioning support.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Remove impure derivations and dynamic derivations [fj#815](https://git.lix.systems/lix-project/lix/issues/815) [cl/3210](https://gerrit.lix.systems/c/lix/+/3210)
The `impure-derivations` and `dynamic-derivations` experimental feature have
been removed.
New impure or dynamic derivations cannot be created from this point forward, and
any such pre-existing store derivations canot be read or built any more.
Derivation outputs created by building such a derivation are still valid
until garbage collected; existing store derivations can only be garbage
collected.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- First argument to `--arg`/`--argstr` must be a valid Nix identifier [fj#496](https://git.lix.systems/lix-project/lix/issues/496)
The first argument to `--arg`/`--argstr` must be a valid Nix identifier, i.e.
`nix-build --arg config.allowUnfree true` is now rejected.
This is because that invocation is a false friend since it doesn't set
`{ config = { allowUnfree = true; }; }`, but `{ "config.allowUnfree" = true; }`.
The idea is to change the behavior to the latter in the long-term. For that,
non-identifiers started giving a warning since 2.92 and are now rejected to give people
who depend on that a chance to notice and potentially weigh in on the discussion.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- New cgroup delegation model [fj#537](https://git.lix.systems/lix-project/lix/issues/537) [fj#77](https://git.lix.systems/lix-project/lix/issues/77) [cl/3230](https://gerrit.lix.systems/c/lix/+/3230)
Builds using cgroups (i.e. `use-cgroups = true` and the experimental feature
`cgroups`) now always delegate a cgroup tree to the sandbox.
Compared to the original C++ Nix project, our delegation includes the
`subtree_control` file as well, which means that the sandbox can disable
certain controllers in its own cgroup tree.
This is a breaking change because this requires the Nix daemon to run with an
already delegated cgroup tree by the service manager.
## How to setup the cgroup tree with systemd?
systemd offers knobs to perform the required setup using:
```
[Service]
Delegate=yes
DelegateSubtree=supervisor
```
These directives are now included in our systemd packaging.
## What about using Nix as root without connecting to the daemon?
Builds run as `root` without connecting to the daemon relying on the cgroup
feature are now broken, i.e.
```console
# nix-build --use-cgroups --sandbox ... # will not work
```
Consider doing instead:
```console
# systemd-run --same-dir --wait -p Delegate=yes -p DelegateSubgroup=supervisor nix-build --use-cgroups ...
```
If you need to disable cgroups temporarily, remember that you can do
`NIX_CONF='include /etc/nix/nix.conf\nuse-cgroups = false' nix-build ...` or
`nix-build --no-use-cgroups ...`.
## What about other service managers than systemd?
systemd has a [documentation](https://systemd.io/CGROUP_DELEGATION/) on how to
handle cgroup delegation from service management perspective.
If your service manager adheres to systemd semantics, e.g. writing an extended
attribute `user.delegate=1` on the delegated cgroup tree directory and moving
the `nix-daemon` process inside a cgroup tree to respect the inner process
rule, then, the feature will work as well.
## Why is the cgroup feature still experimental?
While the cgroup feature unlocks many use cases, its behavior and integration (e.g. user experience), especially at scale on build farms or in multi-tenant environments, are not yet fully matured. Theres also potential for deeper systemd integration (e.g. using slices and scopes) that has not been fully explored.
To avoid locking in an unstable interface, were keeping the experimental flag until we have validated the feature across a broader range of scenarios, including but not limited to:
* Nix as root
* Hydra-style build farms
* Forgejo CI runners
* Shared remote builders
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [eldritch horrors](https://git.lix.systems/pennae), and [lheckemann](https://git.lix.systems/lheckemann) for this.
- Enable high compress ratio zstd compression by default for binary caches uploads [fj#945](https://git.lix.systems/lix-project/lix/issues/945) [cl/4503](https://gerrit.lix.systems/c/lix/+/4503)
The default compression method for binary cache uploads has been switched from
[`xz`](https://github.com/tukaani-project/xz) to
[`zstd`](https://github.com/facebook/zstd) to address performance and usability
issues related to modern hardware and high-speed connections.
## Why?
`xz` offers compression ratios but is single-threaded in our implementation and
very slow (~10-20 Mbps in our test), preventing full utilization of 100Mbps+
connections and significantly slowing decompression for end users.
Lix is a "compress once, decompress many" application: build farms can afford
to spend more time compressing to achieve a faster download transfer for the
end user. More importantly, it matters that all end users spend the least
amount of time decompressing.
## What about compression ratios?
`zstd` cannot achieve the same peaks as `xz`, nonetheless, `zstd` compression
level has been increased to level 12 by default to balance compression ratio
and performance.
## Synthetic test case data
* **xz** (default compression level) on a 4.4GB file: ~632MB (77s)
* **zstd** (level 12) on the same file: ~775MB (18s), 18% larger but 50% faster
* **zstd** (level 14): ~773MB (37s)
* **zstd** (level 16): ~735MB (66s)
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [Raito Bezarius](https://git.lix.systems/raito) for this.
- Repl debugger uses `--ignore-try` by default [lix#666](https://git.lix.systems/lix-project/lix/issues/666) [cl/3488](https://gerrit.lix.systems/c/lix/+/3488)
Previously, using the debugger meant that exceptions thrown in `builtins.tryEval` would trigger the debugger.
However, this caught nixpkgs initialization code, which is unhelpful in the majority of cases, so we changed the default.
To get the old behaviour, use `--no-ignore-try`.
```
$ nix repl --debugger --expr 'with import <nixpkgs> {}; pkgs.hello'
Lix 2.94.0-dev-pre20250625-9a59106
Type :? for help.
error: file 'nixpkgs-overlays' was not found in the Nix search path (add it using $NIX_PATH or -I)
This exception occurred in a 'tryEval' call. Use --ignore-try to skip these.
Added 13 variables.
nix-repl>
```
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Strings may now contain NUL bytes [cl/3968](https://gerrit.lix.systems/c/lix/+/3968)
Lix now allows strings to contain NUL bytes instead of silently truncating the
string before the first such byte. Notably NUL-bearing strings were allowed as
attribute names—even though the corresponding strings were not representable!—
leading to very surprising and incorrect behavior in corner cases, for example
```
nix-repl> builtins.fromJSON ''{"a": 1, "a\u0000b": 2}''
{
a = 1;
"ab" = 2;
}
nix-repl> builtins.attrNames (builtins.fromJSON ''{"a": 1, "a\u0000b": 2}'')
[
"a"
"a"
]
```
rather than the more correct but still with the terminal eating NUL on display
```
nix-repl> builtins.fromJSON ''{"a": 1, "a\u0000b": 2}''
{
a = 1;
"ab" = 2;
}
nix-repl> builtins.attrNames (builtins.fromJSON ''{"a": 1, "a\u0000b": 2}'')
[
"a"
"ab"
]
```
We consider this a breaking change since eval results *will* change if strings
with embedded NUL bytes were used, but we also consider the old behavior to be
not intentional (seeing how inconsistent it was) but merely fallout from a old
and misguided implementation decision to be worked around, not actually fixed.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Fixed output derivations can be run using `pasta` network isolation [fj#285](https://git.lix.systems/lix-project/lix/issues/285) [cl/3452](https://gerrit.lix.systems/c/lix/+/3452)
Fixed output derivations traditionally run in the host network namespace.
On Linux this allows such derivations to communicate with other sandboxes
or the host using the abstract Unix domains socket namespace; this hasn't
been unproblematic in the past and has been used in two distinct exploits
to break out of the sandbox. For this reason fixed output derivations can
now run in a network namespace (provided by [`pasta`]), restricted to TCP
and UDP communication with the rest of the world. When enabled this could
be a breaking change and we classify it as such, even though we don't yet
enable or require such isolation by default. We may enforce this in later
releases of Lix once we have sufficient confidence that breakage is rare.
[`pasta`]: https://passt.top/
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [puck](https://git.lix.systems/puck) for this.
- Function equality semantics are more consistent, but still bad [cl/4556](https://gerrit.lix.systems/c/lix/+/4556) [cl/4244](https://gerrit.lix.systems/c/lix/+/4244)
Lix has inherited a historic misfeature from CppNix in the form of pointer
equality checks built into the `==` operator. These checks were originally
meant to optimize comparison for large sets, but they have the unfortunate
side effect of producing unexpected results when sets containing functions
are compared. **Lix 2.93 and earlier** behave as shown in the repl session
```
Lix 2.93.3
Type :? for help.
nix-repl> f = x: x
Added f.
nix-repl> f == f
false
nix-repl> let s.f = f; in s.f == s.f
false
nix-repl> # however!
{ inherit f; } == { inherit f; }
true
nix-repl> [ f ] == [ f ]
true
nix-repl> # and, in another twist:
[ f ] == map f [ f ]
false
```
Nixpkgs relies on sets containing functions being comparable, so we cannot
simply deprecate this behavior. Due to changes to the object model used by
Lix ***all* comparisons above now evaluate to `true`**. This is considered
a breaking change because eval results may differ, but we also consider it
minor because the optimization is unsound (c.f. `let l = [NaN]; in l == l`
evaluates to `true` even though floating point `NaN` is incomparable). Lix
intends to remove this optimization altogether in the future, but until we
can do that we instead make it slightly less broken to allow other, *real*
optimizations. Function equality comparison remains **undefined behavior**
and should not be relied upon in Nixlang code that intends to be portable.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- `nix eval --write-to` has been removed [fj#974](https://git.lix.systems/lix-project/lix/issues/974) [fj#227](https://git.lix.systems/lix-project/lix/issues/227) [cl/4045](https://gerrit.lix.systems/c/lix/+/4045)
`nix eval --write-to` has been removed since it was underspecified, not widely
useful, and prone to security-sensitive misbehaviors. The feature was added in
Nix 2.4 purely for internal use in the build system. According to our research
it hasn't found any use outside of some distribution packaging scripts. Please
use structured outputs formats (such as JSON) instead as they have better type
fidelity, don't conflate attributes with paths, and are useful to other tools.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Remove the `parse-toml-timestamps` experimental feature
The `parse-toml-timestamps` experimental feature has been removed.
This feature used inband signalling to mark timestamps, making it
impossible to unambiguously parse TOML documents. It also exposed
implementationdefined behaviour in the TOML specification that
changed in the toml11 parser library.
Any interface for parsing TOML timestamps suitable for future
stabilization would necessarily involve breaking changes, and there
is no evidence this experimental feature is being relied upon in the
wild, so it has been removed.
Many thanks to [Emily](https://git.lix.systems/emilazy) for this.
- Reject overflowing TOML integer literals [cl/3916](https://gerrit.lix.systems/c/lix/+/3916)
The toml11 library used by Lix was updated. The new
version aligns with the [TOML v1.0.0 specifications
requirement](https://toml.io/en/v1.0.0#integer) to reject integer
literals that cannot be losslessly parsed. This means that code like
`builtins.fromTOML "v=0x8000000000000000"` will now produce an error
rather than silently saturating the integer result.
Many thanks to [Emily](https://git.lix.systems/emilazy) for this.
- uid-range depends on cgroups [cl/3230](https://gerrit.lix.systems/c/lix/+/3230)
`uid-range` builds now depends on `cgroups`, an experimental feature.
`uid-range` builds already depended upon `auto-allocate-uids`, another experimental feature.
The rationale for doing so is that `uid-range` provides a sandbox with many
UIDs, this is useful for re-mapping them into a nested namespace, e.g. a
container.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
## Features
- Add `inputs.self.submodules` flake attribute [fj#942](https://git.lix.systems/lix-project/lix/issues/942) [cl/3839](https://gerrit.lix.systems/c/lix/+/3839)
A port of <https://github.com/NixOS/nix/pull/12421> to Lix, which:
- adds a general `inputs.self` flake attribute that retroactively applies
configurations to a flake after it's been fetched, then triggers a refetch of
the flake with the new config.
- implements `inputs.self.submodules` that allows a flake to declare its need
for submodules, which are then fetched automatically with no need to pass
`?submodules=1` anywhere.
Many thanks to [Eelco Dolstra](https://github.com/edolstra) and [ورد](https://git.lix.systems/janw4ld) for this.
- Lix supports HTTP/3 behind `--http3` [fj#1033](https://git.lix.systems/lix-project/lix/issues/1033)
Lix now supports HTTP/3 for file transfers when the linked curl version
supports it.
By default, HTTP/3 is disabled notably due to performance issues reported in
mid-2024. [More details
here](https://daniel.haxx.se/blog/2024/06/10/http-3-in-curl-mid-2024/).
As of 2025-11-14, [NixOS official cache](https://cache.nixos.org) supports
HTTP/3 via Fastly. [More info
here](https://github.com/NixOS/infra/commit/157fa70e46afbd6338a32407be461fce05c57bf8).
To enable HTTP/3:
* Use `--http3` for individual transfers.
* Add `http3 = true` in your Nix configuration for permanent activation.
To disable it, use `--no-http3`.
**Note**:
* `--no-http2 --http3` will still enable both HTTP/2 and HTTP/3.
* `--http2 --http3` will prioritize HTTP/3 and fall back to HTTP/2 (and then
HTTP/1.1).
These are current CLI limitations. In the future, we plan to replace `--httpX`
options with `--max-http-version [1,2,3]` for easier version selection in Lix
transfers.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Add hyperlinks in attr set printing [cl/3790](https://gerrit.lix.systems/c/lix/+/3790)
The attribute set printer, such as is seen in `nix repl` or in type errors, now prints hyperlinks on each attribute name to its definition site if it is known.
Example: all of the attributes shown here are hyperlinks to the exact definition site of the attribute in question:
```
$ nix eval -f '<nixpkgs>' lib.licenses.mit
{ deprecated = false; free = true; fullName = "MIT License"; redistributable = true; shortName = "mit"; spdxId = "MIT"; url = "https://spdx.org/licenses/MIT.html"; }
```
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Experimental integer coercion in interpolated strings [cl/3198](https://gerrit.lix.systems/c/lix/+/3198)
Ever tried interpolating a port number in Lix and ended up with something like this?
```nix
"http://${config.network.host}:${builtins.toString config.network.port}/"
```
You're not alone. Thousands of Lix users suffer every day from excessive `builtins.toString` syndrome. Its 2025, and we still have to cast integers to use them in strings.
To address this, Lix introduces the **`coerce-integers`** experimental feature. When enabled, interpolated integers within `"${...}"` are automatically coerced to strings. This allows writing:
```nix
"http://${config.network.host}:${config.network.port}/"
```
without additional conversion.
To enable the feature, you need to add `coerce-integers` to your set of experimental features.
### Stabilization criteria
The `coerce-integers` feature is experimental and limited strictly to string interpolation (`"${...}"`). Before stabilization, the following must hold:
1. **Interpolation-only**
Coercion must not occur outside interpolation. Expressions like `"" + 42` must continue to fail.
2. **Expectation that no explicit cast are being observed**
Cases observing explicit coercion (e.g., via `tryEval` gadget or similar) are expected not to be load-bearing in actual production code.
### Timeline for stabilization
If the feature proves safe and is widely adopted across typical usage (e.g., actual configurations in the wild turning on the flag, non-trivial out-of-tree projects using it), the experimental flag will be removed **after six months of active use or two Lix releases**, whichever is longer.
This avoids locking the feature in experimental status indefinitely, as happened with Flakes, while allowing time for validation and ecosystem integration.
### What about coercing floats or more?
Coercion beyond integers -- such as for floats or other types -- is **not planned**, even under an experimental flag. Questions like "what is the canonical string representation of a float?" involve subtle and context-dependent trade-offs. Without a robust and principled mechanism to define and audit such behavior, introducing broader coercion risks setting unintended and hard-to-reverse precedents. The scope of `coerce-integers` is intentionally narrow and will remain so.
In terms of outlook, a proposal like https://git.lix.systems/lix-project/lix/issues/835 could pave the way for a better solution.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [delroth](https://github.com/delroth), [eldritch horrors](https://git.lix.systems/pennae), and [winter](https://git.lix.systems/winter) for this.
- nix-eval-jobs: support `--no-instantiate` flag [fj#987](https://git.lix.systems/lix-project/lix/issues/987)
`nix-eval-jobs` now supports a flag called `--no-instantiate`. With this enabled,
no write operations on the eval store are performed. That means, only evaluation is
performed, but derivations (and their gcroots) aren't created.
Many thanks to [mic92](https://github.com/mic92) and [ma27](https://git.lix.systems/ma27) for this.
## Improvements
- Assess current profile generations pointers in `nix doctor` [cl/3108](https://gerrit.lix.systems/c/lix/+/3108)
Added a new check to `nix doctor` that verifies whether the current generation of
a Nix profile can be resolved. This helps users diagnose issues with broken or
misconfigured profile symlinks.
This helps determining if you have broken symlinks or misconfigured packaging.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Improved susbtituter query speed
The code used to query substituters for derivations has been rewritten slightly
to take advantage of our asynchronous runtime. Such queries run for every build
that could download from substituters and processes every derivation that isn't
yet present on the local system. Previously Lix would use `http-connections` to
limit query concurrency, even for modern caches that support HTTP/2 and have no
limit on how many queries can be run concurrently on one single connection. Lix
no longer does this, resulting in approximately 60% reduction in query time for
medium-sized closures (e.g. NixOS system closures) during testing, although the
exact number depends greatly on local network latency and generally improves as
latency increases. Unlike previously setting `http-connections` to `1` or other
low values no longer brings a massive penalty in query performance if the cache
in use by the querying system supports HTTP/2 (as e.g. `cache.nixos.org` does).
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Hitting Control-C twice always terminates Lix [cl/3574](https://gerrit.lix.systems/c/lix/+/3574)
Hitting Control-C or sending `SIGINT` to Lix now prints an informational message
if it is still running after on second, the second Control-C/`SIGINT` terminates
Lix immediately without waiting for any shutdown code to finish running. Lix did
not treat the second such event differently from first in the past; this made it
impossible to easily terminate running Lix processes that got stuck in e.g. very
expensive Nixlang code that never interacted with the store. We now terminate as
soon as the user hits Control-C again without waiting any more, to much the same
effect as putting Lix into the background and killing it immediately afterwards.
This means you can now more conveniently break out of stuck Nixlang evaluations:
```
nix-instantiate --eval --expr 'let f = n: if n == 0 then 0 else f (n - 1) + f (n - 1); in f 32'
^CStill shutting down. Press ^C again to abort all operations immediately.
^C
❌130
```
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- `--keep-failed` chowns the build directory to the user that request the build
Running a build with `--keep-failed` now chowns the temporary directory from the
builder user and group to the user that request the build if the build came from
a local user connected to the daemon. This makes inspecting failed derivations a
lot easier. On Linux the build directory made visible to the user will not be in
the same path as it was in the sandbox and continuing builds will usually break.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Better debuggability on fixed-output hash mismatches
Fixed-output derivation hash mismatch error messages will now include the path that was
produced unexpectedly, and this path will be registered as valid even if `--check`
(`nix-store`, `nix-build`) or `--rebuild` (`nix build`) was passed. This makes comparing
the expected path with the obtained path easier, and is useful for debugging when
upstreams modify previously-published releases or when changes in fixed-output
derivations' dependencies affect their output unexpectedly.
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Add --raw flag to `nix-instantiate --eval` for unescaped output [gh#12119](https://github.com/NixOS/nix/pull/12119) [cl/2886](https://gerrit.lix.systems/c/lix/+/2886)
The `nix-instantiate --eval` command now supports a `--raw` flag. When used,
the result must be coercible to a string (as with `${...}`) and is printed
verbatim, without quotes or escaping.
Many thanks to [Martin Fischer](https://github.com/not-my-profile), [infinisil](https://github.com/infinisil), and [Raito Bezarius](https://git.lix.systems/raito) for this.
- Allow `nix store ls` to read nar listings from binary cache stores. [cl/3225](https://gerrit.lix.systems/c/lix/+/3225)
The `nix store ls` command now supports reading `.ls` nar listings from binary cache stores.
If a listing is detected for the store path being queried, the nar is no longer downloaded.
These nar listings are available in binary cache stores where the `write-nar-listing` option is
enabled, such as cache.nixos.org.
Many thanks to [Victor Fuentes](https://git.lix.systems/vlinkz) for this.
- show tree with references that lead to an output cycle [fj#551](https://git.lix.systems/lix-project/lix/issues/551)
When Lix determines a cyclic dependency between several outputs of a derivation,
it now displays which files in which outputs lead to an output cycle:
```
error: cycle detected in build of '/nix/store/gc5h2whz3rylpf34n99nswvqgkjkigmy-demo.drv' in the references of output 'bar' from output 'foo'.
Shown below are the files inside the outputs leading to the cycle:
/nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar
└───lib/libfoo: …stuffbefore /nix/store/h680k7k53rjl9p15g6h7kpym33250w0y-demo-baz andafter.…
→ /nix/store/h680k7k53rjl9p15g6h7kpym33250w0y-demo-baz
└───share/snenskek: …???? /nix/store/dm24c76p9y2mrvmwgpmi64rryw6x5qmm-demo-foo ....…
→ /nix/store/dm24c76p9y2mrvmwgpmi64rryw6x5qmm-demo-foo
└───bin/alarm: …textexttext/nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar abcabcabc.…
→ /nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar
```
Please note that showing the files and its contents while displaying the cycles only works
on Linux.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- Lix now enables parallel marking in boehm-gc [fj#983](https://git.lix.systems/lix-project/lix/issues/983) [cl/3880](https://gerrit.lix.systems/c/lix/+/3880)
This brings a fairly modest performance improvement (~38% for `nixpkgs search hello`) to evaluation, especially in scenarios that necessitate larger heap sizes.
Many thanks to [Eelco Dolstra](https://github.com/edolstra) and [Seth Flynn](https://git.lix.systems/getchoo) for this.
- `disallowedRequisites` now reports chains of disallowed requisites [fj#334](https://git.lix.systems/lix-project/lix/issues/334) [fj#626](https://git.lix.systems/lix-project/lix/issues/626) [gh#10877](https://github.com/NixOS/nix/issues/10877)
When a build fails because of [`disallowedRequisites`](@docroot@/language/advanced-attributes.md#adv-attr-disallowedRequisites), the error message now includes the chain of references that led to the failure. This makes it easier to see in which derivations the chain can be broken, to resolve the problem.
Example:
```
$ nix-build -A hello
error: output '/nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2' is not allowed to refer to the following paths:
/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-glibc-2.40-66
Shown below are chains that lead to the forbidden path(s).
/nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2
└───/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-glibc-2.40-66
```
Many thanks to [ma27](https://git.lix.systems/ma27) and [Robert Hensing](https://github.com/roberth) for this.
- Stack traces now summarize involved derivations at the bottom [cl/4493](https://gerrit.lix.systems/c/lix/+/4493)
When evaluation errors and a stack trace is printed,
For example, if I add Nheko to a NixOS `environment.systemPackages` without adding `olm-3.2.16` `nixpkgs.config.permittedInsecurePackages`, then without `--show-trace`, I previously got this:
```
error:
… while calling the 'head' builtin
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/attrsets.nix:1701:13:
1700| if length values == 1 || pred here (elemAt values 1) (head values) then
1701| head values
| ^
1702| else
… while evaluating the attribute 'value'
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/modules.nix:1118:7:
1117| // {
1118| value = addErrorContext "while evaluating the option `${showOption loc}':" value;
| ^
1119| inherit (res.defsFinal') highestPrio;
(stack trace truncated; use '--show-trace' to show the full trace)
error: Package olm-3.2.16 in /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/pkgs/by-name/ol/olm/package.nix:37 is marked as insecure, refusing to evaluate.
< -snip the whole explanation about olm's CVEs- >
```
This doesn't tell me anything about where `olm-3.2.16` came from.
With `--show-trace`, there's 1155 lines to sift through, but does contain lines like "while evaluating derivation 'nheko-0.12.1'".
With this change, those lines are summarized and collected at the bottom, regardless of `--show-trace`:
```
error:
… while calling the 'head' builtin
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/attrsets.nix:1701:13:
1700| if length values == 1 || pred here (elemAt values 1) (head values) then
1701| head values
| ^
1702| else
… while evaluating the attribute 'value'
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/modules.nix:1118:7:
1117| // {
1118| value = addErrorContext "while evaluating the option `${showOption loc}':" value;
| ^
1119| inherit (res.defsFinal') highestPrio;
(stack trace truncated; use '--show-trace' to show the full trace)
error: Package olm-3.2.16 in /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/pkgs/by-name/ol/olm/package.nix:37 is marked as insecure, refusing to evaluate.
< -snip the whole explanation about olm's CVEs- >
note: trace involved the following derivations:
derivation 'etc'
derivation 'dbus-1'
derivation 'system-path'
derivation 'nheko-0.12.1'
derivation 'mtxclient-0.10.1'
```
Now we finally know that olm was evaluated because of Nheko, without sifting through *thousands* of lines of error message.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- Symbols reuses once-allocated Value to reduce garbage collected allocations [cl/3308](https://gerrit.lix.systems/c/lix/+/3308) [cl/3300](https://gerrit.lix.systems/c/lix/+/3300) [cl/3314](https://gerrit.lix.systems/c/lix/+/3314) [cl/3310](https://gerrit.lix.systems/c/lix/+/3310) [cl/3312](https://gerrit.lix.systems/c/lix/+/3312) [cl/3313](https://gerrit.lix.systems/c/lix/+/3313)
In the Lix evaluator, **symbols** represent immutable strings, like those used
for attribute names.
In evaluator design, such strings are typically [**interned**](https://en.wikipedia.org/wiki/String_interning), stored uniquely
to save memory, and Lix inherits this approach from the original C++ codebase.
However, some builtins, like `builtins.attrNames`, must return a `Value` type
that can represent any Nix value (strings, integers, lists, etc.).
Before this change, these builtins would create lists of `Value` objects by
allocating them through the garbage collector, copying the symbols string
content each time.
This allocation is unnecessary if the interned symbols themselves also hold a
`Value` representation allocated outside the garbage collector, since these
live for the full duration of evaluation.
As a result, this reduces the number of allocations, leading to:
* A significant drop in maximum [resident set memory](https://en.wikipedia.org/wiki/Resident_set_size) (RSS), with some large-scale
tests showing up to 11% (about 500 MiB) savings in large colmena deployments.
* A slight decrease in CPU usage during Nix evaluations.
This change is inspired by https://github.com/NixOS/nix/pull/13258 but the approach is different.
**Note** : [`xokdvium`](https://github.com/xokdvium) is the rightful author of https://gerrit.lix.systems/c/lix/+/3300 and the credit was missed on our end during the development process. We are deeply sorry for this mistake.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [eldritch horrors](https://git.lix.systems/pennae), [Tom Hubrecht](https://git.lix.systems/tom-hubrecht), [xokdvium](https://github.com/xokdvium), and [NaN-git](https://github.com/NaN-git) for this.
## Fixes
- `build-dir` no longer defaults to `temp-dir` [cl/3453](https://gerrit.lix.systems/c/lix/+/3453)
The directory in which temporary build directories are created no longer defaults
to the value of the `temp-dir` setting to avoid builders making their directories
world-accessible. This behavior has been used to escape the build sandbox and can
cause build impurities even when not used maliciously. We now default to `builds`
in `NIX_STATE_DIR` (which is `/nix/var/nix/b` in the default configuration).
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Global certificate authorities are copied inside the builder's environment [gh#12698](https://github.com/NixOS/nix/issues/12698) [fj#885](https://git.lix.systems/lix-project/lix/issues/885) [cl/3765](https://gerrit.lix.systems/c/lix/+/3765)
Previously, CA certificates were only installed at
`/etc/ssl/certs/ca-certificates.crt` for sandboxed builds on Linux.
This setup was insufficient in light of recent changes in `nixpkgs`, which now
enforce HTTPS usage for `fetchurl`, even for fixed-output derivations, to
mitigate confidentiality risks such as `netrc` or credentials leakage.
`nixpkgs` still make use of a special package called `cacerts` which contains a
copy of the CA certificates maintained by Nixpkgs and added as a reference for
TLS-enabled fetchers.
As a result, having a consistent and trusted certificate authority in all
builder environments is becoming more essential.
On `nix-darwin`, the `NIX_SSL_CERT_FILE` environment variable is always
explicitly defined, but it is ignored by the sandbox setup.
Simultaneously, Nix evaluates and propagates impure environment variables via
`lib.proxyImpureEnvVars`, meaning that if `NIX_SSL_CERT_FILE` is set (which
influences the default value for `ssl-cert-file`), it will be forwarded
unchanged into the builder environment.
However, on Linux, Nix also *copies* the CA file into the sandbox, creating a
discrepancy between the value of `NIX_SSL_CERT_FILE` and the actual trusted
certificate path used during the build.
This divergence caused confusion and was partially addressed by attempts to
whitelist the CA path in the Darwin sandbox (see cl/2906), but that approach
involved a non-trivial path canonicalization step and is not as general as this one.
To address this properly, we now emit a warning and override
`NIX_SSL_CERT_FILE` inside the builder, explicitly pointing it to the CA file
copied into the sandbox.
This eliminates ambiguity between `NIX_SSL_CERT_FILE`
and `ssl-cert-file`, ensuring consistent trust anchors across platforms.
This warning might become a hard error as we figure out what to do regarding
`lib.proxyImpureEnvVars` in nixpkgs.
The behavior has been verified across sandboxed and unsandboxed builds on both
Linux and Darwin.
As a consequence of this change, approximately 500KB of CA certificate data is
now unconditionally copied into the build directory for fixed-output
derivations.
While this ensures consistent trust verification without having to restart the
daemon after system upgrades, it may introduce a slight overhead in build
performance. At present, no optimizations have been implemented to avoid this
copy, but if this overhead proves noticeable in your workflows, please open an
issue so we can evaluate and possibly implement different strategies to render
trust anchors visible.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [Emily](https://git.lix.systems/emilazy) for this.
- libstore: exponential backoff for downloads [lix#932](https://git.lix.systems/lix-project/lix/issues/932) [cl/3856](https://gerrit.lix.systems/c/lix/+/3856)
The connection timeout when downloading from e.g. a binary cache is exponentially
increased per failure. The option `connect-timeout` is now an alias to `max-connect-timeout`
which is the maximum value for a timeout. The start value is controlled
by `initial-connect-timeout` which is `5` by default.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- Fix develop shells for derivations with escape codes [fj#991](https://git.lix.systems/lix-project/lix/issues/991) [cl/4154](https://gerrit.lix.systems/c/lix/+/4154) [cl/4155](https://gerrit.lix.systems/c/lix/+/4155)
ASCII control characters (including `\e`, used for ANSI escape codes) in derivation variables are now correctly escaped for `nix develop` and `nix print-dev-env`, instead of erroring.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- nix-store --delete: always remove obsolete hardlinks [cl/3188](https://gerrit.lix.systems/c/lix/+/3188)
Deleting specific paths using `nix-store --delete` or `nix store
delete` previously did not delete hard links created by `nix-store
--optimise` even if they became obsolete, unless _all_ of the given
paths were deleted successfully. Now, hard links are always cleaned
up, even if some of the given paths could not be deleted.
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Report GC statistics correctly [cl/3188](https://gerrit.lix.systems/c/lix/+/3188)
Deleting specific paths using `nix-store --delete` or `nix store delete` previously did
not report statistics correctly when some of the paths could not be deleted, even if
others were deleted:
```
$ nix store delete /nix/store/9bwryidal9q3g91cjm6xschfn4ikd82q-hello-2.12.1 --delete-closure -v
finding garbage collector roots...
deleting '/nix/store/9bwryidal9q3g91cjm6xschfn4ikd82q-hello-2.12.1'
0 store paths deleted, 0.00 MiB freed
error: Cannot delete some of the given paths because they are still alive. Paths not deleted:
k9bxzr1l92r5y6mihrkbpbr3fmc8qszx-libidn2-2.3.8
mbx9ii53lzjlrsnlrfmzpwm33ynljwdn-libunistring-1.3
rf8hcy6bldxdqc0g6q1dcka1vh47x69s-xgcc-14.2.1.20250322-libgcc
vbrdc5wgzn0w1zdp10xd2favkjn5fk7y-glibc-2.40-66
To find out why, use nix-store --query --roots and nix-store --query --referrers.
```
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Fallback to safe temp dir when build-dir is unwritable [fj#876](https://git.lix.systems/lix-project/lix/issues/876) [cl/3501](https://gerrit.lix.systems/c/lix/+/3501)
Non-daemon builds started failing with a permission error after introducing the `build-dir` option:
```
$ nix build --store ~/scratch nixpkgs#hello --rebuild
error: creating directory '/nix/var/nix/builds/nix-build-hello-2.12.2.drv-0': Permission denied
```
This happens because:
1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
2. The user lacks permissions for that path.
We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
**Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Parse overflowing JSON number literals as floatingpoint [cl/3919](https://gerrit.lix.systems/c/lix/+/3919)
Previously, `builtins.fromJSON "-9223372036854775809"` would
return a floatingpoint number, while `builtins.fromJSON
"9223372036854775808"` would cause an evaluation error. This was
introduced with the banning of integer overflow in Lix 2.91; previously
the latter would result in C++ undefined behaviour. These cases are
now treated consistently with JSONs model of a single numeric type,
and JSON number literals that do not fit in a Nixlanguage integer
will be parsed as floatingpoint numbers.
Many thanks to [Emily](https://git.lix.systems/emilazy) for this.
- Fix handling of OSC codes in terminal output [fj#160](https://git.lix.systems/lix-project/lix/issues/160) [cl/3143](https://gerrit.lix.systems/c/lix/+/3143)
OSC codes in terminal output are now handled correctly, where OSC 8 (hyperlink) is preserved any
time color codes are allowed and all other OSC codes are stripped out. This applies not only to
output from build commands but also to rendered documentation in the REPL.
Many thanks to [lilyball](https://git.lix.systems/lilyball) for this.
- Fix nix develop for derivations that rejects dependencies with structured attrs [fj#997](https://git.lix.systems/lix-project/lix/issues/997) [cl/4182](https://gerrit.lix.systems/c/lix/+/4182)
For the sake of concision, we refer to `disallowedReferences` in what follows,
but all output checks were equally fixed:
`{dis,}allowed{References,Requisites}`.
Derivations can define *output checks* to reject unwanted dependencies, such as
interpreters like `bash` or compilers like `gcc`. This can be done in two ways:
* **Legacy style**: `disallowedReferences = [ ... ]` in the environment.
* **Structured attrs**: `outputChecks.<output>.disallowedReferences = [ ... ]`,
typically used in `__json`.
Only the structured form supports derivations with multiple outputs.
`nix develop` internally rewrites derivations to create development shells. It
relied on the legacy `disallowedReferences`, and failed to honor the structured
variant. This led to broken shells in cases where `bashInteractive` was
explicitly disallowed using structured output checks, e.g. `nix develop
nixpkgs#systemd` after the "bash-less NixOS" changes.
This fix teaches `nix develop` to respect structured output checks, restoring
support for such derivations.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- nix-eval-jobs: retain NIX_PATH [cl/3859](https://gerrit.lix.systems/c/lix/+/3859)
`nix-eval-jobs` doesn't clear the `NIX_PATH` from the environment anymore. This matches the behavior
of [upstream version `2.30`](https://github.com/nix-community/nix-eval-jobs/releases/tag/v2.30.0).
Many thanks to [ma27](https://git.lix.systems/ma27) and [mic92](https://github.com/mic92) for this.
- Remove reliance on Bash for remote stores via SSH [fj#830](https://git.lix.systems/lix-project/lix/issues/830) [fj#805](https://git.lix.systems/lix-project/lix/issues/805) [fj#304](https://git.lix.systems/lix-project/lix/issues/304) [cl/3159](https://gerrit.lix.systems/c/lix/+/3159)
The pre-flight `echo started` handshake -- added years ago to catch race conditions -- has been removed.
After removal of connection sharing in Lix 2.93, it required a Bash-compatible shell and a standard `echo`, so it failed on:
* builders protected by `ForceCommand` wrappers (e.g. `nix-remote-build`),
* BusyBox / initrd images with no Bash,
* hosts using non-POSIX shells such as Nushell.
The race the probe once addressed was tied to SSH connection-sharing -- since connection-sharing code has already been removed, the probe is now pointless.
Real connection or protocol errors are now left to SSH/Nix to report directly.
This is technically a breaking change if you had scripts that relied on the literal "started" which needs to be updated to rely on other signals, e.g., exit codes.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- repl-overlays now work in the debugger for flakes [fj#777](https://git.lix.systems/lix-project/lix/issues/777) [cl/3398](https://gerrit.lix.systems/c/lix/+/3398)
Due to a bug, it was previously not possible to use the debugger on flakes with repl-overlays, or with pure evaluation in general:
```
$ nix repl --pure-eval
Lix 2.94.0-dev-pre20250617-87d99da
Type :? for help.
Loading 'repl-overlays'...
error: access to absolute path '/Users/jade/.config/nix/repl.nix' is forbidden in pure eval mode (use '--impure' to override)
```
This is now fixed.
The contents of the repl-overlays file itself (i.e. most typically the top level lambda in it) will be evaluated in impure mode.
It may be necessary to use `builtins.seq` to force the impure operations to happen first if one wants to do impure operations inside a repl-overlays file in pure evaluation mode.
Many thanks to [jade](https://git.lix.systems/jade) for this.
- `nix-shell` default shell directory is not `/tmp` anymore for `$NIX_BUILD_TOP` [fj#940](https://git.lix.systems/lix-project/lix/issues/940)
Previously, Lix `nix-shell`s could exit non-zero status when `stdenv`'s `dumpVars` phase failed to write to `$NIX_BUILD_TOP/env-vars`, despite `dumpVars` being intended as a debugging aid.
This happens when `TMPDIR` is not set and defaults therefore to `/tmp`, resulting in a `/tmp/env-vars` global file that every `nix-shell` wants to write.
We fix this issue by reusing a pre-created, unique, and writable location, as the build top directory, avoiding shell exiting from write failures silently.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- libstore/binary-cache-store: don't cache narinfo on nix copy, remove negative entry [cl/3789](https://gerrit.lix.systems/c/lix/+/3789)
When using e.g. [Snix's nar-bridge](https://snix.dev/docs/components/overview/#nar-bridge) via
an `http`-store, Lix would create cache entries with a wrong URL to the NAR when uploading
a store-path.
This caused hard build failures for Hydra.
Lix doesn't create these entries on upload anymore. Instead, it only removes negative cache entries.
The cache entry for a narinfo is now created the first time, Lix queries the cache
for the previously uploaded store-path again.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- Lix libraries can now be linked statically [fj#789](https://git.lix.systems/lix-project/lix/issues/789) [cl/3775](https://gerrit.lix.systems/c/lix/+/3775) [cl/3778](https://gerrit.lix.systems/c/lix/+/3778)
Previously the pkg-config files distributed with Lix were only suitable for dynamic linkage, causing "undefined reference to…" linker errors when trying to link statically.
Private dependency information has now been added to make static linkage work as expected without user intervention.
In addition, relevant static libraries are now prelinked to avoid strange failures due to missing static initializers.
Many thanks to [alois31](https://git.lix.systems/alois31) for this.
- add description to zsh completions [fj#910](https://git.lix.systems/lix-project/lix/issues/910) [cl/3632](https://gerrit.lix.systems/c/lix/+/3632)
Emit descriptions when completing args in zsh completions. This uses the descriptions we already
provided in NIX\_GET\_COMPLETIONS.
Many thanks to [matthewbauer](https://github.com/matthewbauer) for this.
## Miscellany
- Deprecation of CA derivations, dynamic derivations, and impure derivations [fj#815](https://git.lix.systems/lix-project/lix/issues/815)
Content-addressed derivations are now deprecated and slated for removal in Lix 2.94.
We're doing this because the CA derivation system has been a known cause of problems
and inconsistencies, is unmaintained, habitually makes improving the store code very
difficult (or blocks such improvements outright), and is beset by a number of design
flaws that in our opinion cannot be fixed without a full reimplementation from zero.
Dynamic derivations and impure derivations are built on the CA derivation framework,
and owing to this they too are deprecated and slated for removal in another release.
+1 -1
View File
@@ -8,7 +8,7 @@
tag ? "latest",
bundleNixpkgs ? true,
channelName ? "nixpkgs",
channelURL ? "https://nixos.org/channels/nixpkgs-unstable",
channelURL ? "https://channels.nixos.org/nixpkgs-unstable",
extraPkgs ? [ ],
maxLayers ? 100,
nixConf ? { },
Generated
+3 -3
View File
@@ -108,11 +108,11 @@
},
"nixpkgs_2": {
"locked": {
"lastModified": 1758391731,
"narHash": "sha256-UuwQoPWv13DVKMveeev+F0OC/N95AOmAz6SzCuGhxjQ=",
"lastModified": 1757198069,
"narHash": "sha256-m3VUcOD4rTs8J7S+3dOjWMrAjw6RcITC3XYQ98zhEFs=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "3f00d36f15e16e0471d9ca1e8f88958941fa970a",
"rev": "0747026fc57ecb9c28901c7f7a2b5dc40e8af43c",
"type": "github"
},
"original": {
+19 -2
View File
@@ -88,7 +88,7 @@
else
"pre${
builtins.substring 0 8 (self.lastModifiedDate or self.lastModified or "19700101")
}_${self.shortRev or "dirty"}";
}-dev_${self.shortRev or "dirty"}";
linux32BitSystems = [ "i686-linux" ];
linux64BitSystems = [
@@ -100,6 +100,7 @@
"x86_64-darwin"
"aarch64-darwin"
];
nonDarwinSystems = linuxSystems;
systems = linuxSystems ++ darwinSystems;
# If you add something here, please update the list in doc/manual/src/contributing/hacking.md.
@@ -259,6 +260,19 @@
nativeBuildInputs = prevAttrs.nativeBuildInputs ++ [ final.buildPackages.bmake ];
postInstall = lib.replaceStrings [ "lowdown.so.1" ] [ "lowdown.so.2" ] prevAttrs.postInstall;
});
capnproto = prev.capnproto.overrideAttrs (old: {
patches =
old.patches or [ ]
++ [
# backport of https://github.com/capnproto/capnproto/pull/1810
./misc/capnproto-promise-nodiscard.patch
]
++ lib.optionals (lib.versionOlder old.version "1.2.0") [
# backport of https://github.com/capnproto/capnproto/pull/2296
./misc/capnproto-monotonic-clocks-are-a-lie.patch
];
});
};
in
{
@@ -371,7 +385,10 @@
;
}
// {
nix-eval-jobs = forAllSystems (system: self.packages.${system}.nix-eval-jobs.tests.nix-eval-jobs);
# the n-e-j test suite is unusably slow in darwin ci. disbled until anywho fixes this.
nix-eval-jobs = (lib.genAttrs nonDarwinSystems) (
system: self.packages.${system}.nix-eval-jobs.tests.nix-eval-jobs
);
# This is x86_64-linux only, just because we have significantly
# cheaper x86_64-linux compute in CI.
+195 -110
View File
@@ -1,15 +1,18 @@
#include "lix/libstore/path.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/error.hh"
#include "lix/libutil/file-descriptor.hh"
#include "lix/libutil/logging-rpc.hh"
#include "lix/libutil/logging.hh"
#include "lix/libutil/rpc.hh"
#include "lix/libutil/types-rpc.hh"
#include "lix/libutil/types-rpc.hh" // IWYU pragma: keep
#include "lix/libutil/types.hh"
#include <algorithm>
#include <capnp/rpc-twoparty.h>
#include <chrono>
#include <cstring>
#include <future>
#include <exception>
#include <kj/async.h>
#include <kj/time.h>
#include <set>
#include <memory>
@@ -43,9 +46,11 @@ namespace {
struct Instance final : rpc::build_remote::HookInstance::Server
{
unsigned int maxBuildJobs;
bool initialized = false, used = false;
Instance(unsigned int maxBuildJobs) : maxBuildJobs(maxBuildJobs) {}
kj::Promise<void> init(InitContext context) override;
kj::Promise<void> buildImpl(BuildContext context);
kj::Promise<void> build(BuildContext context) override;
};
}
@@ -183,27 +188,62 @@ struct BuilderConnection
// start the thread that reads ssh stderr and turns it into log items.
// this future *must* outlive sshStore, otherwise it will never finish
std::future<void> startLogThread(int intoFD)
{
kj::Promise<Result<void>> startLogThread(std::string buildDescription, std::string drvPath)
try {
if (!logPipe.readSide) {
return {};
co_return result::success();
}
logPipe.writeSide.close();
return std::async(
std::launch::async,
[](int from, int to) {
AsyncIoRoot aio;
auto reader = AIO().lowLevelProvider.wrapInputFd(from);
auto writer = AIO().lowLevelProvider.wrapOutputFd(to);
reader->pumpTo(*writer).wait(aio.kj.waitScope);
},
logPipe.readSide.get(),
intoFD
// NOTE this is very similar to handleBuilderOutput in DerivationGoal, but unlike
// the derivation goal we do not need to handle EIO from a pty here. we also have
// no timeouts or limits to keep track of, which makes deduplication less useful.
auto act = logger->startActivity(
lvlInfo, actBuild, buildDescription, Logger::Fields{drvPath, storeUri, 1, 1}
);
std::map<ActivityId, Activity> activities;
auto reader = AIO().lowLevelProvider.wrapInputFd(logPipe.readSide.get());
LogLineSplitter splitter;
auto flushLine = [&](const std::string & line) {
if (const auto state =
handleJSONLogMessage(line, act, activities, "the derivation builder"))
{
return *state;
} else {
return act.result(resBuildLogLine, line);
}
};
auto buf = kj::heapArray<char>(4096);
while (true) {
const auto got = co_await reader->tryRead(buf.begin(), 1, buf.size());
if (got == 0) {
break;
}
std::string_view data{buf.begin(), got};
while (!data.empty()) {
if (auto line = splitter.feed(data)) {
if (flushLine(*line) == Logger::BufferState::NeedsFlush) {
TRY_AWAIT(act.getLogger().flush());
}
}
}
}
if (auto line = splitter.finish(); !line.empty()) {
(void) flushLine(line);
TRY_AWAIT(act.getLogger().flush());
}
co_return result::success();
} catch (...) {
co_return result::current_exception();
}
};
@@ -212,21 +252,16 @@ struct AcceptedBuild final : rpc::build_remote::HookInstance::AcceptedBuild::Ser
ref<Store> store;
StorePath drvPath;
BuilderConnection builder;
rpc::build_remote::HookInstance::BuildLogger::Client buildLogger;
bool used = false;
AcceptedBuild(
ref<Store> store,
StorePath drvPath,
BuilderConnection builder,
rpc::build_remote::HookInstance::BuildLogger::Client buildLogger
)
AcceptedBuild(ref<Store> store, StorePath drvPath, BuilderConnection builder)
: store(store)
, drvPath(drvPath)
, builder(std::move(builder))
, buildLogger(std::move(buildLogger))
{
}
kj::Promise<void> runImpl(RunContext context);
kj::Promise<void> run(RunContext context) override;
};
@@ -235,7 +270,7 @@ enum class BuildRejected { Temporarily, Permanently };
static kj::Promise<Result<std::variant<BuildRejected, BuilderConnection>>> connectToBuilder(
const ref<Store> & store,
const std::optional<StorePath> & drvPath,
const StorePath & drvPath,
Machines & machines,
const unsigned int maxBuildJobs,
const bool amWilling,
@@ -255,7 +290,7 @@ try {
bool canBuildLocally = amWilling && couldBuildLocally;
/* Error ignored here, will be caught later */
mkdir(currentLoad.c_str(), 0777);
(void) sys::mkdir(currentLoad, 0777);
while (true) {
bestSlotLock.reset();
@@ -272,7 +307,7 @@ try {
} else {
printSelectionFailureMessage(
couldBuildLocally ? lvlChatty : lvlWarn,
drvPath ? drvPath->to_string() : "<unknown>",
drvPath.to_string(),
machines,
neededSystem,
requiredFeatures
@@ -294,8 +329,8 @@ try {
Pipe logPipe;
try {
Activity act(
*logger, lvlTalkative, actUnknown, fmt("connecting to '%s'", bestMachine->storeUri)
auto act = logger->startActivity(
lvlTalkative, actUnknown, fmt("connecting to '%s'", bestMachine->storeUri)
);
std::tie(sshStore, logPipe) = TRY_AWAIT(bestMachine->openStore());
@@ -321,8 +356,6 @@ try {
static int main_build_remote(AsyncIoRoot & aio, std::string programName, Strings argv)
{
{
logger = makeJSONLogger(*logger);
/* Ensure we don't get any SSH passphrase or host key popups. */
unsetenv("DISPLAY");
unsetenv("SSH_ASKPASS");
@@ -335,104 +368,146 @@ static int main_build_remote(AsyncIoRoot & aio, std::string programName, Strings
verbosity = (Verbosity) std::stoll(argv.front());
FdSource source(STDIN_FILENO);
/* Read the parent's settings. */
while (readNum<unsigned>(source)) {
auto name = readString(source);
auto value = readString(source);
settings.set(name, value);
}
auto maxBuildJobs = settings.maxBuildJobs;
settings.maxBuildJobs.set("1"); // hack to make tests with local?root= work
initPlugins();
auto conn = aio.kj.lowLevelProvider->wrapUnixSocketFd(1);
capnp::TwoPartyServer srv(kj::heap<Instance>(maxBuildJobs));
capnp::TwoPartyServer srv(kj::heap<Instance>());
srv.accept(*conn, 1).wait(aio.kj.waitScope);
return 0;
}
}
kj::Promise<void> Instance::build(BuildContext context)
kj::Promise<void> Instance::init(InitContext context)
{
try {
// FIXME this does not open a daemon connection for historical reasons.
// we may create a lot of build hook instances, and having each of them
// also create a daemon instance is inefficient and wasteful. in future
// versions of the build hook (where we don't need one hook process per
// build) we should change this to using a daemon connection, ideally a
// daemon connection provided by the parent via file descriptor passing
auto store = TRY_AWAIT(openStore(settings.storeUri, {}, AllowDaemon::Disallow));
/* It would be more appropriate to use $XDG_RUNTIME_DIR, since
that gets cleared on reboot, but it wouldn't work on macOS. */
auto currentLoadName = "/current-load";
if (auto localStore = store.try_cast_shared<LocalFSStore>())
currentLoad = std::string { localStore->config().stateDir } + currentLoadName;
else
currentLoad = settings.nixStateDir + currentLoadName;
auto machines = getMachines();
debug("got %d remote builders", machines.size());
if (machines.empty()) {
context.getResults().initResult().initGood().setDeclinePermanently();
co_return;
if (initialized) {
throw Error("build hook can only be initialized once");
}
auto amWilling = context.getParams().getAmWilling();
auto neededSystem = rpc::to<std::string>(context.getParams().getNeededSystem());
auto drvPath = from(context.getParams().getDrvPath(), *store);
auto requiredFeatures =
rpc::to<std::set<std::string>>(context.getParams().getRequiredFeatures());
auto buildLogger = context.getParams().getBuildLogger();
logger = rpc::log::makeRpcLoggerClient(context.getParams().getLogger());
auto result = TRY_AWAIT(connectToBuilder(
store, drvPath, machines, maxBuildJobs, amWilling, neededSystem, requiredFeatures
));
if (auto immediateResponse = std::get_if<BuildRejected>(&result)) {
switch (*immediateResponse) {
case BuildRejected::Temporarily:
context.getResults().initResult().initGood().setPostpone();
co_return;
case BuildRejected::Permanently:
context.getResults().initResult().initGood().setDecline();
co_return;
}
/* Read the parent's settings. */
for (const auto & [name, value] : rpc::to<StringMap>(context.getParams().getSettings())) {
settings.set(name, value);
}
auto builder = std::get_if<BuilderConnection>(&result);
assert(builder);
maxBuildJobs = settings.maxBuildJobs;
settings.maxBuildJobs.set("1"); // hack to make tests with local?root= work
auto ac = context.getResults().initResult().initGood().initAccept();
RPC_FILL(ac, setMachineName, builder->storeUri);
ac.setMachine(kj::heap<AcceptedBuild>(store, drvPath, std::move(*builder), buildLogger));
initPlugins();
initialized = true;
context.getResults().initResult().setGood();
} catch (...) {
RPC_FILL(context.getResults(), initResult, std::current_exception());
}
return kj::READY_NOW;
}
kj::Promise<void> Instance::buildImpl(BuildContext context)
{
if (!initialized) {
throw Error("build hook not fully initialized");
}
// FIXME this does not open a daemon connection for historical reasons.
// we may create a lot of build hook instances, and having each of them
// also create a daemon instance is inefficient and wasteful. in future
// versions of the build hook (where we don't need one hook process per
// build) we should change this to using a daemon connection, ideally a
// daemon connection provided by the parent via file descriptor passing
auto store = TRY_AWAIT(openStore(settings.storeUri, {}, AllowDaemon::Disallow));
/* It would be more appropriate to use $XDG_RUNTIME_DIR, since
that gets cleared on reboot, but it wouldn't work on macOS. */
auto currentLoadName = "/current-load";
if (auto localStore = store.try_cast_shared<LocalFSStore>()) {
currentLoad = std::string{localStore->config().stateDir} + currentLoadName;
} else {
currentLoad = settings.nixStateDir + currentLoadName;
}
auto machines = getMachines();
debug("got %d remote builders", machines.size());
if (machines.empty()) {
context.getResults().initResult().initGood().setDeclinePermanently();
co_return;
}
auto amWilling = context.getParams().getAmWilling();
auto neededSystem = rpc::to<std::string>(context.getParams().getNeededSystem());
auto drvPath = from(context.getParams().getDrvPath(), *store);
auto requiredFeatures =
rpc::to<std::set<std::string>>(context.getParams().getRequiredFeatures());
auto result = TRY_AWAIT(connectToBuilder(
store, drvPath, machines, maxBuildJobs, amWilling, neededSystem, requiredFeatures
));
if (auto immediateResponse = std::get_if<BuildRejected>(&result)) {
switch (*immediateResponse) {
case BuildRejected::Temporarily:
context.getResults().initResult().initGood().setPostpone();
co_return;
case BuildRejected::Permanently:
context.getResults().initResult().initGood().setDecline();
co_return;
}
}
auto builder = std::get_if<BuilderConnection>(&result);
assert(builder);
auto ac = context.getResults().initResult().initGood().initAccept();
ac.setMachine(kj::heap<AcceptedBuild>(store, drvPath, std::move(*builder)));
}
kj::Promise<void> Instance::build(BuildContext context)
try {
if (used) {
throw Error("build hooks can only accept a single job");
}
used = true; // lock out other rpc calls during processing
co_await buildImpl(context);
TRY_AWAIT(logger->flush());
used = context.getResults().getResult().getGood().isAccept();
} catch (...) {
RPC_FILL(context.getResults(), getResult, std::current_exception());
}
kj::Promise<void> AcceptedBuild::run(RunContext context)
{
try {
const int logFD = (co_await buildLogger.getFd()).orDefault(-1);
if (logFD < 0) {
throw Error("build-hook needs a logFD from the builder to build");
}
auto logThread = builder.startLogThread(logFD);
auto oldLogger = logger;
logger = rpc::log::makeRpcLoggerClient(context.getParams().getLogger());
TRY_AWAIT(oldLogger->flush());
KJ_DEFER({
// drop any existing ssh connection so the log thread can exit
builder.sshStore = nullptr;
if (logThread.valid()) {
logThread.get();
}
delete logger;
logger = oldLogger;
});
if (used) {
throw Error("build hooks builds are single-use items");
}
used = true;
co_await runImpl(context);
TRY_AWAIT(logger->flush());
} catch (...) {
RPC_FILL(context.getResults(), getResult, std::current_exception());
}
}
kj::Promise<void> AcceptedBuild::runImpl(RunContext context)
{
try {
auto logHandler = builder.startLogThread(
fmt("%s on '%s'",
rpc::to<std::string_view>(context.getParams().getDescription()),
builder.storeUri),
store->printStorePath(drvPath)
);
auto & sshStore = builder.sshStore;
auto & storeUri = builder.storeUri;
@@ -444,7 +519,9 @@ kj::Promise<void> AcceptedBuild::run(RunContext context)
AutoCloseFD uploadLock = openLockFile(lockFileName, true);
{
Activity act(*logger, lvlTalkative, actUnknown, fmt("waiting for the upload lock to '%s'", storeUri));
auto act = logger->startActivity(
lvlTalkative, actUnknown, fmt("waiting for the upload lock to '%s'", storeUri)
);
auto result = TRY_AWAIT(
AIO().timeoutAfter(15 * kj::MINUTES, lockFileAsync(uploadLock.get(), ltWrite))
@@ -457,7 +534,9 @@ kj::Promise<void> AcceptedBuild::run(RunContext context)
auto substitute = settings.buildersUseSubstitutes ? Substitute : NoSubstitute;
{
Activity act(*logger, lvlTalkative, actUnknown, fmt("copying dependencies to '%s'", storeUri));
auto act = logger->startActivity(
lvlTalkative, actUnknown, fmt("copying dependencies to '%s'", storeUri)
);
TRY_AWAIT(copyPaths(*store, *sshStore, inputs, NoRepair, NoCheckSigs, substitute));
}
@@ -527,7 +606,9 @@ kj::Promise<void> AcceptedBuild::run(RunContext context)
}
if (!missingPaths.empty()) {
Activity act(*logger, lvlTalkative, actUnknown, fmt("copying outputs from '%s'", storeUri));
auto act = logger->startActivity(
lvlTalkative, actUnknown, fmt("copying outputs from '%s'", storeUri)
);
if (auto localStore = store.try_cast_shared<LocalStore>())
for (auto & path : missingPaths)
localStore->locksHeld.insert(store->printStorePath(path)); /* FIXME: ugly */
@@ -536,6 +617,10 @@ kj::Promise<void> AcceptedBuild::run(RunContext context)
);
}
// drop store connection, let log handler process any remaining input
builder.sshStore = nullptr;
TRY_AWAIT(logHandler);
context.getResults().initResult().setGood();
} catch (...) {
RPC_FILL(context.getResults(), initResult, std::current_exception());
+61 -53
View File
@@ -9,6 +9,7 @@
#include "lix/libstore/store-api.hh"
#include "lix/libstore/local-fs-store.hh"
#include "lix/libstore/globals.hh"
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/current-process.hh"
#include "lix/libstore/derivations.hh"
#include "lix/libmain/shared.hh"
@@ -18,6 +19,8 @@
#include "lix/libcmd/common-eval-args.hh"
#include "lix/libexpr/attr-path.hh"
#include "lix/libcmd/legacy.hh"
#include "lix/libutil/finally.hh"
#include "lix/libutil/processes.hh"
#include "lix/libutil/regex.hh"
#include "lix/libutil/shlex.hh"
#include "nix-build.hh"
@@ -29,7 +32,7 @@ namespace nix {
using namespace std::string_literals;
static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings argv)
static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings argv)
{
auto dryRun = false;
auto runEnv = std::regex_search(programName, regex::parse("nix-shell$"));
@@ -187,7 +190,8 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
if (packages && fromArgs)
throw UsageError("'-p' and '-E' are mutually exclusive");
AutoDelete tmpDir(createTempDir("", myName));
AutoDelete tmpDir(createTempDir(myName));
AutoDelete buildTopTmpDir(createTempSubdir(tmpDir, "build-top"));
if (outLink.empty())
outLink = (Path) tmpDir + "/result";
@@ -224,8 +228,9 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
left = {"default.nix"};
}
if (runEnv)
setenv("IN_NIX_SHELL", pure ? "pure" : "impure", 1);
if (runEnv) {
(void) sys::setenv("IN_NIX_SHELL", pure ? "pure" : "impure", 1);
}
DrvInfos drvs;
@@ -272,7 +277,7 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
}
bool add = false;
if (v.type() == nFunction) {
if (auto pattern = dynamic_cast<AttrsPattern *>(v.lambda.fun->pattern.get())) {
if (auto pattern = dynamic_cast<AttrsPattern *>(v.lambda().fun->pattern.get())) {
for (auto & i : pattern->formals) {
if (evaluator->symbols[i.name] == "inNixShell") {
add = true;
@@ -285,12 +290,12 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
};
for (auto & i : attrPaths) {
Value & v(*findAlongAttrPath(
*state,
i,
takesNixShellAttr(vRoot) ? *autoArgsWithInNixShell : *autoArgs,
vRoot
).first);
Value v(
findAlongAttrPath(
*state, i, takesNixShellAttr(vRoot) ? *autoArgsWithInNixShell : *autoArgs, vRoot
)
.first
);
state->forceValue(v, noPos);
getDerivations(
*state,
@@ -397,7 +402,9 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
buildPaths(pathsToBuild);
if (dryRun) return;
if (dryRun) {
return 0;
}
if (shellDrv) {
auto shellDrvOutputs =
@@ -419,24 +426,24 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
}
// Don't use defaultTempDir() here! We want to preserve the user's TMPDIR for the shell
env["NIX_BUILD_TOP"] = env["TMPDIR"] = env["TEMPDIR"] = env["TMP"] = env["TEMP"] = getEnvNonEmpty("TMPDIR").value_or("/tmp");
env["NIX_BUILD_TOP"] = env["TMPDIR"] = env["TEMPDIR"] = env["TMP"] = env["TEMP"] =
getEnvNonEmpty("TMPDIR").value_or(buildTopTmpDir);
env["NIX_STORE"] = store->config().storeDir;
env["NIX_BUILD_CORES"] = std::to_string(settings.buildCores);
auto passAsFile = tokenizeString<StringSet>(getOr(drv.env, "passAsFile", ""));
bool keepTmp = false;
int fileNr = 0;
for (auto & var : drv.env)
if (passAsFile.count(var.first)) {
keepTmp = true;
auto fn = ".attr-" + std::to_string(fileNr++);
Path p = (Path) tmpDir + "/" + fn;
writeFile(p, var.second);
env[var.first + "Path"] = p;
} else
} else {
env[var.first] = var.second;
}
std::string structuredAttrsRC;
@@ -469,7 +476,6 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
env["NIX_ATTRS_SH_FILE"] = attrsSH;
env["NIX_ATTRS_JSON_FILE"] = attrsJSON;
keepTmp = true;
}
}
@@ -479,24 +485,13 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
lose the current $PATH directories. */
auto rcfile = (Path) tmpDir + "/rc";
auto tz = getEnv("TZ");
std::string rc = fmt(
R"(_nix_shell_clean_tmpdir() { command rm -rf %1%; }; )"
"%2%"
"%3%"
// always clear PATH.
// when nix-shell is run impure, we rehydrate it with the `p=$PATH` above
"unset PATH;"
"dontAddDisableDepTrack=1;\n",
shellEscape(tmpDir),
(keepTmp
? "trap _nix_shell_clean_tmpdir EXIT; "
"exitHooks+=(_nix_shell_clean_tmpdir); "
"failureHooks+=(_nix_shell_clean_tmpdir); "
: "_nix_shell_clean_tmpdir; "),
(pure
? ""
: "[ -n \"$PS1\" ] && [ -e ~/.bashrc ] && source ~/.bashrc; p=$PATH; ")
);
std::string rc =
fmt("%1%"
// always clear PATH.
// when nix-shell is run impure, we rehydrate it with the `p=$PATH` above
"unset PATH;"
"dontAddDisableDepTrack=1;\n",
(pure ? "" : "[ -n \"$PS1\" ] && [ -e ~/.bashrc ] && source ~/.bashrc; p=$PATH; "));
rc += structuredAttrsRC;
rc += fmt(
"\n[ -e $stdenv/setup ] && source $stdenv/setup; "
@@ -526,29 +521,38 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
vomit("Sourcing nix-shell with file %s and contents:\n%s", rcfile, rc);
writeFile(rcfile, rc);
Strings envStrs;
for (auto & i : env)
envStrs.push_back(i.first + "=" + i.second);
auto args = interactive
? Strings{"bash", "--rcfile", rcfile}
: Strings{"bash", rcfile};
auto envPtrs = stringsToCharPtrs(envStrs);
environ = envPtrs.data();
auto argPtrs = stringsToCharPtrs(args);
restoreProcessContext();
auto args = interactive ? Strings{"--rcfile", rcfile} : Strings{rcfile};
// We are going to run an interactive command, do not let the logger send a line.
logger->pause();
printMsg(lvlChatty, "running shell: %s", concatMapStringsSep(" ", args, shellEscape));
execvp(shell->c_str(), argPtrs.data());
RunningProgram proc = runProgram2(
{.program = *shell,
.searchPath = true,
.args = args,
.environment = env,
.dieWithParent = true}
);
throw SysError("executing shell '%s'", *shell);
// NOTE: we wait and return the status check immediately.
// If there's interruption, we will swallow it and wait again for termination.
auto toExitStatus = [](int waitRes) {
if (WIFEXITED(waitRes)) {
return WEXITSTATUS(waitRes);
} else if (WIFSIGNALED(waitRes)) {
return 128 + WTERMSIG(waitRes);
} else {
return 255;
}
};
try {
return toExitStatus(proc.wait());
} catch (Interrupted &) {
return toExitStatus(proc.wait());
}
}
else {
@@ -582,7 +586,9 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
buildPaths(pathsToBuild);
if (dryRun) return;
if (dryRun) {
return 0;
}
std::vector<StorePath> outPaths;
@@ -611,6 +617,8 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
for (auto & path : outPaths)
std::cout << store->printStorePath(path) << '\n';
}
return 0;
}
void registerLegacyNixBuildAndNixShell() {
+5 -3
View File
@@ -8,6 +8,7 @@
#include "lix/libexpr/eval-settings.hh" // for defexpr
#include "lix/libstore/temporary-dir.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/regex.hh"
#include "lix/libutil/result.hh"
#include "lix/libutil/users.hh"
@@ -44,7 +45,7 @@ static void readChannels()
// Writes the list of channels.
static void writeChannels()
{
auto channelsFD = AutoCloseFD{open(channelsList.c_str(), O_WRONLY | O_CLOEXEC | O_CREAT | O_TRUNC, 0644)};
auto channelsFD = sys::open(channelsList, O_WRONLY | O_CLOEXEC | O_CREAT | O_TRUNC, 0644);
if (!channelsFD)
throw SysError("opening '%1%' for writing", channelsList);
for (const auto & channel : channels)
@@ -174,11 +175,12 @@ static void update(AsyncIoRoot & aio, const StringSet & channelNames)
// Make the channels appear in nix-env.
struct stat st;
if (lstat(nixDefExpr.c_str(), &st) == 0) {
if (sys::lstat(nixDefExpr, &st) == 0) {
if (S_ISLNK(st.st_mode))
// old-skool ~/.nix-defexpr
if (unlink(nixDefExpr.c_str()) == -1)
if (sys::unlink(nixDefExpr) == -1) {
throw SysError("unlinking %1%", nixDefExpr);
}
} else if (errno != ENOENT) {
throw SysError("getting status of %1%", nixDefExpr);
}
+5 -2
View File
@@ -1,3 +1,4 @@
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/file-system.hh"
#include "lix/libstore/store-api.hh"
#include "lix/libstore/store-cast.hh"
@@ -24,9 +25,11 @@ bool dryRun = false;
static void removeOldGenerations(std::string dir, NeverAsync = {})
{
if (access(dir.c_str(), R_OK) != 0) return;
if (sys::access(dir, R_OK) != 0) {
return;
}
bool canWrite = access(dir.c_str(), W_OK) == 0;
bool canWrite = sys::access(dir, W_OK) == 0;
for (auto & i : readDirectory(dir)) {
checkInterrupt();
+24 -18
View File
@@ -1,6 +1,7 @@
#include "lix/libcmd/cmd-profiles.hh"
#include "lix/libexpr/attr-path.hh"
#include "lix/libcmd/common-eval-args.hh"
#include "lix/libexpr/value.hh"
#include "lix/libstore/derivations.hh"
#include "lix/libutil/terminal.hh"
#include "lix/libexpr/eval.hh"
@@ -150,11 +151,12 @@ static void getAllExprs(Evaluator & state,
continue;
}
/* Load the expression on demand. */
auto vArg = state.mem.allocValue();
vArg->mkString(path2.canonical().abs());
Value vArg;
vArg.mkString(path2.canonical().abs());
if (seen.size() == maxAttrs)
throw Error("too many Nix expressions in directory '%1%'", path);
attrs.alloc(attrName).mkApp(&state.builtins.get("import"), vArg);
attrs.alloc(attrName
) = {NewValueAs::app, state.mem, state.builtins.get("import"), vArg};
}
else if (st.type == InputAccessor::tDirectory)
/* `path2' is a directory (with no default.nix in it);
@@ -181,7 +183,7 @@ static void loadSourceExpr(EvalState & state, const SourcePath & path_, Value &
directory). */
else if (st.type == InputAccessor::tDirectory) {
auto attrs = state.ctx.buildBindings(maxAttrs);
attrs.alloc("_combineChannels").mkList(0);
attrs.alloc("_combineChannels") = Value::EMPTY_LIST;
StringSet seen;
getAllExprs(state.ctx, path, seen, attrs);
v.mkAttrs(attrs);
@@ -198,7 +200,7 @@ static void loadDerivations(EvalState & state, const SourcePath & nixExprPath,
Value vRoot;
loadSourceExpr(state, nixExprPath, vRoot);
Value & v(*findAlongAttrPath(state, pathPrefix, autoArgs, vRoot).first);
Value v(findAlongAttrPath(state, pathPrefix, autoArgs, vRoot).first);
getDerivations(state, v, pathPrefix, autoArgs, elems, true);
@@ -425,7 +427,7 @@ static void queryInstSources(EvalState & state,
Expr & eFun = state.ctx.parseExprFromString(i, CanonPath::fromCwd());
Value vFun, vTmp;
state.eval(eFun, vFun);
vTmp.mkApp(&vFun, &vArg);
vTmp = {NewValueAs::app, state.ctx.mem, vFun, vArg};
getDerivations(state, vTmp, "", *instSource.autoArgs, elems, true);
}
@@ -480,7 +482,7 @@ static void queryInstSources(EvalState & state,
Value vRoot;
loadSourceExpr(state, *instSource.nixExprPath, vRoot);
for (auto & i : args) {
Value & v(*findAlongAttrPath(state, i, *instSource.autoArgs, vRoot).first);
Value v(findAlongAttrPath(state, i, *instSource.autoArgs, vRoot).first);
getDerivations(state, v, "", *instSource.autoArgs, elems, true);
}
break;
@@ -515,8 +517,8 @@ static bool keep(EvalState & state, DrvInfo & drv)
static void setMetaFlag(EvalState & state, DrvInfo & drv,
const std::string & name, const std::string & value)
{
auto v = state.ctx.mem.allocValue();
v->mkString(value);
Value v;
v.mkString(value);
drv.setMeta(state, name, v);
}
@@ -1275,35 +1277,39 @@ static void opQuery(Globals & globals, Strings opFlags, Strings opArgs)
xml.writeEmptyElement("meta", attrs2);
} else if (v->type() == nInt) {
attrs2["type"] = "int";
attrs2["value"] = fmt("%1%", v->integer);
attrs2["value"] = fmt("%1%", v->integer());
xml.writeEmptyElement("meta", attrs2);
} else if (v->type() == nFloat) {
attrs2["type"] = "float";
attrs2["value"] = fmt("%1%", v->fpoint);
attrs2["value"] = fmt("%1%", v->fpoint());
xml.writeEmptyElement("meta", attrs2);
} else if (v->type() == nBool) {
attrs2["type"] = "bool";
attrs2["value"] = v->boolean ? "true" : "false";
attrs2["value"] = v->boolean() ? "true" : "false";
xml.writeEmptyElement("meta", attrs2);
} else if (v->type() == nList) {
attrs2["type"] = "strings";
XMLOpenElement m(xml, "meta", attrs2);
for (auto elem : v->listItems()) {
if (elem->type() != nString) continue;
for (auto & elem : v->listItems()) {
if (elem.type() != nString) {
continue;
}
XMLAttrs attrs3;
attrs3["value"] = elem->str();
attrs3["value"] = elem.str();
xml.writeEmptyElement("string", attrs3);
}
} else if (v->type() == nAttrs) {
attrs2["type"] = "strings";
XMLOpenElement m(xml, "meta", attrs2);
Bindings & attrs = *v->attrs;
Bindings & attrs = *v->attrs();
for (auto &i : attrs) {
const Attr & a(*attrs.get(i.name));
if(a.value->type() != nString) continue;
if (a.value.type() != nString) {
continue;
}
XMLAttrs attrs3;
attrs3["type"] = globals.state->symbols[i.name];
attrs3["value"] = a.value->str();
attrs3["value"] = a.value.str();
xml.writeEmptyElement("string", attrs3);
}
}
+1 -1
View File
@@ -38,7 +38,7 @@ void processExpr(EvalState & state, const Strings & attrPaths,
state.eval(e, vRoot);
for (auto & i : attrPaths) {
Value & v(*findAlongAttrPath(state, i, autoArgs, vRoot).first);
Value v(findAlongAttrPath(state, i, autoArgs, vRoot).first);
state.forceValue(v, noPos);
NixStringContext context;
+11
View File
@@ -25,6 +25,7 @@
#include <iostream>
#include <algorithm>
#include <ranges>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
@@ -88,6 +89,16 @@ try {
if (i == drv.outputs.end())
throw Error("derivation '%s' does not have an output named '%s'",
store2->printStorePath(path.path), j);
if (!outputPaths.contains(i->first)) {
throw Error(
"Possible SQLite database corruption: derivation '%s' output map contains only "
"outputs '{%s}', not '%s'\n"
"Note: derivation output maps are stored in the SQLite database.",
store2->printStorePath(path.path),
concatStringsSep(", ", std::views::keys(outputPaths)),
i->first
);
}
auto outPath = outputPaths.at(i->first);
auto retPath = store->printStorePath(outPath);
if (store2) {
+15 -13
View File
@@ -1,4 +1,5 @@
#include "user-env.hh"
#include "lix/libexpr/value.hh"
#include "lix/libstore/derivations.hh"
#include "lix/libstore/store-api.hh"
#include "lix/libstore/path-with-outputs.hh"
@@ -32,7 +33,8 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
/* Construct the whole top level derivation. */
StorePathSet references;
Value manifest = state.ctx.mem.newList(elems.size());
auto manifest = state.ctx.mem.newList(elems.size());
Value vManifest{NewValueAs::list, manifest};
size_t n = 0;
for (auto & i : elems) {
/* Create a pseudo-derivation containing the name, system,
@@ -55,9 +57,10 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
// Copy each output meant for installation.
auto & vOutputs = attrs.alloc(state.ctx.s.outputs);
vOutputs = state.ctx.mem.newList(outputs.size());
auto outputsList = state.ctx.mem.newList(outputs.size());
vOutputs = {NewValueAs::list, outputsList};
for (const auto & [m, j] : enumerate(outputs)) {
(vOutputs.listElems()[m] = state.ctx.mem.allocValue())->mkString(j.first);
outputsList->elems[m].mkString(j.first);
auto outputAttrs = state.ctx.buildBindings(2);
outputAttrs.alloc(state.ctx.s.outPath).mkString(state.ctx.store->printStorePath(*j.second));
attrs.alloc(j.first).mkAttrs(outputAttrs);
@@ -75,12 +78,12 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
for (auto & j : metaNames) {
Value * v = i.queryMeta(state, j);
if (!v) continue;
meta.insert(state.ctx.symbols.create(j), v);
meta.insert(state.ctx.symbols.create(j), *v);
}
attrs.alloc(state.ctx.s.meta).mkAttrs(meta);
(manifest.listElems()[n++] = state.ctx.mem.allocValue())->mkAttrs(attrs);
manifest->elems[n++].mkAttrs(attrs);
if (drvPath) references.insert(*drvPath);
}
@@ -89,7 +92,7 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
the store; we need it for future modifications of the
environment. */
std::ostringstream str;
printAmbiguous(manifest, state.ctx.symbols, str, nullptr, std::numeric_limits<int>::max());
printAmbiguous(vManifest, state.ctx.symbols, str, nullptr, std::numeric_limits<int>::max());
auto manifestFile = state.aio.blockOn(state.ctx.store->addTextToStore("env-manifest.nix",
str.str(), references));
@@ -103,21 +106,20 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
builder with the manifest as argument. */
auto attrs = state.ctx.buildBindings(3);
state.ctx.paths.mkStorePathString(manifestFile, attrs.alloc("manifest"));
attrs.insert(state.ctx.symbols.create("derivations"), &manifest);
attrs.insert(state.ctx.symbols.create("derivations"), vManifest);
Value args;
args.mkAttrs(attrs);
Value topLevel;
topLevel.mkApp(&envBuilder, &args);
Value topLevel{NewValueAs::app, state.ctx.mem, envBuilder, args};
/* Evaluate it. */
debug("evaluating user environment builder");
state.forceValue(topLevel, noPos);
NixStringContext context;
const Attr & aDrvPath(*topLevel.attrs->get(state.ctx.s.drvPath));
auto topLevelDrv = state.coerceToStorePath(aDrvPath.pos, *aDrvPath.value, context, "");
const Attr & aOutPath(*topLevel.attrs->get(state.ctx.s.outPath));
auto topLevelOut = state.coerceToStorePath(aOutPath.pos, *aOutPath.value, context, "");
const Attr & aDrvPath(*topLevel.attrs()->get(state.ctx.s.drvPath));
auto topLevelDrv = state.coerceToStorePath(aDrvPath.pos, aDrvPath.value, context, "");
const Attr & aOutPath(*topLevel.attrs()->get(state.ctx.s.outPath));
auto topLevelOut = state.coerceToStorePath(aOutPath.pos, aOutPath.value, context, "");
/* Realise the resulting store expression. */
debug("building user environment");
+7 -3
View File
@@ -5,6 +5,8 @@
#include "lix/libstore/profiles.hh"
#include "lix/libcmd/repl.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/error.hh"
extern char * * environ __attribute__((weak));
@@ -303,8 +305,10 @@ void MixEnvironment::setEnviron() {
throw UsageError("--unset does not make sense with --ignore-environment");
for (const auto & var : keep) {
auto val = getenv(var.c_str());
if (val) stringsEnv.emplace_back(fmt("%s=%s", var.c_str(), val));
auto val = sys::getenv(var);
if (val) {
stringsEnv.emplace_back(fmt("%s=%s", var, val));
}
}
vectorEnv = stringsToCharPtrs(stringsEnv);
@@ -314,7 +318,7 @@ void MixEnvironment::setEnviron() {
throw UsageError("--keep does not make sense without --ignore-environment");
for (const auto & var : unset)
unsetenv(var.c_str());
(void) sys::unsetenv(var);
}
}
+4 -4
View File
@@ -113,7 +113,7 @@ MixEvalArgs::MixEvalArgs()
```
-I nixpkgs=channel:nixos-21.05
-I nixpkgs=https://nixos.org/channels/nixos-21.05/nixexprs.tar.xz
-I nixpkgs=https://channels.nixos.org/nixos-21.05/nixexprs.tar.xz
```
You can also fetch source trees using [flake URLs](./nix3-flake.md#url-like-syntax) and add them to the
@@ -183,13 +183,13 @@ Bindings * MixEvalArgs::getAutoArgs(Evaluator & state)
{
auto res = state.buildBindings(autoArgs.size());
for (auto & i : autoArgs) {
auto v = state.mem.allocValue();
Value v;
if (i.second[0] == 'E')
state.evalLazily(
state.parseExprFromString(i.second.substr(1), CanonPath::fromCwd()), *v
state.parseExprFromString(i.second.substr(1), CanonPath::fromCwd()), v
);
else
v->mkString(((std::string_view) i.second).substr(1));
v.mkString(((std::string_view) i.second).substr(1));
res.insert(state.symbols.create(i.first), v);
}
return res.finish();
+11 -9
View File
@@ -12,9 +12,10 @@ namespace nix {
InstallableAttrPath::InstallableAttrPath(
ref<eval_cache::CachingEvaluator> state,
SourceExprCommand & cmd,
Value * v,
Value & v,
const std::string & attrPath,
ExtendedOutputsSpec extendedOutputsSpec)
ExtendedOutputsSpec extendedOutputsSpec
)
: InstallableValue(state)
, cmd(cmd)
, v(allocRootValue(v))
@@ -22,10 +23,10 @@ InstallableAttrPath::InstallableAttrPath(
, extendedOutputsSpec(std::move(extendedOutputsSpec))
{ }
std::pair<Value *, PosIdx> InstallableAttrPath::toValue(EvalState & state)
std::pair<Value, PosIdx> InstallableAttrPath::toValue(EvalState & state)
{
auto [vRes, pos] = findAlongAttrPath(state, attrPath, *cmd.getAutoArgs(*evaluator), **v);
state.forceValue(*vRes, pos);
auto [vRes, pos] = findAlongAttrPath(state, attrPath, *cmd.getAutoArgs(*evaluator), *v);
state.forceValue(vRes, pos);
return {vRes, pos};
}
@@ -34,7 +35,7 @@ DerivedPathsWithInfo InstallableAttrPath::toDerivedPaths(EvalState & state)
auto [v, pos] = toValue(state);
if (std::optional derivedPathWithInfo = trySinglePathToDerivedPaths(
state, *v, pos, fmt("while evaluating the attribute '%s'", attrPath)
state, v, pos, fmt("while evaluating the attribute '%s'", attrPath)
))
{
return { *derivedPathWithInfo };
@@ -43,7 +44,7 @@ DerivedPathsWithInfo InstallableAttrPath::toDerivedPaths(EvalState & state)
Bindings & autoArgs = *cmd.getAutoArgs(*evaluator);
DrvInfos drvInfos;
getDerivations(state, *v, "", autoArgs, drvInfos, false);
getDerivations(state, v, "", autoArgs, drvInfos, false);
// Backward compatibility hack: group results by drvPath. This
// helps keep .all output together.
@@ -92,9 +93,10 @@ DerivedPathsWithInfo InstallableAttrPath::toDerivedPaths(EvalState & state)
InstallableAttrPath InstallableAttrPath::parse(
ref<eval_cache::CachingEvaluator> state,
SourceExprCommand & cmd,
Value * v,
Value & v,
std::string_view prefix,
ExtendedOutputsSpec extendedOutputsSpec)
ExtendedOutputsSpec extendedOutputsSpec
)
{
return {
state, cmd, v,
+7 -5
View File
@@ -20,13 +20,14 @@ class InstallableAttrPath : public InstallableValue
InstallableAttrPath(
ref<eval_cache::CachingEvaluator> state,
SourceExprCommand & cmd,
Value * v,
Value & v,
const std::string & attrPath,
ExtendedOutputsSpec extendedOutputsSpec);
ExtendedOutputsSpec extendedOutputsSpec
);
std::string what() const override { return attrPath; };
std::pair<Value *, PosIdx> toValue(EvalState & state) override;
std::pair<Value, PosIdx> toValue(EvalState & state) override;
DerivedPathsWithInfo toDerivedPaths(EvalState & state) override;
@@ -35,9 +36,10 @@ public:
static InstallableAttrPath parse(
ref<eval_cache::CachingEvaluator> state,
SourceExprCommand & cmd,
Value * v,
Value & v,
std::string_view prefix,
ExtendedOutputsSpec extendedOutputsSpec);
ExtendedOutputsSpec extendedOutputsSpec
);
};
}
+4 -3
View File
@@ -60,7 +60,8 @@ InstallableFlake::InstallableFlake(
DerivedPathsWithInfo InstallableFlake::toDerivedPaths(EvalState & state)
{
Activity act(*logger, lvlTalkative, actUnknown, fmt("evaluating derivation '%s'", what()));
auto act =
logger->startActivity(lvlTalkative, actUnknown, fmt("evaluating derivation '%s'", what()));
auto attr = getCursor(state);
@@ -136,9 +137,9 @@ DerivedPathsWithInfo InstallableFlake::toDerivedPaths(EvalState & state)
}};
}
std::pair<Value *, PosIdx> InstallableFlake::toValue(EvalState & state)
std::pair<Value, PosIdx> InstallableFlake::toValue(EvalState & state)
{
return {&getCursor(state)->forceValue(state), noPos};
return {getCursor(state)->forceValue(state), noPos};
}
std::vector<ref<eval_cache::AttrCursor>>
+1 -1
View File
@@ -55,7 +55,7 @@ struct InstallableFlake : InstallableValue
DerivedPathsWithInfo toDerivedPaths(EvalState & state) override;
std::pair<Value *, PosIdx> toValue(EvalState & state) override;
std::pair<Value, PosIdx> toValue(EvalState & state) override;
/**
* Get a cursor to every attrpath in getActualAttrPaths() that
+3 -2
View File
@@ -9,8 +9,9 @@ std::vector<ref<eval_cache::AttrCursor>>
InstallableValue::getCursors(EvalState & state)
{
auto evalCache =
std::make_shared<nix::eval_cache::EvalCache>(std::nullopt,
[&](EvalState & state) { return toValue(state).first; });
std::make_shared<nix::eval_cache::EvalCache>(std::nullopt, [&](EvalState & state) {
return toValue(state).first;
});
return {evalCache->getRoot()};
}
+1 -1
View File
@@ -77,7 +77,7 @@ struct InstallableValue : Installable
virtual ~InstallableValue() { }
virtual std::pair<Value *, PosIdx> toValue(EvalState & state) = 0;
virtual std::pair<Value, PosIdx> toValue(EvalState & state) = 0;
/**
* Get a cursor to each value this Installable could refer to.
+13 -15
View File
@@ -235,14 +235,13 @@ void SourceExprCommand::completeInstallable(EvalState & state, AddCompletions &
prefix_ = "";
}
auto [v, pos] = findAlongAttrPath(state, prefix_, *autoArgs, root);
Value &v1(*v);
auto [v1, pos] = findAlongAttrPath(state, prefix_, *autoArgs, root);
state.forceValue(v1, pos);
Value v2;
state.autoCallFunction(*autoArgs, v1, v2, pos);
if (v2.type() == nAttrs) {
for (auto & i : *v2.attrs) {
for (auto & i : *v2.attrs()) {
std::string name{evaluator->symbols[i.name]};
if (name.find(searchWord) == 0) {
if (prefix_ == "")
@@ -412,12 +411,12 @@ ref<eval_cache::EvalCache> openEvalCache(
if (getEnv("NIX_ALLOW_EVAL").value_or("1") == "0")
throw Error("not everything is cached, but evaluation is not allowed");
auto vFlake = state.ctx.mem.allocValue();
flake::callFlake(state, *lockedFlake, *vFlake);
Value vFlake;
flake::callFlake(state, *lockedFlake, vFlake);
state.forceAttrs(*vFlake, noPos, "while parsing cached flake data");
state.forceAttrs(vFlake, noPos, "while parsing cached flake data");
auto aOutputs = vFlake->attrs->get(state.ctx.symbols.create("outputs"));
auto aOutputs = vFlake.attrs()->get(state.ctx.symbols.create("outputs"));
assert(aOutputs);
return aOutputs->value;
@@ -450,25 +449,24 @@ Installables SourceExprCommand::parseInstallables(
throw UsageError("'--file' and '--expr' are exclusive");
auto evaluator = getEvaluator();
auto vFile = evaluator->mem.allocValue();
Value vFile;
if (file == "-") {
auto & e = evaluator->parseStdin();
state.eval(e, *vFile);
state.eval(e, vFile);
}
else if (file)
state.evalFile(state.aio.blockOn(lookupFileArg(*evaluator, *file)).unwrap(), *vFile);
state.evalFile(state.aio.blockOn(lookupFileArg(*evaluator, *file)).unwrap(), vFile);
else {
auto & e = evaluator->parseExprFromString(*expr, CanonPath::fromCwd());
state.eval(e, *vFile);
state.eval(e, vFile);
}
for (auto & s : ss) {
auto [prefix, extendedOutputsSpec] = ExtendedOutputsSpec::parse(s);
result.push_back(
make_ref<InstallableAttrPath>(
InstallableAttrPath::parse(
evaluator, *this, vFile, std::move(prefix), std::move(extendedOutputsSpec))));
result.push_back(make_ref<InstallableAttrPath>(InstallableAttrPath::parse(
evaluator, *this, vFile, std::move(prefix), std::move(extendedOutputsSpec)
)));
}
} else {
+1 -1
View File
@@ -9,7 +9,7 @@
namespace nix {
typedef std::function<void(AsyncIoRoot &, std::string, std::list<std::string>)> MainFunction;
typedef std::function<int(AsyncIoRoot &, std::string, std::list<std::string>)> MainFunction;
struct LegacyCommandRegistry
{
+6 -5
View File
@@ -1,3 +1,4 @@
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/error.hh"
#include "lix/libutil/file-system.hh"
#include "lix/libutil/logging.hh"
@@ -110,7 +111,7 @@ static el_status_t doCompletion() {
if (possible.size() == 1) {
const auto completion = *possible.cbegin();
if (completion.size() > s.size()) {
rl_insert_text(completion.c_str() + s.size());
rl_insert_text(requireCString(completion.substr(s.size())));
return redisplay();
}
@@ -134,7 +135,7 @@ static el_status_t doCompletion() {
}
if (len > 0) {
auto commonPrefix = possible.begin()->substr(start, len);
rl_insert_text(commonPrefix.c_str());
rl_insert_text(requireCString(commonPrefix));
el_ring_bell();
return redisplay();
}
@@ -154,7 +155,7 @@ ReadlineLikeInteracter::Guard ReadlineLikeInteracter::init(detail::ReplCompleter
logWarning(e.info());
}
el_hist_size = 1000;
read_history(historyFile.c_str());
read_history(requireCString(historyFile));
auto oldRepl = curRepl;
curRepl = repl;
Guard restoreRepl([oldRepl] { curRepl = oldRepl; });
@@ -202,7 +203,7 @@ bool ReadlineLikeInteracter::getLine(std::string & input, ReplPromptType promptT
};
setupSignals();
char * s = readline(promptForType(promptType));
char * s = readline(promptForType(promptType)); // NOLINT(lix-unsafe-c-calls)
Finally doFree([&]() { free(s); });
restoreSignals();
@@ -223,7 +224,7 @@ bool ReadlineLikeInteracter::getLine(std::string & input, ReplPromptType promptT
void ReadlineLikeInteracter::writeHistory()
{
int ret = write_history(historyFile.c_str());
int ret = write_history(requireCString(historyFile));
int writeHistErr = errno;
if (ret == 0) {
+66 -57
View File
@@ -5,6 +5,7 @@
#include <cstring>
#include <string_view>
#include "lix/libexpr/value.hh"
#include "lix/libutil/box_ptr.hh"
#include "lix/libcmd/repl-interacter.hh"
#include "lix/libcmd/repl.hh"
@@ -55,6 +56,7 @@ using NdString = std::unique_ptr<const char, decltype(&lixdoc_free_string)>;
*/
NdString lambdaDocsForPos(SourcePath const path, nix::Pos const &pos) {
std::string const file = path.to_string();
// NOLINTNEXTLINE(lix-unsafe-c-calls): paths are safe
return NdString{lixdoc_get_function_docs(file.c_str(), pos.line, pos.column), &lixdoc_free_string};
}
@@ -173,35 +175,37 @@ struct NixRepl
/**
* Get a list of each of the `repl-overlays` (parsed and evaluated).
*/
Value * replOverlays();
Value replOverlays();
/**
* Get the Nix function that composes the `repl-overlays` together.
*/
Value * getReplOverlaysEvalFunction();
Value getReplOverlaysEvalFunction();
/**
* Cached return value of `getReplOverlaysEvalFunction`.
*
* Note: This is `shared_ptr` to avoid garbage collection.
*/
std::shared_ptr<Value *> replOverlaysEvalFunction =
std::allocate_shared<Value *>(TraceableAllocator<Value *>(), nullptr);
std::shared_ptr<std::optional<Value>> replOverlaysEvalFunction =
std::allocate_shared<std::optional<Value>>(
TraceableAllocator<std::optional<Value>>(), std::nullopt
);
/**
* Get the `info` AttrSet that's passed as the first argument to each
* of the `repl-overlays`.
*/
Value * replInitInfo();
Value replInitInfo();
/**
* Get the current top-level bindings as an AttrSet.
*/
Value * bindingsToAttrs();
Value bindingsToAttrs();
/**
* Parse a file, evaluate its result, and force the resulting value.
*/
Value * evalFile(SourcePath & path);
Value evalFile(SourcePath & path);
void printValue(std::ostream & str,
Value & v,
@@ -450,7 +454,7 @@ StringSet NixRepl::completePrefix(const std::string &prefix)
e.eval(state, *env, v);
state.forceAttrs(v, noPos, "while evaluating an attrset for the purpose of completion (this error should not be displayed; file an issue?)");
for (auto & i : *v.attrs) {
for (auto & i : *v.attrs()) {
std::ostringstream output;
printAttributeName(output, evaluator.symbols[i.name]);
std::string name = output.str();
@@ -653,7 +657,7 @@ ProcessLineResult NixRepl::processLine(std::string line)
auto path = state.coerceToPath(noPos, v, context, "while evaluating the filename to edit");
return {path, 0};
} else if (v.isLambda()) {
auto pos = evaluator.positions[v.lambda.fun->pos];
auto pos = evaluator.positions[v.lambda().fun->pos];
if (auto path = std::get_if<CheckedSourcePath>(&pos.origin))
return {*path, pos.line};
else
@@ -822,7 +826,7 @@ ProcessLineResult NixRepl::processLine(std::string line)
logger->cout(trim(renderMarkdownToTerminal(markdown)));
} else if (v.isLambda()) {
auto pos = evaluator.positions[v.lambda.fun->pos];
auto pos = evaluator.positions[v.lambda().fun->pos];
if (auto path = std::get_if<CheckedSourcePath>(&pos.origin)) {
// Path and position have now been obtained, feed to nix-doc library to get data.
auto docComment = lambdaDocsForPos(*path, pos);
@@ -862,10 +866,10 @@ ProcessLineResult NixRepl::processLine(std::string line)
std::visit(overloaded {
[&](ExprReplBindings & b) {
for (auto & [name, e] : b.symbols) {
Value * v = state.ctx.mem.allocValue();
e->eval(state, *env, *v);
Value v;
e->eval(state, *env, v);
(void) e.release(); // NOLINT(bugprone-unused-return-value): leak because of thunk references
addVarToScope(name, *v);
addVarToScope(name, v);
}
},
[&](std::unique_ptr<Expr> & e) {
@@ -948,7 +952,7 @@ void NixRepl::loadFiles()
for (auto & [i, what] : getValues()) {
notice("Loading installable '%1%'...", Magenta(what));
addAttrsToScope(*i);
addAttrsToScope(i);
}
loadReplOverlays();
@@ -963,9 +967,9 @@ void NixRepl::loadReplOverlays()
notice("Loading '%1%'...", "repl-overlays");
auto replInitFilesFunction = getReplOverlaysEvalFunction();
Value &newAttrs(*evaluator.mem.allocValue());
SmallValueVector<3> args = {replInitInfo(), bindingsToAttrs(), replOverlays()};
state.callFunction(*replInitFilesFunction, args.size(), args.data(), newAttrs, noPos);
Value newAttrs;
Value args[] = {replInitInfo(), bindingsToAttrs(), replOverlays()};
state.callFunction(replInitFilesFunction, args, newAttrs, noPos);
// n.b. this does in fact load the stuff into the environment twice (once
// from the superset of the environment returned by repl-overlays and once
@@ -975,14 +979,14 @@ void NixRepl::loadReplOverlays()
addAttrsToScope(newAttrs);
}
Value * NixRepl::getReplOverlaysEvalFunction()
Value NixRepl::getReplOverlaysEvalFunction()
{
if (replOverlaysEvalFunction && *replOverlaysEvalFunction) {
return *replOverlaysEvalFunction;
return **replOverlaysEvalFunction;
}
auto evalReplInitFilesPath = CanonPath::root + "repl-overlays.nix";
*replOverlaysEvalFunction = evaluator.mem.allocValue();
*replOverlaysEvalFunction = Value{};
auto code =
#include "repl-overlays.nix.gen.hh"
;
@@ -994,14 +998,14 @@ Value * NixRepl::getReplOverlaysEvalFunction()
state.eval(expr, **replOverlaysEvalFunction);
return *replOverlaysEvalFunction;
return **replOverlaysEvalFunction;
}
Value * NixRepl::replOverlays()
Value NixRepl::replOverlays()
{
Value * replInits(evaluator.mem.allocValue());
*replInits = evaluator.mem.newList(evalSettings.replOverlays.get().size());
Value ** replInitElems = replInits->listElems();
Value replInits;
auto replInitStorage = evaluator.mem.newList(evalSettings.replOverlays.get().size());
replInits = {NewValueAs::list, replInitStorage};
size_t i = 0;
for (auto path : evalSettings.replOverlays.get()) {
@@ -1017,27 +1021,32 @@ Value * NixRepl::replOverlays()
auto replInit = evalFile(sourcePath);
evalSettings.pureEval.setDefault(prevPureEval);
if (!replInit->isLambda()) {
evaluator.errors.make<TypeError>(
"Expected `repl-overlays` entry %s to be a lambda but found %s: %s",
path,
showType(*replInit),
ValuePrinter(state, *replInit, errorPrintOptions)
)
.debugThrow();
}
if (auto attrs = dynamic_cast<AttrsPattern *>(replInit->lambda.fun->pattern.get()); attrs && !attrs->ellipsis) {
evaluator.errors.make<TypeError>(
"Expected first argument of %1% to have %2% to allow future versions of Lix to add additional attributes to the argument",
"repl-overlays",
"..."
)
.atPos(replInit->lambda.fun->pos)
if (!replInit.isLambda()) {
evaluator.errors
.make<TypeError>(
"Expected `repl-overlays` entry %s to be a lambda but found %s: %s",
path,
showType(replInit),
ValuePrinter(state, replInit, errorPrintOptions)
)
.debugThrow();
}
replInitElems[i] = replInit;
if (auto attrs = dynamic_cast<AttrsPattern *>(replInit.lambda().fun->pattern.get());
attrs && !attrs->ellipsis)
{
evaluator.errors
.make<TypeError>(
"Expected first argument of %1% to have %2% to allow future versions of Lix to "
"add additional attributes to the argument",
"repl-overlays",
"..."
)
.atPos(replInit.lambda().fun->pos)
.debugThrow();
}
replInitStorage->elems[i] = replInit;
i++;
}
@@ -1045,16 +1054,16 @@ Value * NixRepl::replOverlays()
return replInits;
}
Value * NixRepl::replInitInfo()
Value NixRepl::replInitInfo()
{
auto builder = evaluator.buildBindings(2);
Value * currentSystem(evaluator.mem.allocValue());
currentSystem->mkString(evalSettings.getCurrentSystem());
Value currentSystem;
currentSystem.mkString(evalSettings.getCurrentSystem());
builder.insert(evaluator.symbols.create("currentSystem"), currentSystem);
Value * info(evaluator.mem.allocValue());
info->mkAttrs(builder.finish());
Value info;
info.mkAttrs(builder.finish());
return info;
}
@@ -1090,7 +1099,7 @@ void NixRepl::addAttrsToScope(Value & attrs)
{
state.forceAttrs(attrs, noPos, "while evaluating an attribute set to be merged in the global scope");
addToScope(
*attrs.attrs, [](const Attr & a) { return a.name; }, [](const Attr & a) { return a.value; }
*attrs.attrs(), [](const Attr & a) { return a.name; }, [](const Attr & a) { return a.value; }
);
}
@@ -1112,19 +1121,19 @@ void NixRepl::addVarToScope(const Symbol name, Value & v)
} else {
notice("Added %s.", evaluator.symbols[name]);
}
env->values[displ++] = &v;
env->values[displ++] = v;
varNames.emplace(evaluator.symbols[name]);
}
Value * NixRepl::bindingsToAttrs()
Value NixRepl::bindingsToAttrs()
{
auto builder = evaluator.buildBindings(staticEnv->vars.size());
for (auto & [symbol, displacement] : staticEnv->vars) {
builder.insert(symbol, env->values[displacement]);
}
Value * attrs(evaluator.mem.allocValue());
attrs->mkAttrs(builder.finish());
Value attrs;
attrs.mkAttrs(builder.finish());
return attrs;
}
@@ -1147,12 +1156,12 @@ void NixRepl::evalString(std::string s, Value & v)
state.forceValue(v, noPos);
}
Value * NixRepl::evalFile(SourcePath & path)
Value NixRepl::evalFile(SourcePath & path)
{
auto & expr = evaluator.parseExprFromFile(evaluator.paths.checkSourcePath(path), staticEnv);
Value * result(evaluator.mem.allocValue());
expr.eval(state, *env, *result);
state.forceValue(*result, noPos);
Value result;
expr.eval(state, *env, result);
state.forceValue(result, noPos);
return result;
}
+1 -1
View File
@@ -8,7 +8,7 @@ namespace nix {
struct AbstractNixRepl : NeverAsync
{
typedef std::vector<std::pair<Value*,std::string>> AnnotatedValues;
typedef std::vector<std::pair<Value, std::string>> AnnotatedValues;
static ReplExitStatus
run(const SearchPath & searchPath,
+24 -22
View File
@@ -69,13 +69,12 @@ std::string unparseAttrPath(std::vector<std::string> const & attrPath)
return ret.str();
}
std::pair<Value *, PosIdx> findAlongAttrPath(EvalState & state, const std::string & attrPath,
Bindings & autoArgs, Value & vIn)
std::pair<Value, PosIdx>
findAlongAttrPath(EvalState & state, const std::string & attrPath, Bindings & autoArgs, Value & vIn)
{
auto tokens = parseAttrPath(attrPath);
Value * v = &vIn;
Value v = vIn;
PosIdx pos = noPos;
for (auto [attrPathIdx, attr] : enumerate(tokens)) {
@@ -84,10 +83,10 @@ std::pair<Value *, PosIdx> findAlongAttrPath(EvalState & state, const std::strin
auto attrIndex = string2Int<unsigned int>(attr);
/* Evaluate the expression. */
Value * vNew = state.ctx.mem.allocValue();
state.autoCallFunction(autoArgs, *v, *vNew, pos);
Value vNew;
state.autoCallFunction(autoArgs, v, vNew, pos);
v = vNew;
state.forceValue(*v, noPos);
state.forceValue(v, noPos);
/* It should evaluate to either a set or an expression,
according to what is specified in the attrPath. */
@@ -96,7 +95,7 @@ std::pair<Value *, PosIdx> findAlongAttrPath(EvalState & state, const std::strin
if (attr.empty())
throw Error("empty attribute name in selection path '%1%'", attrPath);
if (v->type() != nAttrs) {
if (v.type() != nAttrs) {
auto pathPart =
std::vector<std::string>(tokens.begin(), tokens.begin() + attrPathIdx);
state.ctx.errors
@@ -105,17 +104,18 @@ std::pair<Value *, PosIdx> findAlongAttrPath(EvalState & state, const std::strin
"set but is %3%: %4%",
attrPath,
unparseAttrPath(pathPart),
showType(*v),
ValuePrinter(state, *v, errorPrintOptions)
showType(v),
ValuePrinter(state, v, errorPrintOptions)
)
.debugThrow();
}
auto a = v->attrs->get(state.ctx.symbols.create(attr));
auto a = v.attrs()->get(state.ctx.symbols.create(attr));
if (!a) {
std::set<std::string> attrNames;
for (auto & attr : *v->attrs)
for (auto & attr : *v.attrs()) {
attrNames.emplace(state.ctx.symbols[attr.name]);
}
auto suggestions = Suggestions::bestMatches(attrNames, attr);
auto pathPart =
@@ -127,33 +127,33 @@ std::pair<Value *, PosIdx> findAlongAttrPath(EvalState & state, const std::strin
attr,
attrPath,
unparseAttrPath(pathPart),
ValuePrinter(state, *v, errorPrintOptions)
ValuePrinter(state, v, errorPrintOptions)
);
}
v = &*a->value;
v = a->value;
pos = a->pos;
} else {
if (!v->isList()) {
if (!v.isList()) {
state.ctx.errors
.make<TypeError>(
"the expression selected by the selection path '%1%' should be a list but "
"is %2%: %3%",
attrPath,
showType(*v),
ValuePrinter(state, *v, errorPrintOptions)
showType(v),
ValuePrinter(state, v, errorPrintOptions)
)
.debugThrow();
}
if (*attrIndex >= v->listSize()) {
if (*attrIndex >= v.listSize()) {
throw AttrPathNotFound(
"list index %1% in selection path '%2%' is out of range for list %3%",
*attrIndex,
attrPath,
ValuePrinter(state, *v, errorPrintOptions)
ValuePrinter(state, v, errorPrintOptions)
);
}
v = v->listElems()[*attrIndex];
v = v.listElems()[*attrIndex];
pos = noPos;
}
@@ -165,7 +165,7 @@ std::pair<Value *, PosIdx> findAlongAttrPath(EvalState & state, const std::strin
std::pair<SourcePath, uint32_t> findPackageFilename(EvalState & state, Value & v, std::string what)
{
Value * v2;
Value v2;
try {
auto dummyArgs = state.ctx.mem.allocBindings(0);
v2 = findAlongAttrPath(state, "meta.position", *dummyArgs, v).first;
@@ -176,7 +176,9 @@ std::pair<SourcePath, uint32_t> findPackageFilename(EvalState & state, Value & v
// FIXME: is it possible to extract the Pos object instead of doing this
// toString + parsing?
NixStringContext context;
auto path = state.coerceToPath(noPos, *v2, context, "while evaluating the 'meta.position' attribute of a derivation");
auto path = state.coerceToPath(
noPos, v2, context, "while evaluating the 'meta.position' attribute of a derivation"
);
auto fn = path.canonical().abs();
+3 -5
View File
@@ -10,11 +10,9 @@ namespace nix {
MakeError(AttrPathNotFound, Error);
MakeError(NoPositionInfo, Error);
std::pair<Value *, PosIdx> findAlongAttrPath(
EvalState & state,
const std::string & attrPath,
Bindings & autoArgs,
Value & vIn);
std::pair<Value, PosIdx> findAlongAttrPath(
EvalState & state, const std::string & attrPath, Bindings & autoArgs, Value & vIn
);
/**
* Heuristic to find the filename and lineno or a nix value.
+3 -4
View File
@@ -20,15 +20,14 @@ Bindings * EvalMemory::allocBindings(size_t capacity)
throw Error("attribute set of size %d is too big", capacity);
stats.nrAttrsets++;
stats.nrAttrsInAttrsets += capacity;
return new (gcAllocBytes(sizeof(Bindings) + sizeof(Attr) * capacity)) Bindings();
return new (allocBytes(sizeof(Bindings) + sizeof(Attr) * capacity)) Bindings();
}
Value & BindingsBuilder::alloc(Symbol name, PosIdx pos)
{
auto value = mem.allocValue();
bindings->push_back(Attr(name, value, pos));
return *value;
bindings->push_back(Attr(name, {}, pos));
return (bindings->end() - 1)->value;
}
+5 -6
View File
@@ -23,9 +23,8 @@ struct Attr
way we keep Attr size at two words with no wasted space. */
Symbol name;
PosIdx pos;
Value * value;
Attr(Symbol name, Value * value, PosIdx pos = noPos)
: name(name), pos(pos), value(value) { };
mutable Value value;
Attr(Symbol name, Value value, PosIdx pos = noPos) : name(name), pos(pos), value(value) {}
Attr() { };
bool operator < (const Attr & a) const
{
@@ -44,7 +43,7 @@ static_assert(sizeof(Attr) == 2 * sizeof(uint32_t) + sizeof(Value *),
* elements allocated after this structure, while the size corresponds to
* the number of elements already inserted in this structure.
*/
class Bindings
class alignas(Value::TAG_ALIGN) Bindings
{
public:
using Size = uint32_t;
@@ -73,7 +72,7 @@ public:
const Attr * get(Symbol name)
{
Attr key(name, 0);
Attr key(name, {});
iterator i = std::lower_bound(begin(), end(), key);
if (i != end() && i->name == name) return &*i;
return nullptr;
@@ -136,7 +135,7 @@ public:
{
}
void insert(Symbol name, Value * value, PosIdx pos = noPos)
void insert(Symbol name, Value value, PosIdx pos = noPos)
{
insert(Attr(name, value, pos));
}
+22 -16
View File
@@ -341,7 +341,7 @@ EvalCache::EvalCache(
{
}
Value * EvalCache::getRootValue(EvalState & state)
Value & EvalCache::getRootValue(EvalState & state)
{
if (!value) {
debug("getting root value");
@@ -362,8 +362,9 @@ AttrCursor::AttrCursor(
std::optional<std::pair<AttrId, AttrValue>> && cachedValue)
: root(root), parent(parent), cachedValue(std::move(cachedValue))
{
if (value)
_value = allocRootValue(value);
if (value) {
_value = allocRootValue(*value);
}
}
AttrKey AttrCursor::getKey()
@@ -383,14 +384,14 @@ Value & AttrCursor::getValue(EvalState & state)
if (parent) {
auto & vParent = parent->first->getValue(state);
state.forceAttrs(vParent, noPos, "while searching for an attribute");
auto attr = vParent.attrs->get(state.ctx.symbols.create(parent->second));
auto attr = vParent.attrs()->get(state.ctx.symbols.create(parent->second));
if (!attr)
throw Error("attribute '%s' is unexpectedly missing", getAttrPathStr(state));
_value = allocRootValue(attr->value);
} else
_value = allocRootValue(root->getRootValue(state));
}
return **_value;
return *_value;
}
std::vector<std::string> AttrCursor::getAttrPath(EvalState & state) const
@@ -438,16 +439,16 @@ Value & AttrCursor::forceValue(EvalState & state)
if (root->db && (!cachedValue || std::get_if<placeholder_t>(&cachedValue->second))) {
if (v.type() == nString)
cachedValue = {
root->db->setString(getKey(), v.str(), v.string.context), string_t{v.str(), {}}
root->db->setString(getKey(), v.str(), v.string().context), string_t{v.str(), {}}
};
else if (v.type() == nPath) {
auto path = v.path().canonical().abs();
cachedValue = {root->db->setString(getKey(), path), string_t{path, {}}};
}
else if (v.type() == nBool)
cachedValue = {root->db->setBool(getKey(), v.boolean), v.boolean};
cachedValue = {root->db->setBool(getKey(), v.boolean()), v.boolean()};
else if (v.type() == nInt)
cachedValue = {root->db->setInt(getKey(), v.integer.value), int_t{v.integer}};
cachedValue = {root->db->setInt(getKey(), v.integer().value), int_t{v.integer()}};
else if (v.type() == nAttrs)
; // FIXME: do something?
else
@@ -500,7 +501,7 @@ std::shared_ptr<AttrCursor> AttrCursor::maybeGetAttr(EvalState & state, const st
return nullptr;
//errors.make<TypeError>("'%s' is not an attribute set", getAttrPathStr()).debugThrow();
auto attr = v.attrs->get(state.ctx.symbols.create(name));
auto attr = v.attrs()->get(state.ctx.symbols.create(name));
if (!attr) {
if (root->db) {
@@ -519,7 +520,8 @@ std::shared_ptr<AttrCursor> AttrCursor::maybeGetAttr(EvalState & state, const st
}
return make_ref<AttrCursor>(
root, std::make_pair(shared_from_this(), name), attr->value, std::move(cachedValue2));
root, std::make_pair(shared_from_this(), name), &attr->value, std::move(cachedValue2)
);
}
ref<AttrCursor> AttrCursor::getAttr(EvalState & state, const std::string & name)
@@ -633,7 +635,7 @@ bool AttrCursor::getBool(EvalState & state)
if (v.type() != nBool)
state.ctx.errors.make<TypeError>("'%s' is not a Boolean", getAttrPathStr(state)).debugThrow();
return v.boolean;
return v.boolean();
}
NixInt AttrCursor::getInt(EvalState & state)
@@ -655,7 +657,7 @@ NixInt AttrCursor::getInt(EvalState & state)
if (v.type() != nInt)
state.ctx.errors.make<TypeError>("'%s' is not an integer", getAttrPathStr(state)).debugThrow();
return v.integer;
return v.integer();
}
std::vector<std::string> AttrCursor::getListOfStrings(EvalState & state)
@@ -682,11 +684,15 @@ std::vector<std::string> AttrCursor::getListOfStrings(EvalState & state)
std::vector<std::string> res;
for (auto & elem : v.listItems())
res.push_back(std::string(state.forceStringNoCtx(*elem, noPos, "while evaluating an attribute for caching")));
for (auto & elem : v.listItems()) {
res.push_back(std::string(
state.forceStringNoCtx(elem, noPos, "while evaluating an attribute for caching")
));
}
if (root->db)
if (root->db) {
cachedValue = {root->db->setListOfStrings(getKey(), res), res};
}
return res;
}
@@ -711,7 +717,7 @@ std::vector<std::string> AttrCursor::getAttrs(EvalState & state)
state.ctx.errors.make<TypeError>("'%s' is not an attribute set", getAttrPathStr(state)).debugThrow();
fullattr_t attrs;
for (auto & attr : *getValue(state).attrs)
for (auto & attr : *getValue(state).attrs())
attrs.p.emplace_back(state.ctx.symbols[attr.name]);
std::sort(attrs.p.begin(), attrs.p.end());
+2 -2
View File
@@ -12,7 +12,7 @@ namespace nix::eval_cache {
struct AttrDb;
class AttrCursor;
typedef std::function<Value *(EvalState &)> RootLoader;
typedef std::function<Value(EvalState &)> RootLoader;
/**
* EvalState with caching support. Historically this was part of EvalState,
@@ -42,7 +42,7 @@ class EvalCache : public std::enable_shared_from_this<EvalCache>
RootLoader rootLoader;
RootValue value;
Value * getRootValue(EvalState & state);
Value & getRootValue(EvalState & state);
public:
+105 -43
View File
@@ -5,59 +5,105 @@
#include "lix/libexpr/eval.hh"
#include "lix/libexpr/eval-error.hh"
#include "lix/libexpr/gc-alloc.hh"
#include "value.hh"
#include <cstdint>
namespace nix {
inline Value::Value(app_t, EvalMemory & mem, Value & lhs, Value & rhs)
{
auto app = static_cast<Value::App *>(mem.allocBytes(sizeof(Value::App) + sizeof(Value *)));
app->_left = lhs;
app->_n = 1;
app->_args[0] = rhs;
raw = tag(tApp, app);
}
inline Value::Value(app_t, EvalMemory & mem, Value & lhs, std::span<Value> args)
: Value(app_t{}, mem, lhs, args, {})
{
}
inline Value::Value(
app_t, EvalMemory & mem, const Value & lhs, std::span<Value> baseArgs, std::span<Value> moreArgs
)
{
auto app = static_cast<Value::App *>(
mem.allocBytes(sizeof(Value::App) + baseArgs.size_bytes() + moreArgs.size_bytes())
);
app->_left = lhs;
app->_n = baseArgs.size() + moreArgs.size();
std::copy(baseArgs.begin(), baseArgs.end(), app->_args);
std::copy(moreArgs.begin(), moreArgs.end(), app->_args + baseArgs.size());
raw = tag(tApp, app);
}
inline Value::Value(thunk_t, EvalMemory & mem, Env & env, Expr & expr)
{
auto thunk = mem.allocType<Thunk>();
*thunk = {._env = &env, .expr = &expr};
raw = tag(tThunk, thunk);
}
inline Value::Value(lambda_t, EvalMemory & mem, Env & env, ExprLambda & lambda)
{
auto lp = mem.allocType<Lambda>();
new (lp) Lambda{env, lambda};
raw = tag(tAuxiliary, lp);
}
[[gnu::always_inline]]
Value * EvalMemory::allocValue()
void * EvalMemory::allocBytes(size_t size)
{
#if HAVE_BOEHMGC
/* We use the boehm batch allocator to speed up allocations of Values (of which there are many).
GC_malloc_many returns a linked list of objects of the given size, where the first word
of each object is also the pointer to the next object in the list. This also means that we
have to explicitly clear the first word of every object we take. */
if (!*valueAllocCache) {
*valueAllocCache = GC_malloc_many(sizeof(Value));
if (!*valueAllocCache) throw std::bad_alloc();
}
// NOTE: we purposely do not allocate 0 byte blocks on caches; we never allocate
// zero bytes anyway, and it makes cache index calculation a little bit simpler.
const auto cacheIdx = (size - 1) / CACHE_INCREMENT;
if (cacheIdx < CACHES) {
const auto roundedSize = (cacheIdx + 1) * CACHE_INCREMENT;
auto & cache = gcCache[cacheIdx];
if (!cache) {
cache = GC_malloc_many(roundedSize);
if (!cache) {
throw std::bad_alloc();
}
}
/* GC_NEXT is a convenience macro for accessing the first word of an object.
Take the first list item, advance the list to the next item, and clear the next pointer. */
void * p = *valueAllocCache;
*valueAllocCache = GC_NEXT(p);
GC_NEXT(p) = nullptr;
#else
void * p = gcAllocBytes(sizeof(Value));
/* GC_NEXT is a convenience macro for accessing the first word of an object.
Take the first list item, advance the list to the next item, and clear the next pointer.
*/
void * p = cache;
cache = GC_NEXT(p);
GC_NEXT(p) = nullptr;
return p;
}
#endif
stats.nrValues++;
return static_cast<Value *>(p);
return gcAllocBytes(size);
}
/// `gcAllocType`, but using allocation caches to amortize allocation overhead.
template<typename T>
[[gnu::always_inline]]
T * EvalMemory::allocType(size_t n)
{
// NOLINTNEXTLINE(bugprone-sizeof-expression)
return static_cast<T *>(allocBytes(checkedArrayAllocSize(sizeof(T), n)));
}
[[gnu::always_inline]]
Env & EvalMemory::allocEnv(size_t size)
{
static_assert(CACHES * CACHE_INCREMENT >= sizeof(Env) + sizeof(Value *));
stats.nrEnvs++;
stats.nrValuesInEnvs += size;
Env * env;
#if HAVE_BOEHMGC
if (size == 1) {
/* see allocValue for explanations. */
if (!*env1AllocCache) {
*env1AllocCache = GC_malloc_many(sizeof(Env) + sizeof(Value *));
if (!*env1AllocCache) throw std::bad_alloc();
}
void * p = *env1AllocCache;
*env1AllocCache = GC_NEXT(p);
GC_NEXT(p) = nullptr;
env = static_cast<Env *>(p);
} else
#endif
env = static_cast<Env *>(gcAllocBytes(sizeof(Env) + size * sizeof(Value *)));
Env * env = static_cast<Env *>(allocBytes(sizeof(Env) + size * sizeof(Value *)));
/* We assume that env->values has been cleared by the allocator; maybeThunk() and lookupVar fromWith expect this. */
@@ -69,22 +115,38 @@ Env & EvalMemory::allocEnv(size_t size)
void EvalState::forceValue(Value & v, const PosIdx pos)
{
if (v.isThunk()) {
Env * env = v.thunk.env;
Expr & expr = *v.thunk.expr;
try {
v.mkBlackhole();
expr.eval(*this, *env, v);
} catch (...) {
v.mkThunk(env, expr);
tryFixupBlackHolePos(v, pos);
throw;
auto & thunk = v.thunk();
if (thunk.resolved()) {
v = thunk.result();
} else {
const auto backup = thunk;
Env * env = thunk.env();
Expr & expr = *thunk.expr;
thunk = Value::blackHole;
try {
expr.eval(*this, *env, v);
thunk.resolve(v);
} catch (...) {
thunk = backup;
tryFixupBlackHolePos(v, pos);
throw;
}
}
} else if (v.isApp()) {
auto & app = v.app();
if (app.resolved()) {
v = app.result();
} else {
auto target = app.target();
if (!target.isPrimOp() || target.primOp()->arity <= app.totalArgs()) {
auto tmp = v.app().left();
callFunction(tmp, v.app().args(), v, pos);
app.resolve(v);
}
}
}
else if (v.isApp())
callFunction(*v.app.left, *v.app.right, v, pos);
}
[[gnu::always_inline]]
inline void EvalState::forceAttrs(Value & v, const PosIdx pos, std::string_view errorCtx)
{
+1 -1
View File
@@ -82,7 +82,7 @@ bool EvalSettings::isPseudoUrl(std::string_view s)
std::string EvalSettings::resolvePseudoUrl(std::string_view url)
{
if (url.starts_with("channel:"))
return "https://nixos.org/channels/" + std::string(url.substr(8)) + "/nixexprs.tar.xz";
return "https://channels.nixos.org/" + std::string(url.substr(8)) + "/nixexprs.tar.xz";
else
return std::string(url);
}
+494 -370
View File
File diff suppressed because it is too large Load Diff
+20 -67
View File
@@ -36,52 +36,7 @@ namespace eval_cache {
class EvalCache;
}
/**
* Function that implements a primop.
*/
using PrimOpImpl = void(EvalState & state, Value ** args, Value & v);
/**
* Info about a primitive operation, and its implementation
*/
struct PrimOp
{
/**
* Name of the primop. `__` prefix is treated specially.
*/
std::string name;
/**
* Names of the parameters of a primop, for primops that take a
* fixed number of arguments to be substituted for these parameters.
*/
std::vector<std::string> args;
/**
* Aritiy of the primop.
*
* If `args` is not empty, this field will be computed from that
* field instead, so it doesn't need to be manually set.
*/
size_t arity = 0;
/**
* Optional free-form documentation about the primop.
*/
const char * doc = nullptr;
/**
* Implementation of the primop.
*/
std::function<PrimOpImpl> fun;
/**
* Optional experimental for this to be gated on.
*/
std::optional<ExperimentalFeature> experimentalFeature;
};
std::ostream & operator<<(std::ostream & output, PrimOp & primOp);
std::ostream & operator<<(std::ostream & output, const PrimOp & primOp);
/**
* Info about a constant
@@ -106,12 +61,12 @@ struct Constant
bool impureOnly = false;
};
using ValMap = GcMap<std::string, Value *>;
using ValMap = GcMap<std::string, Value>;
struct Env
struct alignas(Value::Acb::TAG_ALIGN) Env
{
Env * up;
Value * values[0];
Value values[0];
};
void printEnvBindings(const EvalState &es, const Expr & expr, const Env & env);
@@ -224,39 +179,40 @@ struct StaticSymbols
class EvalMemory
{
/**
* Allocation cache for GC'd Value objects.
*/
std::shared_ptr<void *> valueAllocCache;
static constexpr size_t CACHES = 8;
static constexpr size_t CACHE_INCREMENT = sizeof(void *);
/**
* Allocation cache for size-1 Env objects.
* Allocation caches for small values.
*/
std::shared_ptr<void *> env1AllocCache;
void * gcCache[CACHES] = {};
public:
struct Statistics
{
unsigned long nrEnvs = 0;
unsigned long nrValuesInEnvs = 0;
unsigned long nrValues = 0;
unsigned long nrAttrsets = 0;
unsigned long nrAttrsInAttrsets = 0;
unsigned long nrListElems = 0;
};
EvalMemory();
~EvalMemory();
EvalMemory(const EvalMemory &) = delete;
EvalMemory(EvalMemory &&) = delete;
EvalMemory & operator=(const EvalMemory &) = delete;
EvalMemory & operator=(EvalMemory &&) = delete;
inline Value * allocValue();
inline void * allocBytes(size_t size);
template<typename T>
inline T * allocType(size_t n = 1);
inline Env & allocEnv(size_t size);
Bindings * allocBindings(size_t capacity);
Value newList(size_t length);
Value::List * newList(size_t length);
BindingsBuilder buildBindings(SymbolTable & symbols, size_t capacity)
{
@@ -307,11 +263,9 @@ private:
void createBaseEnv(const SearchPath & searchPath, const Path & storeDir);
Value * addConstant(const std::string & name, const Value & v, Constant info);
void addConstant(const std::string & name, const Value & v, Constant info);
void addConstant(const std::string & name, Value * v, Constant info);
Value * addPrimOp(PrimOp && primOp);
void addPrimOp(PrimOpDetails primOp);
Value prepareNixPath(const SearchPath & searchPath);
@@ -839,13 +793,11 @@ public:
bool isFunctor(Value & fun);
// FIXME: use std::span
void callFunction(Value & fun, size_t nrArgs, Value * * args, Value & vRes, const PosIdx pos);
void callFunction(Value & fun, std::span<Value> args, Value & vRes, const PosIdx pos);
void callFunction(Value & fun, Value & arg, Value & vRes, const PosIdx pos)
{
Value * args[] = {&arg};
callFunction(fun, 1, args, vRes, pos);
callFunction(fun, {&arg, 1}, vRes, pos);
}
/**
@@ -885,7 +837,8 @@ public:
const SingleDerivedPath & p,
Value & v);
void concatLists(Value & v, size_t nrLists, Value * * lists, const PosIdx pos, std::string_view errorCtx);
void
concatLists(Value & v, std::span<Value> lists, const PosIdx pos, std::string_view errorCtx);
private:
+121 -81
View File
@@ -96,7 +96,7 @@ static void expectType(EvalState & state, ValueType type,
static std::pair<std::map<FlakeId, FlakeInput>, std::optional<fetchers::Attrs>> parseFlakeInputs(
EvalState & state,
Value * value,
Value & value,
const PosIdx pos,
const std::optional<Path> & baseDir,
InputPath lockRootPath,
@@ -109,15 +109,15 @@ static void parseFlakeInputAttr(EvalState & state, const Attr & attr, fetchers::
// Allow selecting a subset of enum values
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wswitch-enum"
switch (attr.value->type()) {
switch (attr.value.type()) {
case nString:
attrs.emplace(state.ctx.symbols[attr.name], std::string(attr.value->str()));
attrs.emplace(state.ctx.symbols[attr.name], std::string(attr.value.str()));
break;
case nBool:
attrs.emplace(state.ctx.symbols[attr.name], Explicit<bool>{attr.value->boolean});
attrs.emplace(state.ctx.symbols[attr.name], Explicit<bool>{attr.value.boolean()});
break;
case nInt: {
auto intValue = attr.value->integer.value;
auto intValue = attr.value.integer().value;
if (intValue < 0) {
state.ctx.errors
@@ -137,18 +137,24 @@ static void parseFlakeInputAttr(EvalState & state, const Attr & attr, fetchers::
.make<TypeError>(
"flake input attribute '%s' is %s while a string, Boolean, or integer is expected",
state.ctx.symbols[attr.name],
showType(*attr.value)
showType(attr.value)
)
.debugThrow();
}
#pragma GCC diagnostic pop
}
static FlakeInput parseFlakeInput(EvalState & state,
const std::string & inputName, Value * value, const PosIdx pos,
const std::optional<Path> & baseDir, InputPath lockRootPath, unsigned depth)
static FlakeInput parseFlakeInput(
EvalState & state,
const std::string & inputName,
Value & value,
const PosIdx pos,
const std::optional<Path> & baseDir,
InputPath lockRootPath,
unsigned depth
)
{
expectType(state, nAttrs, *value, pos);
expectType(state, nAttrs, value, pos);
FlakeInput input;
@@ -160,15 +166,15 @@ static FlakeInput parseFlakeInput(EvalState & state,
fetchers::Attrs attrs;
std::optional<std::string> url;
for (nix::Attr attr : *(value->attrs)) {
for (nix::Attr attr : *(value.attrs())) {
try {
if (attr.name == sUrl) {
expectType(state, nString, *attr.value, attr.pos);
url = attr.value->str();
expectType(state, nString, attr.value, attr.pos);
url = attr.value.str();
attrs.emplace("url", *url);
} else if (attr.name == sFlake) {
expectType(state, nBool, *attr.value, attr.pos);
input.isFlake = attr.value->boolean;
expectType(state, nBool, attr.value, attr.pos);
input.isFlake = attr.value.boolean();
} else if (attr.name == sInputs) {
input.overrides =
parseFlakeInputs(
@@ -176,8 +182,8 @@ static FlakeInput parseFlakeInput(EvalState & state,
)
.first;
} else if (attr.name == sFollows) {
expectType(state, nString, *attr.value, attr.pos);
auto follows(parseInputPath(attr.value->str()));
expectType(state, nString, attr.value, attr.pos);
auto follows(parseInputPath(attr.value.str()));
follows.insert(follows.begin(), lockRootPath.begin(), lockRootPath.end());
input.follows = follows;
} else {
@@ -218,7 +224,7 @@ static FlakeInput parseFlakeInput(EvalState & state,
static std::pair<std::map<FlakeId, FlakeInput>, std::optional<fetchers::Attrs>> parseFlakeInputs(
EvalState & state,
Value * value,
Value & value,
const PosIdx pos,
const std::optional<Path> & baseDir,
InputPath lockRootPath,
@@ -228,10 +234,10 @@ static std::pair<std::map<FlakeId, FlakeInput>, std::optional<fetchers::Attrs>>
{
std::map<FlakeId, FlakeInput> inputs;
expectType(state, nAttrs, *value, pos);
expectType(state, nAttrs, value, pos);
std::optional<fetchers::Attrs> selfAttrs = std::nullopt;
for (const nix::Attr & inputAttr : *(*value).attrs) {
for (const nix::Attr & inputAttr : *value.attrs()) {
std::string inputName{state.ctx.symbols[inputAttr.name]};
if (inputName == "self") {
experimentalFeatureSettings.require(Xp::FlakeSelfAttrs);
@@ -241,10 +247,10 @@ static std::pair<std::map<FlakeId, FlakeInput>, std::optional<fetchers::Attrs>>
"'self' input attributes not allowed at %s", state.ctx.positions[inputAttr.pos]
);
}
expectType(state, nAttrs, *inputAttr.value, inputAttr.pos);
expectType(state, nAttrs, inputAttr.value, inputAttr.pos);
selfAttrs = selfAttrs.value_or(fetchers::Attrs{});
for (auto & attr : *inputAttr.value->attrs) {
for (auto & attr : *inputAttr.value.attrs()) {
parseFlakeInputAttr(state, attr, *selfAttrs);
}
} else {
@@ -328,14 +334,14 @@ static Flake getFlake(
Value vInfo;
state.eval(flakeExpr, vInfo);
if (auto description = vInfo.attrs->get(state.ctx.s.description)) {
expectType(state, nString, *description->value, description->pos);
flake.description = description->value->str();
if (auto description = vInfo.attrs()->get(state.ctx.s.description)) {
expectType(state, nString, description->value, description->pos);
flake.description = description->value.str();
}
auto sInputs = state.ctx.symbols.create("inputs");
if (auto inputs = vInfo.attrs->get(sInputs)) {
if (auto inputs = vInfo.attrs()->get(sInputs)) {
auto [flakeInputs, selfAttrs] =
parseFlakeInputs(state, inputs->value, inputs->pos, flakeDir, lockRootPath, 0, true);
flake.inputs = std::move(flakeInputs);
@@ -361,11 +367,14 @@ static Flake getFlake(
flake.resolvedRef = resolvedRef;
}
if (auto outputs = vInfo.attrs->get(state.ctx.s.outputs)) {
expectType(state, nFunction, *outputs->value, outputs->pos);
if (auto outputs = vInfo.attrs()->get(state.ctx.s.outputs)) {
expectType(state, nFunction, outputs->value, outputs->pos);
if (outputs->value->isLambda()) {
if (auto pattern = dynamic_cast<AttrsPattern *>(outputs->value->lambda.fun->pattern.get()); pattern) {
if (outputs->value.isLambda()) {
if (auto pattern =
dynamic_cast<AttrsPattern *>(outputs->value.lambda().fun->pattern.get());
pattern)
{
for (auto & formal : pattern->formals) {
if (formal.name != state.ctx.s.self)
flake.inputs.emplace(
@@ -383,46 +392,71 @@ static Flake getFlake(
auto sNixConfig = state.ctx.symbols.create("nixConfig");
if (auto nixConfig = vInfo.attrs->get(sNixConfig)) {
expectType(state, nAttrs, *nixConfig->value, nixConfig->pos);
if (auto nixConfig = vInfo.attrs()->get(sNixConfig)) {
expectType(state, nAttrs, nixConfig->value, nixConfig->pos);
for (auto & setting : *nixConfig->value->attrs) {
forceTrivialValue(state, *setting.value, setting.pos);
if (setting.value->type() == nString)
for (auto & setting : *nixConfig->value.attrs()) {
forceTrivialValue(state, setting.value, setting.pos);
if (setting.value.type() == nString) {
flake.config.settings.emplace(
state.ctx.symbols[setting.name],
std::string(state.forceStringNoCtx(*setting.value, setting.pos, "")));
else if (setting.value->type() == nPath) {
std::string(state.forceStringNoCtx(setting.value, setting.pos, ""))
);
} else if (setting.value.type() == nPath) {
NixStringContext emptyContext = {};
flake.config.settings.emplace(
state.ctx.symbols[setting.name],
state.coerceToString(setting.pos, *setting.value, emptyContext, "", StringCoercionMode::Strict, true, true) .toOwned());
}
else if (setting.value->type() == nInt)
state
.coerceToString(
setting.pos,
setting.value,
emptyContext,
"",
StringCoercionMode::Strict,
true,
true
)
.toOwned()
);
} else if (setting.value.type() == nInt) {
flake.config.settings.emplace(
state.ctx.symbols[setting.name],
state.forceInt(*setting.value, setting.pos, "").value);
else if (setting.value->type() == nBool)
state.forceInt(setting.value, setting.pos, "").value
);
} else if (setting.value.type() == nBool) {
flake.config.settings.emplace(
state.ctx.symbols[setting.name],
Explicit<bool> { state.forceBool(*setting.value, setting.pos, "") });
else if (setting.value->type() == nList) {
Explicit<bool>{state.forceBool(setting.value, setting.pos, "")}
);
} else if (setting.value.type() == nList) {
std::vector<std::string> ss;
for (auto elem : setting.value->listItems()) {
if (elem->type() != nString)
state.ctx.errors.make<TypeError>("list element in flake configuration setting '%s' is %s while a string is expected",
state.ctx.symbols[setting.name], showType(*setting.value)).debugThrow();
ss.emplace_back(state.forceStringNoCtx(*elem, setting.pos, ""));
for (auto & elem : setting.value.listItems()) {
if (elem.type() != nString) {
state.ctx.errors
.make<TypeError>(
"list element in flake configuration setting '%s' is %s while a "
"string is expected",
state.ctx.symbols[setting.name],
showType(setting.value)
)
.debugThrow();
}
ss.emplace_back(state.forceStringNoCtx(elem, setting.pos, ""));
}
flake.config.settings.emplace(state.ctx.symbols[setting.name], ss);
} else {
state.ctx.errors
.make<TypeError>(
"flake configuration setting '%s' is %s",
state.ctx.symbols[setting.name],
showType(setting.value)
)
.debugThrow();
}
else
state.ctx.errors.make<TypeError>("flake configuration setting '%s' is %s",
state.ctx.symbols[setting.name], showType(*setting.value)).debugThrow();
}
}
for (auto & attr : *vInfo.attrs) {
for (auto & attr : *vInfo.attrs()) {
if (attr.name != state.ctx.s.description &&
attr.name != sInputs &&
attr.name != state.ctx.s.outputs &&
@@ -907,34 +941,39 @@ void callFlake(EvalState & state,
const LockedFlake & lockedFlake,
Value & vRes)
{
auto vLocks = state.ctx.mem.allocValue();
auto vRootSrc = state.ctx.mem.allocValue();
auto vRootSubdir = state.ctx.mem.allocValue();
auto vTmp1 = state.ctx.mem.allocValue();
auto vTmp2 = state.ctx.mem.allocValue();
Value vLocks;
Value vRootSrc;
Value vRootSubdir;
Value vTmp1;
Value vTmp2;
vLocks->mkString(lockedFlake.lockFile.to_string());
vLocks.mkString(lockedFlake.lockFile.to_string());
emitTreeAttrs(
state.ctx,
*lockedFlake.flake.sourceInfo,
lockedFlake.flake.lockedRef.input,
*vRootSrc,
vRootSrc,
false,
lockedFlake.flake.forceDirty);
lockedFlake.flake.forceDirty
);
vRootSubdir->mkString(lockedFlake.flake.lockedRef.subdir);
vRootSubdir.mkString(lockedFlake.flake.lockedRef.subdir);
if (!state.ctx.caches.vCallFlake) {
state.ctx.caches.vCallFlake = allocRootValue(state.ctx.mem.allocValue());
state.eval(state.ctx.parseExprFromString(
#include "call-flake.nix.gen.hh"
, CanonPath::root), **state.ctx.caches.vCallFlake);
state.ctx.caches.vCallFlake = allocRootValue({});
state.eval(
state.ctx.parseExprFromString(
#include "call-flake.nix.gen.hh"
, CanonPath::root
),
*state.ctx.caches.vCallFlake
);
}
state.callFunction(**state.ctx.caches.vCallFlake, *vLocks, *vTmp1, noPos);
state.callFunction(*vTmp1, *vRootSrc, *vTmp2, noPos);
state.callFunction(*vTmp2, *vRootSubdir, vRes, noPos);
state.callFunction(*state.ctx.caches.vCallFlake, vLocks, vTmp1, noPos);
state.callFunction(vTmp1, vRootSrc, vTmp2, noPos);
state.callFunction(vTmp2, vRootSubdir, vRes, noPos);
}
void prim_getFlake(EvalState & state, Value * * args, Value & v)
@@ -984,10 +1023,10 @@ void prim_flakeRefToString(
state.forceAttrs(*args[0], noPos,
"while evaluating the argument passed to builtins.flakeRefToString");
fetchers::Attrs attrs;
for (const auto & attr : *args[0]->attrs) {
auto t = attr.value->type();
for (const auto & attr : *args[0]->attrs()) {
auto t = attr.value.type();
if (t == nInt) {
auto intValue = attr.value->integer.value;
auto intValue = attr.value.integer().value;
if (intValue < 0) {
state.ctx.errors.make<EvalError>("negative value given for flake ref attr %1%: %2%", state.ctx.symbols[attr.name], intValue).debugThrow();
@@ -996,17 +1035,18 @@ void prim_flakeRefToString(
attrs.emplace(state.ctx.symbols[attr.name], asUnsigned);
} else if (t == nBool) {
attrs.emplace(state.ctx.symbols[attr.name],
Explicit<bool> { attr.value->boolean });
attrs.emplace(state.ctx.symbols[attr.name], Explicit<bool>{attr.value.boolean()});
} else if (t == nString) {
attrs.emplace(state.ctx.symbols[attr.name],
std::string(attr.value->str()));
attrs.emplace(state.ctx.symbols[attr.name], std::string(attr.value.str()));
} else {
state.ctx.errors.make<EvalError>(
"flake reference attribute sets may only contain integers, Booleans, "
"and strings, but attribute '%s' is %s",
state.ctx.symbols[attr.name],
showType(*attr.value)).debugThrow();
state.ctx.errors
.make<EvalError>(
"flake reference attribute sets may only contain integers, Booleans, "
"and strings, but attribute '%s' is %s",
state.ctx.symbols[attr.name],
showType(attr.value)
)
.debugThrow();
}
}
auto flakeRef = FlakeRef::fromAttrs(attrs);
+19 -15
View File
@@ -95,6 +95,24 @@ inline void * gcAllocBytes(size_t n)
return ptr;
}
[[gnu::always_inline]]
inline size_t checkedArrayAllocSize(size_t size, size_t howMany)
{
// NOTE: size_t * size_t, which can definitely overflow.
// Unsigned integer overflow is definitely a bug, but isn't undefined
// behavior, so we can just check if we overflowed after the fact.
// However, people can and do request zero sized allocations, so we need
// to check that neither of our multiplicands were zero before complaining
// about it.
auto checkedSz = checked::Checked<size_t>(howMany) * size;
if (checkedSz.overflowed()) {
// Congrats, you done did an overflow.
throw std::bad_alloc();
}
return checkedSz.valueWrapping();
}
/// Typed, safe wrapper around calloc() (transparently GC-enabled). Allocates
/// enough for the requested count of the specified type. Also checks for
/// nullptr (and throws @ref std::bad_alloc), and casts the void pointer to
@@ -103,21 +121,7 @@ template<typename T>
[[gnu::always_inline]]
inline T * gcAllocType(size_t howMany = 1)
{
// NOTE: size_t * size_t, which can definitely overflow.
// Unsigned integer overflow is definitely a bug, but isn't undefined
// behavior, so we can just check if we overflowed after the fact.
// However, people can and do request zero sized allocations, so we need
// to check that neither of our multiplicands were zero before complaining
// about it.
// NOLINTNEXTLINE(bugprone-sizeof-expression): yeah we only seem to alloc pointers with this. the calculation *is* correct though!
auto checkedSz = checked::Checked<size_t>(howMany) * sizeof(T);
size_t sz = checkedSz.valueWrapping();
if (checkedSz.overflowed()) {
// Congrats, you done did an overflow.
throw std::bad_alloc();
}
return static_cast<T *>(gcAllocBytes(sz));
return static_cast<T *>(gcAllocBytes(checkedArrayAllocSize(sizeof(T), howMany)));
}
/// GC-transparently allocates a buffer for a C-string of @ref size *bytes*,
+2 -2
View File
@@ -17,8 +17,8 @@ using SmallVector = boost::container::small_vector<T, nItems, TraceableAllocator
/**
* A vector of value pointers. See `SmallVector`.
*/
template <size_t nItems>
using SmallValueVector = SmallVector<Value *, nItems>;
template<size_t nItems>
using SmallValueVector = SmallVector<Value, nItems>;
/**
* A vector of values that must not be referenced after the vector is destroyed.
+58 -62
View File
@@ -68,7 +68,9 @@ std::string DrvInfo::queryName(EvalState & state)
if (!i) {
state.ctx.errors.make<TypeError>("derivation name missing").debugThrow();
}
name = state.forceStringNoCtx(*i->value, noPos, "while evaluating the 'name' attribute of a derivation");
name = state.forceStringNoCtx(
i->value, noPos, "while evaluating the 'name' attribute of a derivation"
);
}
return name;
}
@@ -81,7 +83,7 @@ std::string DrvInfo::querySystem(EvalState & state)
system = !i
? "unknown"
: state.forceStringNoCtx(
*i->value, i->pos, "while evaluating the 'system' attribute of a derivation"
i->value, i->pos, "while evaluating the 'system' attribute of a derivation"
);
}
return system;
@@ -98,7 +100,7 @@ std::optional<StorePath> DrvInfo::queryDrvPath(EvalState & state)
} else {
drvPath = {state.coerceToStorePath(
i->pos,
*i->value,
i->value,
context,
"while evaluating the 'drvPath' attribute of a derivation"
)};
@@ -123,7 +125,7 @@ StorePath DrvInfo::queryOutPath(EvalState & state)
NixStringContext context;
if (i) {
outPath = state.coerceToStorePath(
i->pos, *i->value, context, "while evaluating the output path of a derivation"
i->pos, i->value, context, "while evaluating the output path of a derivation"
);
}
}
@@ -156,23 +158,17 @@ void DrvInfo::fillOutputs(EvalState & state, bool withPaths)
// NOTE(Qyriad): I don't think there is any codepath that can cause this to error.
state.forceList(
*outputs->value,
outputs->pos,
"while evaluating the 'outputs' attribute of a derivation"
outputs->value, outputs->pos, "while evaluating the 'outputs' attribute of a derivation"
);
for (auto [idx, elem] : enumerate(outputs->value->listItems())) {
for (auto && [idx, elem] : enumerate(outputs->value.listItems())) {
// NOTE(Qyriad): This error should be *extremely* rare in practice.
// It is impossible to construct with `stdenv.mkDerivation`,
// `builtins.derivation`, or even `derivationStrict`. As far as we can tell,
// it is only possible by overriding a derivation attrset already created by
// one of those with `//` to introduce the failing `outputs` entry.
auto errMsg = fmt("while evaluating output %d of a derivation", idx);
std::string_view outputName = state.forceStringNoCtx(
*elem,
outputs->pos,
errMsg
);
std::string_view outputName = state.forceStringNoCtx(elem, outputs->pos, errMsg);
if (withPaths) {
// Find the attr with this output's name...
@@ -184,10 +180,10 @@ void DrvInfo::fillOutputs(EvalState & state, bool withPaths)
// Meanwhile we couldn't figure out any circumstances
// that cause this to error.
state.forceAttrs(*out->value, outputs->pos, errMsg);
state.forceAttrs(out->value, outputs->pos, errMsg);
// ...and evaluate its `outPath` attribute.
const Attr * outPath = out->value->attrs->get(state.ctx.s.outPath);
const Attr * outPath = out->value.attrs()->get(state.ctx.s.outPath);
if (outPath == nullptr) {
continue;
// FIXME: throw error?
@@ -196,12 +192,7 @@ void DrvInfo::fillOutputs(EvalState & state, bool withPaths)
NixStringContext context;
// And idk what could possibly cause this one to error
// that wouldn't error before here.
auto storePath = state.coerceToStorePath(
outPath->pos,
*outPath->value,
context,
errMsg
);
auto storePath = state.coerceToStorePath(outPath->pos, outPath->value, context, errMsg);
this->outputs.emplace(outputName, storePath);
} else {
this->outputs.emplace(outputName, std::nullopt);
@@ -233,7 +224,7 @@ DrvInfo::Outputs DrvInfo::queryOutputs(EvalState & state, bool withPaths, bool o
// explicitly selected-into output.
if (const Attr * outSpecAttr = attrs->get(state.ctx.s.outputSpecified)) {
bool outputSpecified = state.forceBool(
*outSpecAttr->value,
outSpecAttr->value,
outSpecAttr->pos,
"while evaluating the 'outputSpecified' attribute of a derivation"
);
@@ -253,15 +244,16 @@ DrvInfo::Outputs DrvInfo::queryOutputs(EvalState & state, bool withPaths, bool o
/* ^ this shows during `nix-env -i` right under the bad derivation */
if (!outTI->isList()) throw Error(errMsg + "expected a list but got %s", Uncolored(showType(outTI->type())));
Outputs result;
for (auto elem : outTI->listItems()) {
if (elem->type() != nString)
for (auto & elem : outTI->listItems()) {
if (elem.type() != nString) {
throw Error(
errMsg + "element is %s where a string was expected",
Uncolored(showType(elem->type()))
Uncolored(showType(elem.type()))
);
auto out = outputs.find(std::string(elem->str()));
}
auto out = outputs.find(std::string(elem.str()));
if (out == outputs.end()) {
throw Error(errMsg + "output '%s' does not exist", elem->str());
throw Error(errMsg + "output '%s' does not exist", elem.str());
}
result.insert(*out);
}
@@ -274,7 +266,7 @@ std::string DrvInfo::queryOutputName(EvalState & state)
if (outputName == "" && attrs) {
auto i = attrs->get(state.ctx.s.outputName);
outputName = i ? state.forceStringNoCtx(
*i->value, noPos, "while evaluating the output name of a derivation"
i->value, noPos, "while evaluating the output name of a derivation"
)
: "";
}
@@ -290,8 +282,8 @@ Bindings * DrvInfo::getMeta(EvalState & state)
if (!a) {
return 0;
}
state.forceAttrs(*a->value, a->pos, "while evaluating the 'meta' attribute of a derivation");
meta = a->value->attrs;
state.forceAttrs(a->value, a->pos, "while evaluating the 'meta' attribute of a derivation");
meta = a->value.attrs();
return meta;
}
@@ -310,17 +302,23 @@ bool DrvInfo::checkMeta(EvalState & state, Value & v)
{
state.forceValue(v, noPos);
if (v.type() == nList) {
for (auto elem : v.listItems())
if (!checkMeta(state, *elem)) return false;
for (auto & elem : v.listItems()) {
if (!checkMeta(state, elem)) {
return false;
}
}
return true;
}
else if (v.type() == nAttrs) {
auto i = v.attrs->get(state.ctx.s.outPath);
auto i = v.attrs()->get(state.ctx.s.outPath);
if (i) {
return false;
}
for (auto & i : *v.attrs)
if (!checkMeta(state, *i.value)) return false;
for (auto & i : *v.attrs()) {
if (!checkMeta(state, i.value)) {
return false;
}
}
return true;
}
else return v.type() == nInt || v.type() == nBool || v.type() == nString ||
@@ -332,10 +330,10 @@ Value * DrvInfo::queryMeta(EvalState & state, const std::string & name)
{
if (!getMeta(state)) return 0;
auto a = meta->get(state.ctx.symbols.create(name));
if (!a || !checkMeta(state, *a->value)) {
if (!a || !checkMeta(state, a->value)) {
return 0;
}
return a->value;
return &a->value;
}
@@ -351,7 +349,9 @@ NixInt DrvInfo::queryMetaInt(EvalState & state, const std::string & name, NixInt
{
Value * v = queryMeta(state, name);
if (!v) return def;
if (v->type() == nInt) return v->integer;
if (v->type() == nInt) {
return v->integer();
}
if (v->type() == nString) {
/* Backwards compatibility with before we had support for
integer meta fields. */
@@ -366,7 +366,9 @@ bool DrvInfo::queryMetaBool(EvalState & state, const std::string & name, bool de
{
Value * v = queryMeta(state, name);
if (!v) return def;
if (v->type() == nBool) return v->boolean;
if (v->type() == nBool) {
return v->boolean();
}
if (v->type() == nString) {
/* Backwards compatibility with before we had support for
Boolean meta fields. */
@@ -380,8 +382,7 @@ bool DrvInfo::queryMetaBool(EvalState & state, const std::string & name, bool de
return def;
}
void DrvInfo::setMeta(EvalState & state, const std::string & name, Value * v)
void DrvInfo::setMeta(EvalState & state, const std::string & name, Value & v)
{
getMeta(state);
auto attrs = state.ctx.buildBindings(1 + (meta ? meta->size() : 0));
@@ -390,7 +391,7 @@ void DrvInfo::setMeta(EvalState & state, const std::string & name, Value * v)
for (auto i : *meta)
if (i.name != sym)
attrs.insert(i);
if (v) attrs.insert(sym, v);
attrs.insert(sym, v);
meta = attrs.finish();
}
@@ -409,7 +410,7 @@ static bool getDerivation(EvalState & state, Value & v,
state.forceValue(v, noPos);
if (!state.isDerivation(v)) return true;
DrvInfo drv(attrPath, v.attrs);
DrvInfo drv(attrPath, v.attrs());
drv.queryName(state);
@@ -462,19 +463,13 @@ static void getDerivations(EvalState & state, Value & vIn, PosIdx pos,
if (v.type() == nList) {
// NOTE we can't really deduplicate here because small lists don't have stable addresses
// and can cause spurious duplicate detections due to v being on the stack.
for (auto [n, elem] : enumerate(v.listItems())) {
for (auto && [n, elem] : enumerate(v.listItems())) {
std::string joinedAttrPath = addToPath(pathPrefix, fmt("%d", n));
bool shouldRecurse = getDerivation(state, *elem, joinedAttrPath, drvs, ignoreAssertionFailures);
bool shouldRecurse =
getDerivation(state, elem, joinedAttrPath, drvs, ignoreAssertionFailures);
if (shouldRecurse) {
getDerivations(
state,
*elem,
pos,
joinedAttrPath,
autoArgs,
drvs,
done,
ignoreAssertionFailures
state, elem, pos, joinedAttrPath, autoArgs, drvs, done, ignoreAssertionFailures
);
}
}
@@ -489,7 +484,7 @@ static void getDerivations(EvalState & state, Value & vIn, PosIdx pos,
/* Dont consider sets we've already seen, e.g. y in
`rec { x.d = derivation {...}; y = x; }`. */
auto const &[_, didInsert] = done.insert(v.attrs);
auto const &[_, didInsert] = done.insert(v.attrs());
if (!didInsert) {
return;
}
@@ -497,14 +492,14 @@ static void getDerivations(EvalState & state, Value & vIn, PosIdx pos,
// FIXME: what the fuck???
/* !!! undocumented hackery to support combining channels in
nix-env.cc. */
bool combineChannels = v.attrs->get(state.ctx.symbols.create("_combineChannels"));
bool combineChannels = v.attrs()->get(state.ctx.symbols.create("_combineChannels"));
/* Consider the attributes in sorted order to get more
deterministic behaviour in nix-env operations (e.g. when
there are names clashes between derivations, the derivation
bound to the attribute with the "lower" name should take
precedence). */
for (auto & attr : v.attrs->lexicographicOrder(state.ctx.symbols)) {
for (auto & attr : v.attrs()->lexicographicOrder(state.ctx.symbols)) {
debug("evaluating attribute '%1%'", state.ctx.symbols[attr->name]);
// FIXME: only consider attrs with identifier-like names?? Why???
if (!std::regex_match(std::string(state.ctx.symbols[attr->name]), attrRegex)) {
@@ -514,7 +509,7 @@ static void getDerivations(EvalState & state, Value & vIn, PosIdx pos,
if (combineChannels) {
getDerivations(
state,
*attr->value,
attr->value,
attr->pos,
joinedAttrPath,
autoArgs,
@@ -522,18 +517,19 @@ static void getDerivations(EvalState & state, Value & vIn, PosIdx pos,
done,
ignoreAssertionFailures
);
} else if (getDerivation(state, *attr->value, joinedAttrPath, drvs, ignoreAssertionFailures)) {
} else if (getDerivation(state, attr->value, joinedAttrPath, drvs, ignoreAssertionFailures))
{
/* If the value of this attribute is itself a set,
should we recurse into it? => Only if it has a
`recurseForDerivations = true' attribute. */
if (attr->value->type() == nAttrs) {
if (attr->value.type() == nAttrs) {
const Attr * recurseForDrvs =
attr->value->attrs->get(state.ctx.s.recurseForDerivations);
attr->value.attrs()->get(state.ctx.s.recurseForDerivations);
if (recurseForDrvs == nullptr) {
continue;
}
bool shouldRecurse = state.forceBool(
*recurseForDrvs->value,
recurseForDrvs->value,
attr->pos,
fmt("while evaluating the '%s' attribute", Magenta("recurseForDerivations"))
);
@@ -543,7 +539,7 @@ static void getDerivations(EvalState & state, Value & vIn, PosIdx pos,
getDerivations(
state,
*attr->value,
attr->value,
attr->pos,
joinedAttrPath,
autoArgs,
+1 -1
View File
@@ -73,7 +73,7 @@ public:
std::string queryMetaString(EvalState & state, const std::string & name);
NixInt queryMetaInt(EvalState & state, const std::string & name, NixInt def);
bool queryMetaBool(EvalState & state, const std::string & name, bool def);
void setMeta(EvalState & state, const std::string & name, Value * v);
void setMeta(EvalState & state, const std::string & name, Value & v);
/*
MetaInfo queryMetaInfo(EvalState & state) const;
+36 -27
View File
@@ -1,4 +1,5 @@
#include "lix/libexpr/json-to-value.hh"
#include "gc-alloc.hh"
#include "lix/libexpr/value.hh"
#include "lix/libexpr/eval.hh"
#include "lix/libutil/json.hh"
@@ -7,11 +8,6 @@
namespace nix {
/*
* Used for `JSONObjectState`
*/
using ValueMap = GcMap<Symbol, Value *>;
// for more information, refer to
// https://github.com/nlohmann/json/blob/master/include/nlohmann/detail/input/json_sax.hpp
class JSONSax : nlohmann::json_sax<JSON> {
@@ -25,13 +21,14 @@ class JSONSax : nlohmann::json_sax<JSON> {
assert(false && "tried to close toplevel json parser state");
}
explicit JSONState(std::unique_ptr<JSONState> && p) : parent(std::move(p)) {}
explicit JSONState(Value * v) : v(allocRootValue(v)) {}
JSONState() = default;
JSONState(JSONState & p) = delete;
Value & value(EvalState & state)
Value & value()
{
if (!v)
v = allocRootValue(state.ctx.mem.allocValue());
return **v;
if (!v) {
v = allocRootValue({});
}
return *v;
}
virtual ~JSONState() {}
virtual void add() {}
@@ -39,35 +36,41 @@ class JSONSax : nlohmann::json_sax<JSON> {
class JSONObjectState : public JSONState {
using JSONState::JSONState;
ValueMap attrs;
GcMap<Symbol, Value> attrs;
Symbol _key;
std::unique_ptr<JSONState> resolve(EvalState & state) override
{
auto attrs2 = state.ctx.buildBindings(attrs.size());
for (auto & i : attrs)
attrs2.insert(i.first, i.second);
parent->value(state).mkAttrs(attrs2.alreadySorted());
parent->value().mkAttrs(attrs2.alreadySorted());
return std::move(parent);
}
void add() override { v = nullptr; }
void add() override
{
attrs.insert_or_assign(_key, value());
v = nullptr;
}
public:
void key(string_t & name, EvalState & state)
{
attrs.insert_or_assign(state.ctx.symbols.create(name), &value(state));
_key = state.ctx.symbols.create(name);
}
};
class JSONListState : public JSONState {
ValueVector values;
GcVector<Value> values;
std::unique_ptr<JSONState> resolve(EvalState & state) override
{
Value & v = parent->value(state);
v = state.ctx.mem.newList(values.size());
auto list = state.ctx.mem.newList(values.size());
parent->value() = {NewValueAs::list, list};
for (size_t n = 0; n < values.size(); ++n) {
v.listElems()[n] = values[n];
list->elems[n] = values[n];
}
return std::move(parent);
}
void add() override {
void add() override
{
values.push_back(*v);
v = nullptr;
}
@@ -82,25 +85,30 @@ class JSONSax : nlohmann::json_sax<JSON> {
std::unique_ptr<JSONState> rs;
public:
JSONSax(EvalState & state, Value & v) : state(state), rs(new JSONState(&v)) {};
JSONSax(EvalState & state) : state(state), rs(new JSONState()) {};
Value result()
{
return rs->value();
}
bool null() override
{
rs->value(state).mkNull();
rs->value().mkNull();
rs->add();
return true;
}
bool boolean(bool val) override
{
rs->value(state).mkBool(val);
rs->value().mkBool(val);
rs->add();
return true;
}
bool number_integer(number_integer_t val) override
{
rs->value(state).mkInt(val);
rs->value().mkInt(val);
rs->add();
return true;
}
@@ -113,21 +121,21 @@ public:
return number_float(static_cast<number_float_t>(val_), "");
}
NixInt::Inner val = val_;
rs->value(state).mkInt(val);
rs->value().mkInt(val);
rs->add();
return true;
}
bool number_float(number_float_t val, const string_t & s) override
{
rs->value(state).mkFloat(val);
rs->value().mkFloat(val);
rs->add();
return true;
}
bool string(string_t & val) override
{
rs->value(state).mkString(val);
rs->value().mkString(val);
rs->add();
return true;
}
@@ -178,10 +186,11 @@ public:
void parseJSON(EvalState & state, const std::string_view & s_, Value & v)
{
JSONSax parser(state, v);
JSONSax parser(state);
bool res = JSON::sax_parse(s_, &parser);
if (!res)
throw JSONParseError("Invalid JSON Value");
v = parser.result();
}
}
+5 -3
View File
@@ -9,7 +9,9 @@
namespace nix {
ExprBlackHole eBlackHole;
Expr *eBlackHoleAddr = &eBlackHole;
static Env nullEnv;
Value::Thunk Value::blackHole{{&nullEnv}, &eBlackHole};
// FIXME: remove, because *symbols* are abstract and do not have a single
// textual representation; see printIdentifier()
@@ -47,11 +49,11 @@ JSON ExprLiteral::toJSON(const SymbolTable & symbols) const
switch (v.type()) {
case nInt:
valueType = "Int";
value = v.integer.value;
value = v.integer().value;
break;
case nFloat:
valueType = "Float";
value = v.fpoint;
value = v.fpoint();
break;
case nString:
valueType = "String";
+58 -15
View File
@@ -127,7 +127,7 @@ public:
virtual JSON toJSON(const SymbolTable & symbols) const;
virtual void accept(ExprVisitor & ev, std::unique_ptr<Expr> & ptr) = 0;
virtual void eval(EvalState & state, Env & env, Value & v);
virtual Value * maybeThunk(EvalState & state, Env & env);
virtual Value maybeThunk(EvalState & state, Env & env);
virtual void setName(Symbol name);
PosIdx getPos() const { return pos; }
@@ -175,26 +175,61 @@ protected:
Value v;
ExprLiteral(const PosIdx pos) : Expr(pos) {};
public:
ExprLiteral(const PosIdx pos, NewValueAs::integer_t, NixInt n) : Expr(pos) { v.mkInt(n); };
ExprLiteral(const PosIdx pos, NewValueAs::integer_t, NixInt::Inner n) : Expr(pos) { v.mkInt(n); };
ExprLiteral(const PosIdx pos, NewValueAs::floating_t, NixFloat nf) : Expr(pos) { v.mkFloat(nf); };
Value * maybeThunk(EvalState & state, Env & env) override;
Value maybeThunk(EvalState & state, Env & env) override;
JSON toJSON(const SymbolTable & symbols) const override;
void eval(EvalState & state, Env & env, Value & v) override;
void accept(ExprVisitor & ev, std::unique_ptr<Expr> & ptr) override { ev.visit(*this, ptr); }
};
struct ExprInt : ExprLiteral
{
Value::Int i;
ExprInt(const PosIdx pos, NixInt n) : ExprLiteral(pos), i{{Value::Acb::tInt}, n}
{
v = Value::isTaggableInteger(n) ? Value{NewValueAs::integer, n} : Value(i);
}
ExprInt(const PosIdx pos, NixInt::Inner n) : ExprInt(pos, NixInt(n)) {}
};
struct ExprFloat : ExprLiteral
{
Value::Float f;
ExprFloat(const PosIdx pos, NewValueAs::floating_t, double f)
: ExprLiteral(pos)
, f{{Value::Acb::tFloat}, f}
{
v = Value(this->f);
}
};
struct ExprString : ExprLiteral
{
std::string s;
ExprString(const PosIdx pos, std::string &&s) : ExprLiteral(pos), s(std::move(s)) { v.mkString(this->s.data()); };
std::unique_ptr<Value::Str, Value::Str::Deleter> contents;
Value::String strcb{.content = contents.get(), .context = nullptr};
ExprString(const PosIdx pos, std::string s) : ExprLiteral(pos), contents(Value::Str::copy(s))
{
v = {NewValueAs::string, &strcb};
}
std::string_view str() const
{
return contents->str();
}
};
struct ExprPath : ExprLiteral
{
std::string s;
ExprPath(const PosIdx pos, std::string s) : ExprLiteral(pos), s(std::move(s)) { v.mkPath(this->s.c_str()); };
std::unique_ptr<Value::Str, Value::Str::Deleter> contents;
Value::String strcb{.content = contents.get(), .context = Value::String::path};
ExprPath(const PosIdx pos, std::string s) : ExprLiteral(pos), contents(Value::Str::copy(s))
{
v = Value{NewValueAs::path, &strcb};
}
std::string_view str() const
{
return contents->str();
}
};
typedef uint32_t Level;
@@ -228,7 +263,7 @@ struct ExprVar : Expr
ExprVar(Symbol name) : name(name), needsRoot(false) { };
ExprVar(const PosIdx & pos, Symbol name, bool needsRoot = false) : Expr(pos), name(name), needsRoot(needsRoot) { };
Value * maybeThunk(EvalState & state, Env & env) override;
Value maybeThunk(EvalState & state, Env & env) override;
JSON toJSON(const SymbolTable & symbols) const override;
void eval(EvalState & state, Env & env, Value & v) override;
void accept(ExprVisitor & ev, std::unique_ptr<Expr> & ptr) override { ev.visit(*this, ptr); }
@@ -254,6 +289,8 @@ struct ExprInheritFrom : Expr
void accept(ExprVisitor & ev, std::unique_ptr<Expr> & ptr) override { ev.visit(*this, ptr); }
};
struct Attr;
struct ExprSelect : Expr
{
/** The expression attributes are being selected on. e.g. `foo` in `foo.bar.baz`. */
@@ -272,6 +309,9 @@ struct ExprSelect : Expr
JSON toJSON(const SymbolTable & symbols) const override;
void eval(EvalState & state, Env & env, Value & v) override;
void accept(ExprVisitor & ev, std::unique_ptr<Expr> & ptr) override { ev.visit(*this, ptr); }
private:
Attr const * selectSingleAttr(EvalState & state, Env & env, AttrName const & attrName, Value & vCurrent);
};
struct ExprOpHasAttr : Expr
@@ -370,7 +410,7 @@ struct ExprList : Expr
JSON toJSON(const SymbolTable & symbols) const override;
void eval(EvalState & state, Env & env, Value & v) override;
void accept(ExprVisitor & ev, std::unique_ptr<Expr> & ptr) override { ev.visit(*this, ptr); }
Value * maybeThunk(EvalState & state, Env & env) override;
Value maybeThunk(EvalState & state, Env & env) override;
};
struct Pattern {
@@ -384,7 +424,8 @@ struct Pattern {
virtual std::shared_ptr<const StaticEnv> buildEnv(const StaticEnv * up) = 0;
virtual void accept(ExprVisitor & ev) = 0;
virtual Env & match(ExprLambda & lambda, EvalState & state, Env & up, Value * arg, const PosIdx pos) = 0;
virtual Env &
match(ExprLambda & lambda, EvalState & state, Env & up, Value & arg, const PosIdx pos) = 0;
virtual void addBindingsToJSON(JSON & out, const SymbolTable & symbols) const = 0;
};
@@ -399,7 +440,8 @@ struct SimplePattern : Pattern
virtual std::shared_ptr<const StaticEnv> buildEnv(const StaticEnv * up) override;
virtual void accept(ExprVisitor & ev) override;
virtual Env & match(ExprLambda & lambda, EvalState & state, Env & up, Value * arg, const PosIdx pos) override;
virtual Env &
match(ExprLambda & lambda, EvalState & state, Env & up, Value & arg, const PosIdx pos) override;
virtual void addBindingsToJSON(JSON & out, const SymbolTable & symbols) const override;
};
@@ -420,7 +462,8 @@ struct AttrsPattern : Pattern
virtual std::shared_ptr<const StaticEnv> buildEnv(const StaticEnv * up) override;
virtual void accept(ExprVisitor & ev) override;
virtual Env & match(ExprLambda & lambda, EvalState & state, Env & up, Value * arg, const PosIdx pos) override;
virtual Env &
match(ExprLambda & lambda, EvalState & state, Env & up, Value & arg, const PosIdx pos) override;
virtual void addBindingsToJSON(JSON & out, const SymbolTable & symbols) const override;
+14 -11
View File
@@ -148,7 +148,7 @@ struct ExprState
std::unique_ptr<Expr> negate(PosIdx pos, State & state)
{
std::vector<std::unique_ptr<Expr>> args(2);
args[0] = std::make_unique<ExprLiteral>(pos, NewValueAs::integer, 0);
args[0] = std::make_unique<ExprInt>(pos, 0);
args[1] = popExprOnly();
return std::make_unique<ExprCall>(pos, state.mkInternalVar(pos, state.s.sub), std::move(args));
}
@@ -326,10 +326,11 @@ template<> struct BuildAST<grammar::v1::attr::simple> {
template<> struct BuildAST<grammar::v1::attr::string> {
static void apply(const auto & in, auto & s, State & ps) {
auto e = s->popExprOnly();
if (auto str = dynamic_cast<ExprString *>(e.get()))
s.pushAttr(ps.symbols.create(str->s), ps.at(in));
else
if (auto estr = dynamic_cast<ExprString *>(e.get())) {
s.pushAttr(ps.symbols.create(estr->str()), ps.at(in));
} else {
s.pushAttr(std::move(e), ps.at(in));
}
}
};
@@ -387,9 +388,9 @@ template<> struct BuildAST<grammar::v1::inherit> : change_head<InheritState> {
for (auto & i : s.attrs) {
if (i.symbol)
continue;
if (auto str = dynamic_cast<ExprString *>(i.expr.get()))
i = AttrName(i.pos, ps.symbols.create(str->s));
else {
if (auto estr = dynamic_cast<ExprString *>(i.expr.get())) {
i = AttrName(i.pos, ps.symbols.create(estr->str()));
} else {
throw ParseError({
.msg = HintFmt("dynamic attributes not allowed in inherit"),
.pos = ps.positions[i.pos]
@@ -507,7 +508,7 @@ template<> struct BuildAST<grammar::v1::expr::int_> {
.pos = ps.positions[ps.at(in)],
});
}
s.emplaceExpr<ExprLiteral>(ps.at(in), NewValueAs::integer, v);
s.emplaceExpr<ExprInt>(ps.at(in), v);
}
};
@@ -542,7 +543,7 @@ template<> struct BuildAST<grammar::v1::expr::float_> {
});
}
}();
s.emplaceExpr<ExprLiteral>(ps.at(in), NewValueAs::floating, v);
s.emplaceExpr<ExprFloat>(ps.at(in), NewValueAs::floating, v);
}
};
@@ -770,13 +771,15 @@ template<> struct BuildAST<grammar::v1::path> : change_head<StringState> {
template<typename E>
static void check_slash(PosIdx end, StringState & s, State & ps) {
auto e = dynamic_cast<E *>(s.parts.back().second.get());
if (!e || !e->s.ends_with('/'))
if (!e || !e->str().ends_with('/')) {
return;
if (s.parts.size() > 1 || e->s != "/")
}
if (s.parts.size() > 1 || e->str() != "/") {
throw ParseError({
.msg = HintFmt("path has a trailing slash"),
.pos = ps.positions[end],
});
}
}
static void success(const auto & in, StringState & s, ExprState & e, State & ps) {
+527 -331
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -33,7 +33,7 @@ struct RegisterPrimOp
* will get called during EvalState initialization, so there
* may be primops not yet added and builtins is not yet sorted.
*/
RegisterPrimOp(PrimOp && primOp);
RegisterPrimOp(PrimOpDetails && primOp);
};
/* These primops are disabled without enableNativeCode, but plugins
+15 -13
View File
@@ -3,6 +3,7 @@
#include "lix/libstore/derivations.hh"
#include "lix/libstore/store-api.hh"
#include "lix/libutil/types.hh"
#include "value.hh"
namespace nix {
@@ -147,9 +148,10 @@ void prim_getContext(EvalState & state, Value * * args, Value & v)
infoAttrs.alloc(sAllOutputs).mkBool(true);
if (!info.second.outputs.empty()) {
auto & outputsVal = infoAttrs.alloc(state.ctx.s.outputs);
outputsVal = state.ctx.mem.newList(info.second.outputs.size());
auto content = state.ctx.mem.newList(info.second.outputs.size());
outputsVal = {NewValueAs::list, content};
for (const auto & [i, output] : enumerate(info.second.outputs))
(outputsVal.listElems()[i] = state.ctx.mem.allocValue())->mkString(output);
content->elems[i].mkString(output);
}
attrs.alloc(state.ctx.store->printStorePath(info.first)).mkAttrs(infoAttrs);
}
@@ -171,7 +173,7 @@ static void prim_appendContext(EvalState & state, Value * * args, Value & v)
state.forceAttrs(*args[1], noPos, "while evaluating the second argument passed to builtins.appendContext");
auto sAllOutputs = state.ctx.symbols.create("allOutputs");
for (auto & i : *args[1]->attrs) {
for (auto & i : *args[1]->attrs()) {
const auto & name = state.ctx.symbols[i.name];
if (!state.ctx.store->isStorePath(name))
state.ctx.errors.make<EvalError>(
@@ -181,11 +183,11 @@ static void prim_appendContext(EvalState & state, Value * * args, Value & v)
auto namePath = state.ctx.store->parseStorePath(name);
if (!settings.readOnlyMode)
state.aio.blockOn(state.ctx.store->ensurePath(namePath));
state.forceAttrs(*i.value, i.pos, "while evaluating the value of a string context");
auto a = i.value->attrs->get(state.ctx.s.path);
state.forceAttrs(i.value, i.pos, "while evaluating the value of a string context");
auto a = i.value.attrs()->get(state.ctx.s.path);
if (a) {
if (state.forceBool(
*a->value, a->pos, "while evaluating the `path` attribute of a string context"
a->value, a->pos, "while evaluating the `path` attribute of a string context"
))
{
context.emplace(NixStringContextElem::Opaque{
@@ -194,10 +196,10 @@ static void prim_appendContext(EvalState & state, Value * * args, Value & v)
}
}
a = i.value->attrs->get(sAllOutputs);
a = i.value.attrs()->get(sAllOutputs);
if (a) {
if (state.forceBool(
*a->value,
a->value,
a->pos,
"while evaluating the `allOutputs` attribute of a string context"
))
@@ -214,20 +216,20 @@ static void prim_appendContext(EvalState & state, Value * * args, Value & v)
}
}
a = i.value->attrs->get(state.ctx.s.outputs);
a = i.value.attrs()->get(state.ctx.s.outputs);
if (a) {
state.forceList(
*a->value, a->pos, "while evaluating the `outputs` attribute of a string context"
a->value, a->pos, "while evaluating the `outputs` attribute of a string context"
);
if (a->value->listSize() && !isDerivation(name)) {
if (a->value.listSize() && !isDerivation(name)) {
state.ctx.errors.make<EvalError>(
"tried to add derivation output context of %s, which is not a derivation, to a string",
name
).atPos(i.pos).debugThrow();
}
for (auto elem : a->value->listItems()) {
for (auto & elem : a->value.listItems()) {
auto outputName = state.forceStringNoCtx(
*elem, a->pos, "while evaluating an output name within a string context"
elem, a->pos, "while evaluating an output name within a string context"
);
context.emplace(NixStringContextElem::Built {
.drvPath = makeConstantStorePath(namePath),
+7 -8
View File
@@ -118,7 +118,7 @@ void prim_fetchClosure(EvalState & state, Value * * args, Value & v)
std::optional<StorePathOrGap> toPath;
std::optional<bool> inputAddressedMaybe;
for (auto & attr : *args[0]->attrs) {
for (auto & attr : *args[0]->attrs()) {
const auto & attrName = state.ctx.symbols[attr.name];
auto attrHint = [&]() -> std::string {
return "while evaluating the '" + attrName + "' attribute passed to builtins.fetchClosure";
@@ -126,27 +126,26 @@ void prim_fetchClosure(EvalState & state, Value * * args, Value & v)
if (attrName == "fromPath") {
NixStringContext context;
fromPath = state.coerceToStorePath(attr.pos, *attr.value, context, attrHint());
fromPath = state.coerceToStorePath(attr.pos, attr.value, context, attrHint());
}
else if (attrName == "toPath") {
state.forceValue(*attr.value, attr.pos);
bool isEmptyString = attr.value->type() == nString && attr.value->str().empty();
state.forceValue(attr.value, attr.pos);
bool isEmptyString = attr.value.type() == nString && attr.value.str().empty();
if (isEmptyString) {
toPath = StorePathOrGap {};
}
else {
NixStringContext context;
toPath = state.coerceToStorePath(attr.pos, *attr.value, context, attrHint());
toPath = state.coerceToStorePath(attr.pos, attr.value, context, attrHint());
}
}
else if (attrName == "fromStore")
fromStoreUrl = state.forceStringNoCtx(*attr.value, attr.pos,
attrHint());
fromStoreUrl = state.forceStringNoCtx(attr.value, attr.pos, attrHint());
else if (attrName == "inputAddressed")
inputAddressedMaybe = state.forceBool(*attr.value, attr.pos, attrHint());
inputAddressedMaybe = state.forceBool(attr.value, attr.pos, attrHint());
else
throw Error({

Some files were not shown because too many files have changed in this diff Show More