Compare commits

..
Author SHA1 Message Date
Raito Bezarius a6f0e87fa8 libexpr/static-analyzer: init
Change-Id: I606718fbe33db1f6c00ca34f45c6553ce468398c
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-11-21 17:36:35 +01:00
eldritch horrors 17879c9a83 libexpr: de-ptr-ize Value references
thunk values are shareable, and we can represent invalid/uninitialized
values with a special bit pattern that makes no sense otherwise. there
is no need to keep allocating values on the heap, instead we can treat
values like reference-counted smart pointers to heap objects, which in
turn lets us save a lot of allocations and, ultimately, gc heap space.

compared to our baseline (main of 2025-09-27) we save 15%+ memory on a
system rebuild and 17% on nix search. eval time regresses by ~3% for a
system rebuild, while nix search is 7% faster. further optimization is
probably possible (but for now this will just have to be good enough).

Change-Id: Ib6c47acdbe2fac4f76a83c2269f16f30ef66b2e1
2025-09-28 00:02:21 +02:00
eldritch horrors 9ba7a7eee7 libexpr: reformat stuff we'll change soon
mainly to keep the next diff smaller. it'll be large enough as is.

Change-Id: Ib8a34520f03539cbf6aa2f0e66cbed05fe1225eb
2025-09-28 00:02:21 +02:00
eldritch horrors 5f070b2297 libexpr: de-ptr-ize many Value uses
with thunk state being shareable we no longer need to worry about value
uniqueness, only about value lifetime. this means we can liberally drop
indirections and allocations, passing references instead of pointers or
using stack memory instead of gc-managed memory for some intermediates.

Change-Id: I2d48a6fd57a376d544bd9bd2d05e5420611986d1
2025-09-28 00:02:21 +02:00
eldritch horrors 3b093988fb libexpr: remove ValueVector{,Map}
these typedefs were convenient in the past, but now they're not really.

Change-Id: I7522f7582ead545148af8e6444295d620ee0872a
2025-09-28 00:02:21 +02:00
eldritch horrors fe6dfa5ace libexpr: remove unused return types
Change-Id: Ief898b70f9781bd3dbe66734710f388daa2f2fed
2025-09-28 00:02:21 +02:00
eldritch horrors acd85805ab libexpr: remove type punning in primt_attrValues
there's no measurable performance gain in real-world testing to abusing
our list value storage like this. we haven't tested how much storage we
actually need on the stack to offset most of the temporary storage cost
and used 128 as a "good enough" value instead, reserving 1 kiB of stack
space on 64 bit platforms in a leaf function. this should do for a bit.

Change-Id: Ie98519b1da3e6fe685da88d1c44ffb4580fb592a
2025-09-28 00:02:21 +02:00
eldritch horrors af86b74467 libexpr: tag Value::Acb
Value is already tagged. Value::Acb blocks are allocated for lambdas (so
we can fit the value tag into the three bits we have available), but the
current layout is rather wasteful for this purpose. the type bits can be
stored together with parts of pointers, which in the lambda case will be
the scope the lambda captures. the expr could also be used, but Env is a
gc-allocated item and thus guaranteed to be aligned properly for tagging

Change-Id: Ia685875387c7795bc4a00d73d1ce3cfea84e7297
2025-09-28 00:02:21 +02:00
eldritch horrors 85ed12485e libexpr: make thunk state shareable
this is a strong prerequisite for making values themselves copyable
without duplicating evaluation side effects. with this we can treat
`Value` the way we treated `Value *` to date and drop indirections.

Change-Id: I08f30d12697614a3ae7149615f6f1da83b13f52b
2025-09-28 00:02:21 +02:00
eldritch horrors 30b971f6e0 libexpr: reduce Value size to one pointer
Change-Id: Id6e4a2f68eaa4afdd006379ebcf839c4a126b819
2025-09-28 00:02:21 +02:00
eldritch horrors 32440b5fae libexpr: heap-allocate "large" integers
Change-Id: Ic391f2f1bf87f044d7a688196ba9e0ad766d65aa
2025-09-28 00:02:21 +02:00
eldritch horrors aa39e14fcf libexpr: heap-allocate app nodes
Change-Id: I9a39dcf0be7589cedf494757e21665c5d50e446b
2025-09-28 00:02:21 +02:00
eldritch horrors b0d11f9da1 libexpr: heap-allocate thunk control state
Change-Id: I20d8ab1d6f683c0a2f3b77edf9bdad147d62c8fa
2025-09-28 00:02:20 +02:00
eldritch horrors 002dfbb2e3 libexpr: move lambdas to auxiliary storage
Change-Id: Ibe4885f17c0ba1634ed6dbca0a45f8bd4619d69b
2025-09-28 00:02:20 +02:00
eldritch horrors 6e242e8b9b libexpr: move primops to auxiliary storage
same as for null: we have few of them, they're statically allocated,
and they're not the largest contributor to the value population. not
storing them in Value itself frees up resources we *will* use later.

Change-Id: I521f9f243f48f56a78f7bffdf1dc1f0bc40a5c2d
2025-09-28 00:02:20 +02:00
eldritch horrors b1ffae3ccd libexpr: move null to auxiliary storage
we only need the one object for it. there's no need to waste precious
bits of the value internal type enum for this simple singleton datum.

Change-Id: Ie314b5bf429015e518798d9d65ad8ab2bb84a38e
2025-09-28 00:02:20 +02:00
eldritch horrors 5def7559a6 libexpr: move floats to auxiliary storage
floats are used very rarely, and our float support is bad enough to
strongly discourage using them on reproducibility grounds alone. we
can thus move them to more expensive storage without hurting folks.

Change-Id: I1086f612f85e294dd3fae4a2d334e09f52bbe4a8
2025-09-28 00:02:20 +02:00
eldritch horrors ce70234904 libexpr: move external value refs to aux storage
external values very rarely appear during eval "normal" eval, and
creating them is pretty expensive. does *anything* even use them?

Change-Id: Id50fa3f76b7e1f551d550d99996a1ed5880b2531
2025-09-28 00:02:20 +02:00
eldritch horrors cbc378b277 libexpr: heap-alloc string control blocks
despite not using allocation caches this does not have a statistically
significant performance impact, with less than 1% extra memory needed.

Change-Id: Ibe51a55ba986e471f217f3724977af17880fafff
2025-09-28 00:02:20 +02:00
eldritch horrors 5d6bb8c350 libexpr: remove tPrimOpApp
using the same nodes as tApp is possible, and thanks to multi-arg app
nodes it can even be a bit faster than the linked lists used to date.

Change-Id: Idccb7c0b54c808e62da85d1c42ee09e6e92c4f7b
2025-09-28 00:02:20 +02:00
eldritch horrors 93bda92508 libexpr: add multi-arg app nodes
these behave like the old chains of app nodes, but they can store more
than one argument per node. for tApp values themselves this is not all
that useful, but if we could share tApp and tPrimOpApp backing storage
we could avoid creating and traversing the linked lists of values that
are currently needed to represent partially applied builtin functions.

Change-Id: I5a2a02d9733e1e0be5443459e2998d62fd3b9a5b
2025-09-28 00:02:20 +02:00
eldritch horrors dbee9d15c5 libexpr: use std::span for callFunction
Change-Id: I8c94bafabdb2416c85d9721d3d6424f52fbd45e0
2025-09-28 00:02:20 +02:00
eldritch horrors 7642b7227a libexpr: unify strings and paths
paths already are just strings with slightly magic semantics. the type
can mirror that at no perf cost, letting us drop one internalType tag.

Change-Id: I98acaa4fe3bedd28fc0841e1b81184d8dcddacc5
2025-09-28 00:02:20 +02:00
eldritch horrors aa96b00182 libexpr: alloc list storage as a (length, vla) type
memory overhead is minimal and performance impact not measurable. once
we've done something like this for all value types that don't fit in a
single machine word we can cut a word from Value, offsetting the cost.

Change-Id: I9813bacd7e851957ad3426aed8f74033179a4212
2025-09-28 00:02:20 +02:00
eldritch horrors d37a5d4000 libexpr: don't inline small lists into values
this has no measurable performance impact thanks to the new caches.

Change-Id: Ib403a9a567161675f78e8c5d314d6340183d181d
2025-09-28 00:02:20 +02:00
eldritch horrors b36a19e50c libexpr: cache more allocation sizes
we now use a single cache set for a number of sizes from one to eight
words. this also matches small attrsets, but perf impact seems small.

Change-Id: Icf16b329b98a20fcc9fe75e6395e148f0852c798
2025-09-28 00:02:20 +02:00
eldritch horrors f08b8532be libexpr: "hide" Value union members
on its own this is not very useful, but having accessors for every value
kind is a prerequisite for doing smart things with Value than the union.
the net effect for now is only to add a few parentheses across the tree.

Change-Id: I88688ac09eb08495dad1eb221034ca540f094950
2025-09-28 00:02:20 +02:00
eldritch horrors f3fcd7c02f bench: add memory benchmark
Change-Id: I4b2aa305b452a3a0c73e37953a5c332219d43d2c
2025-09-28 00:02:20 +02:00
eldritch horrors 1f0b25a92c bench: allow benchmarking a single build
Change-Id: Ie77247f9348eaeea740b0f894819ee2383d92312
2025-09-28 00:02:20 +02:00
eldritch horrors 82b771951c testing: disable substitution in f2
mirrors f1. much faster.

Change-Id: I38bbbd5b26220f480afe76ef9302e5c90f21507a
2025-09-28 00:02:20 +02:00
716 changed files with 6079 additions and 10805 deletions
+3 -3
View File
@@ -24,8 +24,8 @@ def map_contents_recursively(transformer):
def process_command:
.[0] as $context |
.[1] as $body |
# XXX FUTURE: drop sections once mdBook is at 0.5.0 or above in nixpkgs
$body | (.items? // .sections) |= map(map_contents_recursively(if $context.renderer == "html" then transform_anchors_html else transform_anchors_strip end))
;
$body + {
sections: $body.sections | map(map_contents_recursively(if $context.renderer == "html" then transform_anchors_html else transform_anchors_strip end)),
};
process_command
+5 -6
View File
@@ -22,21 +22,20 @@ fold.level = 30
# not want to disable the links preprocessor entirely though because that requires
# disabling *all* built-in preprocessors and selectively reenabling those we want.
[preprocessor.substitute]
command = "python3 substitute.py"
command = "python3 doc/manual/substitute.py"
before = ["anchors", "links"]
[preprocessor.anchors]
renderers = ["html"]
command = "jq --from-file anchors.jq"
command = "jq --from-file doc/manual/anchors.jq"
[output.markdown]
# XXX FUTURE: may be reenabled once mdBook 0.5.0 or above and matching mdbook-linkchecker are in nixpkgs
#[output.linkcheck]
[output.linkcheck]
# no Internet during the build (in the sandbox)
#follow-web-links = false
follow-web-links = false
# mdbook-linkcheck does not understand [foo]{#bar} style links, resulting in
# excessive "Potential incomplete link" warnings. No other kind of warning was
# produced at the time of writing.
#warning-policy = "ignore"
warning-policy = "ignore"
-20
View File
@@ -61,11 +61,6 @@ cole-h:
display_name: Cole Helbling
github: cole-h
commentator2.0:
display_name: Commentator2.0 (Rutile)
forgejo: commentatorforall
github: CommentatorForAll
delan:
display_name: delan
forgejo: delan
@@ -91,11 +86,6 @@ ericson:
display_name: John Ericson
github: ericson2314
getchoo:
display_name: Seth Flynn
forgejo: getchoo
github: getchoo
gilice:
forgejo: gilice
@@ -130,11 +120,6 @@ jade:
just1602:
forgejo: just1602
k900:
display_name: K900
forgejo: K900
github: K900
kasimeka:
display_name: ورد
forgejo: janw4ld
@@ -186,11 +171,6 @@ midnightveil:
forgejo: midnightveil
github: midnightveil
milibopp:
display_name: Emilia Bopp
forgejo: milibopp
github: milibopp
nan-git:
display_name: NaN-git
github: NaN-git
+2 -4
View File
@@ -42,9 +42,8 @@ manual = custom_target(
'-c',
'''
@0@ @INPUT0@ @CURRENT_SOURCE_DIR@ > @DEPFILE@
cd @3@
@1@ build . -d @2@ | { grep -Fv "because fragment resolution isn't implemented" || :; }
cd @SOURCE_ROOT@
@1@ build doc/manual -d @2@ | { grep -Fv "because fragment resolution isn't implemented" || :; }
rm -rf @2@/manual
mv @2@/html @2@/manual
find @2@/manual -iname meson.build -delete
@@ -52,7 +51,6 @@ manual = custom_target(
python.full_path(),
mdbook.full_path(),
meson.current_build_dir(),
meson.current_source_dir()
),
],
input : [
@@ -83,7 +81,7 @@ manual = custom_target(
depfile : 'manual.d',
env : {
'RUST_LOG': 'info',
'MANUAL_SUBSTITUTE_SEARCH': meson.current_build_dir() / 'src',
'MDBOOK_SUBSTITUTE_SEARCH': meson.current_build_dir() / 'src',
},
)
manual_md = manual[1]
+16
View File
@@ -0,0 +1,16 @@
---
synopsis: First argument to `--arg`/`--argstr` must be a valid Nix identifier
issues: [fj#496]
category: "Breaking Changes"
credits: [ma27]
---
The first argument to `--arg`/`--argstr` must be a valid Nix identifier, i.e.
`nix-build --arg config.allowUnfree true` is now rejected.
This is because that invocation is a false friend since it doesn't set
`{ config = { allowUnfree = true; }; }`, but `{ "config.allowUnfree" = true; }`.
The idea is to change the behavior to the latter in the long-term. For that,
non-identifiers started giving a warning since 2.92 and are now rejected to give people
who depend on that a chance to notice and potentially weigh in on the discussion.
+20
View File
@@ -0,0 +1,20 @@
---
synopsis: "Improved susbtituter query speed"
issues: []
cls: []
category: Improvements
credits: [horrors]
---
The code used to query substituters for derivations has been rewritten slightly
to take advantage of our asynchronous runtime. Such queries run for every build
that could download from substituters and processes every derivation that isn't
yet present on the local system. Previously Lix would use `http-connections` to
limit query concurrency, even for modern caches that support HTTP/2 and have no
limit on how many queries can be run concurrently on one single connection. Lix
no longer does this, resulting in approximately 60% reduction in query time for
medium-sized closures (e.g. NixOS system closures) during testing, although the
exact number depends greatly on local network latency and generally improves as
latency increases. Unlike previously setting `http-connections` to `1` or other
low values no longer brings a massive penalty in query performance if the cache
in use by the querying system supports HTTP/2 (as e.g. `cache.nixos.org` does).
+12
View File
@@ -0,0 +1,12 @@
---
synopsis: "`build-dir` no longer defaults to `temp-dir`"
cls: [3453]
category: "Fixes"
credits: [horrors]
---
The directory in which temporary build directories are created no longer defaults
to the value of the `temp-dir` setting to avoid builders making their directories
world-accessible. This behavior has been used to escape the build sandbox and can
cause build impurities even when not used maliciously. We now default to `builds`
in `NIX_STATE_DIR` (which is `/nix/var/nix/builds` in the default configuration).
-15
View File
@@ -1,15 +0,0 @@
---
synopsis: "Add `builtins.warn` for emitting warnings from Nix code"
cls: [2248]
category: "Features"
credits: [milibopp, Qyriad]
---
Lix now has a builtin function for emitting warnings.
Like `builtins.trace`, it takes two arguments: the message to emit, and the expression to return.
_Unlike_ `builtins.trace`, `builtins.warn` requires the first argument — the message — to be a string.
In the future we may extend `builtins.warn` to accept a more structured API.
To go along with this, we also have two new config settings:
- [`debugger-on-warn`](@docroot@/command-ref/conf-file.md#conf-debugger-on-warn), which, when used with `--debugger`, makes `builtins.warn` also function like [`builtins.break`](@docroot@/language/builtins.md#builtins-break).
- [`abort-on-warn`](@docroot@/command-ref/conf-file.md#conf-abort-on-warn), which aborts evaluation entirely after the warning is emitted.
@@ -0,0 +1,60 @@
---
synopsis: "Global certificate authorities are copied inside the builder's environment"
issues: [gh#12698, fj#885]
cls: [3765]
category: Fixes
credits: [raito, emilazy]
---
Previously, CA certificates were only installed at
`/etc/ssl/certs/ca-certificates.crt` for sandboxed builds on Linux.
This setup was insufficient in light of recent changes in `nixpkgs`, which now
enforce HTTPS usage for `fetchurl`, even for fixed-output derivations, to
mitigate confidentiality risks such as `netrc` or credentials leakage.
`nixpkgs` still make use of a special package called `cacerts` which contains a
copy of the CA certificates maintained by Nixpkgs and added as a reference for
TLS-enabled fetchers.
As a result, having a consistent and trusted certificate authority in all
builder environments is becoming more essential.
On `nix-darwin`, the `NIX_SSL_CERT_FILE` environment variable is always
explicitly defined, but it is ignored by the sandbox setup.
Simultaneously, Nix evaluates and propagates impure environment variables via
`lib.proxyImpureEnvVars`, meaning that if `NIX_SSL_CERT_FILE` is set (which
influences the default value for `ssl-cert-file`), it will be forwarded
unchanged into the builder environment.
However, on Linux, Nix also *copies* the CA file into the sandbox, creating a
discrepancy between the value of `NIX_SSL_CERT_FILE` and the actual trusted
certificate path used during the build.
This divergence caused confusion and was partially addressed by attempts to
whitelist the CA path in the Darwin sandbox (see cl/2906), but that approach
involved a non-trivial path canonicalization step and is not as general as this one.
To address this properly, we now emit a warning and override
`NIX_SSL_CERT_FILE` inside the builder, explicitly pointing it to the CA file
copied into the sandbox.
This eliminates ambiguity between `NIX_SSL_CERT_FILE`
and `ssl-cert-file`, ensuring consistent trust anchors across platforms.
This warning might become a hard error as we figure out what to do regarding
`lib.proxyImpureEnvVars` in nixpkgs.
The behavior has been verified across sandboxed and unsandboxed builds on both
Linux and Darwin.
As a consequence of this change, approximately 500KB of CA certificate data is
now unconditionally copied into the build directory for fixed-output
derivations.
While this ensures consistent trust verification without having to restart the
daemon after system upgrades, it may introduce a slight overhead in build
performance. At present, no optimizations have been implemented to avoid this
copy, but if this overhead proves noticeable in your workflows, please open an
issue so we can evaluate and possibly implement different strategies to render
trust anchors visible.
+69
View File
@@ -0,0 +1,69 @@
---
synopsis: New cgroup delegation model
issues: [fj#537, fj#77]
cls: [3230]
category: "Breaking Changes"
credits: [raito, horrors, lheckemann]
---
Builds using cgroups (i.e. `use-cgroups = true` and the experimental feature
`cgroups`) now always delegate a cgroup tree to the sandbox.
Compared to the original C++ Nix project, our delegation includes the
`subtree_control` file as well, which means that the sandbox can disable
certain controllers in its own cgroup tree.
This is a breaking change because this requires the Nix daemon to run with an
already delegated cgroup tree by the service manager.
## How to setup the cgroup tree with systemd?
systemd offers knobs to perform the required setup using:
```
[Service]
Delegate=yes
DelegateSubtree=supervisor
```
These directives are now included in our systemd packaging.
## What about using Nix as root without connecting to the daemon?
Builds run as `root` without connecting to the daemon relying on the cgroup
feature are now broken, i.e.
```console
# nix-build --use-cgroups --sandbox ... # will not work
```
Consider doing instead:
```console
# systemd-run --same-dir --wait -p Delegate=yes -p DelegateSubgroup=supervisor nix-build --use-cgroups ...
```
If you need to disable cgroups temporarily, remember that you can do
`NIX_CONF='include /etc/nix/nix.conf\nuse-cgroups = false' nix-build ...` or
`nix-build --no-use-cgroups ...`.
## What about other service managers than systemd?
systemd has a [documentation](https://systemd.io/CGROUP_DELEGATION/) on how to
handle cgroup delegation from service management perspective.
If your service manager adheres to systemd semantics, e.g. writing an extended
attribute `user.delegate=1` on the delegated cgroup tree directory and moving
the `nix-daemon` process inside a cgroup tree to respect the inner process
rule, then, the feature will work as well.
## Why is the cgroup feature still experimental?
While the cgroup feature unlocks many use cases, its behavior and integration (e.g. user experience), especially at scale on build farms or in multi-tenant environments, are not yet fully matured. Theres also potential for deeper systemd integration (e.g. using slices and scopes) that has not been fully explored.
To avoid locking in an unstable interface, were keeping the experimental flag until we have validated the feature across a broader range of scenarios, including but not limited to:
* Nix as root
* Hydra-style build farms
* Forgejo CI runners
* Shared remote builders
@@ -1,13 +0,0 @@
---
synopsis: "Deprecate shadowing internal files through the Nix search path"
issues: [998]
cls: [4632]
category: "Breaking Changes"
credits: [thubrecht]
---
As Lix uses the path `<nix/fetchurl.nix>` for bootstrapping purposes, the ability to shadow it by adding `nix=/some/path` (or `/other/path` that contains a `nix` directory) to the search path is not desirable.
To alleviate potential issues, Lix now emits a warning when the Nix search path contains potential shadows for internal files, which will be changed to an error in a future release.
The warning can be disabled by enabling the deprecated feature `nix-path-shadow`.
@@ -0,0 +1,16 @@
---
synopsis: Deprecation of CA derivations, dynamic derivations, and impure derivations
issues: [fj#815]
cls: []
significance: significant
category: Miscellany
credits: []
---
Content-addressed derivations are now deprecated and slated for removal in Lix 2.94.
We're doing this because the CA derivation system has been a known cause of problems
and inconsistencies, is unmaintained, habitually makes improving the store code very
difficult (or blocks such improvements outright), and is beset by a number of design
flaws that in our opinion cannot be fixed without a full reimplementation from zero.
Dynamic derivations and impure derivations are built on the CA derivation framework,
and owing to this they too are deprecated and slated for removal in another release.
+25
View File
@@ -0,0 +1,25 @@
---
synopsis: "Hitting Control-C twice always terminates Lix"
cls: [3574]
issues: []
category: "Improvements"
credits: [horrors]
---
Hitting Control-C or sending `SIGINT` to Lix now prints an informational message
if it is still running after on second, the second Control-C/`SIGINT` terminates
Lix immediately without waiting for any shutdown code to finish running. Lix did
not treat the second such event differently from first in the past; this made it
impossible to easily terminate running Lix processes that got stuck in e.g. very
expensive Nixlang code that never interacted with the store. We now terminate as
soon as the user hits Control-C again without waiting any more, to much the same
effect as putting Lix into the background and killing it immediately afterwards.
This means you can now more conveniently break out of stuck Nixlang evaluations:
```
nix-instantiate --eval --expr 'let f = n: if n == 0 then 0 else f (n - 1) + f (n - 1); in f 32'
^CStill shutting down. Press ^C again to abort all operations immediately.
^C
❌130
```
+12
View File
@@ -0,0 +1,12 @@
---
synopsis: "libstore: exponential backoff for downloads"
issues: [lix#932]
cls: [3856]
category: Fixes
credits: [ma27]
---
The connection timeout when downloading from e.g. a binary cache is exponentially
increased per failure. The option `connect-timeout` is now an alias to `max-connect-timeout`
which is the maximum value for a timeout. The start value is controlled
by `initial-connect-timeout` which is `5` by default.
@@ -1,10 +0,0 @@
---
synopsis: "Warn instead of erroring when the final destination of a transfer changes in-flight"
cls: [4641]
issues: [fj#1004]
category: "Miscellany"
credits: [thubrecht]
---
Lix will now emit a warning during downloads where the final destination changes suddently mid-transfer instead of throwing an error.
This transfer behavior has been known to happen very rarely while fetching from some CDNs.
+9
View File
@@ -0,0 +1,9 @@
---
synopsis: Fix develop shells for derivations with escape codes
issues: [fj#991]
cls: [4154, 4155]
category: Fixes
credits: [Qyriad]
---
ASCII control characters (including `\e`, used for ANSI escape codes) in derivation variables are now correctly escaped for `nix develop` and `nix print-dev-env`, instead of erroring.
+13
View File
@@ -0,0 +1,13 @@
---
synopsis: "nix-store --delete: always remove obsolete hardlinks"
issues: []
cls: [3188]
category: Fixes
credits: [lheckemann]
---
Deleting specific paths using `nix-store --delete` or `nix store
delete` previously did not delete hard links created by `nix-store
--optimise` even if they became obsolete, unless _all_ of the given
paths were deleted successfully. Now, hard links are always cleaned
up, even if some of the given paths could not be deleted.
+24
View File
@@ -0,0 +1,24 @@
---
synopsis: "Report GC statistics correctly"
issues: []
cls: [3188]
category: Fixes
credits: [lheckemann]
---
Deleting specific paths using `nix-store --delete` or `nix store delete` previously did
not report statistics correctly when some of the paths could not be deleted, even if
others were deleted:
```
$ nix store delete /nix/store/9bwryidal9q3g91cjm6xschfn4ikd82q-hello-2.12.1 --delete-closure -v
finding garbage collector roots...
deleting '/nix/store/9bwryidal9q3g91cjm6xschfn4ikd82q-hello-2.12.1'
0 store paths deleted, 0.00 MiB freed
error: Cannot delete some of the given paths because they are still alive. Paths not deleted:
k9bxzr1l92r5y6mihrkbpbr3fmc8qszx-libidn2-2.3.8
mbx9ii53lzjlrsnlrfmzpwm33ynljwdn-libunistring-1.3
rf8hcy6bldxdqc0g6q1dcka1vh47x69s-xgcc-14.2.1.20250322-libgcc
vbrdc5wgzn0w1zdp10xd2favkjn5fk7y-glibc-2.40-66
To find out why, use nix-store --query --roots and nix-store --query --referrers.
```
+16
View File
@@ -0,0 +1,16 @@
---
synopsis: Add `inputs.self.submodules` flake attribute
issues: [fj#942]
cls: [3839]
category: Features
credits: [edolstra, kasimeka]
---
A port of <https://github.com/NixOS/nix/pull/12421> to Lix, which:
- adds a general `inputs.self` flake attribute that retroactively applies
configurations to a flake after it's been fetched, then triggers a refetch of
the flake with the new config.
- implements `inputs.self.submodules` that allows a flake to declare its need
for submodules, which are then fetched automatically with no need to pass
`?submodules=1` anywhere.
@@ -1,10 +0,0 @@
---
synopsis: 'functional lang migration'
issues: [lix#856]
cls: [3213, 3214, 3215, 3224, 4092, 4093, 4094, 4095, 4096, 4097, 4098, 4099, 4100, 4101, 4102, 4103, 4104, 4105, 4106, 4107, 4108, 4109, 4110, 4111, 4112, 4113, 4114, 4115, 4116, 4117, 4122, 4123, 4269, 4270, 4271, 4272, 4273, 4274, 4347, 4348, 4349, 4350, 4351, 4352, 4569, 4570, 4571, 4572, 4573, 4574, 4596, 4597, 4598, 4599, 4600, 4601, 4602, 4603, 4604, 4605]
category: Development
credits: [piegames, commentator2.0]
---
We have done it! The functional/lang framework has now been fully migrated to functional2/lang.
This means: no more `just clean` and `just install` mess and whatever because one removed a test.
The lang test suite is also getting a face lift, with an improved folder structure and restructuring of many tests.
@@ -0,0 +1,16 @@
---
synopsis: Add hyperlinks in attr set printing
issues: []
cls: [3790]
category: Features
credits: [jade]
---
The attribute set printer, such as is seen in `nix repl` or in type errors, now prints hyperlinks on each attribute name to its definition site if it is known.
Example: all of the attributes shown here are hyperlinks to the exact definition site of the attribute in question:
```
$ nix eval -f '<nixpkgs>' lib.licenses.mit
{ deprecated = false; free = true; fullName = "MIT License"; redistributable = true; shortName = "mit"; spdxId = "MIT"; url = "https://spdx.org/licenses/MIT.html"; }
```
+24
View File
@@ -0,0 +1,24 @@
---
synopsis: Repl debugger uses `--ignore-try` by default
issues: [lix#666]
cls: [3488]
category: Breaking Changes
credits: [jade]
---
Previously, using the debugger meant that exceptions thrown in `builtins.tryEval` would trigger the debugger.
However, this caught nixpkgs initialization code, which is unhelpful in the majority of cases, so we changed the default.
To get the old behaviour, use `--no-ignore-try`.
```
$ nix repl --debugger --expr 'with import <nixpkgs> {}; pkgs.hello'
Lix 2.94.0-dev-pre20250625-9a59106
Type :? for help.
error: file 'nixpkgs-overlays' was not found in the Nix search path (add it using $NIX_PATH or -I)
This exception occurred in a 'tryEval' call. Use --ignore-try to skip these.
Added 13 variables.
nix-repl>
```
@@ -0,0 +1,25 @@
---
synopsis: "Fallback to safe temp dir when build-dir is unwritable"
issues: [fj#876]
cls: [3501]
category: "Fixes"
credits: ["raito", "horrors"]
---
Non-daemon builds started failing with a permission error after introducing the `build-dir` option:
```
$ nix build --store ~/scratch nixpkgs#hello --rebuild
error: creating directory '/nix/var/nix/builds/nix-build-hello-2.12.2.drv-0': Permission denied
```
This happens because:
1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
2. The user lacks permissions for that path.
We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
**Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.
+47
View File
@@ -0,0 +1,47 @@
---
synopsis: Experimental integer coercion in interpolated strings
issues: []
cls: [3198]
category: "Features"
credits: [raito, delroth, horrors, winter]
---
Ever tried interpolating a port number in Lix and ended up with something like this?
```nix
"http://${config.network.host}:${builtins.toString config.network.port}/"
```
You're not alone. Thousands of Lix users suffer every day from excessive `builtins.toString` syndrome. Its 2025, and we still have to cast integers to use them in strings.
To address this, Lix introduces the **`coerce-integers`** experimental feature. When enabled, interpolated integers within `"${...}"` are automatically coerced to strings. This allows writing:
```nix
"http://${config.network.host}:${config.network.port}/"
```
without additional conversion.
To enable the feature, you need to add `coerce-integers` to your set of experimental features.
### Stabilization criteria
The `coerce-integers` feature is experimental and limited strictly to string interpolation (`"${...}"`). Before stabilization, the following must hold:
1. **Interpolation-only**
Coercion must not occur outside interpolation. Expressions like `"" + 42` must continue to fail.
2. **Expectation that no explicit cast are being observed**
Cases observing explicit coercion (e.g., via `tryEval` gadget or similar) are expected not to be load-bearing in actual production code.
### Timeline for stabilization
If the feature proves safe and is widely adopted across typical usage (e.g., actual configurations in the wild turning on the flag, non-trivial out-of-tree projects using it), the experimental flag will be removed **after six months of active use or two Lix releases**, whichever is longer.
This avoids locking the feature in experimental status indefinitely, as happened with Flakes, while allowing time for validation and ecosystem integration.
### What about coercing floats or more?
Coercion beyond integers -- such as for floats or other types -- is **not planned**, even under an experimental flag. Questions like "what is the canonical string representation of a float?" involve subtle and context-dependent trade-offs. Without a robust and principled mechanism to define and audit such behavior, introducing broader coercion risks setting unintended and hard-to-reverse precedents. The scope of `coerce-integers` is intentionally narrow and will remain so.
In terms of outlook, a proposal like https://git.lix.systems/lix-project/lix/issues/835 could pave the way for a better solution.
@@ -0,0 +1,16 @@
---
synopsis: Parse overflowing JSON number literals as floatingpoint
issues: []
cls: [3919]
category: "Fixes"
credits: [emilazy]
---
Previously, `builtins.fromJSON "-9223372036854775809"` would
return a floatingpoint number, while `builtins.fromJSON
"9223372036854775808"` would cause an evaluation error. This was
introduced with the banning of integer overflow in Lix 2.91; previously
the latter would result in C++ undefined behaviour. These cases are
now treated consistently with JSONs model of a single numeric type,
and JSON number literals that do not fit in a Nixlanguage integer
will be parsed as floatingpoint numbers.
+13
View File
@@ -0,0 +1,13 @@
---
synopsis: "`--keep-failed` chowns the build directory to the user that request the build"
issues: []
cls: []
category: Improvements
credits: [horrors]
---
Running a build with `--keep-failed` now chowns the temporary directory from the
builder user and group to the user that request the build if the build came from
a local user connected to the daemon. This makes inspecting failed derivations a
lot easier. On Linux the build directory made visible to the user will not be in
the same path as it was in the sandbox and continuing builds will usually break.
@@ -0,0 +1,11 @@
---
synopsis: Fix handling of OSC codes in terminal output
issues: [fj#160]
cls: [3143]
category: Fixes
credits: [lilyball]
---
OSC codes in terminal output are now handled correctly, where OSC 8 (hyperlink) is preserved any
time color codes are allowed and all other OSC codes are stripped out. This applies not only to
output from build commands but also to rendered documentation in the REPL.
-10
View File
@@ -1,10 +0,0 @@
---
synopsis: "Allow remote builders to be configured using TOML"
cls: [4533]
category: "Features"
credits: [commentator2.0, Qyriad]
---
Lix now supports configuring remote builders using a TOML file instead of the old, very cursed and incomprehensible format.
This comes with not only a human-understandable file, but also with better messages and error reports on misconfiguration.
A more detailed Documentation can be found on the [distributed-builds](@docroot@/advanced-topics/distributed-builds.md) Wiki-page
+14
View File
@@ -0,0 +1,14 @@
---
synopsis: Better debuggability on fixed-output hash mismatches
issues: []
cls: []
category: Improvements
credits: [lheckemann]
---
Fixed-output derivation hash mismatch error messages will now include the path that was
produced unexpectedly, and this path will be registered as valid even if `--check`
(`nix-store`, `nix-build`) or `--rebuild` (`nix build`) was passed. This makes comparing
the expected path with the obtained path easier, and is useful for debugging when
upstreams modify previously-published releases or when changes in fixed-output
derivations' dependencies affect their output unexpectedly.
-8
View File
@@ -1,8 +0,0 @@
---
synopsis: "Default to showing build logs in the new-style (nix3) CLI"
cls: [4674]
category: "Miscellany"
credits: [k900]
---
Lix will now show logs by default, in addition to the progress bar, when invoked through the new-style "nix3" CLI (`nix build`, etc)
@@ -1,31 +0,0 @@
---
synopsis: "Move /root/.cache/nix to /var/cache/nix by default"
cls: [4671]
issues: [fj#634]
category: "Breaking Changes"
credits: [raito]
---
By default, Lix attempts to locate a cache directory for its operations (such
as the narinfo cache) by checking the value of `$XDG_CACHE_DIR`.
However, since the Nix daemon is a system service, using `$XDG_CACHE_DIR` is
not typical in this context.
To address this, systemd provides a better solution. Specifically, when
`CacheDirectory=` is set in the `[Service]` section of a systemd unit, it
automatically sets the `$CACHE_DIRECTORY` environment variable and systemd will
manage that cache directory for us.
Now, our systemd unit includes `CacheDirectory=nix`, which sets the
`$CACHE_DIRECTORY` and takes precedence over `$XDG_CACHE_DIR`.
If the daemon is run under user units, systemd will automatically set
`$XDG_CACHE_DIR`.
If neither of these variables is set, Lix falls back to its default behavior.
By default, Lix will try to find a cache directory for its various operations
(e.g. narinfo cache) by looking into `$XDG_CACHE_DIR`.
In summary, what was stored in `/root/.cache/nix` is now moved to
`/var/cache/nix/nix`.
+10
View File
@@ -0,0 +1,10 @@
---
synopsis: "nix-eval-jobs: support `--no-instantiate` flag"
issues: [fj#987]
category: Features
credits: [mic92,ma27]
---
`nix-eval-jobs` now supports a flag called `--no-instantiate`. With this enabled,
no write operations on the eval store are performed. That means, only evaluation is
performed, but derivations (and their gcroots) aren't created.
@@ -0,0 +1,29 @@
---
synopsis: "Fix nix develop for derivations that rejects dependencies with structured attrs"
issues: [fj#997]
cls: [4182]
category: Fixes
credits: [raito]
---
For the sake of concision, we refer to `disallowedReferences` in what follows,
but all output checks were equally fixed:
`{dis,}allowed{References,Requisites}`.
Derivations can define *output checks* to reject unwanted dependencies, such as
interpreters like `bash` or compilers like `gcc`. This can be done in two ways:
* **Legacy style**: `disallowedReferences = [ ... ]` in the environment.
* **Structured attrs**: `outputChecks.<output>.disallowedReferences = [ ... ]`,
typically used in `__json`.
Only the structured form supports derivations with multiple outputs.
`nix develop` internally rewrites derivations to create development shells. It
relied on the legacy `disallowedReferences`, and failed to honor the structured
variant. This led to broken shells in cases where `bashInteractive` was
explicitly disallowed using structured output checks, e.g. `nix develop
nixpkgs#systemd` after the "bash-less NixOS" changes.
This fix teaches `nix develop` to respect structured output checks, restoring
support for such derivations.
+12
View File
@@ -0,0 +1,12 @@
---
synopsis: "Add --raw flag to `nix-instantiate --eval` for unescaped output"
issues: []
prs: [gh#12119]
cls: [2886]
category: Improvements
credits: [not-my-profile, infinisil, raito]
---
The `nix-instantiate --eval` command now supports a `--raw` flag. When used,
the result must be coercible to a string (as with `${...}`) and is printed
verbatim, without quotes or escaping.
-9
View File
@@ -1,9 +0,0 @@
---
synopsis: "Add an indication of nix-shell nesting depth"
cls: [4657]
issues: [fj#826]
category: "Improvements"
credits: [thubrecht]
---
When in a nix shell (either via a `nix-shell` or a `nix develop` invocation), a variable `NIX_SHELL_LEVEL` is exported to indicate the nesting depth of nix shells.
@@ -0,0 +1,12 @@
---
synopsis: Allow `nix store ls` to read nar listings from binary cache stores.
issues: []
cls: [3225]
category: Improvements
credits: [vlinkz]
---
The `nix store ls` command now supports reading `.ls` nar listings from binary cache stores.
If a listing is detected for the store path being queried, the nar is no longer downloaded.
These nar listings are available in binary cache stores where the `write-nar-listing` option is
enabled, such as cache.nixos.org.
@@ -0,0 +1,10 @@
---
synopsis: "nix-eval-jobs: retain NIX_PATH"
issues: []
cls: [3859]
category: Fixes
credits: [ma27,mic92]
---
`nix-eval-jobs` doesn't clear the `NIX_PATH` from the environment anymore. This matches the behavior
of [upstream version `2.30`](https://github.com/nix-community/nix-eval-jobs/releases/tag/v2.30.0).
+25
View File
@@ -0,0 +1,25 @@
---
synopsis: "show tree with references that lead to an output cycle"
issues: [fj#551]
category: Improvements
credits: [ma27]
---
When Lix determines a cyclic dependency between several outputs of a derivation,
it now displays which files in which outputs lead to an output cycle:
```
error: cycle detected in build of '/nix/store/gc5h2whz3rylpf34n99nswvqgkjkigmy-demo.drv' in the references of output 'bar' from output 'foo'.
Shown below are the files inside the outputs leading to the cycle:
/nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar
└───lib/libfoo: …stuffbefore /nix/store/h680k7k53rjl9p15g6h7kpym33250w0y-demo-baz andafter.…
→ /nix/store/h680k7k53rjl9p15g6h7kpym33250w0y-demo-baz
└───share/snenskek: …???? /nix/store/dm24c76p9y2mrvmwgpmi64rryw6x5qmm-demo-foo ....…
→ /nix/store/dm24c76p9y2mrvmwgpmi64rryw6x5qmm-demo-foo
└───bin/alarm: …textexttext/nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar abcabcabc.…
→ /nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar
```
Please note that showing the files and its contents while displaying the cycles only works
on Linux.
+20
View File
@@ -0,0 +1,20 @@
---
synopsis: "Fixed output derivations can be run using `pasta` network isolation"
cls: [3452]
issues: [fj#285]
category: "Breaking Changes"
credits: [horrors, puck]
---
Fixed output derivations traditionally run in the host network namespace.
On Linux this allows such derivations to communicate with other sandboxes
or the host using the abstract Unix domains socket namespace; this hasn't
been unproblematic in the past and has been used in two distinct exploits
to break out of the sandbox. For this reason fixed output derivations can
now run in a network namespace (provided by [`pasta`]), restricted to TCP
and UDP communication with the rest of the world. When enabled this could
be a breaking change and we classify it as such, even though we don't yet
enable or require such isolation by default. We may enforce this in later
releases of Lix once we have sufficient confidence that breakage is rare.
[`pasta`]: https://passt.top/
-20
View File
@@ -1,20 +0,0 @@
---
synopsis: Derivations can now be printed in detail in `nix repl`
cls: [3842]
category: Improvements
credits: [Lunaphied]
---
Traditionally derivations printed in the REPL would only print a formatted object
representing the path of the derivation file it refers to. This makes inspecting
the enhanced derivation attribute sets encountered from `mkDerivation` or similar
wrappers more difficult. Even the `:p`/`:print` command would not elaborate attribute sets
tagged as a derivation.
With this change you can now use `:p`/`:print` to directly inspect a derivation
by providing one as the top-level object. Derivation attribute sets will only be
printed two levels deep and internal derivation attrsets will remain in unexpanded
path form as before. `drvAttrs` will also be elided as these attributes are already
present in the top-level attribute set of the derivation. These heuristics provide
a balance between readability and functionality. When the `:p`/`:print` is omitted,
a bare derivation is printed in the path format as before.
@@ -0,0 +1,21 @@
---
synopsis: Remove reliance on Bash for remote stores via SSH
issues: [fj#830, fj#805, fj#304]
cls: [3159]
category: "Fixes"
credits: [raito]
---
The pre-flight `echo started` handshake -- added years ago to catch race conditions -- has been removed.
After removal of connection sharing in Lix 2.93, it required a Bash-compatible shell and a standard `echo`, so it failed on:
* builders protected by `ForceCommand` wrappers (e.g. `nix-remote-build`),
* BusyBox / initrd images with no Bash,
* hosts using non-POSIX shells such as Nushell.
The race the probe once addressed was tied to SSH connection-sharing -- since connection-sharing code has already been removed, the probe is now pointless.
Real connection or protocol errors are now left to SSH/Nix to report directly.
This is technically a breaking change if you had scripts that relied on the literal "started" which needs to be updated to rely on other signals, e.g., exit codes.
+20
View File
@@ -0,0 +1,20 @@
---
synopsis: Remove support for daemon protocols before 2.18
issues: [fj#510]
cls: [3249]
significance: significant
category: "Breaking Changes"
credits: [horrors]
---
Support for daemon wire protocols belonging to Nix 2.17 or older have been
removed. This impacts clients connecting to the local daemon socket or any
remote builder configured using the `ssh-ng` protocol. Builders configured
with the `ssh` protocol are still accessible from clients such as Nix 2.3.
Additionally Lix will not be able to connect to an old daemon locally, and
remote build connections to old daemons is likewise limited to `ssh` urls.
We have decided to take this step because the old protocols are very badly
tested (if at all), maintenance overhead is high, and a number of problems
with their design makes it infeasible to remain backwards compatible while
we move Lix to a more modern RPC mechanism with better versioning support.
+14
View File
@@ -0,0 +1,14 @@
---
synopsis: "`nix eval --write-to` has been removed"
cls: [4045]
issues: [fj#974, fj#227]
category: "Breaking Changes"
credits: [horrors]
---
`nix eval --write-to` has been removed since it was underspecified, not widely
useful, and prone to security-sensitive misbehaviors. The feature was added in
Nix 2.4 purely for internal use in the build system. According to our research
it hasn't found any use outside of some distribution packaging scripts. Please
use structured outputs formats (such as JSON) instead as they have better type
fidelity, don't conflate attributes with paths, and are useful to other tools.
@@ -0,0 +1,17 @@
---
synopsis: Remove impure derivations and dynamic derivations
issues: [fj#815]
cls: [3210]
significance: significant
category: "Breaking Changes"
credits: [horrors]
---
The `impure-derivations` and `dynamic-derivations` experimental feature have
been removed.
New impure or dynamic derivations cannot be created from this point forward, and
any such pre-existing store derivations canot be read or built any more.
Derivation outputs created by building such a derivation are still valid
until garbage collected; existing store derivations can only be garbage
collected.
@@ -1,37 +0,0 @@
---
synopsis: Remove `fetch-closure` experimental feature
issues: [fj#1010]
cls: [4595]
category: "Breaking Changes"
credits: [just1602]
---
The `fetch-closure` experimental feature has been removed.
Outside of allowing the user to import closure from binary cache,
`fetchClosure` also allow you to do the following:
* rewrite non-CA path to CA
* reject non-CA paths at fetching time
* reject CA paths at fetching time
Some people are using those mechanism to prevent users from having to build any
package and force going via the declared cache or as a way to use ancient/old
software without paying the evaluation cost of a second nixpkgs.
Both use cases are somewhat of an antipattern in Nix semantics. If the user
cannot fetch a program directly via the substituter mechanism and fall back to
local build, this is a feature AND a misconfiguration. If the user cannot build
certain derivations because they are too expensive, the build directives should
pass `-j0` or similar.
As for the second usecase, there's a different way to do it that also allows to
have a way to reproduce the paths that are hardcoded in that file, perform
`import (fetchurl "https://my-cache/${hashparts storepath}.drv")` rather, i.e.
an IFD to a possibly well known name. The backend can generate them on the fly
or once, and possess stable names.
Finally, as for the non-CA → CA features, Lix removed ca-derivations.
fetchClosure offers ca-derivations-like features which suffers from similar
shortcomings albeit lessened. It only follows that we should rather deprecate
and remove these capabilities.
@@ -0,0 +1,17 @@
---
synopsis: Remove the `parse-toml-timestamps` experimental feature
category: "Breaking Changes"
credits: [emilazy]
---
The `parse-toml-timestamps` experimental feature has been removed.
This feature used inband signalling to mark timestamps, making it
impossible to unambiguously parse TOML documents. It also exposed
implementationdefined behaviour in the TOML specification that
changed in the toml11 parser library.
Any interface for parsing TOML timestamps suitable for future
stabilization would necessarily involve breaking changes, and there
is no evidence this experimental feature is being relied upon in the
wild, so it has been removed.
+20
View File
@@ -0,0 +1,20 @@
---
synopsis: "repl-overlays now work in the debugger for flakes"
issues: [fj#777]
cls: [3398]
category: Fixes
credits: [jade]
---
Due to a bug, it was previously not possible to use the debugger on flakes with repl-overlays, or with pure evaluation in general:
```
$ nix repl --pure-eval
Lix 2.94.0-dev-pre20250617-87d99da
Type :? for help.
Loading 'repl-overlays'...
error: access to absolute path '/Users/jade/.config/nix/repl.nix' is forbidden in pure eval mode (use '--impure' to override)
```
This is now fixed.
The contents of the repl-overlays file itself (i.e. most typically the top level lambda in it) will be evaluated in impure mode.
It may be necessary to use `builtins.seq` to force the impure operations to happen first if one wants to do impure operations inside a repl-overlays file in pure evaluation mode.
@@ -0,0 +1,19 @@
---
synopsis: "`disallowedRequisites` now reports chains of disallowed requisites"
issues: [fj#334,fj#626,gh#10877]
category: Improvements
credits: [ma27,roberth]
---
When a build fails because of [`disallowedRequisites`](@docroot@/language/advanced-attributes.md#adv-attr-disallowedRequisites), the error message now includes the chain of references that led to the failure. This makes it easier to see in which derivations the chain can be broken, to resolve the problem.
Example:
```
$ nix-build -A hello
error: output '/nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2' is not allowed to refer to the following paths:
/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-glibc-2.40-66
Shown below are chains that lead to the forbidden path(s).
/nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2
└───/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-glibc-2.40-66
```
@@ -0,0 +1,17 @@
---
synopsis: "libstore/binary-cache-store: don't cache narinfo on nix copy, remove negative entry"
issues: []
cls: [3789]
category: Fixes
credits: [ma27]
---
When using e.g. [Snix's nar-bridge](https://snix.dev/docs/components/overview/#nar-bridge) via
an `http`-store, Lix would create cache entries with a wrong URL to the NAR when uploading
a store-path.
This caused hard build failures for Hydra.
Lix doesn't create these entries on upload anymore. Instead, it only removes negative cache entries.
The cache entry for a narinfo is now created the first time, Lix queries the cache
for the previously uploaded store-path again.
-15
View File
@@ -1,15 +0,0 @@
---
synopsis: "Lix daemons are now fully socket-activated on systemd setups"
cls: []
issues: [1030]
category: "Miscellany"
credits: [horrors]
---
When launched by systemd, Lix no longer uses a persistent daemon process and uses systemd socket
activation instead. This is necessary to support the `cgroups` and `auto-allocate-uids` features
and may improve observability of daemon behavior with common systemd-based monitoring solutions.
The old behavior with a single persistent daemon is still available, but disabled by default. It
is not possible to enable both a persistent daemon and socket activation, starting one stops the
other automatically. Existing installations should not require any changes when they're updated.
+10
View File
@@ -0,0 +1,10 @@
---
synopsis: "Lix libraries can now be linked statically"
issues: [fj#789]
cls: [3775, 3778]
category: Fixes
credits: [alois31]
---
Previously the pkg-config files distributed with Lix were only suitable for dynamic linkage, causing "undefined reference to…" linker errors when trying to link statically.
Private dependency information has now been added to make static linkage work as expected without user intervention.
In addition, relevant static libraries are now prelinked to avoid strange failures due to missing static initializers.
+34
View File
@@ -0,0 +1,34 @@
---
synopsis: Symbols reuses once-allocated Value to reduce garbage collected allocations
issues: []
cls: [3308, 3300, 3314, 3310, 3312, 3313]
category: Improvements
credits: [raito, horrors, thubrecht, xokdvium, nan-git]
---
In the Lix evaluator, **symbols** represent immutable strings, like those used
for attribute names.
In evaluator design, such strings are typically [**interned**](https://en.wikipedia.org/wiki/String_interning), stored uniquely
to save memory, and Lix inherits this approach from the original C++ codebase.
However, some builtins, like `builtins.attrNames`, must return a `Value` type
that can represent any Nix value (strings, integers, lists, etc.).
Before this change, these builtins would create lists of `Value` objects by
allocating them through the garbage collector, copying the symbols string
content each time.
This allocation is unnecessary if the interned symbols themselves also hold a
`Value` representation allocated outside the garbage collector, since these
live for the full duration of evaluation.
As a result, this reduces the number of allocations, leading to:
* A significant drop in maximum [resident set memory](https://en.wikipedia.org/wiki/Resident_set_size) (RSS), with some large-scale
tests showing up to 11% (about 500 MiB) savings in large colmena deployments.
* A slight decrease in CPU usage during Nix evaluations.
This change is inspired by https://github.com/NixOS/nix/pull/13258 but the approach is different.
**Note** : [`xokdvium`](https://github.com/xokdvium) is the rightful author of https://gerrit.lix.systems/c/lix/+/3300 and the credit was missed on our end during the development process. We are deeply sorry for this mistake.
@@ -0,0 +1,14 @@
---
synopsis: Reject overflowing TOML integer literals
issues: []
cls: [3916]
category: "Breaking Changes"
credits: [emilazy]
---
The toml11 library used by Lix was updated. The new
version aligns with the [TOML v1.0.0 specifications
requirement](https://toml.io/en/v1.0.0#integer) to reject integer
literals that cannot be losslessly parsed. This means that code like
`builtins.fromTOML "v=0x8000000000000000"` will now produce an error
rather than silently saturating the integer result.
@@ -0,0 +1,15 @@
---
synopsis: uid-range depends on cgroups
issues: []
cls: [3230]
category: "Breaking Changes"
credits: [raito, horrors]
---
`uid-range` builds now depends on `cgroups`, an experimental feature.
`uid-range` builds already depended upon `auto-allocate-uids`, another experimental feature.
The rationale for doing so is that `uid-range` provides a sandbox with many
UIDs, this is useful for re-mapping them into a nested namespace, e.g. a
container.
@@ -0,0 +1,10 @@
---
synopsis: add description to zsh completions
issues: [fj#910]
cls: [3632]
category: "Fixes"
credits: [matthewbauer]
---
Emit descriptions when completing args in zsh completions. This uses the descriptions we already
provided in NIX\_GET\_COMPLETIONS.
-4
View File
@@ -39,9 +39,6 @@
- [Tuning Cores and Jobs](advanced-topics/cores-vs-jobs.md)
- [Verifying Build Reproducibility](advanced-topics/diff-hook.md)
- [Using the `post-build-hook`](advanced-topics/post-build-hook.md)
- [Pasta](advanced-topics/pasta.md)
- [Known Issues](known-issues/known-issues.md)
- [Limitations around non-isolated builds](known-issues/non-isolated-build-limits.md)
- [Command Reference](command-ref/command-ref.md)
- [Common Options](command-ref/opt-common.md)
- [Common Environment Variables](command-ref/env-common.md)
@@ -200,7 +197,6 @@
- [Release Notes](release-notes/release-notes.md)
- [Upcoming release](release-notes/rl-next.md)
<!-- RELENG-AUTO-INSERTION-MARKER (see releng/release_notes.py) -->
- [Lix 2.94 (2025-11-17)](release-notes/rl-2.94.md)
- [Lix 2.93 (2025-05-09)](release-notes/rl-2.93.md)
- [Lix 2.92 (2025-01-18)](release-notes/rl-2.92.md)
- [Lix 2.91 (2024-08-12)](release-notes/rl-2.91.md)
@@ -41,17 +41,106 @@ contains Nix.
> If you are building via the Lix daemon (default on Linux and macOS), it is the Lix daemon user account (that is, `root`) that should have SSH access to a user (not necessarily `root`) on the remote machine.
>
> Furthermore, `root` needs to have the public host keys for the remote system in its `.ssh/known_hosts`.
> To add them to `known_hosts` for root, do `ssh-keyscan HOST | sudo tee -a ~root/.ssh/known_hosts`.
> To add them to `known_hosts` for root, do `ssh-keyscan USER@HOST | sudo tee -a ~root/.ssh/known_hosts`.
>
> If you cant or dont want to configure `root` to be able to access the remote machine, you can use a private Nix store instead by passing e.g. `--store ~/my-nix` when running a Nix command from the local machine.
## Configuration
The list of remote machines can be specified on the command line or in
the Lix configuration file. The former is convenient for testing.
Additionally, there are two supported formats to configure remote builders:
The legacy, "space"-separated format and starting with Lix 2.95.0, a TOML.
the Lix configuration file. The former is convenient for testing. For
example, the following command allows you to build a derivation for
`x86_64-darwin` on a Linux machine:
```console
$ uname
Linux
$ nix build --impure \
--expr '(with import <nixpkgs> { system = "x86_64-darwin"; }; runCommand "foo" {} "uname > $out")' \
--builders 'ssh://mac x86_64-darwin'
[1/0/1 built, 0.0 MiB DL] building foo on ssh://mac
$ cat ./result
Darwin
```
It is possible to specify multiple builders separated by a semicolon or
a newline, e.g.
```console
--builders 'ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd'
```
Each machine specification consists of the following elements, separated
by spaces. Only the first element is required. To leave a field at its
default, set it to `-`.
1. The URI of the remote store in the format
`ssh://[username@]hostname[?port=<port>]`, e.g. `ssh://nix@mac` or `ssh://mac`.
If the ssh server is not listening on port 22 (e.g. port 1337 in this case)
the URI would be `ssh://nix@mac?port=1337`
For backward compatibility, `ssh://` may be omitted. The hostname
may be an alias defined in your `~/.ssh/config`.
2. A comma-separated list of Nix platform type identifiers, such as
`x86_64-darwin`. It is possible for a machine to support multiple
platform types, e.g., `i686-linux,x86_64-linux`. If omitted, this
defaults to the local platform type.
3. The SSH identity file to be used to log in to the remote machine. If
omitted, SSH will use its regular identities.
4. The maximum number of builds that Lix will execute in parallel on
the machine. Typically this should be equal to the number of CPU
cores. For instance, the machine `itchy` in the example will execute
up to 8 builds in parallel.
5. The “speed factor”, indicating the relative speed of the machine. If
there are multiple machines of the right type, Lix will prefer the
fastest, taking load into account.
6. A comma-separated list of *supported features*. If a derivation has
the `requiredSystemFeatures` attribute, then Lix will only perform
the derivation on a machine that has the specified features. For
instance, the attribute
```nix
requiredSystemFeatures = [ "kvm" ];
```
will cause the build to be performed on a machine that has the `kvm`
feature.
7. A comma-separated list of *mandatory features*. A machine will only
be used to build a derivation if all of the machines mandatory
features appear in the derivations `requiredSystemFeatures`
attribute.
8. The (base64-encoded) public host key of the remote machine. If omitted, SSH
will use its regular known-hosts file. Specifically, the field is calculated
via `base64 -w0 /etc/ssh/ssh_host_ed25519_key.pub`.
For example, the machine specification
nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm
nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2
nix@poochie.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 1 2 kvm benchmark
specifies several machines that can perform `i686-linux` builds.
However, `poochie` will only do builds that have the attribute
```nix
requiredSystemFeatures = [ "benchmark" ];
```
or
```nix
requiredSystemFeatures = [ "benchmark" "kvm" ];
```
`itchy` cannot do builds that require `kvm`, but `scratchy` does support
such builds. For regular builds, `itchy` will be preferred over
`scratchy` because it has a higher speed factor.
Remote builders can also be configured in `nix.conf`, e.g.
@@ -70,180 +159,3 @@ option `builders-use-substitutes` in your local `nix.conf`.
To build only on remote builders and disable building on the local
machine, you can use the option `--max-jobs 0`.
---
Each machine specification consists of the following attributes.
How those are combined within the configuration file differs for the formats, and will be explained further down.
1. `uri` (**required**)
The URI of the remote store in the format
`ssh[-ng]://[username@]hostname[?port=<port>]`, e.g. `ssh://nix@mac` or `ssh://mac`.
If the ssh server is not listening on port 22 (e.g. port 1337 in this case)
the URI would be `ssh[-ng]://nix@mac?port=1337`. The hostname
may be an alias defined in your `~/.ssh/config`.
2. `system-types` (**optional**)
A list of Nix platform type identifiers, such as
`x86_64-darwin`. It is possible for a machine to support multiple
platform types, e.g., `i686-linux` and `x86_64-linux`.
Defaults to the local platform type
3. `ssh-key` (**optional**)
The SSH identity file to be used to log in to the remote machine.
Defaults to SSHs regular identities.
4. `jobs` (**optional**)
The maximum number of builds that Lix will execute in parallel on
the machine. Typically, this should be equal to the number of CPU
cores divided by the cores within the target machines configuration, i.e. `jobs * cores ~= cpu cores`
Defaults to 1; must be a positive integer.
5. `speed-factor`
The “speed factor”, indicating the relative speed of the machine. If
there are multiple machines of the right type, Lix will prefer the
fastest, taking load into account.
Defaults to 1; must be a positive float.
6. `supported-features` (**optional**)
A list of *supported features*. If a derivation has
the `requiredSystemFeatures` attribute, then Lix will only schedule
the derivation on a machine that has the specified features. For
example, the attribute
```nix
requiredSystemFeatures = [ "kvm" ];
```
will cause the build to be performed on a machine that has the `kvm`
feature.
Defaults to an empty list.
7. `mandatory-features` (**optional**)
A list of *mandatory features*. A machine will only
be used to build a derivation if all the machines mandatory
features appear in the derivations `requiredSystemFeatures`
attribute.
Defaults to an empty list.
8. `ssh-public-host-key` (**optional**)
The public host key of the remote machine.
Defaults to basic ssh behavior (checking contests of the known-hosts file)
### Using a TOML configuration
Each machine is configured as an attribute within the map called `machines`.
The attributes name is the machines name.
Attributes can be in any order.
For example:
```toml
version = 1
[machines.andesite]
uri = "ssh://lix@andesite.lix.systems" # toml also allows for comments
system-types = ["i686-linux"]
jobs = 8
speed-factor = 1.0
supported-features = ["kvm"]
ssh-key = "/home/deepslate/.ssh/id_ed25519"
[machines.diorite]
uri = "ssh://lix@diorite.lix.systems"
system-types = ["i686-linux"]
jobs = 8
speed-factor = 2.0
ssh-key = "/home/deepslate/.ssh/id_ed25519"
[machines.granite]
uri = "ssh://lix@granite.lix.systems"
system-types = ["i686-linux"]
jobs = 1
speed-factor = 2.0
supported-features = ["kvm", "benchmark"]
ssh-key = "/home/deepslate/.ssh/id_ed25519"
[machines.legacy]
uri = "ssh://nix@nix-15-11.nixos.org"
enable = false
```
> **Note**
>
> If the version tag is omitted (e.g. in the CLI), it defaults to the latest version.
> It is strongly recommended to always provide a version tag for configuration within files to avoid breakage.
For testing purposes, one can also define a builder ad hoc on the CLI as follows:
`--builders 'machines.andesite = {uri = "ssh://lix@andesite.lix.systems", jobs = 8}'`
#### Special handling of fields
- `enable` (**optional**)
If set to false, the declared machine will not be loaded.
This allows one to statically disable machines.
Defaults to true
### Using the legacy format
> **Warning**
>
> This format is frozen and new features / configuration options will not be backported to this format.
It is possible to specify multiple builders separated by a semicolon or
a newline, e.g.
```console
--builders 'ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd'
```
Every machine specification consists of the elements listed in the section above, seperated by any amount of spaces or tabs.
The Attributes need to be provided **in order** and without names.
To leave a field at its default, set it to `-`.
Lists are colon seperated, without additional spaces.
```
lix@andesite.lix.systems i686-linux /home/deepslate/.ssh/id_ed25519 8 1 kvm
lix@diorite.lix.systems i686-linux /home/deepslate/.ssh/id_ed25519 8 2
lix@granite.lix.systems i686-linux /home/deepslate/.ssh/id_ed25519 1 2 kvm benchmark
```
#### Special handling of fields
- `uri`: Due to backward compatibility, the `ssh://` may be omitted for the store-uri.
- `ssh-public-host-key`: The key must be provided encoded in base64. Specifically calculated via `base64 -w0 /etc/ssh/ssh_host_ed25519_key.pub`
### Format detection
At first, the given configuration is being parsed syntactically as a toml.
If parsing fails and the given configuration contains a `"` the error is presented to the user, as those characters are necessary for TOML, but disallowed for the legacy format.
Otherwise, parsing is retried using the legacy format.
If non-syntactic errors are detected within the toml, the exception will always be shown to the user directly.
## Builder selection
The configuration(s) above specify several machines that can perform `i686-linux` builds.
However, `granite` will only do builds that have the attribute
```nix
requiredSystemFeatures = [ "benchmark" ];
```
or
```nix
requiredSystemFeatures = [ "benchmark" "kvm" ];
```
`diorite` cannot do builds that require `kvm`, but `andesite` does support
such builds. For regular builds, `diorite` will be preferred over
`andesite` because it has a higher speed factor.
-19
View File
@@ -1,19 +0,0 @@
# [Pasta](https://passt.top/passt/about/): a network sandbox for fixed-output derivations
## Introduction
This section only applies to **Linux systems** as Pasta is a Linux-only measure.
Since [CVE-2025-46416](https://lix.systems/blog/2025-06-24-lix-cves/), the Lix project decided to adopt [Pasta](https://passt.top/passt/about/) for all fixed-output derivations, protecting against various attack vectors such as UNIX abstract domain sockets or more manipulation at the network layer from a malicious fixed-output derivation code.
Pasta acts as a translation layer between a layer-2 network interface and layer-4 sockets (TCP, UDP, ICMP/ICMPv6 echo) on the host. It requires no special privileges and can serve as a alternative to [SLiRP](https://en.wikipedia.org/wiki/Slirp) which was used [by Guix to mitigate the same problem](https://codeberg.org/guix/guix/commit/fb42611b8f27960304db5a1c0d33b8371dcde2a8).
## How to disable Pasta?
It's sufficient to pass `pasta-path = ""` in your `/etc/nix/nix.conf` or on the command line `--pasta-path ""` of a Lix invocation.
## Known issues surrounding Pasta
- Only the first DNS server in `/etc/resolv.conf` is considered: failover is not possible.
- [Reduced feature set compared to the Linux kernel](https://passt.top/passt/about/#features)
- [Performance overhead in multi-gigabits contexts and IMIX MTUs](https://passt.top/passt/about/#performance_1)
+1 -1
View File
@@ -58,7 +58,7 @@ $ nix-build flake:nixpkgs -A firefox
$ nix-build flake:github:NixOS/nixpkgs/release-23.11 -A firefox
```
Finally, for legacy reasons, if a path starts with `channel:`, the rest of the argument is interpreted as the name of a *nixpkgs* channel tarball to fetch from `https://channels.nixos.org/$CHANNEL_NAME/nixexprs.tar.xz`.
Finally, for legacy reasons, if a path starts with `channel:`, the rest of the argument is interpreted as the name of a *nixpkgs* channel tarball to fetch from `https://nixos.org/channels/$CHANNEL_NAME/nixexprs.tar.xz`.
This is a **hard coded URL** pattern and is *not* related to the subscribed channels managed by the [nix-channel](./nix-channel.md) command.
> **Note**: any of the special syntaxes may always be disambiguated by prefixing the path.
+3 -3
View File
@@ -11,7 +11,7 @@
Channels are a mechanism for referencing remote Nix expressions and conveniently retrieving their latest version.
The moving parts of channels are:
- The official channels listed at <https://channels.nixos.org>
- The official channels listed at <https://nixos.org/channels>
- The user-specific list of [subscribed channels](#subscribed-channels)
- The [downloaded channel contents](#channels)
- The [Nix expression search path](@docroot@/command-ref/conf-file.md#conf-nix-path), set with the [`-I` option](#opt-I) or the [`NIX_PATH` environment variable](#env-NIX_PATH)
@@ -77,9 +77,9 @@ This command has the following operations:
Subscribe to the Nixpkgs channel and run `hello` from the GNU Hello package:
```console
$ nix-channel --add https://channels.nixos.org/nixpkgs-unstable
$ nix-channel --add https://nixos.org/channels/nixpkgs-unstable
$ nix-channel --list
nixpkgs https://channels.nixos.org/nixpkgs
nixpkgs https://nixos.org/channels/nixpkgs
$ nix-channel --update
$ nix-shell -p hello --run hello
hello
-23
View File
@@ -141,21 +141,6 @@ To inspect the canonical source of truth on what the state of the buildsystem co
$ meson introspect
```
#### LLD
The development shell on Linux uses LLD by default for faster link times.
This is set using `mesonFlags`, so to override it, you can simplify re-specify the linker to Meson:
```bash
$ just setup-custom -Dc_link_args=-fuse-ld=ld -Dcpp_link_args=-fuse-ld=ld
```
While using LLD, you may find it helpful to use ThinLTO for even further improvements to link times for incremental builds:
```bash
$ just setup-custom -Db_lto=true -Db_lto_mode=thin -Db_thinlto_cache=true
```
## Sending changes to Gerrit for review {#sending-to-gerrit}
We use Gerrit for all our code review in Lix.
@@ -185,14 +170,6 @@ Once you get a `Code-Review+2` vote on your change, it's rebased on `main` and C
If the change appears as "Rebase Required", you need to rebase it on `main` locally or via the Gerrit UI and wait for `Verified+1` before the Submit button is made active
The `Code-Review+2` from before will stick around through trivial rebases so no need to re-request review for a mere rebase.
## Interacting with the CI, Buildkite
We use Buildkite for our CI, usually you will not have to interact directly with it other than reviewing any errors it produces, which are linked from Gerrit.
However in certain cases a CI run will fail due to transient issues not related to your code and you will need to rerun it by hand.
You can log in to the CI via [SSO](https://buildkite.com/sso/lix-project). On your job you can then hit the "Retry failed" button to rerun it, normally you will not have a repeat of the transient issue.
If the build still fails on CI issues or all builds are failing this should be reported via [Zulip on #T-infra](https://zulip.lix.systems/#narrow/channel/7-T-infra) or [Matrix on #dev](https://matrix.to/#/%23dev%3Alix.systems?via=lix.systems).
## Building Lix with `nix`
To build a release version of Lix for the current operating system and CPU architecture:
+1 -6
View File
@@ -62,12 +62,6 @@ For `installcheck` specifically, first run `just install` before running the tes
Finer-grained filtering within a test suite is also possible using the [--gtest_filter](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) command-line option to a test suite executable, or the `GTEST_FILTER` environment variable.
### Inspecting failures
The test suite emits logs in `build/meson-logs/`; the full textual failure logs are in `build/meson-logs/testlog.txt`.
If you want a much nicer experience of viewing the logs in a structured manner, use `xunit-viewer --results build/meson-logs/testlog.junit.xml --server` to view them in a web browser.
### Unit test support libraries
There are headers and code which are not just used to test the library in question, but also downstream libraries.
@@ -401,6 +395,7 @@ I grepped `lix/` for `get[eE]nv\("` to find the mentions in Lix code.
**Expected value**: the path to an executable shell
- `PRINT_PATH` - Undocumented. Used by `nix-prefetch-url` as an alternative form of `--print-path`. Why???
- `_NIX_IN_TEST` - If present with any value, makes `fetchClosure` accept file URLs in addition to HTTP ones. Why is this not `_NIX_FORCE_HTTP`??
Not used anywhere else.
- `NIX_ALLOW_EVAL` - Used by eval-cache tests to block evaluation if set to `0`.
@@ -60,10 +60,3 @@ Then:
```console
$ docker run -ti lix
```
# Known issues
Lix in Docker is very sensitive to **functional** DNS resolution if you are running with [Pasta protections](../advanced-topics/pasta.md) which are enabled by default since Lix 2.93.0 on most distributions.
If you notice failure to download things, double check whether your **first** DNS entry in `/etc/resolv.conf` is functional.
Lix with [Pasta protections](../advanced-topics/pasta.md) does not support failing over the next entries.
@@ -1 +0,0 @@
This section lists known issues around Lix.
@@ -1,21 +0,0 @@
# Limitations of non-isolated builds
## What are non-isolated builds?
In Lix, only builds done on Linux with `sandbox = true` and a functioning
`pasta-path` are isolated from the rest of the system, all other builds are
considered non-isolated to some degree.
For example, running Lix with [Pasta](@docroot@/advanced-topics/pasta.md)
disabled makes the host network visible to fixed-output derivations, reducing
isolation somewhat.
## Clean termination of non-isolated builds
Non-isolated builds may not terminate cleanly in all cases due to limitations in Lix's process management.
This occurs when a build keeps the build log file descriptor open past the end of the actual build. A common cause of this are background tasks that aren't properly terminated before the main build process exits, for example: HTTP servers run as part of a test suite.
See [issue #1018](https://git.lix.systems/lix-project/lix/issues/1018) for an example.
The only solution is to manually terminate leftover processes in your derivation, including during failure scenarios.
@@ -5,7 +5,7 @@
FIXME(Lix): This section does not document the most common modern practices in terms of avoiding channels, pinning, declarative software installation (see flakey-profile or home-manager or NixOS), or using flakes, etc.
It is, however, likely correct at a technical level.
For more information on modern practices, see the [resources](https://wiki.lix.systems/books/lix-users/page/nix-resources) page on the Lix site.
For more information on modern practices, see the [resources](https://lix.systems/resources) page on the Lix site.
</div>
@@ -41,7 +41,7 @@ install Lix. If this is not the case for some reason, you can add it
as follows:
```console
$ nix-channel --add https://channels.nixos.org/nixpkgs-unstable
$ nix-channel --add https://nixos.org/channels/nixpkgs-unstable
$ nix-channel --update
```
@@ -49,7 +49,7 @@ $ nix-channel --update
>
> On NixOS, youre automatically subscribed to a NixOS channel
> corresponding to your NixOS major release (e.g.
> <https://channels.nixos.org/nixos-21.11>). A NixOS channel is identical
> <http://nixos.org/channels/nixos-21.11>). A NixOS channel is identical
> to the Nixpkgs channel, except that it contains only Linux binaries
> and is updated only if a set of regression tests succeed.
-955
View File
@@ -1,955 +0,0 @@
# Lix 2.94 "Açaí na tigela" (2025-11-17)
# Lix 2.94.0 (2025-11-17)
## Breaking Changes
- Remove support for daemon protocols before 2.18 [fj#510](https://git.lix.systems/lix-project/lix/issues/510) [cl/3249](https://gerrit.lix.systems/c/lix/+/3249)
Support for daemon wire protocols belonging to Nix 2.17 or older have been
removed. This impacts clients connecting to the local daemon socket or any
remote builder configured using the `ssh-ng` protocol. Builders configured
with the `ssh` protocol are still accessible from clients such as Nix 2.3.
Additionally Lix will not be able to connect to an old daemon locally, and
remote build connections to old daemons is likewise limited to `ssh` urls.
We have decided to take this step because the old protocols are very badly
tested (if at all), maintenance overhead is high, and a number of problems
with their design makes it infeasible to remain backwards compatible while
we move Lix to a more modern RPC mechanism with better versioning support.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Remove impure derivations and dynamic derivations [fj#815](https://git.lix.systems/lix-project/lix/issues/815) [cl/3210](https://gerrit.lix.systems/c/lix/+/3210)
The `impure-derivations` and `dynamic-derivations` experimental feature have
been removed.
New impure or dynamic derivations cannot be created from this point forward, and
any such pre-existing store derivations canot be read or built any more.
Derivation outputs created by building such a derivation are still valid
until garbage collected; existing store derivations can only be garbage
collected.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- First argument to `--arg`/`--argstr` must be a valid Nix identifier [fj#496](https://git.lix.systems/lix-project/lix/issues/496)
The first argument to `--arg`/`--argstr` must be a valid Nix identifier, i.e.
`nix-build --arg config.allowUnfree true` is now rejected.
This is because that invocation is a false friend since it doesn't set
`{ config = { allowUnfree = true; }; }`, but `{ "config.allowUnfree" = true; }`.
The idea is to change the behavior to the latter in the long-term. For that,
non-identifiers started giving a warning since 2.92 and are now rejected to give people
who depend on that a chance to notice and potentially weigh in on the discussion.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- New cgroup delegation model [fj#537](https://git.lix.systems/lix-project/lix/issues/537) [fj#77](https://git.lix.systems/lix-project/lix/issues/77) [cl/3230](https://gerrit.lix.systems/c/lix/+/3230)
Builds using cgroups (i.e. `use-cgroups = true` and the experimental feature
`cgroups`) now always delegate a cgroup tree to the sandbox.
Compared to the original C++ Nix project, our delegation includes the
`subtree_control` file as well, which means that the sandbox can disable
certain controllers in its own cgroup tree.
This is a breaking change because this requires the Nix daemon to run with an
already delegated cgroup tree by the service manager.
## How to setup the cgroup tree with systemd?
systemd offers knobs to perform the required setup using:
```
[Service]
Delegate=yes
DelegateSubtree=supervisor
```
These directives are now included in our systemd packaging.
## What about using Nix as root without connecting to the daemon?
Builds run as `root` without connecting to the daemon relying on the cgroup
feature are now broken, i.e.
```console
# nix-build --use-cgroups --sandbox ... # will not work
```
Consider doing instead:
```console
# systemd-run --same-dir --wait -p Delegate=yes -p DelegateSubgroup=supervisor nix-build --use-cgroups ...
```
If you need to disable cgroups temporarily, remember that you can do
`NIX_CONF='include /etc/nix/nix.conf\nuse-cgroups = false' nix-build ...` or
`nix-build --no-use-cgroups ...`.
## What about other service managers than systemd?
systemd has a [documentation](https://systemd.io/CGROUP_DELEGATION/) on how to
handle cgroup delegation from service management perspective.
If your service manager adheres to systemd semantics, e.g. writing an extended
attribute `user.delegate=1` on the delegated cgroup tree directory and moving
the `nix-daemon` process inside a cgroup tree to respect the inner process
rule, then, the feature will work as well.
## Why is the cgroup feature still experimental?
While the cgroup feature unlocks many use cases, its behavior and integration (e.g. user experience), especially at scale on build farms or in multi-tenant environments, are not yet fully matured. Theres also potential for deeper systemd integration (e.g. using slices and scopes) that has not been fully explored.
To avoid locking in an unstable interface, were keeping the experimental flag until we have validated the feature across a broader range of scenarios, including but not limited to:
* Nix as root
* Hydra-style build farms
* Forgejo CI runners
* Shared remote builders
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [eldritch horrors](https://git.lix.systems/pennae), and [lheckemann](https://git.lix.systems/lheckemann) for this.
- Enable high compress ratio zstd compression by default for binary caches uploads [fj#945](https://git.lix.systems/lix-project/lix/issues/945) [cl/4503](https://gerrit.lix.systems/c/lix/+/4503)
The default compression method for binary cache uploads has been switched from
[`xz`](https://github.com/tukaani-project/xz) to
[`zstd`](https://github.com/facebook/zstd) to address performance and usability
issues related to modern hardware and high-speed connections.
## Why?
`xz` offers compression ratios but is single-threaded in our implementation and
very slow (~10-20 Mbps in our test), preventing full utilization of 100Mbps+
connections and significantly slowing decompression for end users.
Lix is a "compress once, decompress many" application: build farms can afford
to spend more time compressing to achieve a faster download transfer for the
end user. More importantly, it matters that all end users spend the least
amount of time decompressing.
## What about compression ratios?
`zstd` cannot achieve the same peaks as `xz`, nonetheless, `zstd` compression
level has been increased to level 12 by default to balance compression ratio
and performance.
## Synthetic test case data
* **xz** (default compression level) on a 4.4GB file: ~632MB (77s)
* **zstd** (level 12) on the same file: ~775MB (18s), 18% larger but 50% faster
* **zstd** (level 14): ~773MB (37s)
* **zstd** (level 16): ~735MB (66s)
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [Raito Bezarius](https://git.lix.systems/raito) for this.
- Repl debugger uses `--ignore-try` by default [lix#666](https://git.lix.systems/lix-project/lix/issues/666) [cl/3488](https://gerrit.lix.systems/c/lix/+/3488)
Previously, using the debugger meant that exceptions thrown in `builtins.tryEval` would trigger the debugger.
However, this caught nixpkgs initialization code, which is unhelpful in the majority of cases, so we changed the default.
To get the old behaviour, use `--no-ignore-try`.
```
$ nix repl --debugger --expr 'with import <nixpkgs> {}; pkgs.hello'
Lix 2.94.0-dev-pre20250625-9a59106
Type :? for help.
error: file 'nixpkgs-overlays' was not found in the Nix search path (add it using $NIX_PATH or -I)
This exception occurred in a 'tryEval' call. Use --ignore-try to skip these.
Added 13 variables.
nix-repl>
```
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Strings may now contain NUL bytes [cl/3968](https://gerrit.lix.systems/c/lix/+/3968)
Lix now allows strings to contain NUL bytes instead of silently truncating the
string before the first such byte. Notably NUL-bearing strings were allowed as
attribute names—even though the corresponding strings were not representable!—
leading to very surprising and incorrect behavior in corner cases, for example
```
nix-repl> builtins.fromJSON ''{"a": 1, "a\u0000b": 2}''
{
a = 1;
"ab" = 2;
}
nix-repl> builtins.attrNames (builtins.fromJSON ''{"a": 1, "a\u0000b": 2}'')
[
"a"
"a"
]
```
rather than the more correct but still with the terminal eating NUL on display
```
nix-repl> builtins.fromJSON ''{"a": 1, "a\u0000b": 2}''
{
a = 1;
"ab" = 2;
}
nix-repl> builtins.attrNames (builtins.fromJSON ''{"a": 1, "a\u0000b": 2}'')
[
"a"
"ab"
]
```
We consider this a breaking change since eval results *will* change if strings
with embedded NUL bytes were used, but we also consider the old behavior to be
not intentional (seeing how inconsistent it was) but merely fallout from a old
and misguided implementation decision to be worked around, not actually fixed.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Fixed output derivations can be run using `pasta` network isolation [fj#285](https://git.lix.systems/lix-project/lix/issues/285) [cl/3452](https://gerrit.lix.systems/c/lix/+/3452)
Fixed output derivations traditionally run in the host network namespace.
On Linux this allows such derivations to communicate with other sandboxes
or the host using the abstract Unix domains socket namespace; this hasn't
been unproblematic in the past and has been used in two distinct exploits
to break out of the sandbox. For this reason fixed output derivations can
now run in a network namespace (provided by [`pasta`]), restricted to TCP
and UDP communication with the rest of the world. When enabled this could
be a breaking change and we classify it as such, even though we don't yet
enable or require such isolation by default. We may enforce this in later
releases of Lix once we have sufficient confidence that breakage is rare.
[`pasta`]: https://passt.top/
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [puck](https://git.lix.systems/puck) for this.
- Function equality semantics are more consistent, but still bad [cl/4556](https://gerrit.lix.systems/c/lix/+/4556) [cl/4244](https://gerrit.lix.systems/c/lix/+/4244)
Lix has inherited a historic misfeature from CppNix in the form of pointer
equality checks built into the `==` operator. These checks were originally
meant to optimize comparison for large sets, but they have the unfortunate
side effect of producing unexpected results when sets containing functions
are compared. **Lix 2.93 and earlier** behave as shown in the repl session
```
Lix 2.93.3
Type :? for help.
nix-repl> f = x: x
Added f.
nix-repl> f == f
false
nix-repl> let s.f = f; in s.f == s.f
false
nix-repl> # however!
{ inherit f; } == { inherit f; }
true
nix-repl> [ f ] == [ f ]
true
nix-repl> # and, in another twist:
[ f ] == map f [ f ]
false
```
Nixpkgs relies on sets containing functions being comparable, so we cannot
simply deprecate this behavior. Due to changes to the object model used by
Lix ***all* comparisons above now evaluate to `true`**. This is considered
a breaking change because eval results may differ, but we also consider it
minor because the optimization is unsound (c.f. `let l = [NaN]; in l == l`
evaluates to `true` even though floating point `NaN` is incomparable). Lix
intends to remove this optimization altogether in the future, but until we
can do that we instead make it slightly less broken to allow other, *real*
optimizations. Function equality comparison remains **undefined behavior**
and should not be relied upon in Nixlang code that intends to be portable.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- `nix eval --write-to` has been removed [fj#974](https://git.lix.systems/lix-project/lix/issues/974) [fj#227](https://git.lix.systems/lix-project/lix/issues/227) [cl/4045](https://gerrit.lix.systems/c/lix/+/4045)
`nix eval --write-to` has been removed since it was underspecified, not widely
useful, and prone to security-sensitive misbehaviors. The feature was added in
Nix 2.4 purely for internal use in the build system. According to our research
it hasn't found any use outside of some distribution packaging scripts. Please
use structured outputs formats (such as JSON) instead as they have better type
fidelity, don't conflate attributes with paths, and are useful to other tools.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Remove the `parse-toml-timestamps` experimental feature
The `parse-toml-timestamps` experimental feature has been removed.
This feature used inband signalling to mark timestamps, making it
impossible to unambiguously parse TOML documents. It also exposed
implementationdefined behaviour in the TOML specification that
changed in the toml11 parser library.
Any interface for parsing TOML timestamps suitable for future
stabilization would necessarily involve breaking changes, and there
is no evidence this experimental feature is being relied upon in the
wild, so it has been removed.
Many thanks to [Emily](https://git.lix.systems/emilazy) for this.
- Reject overflowing TOML integer literals [cl/3916](https://gerrit.lix.systems/c/lix/+/3916)
The toml11 library used by Lix was updated. The new
version aligns with the [TOML v1.0.0 specifications
requirement](https://toml.io/en/v1.0.0#integer) to reject integer
literals that cannot be losslessly parsed. This means that code like
`builtins.fromTOML "v=0x8000000000000000"` will now produce an error
rather than silently saturating the integer result.
Many thanks to [Emily](https://git.lix.systems/emilazy) for this.
- uid-range depends on cgroups [cl/3230](https://gerrit.lix.systems/c/lix/+/3230)
`uid-range` builds now depends on `cgroups`, an experimental feature.
`uid-range` builds already depended upon `auto-allocate-uids`, another experimental feature.
The rationale for doing so is that `uid-range` provides a sandbox with many
UIDs, this is useful for re-mapping them into a nested namespace, e.g. a
container.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
## Features
- Add `inputs.self.submodules` flake attribute [fj#942](https://git.lix.systems/lix-project/lix/issues/942) [cl/3839](https://gerrit.lix.systems/c/lix/+/3839)
A port of <https://github.com/NixOS/nix/pull/12421> to Lix, which:
- adds a general `inputs.self` flake attribute that retroactively applies
configurations to a flake after it's been fetched, then triggers a refetch of
the flake with the new config.
- implements `inputs.self.submodules` that allows a flake to declare its need
for submodules, which are then fetched automatically with no need to pass
`?submodules=1` anywhere.
Many thanks to [Eelco Dolstra](https://github.com/edolstra) and [ورد](https://git.lix.systems/janw4ld) for this.
- Lix supports HTTP/3 behind `--http3` [fj#1033](https://git.lix.systems/lix-project/lix/issues/1033)
Lix now supports HTTP/3 for file transfers when the linked curl version
supports it.
By default, HTTP/3 is disabled notably due to performance issues reported in
mid-2024. [More details
here](https://daniel.haxx.se/blog/2024/06/10/http-3-in-curl-mid-2024/).
As of 2025-11-14, [NixOS official cache](https://cache.nixos.org) supports
HTTP/3 via Fastly. [More info
here](https://github.com/NixOS/infra/commit/157fa70e46afbd6338a32407be461fce05c57bf8).
To enable HTTP/3:
* Use `--http3` for individual transfers.
* Add `http3 = true` in your Nix configuration for permanent activation.
To disable it, use `--no-http3`.
**Note**:
* `--no-http2 --http3` will still enable both HTTP/2 and HTTP/3.
* `--http2 --http3` will prioritize HTTP/3 and fall back to HTTP/2 (and then
HTTP/1.1).
These are current CLI limitations. In the future, we plan to replace `--httpX`
options with `--max-http-version [1,2,3]` for easier version selection in Lix
transfers.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Add hyperlinks in attr set printing [cl/3790](https://gerrit.lix.systems/c/lix/+/3790)
The attribute set printer, such as is seen in `nix repl` or in type errors, now prints hyperlinks on each attribute name to its definition site if it is known.
Example: all of the attributes shown here are hyperlinks to the exact definition site of the attribute in question:
```
$ nix eval -f '<nixpkgs>' lib.licenses.mit
{ deprecated = false; free = true; fullName = "MIT License"; redistributable = true; shortName = "mit"; spdxId = "MIT"; url = "https://spdx.org/licenses/MIT.html"; }
```
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Experimental integer coercion in interpolated strings [cl/3198](https://gerrit.lix.systems/c/lix/+/3198)
Ever tried interpolating a port number in Lix and ended up with something like this?
```nix
"http://${config.network.host}:${builtins.toString config.network.port}/"
```
You're not alone. Thousands of Lix users suffer every day from excessive `builtins.toString` syndrome. Its 2025, and we still have to cast integers to use them in strings.
To address this, Lix introduces the **`coerce-integers`** experimental feature. When enabled, interpolated integers within `"${...}"` are automatically coerced to strings. This allows writing:
```nix
"http://${config.network.host}:${config.network.port}/"
```
without additional conversion.
To enable the feature, you need to add `coerce-integers` to your set of experimental features.
### Stabilization criteria
The `coerce-integers` feature is experimental and limited strictly to string interpolation (`"${...}"`). Before stabilization, the following must hold:
1. **Interpolation-only**
Coercion must not occur outside interpolation. Expressions like `"" + 42` must continue to fail.
2. **Expectation that no explicit cast are being observed**
Cases observing explicit coercion (e.g., via `tryEval` gadget or similar) are expected not to be load-bearing in actual production code.
### Timeline for stabilization
If the feature proves safe and is widely adopted across typical usage (e.g., actual configurations in the wild turning on the flag, non-trivial out-of-tree projects using it), the experimental flag will be removed **after six months of active use or two Lix releases**, whichever is longer.
This avoids locking the feature in experimental status indefinitely, as happened with Flakes, while allowing time for validation and ecosystem integration.
### What about coercing floats or more?
Coercion beyond integers -- such as for floats or other types -- is **not planned**, even under an experimental flag. Questions like "what is the canonical string representation of a float?" involve subtle and context-dependent trade-offs. Without a robust and principled mechanism to define and audit such behavior, introducing broader coercion risks setting unintended and hard-to-reverse precedents. The scope of `coerce-integers` is intentionally narrow and will remain so.
In terms of outlook, a proposal like https://git.lix.systems/lix-project/lix/issues/835 could pave the way for a better solution.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [delroth](https://github.com/delroth), [eldritch horrors](https://git.lix.systems/pennae), and [winter](https://git.lix.systems/winter) for this.
- nix-eval-jobs: support `--no-instantiate` flag [fj#987](https://git.lix.systems/lix-project/lix/issues/987)
`nix-eval-jobs` now supports a flag called `--no-instantiate`. With this enabled,
no write operations on the eval store are performed. That means, only evaluation is
performed, but derivations (and their gcroots) aren't created.
Many thanks to [mic92](https://github.com/mic92) and [ma27](https://git.lix.systems/ma27) for this.
## Improvements
- Assess current profile generations pointers in `nix doctor` [cl/3108](https://gerrit.lix.systems/c/lix/+/3108)
Added a new check to `nix doctor` that verifies whether the current generation of
a Nix profile can be resolved. This helps users diagnose issues with broken or
misconfigured profile symlinks.
This helps determining if you have broken symlinks or misconfigured packaging.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Improved susbtituter query speed
The code used to query substituters for derivations has been rewritten slightly
to take advantage of our asynchronous runtime. Such queries run for every build
that could download from substituters and processes every derivation that isn't
yet present on the local system. Previously Lix would use `http-connections` to
limit query concurrency, even for modern caches that support HTTP/2 and have no
limit on how many queries can be run concurrently on one single connection. Lix
no longer does this, resulting in approximately 60% reduction in query time for
medium-sized closures (e.g. NixOS system closures) during testing, although the
exact number depends greatly on local network latency and generally improves as
latency increases. Unlike previously setting `http-connections` to `1` or other
low values no longer brings a massive penalty in query performance if the cache
in use by the querying system supports HTTP/2 (as e.g. `cache.nixos.org` does).
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Hitting Control-C twice always terminates Lix [cl/3574](https://gerrit.lix.systems/c/lix/+/3574)
Hitting Control-C or sending `SIGINT` to Lix now prints an informational message
if it is still running after on second, the second Control-C/`SIGINT` terminates
Lix immediately without waiting for any shutdown code to finish running. Lix did
not treat the second such event differently from first in the past; this made it
impossible to easily terminate running Lix processes that got stuck in e.g. very
expensive Nixlang code that never interacted with the store. We now terminate as
soon as the user hits Control-C again without waiting any more, to much the same
effect as putting Lix into the background and killing it immediately afterwards.
This means you can now more conveniently break out of stuck Nixlang evaluations:
```
nix-instantiate --eval --expr 'let f = n: if n == 0 then 0 else f (n - 1) + f (n - 1); in f 32'
^CStill shutting down. Press ^C again to abort all operations immediately.
^C
❌130
```
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- `--keep-failed` chowns the build directory to the user that request the build
Running a build with `--keep-failed` now chowns the temporary directory from the
builder user and group to the user that request the build if the build came from
a local user connected to the daemon. This makes inspecting failed derivations a
lot easier. On Linux the build directory made visible to the user will not be in
the same path as it was in the sandbox and continuing builds will usually break.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Better debuggability on fixed-output hash mismatches
Fixed-output derivation hash mismatch error messages will now include the path that was
produced unexpectedly, and this path will be registered as valid even if `--check`
(`nix-store`, `nix-build`) or `--rebuild` (`nix build`) was passed. This makes comparing
the expected path with the obtained path easier, and is useful for debugging when
upstreams modify previously-published releases or when changes in fixed-output
derivations' dependencies affect their output unexpectedly.
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Add --raw flag to `nix-instantiate --eval` for unescaped output [gh#12119](https://github.com/NixOS/nix/pull/12119) [cl/2886](https://gerrit.lix.systems/c/lix/+/2886)
The `nix-instantiate --eval` command now supports a `--raw` flag. When used,
the result must be coercible to a string (as with `${...}`) and is printed
verbatim, without quotes or escaping.
Many thanks to [Martin Fischer](https://github.com/not-my-profile), [infinisil](https://github.com/infinisil), and [Raito Bezarius](https://git.lix.systems/raito) for this.
- Allow `nix store ls` to read nar listings from binary cache stores. [cl/3225](https://gerrit.lix.systems/c/lix/+/3225)
The `nix store ls` command now supports reading `.ls` nar listings from binary cache stores.
If a listing is detected for the store path being queried, the nar is no longer downloaded.
These nar listings are available in binary cache stores where the `write-nar-listing` option is
enabled, such as cache.nixos.org.
Many thanks to [Victor Fuentes](https://git.lix.systems/vlinkz) for this.
- show tree with references that lead to an output cycle [fj#551](https://git.lix.systems/lix-project/lix/issues/551)
When Lix determines a cyclic dependency between several outputs of a derivation,
it now displays which files in which outputs lead to an output cycle:
```
error: cycle detected in build of '/nix/store/gc5h2whz3rylpf34n99nswvqgkjkigmy-demo.drv' in the references of output 'bar' from output 'foo'.
Shown below are the files inside the outputs leading to the cycle:
/nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar
└───lib/libfoo: …stuffbefore /nix/store/h680k7k53rjl9p15g6h7kpym33250w0y-demo-baz andafter.…
→ /nix/store/h680k7k53rjl9p15g6h7kpym33250w0y-demo-baz
└───share/snenskek: …???? /nix/store/dm24c76p9y2mrvmwgpmi64rryw6x5qmm-demo-foo ....…
→ /nix/store/dm24c76p9y2mrvmwgpmi64rryw6x5qmm-demo-foo
└───bin/alarm: …textexttext/nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar abcabcabc.…
→ /nix/store/3lrgm74j85nzpnkz127rkwbx3fz5320q-demo-bar
```
Please note that showing the files and its contents while displaying the cycles only works
on Linux.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- Lix now enables parallel marking in boehm-gc [fj#983](https://git.lix.systems/lix-project/lix/issues/983) [cl/3880](https://gerrit.lix.systems/c/lix/+/3880)
This brings a fairly modest performance improvement (~38% for `nixpkgs search hello`) to evaluation, especially in scenarios that necessitate larger heap sizes.
Many thanks to [Eelco Dolstra](https://github.com/edolstra) and [Seth Flynn](https://git.lix.systems/getchoo) for this.
- `disallowedRequisites` now reports chains of disallowed requisites [fj#334](https://git.lix.systems/lix-project/lix/issues/334) [fj#626](https://git.lix.systems/lix-project/lix/issues/626) [gh#10877](https://github.com/NixOS/nix/issues/10877)
When a build fails because of [`disallowedRequisites`](@docroot@/language/advanced-attributes.md#adv-attr-disallowedRequisites), the error message now includes the chain of references that led to the failure. This makes it easier to see in which derivations the chain can be broken, to resolve the problem.
Example:
```
$ nix-build -A hello
error: output '/nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2' is not allowed to refer to the following paths:
/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-glibc-2.40-66
Shown below are chains that lead to the forbidden path(s).
/nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2
└───/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-glibc-2.40-66
```
Many thanks to [ma27](https://git.lix.systems/ma27) and [Robert Hensing](https://github.com/roberth) for this.
- Stack traces now summarize involved derivations at the bottom [cl/4493](https://gerrit.lix.systems/c/lix/+/4493)
When evaluation errors and a stack trace is printed,
For example, if I add Nheko to a NixOS `environment.systemPackages` without adding `olm-3.2.16` `nixpkgs.config.permittedInsecurePackages`, then without `--show-trace`, I previously got this:
```
error:
… while calling the 'head' builtin
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/attrsets.nix:1701:13:
1700| if length values == 1 || pred here (elemAt values 1) (head values) then
1701| head values
| ^
1702| else
… while evaluating the attribute 'value'
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/modules.nix:1118:7:
1117| // {
1118| value = addErrorContext "while evaluating the option `${showOption loc}':" value;
| ^
1119| inherit (res.defsFinal') highestPrio;
(stack trace truncated; use '--show-trace' to show the full trace)
error: Package olm-3.2.16 in /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/pkgs/by-name/ol/olm/package.nix:37 is marked as insecure, refusing to evaluate.
< -snip the whole explanation about olm's CVEs- >
```
This doesn't tell me anything about where `olm-3.2.16` came from.
With `--show-trace`, there's 1155 lines to sift through, but does contain lines like "while evaluating derivation 'nheko-0.12.1'".
With this change, those lines are summarized and collected at the bottom, regardless of `--show-trace`:
```
error:
… while calling the 'head' builtin
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/attrsets.nix:1701:13:
1700| if length values == 1 || pred here (elemAt values 1) (head values) then
1701| head values
| ^
1702| else
… while evaluating the attribute 'value'
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/modules.nix:1118:7:
1117| // {
1118| value = addErrorContext "while evaluating the option `${showOption loc}':" value;
| ^
1119| inherit (res.defsFinal') highestPrio;
(stack trace truncated; use '--show-trace' to show the full trace)
error: Package olm-3.2.16 in /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/pkgs/by-name/ol/olm/package.nix:37 is marked as insecure, refusing to evaluate.
< -snip the whole explanation about olm's CVEs- >
note: trace involved the following derivations:
derivation 'etc'
derivation 'dbus-1'
derivation 'system-path'
derivation 'nheko-0.12.1'
derivation 'mtxclient-0.10.1'
```
Now we finally know that olm was evaluated because of Nheko, without sifting through *thousands* of lines of error message.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- Symbols reuses once-allocated Value to reduce garbage collected allocations [cl/3308](https://gerrit.lix.systems/c/lix/+/3308) [cl/3300](https://gerrit.lix.systems/c/lix/+/3300) [cl/3314](https://gerrit.lix.systems/c/lix/+/3314) [cl/3310](https://gerrit.lix.systems/c/lix/+/3310) [cl/3312](https://gerrit.lix.systems/c/lix/+/3312) [cl/3313](https://gerrit.lix.systems/c/lix/+/3313)
In the Lix evaluator, **symbols** represent immutable strings, like those used
for attribute names.
In evaluator design, such strings are typically [**interned**](https://en.wikipedia.org/wiki/String_interning), stored uniquely
to save memory, and Lix inherits this approach from the original C++ codebase.
However, some builtins, like `builtins.attrNames`, must return a `Value` type
that can represent any Nix value (strings, integers, lists, etc.).
Before this change, these builtins would create lists of `Value` objects by
allocating them through the garbage collector, copying the symbols string
content each time.
This allocation is unnecessary if the interned symbols themselves also hold a
`Value` representation allocated outside the garbage collector, since these
live for the full duration of evaluation.
As a result, this reduces the number of allocations, leading to:
* A significant drop in maximum [resident set memory](https://en.wikipedia.org/wiki/Resident_set_size) (RSS), with some large-scale
tests showing up to 11% (about 500 MiB) savings in large colmena deployments.
* A slight decrease in CPU usage during Nix evaluations.
This change is inspired by https://github.com/NixOS/nix/pull/13258 but the approach is different.
**Note** : [`xokdvium`](https://github.com/xokdvium) is the rightful author of https://gerrit.lix.systems/c/lix/+/3300 and the credit was missed on our end during the development process. We are deeply sorry for this mistake.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [eldritch horrors](https://git.lix.systems/pennae), [Tom Hubrecht](https://git.lix.systems/tom-hubrecht), [xokdvium](https://github.com/xokdvium), and [NaN-git](https://github.com/NaN-git) for this.
## Fixes
- `build-dir` no longer defaults to `temp-dir` [cl/3453](https://gerrit.lix.systems/c/lix/+/3453)
The directory in which temporary build directories are created no longer defaults
to the value of the `temp-dir` setting to avoid builders making their directories
world-accessible. This behavior has been used to escape the build sandbox and can
cause build impurities even when not used maliciously. We now default to `builds`
in `NIX_STATE_DIR` (which is `/nix/var/nix/b` in the default configuration).
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Global certificate authorities are copied inside the builder's environment [gh#12698](https://github.com/NixOS/nix/issues/12698) [fj#885](https://git.lix.systems/lix-project/lix/issues/885) [cl/3765](https://gerrit.lix.systems/c/lix/+/3765)
Previously, CA certificates were only installed at
`/etc/ssl/certs/ca-certificates.crt` for sandboxed builds on Linux.
This setup was insufficient in light of recent changes in `nixpkgs`, which now
enforce HTTPS usage for `fetchurl`, even for fixed-output derivations, to
mitigate confidentiality risks such as `netrc` or credentials leakage.
`nixpkgs` still make use of a special package called `cacerts` which contains a
copy of the CA certificates maintained by Nixpkgs and added as a reference for
TLS-enabled fetchers.
As a result, having a consistent and trusted certificate authority in all
builder environments is becoming more essential.
On `nix-darwin`, the `NIX_SSL_CERT_FILE` environment variable is always
explicitly defined, but it is ignored by the sandbox setup.
Simultaneously, Nix evaluates and propagates impure environment variables via
`lib.proxyImpureEnvVars`, meaning that if `NIX_SSL_CERT_FILE` is set (which
influences the default value for `ssl-cert-file`), it will be forwarded
unchanged into the builder environment.
However, on Linux, Nix also *copies* the CA file into the sandbox, creating a
discrepancy between the value of `NIX_SSL_CERT_FILE` and the actual trusted
certificate path used during the build.
This divergence caused confusion and was partially addressed by attempts to
whitelist the CA path in the Darwin sandbox (see cl/2906), but that approach
involved a non-trivial path canonicalization step and is not as general as this one.
To address this properly, we now emit a warning and override
`NIX_SSL_CERT_FILE` inside the builder, explicitly pointing it to the CA file
copied into the sandbox.
This eliminates ambiguity between `NIX_SSL_CERT_FILE`
and `ssl-cert-file`, ensuring consistent trust anchors across platforms.
This warning might become a hard error as we figure out what to do regarding
`lib.proxyImpureEnvVars` in nixpkgs.
The behavior has been verified across sandboxed and unsandboxed builds on both
Linux and Darwin.
As a consequence of this change, approximately 500KB of CA certificate data is
now unconditionally copied into the build directory for fixed-output
derivations.
While this ensures consistent trust verification without having to restart the
daemon after system upgrades, it may introduce a slight overhead in build
performance. At present, no optimizations have been implemented to avoid this
copy, but if this overhead proves noticeable in your workflows, please open an
issue so we can evaluate and possibly implement different strategies to render
trust anchors visible.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [Emily](https://git.lix.systems/emilazy) for this.
- libstore: exponential backoff for downloads [lix#932](https://git.lix.systems/lix-project/lix/issues/932) [cl/3856](https://gerrit.lix.systems/c/lix/+/3856)
The connection timeout when downloading from e.g. a binary cache is exponentially
increased per failure. The option `connect-timeout` is now an alias to `max-connect-timeout`
which is the maximum value for a timeout. The start value is controlled
by `initial-connect-timeout` which is `5` by default.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- Fix develop shells for derivations with escape codes [fj#991](https://git.lix.systems/lix-project/lix/issues/991) [cl/4154](https://gerrit.lix.systems/c/lix/+/4154) [cl/4155](https://gerrit.lix.systems/c/lix/+/4155)
ASCII control characters (including `\e`, used for ANSI escape codes) in derivation variables are now correctly escaped for `nix develop` and `nix print-dev-env`, instead of erroring.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- nix-store --delete: always remove obsolete hardlinks [cl/3188](https://gerrit.lix.systems/c/lix/+/3188)
Deleting specific paths using `nix-store --delete` or `nix store
delete` previously did not delete hard links created by `nix-store
--optimise` even if they became obsolete, unless _all_ of the given
paths were deleted successfully. Now, hard links are always cleaned
up, even if some of the given paths could not be deleted.
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Report GC statistics correctly [cl/3188](https://gerrit.lix.systems/c/lix/+/3188)
Deleting specific paths using `nix-store --delete` or `nix store delete` previously did
not report statistics correctly when some of the paths could not be deleted, even if
others were deleted:
```
$ nix store delete /nix/store/9bwryidal9q3g91cjm6xschfn4ikd82q-hello-2.12.1 --delete-closure -v
finding garbage collector roots...
deleting '/nix/store/9bwryidal9q3g91cjm6xschfn4ikd82q-hello-2.12.1'
0 store paths deleted, 0.00 MiB freed
error: Cannot delete some of the given paths because they are still alive. Paths not deleted:
k9bxzr1l92r5y6mihrkbpbr3fmc8qszx-libidn2-2.3.8
mbx9ii53lzjlrsnlrfmzpwm33ynljwdn-libunistring-1.3
rf8hcy6bldxdqc0g6q1dcka1vh47x69s-xgcc-14.2.1.20250322-libgcc
vbrdc5wgzn0w1zdp10xd2favkjn5fk7y-glibc-2.40-66
To find out why, use nix-store --query --roots and nix-store --query --referrers.
```
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Fallback to safe temp dir when build-dir is unwritable [fj#876](https://git.lix.systems/lix-project/lix/issues/876) [cl/3501](https://gerrit.lix.systems/c/lix/+/3501)
Non-daemon builds started failing with a permission error after introducing the `build-dir` option:
```
$ nix build --store ~/scratch nixpkgs#hello --rebuild
error: creating directory '/nix/var/nix/builds/nix-build-hello-2.12.2.drv-0': Permission denied
```
This happens because:
1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
2. The user lacks permissions for that path.
We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
**Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Parse overflowing JSON number literals as floatingpoint [cl/3919](https://gerrit.lix.systems/c/lix/+/3919)
Previously, `builtins.fromJSON "-9223372036854775809"` would
return a floatingpoint number, while `builtins.fromJSON
"9223372036854775808"` would cause an evaluation error. This was
introduced with the banning of integer overflow in Lix 2.91; previously
the latter would result in C++ undefined behaviour. These cases are
now treated consistently with JSONs model of a single numeric type,
and JSON number literals that do not fit in a Nixlanguage integer
will be parsed as floatingpoint numbers.
Many thanks to [Emily](https://git.lix.systems/emilazy) for this.
- Fix handling of OSC codes in terminal output [fj#160](https://git.lix.systems/lix-project/lix/issues/160) [cl/3143](https://gerrit.lix.systems/c/lix/+/3143)
OSC codes in terminal output are now handled correctly, where OSC 8 (hyperlink) is preserved any
time color codes are allowed and all other OSC codes are stripped out. This applies not only to
output from build commands but also to rendered documentation in the REPL.
Many thanks to [lilyball](https://git.lix.systems/lilyball) for this.
- Fix nix develop for derivations that rejects dependencies with structured attrs [fj#997](https://git.lix.systems/lix-project/lix/issues/997) [cl/4182](https://gerrit.lix.systems/c/lix/+/4182)
For the sake of concision, we refer to `disallowedReferences` in what follows,
but all output checks were equally fixed:
`{dis,}allowed{References,Requisites}`.
Derivations can define *output checks* to reject unwanted dependencies, such as
interpreters like `bash` or compilers like `gcc`. This can be done in two ways:
* **Legacy style**: `disallowedReferences = [ ... ]` in the environment.
* **Structured attrs**: `outputChecks.<output>.disallowedReferences = [ ... ]`,
typically used in `__json`.
Only the structured form supports derivations with multiple outputs.
`nix develop` internally rewrites derivations to create development shells. It
relied on the legacy `disallowedReferences`, and failed to honor the structured
variant. This led to broken shells in cases where `bashInteractive` was
explicitly disallowed using structured output checks, e.g. `nix develop
nixpkgs#systemd` after the "bash-less NixOS" changes.
This fix teaches `nix develop` to respect structured output checks, restoring
support for such derivations.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- nix-eval-jobs: retain NIX_PATH [cl/3859](https://gerrit.lix.systems/c/lix/+/3859)
`nix-eval-jobs` doesn't clear the `NIX_PATH` from the environment anymore. This matches the behavior
of [upstream version `2.30`](https://github.com/nix-community/nix-eval-jobs/releases/tag/v2.30.0).
Many thanks to [ma27](https://git.lix.systems/ma27) and [mic92](https://github.com/mic92) for this.
- Remove reliance on Bash for remote stores via SSH [fj#830](https://git.lix.systems/lix-project/lix/issues/830) [fj#805](https://git.lix.systems/lix-project/lix/issues/805) [fj#304](https://git.lix.systems/lix-project/lix/issues/304) [cl/3159](https://gerrit.lix.systems/c/lix/+/3159)
The pre-flight `echo started` handshake -- added years ago to catch race conditions -- has been removed.
After removal of connection sharing in Lix 2.93, it required a Bash-compatible shell and a standard `echo`, so it failed on:
* builders protected by `ForceCommand` wrappers (e.g. `nix-remote-build`),
* BusyBox / initrd images with no Bash,
* hosts using non-POSIX shells such as Nushell.
The race the probe once addressed was tied to SSH connection-sharing -- since connection-sharing code has already been removed, the probe is now pointless.
Real connection or protocol errors are now left to SSH/Nix to report directly.
This is technically a breaking change if you had scripts that relied on the literal "started" which needs to be updated to rely on other signals, e.g., exit codes.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- repl-overlays now work in the debugger for flakes [fj#777](https://git.lix.systems/lix-project/lix/issues/777) [cl/3398](https://gerrit.lix.systems/c/lix/+/3398)
Due to a bug, it was previously not possible to use the debugger on flakes with repl-overlays, or with pure evaluation in general:
```
$ nix repl --pure-eval
Lix 2.94.0-dev-pre20250617-87d99da
Type :? for help.
Loading 'repl-overlays'...
error: access to absolute path '/Users/jade/.config/nix/repl.nix' is forbidden in pure eval mode (use '--impure' to override)
```
This is now fixed.
The contents of the repl-overlays file itself (i.e. most typically the top level lambda in it) will be evaluated in impure mode.
It may be necessary to use `builtins.seq` to force the impure operations to happen first if one wants to do impure operations inside a repl-overlays file in pure evaluation mode.
Many thanks to [jade](https://git.lix.systems/jade) for this.
- `nix-shell` default shell directory is not `/tmp` anymore for `$NIX_BUILD_TOP` [fj#940](https://git.lix.systems/lix-project/lix/issues/940)
Previously, Lix `nix-shell`s could exit non-zero status when `stdenv`'s `dumpVars` phase failed to write to `$NIX_BUILD_TOP/env-vars`, despite `dumpVars` being intended as a debugging aid.
This happens when `TMPDIR` is not set and defaults therefore to `/tmp`, resulting in a `/tmp/env-vars` global file that every `nix-shell` wants to write.
We fix this issue by reusing a pre-created, unique, and writable location, as the build top directory, avoiding shell exiting from write failures silently.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- libstore/binary-cache-store: don't cache narinfo on nix copy, remove negative entry [cl/3789](https://gerrit.lix.systems/c/lix/+/3789)
When using e.g. [Snix's nar-bridge](https://snix.dev/docs/components/overview/#nar-bridge) via
an `http`-store, Lix would create cache entries with a wrong URL to the NAR when uploading
a store-path.
This caused hard build failures for Hydra.
Lix doesn't create these entries on upload anymore. Instead, it only removes negative cache entries.
The cache entry for a narinfo is now created the first time, Lix queries the cache
for the previously uploaded store-path again.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- Lix libraries can now be linked statically [fj#789](https://git.lix.systems/lix-project/lix/issues/789) [cl/3775](https://gerrit.lix.systems/c/lix/+/3775) [cl/3778](https://gerrit.lix.systems/c/lix/+/3778)
Previously the pkg-config files distributed with Lix were only suitable for dynamic linkage, causing "undefined reference to…" linker errors when trying to link statically.
Private dependency information has now been added to make static linkage work as expected without user intervention.
In addition, relevant static libraries are now prelinked to avoid strange failures due to missing static initializers.
Many thanks to [alois31](https://git.lix.systems/alois31) for this.
- add description to zsh completions [fj#910](https://git.lix.systems/lix-project/lix/issues/910) [cl/3632](https://gerrit.lix.systems/c/lix/+/3632)
Emit descriptions when completing args in zsh completions. This uses the descriptions we already
provided in NIX\_GET\_COMPLETIONS.
Many thanks to [matthewbauer](https://github.com/matthewbauer) for this.
## Miscellany
- Deprecation of CA derivations, dynamic derivations, and impure derivations [fj#815](https://git.lix.systems/lix-project/lix/issues/815)
Content-addressed derivations are now deprecated and slated for removal in Lix 2.94.
We're doing this because the CA derivation system has been a known cause of problems
and inconsistencies, is unmaintained, habitually makes improving the store code very
difficult (or blocks such improvements outright), and is beset by a number of design
flaws that in our opinion cannot be fixed without a full reimplementation from zero.
Dynamic derivations and impure derivations are built on the CA derivation framework,
and owing to this they too are deprecated and slated for removal in another release.
+2 -7
View File
@@ -70,15 +70,10 @@ def do_include(content: str, relative_md_path: Path, source_root: Path, search_p
def recursive_replace(data, book_root, search_path):
match data:
# XXX FUTURE: drop sections once mdBook is at 0.5.0 or above in nixpkgs
case {'sections': sections}:
return data | dict(
sections = [recursive_replace(section, book_root, search_path) for section in sections],
)
case {'items': items}:
return data | dict(
items = [recursive_replace(item, book_root, search_path) for item in items],
)
case {'Chapter': chapter}:
path_to_chapter = Path(chapter['path'])
chapter_content = chapter['content']
@@ -124,10 +119,10 @@ def main():
context, book = json.load(sys.stdin)
# book_root is the directory where book contents leave (ie, src/)
book_root = Path(context['root']) / context['config']['book'].get('src', 'src')
book_root = Path(context['root']) / context['config']['book']['src']
# includes pointing into @generated@ will look here
search_path = Path(os.environ['MANUAL_SUBSTITUTE_SEARCH'])
search_path = Path(os.environ['MDBOOK_SUBSTITUTE_SEARCH'])
# Find @var@ in all parts of our recursive book structure.
replaced_content = recursive_replace(book, book_root, search_path)
+1 -1
View File
@@ -8,7 +8,7 @@
tag ? "latest",
bundleNixpkgs ? true,
channelName ? "nixpkgs",
channelURL ? "https://channels.nixos.org/nixpkgs-unstable",
channelURL ? "https://nixos.org/channels/nixpkgs-unstable",
extraPkgs ? [ ],
maxLayers ? 100,
nixConf ? { },
Generated
+3 -3
View File
@@ -108,11 +108,11 @@
},
"nixpkgs_2": {
"locked": {
"lastModified": 1757198069,
"narHash": "sha256-m3VUcOD4rTs8J7S+3dOjWMrAjw6RcITC3XYQ98zhEFs=",
"lastModified": 1758391731,
"narHash": "sha256-UuwQoPWv13DVKMveeev+F0OC/N95AOmAz6SzCuGhxjQ=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "0747026fc57ecb9c28901c7f7a2b5dc40e8af43c",
"rev": "3f00d36f15e16e0471d9ca1e8f88958941fa970a",
"type": "github"
},
"original": {
+8 -36
View File
@@ -88,7 +88,7 @@
else
"pre${
builtins.substring 0 8 (self.lastModifiedDate or self.lastModified or "19700101")
}-dev_${self.shortRev or "dirty"}";
}_${self.shortRev or "dirty"}";
linux32BitSystems = [ "i686-linux" ];
linux64BitSystems = [
@@ -100,7 +100,6 @@
"x86_64-darwin"
"aarch64-darwin"
];
nonDarwinSystems = linuxSystems;
systems = linuxSystems ++ darwinSystems;
# If you add something here, please update the list in doc/manual/src/contributing/hacking.md.
@@ -222,13 +221,9 @@
busybox-sandbox-shell = final.busybox-sandbox-shell or final.default-busybox-sandbox-shell;
};
lix-clang-tidy = final.callPackage ./subprojects/lix-clang-tidy {
# FIXME: To be removed when switching to nixos-25.11-small
llvmPackages = final.llvmPackages_20;
};
lix-clang-tidy = final.callPackage ./subprojects/lix-clang-tidy { };
nix-eval-jobs = final.callPackage ./subprojects/nix-eval-jobs {
stdenv = currentStdenv;
srcDir = ./subprojects/nix-eval-jobs;
};
@@ -262,22 +257,7 @@
sha512 = "2a4d0rqh8gkw4ca3gkzddp0hjpmmw74cbks8k0inhh0vizmgbn188zdv6m1kgmr019b99g7insli8js3ci1ji7y4n5nk704bswf3z3i";
};
nativeBuildInputs = prevAttrs.nativeBuildInputs ++ [ final.buildPackages.bmake ];
postInstall = lib.replaceStrings [ "lowdown.so.1" ] [ "lowdown.so.2" ] (
prevAttrs.postInstall or ""
);
});
capnproto = prev.capnproto.overrideAttrs (old: {
patches =
old.patches or [ ]
++ [
# backport of https://github.com/capnproto/capnproto/pull/1810
./misc/capnproto-promise-nodiscard.patch
]
++ lib.optionals (lib.versionOlder old.version "1.2.0") [
# backport of https://github.com/capnproto/capnproto/pull/2296
./misc/capnproto-monotonic-clocks-are-a-lie.patch
];
postInstall = lib.replaceStrings [ "lowdown.so.1" ] [ "lowdown.so.2" ] prevAttrs.postInstall;
});
};
in
@@ -391,10 +371,7 @@
;
}
// {
# the n-e-j test suite is unusably slow in darwin ci. disbled until anywho fixes this.
nix-eval-jobs = (lib.genAttrs nonDarwinSystems) (
system: self.packages.${system}.nix-eval-jobs.tests.nix-eval-jobs
);
nix-eval-jobs = forAllSystems (system: self.packages.${system}.nix-eval-jobs.tests.nix-eval-jobs);
# This is x86_64-linux only, just because we have significantly
# cheaper x86_64-linux compute in CI.
@@ -433,20 +410,17 @@
'';
# clang-tidy run against the Lix codebase using the Lix clang-tidy plugin
clang-tidy = forAllSystems (
system:
clang-tidy =
let
pkgs = nixpkgsFor.${system}.native;
nixpkgs = nixpkgsFor.x86_64-linux.native;
inherit (nixpkgs) pkgs;
in
pkgs.callPackage ./package.nix {
# Required since we don't support gcc stdenv
stdenv = pkgs.clangStdenv;
# FIXME: To be removed when switching to nixos-25.11-small
llvmPackages = pkgs.llvmPackages_20;
versionSuffix = "";
lintInsteadOfBuild = true;
}
);
};
# Make sure that nix-env still produces the exact same result
# on a particular version of Nixpkgs.
@@ -591,8 +565,6 @@
inherit stdenv versionSuffix;
busybox-sandbox-shell = pkgs.busybox-sandbox-shell or pkgs.default-busybox-sandbox;
internalApiDocs = false;
# Use LLD in the dev shell by default for faster link times.
useLld = stdenv.hostPlatform.isLinux;
};
pre-commit = self.hydraJobs.pre-commit.${pkgs.system} or { };
in
+110 -195
View File
@@ -1,18 +1,15 @@
#include "lix/libstore/path.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/error.hh"
#include "lix/libutil/file-descriptor.hh"
#include "lix/libutil/logging-rpc.hh"
#include "lix/libutil/logging.hh"
#include "lix/libutil/rpc.hh"
#include "lix/libutil/types-rpc.hh" // IWYU pragma: keep
#include "lix/libutil/types.hh"
#include "lix/libutil/types-rpc.hh"
#include <algorithm>
#include <capnp/rpc-twoparty.h>
#include <chrono>
#include <cstring>
#include <exception>
#include <kj/async.h>
#include <future>
#include <kj/time.h>
#include <set>
#include <memory>
@@ -46,11 +43,9 @@ namespace {
struct Instance final : rpc::build_remote::HookInstance::Server
{
unsigned int maxBuildJobs;
bool initialized = false, used = false;
kj::Promise<void> init(InitContext context) override;
Instance(unsigned int maxBuildJobs) : maxBuildJobs(maxBuildJobs) {}
kj::Promise<void> buildImpl(BuildContext context);
kj::Promise<void> build(BuildContext context) override;
};
}
@@ -188,62 +183,27 @@ struct BuilderConnection
// start the thread that reads ssh stderr and turns it into log items.
// this future *must* outlive sshStore, otherwise it will never finish
kj::Promise<Result<void>> startLogThread(std::string buildDescription, std::string drvPath)
try {
std::future<void> startLogThread(int intoFD)
{
if (!logPipe.readSide) {
co_return result::success();
return {};
}
logPipe.writeSide.close();
// NOTE this is very similar to handleBuilderOutput in DerivationGoal, but unlike
// the derivation goal we do not need to handle EIO from a pty here. we also have
// no timeouts or limits to keep track of, which makes deduplication less useful.
auto act = logger->startActivity(
lvlInfo, actBuild, buildDescription, Logger::Fields{drvPath, storeUri, 1, 1}
return std::async(
std::launch::async,
[](int from, int to) {
AsyncIoRoot aio;
auto reader = AIO().lowLevelProvider.wrapInputFd(from);
auto writer = AIO().lowLevelProvider.wrapOutputFd(to);
reader->pumpTo(*writer).wait(aio.kj.waitScope);
},
logPipe.readSide.get(),
intoFD
);
std::map<ActivityId, Activity> activities;
auto reader = AIO().lowLevelProvider.wrapInputFd(logPipe.readSide.get());
LogLineSplitter splitter;
auto flushLine = [&](const std::string & line) {
if (const auto state =
handleJSONLogMessage(line, act, activities, "the derivation builder"))
{
return *state;
} else {
return act.result(resBuildLogLine, line);
}
};
auto buf = kj::heapArray<char>(4096);
while (true) {
const auto got = co_await reader->tryRead(buf.begin(), 1, buf.size());
if (got == 0) {
break;
}
std::string_view data{buf.begin(), got};
while (!data.empty()) {
if (auto line = splitter.feed(data)) {
if (flushLine(*line) == Logger::BufferState::NeedsFlush) {
TRY_AWAIT(act.getLogger().flush());
}
}
}
}
if (auto line = splitter.finish(); !line.empty()) {
(void) flushLine(line);
TRY_AWAIT(act.getLogger().flush());
}
co_return result::success();
} catch (...) {
co_return result::current_exception();
}
};
@@ -252,16 +212,21 @@ struct AcceptedBuild final : rpc::build_remote::HookInstance::AcceptedBuild::Ser
ref<Store> store;
StorePath drvPath;
BuilderConnection builder;
bool used = false;
rpc::build_remote::HookInstance::BuildLogger::Client buildLogger;
AcceptedBuild(ref<Store> store, StorePath drvPath, BuilderConnection builder)
AcceptedBuild(
ref<Store> store,
StorePath drvPath,
BuilderConnection builder,
rpc::build_remote::HookInstance::BuildLogger::Client buildLogger
)
: store(store)
, drvPath(drvPath)
, builder(std::move(builder))
, buildLogger(std::move(buildLogger))
{
}
kj::Promise<void> runImpl(RunContext context);
kj::Promise<void> run(RunContext context) override;
};
@@ -270,7 +235,7 @@ enum class BuildRejected { Temporarily, Permanently };
static kj::Promise<Result<std::variant<BuildRejected, BuilderConnection>>> connectToBuilder(
const ref<Store> & store,
const StorePath & drvPath,
const std::optional<StorePath> & drvPath,
Machines & machines,
const unsigned int maxBuildJobs,
const bool amWilling,
@@ -290,7 +255,7 @@ try {
bool canBuildLocally = amWilling && couldBuildLocally;
/* Error ignored here, will be caught later */
(void) sys::mkdir(currentLoad, 0777);
mkdir(currentLoad.c_str(), 0777);
while (true) {
bestSlotLock.reset();
@@ -307,7 +272,7 @@ try {
} else {
printSelectionFailureMessage(
couldBuildLocally ? lvlChatty : lvlWarn,
drvPath.to_string(),
drvPath ? drvPath->to_string() : "<unknown>",
machines,
neededSystem,
requiredFeatures
@@ -329,8 +294,8 @@ try {
Pipe logPipe;
try {
auto act = logger->startActivity(
lvlTalkative, actUnknown, fmt("connecting to '%s'", bestMachine->storeUri)
Activity act(
*logger, lvlTalkative, actUnknown, fmt("connecting to '%s'", bestMachine->storeUri)
);
std::tie(sshStore, logPipe) = TRY_AWAIT(bestMachine->openStore());
@@ -356,6 +321,8 @@ try {
static int main_build_remote(AsyncIoRoot & aio, std::string programName, Strings argv)
{
{
logger = makeJSONLogger(*logger);
/* Ensure we don't get any SSH passphrase or host key popups. */
unsetenv("DISPLAY");
unsetenv("SSH_ASKPASS");
@@ -368,145 +335,103 @@ static int main_build_remote(AsyncIoRoot & aio, std::string programName, Strings
verbosity = (Verbosity) std::stoll(argv.front());
FdSource source(STDIN_FILENO);
/* Read the parent's settings. */
while (readNum<unsigned>(source)) {
auto name = readString(source);
auto value = readString(source);
settings.set(name, value);
}
auto maxBuildJobs = settings.maxBuildJobs;
settings.maxBuildJobs.set("1"); // hack to make tests with local?root= work
initPlugins();
auto conn = aio.kj.lowLevelProvider->wrapUnixSocketFd(1);
capnp::TwoPartyServer srv(kj::heap<Instance>());
capnp::TwoPartyServer srv(kj::heap<Instance>(maxBuildJobs));
srv.accept(*conn, 1).wait(aio.kj.waitScope);
return 0;
}
}
kj::Promise<void> Instance::init(InitContext context)
kj::Promise<void> Instance::build(BuildContext context)
{
try {
if (initialized) {
throw Error("build hook can only be initialized once");
// FIXME this does not open a daemon connection for historical reasons.
// we may create a lot of build hook instances, and having each of them
// also create a daemon instance is inefficient and wasteful. in future
// versions of the build hook (where we don't need one hook process per
// build) we should change this to using a daemon connection, ideally a
// daemon connection provided by the parent via file descriptor passing
auto store = TRY_AWAIT(openStore(settings.storeUri, {}, AllowDaemon::Disallow));
/* It would be more appropriate to use $XDG_RUNTIME_DIR, since
that gets cleared on reboot, but it wouldn't work on macOS. */
auto currentLoadName = "/current-load";
if (auto localStore = store.try_cast_shared<LocalFSStore>())
currentLoad = std::string { localStore->config().stateDir } + currentLoadName;
else
currentLoad = settings.nixStateDir + currentLoadName;
auto machines = getMachines();
debug("got %d remote builders", machines.size());
if (machines.empty()) {
context.getResults().initResult().initGood().setDeclinePermanently();
co_return;
}
logger = rpc::log::makeRpcLoggerClient(context.getParams().getLogger());
auto amWilling = context.getParams().getAmWilling();
auto neededSystem = rpc::to<std::string>(context.getParams().getNeededSystem());
auto drvPath = from(context.getParams().getDrvPath(), *store);
auto requiredFeatures =
rpc::to<std::set<std::string>>(context.getParams().getRequiredFeatures());
auto buildLogger = context.getParams().getBuildLogger();
/* Read the parent's settings. */
for (const auto & [name, value] : rpc::to<StringMap>(context.getParams().getSettings())) {
settings.set(name, value);
auto result = TRY_AWAIT(connectToBuilder(
store, drvPath, machines, maxBuildJobs, amWilling, neededSystem, requiredFeatures
));
if (auto immediateResponse = std::get_if<BuildRejected>(&result)) {
switch (*immediateResponse) {
case BuildRejected::Temporarily:
context.getResults().initResult().initGood().setPostpone();
co_return;
case BuildRejected::Permanently:
context.getResults().initResult().initGood().setDecline();
co_return;
}
}
maxBuildJobs = settings.maxBuildJobs;
settings.maxBuildJobs.set("1"); // hack to make tests with local?root= work
auto builder = std::get_if<BuilderConnection>(&result);
assert(builder);
initPlugins();
initialized = true;
context.getResults().initResult().setGood();
auto ac = context.getResults().initResult().initGood().initAccept();
RPC_FILL(ac, setMachineName, builder->storeUri);
ac.setMachine(kj::heap<AcceptedBuild>(store, drvPath, std::move(*builder), buildLogger));
} catch (...) {
RPC_FILL(context.getResults(), initResult, std::current_exception());
}
return kj::READY_NOW;
}
kj::Promise<void> Instance::buildImpl(BuildContext context)
{
if (!initialized) {
throw Error("build hook not fully initialized");
}
// FIXME this does not open a daemon connection for historical reasons.
// we may create a lot of build hook instances, and having each of them
// also create a daemon instance is inefficient and wasteful. in future
// versions of the build hook (where we don't need one hook process per
// build) we should change this to using a daemon connection, ideally a
// daemon connection provided by the parent via file descriptor passing
auto store = TRY_AWAIT(openStore(settings.storeUri, {}, AllowDaemon::Disallow));
/* It would be more appropriate to use $XDG_RUNTIME_DIR, since
that gets cleared on reboot, but it wouldn't work on macOS. */
auto currentLoadName = "/current-load";
if (auto localStore = store.try_cast_shared<LocalFSStore>()) {
currentLoad = std::string{localStore->config().stateDir} + currentLoadName;
} else {
currentLoad = settings.nixStateDir + currentLoadName;
}
auto machines = getMachines();
debug("got %d remote builders", machines.size());
if (machines.empty()) {
context.getResults().initResult().initGood().setDeclinePermanently();
co_return;
}
auto amWilling = context.getParams().getAmWilling();
auto neededSystem = rpc::to<std::string>(context.getParams().getNeededSystem());
auto drvPath = from(context.getParams().getDrvPath(), *store);
auto requiredFeatures =
rpc::to<std::set<std::string>>(context.getParams().getRequiredFeatures());
auto result = TRY_AWAIT(connectToBuilder(
store, drvPath, machines, maxBuildJobs, amWilling, neededSystem, requiredFeatures
));
if (auto immediateResponse = std::get_if<BuildRejected>(&result)) {
switch (*immediateResponse) {
case BuildRejected::Temporarily:
context.getResults().initResult().initGood().setPostpone();
co_return;
case BuildRejected::Permanently:
context.getResults().initResult().initGood().setDecline();
co_return;
}
}
auto builder = std::get_if<BuilderConnection>(&result);
assert(builder);
auto ac = context.getResults().initResult().initGood().initAccept();
ac.setMachine(kj::heap<AcceptedBuild>(store, drvPath, std::move(*builder)));
}
kj::Promise<void> Instance::build(BuildContext context)
try {
if (used) {
throw Error("build hooks can only accept a single job");
}
used = true; // lock out other rpc calls during processing
co_await buildImpl(context);
TRY_AWAIT(logger->flush());
used = context.getResults().getResult().getGood().isAccept();
} catch (...) {
RPC_FILL(context.getResults(), getResult, std::current_exception());
}
kj::Promise<void> AcceptedBuild::run(RunContext context)
{
try {
auto oldLogger = logger;
logger = rpc::log::makeRpcLoggerClient(context.getParams().getLogger());
TRY_AWAIT(oldLogger->flush());
KJ_DEFER({
delete logger;
logger = oldLogger;
});
if (used) {
throw Error("build hooks builds are single-use items");
const int logFD = (co_await buildLogger.getFd()).orDefault(-1);
if (logFD < 0) {
throw Error("build-hook needs a logFD from the builder to build");
}
used = true;
co_await runImpl(context);
TRY_AWAIT(logger->flush());
} catch (...) {
RPC_FILL(context.getResults(), getResult, std::current_exception());
}
}
kj::Promise<void> AcceptedBuild::runImpl(RunContext context)
{
try {
auto logHandler = builder.startLogThread(
fmt("%s on '%s'",
rpc::to<std::string_view>(context.getParams().getDescription()),
builder.storeUri),
store->printStorePath(drvPath)
);
auto logThread = builder.startLogThread(logFD);
KJ_DEFER({
// drop any existing ssh connection so the log thread can exit
builder.sshStore = nullptr;
if (logThread.valid()) {
logThread.get();
}
});
auto & sshStore = builder.sshStore;
auto & storeUri = builder.storeUri;
@@ -519,9 +444,7 @@ kj::Promise<void> AcceptedBuild::runImpl(RunContext context)
AutoCloseFD uploadLock = openLockFile(lockFileName, true);
{
auto act = logger->startActivity(
lvlTalkative, actUnknown, fmt("waiting for the upload lock to '%s'", storeUri)
);
Activity act(*logger, lvlTalkative, actUnknown, fmt("waiting for the upload lock to '%s'", storeUri));
auto result = TRY_AWAIT(
AIO().timeoutAfter(15 * kj::MINUTES, lockFileAsync(uploadLock.get(), ltWrite))
@@ -534,9 +457,7 @@ kj::Promise<void> AcceptedBuild::runImpl(RunContext context)
auto substitute = settings.buildersUseSubstitutes ? Substitute : NoSubstitute;
{
auto act = logger->startActivity(
lvlTalkative, actUnknown, fmt("copying dependencies to '%s'", storeUri)
);
Activity act(*logger, lvlTalkative, actUnknown, fmt("copying dependencies to '%s'", storeUri));
TRY_AWAIT(copyPaths(*store, *sshStore, inputs, NoRepair, NoCheckSigs, substitute));
}
@@ -606,9 +527,7 @@ kj::Promise<void> AcceptedBuild::runImpl(RunContext context)
}
if (!missingPaths.empty()) {
auto act = logger->startActivity(
lvlTalkative, actUnknown, fmt("copying outputs from '%s'", storeUri)
);
Activity act(*logger, lvlTalkative, actUnknown, fmt("copying outputs from '%s'", storeUri));
if (auto localStore = store.try_cast_shared<LocalStore>())
for (auto & path : missingPaths)
localStore->locksHeld.insert(store->printStorePath(path)); /* FIXME: ugly */
@@ -617,10 +536,6 @@ kj::Promise<void> AcceptedBuild::runImpl(RunContext context)
);
}
// drop store connection, let log handler process any remaining input
builder.sshStore = nullptr;
TRY_AWAIT(logHandler);
context.getResults().initResult().setGood();
} catch (...) {
RPC_FILL(context.getResults(), initResult, std::current_exception());
+46 -63
View File
@@ -9,7 +9,6 @@
#include "lix/libstore/store-api.hh"
#include "lix/libstore/local-fs-store.hh"
#include "lix/libstore/globals.hh"
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/current-process.hh"
#include "lix/libstore/derivations.hh"
#include "lix/libmain/shared.hh"
@@ -19,13 +18,10 @@
#include "lix/libcmd/common-eval-args.hh"
#include "lix/libexpr/attr-path.hh"
#include "lix/libcmd/legacy.hh"
#include "lix/libutil/finally.hh"
#include "lix/libutil/processes.hh"
#include "lix/libutil/regex.hh"
#include "lix/libutil/shlex.hh"
#include "nix-build.hh"
#include "lix/libstore/temporary-dir.hh"
#include "lix/libutil/strings.hh"
extern char * * environ __attribute__((weak)); // Man what even is this
@@ -33,7 +29,7 @@ namespace nix {
using namespace std::string_literals;
static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings argv)
static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings argv)
{
auto dryRun = false;
auto runEnv = std::regex_search(programName, regex::parse("nix-shell$"));
@@ -191,8 +187,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
if (packages && fromArgs)
throw UsageError("'-p' and '-E' are mutually exclusive");
AutoDelete tmpDir(createTempDir(myName));
AutoDelete buildTopTmpDir(createTempSubdir(tmpDir, "build-top"));
AutoDelete tmpDir(createTempDir("", myName));
if (outLink.empty())
outLink = (Path) tmpDir + "/result";
@@ -229,9 +224,8 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
left = {"default.nix"};
}
if (runEnv) {
(void) sys::setenv("IN_NIX_SHELL", pure ? "pure" : "impure", 1);
}
if (runEnv)
setenv("IN_NIX_SHELL", pure ? "pure" : "impure", 1);
DrvInfos drvs;
@@ -403,9 +397,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
buildPaths(pathsToBuild);
if (dryRun) {
return 0;
}
if (dryRun) return;
if (shellDrv) {
auto shellDrvOutputs =
@@ -426,33 +418,25 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
env["__ETC_PROFILE_SOURCED"] = "1";
}
// Set NIX_SHELL_LEVEL
env["NIX_SHELL_LEVEL"] = std::to_string(
getEnvNonEmpty("NIX_SHELL_LEVEL")
.and_then([](std::string lvl) { return string2Int<size_t>(lvl); })
.value_or(0)
+ 1
);
// Don't use defaultTempDir() here! We want to preserve the user's TMPDIR for the shell
env["NIX_BUILD_TOP"] = env["TMPDIR"] = env["TEMPDIR"] = env["TMP"] = env["TEMP"] =
getEnvNonEmpty("TMPDIR").value_or(buildTopTmpDir);
env["NIX_BUILD_TOP"] = env["TMPDIR"] = env["TEMPDIR"] = env["TMP"] = env["TEMP"] = getEnvNonEmpty("TMPDIR").value_or("/tmp");
env["NIX_STORE"] = store->config().storeDir;
env["NIX_BUILD_CORES"] = std::to_string(settings.buildCores);
auto passAsFile = tokenizeString<StringSet>(getOr(drv.env, "passAsFile", ""));
bool keepTmp = false;
int fileNr = 0;
for (auto & var : drv.env)
if (passAsFile.count(var.first)) {
keepTmp = true;
auto fn = ".attr-" + std::to_string(fileNr++);
Path p = (Path) tmpDir + "/" + fn;
writeFile(p, var.second);
env[var.first + "Path"] = p;
} else {
} else
env[var.first] = var.second;
}
std::string structuredAttrsRC;
@@ -485,6 +469,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
env["NIX_ATTRS_SH_FILE"] = attrsSH;
env["NIX_ATTRS_JSON_FILE"] = attrsJSON;
keepTmp = true;
}
}
@@ -494,13 +479,24 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
lose the current $PATH directories. */
auto rcfile = (Path) tmpDir + "/rc";
auto tz = getEnv("TZ");
std::string rc =
fmt("%1%"
// always clear PATH.
// when nix-shell is run impure, we rehydrate it with the `p=$PATH` above
"unset PATH;"
"dontAddDisableDepTrack=1;\n",
(pure ? "" : "[ -n \"$PS1\" ] && [ -e ~/.bashrc ] && source ~/.bashrc; p=$PATH; "));
std::string rc = fmt(
R"(_nix_shell_clean_tmpdir() { command rm -rf %1%; }; )"
"%2%"
"%3%"
// always clear PATH.
// when nix-shell is run impure, we rehydrate it with the `p=$PATH` above
"unset PATH;"
"dontAddDisableDepTrack=1;\n",
shellEscape(tmpDir),
(keepTmp
? "trap _nix_shell_clean_tmpdir EXIT; "
"exitHooks+=(_nix_shell_clean_tmpdir); "
"failureHooks+=(_nix_shell_clean_tmpdir); "
: "_nix_shell_clean_tmpdir; "),
(pure
? ""
: "[ -n \"$PS1\" ] && [ -e ~/.bashrc ] && source ~/.bashrc; p=$PATH; ")
);
rc += structuredAttrsRC;
rc += fmt(
"\n[ -e $stdenv/setup ] && source $stdenv/setup; "
@@ -530,38 +526,29 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
vomit("Sourcing nix-shell with file %s and contents:\n%s", rcfile, rc);
writeFile(rcfile, rc);
auto args = interactive ? Strings{"--rcfile", rcfile} : Strings{rcfile};
Strings envStrs;
for (auto & i : env)
envStrs.push_back(i.first + "=" + i.second);
auto args = interactive
? Strings{"bash", "--rcfile", rcfile}
: Strings{"bash", rcfile};
auto envPtrs = stringsToCharPtrs(envStrs);
environ = envPtrs.data();
auto argPtrs = stringsToCharPtrs(args);
restoreProcessContext();
// We are going to run an interactive command, do not let the logger send a line.
logger->pause();
printMsg(lvlChatty, "running shell: %s", concatMapStringsSep(" ", args, shellEscape));
RunningProgram proc = runProgram2(
{.program = *shell,
.searchPath = true,
.args = args,
.environment = env,
.dieWithParent = true}
);
execvp(shell->c_str(), argPtrs.data());
// NOTE: we wait and return the status check immediately.
// If there's interruption, we will swallow it and wait again for termination.
auto toExitStatus = [](int waitRes) {
if (WIFEXITED(waitRes)) {
return WEXITSTATUS(waitRes);
} else if (WIFSIGNALED(waitRes)) {
return 128 + WTERMSIG(waitRes);
} else {
return 255;
}
};
try {
return toExitStatus(proc.wait());
} catch (Interrupted &) {
return toExitStatus(proc.wait());
}
throw SysError("executing shell '%s'", *shell);
}
else {
@@ -595,9 +582,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
buildPaths(pathsToBuild);
if (dryRun) {
return 0;
}
if (dryRun) return;
std::vector<StorePath> outPaths;
@@ -626,8 +611,6 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
for (auto & path : outPaths)
std::cout << store->printStorePath(path) << '\n';
}
return 0;
}
void registerLegacyNixBuildAndNixShell() {
+3 -5
View File
@@ -8,7 +8,6 @@
#include "lix/libexpr/eval-settings.hh" // for defexpr
#include "lix/libstore/temporary-dir.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/regex.hh"
#include "lix/libutil/result.hh"
#include "lix/libutil/users.hh"
@@ -45,7 +44,7 @@ static void readChannels()
// Writes the list of channels.
static void writeChannels()
{
auto channelsFD = sys::open(channelsList, O_WRONLY | O_CLOEXEC | O_CREAT | O_TRUNC, 0644);
auto channelsFD = AutoCloseFD{open(channelsList.c_str(), O_WRONLY | O_CLOEXEC | O_CREAT | O_TRUNC, 0644)};
if (!channelsFD)
throw SysError("opening '%1%' for writing", channelsList);
for (const auto & channel : channels)
@@ -175,12 +174,11 @@ static void update(AsyncIoRoot & aio, const StringSet & channelNames)
// Make the channels appear in nix-env.
struct stat st;
if (sys::lstat(nixDefExpr, &st) == 0) {
if (lstat(nixDefExpr.c_str(), &st) == 0) {
if (S_ISLNK(st.st_mode))
// old-skool ~/.nix-defexpr
if (sys::unlink(nixDefExpr) == -1) {
if (unlink(nixDefExpr.c_str()) == -1)
throw SysError("unlinking %1%", nixDefExpr);
}
} else if (errno != ENOENT) {
throw SysError("getting status of %1%", nixDefExpr);
}
+21 -13
View File
@@ -1,4 +1,3 @@
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/file-system.hh"
#include "lix/libstore/store-api.hh"
#include "lix/libstore/store-cast.hh"
@@ -25,11 +24,9 @@ bool dryRun = false;
static void removeOldGenerations(std::string dir, NeverAsync = {})
{
if (sys::access(dir, R_OK) != 0) {
return;
}
if (access(dir.c_str(), R_OK) != 0) return;
bool canWrite = sys::access(dir, W_OK) == 0;
bool canWrite = access(dir.c_str(), W_OK) == 0;
for (auto & i : readDirectory(dir)) {
checkInterrupt();
@@ -64,7 +61,7 @@ static int main_nix_collect_garbage(AsyncIoRoot & aio, std::string programName,
{
bool removeOld = false;
GCOptions options = {.action = GCOptions::gcDeleteDead};
GCOptions options;
LegacyArgs(aio, programName, [&](Strings::iterator & arg, const Strings::iterator & end) {
if (*arg == "--help")
@@ -75,13 +72,12 @@ static int main_nix_collect_garbage(AsyncIoRoot & aio, std::string programName,
else if (*arg == "--delete-older-than") {
removeOld = true;
deleteOlderThan = getArg(*arg, arg, end);
} else if (*arg == "--dry-run") {
options.action = GCOptions::gcReturnDead;
} else if (*arg == "--max-freed") {
options.maxFreed = std::max(getIntArg<int64_t>(*arg, arg, end, true), (int64_t) 0);
} else {
return false;
}
else if (*arg == "--dry-run") dryRun = true;
else if (*arg == "--max-freed")
options.maxFreed = std::max(getIntArg<int64_t>(*arg, arg, end, true), (int64_t) 0);
else
return false;
return true;
}).parseCmdline(argv);
@@ -93,12 +89,24 @@ static int main_nix_collect_garbage(AsyncIoRoot & aio, std::string programName,
}
// Run the actual garbage collector.
if (!dryRun) {
options.action = GCOptions::gcDeleteDead;
} else {
options.action = GCOptions::gcReturnDead;
}
auto store = aio.blockOn(openStore());
auto & gcStore = require<GcStore>(*store);
GCResults results;
PrintFreed freed(options.action, results);
PrintFreed freed(true, results);
aio.blockOn(gcStore.collectGarbage(options, results));
if (dryRun) {
// Only print results for dry run; when !dryRun, paths will be printed as they're deleted.
for (auto & i : results.paths) {
printInfo("%s", Uncolored(i));
}
}
return 0;
}
}
+6 -13
View File
@@ -25,7 +25,6 @@
#include <iostream>
#include <algorithm>
#include <ranges>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
@@ -89,16 +88,6 @@ try {
if (i == drv.outputs.end())
throw Error("derivation '%s' does not have an output named '%s'",
store2->printStorePath(path.path), j);
if (!outputPaths.contains(i->first)) {
throw Error(
"Possible SQLite database corruption: derivation '%s' output map contains only "
"outputs '{%s}', not '%s'\n"
"Note: derivation output maps are stored in the SQLite database.",
store2->printStorePath(path.path),
concatStringsSep(", ", std::views::keys(outputPaths)),
i->first
);
}
auto outPath = outputPaths.at(i->first);
auto retPath = store->printStorePath(outPath);
if (store2) {
@@ -711,8 +700,12 @@ static void opGC(std::shared_ptr<Store> store, AsyncIoRoot & aio, Strings opFlag
}
else {
PrintFreed freed(options.action, results);
PrintFreed freed(options.action == GCOptions::gcDeleteDead, results);
aio.blockOn(gcStore.collectGarbage(options, results));
if (options.action != GCOptions::gcDeleteDead)
for (auto & i : results.paths)
cout << i << std::endl;
}
}
@@ -745,7 +738,7 @@ opDelete(std::shared_ptr<Store> store, AsyncIoRoot & aio, Strings opFlags, Strin
auto & gcStore = require<GcStore>(*store);
GCResults results;
PrintFreed freed(options.action, results);
PrintFreed freed(true, results);
aio.blockOn(gcStore.collectGarbage(options, results));
}
+6 -16
View File
@@ -5,8 +5,6 @@
#include "lix/libstore/profiles.hh"
#include "lix/libcmd/repl.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/error.hh"
extern char * * environ __attribute__((weak));
@@ -75,15 +73,6 @@ CopyCommand::CopyCommand()
});
}
void CopyCommand::run()
{
if (requireStore && srcUri.empty() && dstUri.empty()) {
throw UsageError("you must pass '--from' and/or '--to'");
}
StoreCommand::run();
}
ref<Store> CopyCommand::createStore(AsyncIoRoot & in)
{
return srcUri.empty() ? StoreCommand::createStore(in) : in.blockOn(openStore(srcUri));
@@ -91,6 +80,9 @@ ref<Store> CopyCommand::createStore(AsyncIoRoot & in)
ref<Store> CopyCommand::getDstStore()
{
if (srcUri.empty() && dstUri.empty())
throw UsageError("you must pass '--from' and/or '--to'");
return aio().blockOn(dstUri.empty() ? openStore() : openStore(dstUri));
}
@@ -311,10 +303,8 @@ void MixEnvironment::setEnviron() {
throw UsageError("--unset does not make sense with --ignore-environment");
for (const auto & var : keep) {
auto val = sys::getenv(var);
if (val) {
stringsEnv.emplace_back(fmt("%s=%s", var, val));
}
auto val = getenv(var.c_str());
if (val) stringsEnv.emplace_back(fmt("%s=%s", var.c_str(), val));
}
vectorEnv = stringsToCharPtrs(stringsEnv);
@@ -324,7 +314,7 @@ void MixEnvironment::setEnviron() {
throw UsageError("--keep does not make sense without --ignore-environment");
for (const auto & var : unset)
(void) sys::unsetenv(var);
unsetenv(var.c_str());
}
}
+2 -2
View File
@@ -15,6 +15,8 @@ namespace nix {
extern std::string programPath;
extern char * * savedArgv;
class EvalState;
struct Pos;
class Store;
@@ -54,11 +56,9 @@ private:
struct CopyCommand : virtual StoreCommand
{
std::string srcUri, dstUri;
bool requireStore = true;
CopyCommand();
void run() override;
ref<Store> createStore(AsyncIoRoot & in) override;
ref<Store> getDstStore();
+1 -1
View File
@@ -113,7 +113,7 @@ MixEvalArgs::MixEvalArgs()
```
-I nixpkgs=channel:nixos-21.05
-I nixpkgs=https://channels.nixos.org/nixos-21.05/nixexprs.tar.xz
-I nixpkgs=https://nixos.org/channels/nixos-21.05/nixexprs.tar.xz
```
You can also fetch source trees using [flake URLs](./nix3-flake.md#url-like-syntax) and add them to the
+1 -2
View File
@@ -60,8 +60,7 @@ InstallableFlake::InstallableFlake(
DerivedPathsWithInfo InstallableFlake::toDerivedPaths(EvalState & state)
{
auto act =
logger->startActivity(lvlTalkative, actUnknown, fmt("evaluating derivation '%s'", what()));
Activity act(*logger, lvlTalkative, actUnknown, fmt("evaluating derivation '%s'", what()));
auto attr = getCursor(state);
+3 -16
View File
@@ -9,27 +9,14 @@
namespace nix {
typedef std::function<void(AsyncIoRoot &, std::string, std::list<std::string>)> MainFunction;
struct LegacyCommandRegistry
{
typedef std::function<int(AsyncIoRoot &, std::string, std::list<std::string>)> MainFunction;
typedef std::function<
int(AsyncIoRoot &, std::string, std::list<std::string>, std::span<char *>)>
RawMainFunction;
using LegacyCommandMap = std::map<std::string, RawMainFunction>;
using LegacyCommandMap = std::map<std::string, MainFunction>;
static LegacyCommandMap * commands;
static void add(const std::string & name, MainFunction fun)
{
addWithRaw(
name,
[fun](AsyncIoRoot & aio, std::string name, std::list<std::string> args, std::span<char *>) {
return fun(aio, name, args);
}
);
}
static void addWithRaw(const std::string & name, RawMainFunction fun)
{
if (!commands) commands = new LegacyCommandMap;
(*commands)[name] = fun;
+5 -6
View File
@@ -1,4 +1,3 @@
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/error.hh"
#include "lix/libutil/file-system.hh"
#include "lix/libutil/logging.hh"
@@ -111,7 +110,7 @@ static el_status_t doCompletion() {
if (possible.size() == 1) {
const auto completion = *possible.cbegin();
if (completion.size() > s.size()) {
rl_insert_text(requireCString(completion.substr(s.size())));
rl_insert_text(completion.c_str() + s.size());
return redisplay();
}
@@ -135,7 +134,7 @@ static el_status_t doCompletion() {
}
if (len > 0) {
auto commonPrefix = possible.begin()->substr(start, len);
rl_insert_text(requireCString(commonPrefix));
rl_insert_text(commonPrefix.c_str());
el_ring_bell();
return redisplay();
}
@@ -155,7 +154,7 @@ ReadlineLikeInteracter::Guard ReadlineLikeInteracter::init(detail::ReplCompleter
logWarning(e.info());
}
el_hist_size = 1000;
read_history(requireCString(historyFile));
read_history(historyFile.c_str());
auto oldRepl = curRepl;
curRepl = repl;
Guard restoreRepl([oldRepl] { curRepl = oldRepl; });
@@ -203,7 +202,7 @@ bool ReadlineLikeInteracter::getLine(std::string & input, ReplPromptType promptT
};
setupSignals();
char * s = readline(promptForType(promptType)); // NOLINT(lix-unsafe-c-calls)
char * s = readline(promptForType(promptType));
Finally doFree([&]() { free(s); });
restoreSignals();
@@ -224,7 +223,7 @@ bool ReadlineLikeInteracter::getLine(std::string & input, ReplPromptType promptT
void ReadlineLikeInteracter::writeHistory()
{
int ret = write_history(requireCString(historyFile));
int ret = write_history(historyFile.c_str());
int writeHistErr = errno;
if (ret == 0) {
+6 -10
View File
@@ -56,7 +56,6 @@ using NdString = std::unique_ptr<const char, decltype(&lixdoc_free_string)>;
*/
NdString lambdaDocsForPos(SourcePath const path, nix::Pos const &pos) {
std::string const file = path.to_string();
// NOLINTNEXTLINE(lix-unsafe-c-calls): paths are safe
return NdString{lixdoc_get_function_docs(file.c_str(), pos.line, pos.column), &lixdoc_free_string};
}
@@ -209,13 +208,12 @@ struct NixRepl
void printValue(std::ostream & str,
Value & v,
unsigned int maxDepth = std::numeric_limits<unsigned int>::max(),
unsigned int derivationPathDepth = 0)
unsigned int maxDepth = std::numeric_limits<unsigned int>::max())
{
::nix::printValue(state, str, v, PrintOptions {
.ansiColors = true,
.force = true,
.derivationPathDepth = derivationPathDepth,
.derivationPaths = true,
.maxDepth = maxDepth,
.prettyIndent = 2,
.errors = ErrorPrintBehavior::ThrowTopLevel,
@@ -794,10 +792,8 @@ ProcessLineResult NixRepl::processLine(std::string line)
evalString(arg, v);
if (v.type() == nString) {
std::cout << v.str();
} else if (v.type() == nAttrs && state.isDerivation(v)) {
printValue(std::cout, v, 2, 1);
} else {
printValue(std::cout, v, std::numeric_limits<unsigned int>::max(), 0);
printValue(std::cout, v);
}
std::cout << std::endl;
}
@@ -1007,8 +1003,8 @@ Value NixRepl::getReplOverlaysEvalFunction()
Value NixRepl::replOverlays()
{
Value replInits;
auto replInitStorage = evaluator.mem.newList(evalSettings.replOverlays.get().size());
replInits = {NewValueAs::list, replInitStorage};
auto replInitElems = evaluator.mem.newList(evalSettings.replOverlays.get().size());
replInits = {NewValueAs::list, replInitElems};
size_t i = 0;
for (auto path : evalSettings.replOverlays.get()) {
@@ -1049,7 +1045,7 @@ Value NixRepl::replOverlays()
.debugThrow();
}
replInitStorage->elems[i] = replInit;
replInitElems->elems[i] = replInit;
i++;
}
+1 -3
View File
@@ -43,7 +43,7 @@ static_assert(sizeof(Attr) == 2 * sizeof(uint32_t) + sizeof(Value *),
* elements allocated after this structure, while the size corresponds to
* the number of elements already inserted in this structure.
*/
class alignas(Value::TAG_ALIGN) Bindings
class Bindings
{
public:
using Size = uint32_t;
@@ -97,8 +97,6 @@ public:
res.reserve(size_);
for (Size n = 0; n < size_; n++)
res.emplace_back(&attrs[n]);
// NOTE: this sort uses a total order, so the iteration over pointers is not an issue
// NOLINTNEXTLINE(bugprone-nondeterministic-pointer-iteration-order)
std::sort(res.begin(), res.end(), [&](const Attr * a, const Attr * b) {
std::string_view sa = symbols[a->name], sb = symbols[b->name];
return sa < sb;
-96
View File
@@ -1,96 +0,0 @@
---
name: addErrorContext
args: [message, expr]
---
This adds a `message` to be shown in the stacktrace in the event of
a failure during the evaluation of `expr`.
For example, if a file `err.nix` contains the following:
```nix
let
countDown =
n:
if n == 0 then
throw "kaboom"
else
builtins.addErrorContext "while counting down; n = ${toString n}" ("x" + countDown (n - 1));
in
countDown 2
```
Then, evaluating the file will give the following stack trace:
```console
$ nix-instantiate --show-trace err.nix
error:
… from call site
at /home/plop/git.lix.systems/lix-project/lix/err.nix:9:1:
8| in
9| countDown 2
| ^
10|
… while calling 'countDown'
at /home/plop/git.lix.systems/lix-project/lix/err.nix:3:5:
2| countDown =
3| n:
| ^
4| if n == 0 then
… while calling the 'addErrorContext' builtin
at /home/plop/git.lix.systems/lix-project/lix/err.nix:7:7:
6| else
7| builtins.addErrorContext "while counting down; n = ${toString n}" ("x" + countDown (n - 1));
| ^
8| in
… while counting down; n = 2
… from call site
at /home/plop/git.lix.systems/lix-project/lix/err.nix:7:80:
6| else
7| builtins.addErrorContext "while counting down; n = ${toString n}" ("x" + countDown (n - 1));
| ^
8| in
… while calling 'countDown'
at /home/plop/git.lix.systems/lix-project/lix/err.nix:3:5:
2| countDown =
3| n:
| ^
4| if n == 0 then
… while calling the 'addErrorContext' builtin
at /home/plop/git.lix.systems/lix-project/lix/err.nix:7:7:
6| else
7| builtins.addErrorContext "while counting down; n = ${toString n}" ("x" + countDown (n - 1));
| ^
8| in
… while counting down; n = 1
… from call site
at /home/plop/git.lix.systems/lix-project/lix/err.nix:7:80:
6| else
7| builtins.addErrorContext "while counting down; n = ${toString n}" ("x" + countDown (n - 1));
| ^
8| in
… while calling 'countDown'
at /home/plop/git.lix.systems/lix-project/lix/err.nix:3:5:
2| countDown =
3| n:
| ^
4| if n == 0 then
… caused by explicit throw
at /home/plop/git.lix.systems/lix-project/lix/err.nix:5:7:
4| if n == 0 then
5| throw "kaboom"
| ^
6| else
error: kaboom
```
-6
View File
@@ -1,6 +0,0 @@
---
name: appendContext
args: [s, ctx]
---
Appends the attribute set `ctx` as a context to the string `s`, see [`getContext`](#builtins-getContext) for details on the format of the context.
-22
View File
@@ -1,22 +0,0 @@
---
name: derivationStrict
args: [args]
renameInGlobalScope: false
---
Constructs a [store derivation](../glossary.md#gloss-store-derivation) from the attribute set `args`
(c.f. [derivation](#builtins-derivation). Unlike `derivation` the produced store derivation is placed
in the store *immediately* when this builtin is called, while `derivation` may defer placing store
derivations in the store until it is proven that they are used.
It then returns a new attrset with *only* the following attributes:
- `drvPath` containing the path of the store derivation;
- For each output of the derivation (`out`, `dev`, etc): an attribute named after that output containing the output path
> **Note**
>
> In contrast to [`builtins.derivation`](#builtins-derivation), this computes
> the derivation set in a fully *strict* manner, i.e. the values of the attributes
> directly computed, whereas using `builtins.derivation` will produce an attrset
> whose values will be evaluated when they are used at a later point.
+72
View File
@@ -0,0 +1,72 @@
---
name: fetchClosure
args: [args]
experimentalFeature: fetch-closure
---
Fetch a store path [closure](@docroot@/glossary.md#gloss-closure) from a binary cache, and return the store path as a string with context.
This function can be invoked in three ways, that we will discuss in order of preference.
**Fetch a content-addressed store path**
Example:
```nix
builtins.fetchClosure {
fromStore = "https://cache.nixos.org";
fromPath = /nix/store/ldbhlwhh39wha58rm61bkiiwm6j7211j-git-2.33.1;
}
```
This is the simplest invocation, and it does not require the user of the expression to configure [`trusted-public-keys`](@docroot@/command-ref/conf-file.md#conf-trusted-public-keys) to ensure their authenticity.
If your store path is [input addressed](@docroot@/glossary.md#gloss-input-addressed-store-object) instead of content addressed, consider the other two invocations.
**Fetch any store path and rewrite it to a fully content-addressed store path**
Example:
```nix
builtins.fetchClosure {
fromStore = "https://cache.nixos.org";
fromPath = /nix/store/r2jd6ygnmirm2g803mksqqjm4y39yi6i-git-2.33.1;
toPath = /nix/store/ldbhlwhh39wha58rm61bkiiwm6j7211j-git-2.33.1;
}
```
This example fetches `/nix/store/r2jd...` from the specified binary cache,
and rewrites it into the content-addressed store path
`/nix/store/ldbh...`.
Like the previous example, no extra configuration or privileges are required.
To find out the correct value for `toPath` given a `fromPath`,
use [`nix store make-content-addressed`](@docroot@/command-ref/new-cli/nix3-store-make-content-addressed.md):
```console
# nix store make-content-addressed --from https://cache.nixos.org /nix/store/r2jd6ygnmirm2g803mksqqjm4y39yi6i-git-2.33.1
rewrote '/nix/store/r2jd6ygnmirm2g803mksqqjm4y39yi6i-git-2.33.1' to '/nix/store/ldbhlwhh39wha58rm61bkiiwm6j7211j-git-2.33.1'
```
Alternatively, set `toPath = ""` and find the correct `toPath` in the error message.
**Fetch an input-addressed store path as is**
Example:
```nix
builtins.fetchClosure {
fromStore = "https://cache.nixos.org";
fromPath = /nix/store/r2jd6ygnmirm2g803mksqqjm4y39yi6i-git-2.33.1;
inputAddressed = true;
}
```
It is possible to fetch an [input-addressed store path](@docroot@/glossary.md#gloss-input-addressed-store-object) and return it as is.
However, this is the least preferred way of invoking `fetchClosure`, because it requires that the input-addressed paths are trusted by the Lix configuration.
**`builtins.storePath`**
`fetchClosure` is similar to [`builtins.storePath`](#builtins-storePath) in that it allows you to use a previously built store path in a Nix expression.
However, `fetchClosure` is more reproducible because it specifies a binary cache from which the path can be fetched.
Also, using content-addressed store paths does not require users to configure [`trusted-public-keys`](@docroot@/command-ref/conf-file.md#conf-trusted-public-keys) to ensure their authenticity.
-48
View File
@@ -1,48 +0,0 @@
---
name: fetchMercurial
args: [args]
renameInGlobalScope: false
---
Fetch a Mercurial repository. *args* can be a URL, in which case the default
branch of the repo at that URL is fetched. Otherwise, it can be an
attribute with the following attributes (all except `url` optional):
- `url`
The URL of the repo.
- `name` (default: `"source"`)
The name of the directory the repo should be exported to in the store.
- `rev`
The revision to fetch.
This is typically a commit hash.
> **Note**
>
> Currently, `rev` can either contain a revision or a branch/tag name.
The return value is an attrset containing the following keys:
- `outPath` (`string`)
Resulting store path of the fetch process.
- `branch` (`string`)
The branch of the fetch repository.
- `rev` (`string`)
The revision that was fetched.
- `revCount` (`int`)
The number of revsets for this branch.
- `shortRev` (`string`)
The first *12* characters of `rev`.
-59
View File
@@ -1,59 +0,0 @@
---
name: scopedImport
implementation: "[](EvalState & state, Value ** args, Value & v) { import(state, *args[1], args[0], v); }"
args: [scope, path]
renameInGlobalScope: false
---
> **Warning**
>
> This builtin's use is heavily discouraged, it has many drawbacks and may be removed
> in a future version of Lix.
Functions like [`import`](#builtins-import) with the exception that
it takes a `scope`, which is a set of attributes to be added to the
lexical scope of the expression.
This essentially allows overriding the ambient builtin variables.
For example, if `foo.nix` is a file containing the following content:
```nix
x
```
then the following expression
```nix
scopedImport { x = 1; } ./foo.nix
```
will evaluate to `1`.
Another application is overriding builtin functions or constants, e.g. to
trace all calls to `map`, one can do:
```nix
let
overrides = {
map = f: xs: builtins.trace "map called!" (map f xs);
# Ensure that our override gets propagated by calls to
# import/scopedImport.
import = fn: scopedImport overrides fn;
scopedImport = attrs: fn: scopedImport (overrides // attrs) fn;
# Also update builtins.
builtins = builtins // overrides;
};
in scopedImport overrides ./bla.nix
```
Similarly, it can be used to extend the set of builtin functions.
> **Warning**
>
> One of the downsides of `scopedImport` is that it bypasses the evaluation cache.
> This means that importing a file multiple times will lead to multiple expensive
> parsings and evaluations.
+2
View File
@@ -13,3 +13,5 @@ in a new path (e.g. `/nix/store/ld01dnzc…-source-source`).
Not available in [pure evaluation mode](@docroot@/command-ref/conf-file.md#conf-pure-eval).
Lix may change this, tracking issue: <https://git.lix.systems/lix-project/lix/issues/402>
See also [`builtins.fetchClosure`](#builtins-fetchClosure).
@@ -1,33 +0,0 @@
---
name: unsafeDiscardStringContext
args: [s]
---
Returns a copy of the string `s` with all string context associated with `s` removed.
Since string context is used for dependency tracking the returned string will also have
*no dependencies* on store objects, even when the original string `s` had such dependencies.
This function is mainly useful when discarding dependencies is explicitly required, e.g.
to produce a string listing all inputs of a derivation without propagating these inputs
as dependencies into all *users* of the listing. For example, the derivation `dep` in the
following example will pull `hello` into its closure despite never using it while `nodep`
will not:
```nix
dep = runCommand "dep" {
inherit hello;
} "echo hello is at $hello >$out";
nodep = runCommand "nodep" {
hello = builtins.unsafeDiscardStringContext hello;
} "echo hello is at $hello >$out";
```
This behavior also makes this function unsafe: if `s` contains the path of a
store object that is not present in the store then any use of `s` in a
derivation tree will attempt to realize that path in the store, but no use
of `unsafeDiscardStringContext s` will. This can lead to derivation outputs that
refer to paths that were never created.
Lix cannot determine whether such reference are safe or not and must pass
this obligation to the user.
-7
View File
@@ -1,7 +0,0 @@
---
name: warn
args: [msg, e2]
---
Evaluate string *msg* and print it on standard error. Then return *e2*.
This function is useful for warning about unexpected conditions without aborting evaluation.
If the [`debugger-on-warn`](@docroot@/command-ref/conf-file.md#conf-debugger-on-trace) option is set to `true` and the `--debugger` flag is given, the interactive debugger will be started when `warn` is called (like [`break`](@docroot@/language/builtins.md#builtins-break)).
+1 -2
View File
@@ -31,7 +31,7 @@ inline Value::Value(app_t, EvalMemory & mem, Value & lhs, std::span<Value> args)
inline Value::Value(thunk_t, EvalMemory & mem, Env & env, Expr & expr)
{
auto thunk = mem.allocType<Thunk>();
*thunk = {._env = &env, .expr = &expr};
*thunk = {._env = reinterpret_cast<uintptr_t>(&env), .expr = &expr};
raw = tag(tThunk, thunk);
}
@@ -81,7 +81,6 @@ template<typename T>
[[gnu::always_inline]]
T * EvalMemory::allocType(size_t n)
{
// NOLINTNEXTLINE(bugprone-sizeof-expression)
return static_cast<T *>(allocBytes(checkedArrayAllocSize(sizeof(T), n)));
}
+1 -1
View File
@@ -82,7 +82,7 @@ bool EvalSettings::isPseudoUrl(std::string_view s)
std::string EvalSettings::resolvePseudoUrl(std::string_view url)
{
if (url.starts_with("channel:"))
return "https://channels.nixos.org/" + std::string(url.substr(8)) + "/nixexprs.tar.xz";
return "https://nixos.org/channels/" + std::string(url.substr(8)) + "/nixexprs.tar.xz";
else
return std::string(url);
}
+98 -221
View File
@@ -4,14 +4,12 @@
#include "lix/libutil/archive.hh"
#include "lix/libutil/ansicolor.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/deprecated-features.hh"
#include "lix/libutil/error.hh"
#include "lix/libutil/english.hh"
#include "lix/libutil/fmt.hh"
#include "lix/libexpr/primops.hh"
#include "lix/libexpr/print-options.hh"
#include "lix/libmain/shared.hh"
#include "lix/libutil/logging.hh"
#include "lix/libutil/suggestions.hh"
#include "lix/libutil/types.hh"
#include "lix/libstore/store-api.hh"
@@ -41,26 +39,10 @@
#include <sys/resource.h>
#include <fstream>
#include <functional>
#include <ranges>
#include <sys/resource.h>
#include <boost/container/small_vector.hpp>
// Ignore all internal signals boehm uses for parallel marking
// FIXME: Find out how to do this with LLDB for macOS!
#ifndef __APPLE__
[[gnu::section(".debug_gdb_scripts"), gnu::used, gnu::aligned(1)]]
// TODO: We should use `SECTION_SCRIPT_ID_PYTHON_TEXT` from
// `<gdb/section-scripts.h>` instead of hardcoding 4.
// But why isn't this header exported by GDB?
static const char printer_script[] =
"\4"
R"(lix-ignore-boehm-signals
import gdb
gdb.execute("handle SIGPWR SIGXCPU ignore")
)";
#endif
#if HAVE_BOEHMGC
#define GC_INCLUDE_NEW
@@ -195,9 +177,6 @@ void initLibExpr()
GC_INIT();
// Enable parallel marking
GC_start_mark_threads();
GC_set_oom_fn(oomHandler);
/* Set the initial heap size to something fairly big (25% of
@@ -349,61 +328,6 @@ EvalPaths::EvalPaths(
allowPath(path);
}
}
#if LIX_MAJOR >= 3 || (LIX_MAJOR == 2 && LIX_MINOR >= 96)
#warning \
"The feature nix-path-shadow was deprecated in 2.95 with a warning, which needs to be turned into an error in 2.96"
#endif
if (!featureSettings.isEnabled(DeprecatedFeature::NixPathShadow)) {
for (auto & [prefix, path] : searchPath_.elements) {
// Match on the 'nix' prefix
if (prefix.s == "nix") {
logWarning(
{.msg = HintFmt(
"The prefix '%s' is reserved for internal use by Lix in the Nix search "
"path, its usage is deprecated and will be forbidden in the future.\n"
"Use %s to silence this warning.\n"
"This is due to adding '%s=%s' in the Nix search path, either through the "
"environment variable '%s' or by passing the flag %s to the nix "
"invocation.",
"nix",
"--extra-deprecated-features nix-path-shadow",
prefix.s,
path.s,
"NIX_PATH",
"-I"
)}
);
} else
// Match prefixless paths that contain a `nix` directory
if (auto res =
prefix.suffixIfPotentialMatch("nix").and_then([&](std::string_view s) {
return aio.blockOn(resolveSearchPathPath(path))
.and_then([&](std::string r) {
Path res = s.length() ? concatStrings(r, "/", s): r;
return pathExists(res) ? std::optional(res) : std::nullopt;
});
}))
{
logWarning(
{.msg = HintFmt(
"Shadowing '%s' by configuring the nix-path is deprecated and "
"will be forbidden in the future.\n"
"Use %s to silence this warning.\n"
"This is due to adding '%s' to the nix-path without a prefix, "
"either by passing the flag '-I %s' to the nix invocation or by "
"adding this path to the environment variable '%s'.",
"<nix/...>",
"--extra-deprecated-features nix-path-shadow",
path.s,
path.s,
"NIX_PATH"
)}
);
}
}
}
}
Evaluator::Evaluator(
@@ -649,20 +573,20 @@ std::ostream & operator<<(std::ostream & output, const PrimOp & primOp)
return output;
}
void EvalBuiltins::addPrimOp(PrimOpDetails primOp)
void EvalBuiltins::addPrimOp(PrimOpDetails && primOp)
{
/* Hack to make constants lazy: turn them into a application of
the primop to a dummy value. */
if (primOp.arity == 0) {
primOp.arity = 1;
Value vPrimOp{NewValueAs::primop, *new PrimOp(primOp)};
Value vPrimOp{NewValueAs::primop, *new PrimOp(std::move(primOp))};
Value v{NewValueAs::app, mem, vPrimOp, vPrimOp};
addConstant(
vPrimOp.primOp()->name,
v,
{
.type = nFunction,
.doc = vPrimOp.primOp()->doc,
.doc = primOp.doc,
}
);
}
@@ -1361,149 +1285,113 @@ static std::string showAttrPath(EvalState & state, Env & env, const AttrPath & a
return out.str();
}
/** Returns `nullptr` if we should be using a default instead. */
Attr const * ExprSelect::selectSingleAttr(
EvalState & state, Env & env, AttrName const & attrName, Value & vCurrent
)
{
Symbol const attrSym = getName(attrName, state, env);
try {
state.forceValue(vCurrent, pos);
} catch (Error & e) {
// clang-format off
e.addTrace(state.ctx.positions[attrName.pos], HintFmt(
"while evaluating an expression to select '%s' on it", state.ctx.symbols[attrSym]
));
// clang-format on
throw;
}
if (vCurrent.type() != nAttrs) {
// If we have an `or` provided default, then it doesn't have to be an attrset.
// Let the caller know there's no attr value here.
if (def != nullptr) {
return nullptr;
}
// Otherwise, we must type error.
// clang-format off
state.ctx.errors.make<TypeError>(
"expected a set but found %s: %s",
showType(vCurrent),
ValuePrinter(state, vCurrent, errorPrintOptions)
).addTrace(
attrName.pos,
HintFmt("while selecting '%s'", state.ctx.symbols[attrSym])
).debugThrow();
// clang-format on
}
// Now that we know it's an attrset, we can actually look for the name.
auto const attrIt = vCurrent.attrs()->get(attrSym);
if (!attrIt) {
// Again if we have an `or` provided default, then missing attr is not an error.
if (def != nullptr) {
return nullptr;
}
// Otherwise, we collect all attr names and throw an attr missing error.
std::set<std::string> const allAttrNames = *vCurrent.attrs()
| std::views::transform([&state](auto const & attr) {
return std::string{state.ctx.symbols[attr.name]};
})
| std::ranges::to<std::set>();
auto suggestions = Suggestions::bestMatches(allAttrNames, state.ctx.symbols[attrSym]);
state.ctx.errors.make<EvalError>("attribute '%s' missing", state.ctx.symbols[attrSym])
.atPos(attrName.pos)
.withSuggestions(suggestions)
.withFrame(env, *this)
.debugThrow();
}
// If we made it here, then we successfully found the attribute.
// Return it to our caller!
return attrIt;
}
void ExprSelect::eval(EvalState & state, Env & env, Value & v)
{
Value vFirst;
// Pointer to the current attrset Value in this select chain.
Value * vCurrent = &vFirst;
// Position for the current attrset Value in this select chain.
PosIdx posCurrent;
// Position for the current selector in this select chain.
PosIdx posCurrentSyntax;
Value baseSelectee;
try {
// Evaluate the original thing we're selecting on.
e->eval(state, env, baseSelectee);
e->eval(state, env, vFirst);
} catch (Error & e) {
// clang-format off
e.addTrace(state.ctx.positions[getPos()], HintFmt(
assert(this->e != nullptr);
e.addTrace(
state.ctx.positions[getPos()],
"while evaluating an expression to select '%s' on it",
showAttrPath(state.ctx.symbols, attrPath)
));
// clang-format on
showAttrPath(state.ctx.symbols, this->attrPath)
);
throw;
}
try {
// With the original selectee evaluated, we'll walk the selection path starting
// with the evaluated original selectee.
std::reference_wrapper<Value> curSelectee = std::ref(baseSelectee);
for (AttrName const & attrName : attrPath) {
for (auto const & [partIdx, currentAttrName] : enumerate(attrPath)) {
state.ctx.stats.nrLookups++;
// Select `attrName` on `curSelectee`.
auto const attr = selectSingleAttr(state, env, attrName, curSelectee.get());
if (!attr) {
// Use default.
try {
Symbol const name = getName(currentAttrName, state, env);
try {
state.forceValue(*vCurrent, pos);
} catch (Error & e) {
e.addTrace(
state.ctx.positions[currentAttrName.pos],
"while evaluating an expression to select '%s' on it",
state.ctx.symbols[name]
);
throw;
}
if (vCurrent->type() != nAttrs) {
// If we have an `or` provided default,
// then this is allowed to not be an attrset.
if (def != nullptr) {
this->def->eval(state, env, v);
} catch (Error & err) {
err.addTrace(
state.ctx.positions[this->def->pos],
"while evaluating fallback for missing attribute '%s'",
state.ctx.symbols[getName(attrName, state, env)]
);
throw;
return;
}
return;
// Otherwise, we must type error.
state.ctx.errors.make<TypeError>(
"expected a set but found %s: %s",
showType(*vCurrent),
ValuePrinter(state, *vCurrent, errorPrintOptions)
).addTrace(
currentAttrName.pos,
HintFmt("while selecting '%s'", state.ctx.symbols[name])
).debugThrow();
}
// The selection worked. If we have another iteration, then we use `attr->value`
// as the thing to select on. If this is the last iteration, then `attr->value`
// is the final value this ExprSelect evaluated to.
curSelectee = std::ref(attr->value);
// Now that we know this is actually an attrset, try to find an attr
// with the selected name.
auto attrIt = vCurrent->attrs()->get(name);
if (!attrIt) {
posCurrent = attr->pos;
posCurrentSyntax = attrName.pos;
if (state.ctx.stats.countCalls) {
state.ctx.stats.attrSelects[posCurrent]++;
// If we have an `or` provided default, then we'll use that.
if (def != nullptr) {
this->def->eval(state, env, v);
return;
}
// Otherwise, missing attr error.
std::set<std::string> allAttrNames;
for (auto const & attr : *vCurrent->attrs()) {
allAttrNames.emplace(state.ctx.symbols[attr.name]);
}
auto suggestions = Suggestions::bestMatches(allAttrNames, state.ctx.symbols[name]);
state.ctx.errors.make<EvalError>("attribute '%s' missing", state.ctx.symbols[name])
.atPos(currentAttrName.pos)
.withSuggestions(suggestions)
.withFrame(env, *this)
.debugThrow();
}
// If we're here, then we successfully found the attribute.
// Set our currently operated-on attrset to this one, and keep going.
vCurrent = &attrIt->value;
posCurrent = attrIt->pos;
posCurrentSyntax = currentAttrName.pos;
if (state.ctx.stats.countCalls) state.ctx.stats.attrSelects[posCurrent]++;
}
state.forceValue(curSelectee.get(), posCurrent ? posCurrent : posCurrentSyntax);
v = curSelectee.get();
} catch (Error & err) {
auto const & lastPos = state.ctx.positions[posCurrent];
if (lastPos && !std::get_if<Pos::Hidden>(&lastPos.origin)) {
err.addTrace(
lastPos, "while evaluating the attribute '%s'", showAttrPath(state, env, attrPath)
);
}
state.forceValue(*vCurrent, (posCurrent ? posCurrent : posCurrentSyntax));
} catch (Error & e) {
auto pos2r = state.ctx.positions[posCurrent];
if (pos2r && !std::get_if<Pos::Hidden>(&pos2r.origin))
e.addTrace(pos2r, "while evaluating the attribute '%1%'",
showAttrPath(state, env, attrPath));
throw;
}
v = *vCurrent;
}
void ExprOpHasAttr::eval(EvalState & state, Env & env, Value & v)
{
Value vTmp;
@@ -2108,16 +1996,17 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
for (const auto & part : s) result += *part;
return result;
};
/* build a gc'd value string directly instead of going through str()
and mkString to save an allocation and copy */
const auto gcStr = [&] {
auto result = Value::Str::gcAlloc(sSize);
char * tmp = result->contents;
/* c_str() is not str().c_str() because we want to create a string
Value. allocating a GC'd string directly and moving it into a
Value lets us avoid an allocation and copy. */
const auto c_str = [&] {
char * result = gcAllocString(sSize + 1);
char * tmp = result;
for (const auto & part : s) {
memcpy(tmp, part->data(), part->size());
tmp += part->size();
}
*tmp = 0;
return result;
};
@@ -2191,7 +2080,7 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
state.ctx.errors.make<EvalError>("a string that refers to a store path cannot be appended to a path").atPos(pos).withFrame(env, *this).debugThrow();
v.mkPath(CanonPath(canonPath(str())));
} else
v.mkStringMove(gcStr(), context);
v.mkStringMove(c_str(), context);
}
@@ -2462,8 +2351,8 @@ BackedStringView EvalState::coerceToString(
if (v.type() == nPath) {
return !canonicalizePath && !copyToStore
// FIXME: hack to preserve path literals that end in a slash, as in /foo/${x}.
? std::string(v.string().content->str())
? v.string().content // FIXME: hack to preserve path literals that end in a slash, as in
// /foo/${x}.
: (copyToStore
? ctx.store->printStorePath(
aio.blockOn(ctx.paths.copyPathToStore(context, v.path(), ctx.repair))
@@ -2669,6 +2558,11 @@ bool EvalState::eqValues(Value & v1, Value & v2, const PosIdx pos, std::string_v
forceValue(v1, pos);
forceValue(v2, pos);
/* !!! Hack to support some old broken code that relies on pointer
equality tests between sets. (Specifically, builderDefs calls
uniqList on a list of sets.) Will remove this eventually. */
if (&v1 == &v2) return true;
// Special case type-compatibility between float and int
if (v1.type() == nInt && v2.type() == nFloat) {
return v1.integer().value == v2.fpoint();
@@ -2680,11 +2574,6 @@ bool EvalState::eqValues(Value & v1, Value & v2, const PosIdx pos, std::string_v
// All other types are not compatible with each other.
if (v1.type() != v2.type()) return false;
/* !!! Hack to support some old broken code that relies on pointer
equality tests between sets. (Specifically, builderDefs calls
uniqList on a list of sets.) Will remove this eventually. */
auto pointerEq = [&] { return v1.pointerEqProxy() == v2.pointerEqProxy(); };
switch (v1.type()) {
case nInt:
return v1.integer() == v2.integer();
@@ -2696,13 +2585,12 @@ bool EvalState::eqValues(Value & v1, Value & v2, const PosIdx pos, std::string_v
return v1.str() == v2.str();
case nPath:
return v1.string().content->str() == v2.string().content->str();
return strcmp(v1.string().content, v2.string().content) == 0;
case nNull:
return true;
case nList:
if (pointerEq()) return true;
if (v1.listSize() != v2.listSize()) return false;
for (size_t n = 0; n < v1.listSize(); ++n) {
if (!eqValues(v1.listElems()[n], v2.listElems()[n], pos, errorCtx)) {
@@ -2712,7 +2600,6 @@ bool EvalState::eqValues(Value & v1, Value & v2, const PosIdx pos, std::string_v
return true;
case nAttrs: {
if (pointerEq()) return true;
/* If both sets denote a derivation (type = "derivation"),
then compare their outPaths. */
if (isDerivation(v1) && isDerivation(v2)) {
@@ -2737,12 +2624,11 @@ bool EvalState::eqValues(Value & v1, Value & v2, const PosIdx pos, std::string_v
return true;
}
/* Functions are incomparable, except for identity (see note above about this nonsense). */
/* Functions are incomparable. */
case nFunction:
return pointerEq();
return false;
case nExternal:
if (pointerEq()) return true;
return *v1.external() == *v2.external();
case nFloat:
@@ -2981,15 +2867,6 @@ Expr & Evaluator::parseStdin()
);
}
std::optional<DebugTrace const *> Evaluator::nextDebugTrace() const
{
if (!debug) {
return std::nullopt;
}
return debug->traces().next();
}
kj::Promise<Result<EvalPaths::PathResult<SourcePath, ThrownError>>>
EvalPaths::findFile(const std::string_view path)
+1 -4
View File
@@ -265,7 +265,7 @@ private:
void addConstant(const std::string & name, const Value & v, Constant info);
void addPrimOp(PrimOpDetails primOp);
void addPrimOp(PrimOpDetails && primOp);
Value prepareNixPath(const SearchPath & searchPath);
@@ -568,9 +568,6 @@ public:
*/
void evalLazily(Expr & e, Value & v);
/** If debugging is enabled, returns the next trace. Otherwise, std::nullopt. */
std::optional<DebugTrace const *> nextDebugTrace() const;
private:
Expr * parse(
char * text,
+1 -3
View File
@@ -9,17 +9,15 @@ class EvalState;
struct Value;
void prim_addDrvOutputDependencies(EvalState & state, Value * * args, Value & v);
void prim_fetchClosure(EvalState & state, Value * * args, Value & v);
void prim_fetchTree(EvalState & state, Value * * args, Value & v);
void prim_fetchGit(EvalState & state, Value * * args, Value & v);
void prim_fetchMercurial(EvalState & state, Value ** args, Value & v);
void prim_fetchTarball(EvalState & state, Value * * args, Value & v);
void prim_fetchurl(EvalState & state, Value * * args, Value & v);
void prim_fromTOML(EvalState & state, Value * * args, Value & v);
void prim_appendContext(EvalState & state, Value ** args, Value & v);
void prim_getContext(EvalState & state, Value * * args, Value & v);
void prim_hasContext(EvalState & state, Value * * args, Value & v);
void prim_unsafeDiscardOutputDependency(EvalState & state, Value * * args, Value & v);
void prim_unsafeDiscardStringContext(EvalState & state, Value ** args, Value & v);
namespace flake {
+2 -9
View File
@@ -15,12 +15,10 @@ subdir('flake')
libexpr_setting_definitions = files(
# keep-sorted start
'settings/abort-on-warn.md',
'settings/allow-import-from-derivation.md',
'settings/allow-unsafe-native-code-during-evaluation.md',
'settings/allowed-uris.md',
'settings/debugger-on-trace.md',
'settings/debugger-on-warn.md',
'settings/eval-cache.md',
'settings/eval-system.md',
'settings/ignore-try.md',
@@ -53,10 +51,8 @@ builtin_definitions = files(
'builtins/abort.md',
'builtins/add.md',
'builtins/addDrvOutputDependencies.md',
'builtins/addErrorContext.md',
'builtins/all.md',
'builtins/any.md',
'builtins/appendContext.md',
'builtins/attrNames.md',
'builtins/attrValues.md',
'builtins/baseNameOf.md',
@@ -71,13 +67,12 @@ builtin_definitions = files(
'builtins/concatMap.md',
'builtins/concatStringsSep.md',
'builtins/deepSeq.md',
'builtins/derivationStrict.md',
'builtins/dirOf.md',
'builtins/div.md',
'builtins/elem.md',
'builtins/elemAt.md',
'builtins/fetchClosure.md',
'builtins/fetchGit.md',
'builtins/fetchMercurial.md',
'builtins/fetchTarball.md',
'builtins/fetchTree.md',
'builtins/fetchurl.md',
@@ -131,7 +126,6 @@ builtin_definitions = files(
'builtins/readFileType.md',
'builtins/removeAttrs.md',
'builtins/replaceStrings.md',
'builtins/scopedImport.md',
'builtins/seq.md',
'builtins/sort.md',
'builtins/split.md',
@@ -152,9 +146,7 @@ builtin_definitions = files(
'builtins/tryEval.md',
'builtins/typeOf.md',
'builtins/unsafeDiscardOutputDependency.md',
'builtins/unsafeDiscardStringContext.md',
'builtins/unsafeGetAttrPos.md',
'builtins/warn.md',
'builtins/zipAttrsWith.md',
# keep-sorted end
)
@@ -227,6 +219,7 @@ libexpr_sources = files(
'parser/parser.cc',
'primops.cc',
'primops/context.cc',
'primops/fetchClosure.cc',
'primops/fetchMercurial.cc',
'primops/fetchTree.cc',
'primops/fromTOML.cc',
+87 -41
View File
@@ -10,8 +10,7 @@ namespace nix {
ExprBlackHole eBlackHole;
static Env nullEnv;
Value::Thunk Value::blackHole{{&nullEnv}, &eBlackHole};
Value::Thunk Value::blackHole{{0}, &eBlackHole};
// FIXME: remove, because *symbols* are abstract and do not have a single
// textual representation; see printIdentifier()
@@ -35,24 +34,6 @@ AttrName::AttrName(PosIdx pos, std::unique_ptr<Expr> e) : pos(pos), expr(std::mo
{
}
static JSON stringToJSON(std::string_view s)
{
try {
JSON value = s;
// dump it now to catch invalid utf8 strings early. this code path is not
// hot, so the extra memory allocation and encoding is not worth avoiding
(void) value.dump();
return value;
} catch (nlohmann::json::type_error & e) { // NOLINT(lix-foreign-exceptions)
if (e.id == 316) {
// invalid utf8 in string! serialize as byte array instead
return s | std::ranges::to<std::vector<unsigned char>>();
} else {
throw; // NOLINT(lix-foreign-exceptions)
}
}
}
JSON Expr::toJSON(const SymbolTable & symbols) const
{
abort();
@@ -75,7 +56,7 @@ JSON ExprLiteral::toJSON(const SymbolTable & symbols) const
break;
case nString:
valueType = "String";
value = stringToJSON(v.str());
value = v.str();
break;
case nPath:
valueType = "Path";
@@ -95,7 +76,10 @@ JSON ExprLiteral::toJSON(const SymbolTable & symbols) const
JSON ExprVar::toJSON(const SymbolTable & symbols) const
{
return {{"_type", "ExprVar"}, {"value", stringToJSON(symbols[name])}};
return {
{"_type", "ExprVar"},
{"value", symbols[name]}
};
}
JSON ExprInheritFrom::toJSON(SymbolTable const & symbols) const
@@ -131,8 +115,6 @@ void ExprAttrs::addBindingsToJSON(JSON & out, const SymbolTable & symbols) const
typedef const decltype(attrs)::value_type * Attr;
std::vector<Attr> sorted;
for (auto & i : attrs) sorted.push_back(&i);
// NOTE: this sort uses a total order, so the iteration over pointers is not an issue
// NOLINTNEXTLINE(bugprone-nondeterministic-pointer-iteration-order)
std::sort(sorted.begin(), sorted.end(), [&](Attr a, Attr b) {
std::string_view sa = symbols[a->first], sb = symbols[b->first];
return sa < sb;
@@ -141,19 +123,11 @@ void ExprAttrs::addBindingsToJSON(JSON & out, const SymbolTable & symbols) const
for (auto & i : sorted) {
switch (i->second.kind) {
case AttrDef::Kind::Plain:
case AttrDef::Kind::Inherited: {
const auto key = i->second.kind == AttrDef::Kind::Plain ? "attrs" : "inherit";
auto name = stringToJSON(symbols[i->first]);
if (name.is_string()) {
out[key][name] = i->second.e->toJSON(symbols);
} else {
out[fmt("binary_%s", key)][key].push_back({
{"name", name},
{"value", i->second.e->toJSON(symbols)},
});
}
out["attrs"][std::string(symbols[i->first])] = i->second.e->toJSON(symbols);
break;
case AttrDef::Kind::Inherited:
out["inherit"][std::string(symbols[i->first])] = i->second.e->toJSON(symbols);
break;
}
case AttrDef::Kind::InheritedFrom: {
auto & select = i->second.e->cast<ExprSelect>();
auto & from = select.e->cast<ExprInheritFrom>();
@@ -173,7 +147,7 @@ void ExprAttrs::addBindingsToJSON(JSON & out, const SymbolTable & symbols) const
for (const auto & [from, syms] : inheritsFrom) {
JSON attrs = JSON::array();
for (auto sym : syms)
attrs.push_back(stringToJSON(symbols[sym]));
attrs.push_back(symbols[sym]);
out["inheritFrom"].push_back({
{"from", inheritFromExprs[from]->toJSON(symbols)},
{"attrs", attrs}
@@ -211,13 +185,11 @@ JSON ExprList::toJSON(const SymbolTable & symbols) const
void SimplePattern::addBindingsToJSON(JSON & out, const SymbolTable & symbols) const
{
// name must be alphanumeric
out["arg"] = symbols[name];
}
void AttrsPattern::addBindingsToJSON(JSON & out, const SymbolTable & symbols) const
{
// name must be alphanumeric
if (name)
out["arg"] = symbols[name];
@@ -225,7 +197,6 @@ void AttrsPattern::addBindingsToJSON(JSON & out, const SymbolTable & symbols) co
// same expression being printed in two different ways depending on its
// context. always use lexicographic ordering to avoid this.
for (const Formal & i : lexicographicOrder(symbols)) {
// names must be alphanumeric
if (i.def)
out["formals"][std::string(symbols[i.name])] = i.def->toJSON(symbols);
else
@@ -339,7 +310,7 @@ JSON printAttrPathToJson(const SymbolTable & symbols, const AttrPath & attrPath)
JSON out = JSON::array();
for (auto & i : attrPath) {
if (i.symbol)
out.push_back(stringToJSON(symbols[i.symbol]));
out.push_back(symbols[i.symbol]);
else
out.push_back(i.expr->toJSON(symbols));
}
@@ -350,6 +321,74 @@ JSON printAttrPathToJson(const SymbolTable & symbols, const AttrPath & attrPath)
/* Computing levels/displacements for variables. */
namespace {
// This is a one-pass static analyzer for
// various topics.
struct StaticAnalyzer : ExprVisitor
{
std::set<Symbol> staticallyUsedVariables;
bool usedDynamicVariables = false;
StaticAnalyzer() {}
using ExprVisitor::visit;
void visit(ExprDebugFrame & e, std::unique_ptr<Expr> & ptr) override
{
visit(e.inner);
}
void visit(ExprLiteral & e, std::unique_ptr<Expr> & ptr) override {}
void visit(ExprVar & e, std::unique_ptr<Expr> & ptr) override
{
staticallyUsedVariables.insert(e.name);
}
void visit(ExprInheritFrom & e, std::unique_ptr<Expr> & ptr) override {}
void visit(ExprSelect & e, std::unique_ptr<Expr> & ptr) override
{
if (e.isDynamic()) {
usedDynamicVariables = true;
}
visit(e.def);
visit(e.e);
}
void visit(ExprOpHasAttr & e, std::unique_ptr<Expr> & ptr) override
{
if (e.isDynamic()) {
usedDynamicVariables = true;
}
visit(e.e);
}
void visit(ExprSet & e, std::unique_ptr<Expr> & ptr) override
{
// TODO: oh bro, we need to analyze dynamic attributes for their value expressions.
}
void visit(ExprList & e, std::unique_ptr<Expr> & ptr) override {}
void visit(ExprLambda & e, std::unique_ptr<Expr> & ptr) override {}
void visit(ExprCall & e, std::unique_ptr<Expr> & ptr) override {}
void visit(ExprLet & e, std::unique_ptr<Expr> & ptr) override {}
void visit(ExprWith & e, std::unique_ptr<Expr> & ptr) override {}
void visit(ExprIf & e, std::unique_ptr<Expr> & ptr) override {}
void visit(ExprAssert & e, std::unique_ptr<Expr> & ptr) override {}
void visit(ExprOpNot & e, std::unique_ptr<Expr> & ptr) override {}
#define BINOP(type) \
/* NOLINTNEXTLINE(bugprone-macro-parentheses) */ \
void visit(type & e, std::unique_ptr<Expr> & ptr) override \
{ \
visit(e.e1); \
visit(e.e2); \
}
BINOP(ExprOpEq)
BINOP(ExprOpNEq)
BINOP(ExprOpAnd)
BINOP(ExprOpOr)
BINOP(ExprOpImpl)
BINOP(ExprOpUpdate)
BINOP(ExprOpConcatLists)
#undef BINOP
void visit(ExprConcatStrings & e, std::unique_ptr<Expr> & ptr) override {}
void visit(ExprPos & e, std::unique_ptr<Expr> & ptr) override {}
void visit(ExprBlackHole & e, std::unique_ptr<Expr> & ptr) override {}
};
struct VarBinder : ExprVisitor
{
Evaluator & es;
@@ -619,6 +658,13 @@ void VarBinder::visit(ExprLambda & e, std::unique_ptr<Expr> & ptr)
{
withEnv(e.pattern->buildEnv(env.get()), [&] {
e.pattern->accept(*this);
/* TODO: If statically, e.body makes only use of some parameters and not the whole scope.
* We shouldn't have to keep around all the environment data which might contain trapped
* pointers. Analyze `e.body` and return its statically known set of used variables.
* */
DirectCallAnalyzer analyzer{es, env};
analyzer.visit(e.body);
e.shortcut = analyzer.shortcut;
visit(e.body);
});
}

Some files were not shown because too many files have changed in this diff Show More