It's software archaeology time.
- In 2019 (Nix 2.4), Eelco added `checkOverlays`, which strictly checked
that the overlay must be a function of two arguments, one called `final`
and the other `prev`. (dc3f52a144,
gh#3573)
- In 2024, NofairKing opened an issue about checking the exact name
being a silly idea (gh#10516). Instead of trying to make the check more
sensible, the first fix attempt simply expanded the check to allow
`previous` instead of `prev` (the author's preferred name for the
attribute, supposedly; gh#10553). After some discussion, instead a PR
got merged which simply removed the `prev` check altogether, while
leaving the `final` argument check intact for some reason
(ad65a50a94a97bf1f1a1902f43542d28a2e8206b, gh#10572).
- In 2025, over at Lix and oblivious to the latest change over at
CppNix, I came across this stupid check while trying to improve the
abstractions of the AST in Nixexpr. I spent (wasted, retrospect)
considerable amounts of time removing the exact name checks while also
improving the check overall (making it check for more than two
arguments, and also generally improved error message).
(0928d4d87a,
Id4244171123dd8a228be71ce9f04d8e9f647c111)
- Fast forward to 2026, where I run into this becursed piece of code
once again, and once again during some Lixexpr cleanups. Now that I have
seen how carlessly the issue was handled at CppNix, I can't be arsed to
give a flying fuck anymore. Out the code goes. Bye.
With this commit, all casts on `Expr` subclasses outside of libexpr
itself have been removed. No more violation of abstraction boundaries.
Good riddance.
Change-Id: I939968bb01d461764cfa0f4ea7152b4fcf1acf93
pour the foundations for rpc, and let them set. this is very much
unstable and must be opted into with explicit store uris (e.g. by
setting `NIX_REMOTE=daemon?protocol=any`). the daemon sockets are
not enabled by default and must be enabled with the `rpc-sockets`
experimental feature. we will not advertise this just yet because
in the current state it one has to be *very* dedicated to the rpc
cause to deploy this, but once we have some more bits migrated we
may want to add release notes and officially as for beta testing.
Co-Authored-By: piegames <git@piegames.de>
Change-Id: I85a96ccb700b91190c1eb37154bcc6ae1c03401a
this adds the protocol registry. we also extend the systemd unit setup
to allow adding new protocols without much fussing, and we move deamon
instances into protocol-dependent scopes. this will make monitoring or
resource limiting of individual protocol daemon instances much easier.
Change-Id: Ic9da08eb074868b6e57385075c5502b081e8461f
string data shares a buffer with the binary string length field. size
calculations for string read buffers always include the length field;
sufficiently large length fields can cause these calculations to wrap.
a malicious nar could use this for OOB writes in the daemon (as root).
since we use strings only as tags for archive members and for symlinks
with their OS-dependent length limits we can simply limit string size.
1 MiB should be sufficient for all symlinks, and tags are always tiny.
Change-Id: I89fb05f73c1dbeda45d91244aba4cd526a3d83e1
This isn't really relevant for a setting of 10k, but now that we want to
set it to 0 for Flakes it makes a difference as to whether or not one
top-level function call is allowed or not. (It shouldn't be)
Change-Id: I6bbf99826af0289c232b9d45d172a378b81abe4b
Caught by edef's harness. I rewrote the structured build log tests in
f2 and added a test for this one.
Before: it failed the build altogether:
```
building '/tmp/jade/pytest-of-jade/pytest-77/test_invalid_fields_files0_0/nix/store/skhjs5zs082nqiakj69qz32gyywz0v1a-unusual-loggi
ng.drv'...
error: unsupported JSON type 7
```
Now the output is as expected:
```
warning: Unable to handle a JSON message from the derivation builder: error: unsupported log field type number
@nix {"action": "start", "fields": [1.5], "id": 2, "type": 1, "level": 1, "text": "abc"}
```
Change-Id: Idbd28c5a4d4cf15f27207f83b730e7946a6a6964
This is a minor adjustment to 2b22dae1ba /
I3e9d7c1c7a6599a8e68302448bbb961d051002b7
Basically having the error point to the operand instead of the operator
kind of makes sense, but it would require error spans to *truly* make
sense, and in the meantime maintaining the logic is more hassle than it
is worth.
Change-Id: Idac98dd77a0f4a6cb386cb74e0d4eb5fd2f503e5
"while evaluating a path segment" was just plain wrong
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I827374635df2887d17a8ee8b1755c078bf0b506b
It's useful to obtain a fully assembled command and then wrap it to
execute into something else, e.g. gdb, strace or fakeroot.
Change-Id: Ida5928fd001925549eab89edfbf0facada7ef7f1
We choose to throw an error in the invalid-input space where we *can*
produce a valid (but wrong) result to allow ourselves to, mirroring
CppNix, change the semantics to not corrupt it at all in the future,
while having a middle state where it is an error.
This is a largely-rewritten version of
https://github.com/NixOS/nix/pull/13013.
Co-authored-by: rootile <lix@rootile.de>
Fixes: https://github.com/NixOS/nix/issues/12899
Upstream-PR: https://github.com/NixOS/nix/pull/13013
Change-Id: I6a6a6964cdd1a88714952e80c660d1fa57d1f2d8
before this change, the innermost frame was all the way at the top,
which means that in very nested code you'd have to scroll though
dozens or hundreds of frames before even seeing where you are, because
the last lines (which are immediately visible when the command returns)
would be the nix code's entry point and other outer frames instead,
which is rarely relevant.
this CL reverses this order, so that the innermost frames are the last
ones printed, and thus are immediately visible. (note that this is
already how errors are printed by nix in other contexts anyway, the
debugger's :bt is the only thing that prints the trace in "forward"
order.)
because `DebugState.traces()` use the homegrown `libutil::Generator`
instead of an stdlib container, we have to do that reversing ourselves,
in this case by just storing every element in an `std::list` (in reverse
order) and then traversing it. this is feels kinda dirty, but i don't
know any other way, and it's not exactly performance-critical.
Change-Id: I9f23e40e57f72a251d65a335b3bba3c3f77b935d
it seems like the repl_characterization tests have not been actually
running correctly for a while, since tests from more than 2 years ago
that should have failed have been in the test suite and OK'd thus far.
the reason why some tests passed when they shouldn't have is because
the AST nodes used by the characterization parser didn't have a correct
comparison operator, so the tests would only fail when one of the output
line was of the wrong *kind*, but not if it had the wrong *value*. this
was fixed by simply adding a default comparator to the TextNode type as
well as every derived class.
i've taken the occasion to fix the tests that were failing now that we
are actually being strict about them. most of these are pretty boring
changes like indentation, but if possible i'd like to get an explicit
ACK from at least one or two people on the core team that these changes
are okay.
for reference, here is the zulip thread that lead to this CL:
https://zulip.lix.systems/#narrow/channel/8-Testing-.2F-functional2/topic/.E2.9C.94.20repl-characterization.20tests.20not.20failing/near/10276
ps: i had to turn clang-format off around the macro for text node
declarations because it re-formatted the entire macro, causing a lot
of noising and producing uglier code; if this isn't okay or if there's
a better alternative, please let me know!
Change-Id: I1d6d92eb3e9cb705ecfeccd7b9294d6cfd7fe25b
This started by noticing that the coverage report generated by cl/5489
showed that the error message for when allow-import-from-derivation is
disabled appeared to be not covered by any tests. Since the existing
tests for IFD were in functional, which is deprecated, I migrated those
into functional2 and added a test for erroring when IFD is disabled.
Change-Id: I565674de3dd693e023ec99d501adc4706a6a6964
notably the bad-store test previously tested *bash*, not lix. we also
need to extend the list of non-fatal errors, but that's probably fine
Change-Id: I983f4adc8047fe800323099de4037711869ee547
I did a pass through all files, fixing bits that have annoyed me here
and there based on vibes. Roughly:
- Replaced some out values like bools or strings with lists where
appropriate. Those tests were likely older than the value printer
- Added some parser tests where it makes sense
- Gently touched some formatting woes, while trying to not cause too
much diff noise
- Removed some dead `with (import ./lib.nix);` code
Change-Id: I8c40b2110f0b7799f68ae38ba61f049c5f1f6ee8
Related to #1162
Currently if JSON with duplicate object keys is passed into `builtins.toJSON`,
it will silently drop all but the last value, keeping only the last.
This may be surprising, but as this has been consistent reliable behavior that
users might depend on, we should test for it.
Change-Id: Icc2adefabb161530e7cbfa7330919bab6a6a6964
Without this, the primop chokes on any thunks on attributes passed in
the attrset. It even is the reason why the test contained `builtins.seq`
to work around this. Supposedly, this might have been an intentional
restriction and changing this might break things in ways I cannot forsee
due to not knowing much about Flakes, however the status quo is equally
broken:
- The error message looks like an internal error and not like some
explicitly forbidden invariant violation.
- Seemingly simple syntax literals like "-1" compile to "__sub 0 1" and
thus create a thunk which then fails, which is utterly confusing ("why
does 1 work but not -1?")
- This is a stark violation of the principle of least surprise.
- Thunking relies on maybeThunk and thunk inlining optimizations, thus
not forcing thunks turns operational details of the evaluator into
language-observable behavior. That's bad.
I am changing this now regardless of the risk of breakage, because the
bytecode evaluator will have different thunk inlining optimizations and
thus inevitably cause mismatches in behavior anyways.
Change-Id: Ifc45c4d2900e40822383670b28e4e50ab8af317a
This is a small tweak to the logic added in cl/1511 to detect explicit
throws when printing stack traces. Now when deciding whether to print
"caused by explicit throw", it checks not only that the error is a
ThrowError and that we are in a throw, but also that the ThrowError was
thrown by *this* throw, and not by another while in the process of
evaluating this throw's operand.
It turns this:
```
let
set = {
inner = throw "nested throw";
};
in
throw set.inner
error:
… caused by explicit throw
at /pwd/in.nix:6:1:
5| in
6| throw set.inner
| ^
7|
… while evaluating the attribute 'inner'
at /pwd/in.nix:3:5:
2| set = {
3| inner = throw "nested throw";
| ^
4| };
… caused by explicit throw
at /pwd/in.nix:3:13:
2| set = {
3| inner = throw "nested throw";
| ^
4| };
error: nested throw
```
into this:
```
error:
… while calling the 'throw' builtin
at /pwd/in.nix:6:1:
5| in
6| throw set.inner
| ^
7|
… while evaluating the attribute 'inner'
at /pwd/in.nix:3:5:
2| set = {
3| inner = throw "nested throw";
| ^
4| };
… caused by explicit throw
at /pwd/in.nix:3:13:
2| set = {
3| inner = throw "nested throw";
| ^
4| };
error: nested throw
```
Notice the difference in the top context frame. Before it incorrectly
attributed the throw error to the both throws instead of just the one
that actually threw.
Change-Id: If3b0b8311f1ae2ff1471e260fe59d9166a6a6964
A significant part of #1136.
This is a rather large cl, because all of the primops need to be changed
together.
Change-Id: I7d92698d5344bd6186ee4fa47f5c21966a6a6964
Closes#496
When running
nix-build ../nixpkgs --arg config.allowUnfree true -A hello-unfree
the package `hello-unfree` is now built rather than getting an
eval-error rejecting instantiation. This is because `config.allowUnfree`
is now interpreted as nested attribute-set declaration, similar to how
it's done in `nix repl`.
To prevent sudden breakage, this behavior was carefully deprecated with
Nix throwing an error if the identifier for `--arg` is not a pure
identifier, but an expression as above.
Any kind of merging is rejected. I.e. doing
nix-build ../nixpkgs --arg config '{cudaSupport = true;}' --arg config.allowUnfree true
is prohibited. That way we don't have to think about merge semantics for
cases like this (or even worse `--arg config 'rec { ... }'`). Another
nice side-effect of this is that we don't need to create an EvalState to
force the values and implement merging.
Change-Id: I8b560883a4468a3f32f915764b08f5fdd8fe71bb
While it doesn't make sense to have `foo..bar`, the attribute-path
`foo."".bar` is valid and shouldn't throw.
Change-Id: Ifcddaad6233c6ba8f17cb5c953c2101d276dfeb6
previously, `builtins.break` didn't force its argument, resulting in
a value wrapped with `break` being opaque to most builtins if not
also wrapped with `seq`. see [lix-project/lix#1165] for more details
on what this can break.
this tiny fix just adds a call to `forceValue` inside `prim_break`,
but unfortunately this "breaks" a few existing tests because it
changes the call stack; those tests' golden outputs have been adjusted
without modifying their intended purpose.
Fixes#1165
[lix-project/lix#1165]: https://git.lix.systems/lix-project/lix/issues/1165
Change-Id: I5fe4ee3ff28b38aaf924125b8978130812e58fef
See [lix-project/lix#1156], but basically currently the `:st <n>`
debugger command doesn't allow any negative indices, and putting a plus
sign in front of the arg doesn't change anything; thus, we can exploit
that "design space" to allow users to move between different stack
frames easily, by simply prepending their arg with a +/- sign.
The actual behavior is little more nuanced when you account for errors:
as suggested by @pennae (thanks! :), when the user inputs an offset that
would result in an invalid frame index, the debugger instead clamps it
to the closest bound (i.e. 0 for negative offsets, $maxFrame for
positive ones) and just prints a warning.
Fixes#1156
[lix-project/lix#1156]: https://git.lix.systems/lix-project/lix/issues/1156
Change-Id: I02a0cdb6aaebbdb0515308880a3bf9c0d2fcd25e
currently, the `:st <n>` command in the debugger will simply silently
fail if the argument cannot be converted to an integer, or if it falls
outside the range of valid stack indices. this isn't too big of problem,
but it can be nicer to tell the user something went wrong, rather than
not give them any output and having them guess (esp. in the second case).
this commit adds two errors, one for each case:
1. the argument is not actually an integer, or is outside INT_MIN/MAX
-> "argument '%arg' is not a valid integer"
2. the argument is an integer outside the range of stack traces
-> "stack index must be between 0 and %max_frame, but was %arg"
Change-Id: I8109feeede79a9ad3db9ee7dc95d37e7dd19741a
not doing this has lix crash if stdio forwarding fails for odd reasons,
such as ssh failing to set up a connection due to transient dns errors.
cf #1148
Change-Id: I543b7444236e69d3c13b47fb6c6eb37c65b96763
This is intended to be a softer version of
`allow-import-from-derivation`, allowing for users to only flag when IFD
is encountered rather than error out
Based-on: https://github.com/NixOS/nix/pull/13279
Co-authored-by: gustavderdrache <alex.ford@determinate.systems>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Change-Id: I879a444e3a24b4e82a8cea3a5a94834c78c3924a
`builtins.storePath` allows you to use "external" knowledge of present
store paths in your store to be used in your expressions.
This is sometimes used to avoid double wrapping store paths, for example
when dealing with nixpkgs sources (`builtins.storePath pkgs.path` is a
frequent pattern).
Unfortunately, using your external knowledge of present store paths is
technically impure because Flakes is about not depending on the *STATE*
of your store, no matter what you know about it or what you do to ensure
that your knowledge is accurate.
That being said, `builtins.storePath` is already possible in the
fragment of "pure Nixlang" via a clever use of `builtins.appendContext`.
Precisely:
```
storePath = path:
let path' = builtins.unsafeDiscardStringContext path;
in
# NOTE: merging the context set in all generality is impossible
because getContext on a pure path doesn't work.
builtins.appendContext path' { ${path'} = { path = true; }; };
```
works in pure contexts.
Because of this, we deem that storePath should be allowed in pure
contexts and users do not have to come up with a `storePath` polyfill.
Fixes#402.
Change-Id: I5fff750a4feed082745431835db451f945249cfb
Signed-off-by: Raito Bezarius <raito@lix.systems>
nix3 CLI never honored that setting.
When we update a nix3 profile element with new built store paths for
which we know the deriver (the derivation path which built it), we add
the derivation paths to the profile element to prevent garbage
collection the same way the nix2 CLI logic does.
Fixes#1095.
Change-Id: Icc236d174c5ce5ffe042d1c8e1968f6cb5b50359
Signed-off-by: Raito Bezarius <raito@lix.systems>
In messages printed by the evaluator, we use showType(.) to say things
like "expected X, got Y" where X, Y can be string, string with context,
etc.
After changes in the Lix data model (pointer tagging and friends), paths
got squished inside a internal string type.
We missed updating `showType` to take this fact into account which
caused an unfortunate:
`error: expected a string but found a string with context:
/nix/store/wjf8rdpp63rhyasbzc7zfms1agf7pwyd-source`
while using a path on `builtins.getContext`
A test is added.
Change-Id: Ifb0129ead2ea1973fc17dc0133f8a694995536b1
Signed-off-by: Raito Bezarius <raito@lix.systems>
This partitions a string into two parts based on the first occurrence of
the separator.
This function handles the case when the separator never occur in the
target string by returning an optional<T> in the second argument of the
pair.
Change-Id: I9ad44375dbaa137a4d21d2408bc2ffa34fab62a7
Signed-off-by: Raito Bezarius <raito@lix.systems>
When `max-jobs = 0`, our async semaphores have null capacity and
therefore no slot token can be acquired.
In addition, a derivation with `preferLocalBuild = true` with `max-jobs
= 0` which is not a built-in derivation is not considered suitable for
being possible to build.
Our documentation in `max-jobs` writes:
> `0` is useful when using remote builders to prevent any local builds
(except for preferLocalBuild derivation attribute which executes locally
regardless).
A valid interpretation is that `preferLocalBuild` derivations (all of
them) will always execute even if `max-jobs = 0`.
This CL achieves this promise by enabling `preferLocalBuild = true`
derivations to draw in priority capacity from a `preferLocalBuild`-only
pool and then fall back to the build pool (`maxBuildJobs`).
If there's non-zero max-jobs, we make the `preferLocalBuild`-only pool
empty.
If there's zero max-jobs, we make the `preferLocalBuild`-only pool
non-empty (capacity: 1).
Remark 1: if `max-jobs = 0`, no non-`preferLocalBuild = true`
derivations can be built without a remote builder and `preferLocalBuild
= true` derivations can be built one at a time if `extra-local-jobs` is
not overridden.
Remark 2: if `max-jobs != 0`, no `preferLocalBuild = true` derivations
can bypass capacity limits, i.e. `max-jobs` unless `extra-local-jobs` is
overridden.
Fixes#855.
Change-Id: I00a0acac7375a881aec6281ebb13a4530b26e460
Signed-off-by: Raito Bezarius <raito@lix.systems>
When ref/rev are very similar looking, e.g. a revision SHA1, the logic
to reconstruct whether we had a ref or rev initially cannot work.
This regression was introduced by
I2c72d5a32e406a7ca308e271730bd0af01c5d18b which passed the information
as an attribute field called `refOrRev`, instead, we pass `ref` or `rev`
but not `refOrRev`.
We apply the same post-processing logic to distinguish rev/ref in
path-style flake references: `github:owner/repo/$refOrRev`.
Fixes#520#841.
Change-Id: I2b1985cd5f351f43fea84232c1972715e2408fb1
Signed-off-by: Raito Bezarius <raito@lix.systems>
piegames