Commit Graph
318 Commits
Author SHA1 Message Date
Jade Lovelace 276add2cd7 repl: fix repl-overlays in pure eval mode
The reason this gets hit is because of the debugger in flakes. Otherwise
you never have a repl in pure mode anyway.

We evaluate the repl-overlay file in impure mode but this doesn't do
what one would initially expect.

Fixes: https://git.lix.systems/lix-project/lix/issues/777
Change-Id: I19b8ed2f5e9ce500b633b13301b42df69ab7deb3
2025-06-25 14:15:37 +00:00
Jade Lovelace 38850e59e1 repl-characterization: delete duplicate extra_data directory
idk how this mistake happened but it was really confusing to figure out
which one of these was right, so let's get rid of the impostor.

Change-Id: If3b6fb543e5976b1edad68fb143bfa994d1d6381
2025-06-24 22:20:28 +00:00
eldritch horrorsandRaito Bezarius 749afbbe99 libstore: don't default build-dir to temp-dir
if a build directory is accessible to other users it is possible to
smuggle data in and out of build directories. usually this ins only
a build purity problem, but in combination with other issues it can
be used to break out of a build sandbox. to prevent this we default
to using a subdirectory of nixStateDir (which is more restrictive).

Fixes CVE-2025-52991.

Change-Id: Iacfc9b50534de158618c815f9fb99d7dae1be4d0
2025-06-24 10:49:49 +00:00
Raito Bezarius 21dbd7745d libstore/binary-cache-store: skip NAR listings if it's not possible to serialize it
Some source trees might not be representable inside of the NAR listing
format v1 as file paths (on Linux) are not guaranteed to be valid UTF-8.

When something like this happens on a large-scale build farm, a
mysterious "queued" but impossible to process job appears, this is
because we cannot write the NAR listing and serialization always fails.

Why did this work before? nlohmann was introduced _after_ such paths
were ingested, see: 09f00dd4d0.

What happened for such previously mis-serialized NAR listings?

```
curl -v 'https://cache.nixos.org/nz8p9hn00r6z7s57581c1hiv39pa1ia6.ls' |
brotli -d | jq .
```

This fixes the build of `sub-batch`
(https://github.com/kl/sub-batch/tree/master/tests/rename_invalid_utf8)
on ForkOS infrastructure.

Many thanks to Puck for the assistance on holding `rr` right on this one
and finding the history of these changes.

Change-Id: I2c2fbac70818e02810f9fd236c3a248187bf5fe7
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 15:22:22 +00:00
piegames 019b17f4e9 tests/functional2/lang: Migrate trivial eval-okay tests
Change-Id: I07a2e70eacd3d7bca3fc4d7074b9892b9ea35346
2025-06-02 21:52:34 +02:00
piegames 9673c6480a tests/functional2/lang: Migrate trivial parse-okay tests
Change-Id: I11b6ea26b7dc6bcf8250c04d3c97ded452dd1eda
2025-06-02 21:52:34 +02:00
piegames 0219434ae9 tests/functional2/lang: Migrate trivial parse-fail tests
Change-Id: If6cd6a2432c081e4b918f480ce3be0da3e691d40
2025-06-02 21:52:34 +02:00
piegames 63edb9c678 tests/functional2/lang: Migrate trivial eval-fail tests
All changes are uniform and done with the same script, so checking only
some should suffice. For that reason, any tests involving multiple files
or custom CLI flags are not included in this commit.

Change-Id: Ib2d0e08937b56e241d99771a58aad34ed3ad308a
2025-06-02 21:52:34 +02:00
Linus Heckemann e753fcb414 gc: delay throwing error until cleanup is complete
Previously, paths not being deleted by gcDeleteSpecific would result in
(a) hardlinks not being cleaned up, and
(b) statistics not being reported correctly.

By throwing the error later, we fix both of these problems.

Change-Id: I8019f3e10d9f22e81ea87bb26b77f04ebc888a19
2025-06-02 20:05:32 +02:00
Victor Fuentes ab1e58e948 feat(nix store ls): support reading nar listings from binary cache
Remote binary caches support `write-nar-listing` options where they create a `HASH.ls` file for quick indexing without having to download the nar.
This commit makes experimental `nix store ls` attempt to read these files instead of downloading the full nar.

The difference is very obvious with large packages like stellarium:

nix store ls --store "https://cache.nixos.org" /nix/store/ijpvwgs9zamqaax5dy2cd0kxgz7lr7an-stellarium-25.1 -R

Change-Id: I6a37e0788b3a91c319331a8de69c51daf3efa955
2025-06-01 22:47:11 -07:00
Commentator2.0 c63ba3c485 tests/functional2/lang: migrated first tests
Change-Id: I4b5755a63d9454db20f63f751f5f33564a2ee5be
2025-06-01 20:19:37 +02:00
Alois Wohlschlager 4505bfac8e libstore/gc: fix auto-GC blocking indefinitely during evaluation
The first auto-GC request would not be registered as a waiter due to a logic
error. As a result, if that request was synchronous (as happens during
evaluation) it would be stuck forever waiting on a promise that will never be
fulfilled.
Register also the first request properly so that it is notified and unblocked
again when the GC has finished. Also add a test verifying that auto-GC
triggering during evaluation will not get stuck.

Fixes: https://git.lix.systems/lix-project/lix/issues/844
Change-Id: I157afdc737415261e48d6d01d46c586a2927a1ad
2025-06-01 18:59:23 +02:00
0c2ced0224 feat(nix-instantiate): add --raw flag
The experimental `nix eval` command already supports a `--raw` flag.
This commit implements the same flag for the stable nix-instantiate command.

Until now instructions and scripts that didn't want to rely on experimental
features had to use workarounds such as:

    nix-instantiate --eval <something> | tr -d \"

(which also undesirably also removes double quotation marks within the string), or

    nix-instantiate --eval <something> | jq -j

(which undesirably depends on another package).

Co-authored-by: Raito Bezarius <raito@lix.systems>
Co-authored-by: Silvan Mosberger <silvan.mosberger@tweag.io>
Change-Id: Iced9a80ee7edd60af2385c5193485f1774175339
2025-05-29 15:25:12 +02:00
Tom Hubrecht e468102508 fix: Consider fetchGit locked when narHash is present
`fetchGit` has been modified a long time ago to use fetchTree, however,
we don't care about `lastModified` because we are not in a flake
context, this hack introduces a `git-locked` type of input that only
cares about `narHash` being present. This is needed to avoid fetching
the remote repo each time `fetchGit` is evaluated whith the result
present in the store.

Change-Id: I521c6fcccf8cf12945594f205d7fd4c8c2cf89e9
2025-05-28 22:24:23 +00:00
Raito Bezarius b792279780 tests/functional/lang: update error path locations
The coerce integer feature was not rebased before merge and we do not
have a merge queue, hence, after merge, the HEAD was in a broken state.

We take a commitment to invest into a merge queue now and do a fixup
here.

Change-Id: Ied9410690b542359859ab5f597f22ebceb857305
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-05-28 22:29:04 +02:00
Tom HubrechtandLix Systems Gerrit 0d72109ada Merge "libfetchers: factorize inputFromAttrs" into main 2025-05-28 19:54:37 +00:00
Raito BezariusandLix Systems Gerrit 316aa591ac Merge "libexpr: coerce integers under the XP feature coerce-integers" into main 2025-05-28 19:39:59 +00:00
git@71rd.net 0e115a4828 libutil/args: fix crash when NIX_GET_COMPLETIONS is not a number
When assigning an a value to NIX_GET_COMPLETIONS that could not be
parsed as an integer lix would just crash, as the value was directly
passed to stoi, without handling the return value.
This change switches the parsing to use string2Int and throws an
exception if the return value is empty.

The behaviour of lix is slightly changed through, as the value of the
variable was previously parsed to an int and then assigned to a variable
of size_t.
This change in behaviour can only be observed in cases where the
value of NIX_GET_COMPLETIONS is chosen so when it overflows it would
be valid index of the provided arguments again.

Through this change the variable is parsed as a size_t and negative
values are rejected.

Change-Id: Idf7c5740274c6e07d5bb13d7e2ed32764bfc27f8
2025-05-27 22:20:22 +00:00
git@71rd.net 20d50b049d libutil/args: dont crash completion when receiving incorrect number of arguments
When using completion, the number of the word for which the shell
requests completion is provided in the environment variable
`NIX_GET_COMPLETIONS`. When the number smaller than 1 is or larger
than the number of arguments nix coredumps as a assert is violated.

This change removes the assert and instead throws an exception informing
the user that their autocomplete is most likely misconfigured.

Change-Id: I821719e470e576b6f63c06beb097338b53d183e0
2025-05-27 14:36:51 +00:00
Raito Bezarius 1e40171ea4 libexpr: coerce integers under the XP feature coerce-integers
This introduces a new (demanded?) feature for coercing integers in
interpolation arguments under the experimental feature
`coerce-integers`.

This feature is being introduced behind an *experimental feature flag*
due to the cautious approach we're taking. The codebase has a track
record of revealing unexpected behaviors, often in subtle ways, so we
want to give this sufficient time and exposure before making it stable.

To remove the experimental flag, we want to see **at least two releases
or six months of real-world usage -- whichever is longer** -- that
demonstrate strong confidence the feature doesn't introduce regressions
or unintended side effects. If that level of confidence is reached,
we'll proceed to stabilize it.

Change-Id: I825904719eeba8f0e2a93cd6b93cfe6cebd7d827
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-05-27 11:42:53 +02:00
piegames 1071643259 tests/functional/lang: Clean up lib.nix
That file was written once in 2008 and never updated since, and let's
just say that a lot of things have changed since

Change-Id: I66b0c87ecbba6ca653470966c9514edb21882ca3
2025-05-24 08:49:09 +02:00
Tom Hubrecht 76524b92ee libfetchers: factorize inputFromAttrs
Each `inputFromAttrs` is roughly the same function in each class, we
check that the attributes given are correct (in term of keys and other
types) then we coppy the attributes. Instead of having the same code
copied in 10 places, set it in the parent class and specify what is
specific per child class.

Change-Id: If9aecb76cff1e28a1ef6668d83d825686cce8353
2025-05-22 14:00:19 +02:00
piegames 5d49e26f71 tests/functional/lang: Remove disabled tail-call test
It was introduced back in 2013, was disabled in 2014 again for dubious
reasons and according to horrors is unsound anyways and can never really
work.

It was the only disabled test, so I removed the "infrastructure" for
that in the test runner as well. functional2/lang will have much better
ways for skipping tests anyways

Change-Id: Icb8697fb85221e3206fb64cb917c03607ef278a7
2025-05-22 01:48:34 +02:00
piegames 76d6b51f5c tests/functional/lang: Don't pipe input into stdin
Back in the days, this used to be the modus operandi, but then, still
many but less years ago, Eelco came along and changed it to passing in
the actual file. Of course, no motivation was provided, and it was only
done on half of the test runners for some reason, leaving us to wonder
what the true intentions of this code are …

Anyways, with this commit now everything standardises on passing in the
file by path instead of via stdin. Motivation:

- We need to `sed` out the path anyways for various other reasons,
including import tests and path value tests
- Given that, the presumed primary motivation for using stdin in the
first place becomes moot
- Bonus points for giving better error messages, especially in tests
that involve multiple input files

Change-Id: Ic6de1ec24f4c4d3c05e33d1ee053614784677513
2025-05-22 01:48:34 +02:00
piegamesandLix Systems Gerrit f92199d651 Merge changes I186b0edb,Ie394691b into main
* changes:
  tests/functional/lang: Don't use tabs for indentation
  tests/functional/lang: Change base path to the lang directory
2025-05-21 22:17:42 +00:00
piegamesandCommentator2.0 e52cad0285 tests/functional/lang: Don't use tabs for indentation
Don't ask me the fuck why, but *somehow* Nix prints error locations
differently if the input file is passed as a path vs through stdin, and
I have a hunch that this might have to do with tabs

Change-Id: I186b0edb90edd48856da3621815463e372c37512
2025-05-21 20:36:20 +02:00
piegamesandCommentator2.0 e2944876cc tests/functional/lang: Change base path to the lang directory
Currently, all tests are relative to `./tests/functional` instead of
`./tests/functional/lang`. Whether this is a historical artefact or as
intended, the current move is to align the tests with the new design of
functional2, preparing them for an easier migration.

Change-Id: Ie394691b071488a8000a005080b9167786d5bd9a
2025-05-21 20:36:20 +02:00
Raito BezariusandLix Systems Gerrit 4f433a6186 Merge changes I9f893374,Ief7a4756 into main
* changes:
  libexpr: rename `forceString` to `isInterpolation`
  libexpr: refactor string coercion modes
2025-05-20 20:45:21 +00:00
eldritch horrors 03da670021 libutil: remove ca-derivations experimental features
also remove all the documentation referencing it, or rewrite the docs
to make sense in the non-floating-content-addressed world we live in.

Change-Id: I724e67839f44cc9f1cfc7d6f1c05252b62752b42
2025-05-20 17:43:46 +00:00
Raito Bezarius 3e4bffcc24 libexpr: rename forceString to isInterpolation
`ExprConcatStrings` tracks whether the expression is an interpolation or
not via an obscure boolean called `forceString`.

Instead, we rename it to `isInterpolation`.

This is a breaking change for the JSON AST representation.

Change-Id: I9f89337449b56f6e99a961e21169761f554c9896
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-05-20 19:14:49 +02:00
eldritch horrors dad28eca75 cli: disallow ca derivations
we no longer have any experimental features depending on ca derivations,
so we can start removing them. since ca derivations are very invasive we
will need a while to remove all of the explicitly experimental code, and
even then we will not have removed *all* code related to ca derivations.
especially in the derivation goals there is a lot of code that is not as
easy to disentangle from experimental features as some would have hoped.

Change-Id: Ia456aadc6164613ded343f571318494d9310a549
2025-05-20 13:27:04 +02:00
Linus HeckemannandLix Systems Gerrit a7634f87aa Merge "libstore/local-derivation-goal: better debuggability for FOD mismatches" into main 2025-05-16 15:18:11 +00:00
Raito BezariusandLix Systems Gerrit 901940df00 Merge "libstore/profiles: do not dereference current generation if it doesn't exist" into main 2025-05-12 22:47:48 +00:00
eldritch horrors ccdd916226 libstore: de-ref {Derived,Built}Path::Built drvPath
they're no longer recursive, so this is perfectly fine.

Change-Id: If565a557f2c2074e2a96a7f89c51ff1c51146b36
2025-05-12 13:37:54 +02:00
eldritch horrors 540071dd77 cli: disallow dynamic derivations
with impure derivations gone we move on to dynamic derivations. this too
is not done in a single commit because dynamic derivations are invasive,
modifying semantics of all references to derivation output paths and all
derivation dependency calculations. removing dynamic derivations cleanly
is made significantly harder by the multiple did-you-mean-sum types, aka
"wrappers for std::variant", holding all derivation outpath information.

Change-Id: Ice7a7700c7b54c6a6061d4beb322b4175923d27a
2025-05-12 13:37:54 +02:00
Linus Heckemann d19593d00b libstore/local-derivation-goal: better debuggability for FOD mismatches
The expected and the obtained path are now printed as part of the
error message, making comparing them easier when they're both at hand.

The extra rethrow for the hash-mismatch exception in the bmCheck case
has been removed, allowing the path to be registered as in the
non-check case. This makes having both paths at hand a lot more likely!

The determinism check logic was incorrect for content-addressed paths,
since it only ever tried to compare the path produced, even if this
was not the path expected (in the case of fixed-output derivations) or
the path previously produced (in the case of non-fixed CA
derivations). This made little sense, because that would always be the
same path if it exists! The determinism check is therefore now
bypassed for CA paths. Having a correct determinism check for
non-fixed CA derivations and running the diff hook for fixed-output
derivations would be nice, but feels out of scope and bypassing the
inapplicable logic isn't a regression from the previous behaviour.

Change-Id: I5fc14fb477c8c7d2f5bdedad5591af916f72b128
2025-05-11 21:22:52 +02:00
eldritch horrors 6599be1a9f cli: disallow impure derivations
we don't remove the entire feature in one go to make review easier.
impure derivations are rather unintrusive on their own, at least if
we compare them to dynamic or ca derivations in general, so we will
be done with this soon. as it stands impure derivations cannot work
without ca derivations, and those we *really* want to leave behind.

Change-Id: I4f01d8d758b2c85dcd6c3078304b5ee1b52f65b0
2025-05-11 17:26:55 +02:00
Raito BezariusandQyriad d19a9e3039 libstore/profiles: do not dereference current generation if it doesn't exist
If the profile inode is invalid, e.g. invalid symlink, the current
generation cannot be discovered.

Nonetheless, this should not be a reason for an assert failure, instead
of crashing, just raise an error.

Fixes fj#801.

Change-Id: I63937672173bc3bf37196de98307800adc5757e1
Signed-off-by: Raito Bezarius <raito@lix.systems>
Co-authored-by: Qyriad <qyriad@qyriad.me>
2025-05-11 01:07:18 +02:00
Tom HubrechtandLix Systems Gerrit 885dde9c3d Merge "libstore: Show illegal path references in fixed-outputs derivations" into main 2025-05-08 20:19:15 +00:00
a8da0df671 libstore: Show illegal path references in fixed-outputs derivations
This allows debugging the errors by pinpointing the offenders, instead
of trying to blindly guess what could have possibly gone wrong.

The wording has been improved to better explain the failure.

Closes #530

Co-authored-by: Ben Millwood <thebenmachine+git@gmail.com>
Change-Id: I84ba5d2d81e5d1867f53bd3bc80e615cab9fe274
2025-05-08 20:23:12 +02:00
eldritch horrors 2059ccaad3 libexpr: remove positions from primops
they're always noPos and have caused quite some confusion.

fixes #781

Change-Id: I15d4210b1aab6ef5d1caabe2e10b45cf98cc4d47
2025-05-02 10:57:15 +00:00
eldritch horrors bdebc3797b cmd/libexpr: don't determinePos in value-to-json
it's not useful. nix-instantiate will always point to the wrong location
because "at the toplevel" is not a position we can point to. all nesting
conversions will point to strange places: the *original* definition of a
lambda value (or partial applications thereof), or nowhere at all (since
only lambda values can have their positions determined in the formatter)

Change-Id: I56bd9e1cf3cae11b4198f148a141ca758e4fe8ac
2025-04-27 17:38:57 +00:00
eldritch horrors eb0280f715 libexpr: add set-to-string error frame
this is much more useful than a frame pointing to the set but claiming
it as a function being called. if the function is actually at fault we
will now point to its attribute, although the position may be slightly
wrong if __toString was defined from set updates or builtin functions.

Change-Id: Ib3eb237a276d94426d9c6fc0e26eea72382d34a2
2025-04-27 11:01:22 +00:00
Raito Bezarius 5ca168f6f2 chore: drop experimental feature recursive-nix
This was discussed as part of https://git.lix.systems/lix-project/lix/issues/767
with a one month long comment period.

This commit removes the recursive-nix feature from Lix. It has seen
limited usage and is difficult to stabilize, especially on non-Linux
systems where its behavior is underspecified.

Maintaining this feature complicates core work on the store, as we must
account for the potential presence of the daemon in the sandbox, adding
unnecessary complexity. Additionally, its inclusion in the
platform-independent local store creates risks for non-Linux platforms.

For more details on this removal, refer to the release note entry or the
issue entry.

Change-Id: I9137202f563c0a317f9c5da79cd9fd07d801427a
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-04-25 17:09:01 +02:00
2b22dae1ba lixexpr: Declutter error traces
Now that we can correctly point to all expressions, we can remove
redundant intermediate traces to reduce clutter.

Co-authored-by: eldritch horrors <pennae@lix.systems>
Co-authored-by: Raito Bezarius <raito@lix.systems>
Change-Id: I3e9d7c1c7a6599a8e68302448bbb961d051002b7
2025-04-24 20:44:57 +02:00
piegames abb8ad29c0 repl: Always use parser, allow trailing ; in assignments
We now properly shell out to the parser instead of hacking stuff
together with a regex. Stuff we get for free by doing this:

- Optional trailing semicolon
- Declaring nested attribute sets
- String identifiers, and future proofing for eventual grammar
improvements to identifiers
- Dynamic attributes

Change-Id: Ibf1ad815e5e27caf162df05ea5ba5b1b4955d9c9
2025-04-21 20:19:25 +02:00
helle 2238f7a8ab Replace existing external command/custom sub command tests with functional2 tests
This removes the bash dependency issue that the previous test had, the
tests are translated roughly 1 to 1 and do not address any of the TOD
list items.

Closes #800, also superceeds I81817b9fe4a5eeb019f7e04b5a60e091bbde26b2

Change-Id: Ie440cd0a602d4ca4a48edcaf517b7358ec572710
2025-04-19 20:09:44 +02:00
Commentator2.0 fb0ef6ca6b Made parser warnings use Error info
no other warn calls were found, which have position information

Change-Id: If9773b86a756e5b2e4096d3f6b3a5114bfed5bee
2025-04-08 10:21:53 +02:00
eldritch horrors 8d8bb60796 treewide: lint against non-lix exceptions
we generally do not want to catch or throw these. catching them to print
and discard is fine, tests are largely exempt, and cases in which we can
be certain where the exception came from are also fine to *catch*. we'll
try to never *throw* (or rethrow) these if possible though because doing
so will make it impossible to construct async traces for the exceptions.

Change-Id: I3b71c32ecd16afc2246c946472f5629a1fa31f2c
2025-04-06 12:14:38 +02:00
Maximilian BoschandLix Systems Gerrit d169c092fc Merge "fetchers: don't consider a path locked if a rev is specified" into main 2025-03-29 18:51:45 +00:00