Commit Graph
2353 Commits
Author SHA1 Message Date
eldritch horrors eb0280f715 libexpr: add set-to-string error frame
this is much more useful than a frame pointing to the set but claiming
it as a function being called. if the function is actually at fault we
will now point to its attribute, although the position may be slightly
wrong if __toString was defined from set updates or builtin functions.

Change-Id: Ib3eb237a276d94426d9c6fc0e26eea72382d34a2
2025-04-27 11:01:22 +00:00
Raito BezariusandLix Systems Gerrit 9cc0cda16f Merge "libstore/ssh: fix the SSH connectivity check with non-POSIXy shells" into main 2025-04-26 11:06:49 +00:00
K900andeldritch horrors 227e4d349f libstore/ssh: fix the SSH connectivity check with non-POSIXy shells
SSH runs the command provided to it in the user's login shell.
That's a feature and is impossible to bypass (trust me, we tried).

The previous implementation of `echo started; nix-store --stdio`
broke with shells that treat `;` differently, e.g. nushell,
which eats the output of everything but the last command in a chain.

In the more general case, this means that a sufficiently weird shell
can do _anything_ it wants to the command we pass via SSH, so
we're forced to rely on as little functionality as possible.

The subset we're hereby settling on is just "running `bash` runs bash".
We then run bash, in non-interactive mode, which gives us a somewhat
consistent environment do to things in.

This whole thing is extremely cursed, but fixing it _correctly_
requires pretty much entirely rewriting how remote stores work,
and I'm not doing that right now.

Fixes #805.

Change-Id: Icac846e8cd821cbca91860ddaa0f657b4317dbf8
Co-authored-by: eldritch horrors <pennae@lix.systems>
2025-04-26 13:46:16 +03:00
Raito Bezarius 5ca168f6f2 chore: drop experimental feature recursive-nix
This was discussed as part of https://git.lix.systems/lix-project/lix/issues/767
with a one month long comment period.

This commit removes the recursive-nix feature from Lix. It has seen
limited usage and is difficult to stabilize, especially on non-Linux
systems where its behavior is underspecified.

Maintaining this feature complicates core work on the store, as we must
account for the potential presence of the daemon in the sandbox, adding
unnecessary complexity. Additionally, its inclusion in the
platform-independent local store creates risks for non-Linux platforms.

For more details on this removal, refer to the release note entry or the
issue entry.

Change-Id: I9137202f563c0a317f9c5da79cd9fd07d801427a
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-04-25 17:09:01 +02:00
Raito BezariusandLix Systems Gerrit a133633ecc Merge "lixexpr: Declutter error traces" into main 2025-04-24 19:07:06 +00:00
2b22dae1ba lixexpr: Declutter error traces
Now that we can correctly point to all expressions, we can remove
redundant intermediate traces to reduce clutter.

Co-authored-by: eldritch horrors <pennae@lix.systems>
Co-authored-by: Raito Bezarius <raito@lix.systems>
Change-Id: I3e9d7c1c7a6599a8e68302448bbb961d051002b7
2025-04-24 20:44:57 +02:00
eldritch horrors feebecd60b treewide: wrap std::regex_error
otherwise lix may crash when e.g. nix search receives invalid regex.
we now also give better error messages for regex errors during eval.

fixes #803

Change-Id: Icc7c578ff488ba520efac5d898572ccf4486e9a8
2025-04-24 13:48:15 +00:00
piegames abb8ad29c0 repl: Always use parser, allow trailing ; in assignments
We now properly shell out to the parser instead of hacking stuff
together with a regex. Stuff we get for free by doing this:

- Optional trailing semicolon
- Declaring nested attribute sets
- String identifiers, and future proofing for eventual grammar
improvements to identifiers
- Dynamic attributes

Change-Id: Ibf1ad815e5e27caf162df05ea5ba5b1b4955d9c9
2025-04-21 20:19:25 +02:00
giliceandLix Systems Gerrit 930ac12346 Merge "libutil: canonPath: error instead of panic on empty path" into main 2025-04-21 15:35:54 +00:00
eldritch horrors f92235e1d2 libstore: stop using ssh connection sharing
our implementation is mostly unused, completely untested, and simply
breaks when users configure connection sharing independently. we can
safely delete this "feature" and inherit user configuration instead.

also have the remote build test check that connection sharing works.

fixes #304, fixes #644

Change-Id: Iea44cc0f8e51a1d231ad186a95c7e310bbfeb303
2025-04-21 13:43:44 +00:00
helle 2238f7a8ab Replace existing external command/custom sub command tests with functional2 tests
This removes the bash dependency issue that the previous test had, the
tests are translated roughly 1 to 1 and do not address any of the TOD
list items.

Closes #800, also superceeds I81817b9fe4a5eeb019f7e04b5a60e091bbde26b2

Change-Id: Ie440cd0a602d4ca4a48edcaf517b7358ec572710
2025-04-19 20:09:44 +02:00
gilice 341e6049a7 libutil: canonPath: error instead of panic on empty path
This could previously crash lix:

Before:
$ nix eval -E '{type="derivation"; drvPath="";}'
nix: lix/libutil/file-system.cc:45: Path nix::canonPath(PathView, bool): Assertion `path != ""' failed.
Aborted (core dumped)

After:
$ nix eval -E '{type="derivation"; drvPath="";}'
error:
       … while evaluating the drvPath of a derivation
         at «string»:1:21:
            1| {type="derivation"; drvPath="";}
             |                     ^

       error: path '' is not in the Nix store

Fixes #536

Change-Id: I406dc9e58047be8f263cf2e4bc3ed5da75a46602
2025-04-12 16:31:28 +02:00
jadeandLix Systems Gerrit 81d4769849 Merge "fix: make test/ headers compliant with styleguide" into main 2025-04-10 16:25:52 +00:00
FireFlyandLix Systems Gerrit 6b3162be9c Merge changes Idd20d965,I27e2c7d8 into main
* changes:
  tests: fix deprecated of googletest macro use
  tests: ignore deprecated uses in rapidcheck
2025-04-10 11:48:09 +00:00
Jade Lovelace c0de512e14 fix: make test/ headers compliant with styleguide
These were not included in that check. Beats me as to why.

Change-Id: Ib44c7f0b10672359026f95daeb088039cf6df4eb
2025-04-09 23:46:41 -07:00
jadeandLix Systems Gerrit 9c2dba4ee2 Merge "Revert "libstore: don't use curl decompression support"" into main 2025-04-09 16:01:37 +00:00
FireFly 478253b16a tests: fix deprecated of googletest macro use
Change-Id: Idd20d9659116c352f05876acd91f5c3d8325f097
2025-04-09 17:15:19 +02:00
FireFly a8c3fbf0cc tests: ignore deprecated uses in rapidcheck
There is a pull request [1] addressing these upstream--it doesn't appear
likely to be merged anytime soon though... this is a no-op til we enable
-Wdeprecated-declarations, but helps in the direction of #744

[1]: https://github.com/emil-e/rapidcheck/pull/325

Change-Id: I27e2c7d81df152de8674696f2a56d5f21c414ce3
2025-04-09 17:04:42 +02:00
eldritch horrorsandLix Systems Gerrit 350bf7e42d Revert "libstore: don't use curl decompression support"
This reverts commit bba678e5c5.

Reason for revert: didn't fix the bug and created new ones (fj#794)

Change-Id: I0450205d3041b6c876737151a4051081c1366f1d
2025-04-09 11:28:50 +00:00
Commentator2.0 fb0ef6ca6b Made parser warnings use Error info
no other warn calls were found, which have position information

Change-Id: If9773b86a756e5b2e4096d3f6b3a5114bfed5bee
2025-04-08 10:21:53 +02:00
eldritch horrors 8d8bb60796 treewide: lint against non-lix exceptions
we generally do not want to catch or throw these. catching them to print
and discard is fine, tests are largely exempt, and cases in which we can
be certain where the exception came from are also fine to *catch*. we'll
try to never *throw* (or rethrow) these if possible though because doing
so will make it impossible to construct async traces for the exceptions.

Change-Id: I3b71c32ecd16afc2246c946472f5629a1fa31f2c
2025-04-06 12:14:38 +02:00
eldritch horrors 63d550938b treewide: derive all lix exceptions from BaseError
even the non-errors. we should probably insert a BaseException here.

Change-Id: I1b1af8ba0bf49251fe9d1a24c6559db3ef4a59d2
2025-04-06 11:52:32 +02:00
jadeandLix Systems Gerrit 7bf3d0fa2a Merge changes Iaa962f04,I19480ab0 into main
* changes:
  MonitorFdHup: introduce a test
  tests: fix the fmt.cc test
2025-04-05 17:12:48 +00:00
FireFly c292fbc46d tests: actually test printing of paths in unit test
This seems to be a copypasta mistake that slipped through when writing the
initial tests.  Changes the ValuePrintingTests::tPath to actually test a path
value instead of a string.

Change-Id: I20fe72635a0b1a6a0d72f674a4619da0de9f3d44
2025-04-05 17:11:17 +02:00
Jade Lovelace 7aec313597 MonitorFdHup: introduce a test
Well, I was trying to figure out
https://git.lix.systems/lix-project/lix/issues/729 in which this feature
is clearly just broken on macOS, but frustratingly, it seems that it
*does* work, except for the daemon. um........ sure.

Change-Id: Iaa962f045c16fdfa82854151c90a03e5cf0eea47
2025-04-04 16:59:23 -07:00
Jade Lovelace 6d8e4337e2 tests: fix the fmt.cc test
Yikes!! I wonder if we have any other ones of these that just .. didn't
get added to a meson file?

Change-Id: I19480ab03cdbecf608e523d5b6c3980233f4f445
2025-04-03 18:40:06 -07:00
Maximilian BoschandLix Systems Gerrit d169c092fc Merge "fetchers: don't consider a path locked if a rev is specified" into main 2025-03-29 18:51:45 +00:00
eldritch horrors 0d47773d76 treewide: handle JSON parse errors
or more accurately, wrap them in a nix::Error subclass so we can display
them properly without crashing, and add some error context if available.

fixes #642
fixes #753
fixes #759
fixes #769

Change-Id: I1aad0c0501fea83f9de3a1335eaa6adc20721616
2025-03-27 08:56:14 +00:00
eldritch horrors fc58e7b217 libexpr: mark eval error handling NeverAsync
technically it doesn't *have* to be NeverAsync, but not marking it as
such unconditionally requires templating DebugState over asyncness of
its callback (which then requires templating EvalState, which, *NO*.)

Change-Id: I4980d45b541c2e40328beac139b18c6c1ba0957c
2025-03-25 21:26:47 +01:00
eldritch horrors 8a90195cdb libexpr: don't debugThrow from async code
running a debugger is never-async since it holds an EvalState reference.

fixes #761

Change-Id: Ie1b4df6f81bb5614f11ea183eb9d1058fde73121
2025-03-25 21:23:05 +01:00
eldritch horrors c3929c78f3 libutil: disallow enum serialization by default
allow opting in to serialization as integers via a trait type instead,
and add string-list serializers for the feature flag set enumerations.

fixes #738

Change-Id: I2746eb5ef1f15c01b4e681f9ba1615b6c6e64f44
2025-03-25 10:44:52 +01:00
eldritch horrors 56df5ba164 libutil: remove json-utils.hh
we can merge it into json.hh instead.

Change-Id: Ic40c25fa759bf52bb69eae5d7c0597f260fd94a6
2025-03-23 22:29:58 +00:00
eldritch horrors 45017f7508 libutil: specialize nix::JSON
we want to own this specialization fully so we can change the default
serializer behavior without also forcing downstream users of our code
to use the same behavior. it'll also let us do things we cannot do in
regular nlohmann::json, such as selectively enabling serialization of
enums as integral types, or using `to_json`/`from_json` overloads for
not-default-constructible types instead of serializer specializations

Change-Id: I91a1db362e37d654090f1824b1cd3ce783d32134
2025-03-23 22:16:30 +00:00
eldritch horrors 19d7c8352e treewide: add nix::JSON
this will become a proper specialization of `nlohmann::basic_json` soon.
specialing basic_json will let us get rid of our `adl_serializer` hacks,
and it'll open the door to better enum serializing behavior without also
forcing all those who use lix as a library to set certain defines (which
may not even be possible depending on how those users use json already).

Change-Id: I5228d2b9df581a189552c993363207cfbd20f445
2025-03-23 20:42:39 +00:00
eldritch horrors a186bc5021 treewide: add json forwarder headers
this doesn't do much, just wrap a few nlohmann headers in headers of our
own (and delete includes we don't need because they're transitively seen
by other includes). doing this now will make the next change much nicer.

Change-Id: I166933102ea86bb5322ebbf9ba9411f96032a53b
2025-03-23 20:42:39 +00:00
eldritch horrors 09769bbf41 treewide: remove unused nlohmann includes
Change-Id: I261e4bd4c859eb3924202e690331c05887aa26b0
2025-03-23 20:42:39 +00:00
jadeandLix Systems Gerrit 3f45d5e020 Merge "fix!: file type flake inputs are always recursive hashed" into main 2025-03-23 17:45:41 +00:00
Jade Lovelace b22bee91f5 fix!: file type flake inputs are always recursive hashed
Well that is a mess. I don't like breaking derivation compatibility,
but I like non-deterministic evaluation much less.

We can break compat if we own up to it, which this does.

Fixes: https://git.lix.systems/lix-project/lix/issues/750

Change-Id: Ic9e2407393f1d42c2be604f80b4aa11bc872bc23
2025-03-21 13:19:15 -07:00
Maximilian Bosch 62fb98ae55 fetchers: don't consider a path locked if a rev is specified
It's possible to put a path into the store in pure mode by pretending
it's locked like this:

    $ echo 'lalala' > testfile
    $ nix eval --expr '(builtins.fetchTree { path = "/home/ma27/testfile"; rev = "0000000000000000000000000000000000000000"; type = "path"; })'
    { lastModified = 1723656303; lastModifiedDate = "20240814172503"; narHash = "sha256-hOMY06A0ohaaCLwnhpZIMoAqi/8kG2vk30NRiqi0dfc="; outPath = "/nix/store/lhfz259iipmv9ky995rml8018jvriynh-source"; rev = "0000000000000000000000000000000000000000"; shortRev = "0000000"; }
    $ cat /nix/store/lhfz259iipmv9ky995rml8018jvriynh-source
    lalala

There was a fix in CppNix[1], but Puck noted that it's breaking
backwards-compatibility because

> [...] a github fetch with a fully specified rev is no longer considered "locked"
> this is for "purity" reasons, but it breaks any existing flake.nix files

I tried a way smaller correctness fix here:

* Each scheme can denote whether a `rev` is enough to consider itself
  locked.
* If a `rev` is given and the scheme is OK with just a `rev` to be
  locked, the input is marked as locked.

For `path` this is not the case anymore, i.e. it requires a NAR hash to
be locked down.

[1] https://github.com/nixos/nix/commit/071dd2b3a4e6c0b2106f1b6f14ec26e153d97446

Change-Id: Ibbbf4733c82bcfa1c24dfe099a896d8aaecd81cc
2025-03-20 13:17:34 +00:00
jadeandLix Systems Gerrit 3fcf823a86 Merge "fix: do not test non-presence of flakes from a subdirectory of lix src" into main 2025-03-18 20:03:40 +00:00
Jade Lovelace ed7c89790e fix: do not test non-presence of flakes from a subdirectory of lix src
Yeah, we could have seen that one coming, it's kind of on the nose,
really.

Fixes: https://git.lix.systems/lix-project/lix/issues/748
Change-Id: I9e89f73ed89105338343fdecd615e398fc758e89
2025-03-18 12:40:10 -07:00
eldritch horrors 27d5209f4d libutil: fix copyNAR not reading the whole nar when dropped early
if a copyNAR generator was not drained to completion it would not read
the full nar data from its source. this could happen if the copier was
passed to parseAndDump wrapped as a source because copyNAR would yield
nar metadata *before* it had read it, and GeneratorSource will drain a
generator fully *only* if the source is allowed to throw EndOfFile. in
the parseAndDump case this never happened because parseAndDump expects
to be given an unterminated stream, and thus the combination left some
nar metadata in the input Source, breaking the remote store protocols.

fixes #732

Change-Id: Ia59a53375992bfcdb7bc6b37764ca779622bc8f7
2025-03-18 19:32:58 +00:00
jadeandLix Systems Gerrit be4ab653eb Merge "fetchTree: make invisible when not available, sorta document" into main 2025-03-18 19:32:12 +00:00
Jade LovelaceandJade Lovelace fcea7379d3 fetchTree: make invisible when not available, sorta document
Documenting fetchTree is an exercise in frustration because of the sheer
amount of stringly typed everything in it. I do not know which fields
exist without reading the entirety of libfetchers. However, we can write
something slightly perfunctory but at least perhaps somewhat helpful
documentation-wise.

Fixes: https://git.lix.systems/lix-project/lix/issues/609
Change-Id: I991391b53fcd69172dbc7efb9d384e62bc847b91
2025-03-18 11:34:28 -07:00
jadeandLix Systems Gerrit a20a1e9692 Merge "libstore: move case hacking to the FS interaction code" into main 2025-03-18 17:56:53 +00:00
jadeandLix Systems Gerrit 6e6ff58272 Merge "feat!: remove experimental repl-flakes" into main 2025-03-18 16:02:55 +00:00
jadeandLix Systems Gerrit ea4cf48879 Merge "post-build-hook: don't emit config settings that aren't overridden" into main 2025-03-17 23:31:32 +00:00
KFearsandJade Lovelace 2c21882767 feat!: remove experimental repl-flakes
This is a backport of CppNix PR 10299

Fixes #557

Change-Id: I848e8c5af4bde09afba053a5911fbbd54004fbb8
2025-03-17 16:01:53 -07:00
Jade Lovelace 523965697d post-build-hook: don't emit config settings that aren't overridden
This is a redesign from 9b1f3cbc13 where
this was introduced.

I deleted the AbstractConfig::toKeyValue since it was conspicuously and
obviously broken for two years since 450e5ec618.
I asked myself if anyone was using it, given that it only emitted
settings that were aliases (broken!), and found that nobody used it.

The motivation for this change is the same for only emitting overridden
settings to the protocol: the nix inside there may not be able to parse
our defaults, as is the case of CppNix since the consensual
accept-flake-config was added to Lix.

Fixes: https://git.lix.systems/lix-project/lix/issues/739
Change-Id: Ib9874a52137f1f22220c25bcfa2425a4802509c7
2025-03-17 15:25:52 -07:00
Jade Lovelace adf45deeca plugins: support nix_plugin_entry, do some minor reworks
Plugins are already allowed to fail to load, let's be even more tolerant
of them failing to load by ignoring inaccessible ones with a warning.

This change also significantly reworks the docs for plugins.

This is a tiny extremely partial backport of the C API at
https://github.com/NixOS/nix/pull/8699.

Fixes: https://git.lix.systems/lix-project/lix/issues/740
CC: https://git.lix.systems/lix-project/lix/issues/359

Change-Id: If4ee20c3daaf26c8184a415eef3e20ca5b5e7aef
2025-03-16 16:43:54 -07:00