Commit Graph
79 Commits
Author SHA1 Message Date
Emilia BoppandQyriad 0ade82d23a libexpr: add builtins.warn
Added builtins.warn` which takes two arguments: a message that is
displayed as a warning during evaluation which must be a string and a
value that is returned from the expression.

The next commits add new settings to control the behavior of the new
builtin: `debugger-on-warn` allows the user to start the debugger and
`abort-on-warn` aborts evaluation with an error.

Unlike upstream, I chose not to mark evaluation warnings from
`builtins.warn` as distinct from other warnings because that breaks the
commonly expected logging format `level: message`.

Co-authored-by: Qyriad <qyriad@qyriad.me>
Upstream-PR: https://github.com/NixOS/nix/pull/10592
Fixes: https://git.lix.systems/lix-project/lix/issues/579
Change-Id: I8658c88e5c27952b65e8b9f5525a572e0680cc1f
2025-11-20 17:52:29 +01:00
Lunaphied 2c73f3c492 cq: improve derivation printing configuration
Change-Id: I7f16afa921cf2a33f750cb0f5cc53fa36a6a6964
2025-11-19 20:52:38 +01:00
eldritch horrors f29dfb3d3c libutil: add buffer state management to loggers
currently all loggers can always accept messages and never suggest
flushing buffers. in the future this may change, and at that point
we're already fully set up for it. local loggers should never keep
asynchronous (i.e. network-backed) buffers, disk buffers are fine.
networked loggers will require buffers and periodic flushes later.

Change-Id: Ide2114f5bc17f4a1d289c92ed4f9381a1d59dacf
2025-10-13 11:26:49 +00:00
eldritch horrors 80654b84b6 libexpr: de-ptr-ize Value references
thunk values are shareable, and we can represent invalid/uninitialized
values with a special bit pattern that makes no sense otherwise. there
is no need to keep allocating values on the heap, instead we can treat
values like reference-counted smart pointers to heap objects, which in
turn lets us save a lot of allocations and, ultimately, gc heap space.

compared to our baseline (main of 2025-09-27) we save 15%+ memory on a
system rebuild and 17% on nix search. eval time regresses by ~3% for a
system rebuild, while nix search is 7% faster. further optimization is
probably possible (but for now this will just have to be good enough).

Change-Id: Ib6c47acdbe2fac4f76a83c2269f16f30ef66b2e1
2025-10-05 16:23:05 +02:00
eldritch horrors 28c4431c69 libexpr: reformat stuff we'll change soon
mainly to keep the next diff smaller. it'll be large enough as is.

Change-Id: Ib8a34520f03539cbf6aa2f0e66cbed05fe1225eb
2025-10-05 16:22:35 +02:00
eldritch horrors db35aa753b libexpr: de-ptr-ize many Value uses
with thunk state being shareable we no longer need to worry about value
uniqueness, only about value lifetime. this means we can liberally drop
indirections and allocations, passing references instead of pointers or
using stack memory instead of gc-managed memory for some intermediates.

Change-Id: I2d48a6fd57a376d544bd9bd2d05e5420611986d1
2025-10-05 16:22:35 +02:00
eldritch horrors 0e9e9fd917 libexpr: heap-allocate "large" integers
Change-Id: Ic391f2f1bf87f044d7a688196ba9e0ad766d65aa
2025-10-04 16:31:45 +02:00
eldritch horrors 79586575c5 libexpr: heap-allocate app nodes
Change-Id: I9a39dcf0be7589cedf494757e21665c5d50e446b
2025-10-04 16:31:45 +02:00
eldritch horrors 94a866d44f libexpr: heap-allocate thunk control state
Change-Id: I20d8ab1d6f683c0a2f3b77edf9bdad147d62c8fa
2025-10-04 16:31:45 +02:00
eldritch horrors d84f3003cf libexpr: move lambdas to auxiliary storage
Change-Id: Ibe4885f17c0ba1634ed6dbca0a45f8bd4619d69b
2025-10-04 16:25:05 +02:00
eldritch horrors 1b5f4eb5eb libexpr: move primops to auxiliary storage
same as for null: we have few of them, they're statically allocated,
and they're not the largest contributor to the value population. not
storing them in Value itself frees up resources we *will* use later.

Change-Id: I521f9f243f48f56a78f7bffdf1dc1f0bc40a5c2d
2025-10-04 16:25:04 +02:00
eldritch horrors 1c4e77387a libexpr: remove tPrimOpApp
using the same nodes as tApp is possible, and thanks to multi-arg app
nodes it can even be a bit faster than the linked lists used to date.

Change-Id: Idccb7c0b54c808e62da85d1c42ee09e6e92c4f7b
2025-09-29 17:56:05 +02:00
eldritch horrors 0b5b14ddc7 libexpr: add multi-arg app nodes
these behave like the old chains of app nodes, but they can store more
than one argument per node. for tApp values themselves this is not all
that useful, but if we could share tApp and tPrimOpApp backing storage
we could avoid creating and traversing the linked lists of values that
are currently needed to represent partially applied builtin functions.

Change-Id: I5a2a02d9733e1e0be5443459e2998d62fd3b9a5b
2025-09-29 17:56:05 +02:00
eldritch horrors d23e3a09a4 libexpr: alloc list storage as a (length, vla) type
memory overhead is minimal and performance impact not measurable. once
we've done something like this for all value types that don't fit in a
single machine word we can cut a word from Value, offsetting the cost.

Change-Id: I9813bacd7e851957ad3426aed8f74033179a4212
2025-09-29 17:55:54 +02:00
eldritch horrors 01f1852360 libexpr: don't inline small lists into values
this has no measurable performance impact thanks to the new caches.

Change-Id: Ib403a9a567161675f78e8c5d314d6340183d181d
2025-09-29 15:22:41 +02:00
eldritch horrors c7cc7d6c31 libexpr: "hide" Value union members
on its own this is not very useful, but having accessors for every value
kind is a prerequisite for doing smart things with Value than the union.
the net effect for now is only to add a few parentheses across the tree.

Change-Id: I88688ac09eb08495dad1eb221034ca540f094950
2025-09-29 15:22:41 +02:00
Raito BezariusandSergei Zimmerman 057b725ae3 libexpr: use const references or pointers to attribute sets
Prior to this change, references or pointers could be mutated. In
practice, we do not require this capability in the codebase except in
zipAttrsWith.

This cleans up all easy sites in preparation to have a smarter
representation of attribute sets albeit one that requires constant
references.

Change-Id: I2be20cce040a9228bde9e5f7b42c0499fba9550b
Signed-off-by: Raito Bezarius <raito@lix.systems>
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2025-09-20 20:31:19 +02:00
Jade Lovelace 61955d0a40 libexpr: hyperlink attr names to their definition locations
Concept: what if you could, in your fancy terminal, in the year of our
lord 2025, just click on the attrs you're looking at to go to where
they're defined. Currently we only expose this info as
builtins.unsafeGetAttrPos, which is inconvenient as it's not
discoverable to users.

By putting it in this more visible yet invisible spot, it's more likely
to be more useful to more people.

In the current state, this is not the most useful ever due to stuff like
https://github.com/neovim/neovim/discussions/35097. However, it can be
expanded by perhaps adding something like the url format setting ripgrep
has.

Change-Id: I3947f97d5c2056d59099af468d7b855486438227
2025-08-20 20:55:54 +00:00
eldritch horrors 8208c6ebb7 libexpr: don't read Value::string.s directly
Change-Id: I0f224459fcbff8bc53c3668bc6ea52881c453fd0
2025-08-17 12:11:19 +00:00
Philipp OtterbeinandRaito Bezarius fc3261c340 libexpr/primops: fix overflows and silent type casts
Avoids incorrect behavior with large integers in `elemAt`, `substring`,
`genList`, etc, which results into crashing the Lix interpreter.

At the same time, unit tests were added for these edge cases with 2^32
as an argument of these primops.

Port of https://github.com/NixOS/nix/pull/13309.
Prior art in https://github.com/NixOS/nix/pull/7222 (forgotten by the original project…).

Change-Id: I1c43ed64f26bcb60e51869e11a74e5de2b7db53a
Co-authored-by: Raito Bezarius <raito@lix.systems>
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-07-07 23:32:18 +00:00
eldritch horrors 34317c0081 libexpr: simplify EvalState::mkOutputString{,Raw}
we no longer need placeholders to represent all derivation output paths
as string context, and thus will not need experimental features either.

Change-Id: I9e86ce86810e976cf8397b2c2f473af11390874c
2025-05-20 17:43:46 +00:00
eldritch horrors dad28eca75 cli: disallow ca derivations
we no longer have any experimental features depending on ca derivations,
so we can start removing them. since ca derivations are very invasive we
will need a while to remove all of the explicitly experimental code, and
even then we will not have removed *all* code related to ca derivations.
especially in the derivation goals there is a lot of code that is not as
easy to disentangle from experimental features as some would have hoped.

Change-Id: Ia456aadc6164613ded343f571318494d9310a549
2025-05-20 13:27:04 +02:00
eldritch horrors ccdd916226 libstore: de-ref {Derived,Built}Path::Built drvPath
they're no longer recursive, so this is perfectly fine.

Change-Id: If565a557f2c2074e2a96a7f89c51ff1c51146b36
2025-05-12 13:37:54 +02:00
eldritch horrors 5f723e96e6 libstore: flatten {,Single}{Built,Derived}Path
only dynamic derivations could produce a non-opaque drvPath. since
dynamic derivations are no longer supported we can have drvPath be
opaque at all times, simplifying downstream code significantly and
making quite a few methods unnecessary. discardOutputPath was only
called on drvPath members anyway and thus reduces to a copy, other
operations at the very least are no longer recursive. some vestige
of dynamic derivations remains in DerivedPathMap though (for now).

Change-Id: Ifb4ad53a3c67800be5a62540068c8279d4ae0046
2025-05-12 13:37:54 +02:00
eldritch horrors 8a539424c8 libexpr: drop support for dyn-drv string context
string context doesn't need any tests because it's never persisted or
shown to the user. getting rid of recursive string context means that
the context string parsers can be a lot simpler from here on forward.

Change-Id: I58443679ad76c0f28ea5f4eb8bfb3874f270e764
2025-05-12 13:37:54 +02:00
eldritch horrors 540071dd77 cli: disallow dynamic derivations
with impure derivations gone we move on to dynamic derivations. this too
is not done in a single commit because dynamic derivations are invasive,
modifying semantics of all references to derivation output paths and all
derivation dependency calculations. removing dynamic derivations cleanly
is made significantly harder by the multiple did-you-mean-sum types, aka
"wrappers for std::variant", holding all derivation outpath information.

Change-Id: Ice7a7700c7b54c6a6061d4beb322b4175923d27a
2025-05-12 13:37:54 +02:00
eldritch horrors 6599be1a9f cli: disallow impure derivations
we don't remove the entire feature in one go to make review easier.
impure derivations are rather unintrusive on their own, at least if
we compare them to dynamic or ca derivations in general, so we will
be done with this soon. as it stands impure derivations cannot work
without ca derivations, and those we *really* want to leave behind.

Change-Id: I4f01d8d758b2c85dcd6c3078304b5ee1b52f65b0
2025-05-11 17:26:55 +02:00
Raito BezariusandLix Systems Gerrit a133633ecc Merge "lixexpr: Declutter error traces" into main 2025-04-24 19:07:06 +00:00
2b22dae1ba lixexpr: Declutter error traces
Now that we can correctly point to all expressions, we can remove
redundant intermediate traces to reduce clutter.

Co-authored-by: eldritch horrors <pennae@lix.systems>
Co-authored-by: Raito Bezarius <raito@lix.systems>
Change-Id: I3e9d7c1c7a6599a8e68302448bbb961d051002b7
2025-04-24 20:44:57 +02:00
eldritch horrors feebecd60b treewide: wrap std::regex_error
otherwise lix may crash when e.g. nix search receives invalid regex.
we now also give better error messages for regex errors during eval.

fixes #803

Change-Id: Icc7c578ff488ba520efac5d898572ccf4486e9a8
2025-04-24 13:48:15 +00:00
FireFly a8c3fbf0cc tests: ignore deprecated uses in rapidcheck
There is a pull request [1] addressing these upstream--it doesn't appear
likely to be merged anytime soon though... this is a no-op til we enable
-Wdeprecated-declarations, but helps in the direction of #744

[1]: https://github.com/emil-e/rapidcheck/pull/325

Change-Id: I27e2c7d81df152de8674696f2a56d5f21c414ce3
2025-04-09 17:04:42 +02:00
FireFly c292fbc46d tests: actually test printing of paths in unit test
This seems to be a copypasta mistake that slipped through when writing the
initial tests.  Changes the ValuePrintingTests::tPath to actually test a path
value instead of a string.

Change-Id: I20fe72635a0b1a6a0d72f674a4619da0de9f3d44
2025-04-05 17:11:17 +02:00
eldritch horrors fc58e7b217 libexpr: mark eval error handling NeverAsync
technically it doesn't *have* to be NeverAsync, but not marking it as
such unconditionally requires templating DebugState over asyncness of
its callback (which then requires templating EvalState, which, *NO*.)

Change-Id: I4980d45b541c2e40328beac139b18c6c1ba0957c
2025-03-25 21:26:47 +01:00
eldritch horrors 09769bbf41 treewide: remove unused nlohmann includes
Change-Id: I261e4bd4c859eb3924202e690331c05887aa26b0
2025-03-23 20:42:39 +00:00
piegames 2e5239d2e5 libexpr: Refactor formals handling
This is kind of a painful change, because a lot of code grew around the
bad abstraction, but the goal here is to abstract formals in a way that
allows adding other means of pattern matching / argument destructuring
in the future.

Change-Id: If4e681a4be3d1f42ceea81a8e07297f0d08acc80
2025-02-24 13:48:29 +01:00
piegames f324574265 libexpr: Track position information in all expressions
Change-Id: Ied79381a917a715cdd8816d6f9df527289217dfb
2025-01-27 20:49:25 +01:00
piegames d0361e9467 libexpr: Unify literal expressions as ExprLiteral
Change-Id: Ifc0f1e27b46803fa2175801ed7d3d108896c8259
2025-01-27 20:49:25 +01:00
piegames 539c340c66 libexpr: Undeprecate overriding __findFile
After gathering more community feedback, more non-trivial use cases for
overriding `__findFile` emerged. Unlike the use case of Tvix mentioned
in #599, these can't easily be worked around by overriding `nixPath`
instead.

This is the second fixup/partial revert for
81d5f0a7d9.
See also #599.

Change-Id: I7ca75e1a2b196c0da341969c61f4c168b5f657f9
2025-01-04 19:07:35 +01:00
jadeandGerrit Code Review cb44fe7c15 Merge changes Ic890f781,I77b2de10,I3202aba0,I6ddc4296,I18776984, ... into main
* changes:
  doc: fix broken table in operators list
  terminal code eaters: implement OSC
  libexpr: significantly improve error messages for bad attr paths
  tests/functional2: add terminal code eater
  tests/functional2: fix occasional pytest haunting
  attr path parser: fix bug in not rejecting empty attr paths, add unparser
2024-12-11 02:28:11 +00:00
Jade Lovelace 21ad02c1d0 attr path parser: fix bug in not rejecting empty attr paths, add unparser
The following behaviour was previously present and has been fixed:

lix/lix2 » nix eval --expr '{x."" = 2;}' 'x.""'
{ "" = 2; }
lix/lix2 » nix eval --expr '{x."".y = 2;}' 'x."".y'
error: empty attribute name in selection path 'x."".y'

Change-Id: Iad21988f1191c33a3661c72a6b7f01a8b8b3e6eb
2024-12-10 13:32:28 -08:00
piegames 7c76053d93 libexpr: Undeprecate overriding __nixPath
It being overridable was an intended feature with good use cases, and
should not have been removed. However, this feature is generally in a
bad state and needs revisiting in the future.

Fixup for 81d5f0a7d9
Fixes #599

Change-Id: I2d93e012caa65aa795bce3a71d8e56d7052ef9df
2024-12-10 19:02:41 +01:00
piegames cdda2454f6 libexpr: Assert: Don't print assertion in error message
The `show` functionality needs to be removed because it is deeply
flawed, and given that we already print position information in the
error message (which probably wasn't always the case in the past) the
assertion printing is redundant anyways.

Change-Id: I1f5e05ab73aaa0ec92994c2211463260fd374898
2024-12-08 20:51:44 +01:00
eldritch horrors 81559ea8ad treewide: add evaluator aliases for eval states
this is not necessary in any way, but it will make the following changes
smaller and easier to review. the aliases could also be added piecemeal,
but doing it here lets us lean heavily on our compilers for correctness.

(teacher notes: here the author foreshadows the shape of things to come.
not all names change, and only the names unchanged are those which will,
over time, become ever more unrecognizable. note especially nix/main.cc,
where `state` is not only cloned, but itself changes pointerness. it can
be seen as a nod to the trans community, but more realistically it is no
more than foreshadowing the future where `state` is only seen by proxy.)

Change-Id: I7732025e58df089b7f8e564fc63960cd91729d09
2024-12-03 20:38:41 +01:00
eldritch horrors 32f7a93f71 libexpr: extract eval path handling into new type
Change-Id: Icf38113076a4dd0f8515a27b501f405033aec73b
2024-12-03 20:38:41 +01:00
eldritch horrors a65e9e5828 libexpr: extract eval error creation into new type
this will let us pass the capability to create debuggable eval errors
without having to pass an entire EvalState. we could pass symbols and
debug states around just as easily, but if we add new capabilities to
our debugger we might have to change many more places than with this.

Change-Id: I2f8893012e5d98a986ef1fc888234c2dd8d5e096
2024-12-03 20:38:41 +01:00
eldritch horrors 1ed0814859 libexpr: use eval() for builtin-using value printer tests
if we're relying on the behavior of builtins we can also use the parser.

Change-Id: I682722b2a22fbb6e748da2cab1bfeb76788bfb48
2024-11-29 17:45:04 +01:00
piegamesandGerrit Code Review 9fb5315d06 Merge "libexpr: Deprecate overriding __sub and the like" into main 2024-11-29 07:56:17 +00:00
piegames 81d5f0a7d9 libexpr: Deprecate overriding __sub and the like
It was never intended to be a feature to be used, and moreover it is
inconsistent: One cannot override `+`, and overriding `__lessThan` won't
affect the builtins which do comparisons.

Change-Id: Iaba54a05aa4c2eb37cdb3dc0d731fcee5a86deba
2024-11-28 18:15:52 +01:00
eldritch horrors f5754dc90a libexpr: move eval memory allocation to own struct
Change-Id: I9d472c9606fe66fdc1cb7cb9dcf6d1b6b46c6686
2024-11-28 15:12:22 +00:00
eldritch horrors cb8262e11c libexpr: remove EvalState::rootPath
this belongs to lazy trees, which we neither have nor intend to have.
we will keep SourcePath as that may come in handy at some later date.

Change-Id: I44b8f1dd6c435d7486c393fabdcd272766b2b56b
2024-11-28 15:12:22 +00:00