Commit Graph
36 Commits
Author SHA1 Message Date
eldritch horrors 56d6ac3114 *: wrap nlohmann exceptions at the source
nlohmann has customization points for exception throws. we can use these
instead and wrap json exceptions at the source instead of playing a game
of whack-a-mole with json errors all over the tree. since nlohmann needs
macros set to achieve this we can no longer precompile its headers *and*
must forbid including it anywhere without the proper defines or ordering
of include directives will break lix with ODR violations, if we see them
at all before we get another json-related bug report. a new lint will be
in charge of ensuring this doesn't happen. we also re-allow direct calls
to nlohmann json parsing since error handling is no longer a problem. we
will keep the wrapper for more convenient error context handling though.

fixes #1092

Change-Id: I54ecc14f5bec5e2177729b41c3703216e76cc6a3
2026-01-04 16:25:06 +00:00
eldritch horrors 2ae74802c4 libexpr: AST-dump non-utf8 strings as byte arrays
our strings need not be utf8. json requires utf. -sigh-

fixes #1052

Change-Id: I50ecd9882252370bb81845b099b11a7190475d48
2025-12-01 21:11:25 +01:00
Tom Hubrecht 395fe68127 tests/clang-tidy: Run on all supported platforms
Fixes #1013

Change-Id: I7388e4eb4728c34f2e21756438020e28b4f1cfca
2025-11-24 07:22:15 +00:00
eldritch horrors b19bbdfee1 libexpr: make thunk state shareable
this is a strong prerequisite for making values themselves copyable
without duplicating evaluation side effects. with this we can treat
`Value` the way we treated `Value *` to date and drop indirections.

Change-Id: I08f30d12697614a3ae7149615f6f1da83b13f52b
2025-10-05 14:56:56 +02:00
eldritch horrors 94a866d44f libexpr: heap-allocate thunk control state
Change-Id: I20d8ab1d6f683c0a2f3b77edf9bdad147d62c8fa
2025-10-04 16:31:45 +02:00
eldritch horrors c7cc7d6c31 libexpr: "hide" Value union members
on its own this is not very useful, but having accessors for every value
kind is a prerequisite for doing smart things with Value than the union.
the net effect for now is only to add a few parentheses across the tree.

Change-Id: I88688ac09eb08495dad1eb221034ca540f094950
2025-09-29 15:22:41 +02:00
eldritch horrors 3de996f521 libexpr: stringviewify some more APIs
notably this also includes the symbol table because it stores real
strings that are referenced by eval values, and an upcoming change
will make it impossible to share those strings with value strings.

Change-Id: I20a3644db8aa0850efe29630e0b73d424cb2aa56
2025-08-17 14:55:13 +02:00
eldritch horrors 8208c6ebb7 libexpr: don't read Value::string.s directly
Change-Id: I0f224459fcbff8bc53c3668bc6ea52881c453fd0
2025-08-17 12:11:19 +00:00
Raito Bezarius 5e98a2159b libexpr/symbol-table: introduce InternedSymbol
The backing storage for symbols becomes a class storing a Value and a
string.

The Value is itself a string which contents points to the owned string.

Recovering a `SymbolStr` is still possible.

Change-Id: I171151abc3c0a513f2150c4b54edd61dea256cce
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 13:58:08 +02:00
Raito BezariusandLix Systems Gerrit 4f433a6186 Merge changes I9f893374,Ief7a4756 into main
* changes:
  libexpr: rename `forceString` to `isInterpolation`
  libexpr: refactor string coercion modes
2025-05-20 20:45:21 +00:00
Raito Bezarius 3e4bffcc24 libexpr: rename forceString to isInterpolation
`ExprConcatStrings` tracks whether the expression is an interpolation or
not via an obscure boolean called `forceString`.

Instead, we rename it to `isInterpolation`.

This is a breaking change for the JSON AST representation.

Change-Id: I9f89337449b56f6e99a961e21169761f554c9896
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-05-20 19:14:49 +02:00
piegames 0fbbb1e49b libexpr: Switch StaticEnv to LinearMap
Change-Id: If98bfafce9fa5235fe962274c03c619fe965dd60
2025-05-19 16:21:10 +02:00
eldritch horrors 6bd6a2416b libexpr: turn debug frames into exprs
this way we don't have to even check whether we need a debug frame when
the debugger isn't enabled. not doing this gives us an eval performance
improvement of roughly 7% on nixos system eval and 2% for `nix search`.

Change-Id: I1cdad3de61f865ea54d6e09d63a281688e828768
2025-05-01 14:28:05 +00:00
eldritch horrors dd2c54d9ad libexpr: generate debug info from a new visitor
Change-Id: I757a9e49dee99b27b0f153ca520c71ee3ecd347f
2025-05-01 14:28:05 +00:00
eldritch horrors 8145c2f5fe libexpr: implement bindVars with a visitor
we can now move debug info generation out of bindVars itself.

Change-Id: I54c88e14d030d2a19f16b57099990f2c3b4334aa
2025-05-01 14:28:05 +00:00
eldritch horrors a91efd63aa libexpr: add an expression visitor
we'll use this to implement bindVars better.

Change-Id: I620d9d1972acdeeee6e345c3128813e53d2074c6
2025-05-01 14:28:05 +00:00
eldritch horrors 8b619b134d libexpr: store inherit source exprs in a list, not vec
it's a surprise tool that will help us later!

Change-Id: Ieac785b00b3d14e5937c03fb2c97c918f4646e56
2025-05-01 14:28:05 +00:00
eldritch horrors 558d921dac libexpr: don't dynamic_cast Exprs after parsing is done
we'll want to wrap some exprs for debug purposes, and dynamic casts
cannot look through such wrappers. dedicated casting functions can.

Change-Id: I1fba0ec52d281a1b8de85a62e4948bfae536bcfc
2025-05-01 14:28:05 +00:00
eldritch horrors 2b4ef8dc11 libexpr: deduplicate set and let recursive env calculation
Change-Id: Ia7f066e1c02f99f071d10e00020869fac9a13db7
2025-05-01 14:28:05 +00:00
skyrelia ba1c9d52ec Clean up includes
This cleans up includes that clangd reports as unused, usually by deleting the offending include. The process was to delete an include and see if it still builds. If not, try to find a more specific include(s) that works, that was previously transitively included. If the original include seems intended to re-export said transitive include, mark the transitive include as `// IWYU pragma: export`. Otherwise, replace the original include with the transitive include(s). If none of the above applies, because the original file depends on code directly in the include somehow, or the direct include is an external dependency that cannot be modified, restore the original include and mark it as `// IWYU pragma: keep`.

Change-Id: I5ce3d34dad76b0cad0a6a7990fea13add393aad3
2025-04-08 12:13:42 +00:00
eldritch horrors 0ff8c2b06f libexpr: make undefined var errors non-debuggable in the parser
this only affects undefined names outside of with scopes. sending those
errors to the debugger when enabled seems rather less useful; unlike in
the case of with scopes there's nothing meaningful to inspect. avoiding
the debugger also lets us avoid marking the entire parser `NeverAsync`.

cf #761

Change-Id: I3599b826ff5b101acf9a3ba70dcdf9e8e02067f0
2025-03-25 20:14:45 +01:00
eldritch horrors 19d7c8352e treewide: add nix::JSON
this will become a proper specialization of `nlohmann::basic_json` soon.
specialing basic_json will let us get rid of our `adl_serializer` hacks,
and it'll open the door to better enum serializing behavior without also
forcing all those who use lix as a library to set certain defines (which
may not even be possible depending on how those users use json already).

Change-Id: I5228d2b9df581a189552c993363207cfbd20f445
2025-03-23 20:42:39 +00:00
piegames 2e5239d2e5 libexpr: Refactor formals handling
This is kind of a painful change, because a lot of code grew around the
bad abstraction, but the goal here is to abstract formals in a way that
allows adding other means of pattern matching / argument destructuring
in the future.

Change-Id: If4e681a4be3d1f42ceea81a8e07297f0d08acc80
2025-02-24 13:48:29 +01:00
piegames d0361e9467 libexpr: Unify literal expressions as ExprLiteral
Change-Id: Ifc0f1e27b46803fa2175801ed7d3d108896c8259
2025-01-27 20:49:25 +01:00
piegames cf57b5c14c libexpr: Remove Expr::show, add JSON expression serialization
The code for serialization Expr nodes back into (pseudo-)Nix has been
removed for being subtly error-prone and tedious to maintain. Instead,
`nix-instantiate --parse` now prints a JSON representation of the AST.

Usage patterns of the --parse flag I've found in the wild:

1. Check if a file is well-formed, i.e. discard output and test exit code
2. Get parser errors from a file, i.e. discard stdout and use stderr
3. Nixfmt uses --parse to test equivalence pre/post format, and that property is (should be?) preserved

None of these should break with the current change

Closes #487

Change-Id: Icdbaad17790f2ad8765fa08e02e6597ee4c7a909
2025-01-21 11:35:51 +01:00
piegames 172515bd8b libexpr: Track position information in attrpaths
This is a pretty small change because the parser already has all
necessary information (needed for parse errors), now we want to keep it
to also provide better eval errors.

Change-Id: Ifc7a9516b9b0c8d9698f1899a6912ae91f6696ab
2024-12-08 20:51:45 +01:00
piegamesandeldritch horrors 9611018c27 libexpr: Split ExprAttrs and ExprLet
ExprLet was previously inheriting from ExprAttrs for the data, while
ignoring all
set-specific operations on it. The set specific code has now been split
off so that
let doesn't inherit it anymore:

- ExprAttrs (not an Expr), containing the attributes and the related
logic
- ExprLet : Expr, ExprAttrs
- ExprSet : Expr, ExprAttrs

Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I63f2fbcd1e790b3cffb56eec1e7565ee3cdbf964
2024-12-08 12:54:38 +01:00
eldritch horrors 61eed2c97c libexpr: fully separate evaluator and running evaluation
this finally gives us a witness type we can use to prove that a certain
call graph subtree can't be used in kj promises using only a single new
assumption: if EvalState& is never held as a reference member of a type
and instead only ever passes as an argument or held on the stack we can
be certain that anything that has access to en EvalState ref must never
be run inside a promise and, crucially, that anything that doesn't have
access to an EvalState& *can* be run inside a promise without problems.

Change-Id: I6c15ada479175ad7e6cd3e4a729a5586b3ba30d6
2024-12-05 13:30:35 +00:00
eldritch horrors 2f2bc90d92 libexpr: associate positions with EvalContext
Change-Id: I43e827a9119324c23a6d72cfc1b7ed4d5ff32dd2
2024-12-04 23:50:31 +00:00
eldritch horrors 014845ddf1 libexpr: associate symbols with EvalContext
Change-Id: I4ca99dc1f5183c7e7383db2b9c896c1946aadd29
2024-12-04 23:50:31 +00:00
eldritch horrors a65e9e5828 libexpr: extract eval error creation into new type
this will let us pass the capability to create debuggable eval errors
without having to pass an entire EvalState. we could pass symbols and
debug states around just as easily, but if we add new capabilities to
our debugger we might have to change many more places than with this.

Change-Id: I2f8893012e5d98a986ef1fc888234c2dd8d5e096
2024-12-03 20:38:41 +01:00
piegamesandGerrit Code Review 9fb5315d06 Merge "libexpr: Deprecate overriding __sub and the like" into main 2024-11-29 07:56:17 +00:00
piegames 81d5f0a7d9 libexpr: Deprecate overriding __sub and the like
It was never intended to be a feature to be used, and moreover it is
inconsistent: One cannot override `+`, and overriding `__lessThan` won't
affect the builtins which do comparisons.

Change-Id: Iaba54a05aa4c2eb37cdb3dc0d731fcee5a86deba
2024-11-28 18:15:52 +01:00
eldritch horrors 985afeeb4d libexpr: allocate debug state only when debugger is active
Change-Id: Id30f388264c5d1e472e3bdce5078db3914f3b475
2024-11-28 15:08:35 +00:00
eldritch horrors 003883306d libexpr: move debug repl state into own struct
just to keep debug-related state closer together. eventually this will
also allow us to not allocate debugger state altogether unless needed.

Change-Id: Id83fea75d96d0ecbe21683cb1b57dd5b11b13535
2024-11-27 02:09:08 +01:00
eldritch horrorsandjade b0d7a81613 fix tooling after include reorganization
clangd broke because it can't look through symlinks. compile_commands
manipulation does not fix it, clangd configuration does not fix it, a
vfs overlay does not fix it, and while a combination of those can fix
it with a bind mount in place that's just too cursed to even consider

clangd bug: https://github.com/llvm/llvm-project/issues/116877

Change-Id: I8e3e8489548eb3a7aa65ac9d12a5ec8abf814aec
2024-11-19 22:55:32 +00:00