Commit Graph
56 Commits
Author SHA1 Message Date
piegames 5cfe785352 tests/functional2/lang: More trivial eval-okay test migrations
In the first pass I erred on the cautious side, only migrating safe
bets, to here's some trivial migrations that I missed

Change-Id: I934011919837b0aa491113afdcad603cf6b9cbbb
2025-09-18 23:38:46 +02:00
piegames 019b17f4e9 tests/functional2/lang: Migrate trivial eval-okay tests
Change-Id: I07a2e70eacd3d7bca3fc4d7074b9892b9ea35346
2025-06-02 21:52:34 +02:00
piegames 9673c6480a tests/functional2/lang: Migrate trivial parse-okay tests
Change-Id: I11b6ea26b7dc6bcf8250c04d3c97ded452dd1eda
2025-06-02 21:52:34 +02:00
piegames 0219434ae9 tests/functional2/lang: Migrate trivial parse-fail tests
Change-Id: If6cd6a2432c081e4b918f480ce3be0da3e691d40
2025-06-02 21:52:34 +02:00
piegames 63edb9c678 tests/functional2/lang: Migrate trivial eval-fail tests
All changes are uniform and done with the same script, so checking only
some should suffice. For that reason, any tests involving multiple files
or custom CLI flags are not included in this commit.

Change-Id: Ib2d0e08937b56e241d99771a58aad34ed3ad308a
2025-06-02 21:52:34 +02:00
piegames a7e5ff0070 tests/functional2: Make symlink handling less confusing
The current `RelativeTo` design is both more complex and more confusing
than necessary. Its four variants are now reduced to only two. They are
now also represented as different classes, to better communicate the
difference in semantics and also intent.

Change-Id: Ia60fc7a2dfa0f62bdef90dde347fd8603fd3fbf9
2025-06-02 21:52:34 +02:00
piegames 1071643259 tests/functional/lang: Clean up lib.nix
That file was written once in 2008 and never updated since, and let's
just say that a lot of things have changed since

Change-Id: I66b0c87ecbba6ca653470966c9514edb21882ca3
2025-05-24 08:49:09 +02:00
piegames 5d49e26f71 tests/functional/lang: Remove disabled tail-call test
It was introduced back in 2013, was disabled in 2014 again for dubious
reasons and according to horrors is unsound anyways and can never really
work.

It was the only disabled test, so I removed the "infrastructure" for
that in the test runner as well. functional2/lang will have much better
ways for skipping tests anyways

Change-Id: Icb8697fb85221e3206fb64cb917c03607ef278a7
2025-05-22 01:48:34 +02:00
piegames 76d6b51f5c tests/functional/lang: Don't pipe input into stdin
Back in the days, this used to be the modus operandi, but then, still
many but less years ago, Eelco came along and changed it to passing in
the actual file. Of course, no motivation was provided, and it was only
done on half of the test runners for some reason, leaving us to wonder
what the true intentions of this code are …

Anyways, with this commit now everything standardises on passing in the
file by path instead of via stdin. Motivation:

- We need to `sed` out the path anyways for various other reasons,
including import tests and path value tests
- Given that, the presumed primary motivation for using stdin in the
first place becomes moot
- Bonus points for giving better error messages, especially in tests
that involve multiple input files

Change-Id: Ic6de1ec24f4c4d3c05e33d1ee053614784677513
2025-05-22 01:48:34 +02:00
piegamesandCommentator2.0 e52cad0285 tests/functional/lang: Don't use tabs for indentation
Don't ask me the fuck why, but *somehow* Nix prints error locations
differently if the input file is passed as a path vs through stdin, and
I have a hunch that this might have to do with tabs

Change-Id: I186b0edb90edd48856da3621815463e372c37512
2025-05-21 20:36:20 +02:00
piegamesandCommentator2.0 e2944876cc tests/functional/lang: Change base path to the lang directory
Currently, all tests are relative to `./tests/functional` instead of
`./tests/functional/lang`. Whether this is a historical artefact or as
intended, the current move is to align the tests with the new design of
functional2, preparing them for an easier migration.

Change-Id: Ie394691b071488a8000a005080b9167786d5bd9a
2025-05-21 20:36:20 +02:00
piegames 0fbbb1e49b libexpr: Switch StaticEnv to LinearMap
Change-Id: If98bfafce9fa5235fe962274c03c619fe965dd60
2025-05-19 16:21:10 +02:00
piegames bd8ec106fa libutil: Introduce LinearMap
Change-Id: I68ce4c1dc17b0742690e49f62206c65f5a1a4a30
2025-05-19 15:35:52 +02:00
2b22dae1ba lixexpr: Declutter error traces
Now that we can correctly point to all expressions, we can remove
redundant intermediate traces to reduce clutter.

Co-authored-by: eldritch horrors <pennae@lix.systems>
Co-authored-by: Raito Bezarius <raito@lix.systems>
Change-Id: I3e9d7c1c7a6599a8e68302448bbb961d051002b7
2025-04-24 20:44:57 +02:00
piegames abb8ad29c0 repl: Always use parser, allow trailing ; in assignments
We now properly shell out to the parser instead of hacking stuff
together with a regex. Stuff we get for free by doing this:

- Optional trailing semicolon
- Declaring nested attribute sets
- String identifiers, and future proofing for eventual grammar
improvements to identifiers
- Dynamic attributes

Change-Id: Ibf1ad815e5e27caf162df05ea5ba5b1b4955d9c9
2025-04-21 20:19:25 +02:00
piegames 336f467b3f repl: Print message when adding variable
I've always been annoyed that it just silently succeeded without any
feedback, but now with the upcoming improvements on defining variables
this is more necessary than ever.

Change-Id: I565897fa2f97cf6f567d4449dcc8d4ad4eb73fce
2025-02-28 17:13:53 +01:00
piegames 708f1ea342 repl: Add changelog entry for recent improvements
I forgot to add a change log earlier, sorry

Change-Id: Idde2f26b8f0b9aa0bdbfcdb10a070ef1dd663d58
2025-02-28 17:13:53 +01:00
piegames 233be9be14 libexpr: Clean up BindingsState
- `pos` is not used anymore and can be removed
- `path` and `value` are more correctly placed in their own sub state,
to mirror `InheritState`

Change-Id: Ie7c8a3aea3fe4af0f9c5c15ab5897a59d1b3f7b0
2025-02-28 17:13:53 +01:00
piegames 65ce95d381 libexpr: Deprecate NUL bytes in strings
I absolutely want to go back to allowing them, I am strongly of the
opinion that nothing should be special about NUL in a string, but this
will have to wait until at least the GC rewrite, so for now the least I
can do is to disarm this gun.

Change-Id: Id14b6037bc8b274c6c60ad970b1c74d436fb62a7
2025-02-28 16:17:17 +01:00
piegames e38ad66ef7 libexpr: Deprecate CR and CRLF line endings
They are broken beyond repair, and barring a language version revision
this is our only option besides doing nothing about it.

Change-Id: I25fa4f032ca9b5ca67356946bcd889289583592e
2025-02-28 16:17:17 +01:00
piegames 2e5239d2e5 libexpr: Refactor formals handling
This is kind of a painful change, because a lot of code grew around the
bad abstraction, but the goal here is to abstract formals in a way that
allows adding other means of pattern matching / argument destructuring
in the future.

Change-Id: If4e681a4be3d1f42ceea81a8e07297f0d08acc80
2025-02-24 13:48:29 +01:00
piegames 0928d4d87a flakes: Make checkOverlay less strict
I'm sorry but checking the name of the function arguments to be "final"
and "prev" is just absurd. That's like worst of
type-checking-meets-static-analysis. And the error message of the second
`throw` was not even good, as it mixed all three failure cases together.

Change-Id: Id4244171123dd8a228be71ce9f04d8e9f647c111
2025-02-24 12:54:48 +01:00
piegames f8a592713d repl: Allow :env outside of debug mode
Printing the current variables plus staticBaseEnv is useful

Change-Id: Ide8bc26c1c6b4ec0b1d68767994ddac8891a649f
2025-01-28 09:37:01 +01:00
piegames 1b2e7e1ab7 repl: Improve error messages for debug commands outside of debug mode
Pretending that these commands are "unknown" when not in debug mode is
just silly.

Change-Id: I7036ae792656ef540bbad00649be4f7309944a07
2025-01-28 09:37:00 +01:00
piegames f324574265 libexpr: Track position information in all expressions
Change-Id: Ied79381a917a715cdd8816d6f9df527289217dfb
2025-01-27 20:49:25 +01:00
piegames d0361e9467 libexpr: Unify literal expressions as ExprLiteral
Change-Id: Ifc0f1e27b46803fa2175801ed7d3d108896c8259
2025-01-27 20:49:25 +01:00
piegames 9efa32d651 libexpr: Refactor ExprState::pushExpr
It has been renamed to `emplaceExpr` to be more consistent with the fact
that it actually creates the expression pointer. Moreover, `pushExpr`
has been added which directly takes a `unique_ptr<Expr>`.

All manual calls to `exprs.emplace_back` have been removed and replaced
with `pushExpr`, and `exprs` is now a protected field. This allows for
enforcing invariants and modifying state within `pushExpr`.

Change-Id: I995f0bdc1c090cf78080c9f0b7737e2be359b3e4
2025-01-21 11:35:51 +01:00
piegames cf57b5c14c libexpr: Remove Expr::show, add JSON expression serialization
The code for serialization Expr nodes back into (pseudo-)Nix has been
removed for being subtly error-prone and tedious to maintain. Instead,
`nix-instantiate --parse` now prints a JSON representation of the AST.

Usage patterns of the --parse flag I've found in the wild:

1. Check if a file is well-formed, i.e. discard output and test exit code
2. Get parser errors from a file, i.e. discard stdout and use stderr
3. Nixfmt uses --parse to test equivalence pre/post format, and that property is (should be?) preserved

None of these should break with the current change

Closes #487

Change-Id: Icdbaad17790f2ad8765fa08e02e6597ee4c7a909
2025-01-21 11:35:51 +01:00
piegames 539c340c66 libexpr: Undeprecate overriding __findFile
After gathering more community feedback, more non-trivial use cases for
overriding `__findFile` emerged. Unlike the use case of Tvix mentioned
in #599, these can't easily be worked around by overriding `nixPath`
instead.

This is the second fixup/partial revert for
81d5f0a7d9.
See also #599.

Change-Id: I7ca75e1a2b196c0da341969c61f4c168b5f657f9
2025-01-04 19:07:35 +01:00
piegames 7c76053d93 libexpr: Undeprecate overriding __nixPath
It being overridable was an intended feature with good use cases, and
should not have been removed. However, this feature is generally in a
bad state and needs revisiting in the future.

Fixup for 81d5f0a7d9
Fixes #599

Change-Id: I2d93e012caa65aa795bce3a71d8e56d7052ef9df
2024-12-10 19:02:41 +01:00
piegames a2ae14bfd8 libexpr: Rework error messages on ExprSelct::eval
Calls to `show` have been removed. To counter the loss of information,
the error positions have been improved and now correctly point to the
current selector instead of the entire select expression.

Change-Id: I4771fe874af1ac15828a9863550cd4369a8f0e94
2024-12-08 20:52:18 +01:00
piegames 172515bd8b libexpr: Track position information in attrpaths
This is a pretty small change because the parser already has all
necessary information (needed for parse errors), now we want to keep it
to also provide better eval errors.

Change-Id: Ifc7a9516b9b0c8d9698f1899a6912ae91f6696ab
2024-12-08 20:51:45 +01:00
piegames cdda2454f6 libexpr: Assert: Don't print assertion in error message
The `show` functionality needs to be removed because it is deeply
flawed, and given that we already print position information in the
error message (which probably wasn't always the case in the past) the
assertion printing is redundant anyways.

Change-Id: I1f5e05ab73aaa0ec92994c2211463260fd374898
2024-12-08 20:51:44 +01:00
piegamesandeldritch horrors 9611018c27 libexpr: Split ExprAttrs and ExprLet
ExprLet was previously inheriting from ExprAttrs for the data, while
ignoring all
set-specific operations on it. The set specific code has now been split
off so that
let doesn't inherit it anymore:

- ExprAttrs (not an Expr), containing the attributes and the related
logic
- ExprLet : Expr, ExprAttrs
- ExprSet : Expr, ExprAttrs

Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I63f2fbcd1e790b3cffb56eec1e7565ee3cdbf964
2024-12-08 12:54:38 +01:00
piegames 81d5f0a7d9 libexpr: Deprecate overriding __sub and the like
It was never intended to be a feature to be used, and moreover it is
inconsistent: One cannot override `+`, and overriding `__lessThan` won't
affect the builtins which do comparisons.

Change-Id: Iaba54a05aa4c2eb37cdb3dc0d731fcee5a86deba
2024-11-28 18:15:52 +01:00
piegames e2d00ac3a8 libexpr: Fix typo in error message
Closes #523

Change-Id: Ib5705e405b74d07a8fcf0163847405e9c791c3e3
2024-10-18 19:37:23 +02:00
piegames 3ba5ef91bc benchmarks: Fix purity
Some stuff wasn't working on my machine because it depended on the host
environment in some ways, fixed those.

Change-Id: Iff4931a9a26c6827978f1ee6434710f406d67a96
2024-10-18 19:37:23 +02:00
piegames e5de1d13c4 libexpr: Optimize complex indented strings
The old behavior results in lots of concatenations happening for no good
reason and is an artifact of the technical limitations of the old parser
(combined with some lack of care for such details).

Change-Id: I0d78d6220ca6aeaa10bc437e48e08bf7922e0bb3
2024-10-18 19:37:23 +02:00
piegames 878e181882 libexpr: Print interpolations more accurately in show
This is only a minor semantical distinction, but we should be able to
properly test it, and the parser tests rely on show for that.

Change-Id: I25e868cf9544e30cdff17deb5fd50a434e0f367e
2024-10-18 11:40:04 +00:00
piegames c852ae60da libexpr: Rewrite stripIndentation for indented strings
This commit should faithfully reproduce the old behavior down to the
bugs. The new code is a lot more readable, all quirks are well
documented, and it is overall much more maintainable.

Change-Id: I629585918e4f2b7d296b6b8330235cdc90b7bade
2024-10-18 11:40:04 +00:00
piegames 765771a355 tests/functional-lang: Add test cases for indented strings
Accidentally came across them elsewhere, wanted them covered in the
ind-strings test as well.

Change-Id: Iba418a687388ba85516d13d4c9d815744297dc5c
2024-10-18 11:40:04 +00:00
piegames 14291856e4 tests/functional-lang: Test indented string parsing
There is already an eval test for it, but it doesn't test the parser's
output

Change-Id: I94809080f90ace8aa3f610bcd315e9df0cf5d12f
2024-10-18 11:40:04 +00:00
piegames e7d6212f77 libexpr: move parser semantics into separate file
Another preparation for forking off and versioning the parser

Change-Id: I7b1225a44a3b81486414c1d37bd3e76a3ab307f9
2024-10-18 11:40:04 +00:00
piegames f98ee07573 libexpr: rename grammar to grammar::v1
Let's make some space in the namespace for a v2

https://wiki.lix.systems/books/lix-contributors/page/nix-lang-v2
Change-Id: If56e6dbf680d931233aa822ef91c8832464471e4
2024-10-18 11:40:04 +00:00
piegames 580df9bdb3 tests/functional-lang: Small cleanup of flagfiles handling
Flag files now properly work for all tests.

Change-Id: If652235960e8cf4120270a1b974c17586a31a157
2024-10-18 11:40:04 +00:00
piegames 0edfea450b libexpr: Soft-deprecate ancient let syntax
Change-Id: I6802b26f038578870ea1fa1ed298f0c4b1f29c4a
2024-08-21 12:59:03 +02:00
piegames 0a8888d1c7 treewide: Stop using ancient let syntax
Shows for how long these tests have not been touched by anyone …

Change-Id: I3d0c1209a86283ddb012db4e7d45073264fdd0eb
2024-08-21 06:55:52 +00:00
piegames 7210ed1b87 libexpr: Soft-deprecate __overrides
Change-Id: I787e69e1dad6edc5ccdb747b74a9ccd6e8e13bb3
2024-08-21 06:55:52 +00:00
piegames 007211e7a2 libutil: Optimize feature checks
Instead of doing a linear search on an std::set, we use a bitset enum.

Change-Id: Ide537f6cffdd16d06e59aaeb2e4ac0acb6493421
2024-08-18 16:56:49 +00:00
piegames 278fddc317 libexpr: Deprecate URL literals
Closes #437.

Change-Id: I9f67fc965bb4a7e7fd849e5067ac1cb3bab064cd
2024-08-17 20:31:57 +02:00
piegames 49d61b2e4b libexpr: Introduce Deprecated features
They are like experimental features, but opt-in instead of opt-out. They
will allow us to gracefully remove language features. See #437

Change-Id: I9ca04cc48e6926750c4d622c2b229b25cc142c42
2024-08-17 19:47:51 +02:00
piegames 1c080a8239 treewide: Stop using URL literals
They must die

Change-Id: Ibe2b1818b21d98ec1a68836d01d5dad729b8c501
2024-08-17 15:48:10 +00:00
piegamesandGerrit Code Review e03cd8b3a6 Merge "libexpr: Add experimental pipe operator" into main 2024-08-08 18:15:21 +00:00
piegames 28ae24f3f7 libexpr: Add experimental pipe operator
The |> operator is a reverse function operator with low binding strength
to replace lib.pipe. Implements RFC 148, see the RFC text for more
details. Closes #438.

Change-Id: I21df66e8014e0d4dd9753dd038560a2b0b7fd805
2024-08-08 11:13:53 +02:00
piegames ec7552ff74 libexpr/parser: Test experimental features
Currently, the parser relies on the global experimental feature flags.
In order to properly test conditional language features, we instead need
to pass it around in the parser::State.

This means that the parser cannot cache the result of isEnabled anymore,
which wouldn't necessarily hurt performance if the function didn't
perform a linear search on the list of enabled features on every single
call. While we could simply evaluate once at the start of parsing and
cache the result in the parser state, the more sustainable solution
would be to fix `isEnabled` such that all callers may profit from the
performance improvement.

Change-Id: Ic9b9c5d882b6270e1114988b63e6064d36c25cf2
2024-08-07 13:07:50 +00:00
piegames 3dced96741 devShell: Fix errors with environment variables
Previous code assumed that $MANPATH always exists

Change-Id: I5a4d012045ba6ff9086373b3f46a75d82285d393
2024-07-10 14:22:47 +02:00