Commit Graph
18393 Commits
Author SHA1 Message Date
eldritch horrors 32440b5fae libexpr: heap-allocate "large" integers
Change-Id: Ic391f2f1bf87f044d7a688196ba9e0ad766d65aa
2025-09-28 00:02:21 +02:00
eldritch horrors aa39e14fcf libexpr: heap-allocate app nodes
Change-Id: I9a39dcf0be7589cedf494757e21665c5d50e446b
2025-09-28 00:02:21 +02:00
eldritch horrors b0d11f9da1 libexpr: heap-allocate thunk control state
Change-Id: I20d8ab1d6f683c0a2f3b77edf9bdad147d62c8fa
2025-09-28 00:02:20 +02:00
eldritch horrors 002dfbb2e3 libexpr: move lambdas to auxiliary storage
Change-Id: Ibe4885f17c0ba1634ed6dbca0a45f8bd4619d69b
2025-09-28 00:02:20 +02:00
eldritch horrors 6e242e8b9b libexpr: move primops to auxiliary storage
same as for null: we have few of them, they're statically allocated,
and they're not the largest contributor to the value population. not
storing them in Value itself frees up resources we *will* use later.

Change-Id: I521f9f243f48f56a78f7bffdf1dc1f0bc40a5c2d
2025-09-28 00:02:20 +02:00
eldritch horrors b1ffae3ccd libexpr: move null to auxiliary storage
we only need the one object for it. there's no need to waste precious
bits of the value internal type enum for this simple singleton datum.

Change-Id: Ie314b5bf429015e518798d9d65ad8ab2bb84a38e
2025-09-28 00:02:20 +02:00
eldritch horrors 5def7559a6 libexpr: move floats to auxiliary storage
floats are used very rarely, and our float support is bad enough to
strongly discourage using them on reproducibility grounds alone. we
can thus move them to more expensive storage without hurting folks.

Change-Id: I1086f612f85e294dd3fae4a2d334e09f52bbe4a8
2025-09-28 00:02:20 +02:00
eldritch horrors ce70234904 libexpr: move external value refs to aux storage
external values very rarely appear during eval "normal" eval, and
creating them is pretty expensive. does *anything* even use them?

Change-Id: Id50fa3f76b7e1f551d550d99996a1ed5880b2531
2025-09-28 00:02:20 +02:00
eldritch horrors cbc378b277 libexpr: heap-alloc string control blocks
despite not using allocation caches this does not have a statistically
significant performance impact, with less than 1% extra memory needed.

Change-Id: Ibe51a55ba986e471f217f3724977af17880fafff
2025-09-28 00:02:20 +02:00
eldritch horrors 5d6bb8c350 libexpr: remove tPrimOpApp
using the same nodes as tApp is possible, and thanks to multi-arg app
nodes it can even be a bit faster than the linked lists used to date.

Change-Id: Idccb7c0b54c808e62da85d1c42ee09e6e92c4f7b
2025-09-28 00:02:20 +02:00
eldritch horrors 93bda92508 libexpr: add multi-arg app nodes
these behave like the old chains of app nodes, but they can store more
than one argument per node. for tApp values themselves this is not all
that useful, but if we could share tApp and tPrimOpApp backing storage
we could avoid creating and traversing the linked lists of values that
are currently needed to represent partially applied builtin functions.

Change-Id: I5a2a02d9733e1e0be5443459e2998d62fd3b9a5b
2025-09-28 00:02:20 +02:00
eldritch horrors dbee9d15c5 libexpr: use std::span for callFunction
Change-Id: I8c94bafabdb2416c85d9721d3d6424f52fbd45e0
2025-09-28 00:02:20 +02:00
eldritch horrors 7642b7227a libexpr: unify strings and paths
paths already are just strings with slightly magic semantics. the type
can mirror that at no perf cost, letting us drop one internalType tag.

Change-Id: I98acaa4fe3bedd28fc0841e1b81184d8dcddacc5
2025-09-28 00:02:20 +02:00
eldritch horrors aa96b00182 libexpr: alloc list storage as a (length, vla) type
memory overhead is minimal and performance impact not measurable. once
we've done something like this for all value types that don't fit in a
single machine word we can cut a word from Value, offsetting the cost.

Change-Id: I9813bacd7e851957ad3426aed8f74033179a4212
2025-09-28 00:02:20 +02:00
eldritch horrors d37a5d4000 libexpr: don't inline small lists into values
this has no measurable performance impact thanks to the new caches.

Change-Id: Ib403a9a567161675f78e8c5d314d6340183d181d
2025-09-28 00:02:20 +02:00
eldritch horrors b36a19e50c libexpr: cache more allocation sizes
we now use a single cache set for a number of sizes from one to eight
words. this also matches small attrsets, but perf impact seems small.

Change-Id: Icf16b329b98a20fcc9fe75e6395e148f0852c798
2025-09-28 00:02:20 +02:00
eldritch horrors f08b8532be libexpr: "hide" Value union members
on its own this is not very useful, but having accessors for every value
kind is a prerequisite for doing smart things with Value than the union.
the net effect for now is only to add a few parentheses across the tree.

Change-Id: I88688ac09eb08495dad1eb221034ca540f094950
2025-09-28 00:02:20 +02:00
eldritch horrors f3fcd7c02f bench: add memory benchmark
Change-Id: I4b2aa305b452a3a0c73e37953a5c332219d43d2c
2025-09-28 00:02:20 +02:00
eldritch horrors 1f0b25a92c bench: allow benchmarking a single build
Change-Id: Ie77247f9348eaeea740b0f894819ee2383d92312
2025-09-28 00:02:20 +02:00
eldritch horrors 82b771951c testing: disable substitution in f2
mirrors f1. much faster.

Change-Id: I38bbbd5b26220f480afe76ef9302e5c90f21507a
2025-09-28 00:02:20 +02:00
Raito Bezarius 992c3ae981 nix3/develop: support structured attrs-based output checks
nix develop should ignore output checks in general.

This was done only for the old way of specifying output checks, the
structured attrs way requires rewriting the JSON and removing the output
checks pieces.

We take a brutal approach of removing as many as possible including
non-recommended ways of doing it.

Fixes #997.

Change-Id: Iaf83029016c71b5171e56e15d4eadc1a60a8be98
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-09-23 14:35:08 +00:00
Alois Wohlschlager 43325669df daemon: let subdaemons outlive their parent again
In 7b37d5ea6a, aside from subdaemons getting
properly executed, they also lost the ability to outlive their parent, due to
now getting set the parent death signal like most other processes spawned by
Lix. This has annoying consequences like all concurrent builds being forcefully
terminated on system updates requiring a nix-daemon restart. As the behaviour
change was not documented and the systemd service file retained
`KillMode=process`, it seems to have been accidental. Restore the old behaviour
of letting the subdaemons outlive their parent.

Change-Id: I6a6a69645312a90dbce55495c2fef3825dd3c097
2025-09-22 19:51:58 +02:00
K900 f68233ec43 flake: unvendor capnproto
Nixpkgs has a recent enough version and all the relevant packaging changes now.

Change-Id: Ifdf814f80689803dd63ba7d16049e7d37e389542
2025-09-22 10:04:17 +03:00
Commentator2.0 1e69b17f0a tests/functional2: migrate timeout.sh
Change-Id: Ieb12c46bc987c74a6adda07e6c6ed33aebf8a96c
2025-09-21 17:56:19 +02:00
Commentator2.0 e0739afb42 tests/functional2: migrate build.sh
Change-Id: Iac3f3dc021436fc693a3729ba522d4bf1f124220
2025-09-21 13:58:26 +02:00
Raito BezariusandSergei Zimmerman a20a83b0a7 libexpr: replace Bindings::find by Bindings::get
In preparations for a new representation of bindings that will make it
impossible to write an efficient `Bindings::find`.

Change-Id: I4e5a25b8d37d01b5728f7fe43978ceda2ab1b9b6
Signed-off-by: Raito Bezarius <raito@lix.systems>
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2025-09-20 23:46:16 +02:00
Raito Bezarius 257e910247 libutil/comparator: support > for generation
Change-Id: Idd40b06c7051b9b0bd4c125359af0eb69996313a
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-09-20 23:45:41 +02:00
Raito Bezarius ccf3c5e30d libexpr/primops: make zipAttrsWith not mutate attribute sets
The way zipAttrsWith works is to replace the attribute set value by a
call to a function (the argument of zipAttrsWith) over the list of
attributes sharing a common key.

Instead of that, we will insert into the resulting attribute set the
various lazy calls and return that.

Change-Id: I2aae054eb99b1d1f8b0e7c658cc8d3488e5cdb01
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-09-20 23:45:41 +02:00
Raito Bezarius 19e4f11e88 libexpr/symbol-table: efficiently generate ≤, <, >, ≥, ==, != for Symbol
C++ has the "spaceship" operator which can be auto-implemented and
generates efficiently a strong ordering.

Change-Id: Idfd1fd68039b395e54401cbe913454e0cbd80fb3
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-09-20 23:45:38 +02:00
Raito BezariusandSergei Zimmerman 057b725ae3 libexpr: use const references or pointers to attribute sets
Prior to this change, references or pointers could be mutated. In
practice, we do not require this capability in the codebase except in
zipAttrsWith.

This cleans up all easy sites in preparation to have a smarter
representation of attribute sets albeit one that requires constant
references.

Change-Id: I2be20cce040a9228bde9e5f7b42c0499fba9550b
Signed-off-by: Raito Bezarius <raito@lix.systems>
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2025-09-20 20:31:19 +02:00
Alois Wohlschlager 3ee839cc5e libutil,nix: fix clashes with stdio stream identifiers
The stdio stream identifiers (stdin, stdout, stderr) are allowed to be macros.
In musl libc they are, for example doing `#define stdout (stdout)`, breaking
compilation with an error when one of the clashing variables is attempted to be
initialized the "wrong" way:

    ../lix/libutil/processes.cc:272:7: error: expected class member or base class name
      272 |     , stdout(stdout ? std::make_unique<AsyncFdIoStream>(std::move(stdout)) : nullptr)
          |       ^
    /nix/store/ziw42d7rvgnf3vkbfc8kry07kipwf1xm-musl-static-x86_64-unknown-linux-musl-1.2.5-dev/include/stdio.h:67:16: note: expanded from macro 'stdout'
       67 | #define stdout (stdout)
          |                ^

Other places only cause warnings on musl:

    ../lix/libutil/processes.cc:254:17: warning: parentheses were disambiguated as redundant parentheses around declaration of variable named 'stdout' [-Wvexing-parse]
      254 |     std::string stdout;
          |                 ^~~~~~
    /nix/store/ziw42d7rvgnf3vkbfc8kry07kipwf1xm-musl-static-x86_64-unknown-linux-musl-1.2.5-dev/include/stdio.h:67:16: note: expanded from macro 'stdout'
       67 | #define stdout (stdout)
          |                ^~~~~~~~
    ../lix/libutil/processes.cc:254:17: note: add a variable name to declare a 'std::string' (aka 'basic_string<char>') initialized with 'stdout'
      254 |     std::string stdout;
          |                 ^
          |                  varname
    /nix/store/ziw42d7rvgnf3vkbfc8kry07kipwf1xm-musl-static-x86_64-unknown-linux-musl-1.2.5-dev/include/stdio.h:67:16: note: expanded from macro 'stdout'
       67 | #define stdout (stdout)
          |                ^
    ../lix/libutil/processes.cc:254:5: note: add enclosing parentheses to perform a function-style cast
      254 |     std::string stdout;
          |     ^                 
          |     (                 )
    ../lix/libutil/processes.cc:254:17: note: remove parentheses to silence this warning
      254 |     std::string stdout;
          |                 ^
    /nix/store/ziw42d7rvgnf3vkbfc8kry07kipwf1xm-musl-static-x86_64-unknown-linux-musl-1.2.5-dev/include/stdio.h:67:16: note: expanded from macro 'stdout'
       67 | #define stdout (stdout)
          |                ^

However they are still wrong, since the macro could be more complicated. Fix
them as well.

Change-Id: I6a6a6964a50ef7dec8f05f0bd8fc8f13f3036d51
2025-09-20 11:25:00 +02:00
Alois Wohlschlager a7e1b23165 libexpr: remove liblixexpr_mstatic
It was supposed to be removed in 480fdf146d, as
it is not needed any more with the prelinked library. Due to a mistake in
rebase conflict resolution it reappeared by accident. Actually remove it now.

Change-Id: I6a6a6964d175fdb0ba0ad9ac55d4d22d7b27ad3f
2025-09-20 08:30:35 +00:00
Commentator2.0 3a4f9488f0 tests/functional2: migrate nix_path.sh
Change-Id: Ibc4f848721d4da628c44aa074faf009c87af165d
2025-09-19 20:51:49 +02:00
Commentator2.0 fc2eedf9db tests/functional2: migrate why-depends.sh
Make builds work on darwin and migrate the first building test :D

Change-Id: Ia301c78d3bc771cd826d93dc114b098167043e97
2025-09-19 20:51:49 +02:00
Commentator2.0 a25a5739c7 tests/functional2: migrate search.sh
Change-Id: Id0f47fc72191d913442ff35d47df1b87ceb9f0be
2025-09-19 15:09:22 +02:00
piegames 2a622d7de7 tests/functional2/lang: Migrate value printing tests
Change-Id: I39303ef44552eb9fc17ccc2da5206c8906e46f27
2025-09-18 23:38:46 +02:00
piegames fe7a8d5e67 tests/functional2/lang: Migrate arithmetic tests
Change-Id: I73259875debc053dca46fe34f9f8d10b8486fa4f
2025-09-18 23:38:46 +02:00
piegames 096aa37cf1 tests/functional2/lang: Migrate ind-string tests
Change-Id: Ibc764ab97a53b751287654300211f3a9d8750b74
2025-09-18 23:38:46 +02:00
piegames 21a0b81a26 tests/functional2/lang: Migrate function args tests
Change-Id: Ia805d7fbf74c671fc419fb8c9db3960b6be62354
2025-09-18 23:38:46 +02:00
piegames f2b64142a9 tests/functional2/lang: Migrate dup-attrs tests
Change-Id: Ia5976229c8b2e6bf89d821b5676a9a397e4539d7
2025-09-18 23:38:46 +02:00
piegames 0d4a6132dc tests/functional2/lang: Migrate builtins.deepSeq test
Change-Id: I5caeb8a606d8cba519267c6011561afc9813598c
2025-09-18 23:38:46 +02:00
piegames 3692cf1ea2 tests/functional2/lang: Migrate inherit tests
Change-Id: I0f09f2b4ffa2edc5031351d0426865d7605ff7fa
2025-09-18 23:38:46 +02:00
piegames c7b8ac7cd9 tests/functional2/lang: Migrate substring tests
Change-Id: I902de6f3aebc902d56a79f8909bd6c9b5ba70cda
2025-09-18 23:38:46 +02:00
piegames 81baa64847 tests/functional2/lang: Migrate eval-okay-eq*
Change-Id: I17f81a765bbdbde3a27dd547e1ef65bffa97bce5
2025-09-18 23:38:46 +02:00
piegames 16db4da8f0 tests/functional2/lang: Rename tests
Most tests for builtins now have `builtins.builtinName` as their name.
This makes navigating the test list a bit easier

Change-Id: Ief5af5c568a419bf9130601f9590e7a696b0dc0a
2025-09-18 23:38:46 +02:00
piegames 5cfe785352 tests/functional2/lang: More trivial eval-okay test migrations
In the first pass I erred on the cautious side, only migrating safe
bets, to here's some trivial migrations that I missed

Change-Id: I934011919837b0aa491113afdcad603cf6b9cbbb
2025-09-18 23:38:46 +02:00
Jade Lovelace 0018b98859 codeowners: add some missing directories
Change-Id: I22aa53074dedd822517a5e8dfa2f0a520d0000bb
2025-09-17 16:26:49 +00:00
Alois Wohlschlager d63a31e7a9 Remove obsolete editline.patch
Its usage has been removed in 3936f9b5de but for
some reason the file remained. Drop it now.

Change-Id: I6a6a696420f3425587f82bd485c6307e1ccff08d
2025-09-17 06:09:44 +00:00
Commentator2.0 a9f28622c8 tests/functional2: migrate pure-eval.sh
Change-Id: Ieadb54a27b1cedcf3a4ce65e024a68440759ce7e
2025-09-16 20:06:07 +02:00
eldritch horrors d648ec55eb libutil: asyncify runProgram2 stdout stream
fun side effect: post-build hooks are no longer single-threaded!

Change-Id: Ie5c75e588e4b2b8d515d4f1ae36de45b0a0807f7
2025-09-16 16:10:59 +00:00