Commit Graph
2574 Commits
Author SHA1 Message Date
eldritch horrors 1210a4e025 libstore: flush log lines only from handleBuilderOutput
also change the tests to ensure that we handle all lines correctly.

Change-Id: I13de286bda8eee57acd53af1ab5c081d3048b500
2025-10-13 11:26:49 +00:00
eldritch horrors f29dfb3d3c libutil: add buffer state management to loggers
currently all loggers can always accept messages and never suggest
flushing buffers. in the future this may change, and at that point
we're already fully set up for it. local loggers should never keep
asynchronous (i.e. network-backed) buffers, disk buffers are fine.
networked loggers will require buffers and periodic flushes later.

Change-Id: Ide2114f5bc17f4a1d289c92ed4f9381a1d59dacf
2025-10-13 11:26:49 +00:00
eldritch horrors b73a7f1815 libutil: clean up the Activity interface a bit
create activities from loggers themselves instead of passing the logger
as a constructor argument and allow direct construction of children, no
direct logger access needed. most call sites are not changed because we
still need to handle the "no parent" case, and the logger method can do
that more cleanly than a ternary at each site that creates an activity.
we may eventually want to create a root activity, which is cleaner too.

Change-Id: I295e056228dabb08a1316eba7973874784baa113
2025-10-12 22:37:31 +00:00
eldritch horrors b88a6e6f11 libexpr: use pascal strings for eval
this has no performance impact in any benchmarks we've run. nul bytes
are still used as implicit truncation points in many places all over:
rejecting them in all locations that treat them as a string end point
requires large changes such as using a proper path library everywhere

Change-Id: I936158bd435f6abf009a689adfbc24496262c578
2025-10-11 12:57:57 +02:00
eldritch horrors 53d172a308 libutil: remove unused CanonPath methods
Change-Id: Id593e0d03373af88e03a9da473935e5c00f86b33
2025-10-10 16:55:34 +00:00
eldritch horrors 2f16cf6014 treewide: lint syscalls for c string safety
Change-Id: I3caf476e59dcb7899ac5a3d83dfa3fb7ceaaabf0
2025-10-10 16:13:54 +00:00
eldritch horrors 3ef7e0711b Revert "functional2: fix in the development shell"
This reverts commit 42691f0d94.

Reason for revert: darwin builds time out now, apparently due to newly appearing infinite loops in the test suite

Change-Id: I81a09efd53636a287f3095ac87ed8854b6e57922
2025-10-10 12:49:45 +00:00
Commentator2.0 5e2412ea7e tests/functional2: migrate add.sh
Change-Id: I699929b61db19a4ab7bea8281df830de9fea6d04
2025-10-08 13:09:17 +02:00
Alois Wohlschlager 42691f0d94 functional2: fix in the development shell
The functional2 test suite was broken in the development shell for two mostly
separate reasons leading to "no such file or directory" errors:

* The `BUILD_TEST_SHELL` (already containing the correct path) would always be
  set from the Meson option `build_test_shell`, even to the empty string if
  that option is not set. Skip the overwrite in this case to make the
  environment variable work again.
* Sandboxed builds would fail to find their builder `/bin/sh`, since in the
  development shell busybox is not found, so no sandbox shell gets configured.
  Use the shell from `BUILD_TEST_SHELL` instead. (How this does not break the
  old functional tests remains a mystery.)

Change-Id: I6a6a696424e8caaef3f9b68e3738bfd58ea0b056
2025-10-07 20:14:48 +00:00
eldritch horrors 80654b84b6 libexpr: de-ptr-ize Value references
thunk values are shareable, and we can represent invalid/uninitialized
values with a special bit pattern that makes no sense otherwise. there
is no need to keep allocating values on the heap, instead we can treat
values like reference-counted smart pointers to heap objects, which in
turn lets us save a lot of allocations and, ultimately, gc heap space.

compared to our baseline (main of 2025-09-27) we save 15%+ memory on a
system rebuild and 17% on nix search. eval time regresses by ~3% for a
system rebuild, while nix search is 7% faster. further optimization is
probably possible (but for now this will just have to be good enough).

Change-Id: Ib6c47acdbe2fac4f76a83c2269f16f30ef66b2e1
2025-10-05 16:23:05 +02:00
eldritch horrors 28c4431c69 libexpr: reformat stuff we'll change soon
mainly to keep the next diff smaller. it'll be large enough as is.

Change-Id: Ib8a34520f03539cbf6aa2f0e66cbed05fe1225eb
2025-10-05 16:22:35 +02:00
eldritch horrors db35aa753b libexpr: de-ptr-ize many Value uses
with thunk state being shareable we no longer need to worry about value
uniqueness, only about value lifetime. this means we can liberally drop
indirections and allocations, passing references instead of pointers or
using stack memory instead of gc-managed memory for some intermediates.

Change-Id: I2d48a6fd57a376d544bd9bd2d05e5420611986d1
2025-10-05 16:22:35 +02:00
eldritch horrorsandRaito Bezarius 57333a0e60 libutil: preserve ForeignException type info
preserve type information of wrapped exception when constructing a
ForeignException. this requires using `std::current_exception` for
construction in all cases as passing the exception by reference to
any function erases dynamic type information: `throw` may copy the
thrown exception from its source expression; this only works as we
need it to if static and dynamic type of the expression are equal.

fixes #1000

Change-Id: I7167f298540de7c43c2566592bdb8d7a93c34d31
2025-10-05 00:40:22 +00:00
eldritch horrors 0e9e9fd917 libexpr: heap-allocate "large" integers
Change-Id: Ic391f2f1bf87f044d7a688196ba9e0ad766d65aa
2025-10-04 16:31:45 +02:00
eldritch horrors 79586575c5 libexpr: heap-allocate app nodes
Change-Id: I9a39dcf0be7589cedf494757e21665c5d50e446b
2025-10-04 16:31:45 +02:00
eldritch horrors 94a866d44f libexpr: heap-allocate thunk control state
Change-Id: I20d8ab1d6f683c0a2f3b77edf9bdad147d62c8fa
2025-10-04 16:31:45 +02:00
eldritch horrors d84f3003cf libexpr: move lambdas to auxiliary storage
Change-Id: Ibe4885f17c0ba1634ed6dbca0a45f8bd4619d69b
2025-10-04 16:25:05 +02:00
eldritch horrors 1b5f4eb5eb libexpr: move primops to auxiliary storage
same as for null: we have few of them, they're statically allocated,
and they're not the largest contributor to the value population. not
storing them in Value itself frees up resources we *will* use later.

Change-Id: I521f9f243f48f56a78f7bffdf1dc1f0bc40a5c2d
2025-10-04 16:25:04 +02:00
eldritch horrors 1c4e77387a libexpr: remove tPrimOpApp
using the same nodes as tApp is possible, and thanks to multi-arg app
nodes it can even be a bit faster than the linked lists used to date.

Change-Id: Idccb7c0b54c808e62da85d1c42ee09e6e92c4f7b
2025-09-29 17:56:05 +02:00
eldritch horrors 0b5b14ddc7 libexpr: add multi-arg app nodes
these behave like the old chains of app nodes, but they can store more
than one argument per node. for tApp values themselves this is not all
that useful, but if we could share tApp and tPrimOpApp backing storage
we could avoid creating and traversing the linked lists of values that
are currently needed to represent partially applied builtin functions.

Change-Id: I5a2a02d9733e1e0be5443459e2998d62fd3b9a5b
2025-09-29 17:56:05 +02:00
eldritch horrors 0d69da184c libexpr: unify strings and paths
paths already are just strings with slightly magic semantics. the type
can mirror that at no perf cost, letting us drop one internalType tag.

Change-Id: I98acaa4fe3bedd28fc0841e1b81184d8dcddacc5
2025-09-29 17:56:05 +02:00
eldritch horrors d23e3a09a4 libexpr: alloc list storage as a (length, vla) type
memory overhead is minimal and performance impact not measurable. once
we've done something like this for all value types that don't fit in a
single machine word we can cut a word from Value, offsetting the cost.

Change-Id: I9813bacd7e851957ad3426aed8f74033179a4212
2025-09-29 17:55:54 +02:00
eldritch horrors 01f1852360 libexpr: don't inline small lists into values
this has no measurable performance impact thanks to the new caches.

Change-Id: Ib403a9a567161675f78e8c5d314d6340183d181d
2025-09-29 15:22:41 +02:00
eldritch horrors c7cc7d6c31 libexpr: "hide" Value union members
on its own this is not very useful, but having accessors for every value
kind is a prerequisite for doing smart things with Value than the union.
the net effect for now is only to add a few parentheses across the tree.

Change-Id: I88688ac09eb08495dad1eb221034ca540f094950
2025-09-29 15:22:41 +02:00
eldritch horrors e4ee5e2eb5 testing: disable substitution in f2
mirrors f1. much faster.

Change-Id: I38bbbd5b26220f480afe76ef9302e5c90f21507a
2025-09-28 13:19:13 +00:00
Commentator2.0 1e69b17f0a tests/functional2: migrate timeout.sh
Change-Id: Ieb12c46bc987c74a6adda07e6c6ed33aebf8a96c
2025-09-21 17:56:19 +02:00
Commentator2.0 e0739afb42 tests/functional2: migrate build.sh
Change-Id: Iac3f3dc021436fc693a3729ba522d4bf1f124220
2025-09-21 13:58:26 +02:00
Raito BezariusandSergei Zimmerman 057b725ae3 libexpr: use const references or pointers to attribute sets
Prior to this change, references or pointers could be mutated. In
practice, we do not require this capability in the codebase except in
zipAttrsWith.

This cleans up all easy sites in preparation to have a smarter
representation of attribute sets albeit one that requires constant
references.

Change-Id: I2be20cce040a9228bde9e5f7b42c0499fba9550b
Signed-off-by: Raito Bezarius <raito@lix.systems>
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2025-09-20 20:31:19 +02:00
Commentator2.0 3a4f9488f0 tests/functional2: migrate nix_path.sh
Change-Id: Ibc4f848721d4da628c44aa074faf009c87af165d
2025-09-19 20:51:49 +02:00
Commentator2.0 fc2eedf9db tests/functional2: migrate why-depends.sh
Make builds work on darwin and migrate the first building test :D

Change-Id: Ia301c78d3bc771cd826d93dc114b098167043e97
2025-09-19 20:51:49 +02:00
Commentator2.0 a25a5739c7 tests/functional2: migrate search.sh
Change-Id: Id0f47fc72191d913442ff35d47df1b87ceb9f0be
2025-09-19 15:09:22 +02:00
piegames 2a622d7de7 tests/functional2/lang: Migrate value printing tests
Change-Id: I39303ef44552eb9fc17ccc2da5206c8906e46f27
2025-09-18 23:38:46 +02:00
piegames fe7a8d5e67 tests/functional2/lang: Migrate arithmetic tests
Change-Id: I73259875debc053dca46fe34f9f8d10b8486fa4f
2025-09-18 23:38:46 +02:00
piegames 096aa37cf1 tests/functional2/lang: Migrate ind-string tests
Change-Id: Ibc764ab97a53b751287654300211f3a9d8750b74
2025-09-18 23:38:46 +02:00
piegames 21a0b81a26 tests/functional2/lang: Migrate function args tests
Change-Id: Ia805d7fbf74c671fc419fb8c9db3960b6be62354
2025-09-18 23:38:46 +02:00
piegames f2b64142a9 tests/functional2/lang: Migrate dup-attrs tests
Change-Id: Ia5976229c8b2e6bf89d821b5676a9a397e4539d7
2025-09-18 23:38:46 +02:00
piegames 0d4a6132dc tests/functional2/lang: Migrate builtins.deepSeq test
Change-Id: I5caeb8a606d8cba519267c6011561afc9813598c
2025-09-18 23:38:46 +02:00
piegames 3692cf1ea2 tests/functional2/lang: Migrate inherit tests
Change-Id: I0f09f2b4ffa2edc5031351d0426865d7605ff7fa
2025-09-18 23:38:46 +02:00
piegames c7b8ac7cd9 tests/functional2/lang: Migrate substring tests
Change-Id: I902de6f3aebc902d56a79f8909bd6c9b5ba70cda
2025-09-18 23:38:46 +02:00
piegames 81baa64847 tests/functional2/lang: Migrate eval-okay-eq*
Change-Id: I17f81a765bbdbde3a27dd547e1ef65bffa97bce5
2025-09-18 23:38:46 +02:00
piegames 16db4da8f0 tests/functional2/lang: Rename tests
Most tests for builtins now have `builtins.builtinName` as their name.
This makes navigating the test list a bit easier

Change-Id: Ief5af5c568a419bf9130601f9590e7a696b0dc0a
2025-09-18 23:38:46 +02:00
piegames 5cfe785352 tests/functional2/lang: More trivial eval-okay test migrations
In the first pass I erred on the cautious side, only migrating safe
bets, to here's some trivial migrations that I missed

Change-Id: I934011919837b0aa491113afdcad603cf6b9cbbb
2025-09-18 23:38:46 +02:00
Commentator2.0 a9f28622c8 tests/functional2: migrate pure-eval.sh
Change-Id: Ieadb54a27b1cedcf3a4ce65e024a68440759ce7e
2025-09-16 20:06:07 +02:00
eldritch horrors d90e4a6581 libutil: reject malformed nars harder with better errors
fixes #993

Change-Id: I56aff32498ecd32e65f1576661d5a15808a4439f
2025-09-12 23:22:11 +02:00
eldritch horrors 95d43d1e39 testing: unrace functional-repl
using sleep(1) as a synchronization mechanism does not work. use fifos instead.

fixes #690 and ci constantly falling over in the same exact fucking source line

Change-Id: I51725f8e439b6753f3212d2897dbb0620ad77a37
2025-09-12 23:22:11 +02:00
Qyriad 138c7161be nix3-develop: correctly escape ASCII control characters for JSON
Fixes #991.

Change-Id: Iafc7d9603fbc3615393d32d9630f0e8fe548950b
2025-09-12 09:42:20 +00:00
Alois Wohlschlager 91bf54d726 flake: update nixpkgs input
Without https://github.com/NixOS/nixpkgs/pull/434761 evaluation of the
`nixpkgsLibTests` will fail in CI with recent enough Lix, due to reliance on
the TOML integer saturation bug.

Reported-by: Sergei Zimmerman <sergei@zimmerman.foo>

Change-Id: I6a6a6964838009d2c525f67035f84072fdfad988
2025-09-07 16:50:37 +02:00
Commentator2.0 9cdc2bf61c tests/functional2: migrate hash.sh
Change-Id: I5061479b972088c904ca1db4d9004e738c2b5a57
2025-08-25 21:29:27 +02:00
Alois Wohlschlager 451a14980b libstore: use OpenSSL for Ed25519 signatures
Previously two cryptography libraries were linked into Lix: OpenSSL used for
hashing and (in usual configurations) indirectly via curl for TLS, and Sodium
used only for handling the Ed25519 path info signatures. The latter is
functionally redundant since OpenSSL supports the same use case as well.
Reimplement the Ed25519 handling using OpenSSL and drop Sodium.

Fixes: https://git.lix.systems/lix-project/lix/issues/969

Change-Id: I6a6a696456b9d3ad7fdc2bf9b0759836a6247a38
2025-08-25 17:11:45 +00:00
Commentator2.0andCommentator2.0 7b6a85982b tests/functional2: provide a way to easily access often used assets
Change-Id: I461ee08d9752d972e7485e15186426f98e68ba13
2025-08-24 10:38:39 +02:00