Commit Graph
2555 Commits
Author SHA1 Message Date
eldritch horrors 0b5b14ddc7 libexpr: add multi-arg app nodes
these behave like the old chains of app nodes, but they can store more
than one argument per node. for tApp values themselves this is not all
that useful, but if we could share tApp and tPrimOpApp backing storage
we could avoid creating and traversing the linked lists of values that
are currently needed to represent partially applied builtin functions.

Change-Id: I5a2a02d9733e1e0be5443459e2998d62fd3b9a5b
2025-09-29 17:56:05 +02:00
eldritch horrors 0d69da184c libexpr: unify strings and paths
paths already are just strings with slightly magic semantics. the type
can mirror that at no perf cost, letting us drop one internalType tag.

Change-Id: I98acaa4fe3bedd28fc0841e1b81184d8dcddacc5
2025-09-29 17:56:05 +02:00
eldritch horrors d23e3a09a4 libexpr: alloc list storage as a (length, vla) type
memory overhead is minimal and performance impact not measurable. once
we've done something like this for all value types that don't fit in a
single machine word we can cut a word from Value, offsetting the cost.

Change-Id: I9813bacd7e851957ad3426aed8f74033179a4212
2025-09-29 17:55:54 +02:00
eldritch horrors 01f1852360 libexpr: don't inline small lists into values
this has no measurable performance impact thanks to the new caches.

Change-Id: Ib403a9a567161675f78e8c5d314d6340183d181d
2025-09-29 15:22:41 +02:00
eldritch horrors c7cc7d6c31 libexpr: "hide" Value union members
on its own this is not very useful, but having accessors for every value
kind is a prerequisite for doing smart things with Value than the union.
the net effect for now is only to add a few parentheses across the tree.

Change-Id: I88688ac09eb08495dad1eb221034ca540f094950
2025-09-29 15:22:41 +02:00
eldritch horrors e4ee5e2eb5 testing: disable substitution in f2
mirrors f1. much faster.

Change-Id: I38bbbd5b26220f480afe76ef9302e5c90f21507a
2025-09-28 13:19:13 +00:00
Commentator2.0 1e69b17f0a tests/functional2: migrate timeout.sh
Change-Id: Ieb12c46bc987c74a6adda07e6c6ed33aebf8a96c
2025-09-21 17:56:19 +02:00
Commentator2.0 e0739afb42 tests/functional2: migrate build.sh
Change-Id: Iac3f3dc021436fc693a3729ba522d4bf1f124220
2025-09-21 13:58:26 +02:00
Raito BezariusandSergei Zimmerman 057b725ae3 libexpr: use const references or pointers to attribute sets
Prior to this change, references or pointers could be mutated. In
practice, we do not require this capability in the codebase except in
zipAttrsWith.

This cleans up all easy sites in preparation to have a smarter
representation of attribute sets albeit one that requires constant
references.

Change-Id: I2be20cce040a9228bde9e5f7b42c0499fba9550b
Signed-off-by: Raito Bezarius <raito@lix.systems>
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2025-09-20 20:31:19 +02:00
Commentator2.0 3a4f9488f0 tests/functional2: migrate nix_path.sh
Change-Id: Ibc4f848721d4da628c44aa074faf009c87af165d
2025-09-19 20:51:49 +02:00
Commentator2.0 fc2eedf9db tests/functional2: migrate why-depends.sh
Make builds work on darwin and migrate the first building test :D

Change-Id: Ia301c78d3bc771cd826d93dc114b098167043e97
2025-09-19 20:51:49 +02:00
Commentator2.0 a25a5739c7 tests/functional2: migrate search.sh
Change-Id: Id0f47fc72191d913442ff35d47df1b87ceb9f0be
2025-09-19 15:09:22 +02:00
piegames 2a622d7de7 tests/functional2/lang: Migrate value printing tests
Change-Id: I39303ef44552eb9fc17ccc2da5206c8906e46f27
2025-09-18 23:38:46 +02:00
piegames fe7a8d5e67 tests/functional2/lang: Migrate arithmetic tests
Change-Id: I73259875debc053dca46fe34f9f8d10b8486fa4f
2025-09-18 23:38:46 +02:00
piegames 096aa37cf1 tests/functional2/lang: Migrate ind-string tests
Change-Id: Ibc764ab97a53b751287654300211f3a9d8750b74
2025-09-18 23:38:46 +02:00
piegames 21a0b81a26 tests/functional2/lang: Migrate function args tests
Change-Id: Ia805d7fbf74c671fc419fb8c9db3960b6be62354
2025-09-18 23:38:46 +02:00
piegames f2b64142a9 tests/functional2/lang: Migrate dup-attrs tests
Change-Id: Ia5976229c8b2e6bf89d821b5676a9a397e4539d7
2025-09-18 23:38:46 +02:00
piegames 0d4a6132dc tests/functional2/lang: Migrate builtins.deepSeq test
Change-Id: I5caeb8a606d8cba519267c6011561afc9813598c
2025-09-18 23:38:46 +02:00
piegames 3692cf1ea2 tests/functional2/lang: Migrate inherit tests
Change-Id: I0f09f2b4ffa2edc5031351d0426865d7605ff7fa
2025-09-18 23:38:46 +02:00
piegames c7b8ac7cd9 tests/functional2/lang: Migrate substring tests
Change-Id: I902de6f3aebc902d56a79f8909bd6c9b5ba70cda
2025-09-18 23:38:46 +02:00
piegames 81baa64847 tests/functional2/lang: Migrate eval-okay-eq*
Change-Id: I17f81a765bbdbde3a27dd547e1ef65bffa97bce5
2025-09-18 23:38:46 +02:00
piegames 16db4da8f0 tests/functional2/lang: Rename tests
Most tests for builtins now have `builtins.builtinName` as their name.
This makes navigating the test list a bit easier

Change-Id: Ief5af5c568a419bf9130601f9590e7a696b0dc0a
2025-09-18 23:38:46 +02:00
piegames 5cfe785352 tests/functional2/lang: More trivial eval-okay test migrations
In the first pass I erred on the cautious side, only migrating safe
bets, to here's some trivial migrations that I missed

Change-Id: I934011919837b0aa491113afdcad603cf6b9cbbb
2025-09-18 23:38:46 +02:00
Commentator2.0 a9f28622c8 tests/functional2: migrate pure-eval.sh
Change-Id: Ieadb54a27b1cedcf3a4ce65e024a68440759ce7e
2025-09-16 20:06:07 +02:00
eldritch horrors d90e4a6581 libutil: reject malformed nars harder with better errors
fixes #993

Change-Id: I56aff32498ecd32e65f1576661d5a15808a4439f
2025-09-12 23:22:11 +02:00
eldritch horrors 95d43d1e39 testing: unrace functional-repl
using sleep(1) as a synchronization mechanism does not work. use fifos instead.

fixes #690 and ci constantly falling over in the same exact fucking source line

Change-Id: I51725f8e439b6753f3212d2897dbb0620ad77a37
2025-09-12 23:22:11 +02:00
Qyriad 138c7161be nix3-develop: correctly escape ASCII control characters for JSON
Fixes #991.

Change-Id: Iafc7d9603fbc3615393d32d9630f0e8fe548950b
2025-09-12 09:42:20 +00:00
Alois Wohlschlager 91bf54d726 flake: update nixpkgs input
Without https://github.com/NixOS/nixpkgs/pull/434761 evaluation of the
`nixpkgsLibTests` will fail in CI with recent enough Lix, due to reliance on
the TOML integer saturation bug.

Reported-by: Sergei Zimmerman <sergei@zimmerman.foo>

Change-Id: I6a6a6964838009d2c525f67035f84072fdfad988
2025-09-07 16:50:37 +02:00
Commentator2.0 9cdc2bf61c tests/functional2: migrate hash.sh
Change-Id: I5061479b972088c904ca1db4d9004e738c2b5a57
2025-08-25 21:29:27 +02:00
Alois Wohlschlager 451a14980b libstore: use OpenSSL for Ed25519 signatures
Previously two cryptography libraries were linked into Lix: OpenSSL used for
hashing and (in usual configurations) indirectly via curl for TLS, and Sodium
used only for handling the Ed25519 path info signatures. The latter is
functionally redundant since OpenSSL supports the same use case as well.
Reimplement the Ed25519 handling using OpenSSL and drop Sodium.

Fixes: https://git.lix.systems/lix-project/lix/issues/969

Change-Id: I6a6a696456b9d3ad7fdc2bf9b0759836a6247a38
2025-08-25 17:11:45 +00:00
Commentator2.0andCommentator2.0 7b6a85982b tests/functional2: provide a way to easily access often used assets
Change-Id: I461ee08d9752d972e7485e15186426f98e68ba13
2025-08-24 10:38:39 +02:00
Maximilian Bosch e2641cb890 libstore/local-derivation-goal: show tree with references that lead to an output cycle
Closes #551

This adds a special accessor that falls back to checking if a store-path
exists within a chroot if it's not a valid path. That way,
`genGraphString` can find out which files have which references before
the outputs are registered.

Change-Id: I03c9d508fa3c72e5c262194461a25d71f3f4de15
2025-08-24 01:15:15 +02:00
Maximilian Bosch 2175d007e5 libstore/path-tree: allow passing a custom accessor
If none is given, we fall back to whatever accessor we get from the
store.

To display which paths actually contain the references leading to
e.g. a cycle or triggering a disallowedRequisites error, we'd
potentially have to look into the chroot from the previously finished
build. This behavior should not be part of the local accessor by
default, but part of a "special" accessor. This change allows using such
an accessor for `genGraphString()`.

Now that we inject the accessor from the outside, we have to mock it
anyways in the tests. Hence, this also adds a testcase for the
precise=True case.

Change-Id: I58465fb944776c2b0262ba054d1f296ed2ae3406
2025-08-23 18:36:49 +02:00
Maximilian Bosch 312e90f4b6 tests: add small testcase for output cycle detection
Change-Id: I186937dfbca4d051e5ad860239720816429a0a8e
2025-08-23 16:23:35 +02:00
Maximilian Bosch f7871fcb57 libutil/topo-sort: return std::variant<std::vector<T>, Cycle>
The variant has on the left-hand side the topologically sorted vector
and the right-hand side is a pair showing the path and its parent that
represent a cycle in the graph making the sort impossible.

The goal is to implement #551 which needs to throw an error if the
topo-sort fails. However, the error-message is supposed to contain a
graph of store-paths and the API to generate this is inherently async.

Now, catching the exception and re-throwing another one is impossible
since `co_await` is forbidden in `catch`-blocks and adding another
topoSort variant that allows an async `makeError` also seems odd. Hence,
I decided to alter the data-structure in use a bit for this use-case.
One out of two uses of the function are affected after all.

Change-Id: I70a987f470437df8beb3b1cc203ff88701d0aa1b
2025-08-23 16:23:35 +02:00
eldritch horrors be438c62e1 nix/eval: remove --write-to
it's broken, can write arbitrary file paths when run as root, and only
supports strings and recursive sets of strings. this was only used for
manpage generation in a build system that has not woken up since 1976.

fixes #974
fixes #227

Change-Id: I4f18599685a3077c15ddc02c759558f986c8c6e4
2025-08-23 10:39:30 +00:00
Maximilian Bosch 5dc847b47b libstore: exponential backoff for downloads
Closes #932

`connect-timeout` gets replaced by an exponential backoff for the
download timeout where the initial value is controlled by the setting
`initial-connect-timeout`.

Per iteration, the upper limit of the timeout is increased set to

    timeout := min(max_connect_timeout, initial_connect_timeout * 2^i)

I decided to move the entire timeout / tracking of attempts into its own
class to not make the filetransfer implementation more complex. Also,
that allows us to write unit-tests for it.

Setting `--download-attempts` to `0` is forbidden now and an exception
will be thrown. For `--offline` we set it to `1`, the behavior is
equivalent to what it was before: whether the max tries were exceeded is
only checked after the first download exception got thrown, i.e. there's
still one attempt being made.

The end-result - with timeouts being caused by a wrongly set proxy -
looks like this:

    $ env HTTPS_PROXY=1.1.1.1 nix store ping --store https://example.com
    warning: error: unable to download 'https://example.com/nix-cache-info': Connection timed out after 5006 milliseconds (curl error code=28); retrying in 422ms ms (attempt 1/5)
    warning: error: unable to download 'https://example.com/nix-cache-info': Connection timed out after 10010 milliseconds (curl error code=28); retrying in 1003ms ms (attempt 2/5)
    warning: error: unable to download 'https://example.com/nix-cache-info': Connection timed out after 20020 milliseconds (curl error code=28); retrying in 2018ms ms (attempt 3/5)
    warning: error: unable to download 'https://example.com/nix-cache-info': Connection timed out after 40007 milliseconds (curl error code=28); retrying in 4087ms ms (attempt 4/5)
    error: unable to download 'https://example.com/nix-cache-info': Connection timed out after 80074 milliseconds (curl error code=28)

Change-Id: I9e8d08d78275bcf60080d663febc9e075243d36b
2025-08-22 16:19:46 +02:00
Commentator2.0andCommentator2.0 7553d0a983 tests/functional2/lang: don't throw unused file errors on invalid configurations
Currenlty, when a test group is invalid already, we also throw unsued
file errors.
This leads to clutter as more often than not, the unused files are
caused by an invalid configuration, making the debug stack bigger
without reason.

With this commit the behavior is changed to only error about unused
files, when no other configuration issues were found

Change-Id: I92a819753f13b8ed5a07dae53ecaee5d84b5ce64
2025-08-22 07:50:20 +02:00
Commentator2.0andCommentator2.0 0a3e43590c tests/functional2: improve files ux
Currenlty one is required to always write the bulky `mark.parametrize`
with indirect and things

This commit adds a custom decorator for usage of files, which hides the
parametrization complexity from the user.

Change-Id: I526e016d12006669dc302dfc5af619735399c503
2025-08-22 07:50:20 +02:00
Raito Bezarius 7622d28dd4 Revert "libutil: extract Base32 helpers from Hash"
Revert submission 3850

Reason for revert: caused multiple regressions noticed in https://git.lix.systems/lix-project/lix/issues/975 and https://git.lix.systems/lix-project/lix/issues/966 (suspected).

Root cause analysis has not been done yet and this breaks Lix on Darwin on HEAD.

Reverted changes: /q/submissionid:3850

Change-Id: I2dae7147030c883a57be8a8c205e492e16425a23
2025-08-21 14:37:46 +00:00
Raito Bezarius c9cbbc4f63 Revert "tests: remove obsolete code to create custom build direc..."
Revert submission 3850

Reason for revert: caused multiple regressions noticed in https://git.lix.systems/lix-project/lix/issues/975 and https://git.lix.systems/lix-project/lix/issues/966 (suspected).

Root cause analysis has not been done yet and this breaks Lix on Darwin on HEAD.

Reverted changes: /q/submissionid:3850

Change-Id: I56f871786760eb9f54c02773f0617de9878c1382
2025-08-21 14:37:46 +00:00
Raito Bezarius 9cabe56fcd Revert "libstore: don’t include derivation names in build direct..."
Revert submission 3850

Reason for revert: caused multiple regressions noticed in https://git.lix.systems/lix-project/lix/issues/975 and https://git.lix.systems/lix-project/lix/issues/966 (suspected).

Root cause analysis has not been done yet and this breaks Lix on Darwin on HEAD.

Reverted changes: /q/submissionid:3850

Change-Id: Iee4c3f071238fa58e0e92f0bcc5584ded4a9d71f
2025-08-21 14:37:46 +00:00
Jade Lovelace 61955d0a40 libexpr: hyperlink attr names to their definition locations
Concept: what if you could, in your fancy terminal, in the year of our
lord 2025, just click on the attrs you're looking at to go to where
they're defined. Currently we only expose this info as
builtins.unsafeGetAttrPos, which is inconvenient as it's not
discoverable to users.

By putting it in this more visible yet invisible spot, it's more likely
to be more useful to more people.

In the current state, this is not the most useful ever due to stuff like
https://github.com/neovim/neovim/discussions/35097. However, it can be
expanded by perhaps adding something like the url format setting ripgrep
has.

Change-Id: I3947f97d5c2056d59099af468d7b855486438227
2025-08-20 20:55:54 +00:00
Commentator2.0 d8b1fb7799 tests/functional2/lang: replace toml with tomllib
currenlty we use the external package `toml`, this just adds an
unnessecary dependency, as python ships its own toml as `tomllib`

Change-Id: Ia63fa7558973e853ada20cbfa21d897d700444f8
2025-08-18 20:08:09 +02:00
Emily ce6dcf18d6 libstore: don’t include derivation names in build directories
They have variable size, which is bad for #913.

Change-Id: I6a6a6964a84804dee281ade48e6517419a6143fa
2025-08-18 09:27:31 +00:00
Emily 246f0eed82 tests: remove obsolete code to create custom build directories
We now do in fact do this and decide this.

Change-Id: I6a6a69643af2449370df001f8be787eba889b857
2025-08-18 09:27:31 +00:00
Emily 76baa4c50d libutil: extract Base32 helpers from Hash
Change-Id: I6a6a6964f95aecf152090a3bf82b5ec287a21481
2025-08-18 09:27:31 +00:00
eldritch horrors 8208c6ebb7 libexpr: don't read Value::string.s directly
Change-Id: I0f224459fcbff8bc53c3668bc6ea52881c453fd0
2025-08-17 12:11:19 +00:00
Emily 27f7075f4d libexpr: remove the parse-toml-timestamps experimental feature
See [my comment] on the Nix PR to restore the previous behaviour
for why I believe we should remove this for the next release. The PR
should still be backported to stable releases to avoid making breaking
changes to their semantics.

[my comment]: <https://github.com/NixOS/nix/pull/13741#issuecomment-3180851635>

Fixing this across supported Lix versions is required for Nixpkgs to
update toml11, which is a blocker for the CMake 4 update.

Change-Id: I6a6a69642e6b6cb13a9fccc0778e9158b53102d5
2025-08-16 14:27:13 +01:00
4de09b6b54 packaging: bump toml11 to 4.4.0
This version changes the handling of TOML timestamps, and throws an
error on out‐of‐range integer literals rather than the previous
saturating behaviour, as required by [the TOML v1.0.0 specification]:

> Arbitrary 64-bit signed integers (from −2^63 to 2^63−1) should be
> accepted and handled losslessly. If an integer cannot be represented
> losslessly, an error must be thrown.

[the TOML v1.0.0 specification]: <https://toml.io/en/v1.0.0#integer>

The only known use of this is a questionable Nixpkgs test that I have
proposed [a fix] for.

[a fix]: <https://github.com/NixOS/nixpkgs/pull/433710>

Bumping this ahead of Nixpkgs ensures we can test these cases on
HEAD in advance. I presume that the next Lix major version will be
released after 25.05 goes out of support, so it should be fine to
drop support for the old version of toml11.

The co‐authors of this commit are the contributors to the vendored
package definition from Nixpkgs.

Co-authored-by: Anderson Torres <torres.anderson.85@protonmail.com>
Co-authored-by: Artturin <Artturin@artturin.com>
Co-authored-by: Silvan Mosberger <silvan.mosberger@moduscreate.com>
Change-Id: I6a6a69644a188b6e09eee5c9cf91ddd3c81d24ee
2025-08-16 14:27:13 +01:00