Commit Graph
2613 Commits
Author SHA1 Message Date
piegames 5f7e032d28 tests/functional2/lang: Migrate eol tests
Also slightly refactor the existing crlf test to better fit in

Change-Id: Ib4dfa4b3318b3dd55a12d99a9f4908cdf1bce08b
2025-11-17 22:10:11 +00:00
piegames b912f91f62 tests/functional2/lang: Migrate with tests
Change-Id: I3a6c2fb38569711d717f8488c7a949a4afbea675
2025-11-17 22:10:11 +00:00
piegames 1a5fddbf99 tests/functional2/lang: Merge tests
Change-Id: Ia62b1b558e67b18c1998207824271826b9a707db
2025-11-17 22:10:11 +00:00
piegames 59dd03bc39 tests/functional2/lang: Migrate undefined-var tests
The second test has also been fixed. I'd do it in two commits if
functional1 wasn't such a PITA (guess why we're doing the migration …).

Basically, the introduced syntax got changed later on, but the test
never got updated, and back then tests didn't test the output so nobody
ever noticed it now failing for a wrong reason.

Change-Id: I5b66127d507c65676b8dd4a82d1e1f2857fef145
2025-11-17 22:10:11 +00:00
piegames 108beda9ae tests/functional2/lang: Migrate another weird regression test
A link to an issue would have come really handy …

Change-Id: I44495a5e193c0bfbf9e905271025b7321daa84c6
2025-11-17 22:10:11 +00:00
piegames fef822494e tests/functional2/lang: Drop trivial "parse-okay-1"
This really looks like one of the first tests ever written, it doesn't
seem to be actively testing anything interesting

Change-Id: I442840db932203c25da2d8400ada791f8aed04ce
2025-11-17 22:10:11 +00:00
piegames d864159f34 tests/functional2/lang: Migrate url-literals test
Change-Id: I9096037b3e246dc6e63adcc8f327398bf14518a5
2025-11-17 22:10:11 +00:00
piegames 5bbd2946fa tests/functional2/lang: Migrate let tests
Change-Id: Ia3fc85dafbfd706d719f1766544f836df23ef7c9
2025-11-17 22:10:11 +00:00
piegames ad5d044b35 tests/functional2/lang: Migrate builtins.foldl'
Change-Id: I244751eee4881794f7a5b810e19e7d19f1257999
2025-11-17 22:10:11 +00:00
piegames 7bb79d0ae6 tests/functional2/lang: Migrate rec-set-overrides tests
Change-Id: I2ffc949773188d9b98629df87ac0d2fbf7b2ee6f
2025-11-17 22:10:11 +00:00
piegames fdfe21954f tests/functional2/lang: Migrate builtins.unsafeGetAttrPos tests
Change-Id: I6b1bd29bd03467715c985ae7ecb50c584ed48e11
2025-11-17 22:10:11 +00:00
piegames d0ef150349 tests/functional2/lang: Migrate builtins.hashFile tests
Change-Id: I2e1c1af59daf8c976877f5ec09153464e7480aea
2025-11-17 22:10:11 +00:00
adam fb4cc15f39 chore: update nixos channel URLs
Channels have moved from `nixos.org/channels` to `cnannels.nixos.org`.
This udates all relevant links (excluding release notes) to use the new
canonical URLs and replaces HTTP with HTTPS.

Fixes #1031.

Change-Id: I212821c44ac5e482c8e9eaa415c7d8ee17ff8341
Signed-off-by: adam <me@adamperkowski.dev>
2025-11-15 18:21:47 +00:00
EmilyandQyriad 82463616f9 libstore: skip the nested build directory on Darwin
This is unnecessary because Darwin builds already cannot create
`set{u,g}id` files due to the minimal sandbox policy, because we can
forbid messing with the top‐level build directory directly in the
sandbox rules, and because Darwin builds can trivially avoid isolation
through temporary directories right now anyway.

This does regress the build directory isolation for builds with
`sandbox = false`, but I can’t imagine that mattering given the
above. The sandbox change prepares us for a world where we close
off shared temporary directories for `sandbox = true` builds and try
harder to achieve proper isolation on Darwin, but probably doesn’t
have a meaningful security impact one way or another for now.

With this change, we get down to 41 byte build directory paths on
Darwin, a ~2–3½ byte improvement over the old status quo. We can
also restore the 0710 permissions on Linux.

Change-Id: I6a6a6964a681c0365241fe7234831db656b76799
2025-11-14 14:58:15 +01:00
EmilyandQyriad 13a37f1246 libstore: don’t include derivation names in build directories
They have variable size, which is bad for #913.

Change-Id: I6a6a6964870e984c66277c7556ff3c2bc34ddca1
2025-11-14 14:58:15 +01:00
EmilyandQyriad 1102cc180a tests: remove obsolete code to create custom build directories
We now do in fact do this and decide this.

Change-Id: I6a6a696493b288ed76d809122bda259dc0225846
2025-11-14 14:58:15 +01:00
EmilyandQyriad 9f3ba30783 libutil: extract Base32 helpers from Hash
base32Encode now takes std::span<std::byte>, with a base32EncodeStr
convenience wrapper which takes std::string_view.

Co-authored-by: Qyriad <qyriad@qyriad.me>

Change-Id: I6a6a6964f799dc84ecbfb55c7ca03a064cff71d9
2025-11-14 14:58:15 +01:00
eldritch horrors 2fa40c9de4 libstore: bring back some pointer equality nonsense
the change to shareable thunks also removed a few cases of pointer
equality checks that allowed structures containing functions to be
considered equal to other sets containing the same functions, even
if the sets themselves were pointer-equal themselves. *so* busted.

Change-Id: If87fdab658f9037ce2a654f69a9e3da6ae2f53e5
2025-11-14 02:56:53 +00:00
Qyriad 3ad1af8a98 libexpr: add trace for if 'foo' errors in 'or foo'
Change-Id: I59365e4c1bf4aed71c008554eca0c9e6d085219d
2025-11-07 14:54:02 +01:00
eldritch horrors 24054c1107 libutil: restore process context before subcommand exec
fixes #1028

Change-Id: Ic50b9cc0c65607cd96dd81fd770cda34b4caf9d5
2025-11-05 13:56:09 +00:00
eldritch horrors 0d24aee673 testing: drop compression from truncated nar test
some versions of zstd enable checksumming, some don't. don't use it.

fixes #1027

Change-Id: I06811072d9e56a682a792f084cf76a81c7b5b4ce
2025-11-05 00:02:10 +01:00
hexchen 57658b2146 tests/functional/daemon-trust: fallback to uid
The test would fail if the identity cannot be determined
(as is the case e.g. when using auto-allocate-uids under darwin).
Instead we use the uid when the user name is not available.

Fixes: #1025

Signed-off-by: hexchen <hexchen@lilwit.ch>
Change-Id: I12f35840b78f989b5fd09bbd44c833dc6a6a6964
2025-11-03 18:24:07 +01:00
Commentator2.0 acf6e1c6b7 tests/functional2/file_helper: add a way to interpolate environment variables into files
So far it was impossible to put absolute paths into files, as one
wouldn't know whre the test would be placed.
This commit adds a new Fileish variant called `EnvTemplate` which uses
the given string as a template and replaces the `@ENV_VARIABLE_NAME@`
placeholders with the according value of the environment variable.
This way one can use `@HOME@` or `@TEST_ROOT@` to build absolute paths

Change-Id: I425cb6408dceb8d7f26d136ace4ac98b1ca31ec3
2025-11-03 12:58:37 +00:00
Commentator2.0 9ed75192ec tests/functional2: fix crash on empty nix.nix([]) calls on darwin
Calling `nix.nix` with an empty argument list, would crash on darwin due
to a Index Out Of Range Error, as the build="auto" functionallity would
try to access the first element of the provided arguments (second
element at the place of call) resulting in a crash

Change-Id: Ia678109808b3f75a30182114baa421cedaa8759b
2025-10-30 18:53:11 +00:00
Raito Bezarius 95f15cf94f libstore/binary-cache: default to zstd for compression
libarchive's xz offers single threaded xz compression which is very slow
and provides ~10-20Mbps compression speed in addition to maxing a core.

In exchange, it achieves optimal compression ratios among all our
compression methods.

Nonetheless, xz prevent the saturation of 1Gbps+ connections and slow
down significantly decompression for end users. As these connections and
faster hardware is becoming prevalent for cache servers and clients, we
offer to default to zstd.

Lix is a "compress once, decompress many times" application. To avoid
incurring a high penalty to end users very sensitive to compress ratio
(very slow Internet connections), we dampen the consequences of
switching to zstd by increasing the default zstd level to 12.

On one example, xz will compress a 4.4GB file to 632MB, zstd on 12 will
compress it to 775MB, that is a ~18 % increase over the optimal xz
compression. zstd took 18 seconds to produce this file.

Increasing to level 14 leads to a 773MB file while taking 37s.
Increasing to level 16 leads to 735MB file while taking 66s.

Finally, xz took 77s, so a 50 % reduction in time taken to compress in
exchange of an increase of 18 % of the compressed size.

This change will reduce issues encountered in #945 but is probably not
the root cause.

References:

- https://discourse.nixos.org/t/switch-cache-nixos-org-to-zstd-to-fix-slow-nixos-updates-nix-downloads/23961

Change-Id: I7beda2bf2c1fed146dcb797b8f85dc290c486ab2
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-10-30 15:10:10 +01:00
Raito Bezariusandeldritch horrors 12b87538e3 lix/nix-shell: cleanup of shell directories happens in Lix itself
Instead of letting inner shell perform the cleanup, we will fork/execvp
and finish the work ourselves. Once an interruption arrives, we wait
again and return the exit status code of the inner shell.

This idea was suggested by eldritch horrors.

Fixes #1020.

Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: Id38af7ac33874aa1c4b7ca9c6123e3cbd8c3cd8c
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-10-29 20:13:23 +01:00
Qyriad 9523446901 libexpr/libutil: summarize derivations in stack traces regardless of --show-trace
Eval errors will now print a simple, no frills chain of involved
derivations at the bottom. For example, trying to evaluate
`pkgs.xonsh.override { python3 = pkgs.python2; }` has the usual

Package ‘python-2.7.18.8’ in /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/pkgs/development/interpreters/python/cpython/2.7/default.nix:398 is marked as insecure, refusing to evaluate.

message, but now also includes the following:

       note: trace involved the following derivations:
       derivation 'xonsh-0.19.9'
       derivation 'python2.7-xonsh-0.19.9'
       derivation 'python2.7-setuptools-44.0.0'

To give the user information about why the erroring derivation was
involved in the first place.

We would like more structured information in the future, but this should
still be a significant improvement.

Change-Id: Icf6da52abd0a043cfb63943bf0b0c160c21ee59e
2025-10-27 20:28:27 +01:00
Commentator2.0 73c019c57b tests/functional2/lang: allow adding requiring of global assets like config.nix
Change-Id: Ia740b91fb3d18f7d01c0d551196b71e7f3e46384
2025-10-27 17:07:47 +01:00
eldritch horrors 18efc848fe libstore: move curl-multi wrapper into own class
the wrapper is needed by transfer streams to restart a failed transfer
if desired. curlFileTransfer itself is more of a fancy handler for the
thread we're dedicating to curl io handling. the thread will stay with
the multi handle for now because quit handling needs to stay there. we
could have CurlMulti keep only a flag, but that does not help us much.

Change-Id: I99550f0bbb635b75898ca7260f08275df86050e3
2025-10-23 22:52:09 +00:00
Alois Wohlschlager 7a0b8834a4 Reapply "functional2: fix in the development shell"
This reverts commit 3ef7e0711b.

Change-Id: I5b76d2c2e9b0dd669818c7218493d3e76a6a6964
2025-10-22 07:41:46 +02:00
Alois Wohlschlager 0e0853cc70 functional2: handle unset or empty BUILD_TEST_SHELL properly
If BUILD_TEST_SHELL is not set or empty, None or the empty string respectively
should not make it into the PATH. Ensure this property.

Change-Id: I4ce9b0c06c407b465308b63b9cb64e7d6a6a6964
2025-10-22 07:41:46 +02:00
Commentator2.0 2541db5321 tests/functional2/lang: fix expected skip message
it seems like pytest is updating the message printed when skipping a set
due to no parametrization
which leads to breakage on staging-next.
this commit changes the expected message to allow both old and new skip
message

Change-Id: I34d11ad84d7075911d8229fbc89889bb880a27c9
2025-10-18 17:19:01 +02:00
Commentator2.0 0ac71ea1cc tests/functional2/lang: fix wrong unused file error for generic test group collection
Change-Id: I1c548fcebe45d2a6621987091e3a852b8a9b5765
2025-10-18 11:24:00 +00:00
Commentator2.0 4fcfdc66fe tests/functional2: migrate debugger.sh
Change-Id: I82c5b739db09530ff216c91608be2f70a2036e11
2025-10-18 11:23:43 +00:00
EmilyandQyriad d9eead0eac tests: add fetchTarball test for symlink temporary directories
Change-Id: I6a6a6964dfe85b5ee4f66b8c442fe448c7c5c87c
2025-10-17 18:41:58 +00:00
eldritch horrors 20ba2b49fb testing: mock ssh during nix-copy prompt test
we don't need full ssh, we only need to know something makes it onto the terminal.

Change-Id: Ic31808e10e1b876b35d0dbdeee0b5ede7b5b9883
2025-10-17 15:13:37 +02:00
Jade LovelaceandCommentator2.0 faa0b4f291 functional2/http_server: add some example misbehaviour routes
Part of https://git.lix.systems/lix-project/lix/issues/949.

This also fixes a bug where our SO_REUSEADDR wasn't done before bind was
called and thus didn't work as intended.

Co-authored-by: Commentator2.0 <lix@crystal-cavern.systems>

Change-Id: Ie69fd594a32d7a6f538fb66d9d2cac6fe117d563
2025-10-13 18:30:51 +02:00
Jade LovelaceandCommentator2.0 556012e409 functional2: forbid chdir and setting environment
These are a footgun and are not acceptable in functional2 due to thread
safety, effects on other tests, etc.

Co-authored-by: Commentator2.0 <lix@crystal-cavern.systems>

Change-Id: I8d7285061eaa9bab27edd52f3646024c8cf605e5
2025-10-13 18:30:51 +02:00
Commentator2.0 844feb17b5 tests/functional2/lang: fix duplicate collection when both err and out are provided
When providing both a `.err.exp` and a `.out.exp` for a lang test
**not** containing a toml, the test was collected twice and hence threw
a duplicate id error.
This commit adresses this issue

Change-Id: Ia781d69e0f1db2809a30192d6cd0a98861e6cc32
2025-10-13 14:33:25 +00:00
eldritch horrors 1210a4e025 libstore: flush log lines only from handleBuilderOutput
also change the tests to ensure that we handle all lines correctly.

Change-Id: I13de286bda8eee57acd53af1ab5c081d3048b500
2025-10-13 11:26:49 +00:00
eldritch horrors f29dfb3d3c libutil: add buffer state management to loggers
currently all loggers can always accept messages and never suggest
flushing buffers. in the future this may change, and at that point
we're already fully set up for it. local loggers should never keep
asynchronous (i.e. network-backed) buffers, disk buffers are fine.
networked loggers will require buffers and periodic flushes later.

Change-Id: Ide2114f5bc17f4a1d289c92ed4f9381a1d59dacf
2025-10-13 11:26:49 +00:00
eldritch horrors b73a7f1815 libutil: clean up the Activity interface a bit
create activities from loggers themselves instead of passing the logger
as a constructor argument and allow direct construction of children, no
direct logger access needed. most call sites are not changed because we
still need to handle the "no parent" case, and the logger method can do
that more cleanly than a ternary at each site that creates an activity.
we may eventually want to create a root activity, which is cleaner too.

Change-Id: I295e056228dabb08a1316eba7973874784baa113
2025-10-12 22:37:31 +00:00
eldritch horrors b88a6e6f11 libexpr: use pascal strings for eval
this has no performance impact in any benchmarks we've run. nul bytes
are still used as implicit truncation points in many places all over:
rejecting them in all locations that treat them as a string end point
requires large changes such as using a proper path library everywhere

Change-Id: I936158bd435f6abf009a689adfbc24496262c578
2025-10-11 12:57:57 +02:00
eldritch horrors 53d172a308 libutil: remove unused CanonPath methods
Change-Id: Id593e0d03373af88e03a9da473935e5c00f86b33
2025-10-10 16:55:34 +00:00
eldritch horrors 2f16cf6014 treewide: lint syscalls for c string safety
Change-Id: I3caf476e59dcb7899ac5a3d83dfa3fb7ceaaabf0
2025-10-10 16:13:54 +00:00
eldritch horrors 3ef7e0711b Revert "functional2: fix in the development shell"
This reverts commit 42691f0d94.

Reason for revert: darwin builds time out now, apparently due to newly appearing infinite loops in the test suite

Change-Id: I81a09efd53636a287f3095ac87ed8854b6e57922
2025-10-10 12:49:45 +00:00
Commentator2.0 5e2412ea7e tests/functional2: migrate add.sh
Change-Id: I699929b61db19a4ab7bea8281df830de9fea6d04
2025-10-08 13:09:17 +02:00
Alois Wohlschlager 42691f0d94 functional2: fix in the development shell
The functional2 test suite was broken in the development shell for two mostly
separate reasons leading to "no such file or directory" errors:

* The `BUILD_TEST_SHELL` (already containing the correct path) would always be
  set from the Meson option `build_test_shell`, even to the empty string if
  that option is not set. Skip the overwrite in this case to make the
  environment variable work again.
* Sandboxed builds would fail to find their builder `/bin/sh`, since in the
  development shell busybox is not found, so no sandbox shell gets configured.
  Use the shell from `BUILD_TEST_SHELL` instead. (How this does not break the
  old functional tests remains a mystery.)

Change-Id: I6a6a696424e8caaef3f9b68e3738bfd58ea0b056
2025-10-07 20:14:48 +00:00
eldritch horrors 80654b84b6 libexpr: de-ptr-ize Value references
thunk values are shareable, and we can represent invalid/uninitialized
values with a special bit pattern that makes no sense otherwise. there
is no need to keep allocating values on the heap, instead we can treat
values like reference-counted smart pointers to heap objects, which in
turn lets us save a lot of allocations and, ultimately, gc heap space.

compared to our baseline (main of 2025-09-27) we save 15%+ memory on a
system rebuild and 17% on nix search. eval time regresses by ~3% for a
system rebuild, while nix search is 7% faster. further optimization is
probably possible (but for now this will just have to be good enough).

Change-Id: Ib6c47acdbe2fac4f76a83c2269f16f30ef66b2e1
2025-10-05 16:23:05 +02:00
eldritch horrors 28c4431c69 libexpr: reformat stuff we'll change soon
mainly to keep the next diff smaller. it'll be large enough as is.

Change-Id: Ib8a34520f03539cbf6aa2f0e66cbed05fe1225eb
2025-10-05 16:22:35 +02:00