Commit Graph
18365 Commits
Author SHA1 Message Date
Raito Bezarius 19e4f11e88 libexpr/symbol-table: efficiently generate ≤, <, >, ≥, ==, != for Symbol
C++ has the "spaceship" operator which can be auto-implemented and
generates efficiently a strong ordering.

Change-Id: Idfd1fd68039b395e54401cbe913454e0cbd80fb3
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-09-20 23:45:38 +02:00
Raito BezariusandSergei Zimmerman 057b725ae3 libexpr: use const references or pointers to attribute sets
Prior to this change, references or pointers could be mutated. In
practice, we do not require this capability in the codebase except in
zipAttrsWith.

This cleans up all easy sites in preparation to have a smarter
representation of attribute sets albeit one that requires constant
references.

Change-Id: I2be20cce040a9228bde9e5f7b42c0499fba9550b
Signed-off-by: Raito Bezarius <raito@lix.systems>
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2025-09-20 20:31:19 +02:00
Alois Wohlschlager 3ee839cc5e libutil,nix: fix clashes with stdio stream identifiers
The stdio stream identifiers (stdin, stdout, stderr) are allowed to be macros.
In musl libc they are, for example doing `#define stdout (stdout)`, breaking
compilation with an error when one of the clashing variables is attempted to be
initialized the "wrong" way:

    ../lix/libutil/processes.cc:272:7: error: expected class member or base class name
      272 |     , stdout(stdout ? std::make_unique<AsyncFdIoStream>(std::move(stdout)) : nullptr)
          |       ^
    /nix/store/ziw42d7rvgnf3vkbfc8kry07kipwf1xm-musl-static-x86_64-unknown-linux-musl-1.2.5-dev/include/stdio.h:67:16: note: expanded from macro 'stdout'
       67 | #define stdout (stdout)
          |                ^

Other places only cause warnings on musl:

    ../lix/libutil/processes.cc:254:17: warning: parentheses were disambiguated as redundant parentheses around declaration of variable named 'stdout' [-Wvexing-parse]
      254 |     std::string stdout;
          |                 ^~~~~~
    /nix/store/ziw42d7rvgnf3vkbfc8kry07kipwf1xm-musl-static-x86_64-unknown-linux-musl-1.2.5-dev/include/stdio.h:67:16: note: expanded from macro 'stdout'
       67 | #define stdout (stdout)
          |                ^~~~~~~~
    ../lix/libutil/processes.cc:254:17: note: add a variable name to declare a 'std::string' (aka 'basic_string<char>') initialized with 'stdout'
      254 |     std::string stdout;
          |                 ^
          |                  varname
    /nix/store/ziw42d7rvgnf3vkbfc8kry07kipwf1xm-musl-static-x86_64-unknown-linux-musl-1.2.5-dev/include/stdio.h:67:16: note: expanded from macro 'stdout'
       67 | #define stdout (stdout)
          |                ^
    ../lix/libutil/processes.cc:254:5: note: add enclosing parentheses to perform a function-style cast
      254 |     std::string stdout;
          |     ^                 
          |     (                 )
    ../lix/libutil/processes.cc:254:17: note: remove parentheses to silence this warning
      254 |     std::string stdout;
          |                 ^
    /nix/store/ziw42d7rvgnf3vkbfc8kry07kipwf1xm-musl-static-x86_64-unknown-linux-musl-1.2.5-dev/include/stdio.h:67:16: note: expanded from macro 'stdout'
       67 | #define stdout (stdout)
          |                ^

However they are still wrong, since the macro could be more complicated. Fix
them as well.

Change-Id: I6a6a6964a50ef7dec8f05f0bd8fc8f13f3036d51
2025-09-20 11:25:00 +02:00
Alois Wohlschlager a7e1b23165 libexpr: remove liblixexpr_mstatic
It was supposed to be removed in 480fdf146d, as
it is not needed any more with the prelinked library. Due to a mistake in
rebase conflict resolution it reappeared by accident. Actually remove it now.

Change-Id: I6a6a6964d175fdb0ba0ad9ac55d4d22d7b27ad3f
2025-09-20 08:30:35 +00:00
Commentator2.0 3a4f9488f0 tests/functional2: migrate nix_path.sh
Change-Id: Ibc4f848721d4da628c44aa074faf009c87af165d
2025-09-19 20:51:49 +02:00
Commentator2.0 fc2eedf9db tests/functional2: migrate why-depends.sh
Make builds work on darwin and migrate the first building test :D

Change-Id: Ia301c78d3bc771cd826d93dc114b098167043e97
2025-09-19 20:51:49 +02:00
Commentator2.0 a25a5739c7 tests/functional2: migrate search.sh
Change-Id: Id0f47fc72191d913442ff35d47df1b87ceb9f0be
2025-09-19 15:09:22 +02:00
piegames 2a622d7de7 tests/functional2/lang: Migrate value printing tests
Change-Id: I39303ef44552eb9fc17ccc2da5206c8906e46f27
2025-09-18 23:38:46 +02:00
piegames fe7a8d5e67 tests/functional2/lang: Migrate arithmetic tests
Change-Id: I73259875debc053dca46fe34f9f8d10b8486fa4f
2025-09-18 23:38:46 +02:00
piegames 096aa37cf1 tests/functional2/lang: Migrate ind-string tests
Change-Id: Ibc764ab97a53b751287654300211f3a9d8750b74
2025-09-18 23:38:46 +02:00
piegames 21a0b81a26 tests/functional2/lang: Migrate function args tests
Change-Id: Ia805d7fbf74c671fc419fb8c9db3960b6be62354
2025-09-18 23:38:46 +02:00
piegames f2b64142a9 tests/functional2/lang: Migrate dup-attrs tests
Change-Id: Ia5976229c8b2e6bf89d821b5676a9a397e4539d7
2025-09-18 23:38:46 +02:00
piegames 0d4a6132dc tests/functional2/lang: Migrate builtins.deepSeq test
Change-Id: I5caeb8a606d8cba519267c6011561afc9813598c
2025-09-18 23:38:46 +02:00
piegames 3692cf1ea2 tests/functional2/lang: Migrate inherit tests
Change-Id: I0f09f2b4ffa2edc5031351d0426865d7605ff7fa
2025-09-18 23:38:46 +02:00
piegames c7b8ac7cd9 tests/functional2/lang: Migrate substring tests
Change-Id: I902de6f3aebc902d56a79f8909bd6c9b5ba70cda
2025-09-18 23:38:46 +02:00
piegames 81baa64847 tests/functional2/lang: Migrate eval-okay-eq*
Change-Id: I17f81a765bbdbde3a27dd547e1ef65bffa97bce5
2025-09-18 23:38:46 +02:00
piegames 16db4da8f0 tests/functional2/lang: Rename tests
Most tests for builtins now have `builtins.builtinName` as their name.
This makes navigating the test list a bit easier

Change-Id: Ief5af5c568a419bf9130601f9590e7a696b0dc0a
2025-09-18 23:38:46 +02:00
piegames 5cfe785352 tests/functional2/lang: More trivial eval-okay test migrations
In the first pass I erred on the cautious side, only migrating safe
bets, to here's some trivial migrations that I missed

Change-Id: I934011919837b0aa491113afdcad603cf6b9cbbb
2025-09-18 23:38:46 +02:00
Jade Lovelace 0018b98859 codeowners: add some missing directories
Change-Id: I22aa53074dedd822517a5e8dfa2f0a520d0000bb
2025-09-17 16:26:49 +00:00
Alois Wohlschlager d63a31e7a9 Remove obsolete editline.patch
Its usage has been removed in 3936f9b5de but for
some reason the file remained. Drop it now.

Change-Id: I6a6a696420f3425587f82bd485c6307e1ccff08d
2025-09-17 06:09:44 +00:00
Commentator2.0 a9f28622c8 tests/functional2: migrate pure-eval.sh
Change-Id: Ieadb54a27b1cedcf3a4ce65e024a68440759ce7e
2025-09-16 20:06:07 +02:00
eldritch horrors d648ec55eb libutil: asyncify runProgram2 stdout stream
fun side effect: post-build hooks are no longer single-threaded!

Change-Id: Ie5c75e588e4b2b8d515d4f1ae36de45b0a0807f7
2025-09-16 16:10:59 +00:00
Maximilian BoschandJörg Thalheim d6b0b8b382 nix-eval-jobs: add --no-instantiate flag
Closes #987

The patch adds a flag `--no-instantiate` which only performs evaluation
without instantiating any derivations. Hence, GC root creation is also
skipped. To achieve that, Lix is also put in read-only mode and all
operations that require reading a derivation (e.g. constituents or
listing input derivations) are disabled fallback values are set.

This is a port of an upstream PR[1]. Given the divergence of the
codebases (different restructurings on both ends, no more CA derivations)
I decided to redo large portions from scratch instead of
cherry-picking the patches. Hence, the authorship.

Additionally the clean up of casts down to a local store are removed or
guarded behind an if, as done in the upstream PR.

[1] https://github.com/nix-community/nix-eval-jobs/pull/379

Co-authored-by: Jörg Thalheim <joerg@thalheim.io>
Change-Id: Ib84f44e7799bc5577fd2ee98912458f16ebeab81
2025-09-16 13:03:53 +02:00
eldritch horrors 812f466e0d libutil: asyncify runProgram
Change-Id: I397d02b1a993dc4f0c2dc2b2f90d0c1aebda4c2e
2025-09-15 16:18:01 +00:00
Marie Ramlow 8981fb5a18 meson: only require libatomic if the test program doesn't link
95448347 made lix require libatomic if the platform is able to link a
simple program using atomics, but it should actually be the other way
around. We need to require libatomic if it fails.

Change-Id: I6a6a6964ca6ee90a59314ddf1865753e83713772
2025-09-13 21:30:52 +02:00
eldritch horrors d90e4a6581 libutil: reject malformed nars harder with better errors
fixes #993

Change-Id: I56aff32498ecd32e65f1576661d5a15808a4439f
2025-09-12 23:22:11 +02:00
eldritch horrors 95d43d1e39 testing: unrace functional-repl
using sleep(1) as a synchronization mechanism does not work. use fifos instead.

fixes #690 and ci constantly falling over in the same exact fucking source line

Change-Id: I51725f8e439b6753f3212d2897dbb0620ad77a37
2025-09-12 23:22:11 +02:00
Sergei Zimmerman 88517c324f libexpr: slim down attrsets by 8 bytes on 64 bit systems
It's pretty bad that Bindings effectively wasted whole 8
bytes (4 for capacity and + 4 for alignment padding) to
store something it doesn't actually need. BindingsBuilder
allows the capacity to be checked at construction time,
after which the Bindings does not get mutated aside from
the ugly case of builtins, which doesn't get built all
at once.

For `nix search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello`
this shaves off around 53MB allocations out of 2GB used
for attrsets in total:

<     "bytes": 2001170768,
---
>     "bytes": 1947398072,

<     "Bindings": 16,
---
>     "Bindings": 8,

Nix PR: https://github.com/NixOS/nix/pull/13919

Change-Id: I939c5ac545f5abbca048370dcf4936346339d75c
2025-09-12 19:23:38 +03:00
eldritch horrors b966d2e53b libfetchers: asyncify some git/hg internals
Change-Id: I3aea82173a610dc9584e956aa74d3e3ad06bfe68
2025-09-12 11:52:03 +00:00
eldritch horrors 9b01455ada libfetchers: asyncify Input{,Scheme}::{putFile,clone}
Change-Id: I084b6fc2271ea99a9325b1969f18b83778702bea
2025-09-12 11:52:03 +00:00
eldritch horrors 078e1d7245 libstore: asyncify LocalStore::findPlatformRoots
Change-Id: I173fae8aa17cb757b9c2aceadb4d68259a236b32
2025-09-12 11:52:03 +00:00
Qyriad 138c7161be nix3-develop: correctly escape ASCII control characters for JSON
Fixes #991.

Change-Id: Iafc7d9603fbc3615393d32d9630f0e8fe548950b
2025-09-12 09:42:20 +00:00
Qyriad ee91eec5cf libutil: add bashEscape, which escapes non-printing characters with $''
Change-Id: I23b416372677acb80bae9c124e9fd6d867e7af6d
2025-09-12 09:42:20 +00:00
Jade LovelaceandRutile b5cf7dff7b package: fix running pytest in a dev shell
I have no idea how seemingly I am the only one hitting this, but either
way, this fixes it.

Fixes: https://git.lix.systems/lix-project/lix/issues/964
Workaround-For: https://github.com/NixOS/nixpkgs/pull/435749
Change-Id: If2a7ad89f98f0054928868eabb62b35c2df28e6e
2025-09-11 19:15:51 +00:00
Marie Ramlow 95448347ea meson: link against libatomic if required
Some platforms like 32-Bit PowerPC need linking against libatomic.
Try to compile and link a very simple snippet of code which uses atomics
and make libatomic required if it fails.
Because we're using `dependency('atomic')`, the required meson versions
gets bumped to 1.7.0. See https://mesonbuild.com/Dependencies.html#atomic-stdatomic

Change-Id: I6a6a696471e1d352fb161c537ba9023b97c2d31e
2025-09-11 17:58:25 +00:00
eldritch horrors b04419e719 libutil: asyncify streaming unpackTarFile
libarchive is not async and cannot be used async without involving green
threads, which have already proven to be very problematic. unpacking tar
archives is rare enough that spawning a new thread for each shouldn't be
too much overhead, and the additional data copy probably also won't hurt
performance too much. we may even benefit from being able to extract not
just one archive per event loop but as many archives as we can keep fed.

Change-Id: Iece82bd566ada0a2a49de54c4e69caf6d93f6720
2025-09-11 16:36:52 +00:00
eldritch horrors 6f0bf9798a libstore: process post-build-hook logs directly
using a sink for this has long been a bit weird anyway. originally it
was necessary due to api limitations, but it hasn't been for a while.

Change-Id: I3dfa157944618349bfd6f398ee1667fc31519d86
2025-09-11 16:36:52 +00:00
Alois Wohlschlager 91bf54d726 flake: update nixpkgs input
Without https://github.com/NixOS/nixpkgs/pull/434761 evaluation of the
`nixpkgsLibTests` will fail in CI with recent enough Lix, due to reliance on
the TOML integer saturation bug.

Reported-by: Sergei Zimmerman <sergei@zimmerman.foo>

Change-Id: I6a6a6964838009d2c525f67035f84072fdfad988
2025-09-07 16:50:37 +02:00
Jade Lovelace f4bdddf0fd tree-wide: log commands executed at --debug --quiet
Fixes: https://git.lix.systems/lix-project/lix/issues/973

Information about which commands were executed is really valuable to
debug Lix and is much more user relevant than the vast majority of the
e.g. build loop junk printed at debug level. Currently we have a *whole
lot* of call sites where we call execv* which should probably be cleaned
up, but that's future work.

I chose to print argv0 rather than the executable path if these differ,
since the code is shorter and since the command could be a fully
resolved symlink or so where argv0 is the actual command name being run.
However, it's not exactly *hard* to write std::ranges::drop_view(args,
1).

Change-Id: I73c3abb20b229d5e2d64277aa29cbbeed7764bab
2025-08-28 17:19:14 -07:00
Qyriad fe6395cd4d ci: add job to ensure static builds eval and configure
Static builds have broken and been fixed multiple times now[1][2]. It's
too expensive to add another "build Lix" job to CI, but a lot
of failures happen at eval time or during build configuration. We can at
least catch those.

[1]: https://git.lix.systems/lix-project/lix/commit/ac80a11300eb60006b7de90fb10ad6789e5beeed
[2]: https://git.lix.systems/lix-project/lix/commit/c82af241f58d7aeaecbb2a6d17bd8d7bfcc834ce

Change-Id: I5618f1ba7554324e60f4f510e70b1ccea6445f4e
2025-08-27 03:29:46 +00:00
eldritch horrors d5970d8a8b libutil: remove unused logger function
Change-Id: I8505346baa88b9f174707e127f1c78088238cf3e
2025-08-25 21:00:15 +00:00
eldritch horrors ed9a78fccd treewide: colorize logs by default
printTaggedWarning already colorized its messages. we can do the same
for most other log messages.

Change-Id: Idcd31bbf4f8d0d703395b0d2b7b9bc33264d969f
2025-08-25 21:00:15 +00:00
eldritch horrors 18285afa76 treewide: force log format strings to be literals
luckily none of these a format strings vulnerabilities because
boost::format is smart enough to throw an exception when given
fewer format string arguments than are requested by specifiers

Change-Id: I5fa78f0d1396263271f6e1dbcee9c0b2e9e18c34
2025-08-25 21:00:15 +00:00
eldritch horrors 466115c9c8 treewide: don't call Logger output functions directly
always use log macros, which also have the benefit of respecting the
verbosity setting without needing virtual function calls to read it.

Change-Id: I1c605562a53e54140724d5225e040abcf49ac996
2025-08-25 21:00:15 +00:00
eldritch horrors 4c6c01786f treewide: turn nix::warn into a macro
we add two variants: one that just prints a message at the warning
level (mirroring the other printer macros), and one that also adds
the colored "warning: " prefix the function added. since there are
no overriders of this function in tree it looks safe to remove it.

Change-Id: I7008fd0f31d59fbc9259472e29359c8df19ff87d
2025-08-25 21:00:15 +00:00
eldritch horrors 03ab20e191 libutil, cli: remove Logger::warn
only use the free function instead, so we can turn that into a macro.

Change-Id: I0319e9f7bdebb96f6159053e8b7b7a82559c9b33
2025-08-25 21:00:15 +00:00
eldritch horrors f40a60f46a libutil: make log macros usable outside of nix namespaces
mostly useful for nix-eval-jobs which currently has to call the logger
functions directly because its main code *isn't* in the nix namespace.

Change-Id: Ia8440d86a293d9006ffef2562b1859e9aaa79a62
2025-08-25 21:00:15 +00:00
Commentator2.0 9cdc2bf61c tests/functional2: migrate hash.sh
Change-Id: I5061479b972088c904ca1db4d9004e738c2b5a57
2025-08-25 21:29:27 +02:00
Alois Wohlschlager 451a14980b libstore: use OpenSSL for Ed25519 signatures
Previously two cryptography libraries were linked into Lix: OpenSSL used for
hashing and (in usual configurations) indirectly via curl for TLS, and Sodium
used only for handling the Ed25519 path info signatures. The latter is
functionally redundant since OpenSSL supports the same use case as well.
Reimplement the Ed25519 handling using OpenSSL and drop Sodium.

Fixes: https://git.lix.systems/lix-project/lix/issues/969

Change-Id: I6a6a696456b9d3ad7fdc2bf9b0759836a6247a38
2025-08-25 17:11:45 +00:00
Raito Bezarius 836644a7a1 libstore/build: put nix log command on its own line
This way, you can copy paste it easily.

Change-Id: Ie5799d1dea59f26cc0a1b3fde17d4449f49361c5
2025-08-25 09:51:07 +00:00