guiwf/lix
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: guiwf/lix:2.95.0
Branches Tags
guiwf/lix:main guiwf/lix:adaptive-routing guiwf/lix:release-2.95 guiwf/lix:ci-config guiwf/lix:release-2.93 guiwf/lix:release-2.94 guiwf/lix:sb/jade/coverage guiwf/lix:sb/raitobezarius/cgroups-observability guiwf/lix:sb/jade/wip-tars guiwf/lix:sb/raito/flakes-out-of-core guiwf/lix:sb/raito/eval-explorations guiwf/lix:sb/pennae/ci-config guiwf/lix:release-2.91 guiwf/lix:release-2.92 guiwf/lix:sb/raito/darwin-kqueue-disablement guiwf/lix:sb/raito/hydra-merge guiwf/lix:sb/raito/phantom-referrers-gc guiwf/lix:repl-overlay-example guiwf/lix:reuse-compliance guiwf/lix:release-2.90 guiwf/lix:sb/pennae/io-rewrite guiwf/lix:git-series/activity-cleanup guiwf/lix:sb/tom-hubrecht/primops.cc-v2 guiwf/lix:sb/bacchanalia/cache-strlen%topic=substr-perf guiwf/lix:sb/qyriad/beta-1 guiwf/lix:sb/qyriad/2.90-beta.0 guiwf/lix:sb/rbt/test-branch guiwf/lix:sb/rbt/justfile guiwf/lix:sb/rbt/repl-overlays guiwf/lix:sb/qyriad/maint-meson guiwf/lix:sb/qyriad/maint/meson guiwf/lix:sb/puck/meson-c-api guiwf/lix:sb/jade/burn-it-all guiwf/lix:dependabot/github_actions/zeebe-io/backport-action-2.4.1 guiwf/lix:dependabot/github_actions/cachix/install-nix-action-25 guiwf/lix:dependabot/github_actions/actions/labeler-5 guiwf/lix:dependabot/github_actions/cachix/cachix-action-14
guiwf/lix:2.95.3 guiwf/lix:2.93.4 guiwf/lix:2.94.2 guiwf/lix:2.95.2 guiwf/lix:2.95.1 guiwf/lix:2.94.1 guiwf/lix:2.95.0 guiwf/lix:2.94.0 guiwf/lix:2.93.3 guiwf/lix:2.92.3 guiwf/lix:2.93.2 guiwf/lix:2.91.3 guiwf/lix:2.92.2 guiwf/lix:2.91.2 guiwf/lix:2.93.1 guiwf/lix:2.93.0 guiwf/lix:2.92.0 guiwf/lix:2.91.1 guiwf/lix:2.91.0 guiwf/lix:2.90.0 guiwf/lix:2.90.0-rc1 guiwf/lix:2.90-beta.1 guiwf/lix:2.90-beta.0 guiwf/lix:1.11.1 guiwf/lix:1.11 guiwf/lix:1.10 guiwf/lix:1.9 guiwf/lix:1.8 guiwf/lix:1.7 guiwf/lix:1.6.1 guiwf/lix:1.6 guiwf/lix:1.5.3 guiwf/lix:1.5.2 guiwf/lix:1.5.1 guiwf/lix:1.5 guiwf/lix:1.4 guiwf/lix:1.3 guiwf/lix:1.2 guiwf/lix:1.1 guiwf/lix:1.0
...
compare: guiwf/lix:main
Branches Tags
guiwf/lix:adaptive-routing guiwf/lix:main guiwf/lix:release-2.95 guiwf/lix:ci-config guiwf/lix:release-2.93 guiwf/lix:release-2.94 guiwf/lix:sb/jade/coverage guiwf/lix:sb/raitobezarius/cgroups-observability guiwf/lix:sb/jade/wip-tars guiwf/lix:sb/raito/flakes-out-of-core guiwf/lix:sb/raito/eval-explorations guiwf/lix:sb/pennae/ci-config guiwf/lix:release-2.91 guiwf/lix:release-2.92 guiwf/lix:sb/raito/darwin-kqueue-disablement guiwf/lix:sb/raito/hydra-merge guiwf/lix:sb/raito/phantom-referrers-gc guiwf/lix:repl-overlay-example guiwf/lix:reuse-compliance guiwf/lix:release-2.90 guiwf/lix:sb/pennae/io-rewrite guiwf/lix:git-series/activity-cleanup guiwf/lix:sb/tom-hubrecht/primops.cc-v2 guiwf/lix:sb/bacchanalia/cache-strlen%topic=substr-perf guiwf/lix:sb/qyriad/beta-1 guiwf/lix:sb/qyriad/2.90-beta.0 guiwf/lix:sb/rbt/test-branch guiwf/lix:sb/rbt/justfile guiwf/lix:sb/rbt/repl-overlays guiwf/lix:sb/qyriad/maint-meson guiwf/lix:sb/qyriad/maint/meson guiwf/lix:sb/puck/meson-c-api guiwf/lix:sb/jade/burn-it-all guiwf/lix:dependabot/github_actions/zeebe-io/backport-action-2.4.1 guiwf/lix:dependabot/github_actions/cachix/install-nix-action-25 guiwf/lix:dependabot/github_actions/actions/labeler-5 guiwf/lix:dependabot/github_actions/cachix/cachix-action-14
guiwf/lix:2.95.3 guiwf/lix:2.93.4 guiwf/lix:2.94.2 guiwf/lix:2.95.2 guiwf/lix:2.95.1 guiwf/lix:2.94.1 guiwf/lix:2.95.0 guiwf/lix:2.94.0 guiwf/lix:2.93.3 guiwf/lix:2.92.3 guiwf/lix:2.93.2 guiwf/lix:2.91.3 guiwf/lix:2.92.2 guiwf/lix:2.91.2 guiwf/lix:2.93.1 guiwf/lix:2.93.0 guiwf/lix:2.92.0 guiwf/lix:2.91.1 guiwf/lix:2.91.0 guiwf/lix:2.90.0 guiwf/lix:2.90.0-rc1 guiwf/lix:2.90-beta.1 guiwf/lix:2.90-beta.0 guiwf/lix:1.11.1 guiwf/lix:1.11 guiwf/lix:1.10 guiwf/lix:1.9 guiwf/lix:1.8 guiwf/lix:1.7 guiwf/lix:1.6.1 guiwf/lix:1.6 guiwf/lix:1.5.3 guiwf/lix:1.5.2 guiwf/lix:1.5.1 guiwf/lix:1.5 guiwf/lix:1.4 guiwf/lix:1.3 guiwf/lix:1.2 guiwf/lix:1.1 guiwf/lix:1.0
271 Commits
2.95.0 ... main
Author SHA1 Message Date
eldritch horrors 1e48ac50ca meson: fix clang-tidy 
meson fixed the bugs that prevented using the sanitizer args, but using
such a fixed meson will provide broken flags to the subprojects. *sigh*

Change-Id: I0ab54351b19a3879e9a97daed1621dc5f730d376
 2026-07-18 00:22:19 +02:00
rootile 3dd53ecdef tests: migrate fetchers.sh 
Change-Id: I3c2856dad2e06993717bcba437c5a0b46e74821f
 2026-07-17 21:06:08 +00:00
eldritch horrors 289fab6a68 libstore: remove ssh{,-ng}:// connection pooling 
it's used very rarely, on ssh it's pretty much broken, and on ssh-ng
it's standing squarely in the way of the rpc transition. we have not
found significant use of this feature in public configs (with all of
*two* repositories using this store parameter), so why even keep it?

Change-Id: Ifcafdf794d815001a1a55a771e5823010e5b174b
 2026-07-17 19:46:00 +00:00
eldritch horrors f5b9f105e2 libstore: drop Store::connect 
there's no reason a store should not open a connection during init if it
needs to open any connection to function. delaying connection setup like
this makes openStore less deterministic and graceful fallback impossible

this caused test failures where the old behaviour was required to ensure
test output stability. because of course something like that must happen

Change-Id: I946adddee026f1f4c74b4699d730b4d1ac9a5072
 2026-07-17 18:00:14 +00:00
eldritch horrors 57fb4f534d libstore: handle ssh{,-ng}:// logs in their store class 
build-remote should not be the only thing that ever reads logs from
remote store connections. ssh-ng is better off here because it does
not transport its logs on the ssh stderr channel, but ssh *does* do
this. neither have any ssh logs shown after connection setup except
when using an ssh:// url as a remote builder, which is not markedly
helpful for debugging anything at all. let's have both kinds of ssh
store classes handle logs themselves to alleviate all of that. this
also improves the utility of remote builder checks in `nix doctor`.

Change-Id: I23947a9666bc07561eac341578ed45993f1b9839
 2026-07-17 18:00:14 +00:00
eldritch horrors 553ae61ffd libstore: fix AddBuildLog denial of service 
adding logs to a daemon store can fail before the entire request has
been read, leaving non-command data in socket buffers. if the daemon
does not terminate immediately after the failure it will try to read
the remaining data as commands, which is very likely to not work and
cause unbounded memory allocation instead. this doesn't give clients
an attack vector they did not have before though, they could've just
as well sent malformed commands without a bad preceding AddBuildLog.

Change-Id: I86c22890eace19164d932cbd342ac9f52cee6531
 2026-07-17 16:48:28 +00:00
rootile 33d0984e27 justfile/clean: also run cargo clean 
Change-Id: I069a106ced8f164f1aaff48a9a9211668cbe012b
 2026-07-17 15:42:05 +00:00
eldritch horrors 93ca179023 f2: fix nix fixture store setup 
log dir and state dir are not taken from the environment if root is set.

Change-Id: I3752dcf7ac81b291e61beafbb3fed9601e3cfcd1
 2026-07-17 13:33:19 +00:00
rootileandeldritch horrors ece46d54c7 lix-rs: link cpp also when building tests 
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I656a2f1b6999395c421f9a5b301b324c77f8ac52
 2026-07-17 10:27:39 +00:00
eldritch horrorsandRutile 4c4d8b8ccd lix-rs: enable iterators 
we don't support range-based iteration yet because it's a huge hassle to
make c++ and rust iteration styles meet in any reasonable way. this is a
good start to make rust collections *actually* usable though, so here we
go. `into_iter` is not explicitly supported, but can be used regardless.

Change-Id: Iebb76c9409059bc81623e5be81ac2656b1ec5138
 2026-07-17 09:56:35 +00:00
blokyk f91bdc9367 libutil: always print addErrorContext frames 
frames manually added with `addErrorContext` are generally a lot more
useful/informative to the average user than other frames, esp. in
the module system, which can create much better error messages than
we can.

however, before this change, frames from addErrorContext were truncated
by default if they weren't in the first 3 frames, so they were basically
useless (with `--show-trace`, you're dredging through 250 frames of
module shenanigans just to spot one singular line).

this change also removes the frames for _the call to_ `addErrorContext`,
which is just pure noise.

the way this change is done hopefully leaves a bit of space for future
similar changes to error printing, by introducing a new `TraceKind` enum
that can be used to categorize traces (i haven't done that in this CL
because that would be a pretty herculean task, given all the calls to
`BaseError::addTrace` in the codebase, and we probably want to be
careful about what categories we choose). the actual printing code could
definitely be improved tho... (e.g. by iterating twice through the trace
stack instead to first pick out the most important traces and _then_
printing less "important" traces if there's space left)

Change-Id: I52acc52f231991a9f2309d9cecae362397c6888c
 2026-07-16 15:20:59 +00:00
rootile 25d47498bd cli/doctor: fix crash when having relative paths in PATH 
fixes #1244

Change-Id: Ifefab62dcedaea4bccd1546bf63e2081fc040ce1
 2026-07-16 13:04:58 +00:00
rootile 99c6ad45b5 cli/doctor: check machines toml and remote stores 
fixes #1076

Change-Id: I37e904f3b83b48618a09cc119067133775cf01b6
 2026-07-16 13:04:10 +00:00
eldritch horrorsandpiegames 0e180a479f libutil/rpc-types: Add map type 
Co-authored-by: piegames <git@piegames.de>
Change-Id: I4215efb0e9839925d34ad45d47879148dbdfc4c2
 2026-07-16 12:09:24 +00:00
eldritch horrorsandpiegames 7307475218 libutil/rpc-types: Add Option type 
Co-authored-by: piegames <git@piegames.de>
Change-Id: I75d5db5f567a3a86870ed2aceb36840ddcbb53db
 2026-07-16 12:09:24 +00:00
eldritch horrors 66090810f1 libutil: add async input-consumer-to-output-stream adapter 
this makes using the zero-copy pipes a lot more ergonomic. error
handling is unfortunately not trivial and duplicating it for all
pipe users would thus be a bad idea. we're also not oblivious to
the fact that this is a `sourceToSink`, but it's async this time
around. (at least we don't need terrible stackful coroutines..?)

Change-Id: I1ba59f27183988ad68e7f88d102935d005690f43
 2026-07-16 12:09:24 +00:00
eldritch horrors 9bd9947f41 libutil: add a zero-copy async pipe 
rpc will often need to transfer data received on push-bashed interfaces
to consumers that expect input stream sources, which are pull-based. we
want to avoid copies for performance reasons (since kj overhead as kind
of on the high side for us already), so we'll use a shared-buffer class
that behaves much like a mutex. we *don't* use mutexes because ours are
cross-thread-capable and thus require syscalls for all wakeups they do.

Change-Id: I3b14925f5d9f5e07ea2cafdf00a88a64f79e4742
 2026-07-16 12:09:24 +00:00
rootile 25f53b755c tests: migrate config.sh 
Change-Id: Ia37a84e0036dd52af112a359817c709a15aa8c90
 2026-07-15 10:53:10 +00:00
Jade Lovelace f51148a75b tidy: delete a bunch of dead code 
Found ... by edef's harness. There was more dead code it found, which I
didn't address because it felt less obviously deletable:

- `AutoDestroyCgroup::path()` / `::delegation()`
- `TeeSource`, `LambdaSink`
- `S3BinaryCacheStore::getS3Stats()` (maybe Hydra? idk)
- `fetchers::getBoolAttr()` (technically deletable but it would be
  inconsistent with the others)

Change-Id: I913d2afd61e24136fb506389fc33a1016a6a6964
 2026-07-14 20:17:11 +00:00
eldritch horrors e6ddca402b libutil: make blanket rpc converters more useful 
having from() work on primitive types lets us specialize less elsewhere.

Change-Id: Ia6eb72ab03c264c6ba3149cb8fe21fa5b3ab4b4d
 2026-07-14 17:47:07 +00:00
piegamesandeldritch horrors c739701077 rpc.hh: Document footgun in LIX_TRY_AWAIT_RPC 
*piegames is tired of C++*

Change-Id: I9d6dd935afaadf809d5a779fbcc64f48b4af1cba
 2026-07-14 17:47:07 +00:00
eldritch horrors 734cfd1d48 libutil/rpc: add more filler macros for rpc conversions 
sometimes we need to disambiguate based on the kind of the thing we are
filling due to how the c++ type works. in templates some constraints on
type resolution are lifted. sometimes we can make the compiler help us.

Co-authored-by: piegames <git@piegames.de>

Change-Id: Id9549764813a862b8d64959157fed258709eb6ac
 2026-07-14 17:47:07 +00:00
Jade Lovelaceandrootile fca58aa044 main: don't insert null function pointers into the legacy commands map 
This is purely aesthetically bothering me, but [] inserting stuff into
the map is such a C++ coded design decision and we don't intend to
mutate it here so we probably should not do it.

Change-Id: Idf37c51175b031fb396e8fd53dc8fe6f2039fb55
 2026-07-13 19:51:09 +02:00
Raito Bezariusandrootile 782d967342 nix/doctor: check nixpkgs provenance 
Many issues stems from `<nixpkgs>` not resolving anywhere or having the
wrong version, let's make it awfully obvious again here.

Towards #230.

Change-Id: If630e9616566e77143f028cdcfe0b7f00a1d486d
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2026-07-13 16:47:18 +02:00
rootile 08f18285ef f2/nix: add fake nixpkgs fixture 
Change-Id: I76fe321bd93d9f79b4b1d98e529c50581d23d3f7
 2026-07-13 16:46:01 +02:00
eldritch horrors c6d22874d6 f2: increase timeouts and max worker count 
without maxprocesses we can utilize core-rich systems better, and test
timeouts are reportedly to be too low to run on small systems as well.

fixes #890

Change-Id: I89386b89fcd69ef4bf77ecf0a49c0b85e1f17c3d
 2026-07-12 20:31:05 +02:00
eldritch horrors 1657d0eb47 f2: add mark for nix settings defaults and use it 
this fixes a large portion of tests currently marked no_daemon. most of
them only needed to set some trusted settings, which is easily done now

Change-Id: Id5a5ee94951cdc92bddd2264c738ca4f98980c8b
 2026-07-12 20:28:05 +02:00
Maximilian Bosch 527ecba696 release.nix: mock rev/lastModified attrs when nixpkgs is a path 
Some code-paths expect e.g. `nixpkgs.rev` to exist.
This change makes sure this always exists. This is done on purpose
within this let-in dance rather than in the default such that
`nix-build release.nix --arg nixpkgs ../nixpkgs` also benefits from this
fix.

Change-Id: I94f1f901823d3df433e3c44ae380bb2f67eef70e
 2026-07-11 21:18:59 +02:00
Maximilian Bosch 72f8000b8f flake: update nixpkgs, fix eval 
attrpathsSuperset got renamed to preEval in nixpkgs[1] breaking Hydra's
evaluation. Updating nixpkgs to make sure this is consistent now, no
matter if flake-inputs or release.nix is being used.

[1] See commit 19a31658dc39324c9acf8d81198cd7137bdc1e92.

Change-Id: I69ab3f894534b1d89b626ddd4feda8ea502b69b3
 2026-07-11 21:18:59 +02:00
eldritch horrors 8294860ccf packaging: set CARGO_HOME 
single-user builds have a writable home directory in the build sandbox,
and the test suite does not like cargo writing anything into there. the
impact of this seems to be localized to f1 (at least on linux), but not
letting cargo write into the homedir is easy enough to do. nixpkgs also
does something much like this when compiling rustc, so we're not alone.
(it also seems that nixpkgs intended to do this for all rustc users but
broke it at some point in the past. some packages set CARGO_HOME in the
same way we do, possibly because they've had exactly the same problems)

fixes #1251

Change-Id: Ie17a0677ef09bb076ef6295bb590a0895924434e
 2026-07-11 12:11:49 +00:00
rootile 3b286bbd62 fetchers/git: cleanup code 
Change-Id: I643ea90f53b4e57980bed45b33837950f5c1e02e
 2026-07-10 18:06:26 +00:00
rootile 28fc206387 rust: add rootcause as error library 
Change-Id: I65460dda21331d79f0dfadfe06a7d6738c9ddefe
 2026-07-10 19:47:06 +02:00
eldritch horrors 64c486e22c libutil/libmain: never mask SIGWINCH 
we need to cache the current terminal size for progress bar reasons, but
we don't want to catch SIGWINCH to update the current terminal size from
a dedicated thread for repl reasons. a SIGWINCH handler function is much
easier to square with these requirements than communication with another
thread to have it change its signal mask, and since any races in handler
code affect only progress bar output and only very rarely (if ever) we'd
better chose the simplest approach. the progress bar could set a handler
of its own for this purpose, but we would much rather replace it instead

fixes #1246

Change-Id: I814d9aaf1b6fbb6a8cefc5af675a3aa372549dc8
 2026-07-10 16:28:11 +00:00
Qyriad 1a2853bf7b package: restore original curl after cl/5842 
Change-Id: I09b5cdd1a91ef471233cfcaf1c7ec1d16a6a6964
 2026-07-09 15:09:26 +00:00
QyriadandSergei Zimmerman 81084c7fc9 libstore: Fix libcurl thread wakeup with curl >= 8.21 
Since https://github.com/curl/curl/commit/2a2104f3cff44bb28bb570a093be52bbeeed8f23
libcurl sometimes just swallows events in curl_multi_perform, we entirely miss them,
and curl_multi_poll blocks forever.

Fixes #1234.

Based on https://github.com/NixOS/nix/pull/16090.

Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
Change-Id: I453dce192de0ea9e92bfb0114020dc9d6a6a6964
 2026-07-09 15:09:26 +00:00
rootileandeldritch horrors 41bad096e3 rust: migrate libutil/git.cc 
Change-Id: Ic3932fdf3afd3dc09019623516597f1c648f7fc4
 2026-07-08 20:03:10 +00:00
eldritch horrors dd8f1843cc lix-rs: allow exposing ord/eq operators from rust 
this is a bit of a hack, but since zngur cannot handle multiple trait
implementations per type yet we will have to commit to singles types.

Change-Id: I60e7b96bbeaa9fb87cf43662d4a9a5d44116bf47
 2026-07-08 18:04:40 +00:00
Linus Heckemann 9302845601 package: use buildPackages for licxxbridge build 
This fixes a regression in cross builds introduced in https://gerrit.lix.systems/c/lix/+/5704.

Change-Id: I3031d7c4daa5554cc16f654d28311f486a6a6964
 2026-07-07 10:59:53 +02:00
rootile 523df65fba tests: migrate misc.sh 
note: only two of the tests actually needed migrating, as the others
were already covered (e.g. by lang)

Change-Id: I30abb2bd728ae2b144ad76566096a216aed3b8f8
 2026-07-06 12:39:47 +00:00
rootile de67d1424d bench: fix outdated nixpkgs reference 
Change-Id: I293f4e4c399c192b0fec30aa8554b64641eee0bf
 2026-07-05 13:21:25 +00:00
isabelMic92Yannik Sander
 		405ff2eb03 nix-eval-jobs: --apply 
Based off of https://github.com/NixOS/nix-eval-jobs/pull/332 and
9f292a402a6edf620b872bedaaad5f0a45c23826

closes https://git.lix.systems/lix-project/lix/issues/1214

Co-authored-by: Mic92 <joerg@thalheim.io>
Co-authored-by: Yannik Sander <yannik@floxdev.com>
Change-Id: Ifba7081b384d18fbdc47437de261d30b65a7930a
 2026-07-04 14:46:08 +00:00
Seth FlynnandBernardo Meurer Costa c1370bc81c build: use mimalloc 
mimalloc is a compact general purpose allocator from Microsoft. It
consistently outperforms glibc's `malloc()` in allocation-heavy
workloads, such as Lix's evaluator

It's currently only linked in the main `nix` executable, as Boehm's GC
uses its own allocator. Other allocations that *do* go through glibc's
`malloc()` are still much faster, though

Benchmarked on x86_64-linux against Nixpkgs `bd07873`:

| attribute         | thunks  | lix@`1396012` | mimalloc | uplift |
|-------------------|---------|---------------|----------|--------|
| hello             | 206444  | 0.726s        | 0.620s   | 1.17x  |
| chromium          | 1177382 | 2.273s        | 2.136s   | 1.06x  |
| firefox-unwrapped | 1394797 | 2.521s        | 2.378s   | 1.06x  |
| texliveFull       | 3186440 | 4.992s        | 4.672s   | 1.07x  |
| nixosTests.gnome  | 7905808 | 6.115s        | 5.723s   | 1.07x  |

Based-on: https://github.com/NixOS/nix/pull/15596
Co-authored-by: Bernardo Meurer Costa <beme@anthropic.com>
Change-Id: I3ad92eacc075efeaf3d9f7730ada7b29835536a4
 2026-07-04 10:39:30 +00:00
Raito Bezariusandrootile 12f10fca7e nix/doctor: check ambient search paths 
Some issues are caused by users confused about their ambient search
paths, let's analyze the current configuration and make it awfully
obvious what is going on.

Towards #230.

Change-Id: I953ef3feaefc56cee68623c5aaa4d568ba1fd034
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2026-07-03 20:12:07 +02:00
Raito Bezariusandrootile b62b37296c nix/doctor: check flake registry 
We try to access to the Flake registry, if we are failing, it's hosed.
Otherwise, let's record some facts we can reuse later.

Towards #230.

Change-Id: I80dd8272d5c27f7f7ff01052e3487a0496ee29fc
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2026-07-03 20:02:35 +02:00
Raito Bezariusandrootile abeefa0e4f nix/doctor: print nix related information 
Based on `nix-info` and `nix --version` output.

Contributes towards #230.

Change-Id: I13bb9608b733e10037c6816456b5312e712c18ad
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2026-07-03 19:44:16 +02:00
Raito Bezariusandrootile 4126e671cc nix/doctor: print general system information 
Based on `nix-info` output and experience (LOCALE issues).

Change-Id: I350fff9592e7acc8fba0babe67c9fe1bf94a5781
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2026-07-03 19:44:16 +02:00
Raito Bezariusandrootile ad199c4723 libutil/systemd/hostnamectl: init 
This is some code that is specific to systemd to fetch hostnamectl
information via the JSON flag.

This will be used for `nix doctor`.

Errors will show up similar to this:
```
❯ ./build/lix/nix/nix doctor
[INFO] Nix system type: 'x86_64-linux'
warning: could not get host information via hostnamectl: error: Expected JSON value to be of type 'string' but it is of type 'number'
```

and makes them non-fatal.

Change-Id: Idd39f29626e770b1cd3896e8eb8545416c74b041
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2026-07-03 17:45:26 +02:00
eldritch horrors e9a01a5be6 libfetchers: don't treat empty/zero hashes as valid locks 
treating a file with an empty or zero hash as locked is not helpful.
these are placeholdes for "hash is not known", thus treating them as
a valid lock makes them completely useless (and confusing to users).

fixes #1233

Change-Id: If42b47281e6973fc86662b973db69e26f6346f5a
 2026-07-03 11:06:57 +00:00
K900 48f7747deb treewide: drop support for mdbook < 0.5 
25.11 is dead, and so is mdbook 0.4

Also reenable linkcheck which was doing nothing this whole time lmao

Change-Id: I4a8b9c763b881de840d6ef1b1d85bf19941f4386
 2026-07-02 18:06:50 +03:00
Qyriad 2c45e056e9 libstore/curl: std::move is not mem::take; fix correctness 
These uses of std::move are taking data out of a repeatedly-used state
object, à la Rust's std::mem::take(). But move-constructing or
move-assigning from an object does not guarantee that the moved-from
object is equivalent to a default-constructed one, or even that the
moved-from object will continue to function like a non-moved-from object
of that type (only that the object remains "valid").

Assuming there's no differing custom allocator shenanigans at play, both
libstdc++ and libc++ do leave moved-from std::map objects (and probably
others) in an empty, well-behaved, default-constructed state, at the
time of this writing. However the correct tool here is std::exchange().


Change-Id: I7cf76762a1b05a542b53fcd74be100646a6a6964
 2026-07-01 23:03:27 +02:00
Qyriad d4e445f251 libstore/filetransfer: reörganize includes 
Change-Id: I74310683b6e721f3ad0a7b2e2e2aa28c6a6a6964
 2026-07-01 23:03:27 +02:00
Qyriad 43c5aa2076 libstore: factor out curlFileTransfer 
As part of efforts to make debugging our Curl usage easier


Change-Id: Ib1f8567daa8b3d3d559a164bc85a8b906a6a6964
 2026-07-01 23:03:27 +02:00
Qyriad dbf1ef3d42 libstore: factor out CurlMulti 
As part of efforts to make debugging our Curl usage easier


Change-Id: I182bb3e968f1dfcd192ecd61103cba376a6a6964
 2026-07-01 23:03:27 +02:00
Qyriad 325364fc26 libstore: factor out TransferItem 
As part of efforts to make debugging our Curl usage easier


Change-Id: Idb8c32b33b6bfd0eebb9c9fded7b251d6a6a6964
 2026-07-01 23:03:27 +02:00
Qyriad cde71e45a1 libstore: fully inline FileTransferError 
To avoid template instantiation problems during the incomming
extractions


Change-Id: Ia0179da00bdfa45034559be88a2b923c6a6a6964
 2026-07-01 23:03:27 +02:00
Qyriad 1705515e79 libstore/filetransfer: reörganize includes 
Change-Id: If1a9c9d45e50af1d06d07a8510ab003e6a6a6964
 2026-07-01 23:03:27 +02:00
Raito Bezariusandrootile d26a73bfe6 libmain/shared: factor out nix features 
This way, other parts of the codebase can use it.

Change-Id: I58673a6a56362692980426a6c8f70f5c6aff24d4
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2026-07-01 17:46:00 +02:00
Samuel Dionne-Riel cf1e565a0a devshell: Fix usage with nix-shell 
Since d0190cff6f, the `justfile` uses
`$out` from the environment as an argument to `--prefix` in its `meson`
invocation.

The `$out` variable is...  A bit spicy to use since its semantics change
depending on whether `nix develop` or `nix-shell` is used.

In a `nix develop` shell, a directory *relative to PWD* (and not
relative to the Nix expression) is exported. In a `nix-shell`, the
`$out` used comes from the derivation.

The alternative to this change is to make the `justfile` use a variable
name that would not conflict with a `nix-shell` and `nix develop` shell.
Since it wants to use that value for the `outdir` variable, this would
be the name I would pick first.

I chose to instead work around the issue within the `nix-shell` since
this would be breaking the existing interface. As tenuous as it is.

Change-Id: I64cd5fd85345457e14880cb133e9ddafd55a25da
 2026-07-01 11:02:38 +00:00
blokyk a92e4b5c42 devshell: fix usage with non-nix rust-analyzer 
rust-analyzer binaries that don't come from nix (such as the one
shipped with vscode's rust-analyzer extension by default) need the
`RUST_SRC_PATH` env var to be the path to rustc's sources, but this
wasn't set inside the devshell. this change explicitely adds it to
the environment. (trying to add pkgs.rustLibSrc and rustcSrc to
the shell's packages unfortunately does nothing, because they are
just pure source packages, nothing more.)

(the reason this probably didn't come up for other devs is because
1. a lot of lix devs use other editors that probably just use the
binary from the path, and the devshell installs pkgs.rust-analyzer,
which is patched to always have `RUST_SRC_PATH` set, and 2. because
pkgs.vscode-extensions.rust-lang.rust-analyzer is also patched to use
pkgs.rust-analyzer by default instead of the bundled one, so installing
the extension using nix instead of from the marketplace would also hide
this problem.)

Change-Id: Ia27daf2a88756311ac5485ea0af44aae242241a9
 2026-07-01 11:55:10 +02:00
eldritch horrors 418918b599 libstore: derive legacy-tunneled rpc id from package version 
this way each git revision trivially gets its own rpc protocol version.

Change-Id: I066c58ebff697e534479db5d02271dac1ea71709
 2026-06-30 19:58:17 +00:00
eldritch horrors 6e19a940b1 rpc: transport errors in encoded exception descriptions, not results 
results don't allow for streaming or pipelining. transporting errors in
exception texts is *bad*, but it's still better than not having streams
that actually work. this isn't a great situation, but with capnp that's
pretty much the best we can do. still beats the other rpc frameworks :/

Change-Id: I2abf17bc5ea5de2baeea44ff4d7b2f4db33e98dd
 2026-06-30 19:56:57 +00:00
eldritch horrors 57c00a20c5 libutil: make exception wrapping accessible outside of TRY_AWAIT 
this will be useful for rpc cases, and it reduces macro code expasions.

Change-Id: I47c837fb5efab8eb408fcf9617055ea1562fc422
 2026-06-30 19:55:45 +00:00
eldritch horrors d0deb1a150 libutil: add error encoding that survives capnp exception transport 
using result types in capnp is fraught. while it makes some sense for
interfaces that need neither streaming nor pipelining and can provide
much better error fidelity there it's also fundamentally incompatible
with those that *do* need streaming or pipelining: streams will never
be stopped early unless an exception flies, and pipelines cannot look
through result types. likely the best thing we can do is to encode an
error for transport in the capnp/kj exception description strings. :(

Change-Id: Icb7d16238fa9a7aaf92c00363f7be4076ac02a61
 2026-06-30 19:55:45 +00:00
Qyriad 22e2ecdc43 build: update integer overflow sanitizer handling for fixed meson bugs 
Change-Id: I27f33e66e107d4af13abe62257be8c696a6a6964
 2026-06-30 18:56:51 +00:00
piegames 5434ddf4cd justfile: Don't require nested escapes on forwarded commands anymore 
This is a hack, but this is the best we can do with Just unfortunately.
Maybe rewriting this file into a python script wouldn't be the worst
idea at this point …

Change-Id: I32d089f6b49165faed99fb592d068dbb8a7e48e6
 2026-06-30 10:58:58 +00:00
piegamesandeldritch horrors 7ef274600d testlib: default nix fixture to also test all daemon protocols (3/3) 
This is the real deal. The new tests have surfaced one bug in the test
suite itself, and one in the RPC legacy protocol wrapper.

Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: Id75b671c10b45655d3627abd88563e2d41719446
 2026-06-28 19:09:49 +00:00
K900 12c1132f07 package: revert the breaking curl change on 8.21+ 
Change-Id: I374b37c5d428c6c7db71341d783b6a1cddf45b19
 2026-06-28 17:19:51 +00:00
piegamesandeldritch horrors d5044f0016 testlib: default nix fixture to also test all daemon protocols (2.5/3) 
All tests should have been clear here, but alas

Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: Id642be1f6e0fb714b844ee5348dc096866cf3906
 2026-06-28 16:52:12 +00:00
eldritch horrors 2605d3eab1 libcmd|rs: use rustyline instead of editline for the repl 
Change-Id: Ic414de3cc2435d9615305f05a85aa9c923e82a69
 2026-06-28 16:37:46 +00:00
piegamesandeldritch horrors 567348a90c testlib: default nix fixture to also test all daemon protocols (2/3) 
This commit enables the parametrization for "legacy-combined" protocol.
All failing tests with that have been either fixed or disabled. Notably:

- A couple of tests (e.g. involving builders) could be made to work with
a non-local store, but this would require some refactoring to testlib in
order to make certain configuration settings generic over local and
remote operations. We've disabled those for now, in order to make
progress
- The store tests all run locally only because of their nature
- The flakes tests also all run locally only because my energy for
fixing them is limited (and it's a *lot* of test failures, with probably
little overall benefit in terms of test coverage)

Change-Id: I56fa249a64f7c17c952f688ec89e9687f2a13f12
 2026-06-28 18:17:57 +02:00
eldritch horrors 1c2b5d4b5a libutil: expose loggers to rust 
this takes the form of a couple of hideous macros for now to get started
without worrying about semantics of rust logging and tracing crates, all
of which are in some way incompatible with the current lix expectations.
we do want to move to using those eventually, but that will have to be a
task for later. as long as only small amounts of lix code live in crates
we don't have to worry much anyway, and by the time we get around to any
Activity beimg ported we'll hopefully have a better understanding of the
requirements on either side. we'll reëvaluate the log infra around then.

Change-Id: I4aec476a486f56d0972294358cbf0b6810ab9394
 2026-06-28 13:02:07 +00:00
eldritch horrors c5bcb9a312 libutil: disable direct access to logger verbosity 
that way we can sync it with rust for nearly free.

Change-Id: Ib3f1e0335884e077a0293f4b13c5919297748522
 2026-06-28 13:01:45 +00:00
eldritch horrors 91663cca2d libutil|rs: bring back c++ interop with zngur 
Change-Id: Ia9e9881f464d5b25110b56fb9d72ed30463eaeee
 2026-06-28 12:47:18 +00:00
Qyriad d9ecb5301e build: cargo entrypoint II 
This is a second attempt at https://gerrit.lix.systems/c/lix/+/5516

instead of letting meson try to be cargo we'll just have cargo at home.
this requires some contortions to link everything together due to quite
a few meson deficiencies, but at the end we get to pretend that rust is
just c++ painted orange. we'll use cxx to bring real interop back soon.

also fixes #1230
also reverts 10845bfe63

co-authored-by: eldritch horrors <pennae@lix.systems>

Change-Id: Ib1fda843fa80d818705d5a65ec9054216a6a6964
 2026-06-28 12:46:20 +00:00
piegamesandeldritch horrors 9e846fbaea testlib: Small feature additions 
Change-Id: I8940e66a1fb89cf8024b1e94e328b28a008a1452
 2026-06-28 13:45:48 +02:00
eldritch horrorsrootilepiegames
 		84fb93a009 testlib: default nix fixture to also test all daemon protocols (1/3) 
This first geralizes the `nix` fixture with `pytest_generate_tests` to
iterate over all protocols (unless they are marked with `no_daemon`),
though for now the list of protocols is set to be empty. The lang tests
are all tagged with `no_daemon` because they are mostly pure and running
them multiple times would be wasteful.

Co-authored-by: rootile <lix@rootile.de>
Co-authored-by: piegames <git@piegames.de>
Change-Id: Ib407edb420ba4bf434cacf9563a71f5ae6fa8eef
 2026-06-28 13:45:46 +02:00
Samuel Dionne-Riel 7868f52c2a tests: Add regression test for daemonAuthorizationSettings 
Regression test for extra-trusted-users being ignored
  https://git.lix.systems/lix-project/lix/issues/1183

Change-Id: I764a6add0215b535fc4abd21b41d1c23b21fc06b
 2026-06-27 18:37:27 +00:00
Samuel Dionne-Riel fc6772a7dd main: Run registerGlobalConfig before loading config 
This is a bit of a trip.

Commit 63b0a01d75 moved config
initialization out of static initializers.

In doing so, an oversight was made with the new
`GlobalConfig::registerGlobalConfig` calls. Previously all global
configuration would be handled well before `loadConfFile` was called.

The change placed `registerGlobalConfig` in `main` *after*
`initNix`. The `initNix` function is what calls `initLibStore`, which in
turn callse `loadConfFile`.

So, in effect, the configuration for `daemonAuthorizationSettings` and
`nixDevelopSettings` are not actually loaded.

Change-Id: I10dba2c98e032f910942ee0e0ca6bb39ef60edc2
 2026-06-27 18:37:27 +00:00
rootileandeldritch horrors fce63f6dbf parser: improve error message for ill-defined escape sequences 
fixes #1169

Change-Id: Ic93e0cb5442b717964de3ae9eb22f4cd48e4ad0a
 2026-06-25 21:03:09 +00:00
Qyriad de68de1d11 tests/test_build_fod: fix overly strict pattern 
Change-Id: Idd3146f41d527a7263ee61e7a5ce0e836a6a6964
 2026-06-25 09:54:50 +00:00
Qyriad 071b810f95 package: override Curl to remove backported patch that breaks our tests 
Cherry-picked-from: https://github.com/nixos-cuda/nixpkgs/commit/3f596088797d2e3a62e20cae2d2936e8548a690e
Co-authored-by: Atemu <git@atemu.net>

Change-Id: I5d060872d4d0ae283d73b4268af612326a6a6964
 2026-06-24 13:45:53 +00:00
Qyriad 63fb295ad8 build: fix C++ dylibs on Musl by working around Nixpkgs brokenness 
Change-Id: Iad30ec38eef8205e2d73553caddbbbc96a6a6964
 2026-06-22 21:18:56 +02:00
Qyriad 2ea4b5d9e1 fix Nixpkgs 26.05 warning about nested lists in nativeBuildInputs 
Full warning was:

warning: Dependency of package 'lix-shell-env' uses a nested list in attribute 'nativeBuildInputs'.
This is deprecated as of Nixpkgs release 26.05, and support will be removed in a future nixpkgs release.


Change-Id: I6b6c7425e1b03d3238ea21138cb3f2cf6a6a6964
 2026-06-22 21:18:56 +02:00
rootile 8902b78442 tests: migrate derivation-json.sh 
Change-Id: I8a648231677bde28611dc9511e3a03d87608e7b4
 2026-06-22 18:16:17 +00:00
Justin !andQyriad 30ca2e38c4 nixpkgs: bump 25.11 -> 26.05 
Change-Id: I7747d38d38f3be69404f5dcc8147e5366a6a6964
 2026-06-22 17:58:20 +02:00
Qyriad 3f7bc8ffb8 HACK: patch out broken Perl version compatability check on macOS 
Change-Id: I85cfc3863299d06f36138753e8f1f0206a6a6964
 2026-06-22 17:58:20 +02:00
eldritch horrors 20453629ab build: remove libutil rust components 
they're not doing anything, and they're slightly in the way of reworking
the rust build infra. we'll add them back once that rework is completed.

Change-Id: Idde8ba3585fd08b89b4b4298349f8f58a29a9661
 2026-06-21 17:16:17 +02:00
eldritch horrors 0cddd2d38a build: always link plugins with unresolved symbols allowed 
we're quite surprised that this hasn't caused problems sooner.

Change-Id: I33b779c1563e15db715c7fd4d3cb6516cf34391d
 2026-06-21 17:16:17 +02:00
Qyriadandeldritch horrors 10845bfe63 build: move all remaining C++ files to a library (except main.cc, yet) 
Change-Id: I2a69701a67faf6f9ad33e30bbe7aa4386a6a6964
 2026-06-21 17:16:17 +02:00
Qyriadandeldritch horrors 05e9ca3e26 build: remove file prefix mapping 
blokyk on Matrix found that is not only no longer needed but now
actually breaks the very thing it sought to fix


Change-Id: I39cc0df611a6d81c763063c9fd2f0f426a6a6964
 2026-06-21 17:16:17 +02:00
ash ff920956a9 libstore/build: fix URL guesswork for failing structuredAttrs FODs 
Fixes #1175

Change-Id: I6a074f489620f6de0615858f4d0d872c6a6a6964
 2026-06-20 14:31:17 +01:00
rootile cf0ce785b7 tests: remove forgotten cli-literate-parser 
When removing the f1 repl characterization test suit, these files were
kept on accident, despite them being dead code now

Change-Id: I13b2c5a96004787c26fe7209e4cb3aeedd8ada15
 2026-06-17 12:03:36 +02:00
RutileandLix Systems Gerrit d748b4e509 Revert "repl-characterization: document the syntax" 
This reverts commit 0d6b372c19.

Reason for revert: In the time between this getting +2 and it being merged, all repl tests and its syntax have been migrated to f2, making these changes null and to dead code

Change-Id: I00f5237bf3f2ca3a60cc1584f67fb67196805e7e
 2026-06-17 10:00:54 +00:00
Jade Lovelace 0d6b372c19 repl-characterization: document the syntax 
Gosh. Why was there never a README in here explaining what it does
without requiring people to read C++? There was a half-assed grammar in
a comment, but it was very half-assed, so let's replace it with a
better-written one.

I'm documenting this so that neither I nor anyone else have to read C++
while rewriting the parser in Python.

See: https://git.lix.systems/lix-project/lix/issues/1196

Change-Id: Ia34ea7b02f109eb7753833e1c03d4b5d1d3c3b75
 2026-06-17 01:01:13 +00:00
rootile 7bf0edcc0d build-remote/logging: use machine names instead of uris 
Change-Id: Iafe7b3e780d3a69dc4bb2d745ff40db6df7111e8
 2026-06-16 17:58:23 +02:00
rootile 4b0371bff3 libstore/machines: add name attribute 
Change-Id: Ie0f246dee219ffb3cade7e8b9a9596eb2a06f2dd
 2026-06-15 22:09:15 +02:00
rootile c019a776e9 python-infra: move f2 specific stuff into f2 
When extracting a global pyproject.toml from the f2 one, some f2
specific things slipped through and ended up in the global one.
This commit moves those parts back where they belong

Change-Id: I95706fabec487410717fb362c95bfee900cf5b8c
 2026-06-14 20:56:18 +02:00
rootile 93c872ab25 tests: migrate binary-cache-build-remote.sh 
Change-Id: Ie159da4f00cf05627a777bae80c1368485dd4fcc
 2026-06-14 20:33:11 +02:00
rootile 2ea51a6c71 tests: migrate structured-attrs.sh 
Change-Id: I6d2b82e3aa236d01b9a345c148b7fef3703a446d
 2026-06-14 20:19:58 +02:00
rootile fe79d62b73 tests: migrate check-reqs.sh 
Change-Id: Icc22331f6f7e1be0919a74155cd93514c5d70a65
 2026-06-14 20:19:58 +02:00
rootile 9614f12908 tests: migrate export.sh 
Change-Id: I9416b71b0a15a7703aa9d7c2dea866a1f556e657
 2026-06-14 20:19:58 +02:00
rootile bc8a95457f tests: migrate export-graph.sh 
Change-Id: I0de67ba6b1f710434f504814dd9b5eb7d486e65a
 2026-06-14 19:02:11 +02:00
rootile ba3a026f40 tests: migrate eval-store.sh 
Change-Id: Id4a721871218160b441e848f6f252c06187320bf
 2026-06-14 18:46:39 +02:00
Astreaprtcl 12baca7bea store/fetchers: fail if ca or netrc file has been specified but doesn't exist 
if the netrcFile setting has been changed tell curl that the file is required, so
fetching fails if the file does not exist

if the caFile setting has been specified, check if the file exists and throw an
error if is it non-existent

Refs: #1106
Change-Id: Icb9330a7a715175d35b9ed894ed945f0fd4d7061
 2026-06-13 23:38:45 +02:00
rootile abfe9d0c3e tests(functional): remove old repl characterization test suite 
Change-Id: I2c13e0332298600cafa97b06113ca56f66668324
 2026-06-11 16:30:56 +02:00
rootile 64b434b87b tests(f2/repl): migrate stacktrace tests 
Change-Id: Ic6795f22d68aea113fa2037cbd0a731ac5e698f8
 2026-06-11 16:30:56 +02:00
rootile da9f0c494e tests(f2/repl): migrate failing pure eval test 
Change-Id: Ic8db38346c2420b2d3c5ab51fe71ef0b88f826b3
 2026-06-11 16:27:49 +02:00
rootile a50b85b7d6 tests(f2/repl): migrate overlay tests 
Change-Id: Ibd2b123f919abd84ecb9333bd952cf0bdb813992
 2026-06-11 16:26:34 +02:00
rootile cb4703019b refactor(f2/repl): move overlays to overlay folder 
Change-Id: I16e42eae00dc7a1f06300a81d69c637b9fa342df
 2026-06-11 16:19:20 +02:00
rootile ab8cdcc9b8 tests(f2/repl): migrate basic tests 
Change-Id: I22241734a7249d376c86b3a226c5f1a82c03fd33
 2026-06-11 16:19:20 +02:00
rootile f7ce20d879 tests(f2/repl): migrate debugger tests 
Change-Id: I2cc1d45db2e5e0b8cea6798c70e13ede57fea6ee
 2026-06-11 16:19:20 +02:00
rootile ab149a6cd8 tests(f2/repl): migrate builtins_warn.test 
Change-Id: Iaed1fb8651840b855ec7bdf5087681dc5eb9d54c
 2026-06-09 19:40:07 +02:00
rootile 99175779c4 tests(f/repl): remove duplicate repl tests 
Change-Id: I07635a7f62bf56b4ee9526b5a34f51049554e875
 2026-06-09 19:39:52 +02:00
rootile a4634bca67 feat(f2/repl): allow should_fail to have repl input 
So apparently we can still use the repl if the input file fails, as long
as we have the debugger enabled.
This Commit removes the "only one output block for should_fails"
requirement to allow this behavior

Change-Id: I02568c8d7e4305ab6414953306650bbe13c6f4bc
 2026-06-09 19:16:46 +02:00
rootile c6fdf7d843 feat(f2/repl): add block for catching startup messages 
Change-Id: I8465ceb70ea698a00e1cdc7cef0c560382bb73a3
 2026-06-09 19:16:46 +02:00
rootile 5d91fcba5f fix(f2/repl): accept tests should not replace delimiter on updates 
Change-Id: I468bf84af1b4a5dab34e186c55830e11ba9b9474
 2026-06-09 19:16:46 +02:00
rootile 28a8c5c42a formatting: disable formatting of repl test files 
Change-Id: I1b5fedbf3f54ada2a8e105f1f318956154169760
 2026-06-09 19:16:46 +02:00
rootile 868ec73e16 docs(f2): move lang and repl tests to their own files, closer to where the docs are actually needed 
Change-Id: I2d003144ef97f12069a27f9a4be4219ec14d6056
 2026-06-09 15:21:06 +02:00
rootile 1a1234cf68 f2: lay foundations for repl testing 
Change-Id: I0f5588adbf4ea11e2846b2a3655308d1dfb711b0
 2026-06-09 14:06:04 +02:00
eldritch horrors daa2bc82bc libstore: warn if http3 && !http2 
curl does not support this configuration. we can't easily change the
config setup to mirror how curl works, but we *can* print a warning.

fixes #1223

Change-Id: Iaa1583275d9cc9e5eed5552210db183a8e0628c8
 2026-06-08 12:47:19 +00:00
blokyk 0cad504c8a libexpr/flake-config: improve untrusted flake config prompt ux 
the case-sensitiveness of the "accept all options" prompt (and the fact
it silently defaults to no) was confusing a few users, so by rewording
it slightly we can avoid that case-sensitiveness and make the prompt
clearer. for consistency, i also rewrote the individual accept prompt to
make it case-insensitive.

at reviewers' requests, i also reworked the code so it re-prompts the
user on invalid answers.

Change-Id: Ie786c0f26a0aa752b22d5a5e8af1b2c7ca2846b5
 2026-06-07 21:23:39 +00:00
Alex James 1396012a03 meson: conditionally define optional dependencies in pkg-config 
lix.pc currently lists several optional libraries (such as libcpuid and
libseccomp) as required dependencies. Update the pkg-config generation
step to only list optional dependencies which are used by the Lix build.

Change-Id: Iceea7ecc9421b5b10fc08edbe95a063a4e39cea1
 2026-06-06 15:49:20 +00:00
Linus Heckemannandeldritch horrors 267a8e7c9b nix-eval-jobs: fix version 
Change-Id: Ie237ddb2b1a0829b0954b05794506afd6a6a6964
 2026-06-06 15:20:47 +00:00
toonn 2698fe8611 manual: Update Attribute Set section to exclude indented strings and URIs 
Indented strings and URIs are string literals but they are not allowed
as names for attrsets, `let` or `inherit`. The following snippets
illustrate that they lead to syntax errors.

==> examples/ind_str-attr.nix <==
{
  ''indented string'' = "error: syntax error, expecting '}'";
}

==> examples/ind_str-inherit.nix <==
{ inherit ({ "non-indented string" = "error: syntax error, expecting ';'"; })
    ''non-indented string'';
}

==> examples/ind_str-let.nix <==
let
  ''indented string'' = "error: syntax error, expecting 'in'";
in
  true

==> examples/uri-attr.nix <==
{
  http://example.org/foo.tar.bz2 = "error: syntax error, expecting '}'";
}

==> examples/uri-inherit.nix <==
{ inherit ({ "http://example.org/foo.tar.bz2" = "error: syntax error, expecting ';'"; })
    http://example.org/foo.tar.bz2;
}

==> examples/uri-let.nix <==
let
  http://example.org/foo.tar.bz2 = "error: syntax error, expecting 'in'";
in
  true

Change-Id: I6774af93baf9f533877642a563775667c518ea88
 2026-06-06 14:52:54 +00:00
eldritch horrors d5cf115a3c libstore: fix file transfer cancellation deadlock 
file transfer cancellation should not get lost, or bad things happen.

fixes #1218, probably

Change-Id: I1be9d523236655f1248883d59b2b7ef5db7873a2
 2026-06-06 14:19:53 +00:00
eldritch horrors 4f0214b4bc tidy: add a lint for std::fs exceptions 
Change-Id: Iba3c6ed858a7c4304c8c8825aaf5df3ec60b91df
 2026-06-05 18:43:08 +00:00
eldritch horrors 6cd8fcd9d8 libutil: handle all rename errors in moveFile 
what the actual fuck. swallowing all errors *except* EXDEV is not okay.
renames do not do this, so moves should not do it either. luckily we do
not use moveFile anywhere except the store path registration code. this
may or may not have caused problems in the past. probably nobody knows.

Change-Id: I2b0255a5703983cbd129abc3219c11ac7171fd12
 2026-06-05 18:26:21 +00:00
eldritch horrors e108432cb7 libutil: catch std::filesystem errors 
catch and rewrap std::fs errors in some meaningful way, otherwise lix
can crash with foreign exception aborts if filesystem failures happen
that aren't already being caught. renames in particular can fail with
incomprehensible errors in even pretty normal circumstances elsewise.

fixes #1219, fixes #995

Change-Id: I0b1e5696d300aac3422e43cffc17a2426803d64e
 2026-06-05 17:17:18 +00:00
Maximilian Bosch c13ca9adcf nix-support/build/outputs: don't expose rl-next.user 
I'm not sure why this wasn't an issue before, but the generation of
jobset constituents in Hydra trips over this. The Hydra-convention is to
have the `system`-attribute last anyways, not in-between.

Since this is a single attribute, get rid of it for the sake of
simplicity.

Change-Id: I80ef1b92ed158c8712ee899534ce82130614f338
 2026-06-04 16:26:04 +02:00
Maximilian Bosch 82d23e8f1b release.nix: init 
This is a Hydra jobset definition to be used to warm up a cache with Lix
versions built against several nixpkgs versions. This is only happening
after a bunch of tests have passed.

Change-Id: Id0b0e50c08581f3f377c7c793d2fd83a6007999d
 2026-06-04 16:26:04 +02:00
Maximilian Bosch 1c70995b25 flake: remove release job 
This is Hydra-specific and will be re-added in the commit adding a
release.nix.

Change-Id: I13900821b02fe268356932f612734187188832b8
 2026-06-04 15:41:13 +02:00
Maximilian Bosch 4931a5f10c nix-supports/outputs: only build packages on platforms that are available 
If e.g. i686-linux is not in the list of systems and thus no attribute in `nixpkgsFor`
exists (e.g. on hydra.afnix.fr), don't try to instantiate it.

Change-Id: I118e2af5289ad7bd95ce7f1e6b0728afb6f0321f
 2026-06-04 15:40:48 +02:00
Maximilian Bosch 9b2a19530c flake: split off common CI artifacts 
...expect for the dev-shell: while it doesn't seem reasonable to me to
even build dev-shells in post-merge CI, this should still remain part of
the pre-merge checks.

Change-Id: I5cb9b4cf53484f884d694770ed94d0dc0862117e
 2026-06-04 15:38:58 +02:00
Maximilian Bosch 9fe22f4e5e flake: split off tests 
Change-Id: Ic7957eb2928bc66e9213051d9de7447d96e55fe6
 2026-06-04 15:30:46 +02:00
Maximilian Bosch 52449f3f8e flake: split off packages attribute 
Change-Id: Ic1e0ab81cb034f6468edfc03b7a716bbf157146f
 2026-06-04 15:30:45 +02:00
Maximilian Bosch eb4997592d nix-support/build/inputs: init from flake bindings 
Change-Id: I84b462d8fa18f656d4a19d2def6d69ecfb90c156
 2026-06-04 15:30:45 +02:00
Maximilian Bosch 8d581f42d9 flake: turn variables into a scope 
Step II to move the code out of flake.nix. Done in a second intermediate
commit to make it easier to review.

Change-Id: I13fb386f2e9b87b1d28c366e9b7f0d2dbe03315c
 2026-06-04 15:30:45 +02:00
Maximilian Bosch 79d6056d26 flake: import nix_2_18 as path 
That way we can use it in non-flake contexts down the chain.

We don't override the `nixpkgs` anyways since this is a test artifact
that needs an old nixpkgs input, so this isn't a concern here anyways.

Change-Id: If2aa8f7171a445e8321ef449026e206cc5a0fe3c
 2026-06-04 15:30:45 +02:00
Maximilian Bosch 13bcd07385 flake: don't use self in let block 
The upcoming change factors out a lot of code to make it reusable for a
"legacy" Hydra project that we use to build a matrix of (Lix x nixpkgs)
to have a warm cache for everyone.

The Hydra project will not use flakes to have some inputs overridable,
so the change is needed. In non-flake mode we don't have a notion of
"self", so rename it to `lixSrc` such that the next commit is purely
about moving stuff without changing any aspect.

Current stock Hydra doesn't expose `lastModified` though, so right now
that is a cache-miss. There's a pending Hydra PR fixing that[1].

[1] https://git.lix.systems/lix-project/hydra/pulls/82

Change-Id: I59d70b8b874f576c782217e7bca099eda4c7add9
 2026-06-04 15:30:20 +02:00
Maximilian Bosch 1b9570162e tests/nixos: don't pass in self 
The reference to the flake is used in a single place only to get the
`doc` output of Lix. Remove it so that we can also instantiate the tests
in a legacy Hydra jobset (happening further down the chain).

Change-Id: Ie3eccf42d67c81fcf991342097783e5067d512bc
 2026-06-04 15:30:08 +02:00
Maximilian Bosch 003946f062 tests.container: fix for 26.05 
Noticed on my quest to make Lix buildable against multiple nixpkgs' to
provide pre-warmed caches for the flake on AFNix's Hydra.

`pkgs.systemd` has a new patch policy[1] that only allows patches for
hotfixes and fundamental incompatibilities. As such, the patch that
allows nspawn workloads to start without a `/usr/bin` was removed with the
recommendation to create that stub yourself.

[1] https://github.com/NixOS/nixpkgs/pull/488508

Change-Id: Iaa35ae3b8971867f12daeac5f94958c20a7d9233
 2026-05-31 12:27:17 +02:00
rootile c64fbcbdb2 f2/command: allow piping stderr to stdout 
Change-Id: I280d4888e9dc11cce93bacfddd9b4383a11525e1
 2026-05-27 11:49:49 +00:00
Erik Jensen eef57410d7 Fix nix-copy-closure --include-outputs 
The option has been broken since Nix 2.4. The flag was accepted, but not
used. This change plumbs it through to computeFSClosure.

Change-Id: Id6adee8ea8a6a4f457b24b650660f319e3daa2b2
 2026-05-26 19:40:07 +00:00
Linus Heckemann 8edb857248 libfetchers: lock inputs before fetching them 
Fixes #1122

Change-Id: I9243e692779a795066417a15c6315dd56a6a6964
 2026-05-26 15:14:13 +00:00
Qyriad b113be5f13 flake: provide devshells for static cross systems too 
Change-Id: I27ec97eaac88236a0958953a81d09ee16a6a6964
 2026-05-23 21:14:54 +02:00
Qyriad 720306b693 package: deduplicate python environment packages 
Change-Id: I2653d2db71a40d763d4cd3b5c4ef05306a6a6964
 2026-05-23 21:14:54 +02:00
Qyriad f6cfd6b3ea package: deduplicate pyxattr override 
Change-Id: I4c8f34cef4d09f2ccb2df23e0679addd6a6a6964
 2026-05-23 21:14:53 +02:00
Qyriad 839ae964fb justfile: clean build and outputs contents, not directories 
This makes it easier to build lix on a different file system from the
source repo, e.g. symlinking ./build to a directory in /tmp.

The ./outputs directory is included for completeness as another
significant source of generated artifacts.

Change-Id: I1bb4c21c700beacb4882a1be473dd31353edd177
 2026-05-23 21:14:53 +02:00
blokyk e5ca862ea9 libfetchers/tarball,git: log when unpacking tarballs 
this logs unpackTarfile calls as an `actUnknown` activity, which
unfortunately doesn't report any progress for now. for the reasoning
behind this choice, see the previous iterations of this CL (cl/5590)
as well as the attached comments (the crux of the conversion is in DMs,
unfortunately). in short:
- i could't make a new ActivityType because that would be a breaking
  change to the daemon protocol, including with previous lix versions
- i can't use actFileTransfer because that would show up as a download,
  which would be confusing for both users and tools
- i can't use actCopyPath /or/ actCopyPaths because that would break
  NAR/substitution accounting when doing remote builds

also, ArchiveDecompressionSource, which is apparently used for
decompressing logs etc, does its own custom decompression without using
unpackTarfile, so it is not affected by this. from what i can tell,
unpackTarfile is only used for:
- fetchtarball fetcher
- git fetcher
- channel unpacking
- nix-prefetch-url

Fixes #1213

Change-Id: I68f3be520681d8889292353dd41ab19a60c5eeea
 2026-05-23 05:21:56 +00:00
Linus Heckemann 1f100371d7 doc/hacking: switch buildkite sso url to afnix 
Change-Id: I7f7e65fe2dc52b40d4771e59198dd66a6a6a6964
 2026-05-18 18:13:47 +02:00
K900 b118785864 nix-eval-jobs: fix nested list in inputs 
Change-Id: I08b02ecb85512288b24a02a500348eaf78f030be
 2026-05-18 11:55:30 +00:00
rootile 1c81bd926d tests/f2: improve error message for failed in assertions 
By default, when one uses `in` assertions, upon failure newlines in any
of the two strings is escaped, and everything is printed in a single
line.

This commit allows one to pass `-vv` to print out the left and righthand
side unescaped, with each line of the output being its own line,
improving readability a lot when dealing with multiline outputs.

Change-Id: Iaf67a20fce57d375e2c62a8e8a0957b14eec26ef
 2026-05-18 10:47:00 +00:00
rootile b34f4d7b17 OWNERS: Do not inherit parent owners for f2/testlib 
see [lix-weekly of
2026-04-22](https://wiki.lix.systems/link/153#bkmrk-code-ownership-of-f2)

Change-Id: I3e3df0d864a5614bb66a2d756df42c63305a7c8f
 2026-05-13 22:32:30 +02:00
rootile 1971824580 OWNERS: remove Helle from test codeowners. 
See [lix-weekly of
2026-04-22](https://wiki.lix.systems/link/153#bkmrk-code-ownership-of-f2)

Change-Id: Ic4345dd70e2cf5f7dd4509b32543c1f61d76cf07
 2026-05-13 22:32:30 +02:00
Qyriad 7831c98a4d package: remove no-op deprecated 'pie' hardeningDisable 
PIE was enabled by default in GCC Nixpkgs 25.11¹, and the hardening flag
was subsequently removed² and has no effect. This commit thus fixes a
warning about it.

[1]: https://github.com/NixOS/nixpkgs/pull/439314
[2]: https://github.com/NixOS/nixpkgs/pull/442510


Change-Id: I27850f947b1888c01045af9bbd5e737f6a6a6964
 2026-05-07 11:45:08 +02:00
Qyriad cc271148b0 treewide: fix sys/poll.h -> poll.h 
poll() and pollfd are defined in poll.h per POSIX¹, and musl complains²
about it.

[1]: https://man.archlinux.org/man/poll.h.0p
[2]: https://github.com/kraj/musl/blob/8cb84492b0245d70b2cd0edd523e2b55c7ad67a9/include/sys/poll.h#L1


Change-Id: I4fff447394903cbf55e330f07ed82b906a6a6964
 2026-05-06 20:51:25 +02:00
skye 2b409ad322 meson.build: Warn on deprecated declarations 
Can finally do this after fixing all Value based deprecations.
Resolves #744

Change-Id: Ic050b42a5af140c6dc74dd1545e3f1d76a6a6964
 2026-05-06 12:08:03 -04:00
piegames 7cdda9c5fa flakes: checkOverlay: Don't check the second argument 
It's software archaeology time.

- In 2019 (Nix 2.4), Eelco added `checkOverlays`, which strictly checked
that the overlay must be a function of two arguments, one called `final`
and the other `prev`. (dc3f52a144,
gh#3573)
- In 2024, NofairKing opened an issue about checking the exact name
being a silly idea (gh#10516). Instead of trying to make the check more
sensible, the first fix attempt simply expanded the check to allow
`previous` instead of `prev` (the author's preferred name for the
attribute, supposedly; gh#10553). After some discussion, instead a PR
got merged which simply removed the `prev` check altogether, while
leaving the `final` argument check intact for some reason
(ad65a50a94a97bf1f1a1902f43542d28a2e8206b, gh#10572).
- In 2025, over at Lix and oblivious to the latest change over at
CppNix, I came across this stupid check while trying to improve the
abstractions of the AST in Nixexpr. I spent (wasted, retrospect)
considerable amounts of time removing the exact name checks while also
improving the check overall (making it check for more than two
arguments, and also generally improved error message).
(0928d4d87a,
Id4244171123dd8a228be71ce9f04d8e9f647c111)
- Fast forward to 2026, where I run into this becursed piece of code
once again, and once again during some Lixexpr cleanups. Now that I have
seen how carlessly the issue was handled at CppNix, I can't be arsed to
give a flying fuck anymore. Out the code goes. Bye.

With this commit, all casts on `Expr` subclasses outside of libexpr
itself have been removed. No more violation of abstraction boundaries.
Good riddance.

Change-Id: I939968bb01d461764cfa0f4ea7152b4fcf1acf93
 2026-05-06 17:22:19 +02:00
piegames 481fc30ae7 libexpr/flakes: Replace the AST checks with maxCallDepth = 0 
Change-Id: I7130cc941b4b7432df76a3995d298edfb28e6a01
 2026-05-06 17:22:19 +02:00
eldritch horrorsandpiegames d0a4b55a0e treewide: pour the rpc foundations 
pour the foundations for rpc, and let them set. this is very much
unstable and must be opted into with explicit store uris (e.g. by
setting `NIX_REMOTE=daemon?protocol=any`). the daemon sockets are
not enabled by default and must be enabled with the `rpc-sockets`
experimental feature. we will not advertise this just yet because
in the current state it one has to be *very* dedicated to the rpc
cause to deploy this, but once we have some more bits migrated we
may want to add release notes and officially as for beta testing.

Co-Authored-By: piegames <git@piegames.de>
Change-Id: I85a96ccb700b91190c1eb37154bcc6ae1c03401a
 2026-05-06 10:55:08 +00:00
eldritch horrors b2c95d41c9 libstore: prepare meson for more rpc definitions 
Change-Id: I4c495d574acdaa976bdd4ab51bf543aae63903af
 2026-05-05 12:03:34 +00:00
eldritch horrors 010d062926 libstore: move legacy request handler loop to new function 
splitting it off lets us reuse it (and all the legacy protocol handling
it encapsulates) in the rpc world as well by just tunneling wire bytes.
doing this allows incremental migration of to a new rpc protocol, until
we are finally ready to call the rpc protocol ready and remove the link
to the legacy protocol altogether (and maybe merge this function back).

Change-Id: I8f2a8795c4a9d713ebbbbfd02e9eec5271a2b7a9
 2026-05-05 12:03:26 +00:00
eldritch horrors af4d064be7 libstore: processConnection -> processLegacyConnection 
Change-Id: I608085c845cce05f4186f83b9ef0d84e8e4a776e
 2026-05-05 12:02:47 +00:00
eldritch horrors 7edb421e1b treewide: generalize socket handling 
this adds the protocol registry. we also extend the systemd unit setup
to allow adding new protocols without much fussing, and we move deamon
instances into protocol-dependent scopes. this will make monitoring or
resource limiting of individual protocol daemon instances much easier.

Change-Id: Ic9da08eb074868b6e57385075c5502b081e8461f
 2026-05-05 12:02:16 +00:00
eldritch horrors de3a2451eb libstore: move daemon protocol struct out of Settings 
we'll use this to create a registry of protocols the system understands.
also use this struct during connection setup to make that simpler later.

Change-Id: Ifa481fea7ea2efa2a1f1be4d81076a9e022d24f9
 2026-05-05 12:02:07 +00:00
eldritch horrors 4da9a53b66 f2: add test for daemon socket connection 
tests both order of sockets tried and that all sockets work as expected.

Change-Id: I5aeb506e159564a0b14b6fbbe5df1e5ccdb437d5
 2026-05-05 12:01:58 +00:00
skye 8ab7547a74 libexpr/json-to-value: Refactor json-to-value.cc from linked list of 
inherited classes to stack of variants

Previously the code represented the stack of unclosed arrays and
objects as a linked list of virtual objects which seemed unnecessary.
There is a closed very small (3) number of possibilities, and one of
them only exists at the top level. A vector of variants seems a lot
simpler, and as a bonus, more performant.

Change-Id: I6bc429d682d22f69ca91dc0518e6be646a6a6964
 2026-05-04 15:42:55 -04:00
eldritch horrorsandRaito Bezarius 2f7d7444f1 libutil: fix nar parser buffer overflow 
string data shares a buffer with the binary string length field. size
calculations for string read buffers always include the length field;
sufficiently large length fields can cause these calculations to wrap.
a malicious nar could use this for OOB writes in the daemon (as root).

since we use strings only as tags for archive members and for symlinks
with their OS-dependent length limits we can simply limit string size.
1 MiB should be sufficient for all symlinks, and tags are always tiny.

Change-Id: I89fb05f73c1dbeda45d91244aba4cd526a3d83e1
 2026-05-04 19:00:19 +02:00
Raito Bezarius c595477002 releng/keys: update the way to receive the ephemeral key 
Change-Id: I791f3f3f49ecd5884c9e86b5d3b617fc139e031f
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2026-05-04 18:57:20 +02:00
Raito Bezarius c71d2ebace releng/environment: update staging parameters 
These parameters are now created on https://s3.afnix.fr.

Change-Id: I96b6fd913429ee46d04c412cb141edd288665ced
Signed-off-by: Raito Bezarius <raito@lix.systems>
 2026-05-04 18:57:20 +02:00
eldritch horrors 65346101a2 libstore: rename openConnectionWrapper to something more meaningful 
Change-Id: I46c57476ef73c6a8b851ab34ba67872c8f6b826b
 2026-05-04 09:44:10 +00:00
eldritch horrors 0d2c48a797 libstore: remove RemoteStore::failed 
this is not read in code paths that are called repeatedly, and the only
reader can never see it being false. we also question the wisdom of the
flag in the first place; why should opening a connection fail because a
previous connection did? conditions may have improved a lot since then!

Change-Id: Ib7c219e6239432d99a22b10024847218469f1cd4
 2026-05-04 09:44:10 +00:00
piegames 078bbe6171 libexpr/eval: Fix off by one in maxCallDepth calculation 
This isn't really relevant for a setting of 10k, but now that we want to
set it to 0 for Flakes it makes a difference as to whether or not one
top-level function call is allowed or not. (It shouldn't be)

Change-Id: I6bbf99826af0289c232b9d45d172a378b81abe4b
 2026-05-03 13:05:46 +02:00
Lunaphied cd573beb0a docs/hacking: fix to respect changes in justfile 
We no longer have setup-custom, just use setup

Change-Id: I2a151d04dc3d90a98b79859b05c4e6706a6a6964
 2026-05-01 11:12:54 +00:00
Jade Lovelace d94bb49cf4 fix: structured build logs: bad field types no longer fail builds 
Caught by edef's harness. I rewrote the structured build log tests in
f2 and added a test for this one.

Before: it failed the build altogether:
```
building '/tmp/jade/pytest-of-jade/pytest-77/test_invalid_fields_files0_0/nix/store/skhjs5zs082nqiakj69qz32gyywz0v1a-unusual-loggi
ng.drv'...
error: unsupported JSON type 7
```

Now the output is as expected:

```
warning: Unable to handle a JSON message from the derivation builder: error: unsupported log field type number
@nix {"action": "start", "fields": [1.5], "id": 2, "type": 1, "level": 1, "text": "abc"}
```

Change-Id: Idbd28c5a4d4cf15f27207f83b730e7946a6a6964
 2026-04-30 17:28:55 +00:00
piegames 076d1aa4cd eval: switch up ExprOpConcatLists error position again 
This is a minor adjustment to 2b22dae1ba /
I3e9d7c1c7a6599a8e68302448bbb961d051002b7

Basically having the error point to the operand instead of the operator
kind of makes sense, but it would require error spans to *truly* make
sense, and in the meantime maintaining the logic is more hassle than it
is worth.

Change-Id: Idac98dd77a0f4a6cb386cb74e0d4eb5fd2f503e5
 2026-04-30 17:14:43 +02:00
piegamesandeldritch horrors 8782c7533f eval: Improve ExprConcatStrings error messages and positions 
"while evaluating a path segment" was just plain wrong

Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I827374635df2887d17a8ee8b1755c078bf0b506b
 2026-04-30 17:14:43 +02:00
rootile 7017def185 f2/testlib: support wrapping commands 
It's useful to obtain a fully assembled command and then wrap it to
execute into something else, e.g. gdb, strace or fakeroot.

Change-Id: Ida5928fd001925549eab89edfbf0facada7ef7f1
 2026-04-30 16:59:28 +02:00
Jade Lovelaceandrootile e70ae36b3f libexpr/builtins: correctly handle integer edge cases in floor/ceil 
We choose to throw an error in the invalid-input space where we *can*
produce a valid (but wrong) result to allow ourselves to, mirroring
CppNix, change the semantics to not corrupt it at all in the future,
while having a middle state where it is an error.

This is a largely-rewritten version of
https://github.com/NixOS/nix/pull/13013.

Co-authored-by: rootile <lix@rootile.de>

Fixes: https://github.com/NixOS/nix/issues/12899
Upstream-PR: https://github.com/NixOS/nix/pull/13013
Change-Id: I6a6a6964cdd1a88714952e80c660d1fa57d1f2d8
 2026-04-30 16:10:47 +02:00
Jade Lovelace 89f5974125 fix: inverted null check in nix-prefetch-url 
This was found by edef's harness. Unfortunately this entire module has
no coverage, so I don't really want to write a test for it immediately
as I'd have to harness it (though we have an http server in f2!).
(I think it will be much more fun and rewarding to do so once we get
coverage up and running).

Filed a bug about the missing docs for this too:
https://git.lix.systems/lix-project/lix/issues/1190

Test plan:

```nix
let pkgs = import <nixpkgs> {}; in {
  foo = pkgs.fetchurl {
    url = "https://jade.fyi/robots.txt";
    name = "blah";
    hash = "";
  };
}
```

Before:

```
$ nix-prefetch-url ./foo.nix -A foo
fetching path input 'path:/nix/store/knf5m0famajw2bzbsg72b052iy1ynpck-source'
path is '/nix/store/5rcgj95as1ywr463rva6bky18088ij4x-robots.txt'
1cr11i0ih4m6kkpks9fxil71wg5k2crikah1rfjgy607qzgy8vad
```

After (note correct path name):

```
$ nix-prefetch-url ./foo.nix -A foo
fetching path input 'path:/nix/store/knf5m0famajw2bzbsg72b052iy1ynpck-source'
path is '/nix/store/hc9drd1l3vki3bbaacwhi8j1dr9907vd-blah'
1cr11i0ih4m6kkpks9fxil71wg5k2crikah1rfjgy607qzgy8vad
```

Change-Id: I6d9da1c54c6a398a0a169ff01f3f3dce6a6a6964
 2026-04-30 13:19:23 +00:00
eldritch horrors faaa14a303 tests/libutil: disable guessOrInventPath sockets test 
not very useful, fails in docker and on macos. let's disable it for now.

fixes #1113

Change-Id: If83ea11ff7120eb7fb30601096374d167107573c
 2026-04-28 10:52:57 +00:00
blokyk f8b13b254b libcmd/repl: print backtraces from outer to innermost frame 
before this change, the innermost frame was all the way at the top,
which means that in very nested code you'd have to scroll though
dozens or hundreds of frames before even seeing where you are, because
the last lines (which are immediately visible when the command returns)
would be the nix code's entry point and other outer frames instead,
which is rarely relevant.

this CL reverses this order, so that the innermost frames are the last
ones printed, and thus are immediately visible. (note that this is
already how errors are printed by nix in other contexts anyway, the
debugger's :bt is the only thing that prints the trace in "forward"
order.)

because `DebugState.traces()` use the homegrown `libutil::Generator`
instead of an stdlib container, we have to do that reversing ourselves,
in this case by just storing every element in an `std::list` (in reverse
order) and then traversing it. this is feels kinda dirty, but i don't
know any other way, and it's not exactly performance-critical.

Change-Id: I9f23e40e57f72a251d65a335b3bba3c3f77b935d
 2026-04-28 12:09:13 +02:00
blokyk 68834b7841 tests/f1/repl-characterization: match golden text correctly 
it seems like the repl_characterization tests have not been actually
running correctly for a while, since tests from more than 2 years ago
that should have failed have been in the test suite and OK'd thus far.

the reason why some tests passed when they shouldn't have is because
the AST nodes used by the characterization parser didn't have a correct
comparison operator, so the tests would only fail when one of the output
line was of the wrong *kind*, but not if it had the wrong *value*. this
was fixed by simply adding a default comparator to the TextNode type as
well as every derived class.

i've taken the occasion to fix the tests that were failing now that we
are actually being strict about them. most of these are pretty boring
changes like indentation, but if possible i'd like to get an explicit
ACK from at least one or two people on the core team that these changes
are okay.

for reference, here is the zulip thread that lead to this CL:
  https://zulip.lix.systems/#narrow/channel/8-Testing-.2F-functional2/topic/.E2.9C.94.20repl-characterization.20tests.20not.20failing/near/10276

ps: i had to turn clang-format off around the macro for text node
declarations because it re-formatted the entire macro, causing a lot
of noising and producing uglier code; if this isn't okay or if there's
a better alternative, please let me know!

Change-Id: I1d6d92eb3e9cb705ecfeccd7b9294d6cfd7fe25b
 2026-04-27 23:40:56 +02:00
Yureka 0c63036c7d releng: Adapt for AFNix S3 
Change-Id: I29dbd62dcc70595ba3f2ac2a466a5c26a28aea99
 2026-04-27 11:39:58 +00:00
Qyriad de212be82b upgrade-nix: use --store local, so we don't kill our own access to the daemon 
Tested in a Debian container.

Fixes #1189
Fixes https://git.lix.systems/lix-project/lix-installer/issues/79

Change-Id: Id7de28bf81eb122dd60755622b7816716a6a6964
 2026-04-24 15:56:19 +00:00
Jade Lovelace bb8bc31e58 fix: lint warnings from our clang-tidy enum cast check 
I missed some cases in https://gerrit.lix.systems/c/lix/+/5490, which is
not surprising given that I didn't have a lint for it at the time.

Some of these have JSON deserializers, which is kind of scary (in that
it was UB). Probably no impact, again, due to the lack of
-fstrict-enums, but still, yikes!

Change-Id: I895a8bfdd9972d8ade9fa6f5aa30845d6a6a6964
 2026-04-23 16:09:33 +00:00
skye be34bc0481 testing: migrate and expand import-derivation.sh and warn-import-from-derivation.sh 
This started by noticing that the coverage report generated by cl/5489
showed that the error message for when allow-import-from-derivation is
disabled appeared to be not covered by any tests. Since the existing
tests for IFD were in functional, which is deprecated, I migrated those
into functional2 and added a test for erroring when IFD is disabled.

Change-Id: I565674de3dd693e023ec99d501adc4706a6a6964
 2026-04-22 10:26:15 -04:00
skye eeac529d5e libexpr/primops: Suppress deprecation warning in prim_importNative 
as of cl/5456 prim_importNative default constructs a `Value` in order to
pass as an out parameter to a ValueInitializer function. Default
construction is deprecated, but there isn't a meaningful value to
initialize it to instead. After cl/5357 this will give a deprecation
warning (which in an asanBuild will be an error), so this commit
suppresses that warning for that `Value` construction.

Change-Id: Ie567a6e81b5753672001320e9fc816c56a6a6964
 2026-04-22 08:39:55 -04:00
skye 47c9f73d36 doc/manual: Update signature of primop fn in hacking.md 
This should have been part of cl/5456, but I didn't notice it at the time.

Change-Id: I3afe75c5862ecd3ce2f616a2ae30aff96a6a6964
 2026-04-22 08:39:49 -04:00
skye 84347f94da libexec/kill-user: suppress syscall deprecation warning on darwin 
The existing function is used with good reason as explained by its
comment. This suppresses the warning it creates.

Change-Id: I58b0de3f5854266fa77b6a06b7821a636a6a6964
 2026-04-22 08:37:17 -04:00
skye cda9e01053 nix-eval-jobs: Remove Value default construction from releaseExprTopLevelValue 
Change-Id: I7461323771a50729c66d9e3160c4768f6a6a6964
 2026-04-22 08:37:17 -04:00
skye c40afdea73 libexpr/eval-expr: extract makeThunk function that constructs and increments counters 
Motivated by needing to remove a Value default construction from
`ExprVar::maybeThunk`, so part of #744

Change-Id: Ia12b5b019e49055ebd7ffa0cc39a59c06a6a6964
 2026-04-22 08:37:17 -04:00
vczf dda519baff gitignore: ignore top-level build and outputs directories 
Change-Id: Id51439c15c0216e7353ba48ae1940d3906473d4f
 2026-04-22 11:47:09 +00:00
Jade Lovelace af99ede1fd fix: UB casts to unspecified width enums in proto 
Found by edef's fuzzing. We::jade don't think this is security-relevant
or likely to be mistreated by a compiler, but UB is bad.

https://eel.is/c++draft/expr.static.cast#8

> If the enumeration type does not have a fixed underlying type, the value is unchanged if the original value is within the range of the enumeration values ([dcl.enum]), and otherwise, the behavior is undefined

Notably, the range is defined as the smallest bitfield type that could hold all the values, not any real type which exists: https://eel.is/c++draft/dcl.enum#8.

This is basically a footgun, and I'm writing a clang-tidy check to forbid casting to such types. But first I needed to write clang-tidy testing infrastructure: https://gerrit.lix.systems/c/lix/+/5493.

Change-Id: Ieaaa0fe2a92fd9f24f60761e312741186a6a6964
 2026-04-22 10:52:04 +00:00
eldritch horrors 0a09782cc8 testing: migrate store-ping.sh 
notably the bad-store test previously tested *bash*, not lix. we also
need to extend the list of non-fatal errors, but that's probably fine

Change-Id: I983f4adc8047fe800323099de4037711869ee547
 2026-04-20 16:08:22 +00:00
Felix Uhl e3ee75633e Fix broken user override of NIX_SSL_CERT_FILE in fish shell 
Change-Id: If15fac01104cc413102e417805c2e92aa9007eba
 2026-04-20 13:02:17 +00:00
eldritch horrors d505a477a4 libstore: asyncify RemoteStore::openConnection 
Change-Id: Icc7e705314b9a099687c37859700959139840631
 2026-04-20 09:28:24 +00:00
eldritch horrors f006b41a3c libutil: transfer bind/connect errors from libexec helper 
this should improve error messages when connecting to long socket paths,
and make it at all possible to detect ENOTDIR from connect helpers. this
will be needed to handle daemon sockets that live in directories we have
not previously considered like the protocol name directories we need for
rpc purposes, all of which would have resulted in hard errors previously

Change-Id: I4e16cf09fedd07c93cfdf7b0f151106984507315
 2026-04-20 08:19:17 +00:00
Michael Daniels a5205265c0 manual: correct broken link to Gerrit wiki 
Change-Id: Ida19e872da316c7d3170f0ef2bc60712c31b2ae1
 2026-04-20 03:30:28 +00:00
Michael Daniels 23b2797321 lix/code-generation: correct link to changes for deprecated features 
The link should be to Gerrit, not Forgejo.

Issue introduced in: https://gerrit.lix.systems/c/lix/+/5005

Change-Id: Ibf93f114303f59959621a265196e6eda29b315f9
 2026-04-19 22:47:10 +00:00
skye 0e971cddee libexpr: Delete EvalState::mkSingleDerivedPathString dead function with Value& out param 
Part of #1136, I believe this cleans up the final instance of `Value&`
used as an out parameter.

Change-Id: I40153b1f11d5ca9af06cd427f22896d06a6a6964
 2026-04-19 12:45:56 -04:00
piegames 5afda956a7 tests/functional2/lang: Some refactorings 
I did a pass through all files, fixing bits that have annoyed me here
and there based on vibes. Roughly:

- Replaced some out values like bools or strings with lists where
appropriate. Those tests were likely older than the value printer
- Added some parser tests where it makes sense
- Gently touched some formatting woes, while trying to not cause too
much diff noise
- Removed some dead `with (import ./lib.nix);` code

Change-Id: I8c40b2110f0b7799f68ae38ba61f049c5f1f6ee8
 2026-04-19 16:07:54 +02:00
rootile ae4b3d585d f2/nix: provide a high-level wrapper for calling builtins 
Change-Id: I357bd7630b96e55d000fbff1032efcf8c98dfb7b
 2026-04-19 10:10:37 +00:00
rootile 62b728e766 f2/nix: provide a function to serialise python objects to nix code 
Change-Id: I863029263edbc701a18b18da4b58fcb35f575614
 2026-04-19 10:10:22 +00:00
skye 1e986c81ab Add test for duplicate JSON keys for builtins.fromJSON 
Related to #1162
Currently if JSON with duplicate object keys is passed into `builtins.toJSON`,
it will silently drop all but the last value, keeping only the last.
This may be surprising, but as this has been consistent reliable behavior that
users might depend on, we should test for it.

Change-Id: Icc2adefabb161530e7cbfa7330919bab6a6a6964
 2026-04-18 17:04:48 -04:00
eldritch horrors 36e784470c f2: turn Nix.daemon into a fixture 
that way we can parametrize it over the list of protocols we have

Change-Id: Ie3fc267ade9c2ca74c163347f4fa58f09a317f40
 2026-04-18 12:16:28 +00:00
Felix Uhl 32f48682de Fix test_doctor_shows_trust on macOS outside sandbox 
Change-Id: I1fd456a8c5563ba63f795bc4e41811ebeaec9c73
 2026-04-17 21:36:17 +02:00
rootile 15c95b95d6 tests/f2/nix: remove obsolete param documentation 
Change-Id: I103d529ad0dd4ad482f9facb7e44baf61ab700d1
 2026-04-16 15:05:05 +00:00
piegames c1f75860d6 builtins.flakeRefToString: Force the arguments 
Without this, the primop chokes on any thunks on attributes passed in
the attrset. It even is the reason why the test contained `builtins.seq`
to work around this. Supposedly, this might have been an intentional
restriction and changing this might break things in ways I cannot forsee
due to not knowing much about Flakes, however the status quo is equally
broken:

- The error message looks like an internal error and not like some
explicitly forbidden invariant violation.
- Seemingly simple syntax literals like "-1" compile to "__sub 0 1" and
thus create a thunk which then fails, which is utterly confusing ("why
does 1 work but not -1?")
- This is a stark violation of the principle of least surprise.
- Thunking relies on maybeThunk and thunk inlining optimizations, thus
not forcing thunks turns operational details of the evaluator into
language-observable behavior. That's bad.

I am changing this now regardless of the risk of breakage, because the
bytecode evaluator will have different thunk inlining optimizations and
thus inevitably cause mismatches in behavior anyways.

Change-Id: Ifc45c4d2900e40822383670b28e4e50ab8af317a
 2026-04-16 15:53:38 +02:00
blokyk 526bcac44a docs: clarify which characters are allowed in a derivation name 
Although the fact that some characters are illegal in derivation names
was referenced in some parts of the docs, the actual list was not
documented anywhere, and the only source of truth seems to be the
[store source code](lix/libstore/path.cc).

This adds an admonition in the `derivation` docs that specifies which
characters and strings are actually allowed (especially since some
of them can be somewhat surprising, like the string "..-foo" not
being a valid name because of `..`).

Change-Id: Ibc4b4a141e262c21306ce9c0392f92cf0abe610a
 2026-04-14 14:52:08 +00:00
blokyk 08d1e0c140 doc: fix incorrect docs for repl-overlays argument values 
the docs say that the arguments for an overlay is info/prev/final, but
it's actually info/final/prev. this fixes the docs both in the nix.conf
manual as well as in the `nix repl` help text

Change-Id: I3de5c122e7e77cc3f4e4550e3741d6a6e906ea20
 2026-04-14 11:07:52 +02:00
Maximilian Bosch ebf48c14f0 flake: static build in CI 
That way we can automatically push the static build (compressed as
tarball) into the AFNix S3 bucket and have it updated on each update to
main.

Change-Id: I1002727ab12c0ab6b95c8d37ae4620996607d7b7
 2026-04-13 22:46:30 +01:00
skye 01f2fabe14 doc: Update Nix Resources link in Quick Start chapter of manual (#1179) 
This applies the suggested change in #1179, replacing a dead link with a
link to the content's new location. Closes #1179.

Change-Id: I543e3308516218e224e495b8cf14a7c06a6a6964
 2026-04-13 14:18:09 -04:00
Yureka 9899ed29cc flake: fix condition for using lowdown 3.0 
Change-Id: I6832c249da77a80b4e1e79015976156ce1eb1b0b
 2026-04-11 22:42:14 +02:00
skye e79278b4fc Check for throw while evaluating throw message 
This is a small tweak to the logic added in cl/1511 to detect explicit
throws when printing stack traces. Now when deciding whether to print
"caused by explicit throw", it checks not only that the error is a
ThrowError and that we are in a throw, but also that the ThrowError was
thrown by *this* throw, and not by another while in the process of
evaluating this throw's operand.

It turns this:

```
let
  set = {
    inner = throw "nested throw";
  };
in
throw set.inner

error:
       … caused by explicit throw
         at /pwd/in.nix:6:1:
            5| in
            6| throw set.inner
             | ^
            7|

       … while evaluating the attribute 'inner'
         at /pwd/in.nix:3:5:
            2|   set = {
            3|     inner = throw "nested throw";
             |     ^
            4|   };

       … caused by explicit throw
         at /pwd/in.nix:3:13:
            2|   set = {
            3|     inner = throw "nested throw";
             |             ^
            4|   };

       error: nested throw
```

into this:

```
error:
       … while calling the 'throw' builtin
         at /pwd/in.nix:6:1:
            5| in
            6| throw set.inner
             | ^
            7|

       … while evaluating the attribute 'inner'
         at /pwd/in.nix:3:5:
            2|   set = {
            3|     inner = throw "nested throw";
             |     ^
            4|   };

       … caused by explicit throw
         at /pwd/in.nix:3:13:
            2|   set = {
            3|     inner = throw "nested throw";
             |             ^
            4|   };

       error: nested throw
```

Notice the difference in the top context frame. Before it incorrectly
attributed the throw error to the both throws instead of just the one
that actually threw.

Change-Id: If3b0b8311f1ae2ff1471e260fe59d9166a6a6964
 2026-04-11 12:46:24 -04:00
Yureka 9fea1b816f flake: allow using nixpkgs lowdown 3.0 
Admittedly, I do not understand what the comment means:

> As soon as Nixpkgs updates to >= 3.0.0, change to lowdown_2_0!

But this should work fine for nixpkgs versions providing lowdown 2.x or
3.x, and not cause rebuilds of the nixpkgs Lix/Nix derivations.

Change-Id: I6fb9e775335b4dbc23add76b98106970956e19fb
 2026-04-11 14:17:37 +00:00
Florian Klink 6c7ccc2588 libcstore: Fix null deref in writeDebugInfo for non-directory NARs 
When index-debug-info is enabled and the store path being copied is a
regular file (not a directory), std::get_if<nar_index::Directory>
returns nullptr since the NAR root is a File variant. The loop then
immediately dereferences buildIdDir->contents on the null pointer,
causing a segfault.

Add a null check at the top of the loop to break early when the NAR
root is not a directory.

Change-Id: I3a6e792b84cc12c837ecaddf4fee889e1bcb6397
 2026-04-11 12:33:46 +00:00
skye fce5777f9c libexpr/primops: Migrate primops to return Values 
A significant part of #1136.
This is a rather large cl, because all of the primops need to be changed
together.

Change-Id: I7d92698d5344bd6186ee4fa47f5c21966a6a6964
 2026-04-07 23:38:08 -04:00
skye bc9fb560ac libexpr/primops: Migrate addPath to return a Value 
Part of #1136

Change-Id: I079da96c1ed7e7396e5f71b444e4e7b66a6a6964
 2026-04-07 23:36:12 -04:00
skye 50aeb770e5 libexpr/primops: Migrate derivationStrictInternal to return a Value 
Part of #1136

Change-Id: If00bc51c7ebd5ccd99a883788bc1ec506a6a6964
 2026-04-07 23:35:12 -04:00
skye fe38b58e50 libexpr/primops: Migrate helper fn fetch to return a Value 
Part of #1136

Change-Id: I5b21998d437aa82fb89e75b0b645269a6a6a6964
 2026-04-07 22:40:04 -04:00
skye a1f52a1ce6 libexpr/primops: Migrate helper fn fetchTree to return a Value 
Part of #1136

Change-Id: Icb79dfc99f530a2965199954ce784e166a6a6964
 2026-04-07 22:39:30 -04:00
skye 6640ba572f libexpr/primops: Migrate helper fn anyorall to return a Value 
Part of #1136

Change-Id: Id1aa3225d7c337dee72d070ef764c6006a6a6964
 2026-04-07 22:38:52 -04:00
skye aed5b5cbce libexpr: Migrate EvalState::concatLists to return a Value 
Part of #1136

Change-Id: Ie28ee8456191b3da20d53e95ea49e0ec6a6a6964
 2026-04-07 22:38:24 -04:00
skye c01bd37a8d libexpr/primops: Migrate helper fn elemAt to return a Value 
Part of #1136

Change-Id: Ib3c7040c7df729737643d3a9b833773d6a6a6964
 2026-04-07 22:38:02 -04:00
skye 658404c2a0 libexpr/json-to-value: Migrate parseJSON to return a Value 
Part of #1136

Change-Id: I5ac23ce461a106360b150e64a6bc0f2f6a6a6964
 2026-04-07 22:37:44 -04:00
skye bb9f9cf553 lixexpr/primops: Migrate import helper fn to return a Value 
Part of #1136

Change-Id: Ia2cdd540a105ae874430c0f6ba456d4d6a6a6964
 2026-04-07 22:37:25 -04:00
skye 86126d6c89 libexpr: Migrate EvalPaths::allowAndSetStorePathString to return a Value 
Part of #1136

Change-Id: Icf16109a63f1b8114e0af9db0a9213c46a6a6964
 2026-04-07 22:35:12 -04:00
Tom Hubrecht cbeb4fcd69 repl: Fix the use command 
The culprit was a space added at the wrong place, which messed with c++
raw strings.

Fixes #1178

Change-Id: Ic1e09cb7215d9a6dd2d13fd92242649b0e1fcd13
 2026-04-07 11:02:53 +02:00
Qyriad d0190cff6f improve justfile, and add more docs about it 
Change-Id: I00286276dc0ce17a5877a7bd13bb254d6a6a6964
 2026-03-28 19:08:37 +01:00
rootile 76499d36ea f2/nix: rename _serialise to _serialise_config 
Change-Id: Icaa10bf615ae5fa6d3a3957f74055c9d76f67483
 2026-03-28 14:06:26 +01:00
rootile 3bc59b6e0a f2/testlib: fix typechecking for None values 
Change-Id: Ie744be23d2b7963d8cd8f80cfeae051249e34d34
 2026-03-28 14:06:26 +01:00
blokyk 001e3fde8f docs: add section about overlays in nix repl --help 
I just copied and slightly edited the `man nix.conf(5)` docs about
`repl-overlays` to the nix repl help text (but cut out the more
complicated example and redirecting to the nix.conf docs for more
info).

Fixes #303

Change-Id: I85efaef606d8779ac66fe72cd3947d663f33fa44
 2026-03-27 10:41:45 +00:00
skye 03ab7b4a76 package.nix: Remove unused linuxPackages input 
This input was added in cl/2884, but was unused even then. The
`linuxPackages` within `buildPackages` is used instead.

Change-Id: I71c522ef683aa098eac0b356b22007ba6a6a6964
 2026-03-24 11:43:30 -04:00
skye b94d615baa libexpr/json-to-value: Replace add with addValue and define TopLevelJSONValue 
Calls to JSONState::add() always followed an assignment to
JSONState::value(), resulting in the new value going through
JSONState::v for no good reason. The only use of `v` outside of a
pointless stepping stone for `add` was to store the final top-level
Value before it is read out by JSONSax::result(), so it really only
makes sense for the top level JSONState to contain Value field.

Change-Id: I1758c7b770eb4b0c122e501b764dd42b6a6a6964
 2026-03-23 19:37:27 -04:00
piegames 35b776540f libutil/LinearMap: Expose rbegin and rend functions 
Change-Id: I9a4350cc147c98c785b26fbef6c235cd5c48a4cc
 2026-03-23 14:47:39 +01:00
piegames d04fcb57fd libexpr: Don't call setName on dynamic attrs 
And also document in great detail why this is a wrong thing to do

Change-Id: Ifd1331ee7ee4e05322593ada801bab1c3ea8d349
 2026-03-23 14:47:39 +01:00
piegames 93edf577b7 libexpr/eval: Factor out attrs updating code into dedicated helper function 
The bytecode evaluator can use it 1:1

Change-Id: I9791a749231fe42e0e534853d3f3cce91913346c
 2026-03-23 14:47:39 +01:00
Maximilian Bosch 4960a217fe flake: fix attr-path of build-lowdown_2_0 job in hydra jobs 
We would've had `build-lowdown_2_0.aarch64-linux.aarch64-linux`
otherwise. This is a bit of a problem because my way of generating a
list of constituents for Hydra's per-architecture[1] release job stops
recursing once it encounters a system.

The alternative would be to switch `hydraJobs` to `<system>.x.y` or
switching Hydra to "legacy" jobsets. For the latter I'd prefer to do the
same for Buildkite such that we don't have diverging things to build
depending on pre/post-merge CI.

[1] per-architecute because the trusted AFNix builders don't support all
    the architectures we support in Lix and with Flakes there's no way
    of parameterizing the list of supported systems.

Change-Id: I33f31260caf86ed5bb0f728770ca3cf1c00adf31
 2026-03-22 12:30:34 +01:00
Maximilian Bosch 0488a0181d libcmd: allow setting nested attributes via --arg/--argstr 
Closes #496

When running

    nix-build ../nixpkgs --arg config.allowUnfree true -A hello-unfree

the package `hello-unfree` is now built rather than getting an
eval-error rejecting instantiation. This is because `config.allowUnfree`
is now interpreted as nested attribute-set declaration, similar to how
it's done in `nix repl`.

To prevent sudden breakage, this behavior was carefully deprecated with
Nix throwing an error if the identifier for `--arg` is not a pure
identifier, but an expression as above.

Any kind of merging is rejected. I.e. doing

    nix-build ../nixpkgs --arg config '{cudaSupport = true;}' --arg config.allowUnfree true

is prohibited. That way we don't have to think about merge semantics for
cases like this (or even worse `--arg config 'rec { ... }'`). Another
nice side-effect of this is that we don't need to create an EvalState to
force the values and implement merging.

Change-Id: I8b560883a4468a3f32f915764b08f5fdd8fe71bb
 2026-03-21 23:16:30 +01:00
Maximilian Bosch 2a11984a58 libexpr: allow empty attr-names in parseAttrPath if they are quoted 
While it doesn't make sense to have `foo..bar`, the attribute-path
`foo."".bar` is valid and shouldn't throw.

Change-Id: Ifcddaad6233c6ba8f17cb5c953c2101d276dfeb6
 2026-03-21 23:16:30 +01:00
Maximilian Bosch c5d21b36c5 libcmd: turn autoArgs into a map that points to std::variant 
This is a little more elegant and easier to reason about than prefixing
strings with whatever type the rest of the string is.

Change-Id: I7769535303dcb9f67b79e89bef162beec990e2a0
 2026-03-21 23:16:29 +01:00
blokyk 66d702d28d libexpr/primops: make break force its argument 
previously, `builtins.break` didn't force its argument, resulting in
a value wrapped with `break` being opaque to most builtins if not
also wrapped with `seq`. see [lix-project/lix#1165] for more details
on what this can break.

this tiny fix just adds a call to `forceValue` inside `prim_break`,
but unfortunately this "breaks" a few existing tests because it
changes the call stack; those tests' golden outputs have been adjusted
without modifying their intended purpose.

Fixes #1165

[lix-project/lix#1165]: https://git.lix.systems/lix-project/lix/issues/1165

Change-Id: I5fe4ee3ff28b38aaf924125b8978130812e58fef
 2026-03-21 18:38:43 +01:00
Maximilian Bosch af2ef44e76 flake: add release job to Hydra 
This job is used to indicate that all relevant Hydra jobs of an
architecture have built. The idea is to build some CD mechanism on
AFNix's Hydra to e.g. auto-update the nightly manual.

See https://hydra.afnix.fr/jobset/lix/demo for the current setup.

Change-Id: I41724c5884a068bbe41407ab30f8edf8e4914001
 2026-03-21 13:00:31 +01:00
blokyk daadfed9ae libcmd/repl: allow :st argument to be relative to current stack index 
See [lix-project/lix#1156], but basically currently the `:st <n>`
debugger command doesn't allow any negative indices, and putting a plus
sign in front of the arg doesn't change anything; thus, we can exploit
that "design space" to allow users to move between different stack
frames easily, by simply prepending their arg with a +/- sign.

The actual behavior is little more nuanced when you account for errors:
as suggested by @pennae (thanks! :), when the user inputs an offset that
would result in an invalid frame index, the debugger instead clamps it
to the closest bound (i.e. 0 for negative offsets, $maxFrame for
positive ones) and just prints a warning.

Fixes #1156

[lix-project/lix#1156]: https://git.lix.systems/lix-project/lix/issues/1156

Change-Id: I02a0cdb6aaebbdb0515308880a3bf9c0d2fcd25e
 2026-03-20 18:22:24 +01:00
skye 774f957599 libexpr/attr-set.hh: Deprecate Attr default constructor 
The default constructor of Attr default constructs a Value, which is
itself deprecated. Therefore the default constructor of Attr must
itself be either deprecated in turn or removed. The default can't be
trivially deleted because Bindings::EMPTY depends on the default
constructor of Bindings which depends on Attr's default constructor, so
I'm settling for deprecating it for now.

Part of work towards #744

Change-Id: Ie34b08788780615c5478a0354122530b6a6a6964
 2026-03-18 22:03:33 -04:00
skye 83bca23d4a libexpr/primops: Avoid Value default construction in primop_removeAttrs 
This makes the removal vector a vector of Symbols instead of Attrs, and
uses a custom Compare to still be able to std::set_difference them.

std::ranges::set_difference **should** be the perfect function for this,
but because for some reason it spuriously requires
`std::indirectly_copyable<I2, Out>`, I can't use it here. This
defficiency has bee recognized before [here](https://github.com/cplusplus/papers/issues/1021),
but no one has driven the fix forward.

Part of #744

Change-Id: I6d2c016ea41e033bf38836f859541b506a6a6964
 2026-03-18 22:03:33 -04:00
blokyk 8294cd534b docs: fix indent of builtins and nix.conf descriptions in manual 
The manual for the builtins and nix.conf currently has inconsistent
indentation, which causes some of the descriptions to end up being
partially treatedas code blocks in markdown (and thus the manual).

This was simply caused by the template string for the docs having
too much indentation before the description is inserted, so this fixes
that 16-bytes mistake.

Change-Id: Ia264e3b1abb20430109029d07a2d2b0a1a726bd4
 2026-03-18 14:33:21 +01:00
blokykandeldritch horrors 6e1e76f10f libcmd/repl: print error for invalid :st argument 
currently, the `:st <n>` command in the debugger will simply silently
fail if the argument cannot be converted to an integer, or if it falls
outside the range of valid stack indices. this isn't too big of problem,
but it can be nicer to tell the user something went wrong, rather than
not give them any output and having them guess (esp. in the second case).

this commit adds two errors, one for each case:

  1. the argument is not actually an integer, or is outside INT_MIN/MAX
     -> "argument '%arg' is not a valid integer"

  2. the argument is an integer outside the range of stack traces
     -> "stack index must be between 0 and %max_frame, but was %arg"

Change-Id: I8109feeede79a9ad3db9ee7dc95d37e7dd19741a
 2026-03-18 13:15:13 +00:00
sterni f7d7b5d93f libcmd: remove support for lowdown < 1.4.0 
NixOS 25.05 which distributed lowdown < 1.4 has been EOL for a bit now,
25.11 ships lowdown 2.0.4.

Dropping support means we can tweak the lowdown options for terminal
output which have been added in 1.4.0 without having diverging behavior
in possible builds of Lix.

Change-Id: Icae97cf5e9e680766b8a6f4e85514f4c4625d1dd
 2026-03-17 21:38:42 +00:00
sterni af0390c27b libcmd: add support for lowdown >= 3.0.0 
lowdown 3.0.0 merged some flags into one to save on bits and did not add
any aliases for backward compatibility.

As with the changes for lowdown >= 1.4, we define a preprocessor flag to
gate the changes on and add a job to CI to ensure that lowdown < 3.0
keeps working (which is used by NixOS 25.11).

Unfortunately, we need to jump through some hoops to prevent nix and lix
from upstream Nixpkgs from being rebuilt due to a changed lowdown. Since
both implementation's packaging in Nixpkgs has their own package set /
fix point now, we can't simply inherit them from `prev` since they will
always be (re-)computed from the `final` fix point. As a consequence,
we need to expose our changed lowdown version at a non-default attribute
or break the builds of Nixpkgs derivations we test against.

Change-Id: I20a3e2fdaa05906f032ff66911c42867557fdd11
 2026-03-17 21:38:35 +00:00
skye cf5e5f599e libexpr/nixexpr: Move backing fields of ExprLiteral subclasses into new 
base classes

`ExprLiteral::v` needs to be initialized in ExprLiteral's constructor,
but at the time that it gets initialized subclass fields don't yet exist
so it can't reference them. Previously, `v` was default constructed and
then later assigned a proper value, but this creates a problem when
attempting to remove all default constructions of `Value` from the
codebase. This commit moves the backing fields of each of the subclasses
into new base classes, which exist just to make sure they get
initialized before the `ExprLiteral` base class.

Work towards #744

Change-Id: Ic6d24cab474460b113f2fbcc8d92ad266a6a6964
 2026-03-17 15:48:12 -04:00
skye 51c6d6a2e8 libexpr: Remove various default constructions of Values 
Progress towards #744

Change-Id: I138ecf7ab712ea570ecbf506c7b6f6be6a6a6964
 2026-03-17 15:48:12 -04:00
skye 8b99b75698 libexpr: Push Values onto vectors instead of default constructing 
ahead of time

Rather than creating fixed size vectors of default constructed Values
before assigning to those elements, reserve the desired capacity and
then push created values onto the vector. This avoids default
constructing any Values.

Part of fixing #744

Change-Id: I36eff4275a893b181eaf3ce145b1ee446a6a6964
 2026-03-17 15:48:12 -04:00
skye 179164cffc libexpr: Migrate EvalState::mkPos to return a Value 
Part of #1136 and progress towards #744

Change-Id: I31d0169077954ea82c9c7d341afdccd76a6a6964
 2026-03-17 15:48:12 -04:00
skye 810a3bad11 libexpr: replace BindingsBuilder::alloc with insert 
Progress towards #744
`alloc` default constructed a `Value` which is a problem because the
defaut constructor of `Value` is deprecated

Change-Id: I789cba20bd98728758395080a3a9cf6e6a6a6964
 2026-03-17 15:48:12 -04:00
skye f0891b440f libexpr/primops/fromTOML: Migrate visit lambda to return a value 
Necessary step to replace BindingsBuilder::alloc uses with insert

Arguably part of #1136

Change-Id: I72acdd1676cf2dda69bce39e9b7feb656a6a6964
 2026-03-17 15:48:12 -04:00
skye 570357c733 libexpr: Replace Value::mkNull with Value::VNULL 
A small step towards fixing #744

Change-Id: If8304d4de20bae07b33eb7825f781e0f6a6a6964
 2026-03-17 15:48:12 -04:00
Yurekaandeldritch horrors 96db7c79cf lix-doc: remove rust_dynamic_args 
This causes a build error with lto, and according to Jade is not
strictly needed anymore.

Change-Id: I41e53a57f40711061effe08f78545011a4b51754
 2026-03-17 15:10:18 +01:00
eldritch horrors 022e43aa7f fix the static build 
- launch-builder-linux.cc was missing an include for musl
  and used function that are not defined in the launchers
- musl caches pids used for raise, breaking sandbox setup
- the mtls contrib plugin won't build, didn't try fixing,
  static builds can't really use plugins reliably anyway.

Change-Id: I5ab1664e45ea977e5bcf05e41d825e6014e62146
 2026-03-17 14:24:13 +01:00
Linus Heckemann f87d753987 libexpr: print flake config warning to stderr 
Fixes #1155

Change-Id: Ie63f9200f7c06b1eec6c52518d6f523f6a6a6964
 2026-03-17 07:10:00 +00:00
piegames 35b46d1fcb libexpr/value: Add constants for the empty set and null 
Change-Id: I4b673cbb81b06f1c415a0059cf1238d2d329725e
 2026-03-16 22:21:35 +01:00
piegames 3c0fcf6836 libutil/chunked-vector: Add default constructor 
Change-Id: I75a2d0daac7ddddbe4bb3791f642e15cfbe5f732
 2026-03-16 22:21:35 +01:00
rootile bcc9350bfe tests/functional2: migrate simple.sh 
Change-Id: I4366ce7d877935df98e35d733650c60f59478ace
 2026-03-16 16:52:14 +00:00
skye c452341b39 libexpr: Replace Value::mkExternal with constructor call 
Change-Id: I7f92a91f6510be8948dfe0765b3fec6f6a6a6964
 2026-03-14 13:55:59 -04:00
skye 3c9f42443e libexpr: Migrate mkStorePathString to return a Value 
part of #1136

Change-Id: I32ba0f8ef369a82ceb8f75816a2ce6dc6a6a6964
 2026-03-14 13:55:59 -04:00
skye 72a456210e libexpr: Migrate makePositionThunks to return Value tuple 
part of #1136 and a step towards #744

Change-Id: Ib2ccea4c098bbb66eaa47645b6fa59e56a6a6964
 2026-03-14 13:55:59 -04:00
skye 3a18ed52e2 libexpr: Migrate EvalState::mkOutputString to return a Value 
part of #1136 and a small step toward resolving #744

Change-Id: I4e1602b37c40517a92cbab8fe4074d7e6a6a6964
 2026-03-14 13:55:59 -04:00
sternenseemann b72f5e9f9d flake: remove nixUnstable attr which has been removed upstream 
Change-Id: I869941c998daa50d4deb0e4ef9740d10919e0d82
 2026-03-14 14:04:12 +01:00
Jade Lovelaceandeldritch horrors 2cea406121 version.json: begin the 2.96 series 
Change-Id: I21c37e20fc2e5786367aa9b6e5ebb7ba12eb8b6c
 2026-03-13 22:59:53 +01:00
Jade Lovelace c08f2336d0 release: merge release 2.95.0 back to mainline 
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh

Change-Id: Ia7a63807681389cf5ccb9d07539b880697a88be5
 2026-03-13 09:06:50 -07:00
561 changed files with 16352 additions and 7628 deletions
Expand all files Collapse all files
.cargo/config.toml
+14
View File
@@ -0,0 +1,14 @@
[target.'cfg(true)']
rustflags = [
# rustc will pass `-nodefaultlibs` without this, but we need the C++ standard library.
'-Cdefault-linker-libraries=yes',
]
[target.'cfg(target_env = "musl")']
rustflags = [
'-Cdefault-linker-libraries=yes',
# musl, at least in Nixpkgs, is not compiled with -fPIE.
# XXX: nevermind? as of Nixpkgs 26.05??
# Oh gods do we need to gate this??
#'-Crelocation-model=static',
]
.gitignore
+2 -1
View File
@@ -1,4 +1,5 @@
outputs/ /build
/outputs
# GNU Global # GNU Global
GPATH GPATH
Cargo.lock
Generated
+878 -4
View File
@@ -2,18 +2,320 @@
# It is not intended for manual editing. # It is not intended for manual editing.
version = 4 version = 4
[[package]]
name = "aho-corasick"
version = "1.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ddd31a130427c27518df266943a5308ed92d4b226cc639f5a8f1002816174301"
dependencies = [
"memchr",
]
[[package]]
name = "allocator-api2"
version = "0.2.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
[[package]]
name = "anstream"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "824a212faf96e9acacdbd09febd34438f8f711fb84e09a8916013cd7815ca28d"
dependencies = [
"anstyle",
"anstyle-parse",
"anstyle-query",
"anstyle-wincon",
"colorchoice",
"is_terminal_polyfill",
"utf8parse",
]
[[package]]
name = "anstyle"
version = "1.0.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000"
[[package]]
name = "anstyle-parse"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52ce7f38b242319f7cabaa6813055467063ecdc9d355bbb4ce0c68908cd8130e"
dependencies = [
"utf8parse",
]
[[package]]
name = "anstyle-query"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc"
dependencies = [
"windows-sys",
]
[[package]]
name = "anstyle-wincon"
version = "3.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d"
dependencies = [
"anstyle",
"once_cell_polyfill",
"windows-sys",
]
[[package]]
name = "ar_archive_writer"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4087686b4b0a3427190bae57a1d9a478dbb2d40c5dc1bd6e2b6d797913bdd348"
dependencies = [
"object",
]
[[package]]
name = "ariadne"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72fe02fc62033df9ba41cba57ee19acf5e742511a140c7dbc3a873e19a19a1bd"
dependencies = [
"unicode-width 0.1.14",
"yansi",
]
[[package]]
name = "askama"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b79091df18a97caea757e28cd2d5fda49c6cd4bd01ddffd7ff01ace0c0ad2c28"
dependencies = [
"askama_derive",
"askama_escape",
"humansize",
"num-traits",
"percent-encoding",
]
[[package]]
name = "askama_derive"
version = "0.12.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "19fe8d6cb13c4714962c072ea496f3392015f0989b1a2847bb4b2d9effd71d83"
dependencies = [
"askama_parser",
"basic-toml",
"mime",
"mime_guess",
"proc-macro2",
"quote",
"serde",
"syn",
]
[[package]]
name = "askama_escape"
version = "0.10.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "619743e34b5ba4e9703bba34deac3427c72507c7159f5fd030aea8cac0cfe341"
[[package]]
name = "askama_parser"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "acb1161c6b64d1c3d83108213c2a2533a342ac225aabd0bda218278c2ddb00c0"
dependencies = [
"nom",
]
[[package]]
name = "autocfg"
version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
[[package]]
name = "basic-toml"
version = "0.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba62675e8242a4c4e806d12f11d136e626e6c8361d6b829310732241652a178a"
dependencies = [
"serde",
]
[[package]]
name = "bitflags"
version = "2.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4388bee8683e3d04af747c73422af53102d2bd24d9eadb6cbc100baef4b43f8"
[[package]]
name = "block-buffer"
version = "0.10.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71"
dependencies = [
"generic-array",
]
[[package]]
name = "cc"
version = "1.2.65"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e228eec9be7c17ccb640b59b36a5cd805ea2a564a4c5e162c2f659fea30d3b96"
dependencies = [
"find-msvc-tools",
"shlex",
]
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "cfg_aliases"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724"
[[package]]
name = "chumsky"
version = "1.0.0-alpha.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0e82d74e6c83060ec269fe9e0d408d6de4a1645d525f9a0bbbb841ba4efd91ac"
dependencies = [
"hashbrown 0.15.5",
"regex-automata 0.3.9",
"serde",
"stacker",
"unicode-ident",
"unicode-segmentation",
]
[[package]]
name = "clap"
version = "4.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ddb117e43bbf7dacf0a4190fef4d345b9bad68dfc649cb349e7d17d28428e51"
dependencies = [
"clap_builder",
"clap_derive",
]
[[package]]
name = "clap_builder"
version = "4.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "714a53001bf66416adb0e2ef5ac857140e7dc3a0c48fb28b2f10762fc4b5069f"
dependencies = [
"anstream",
"anstyle",
"clap_lex",
"strsim",
]
[[package]]
name = "clap_derive"
version = "4.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2ce8604710f6733aa641a2b3731eaa1e8b3d9973d5e3565da11800813f997a9"
dependencies = [
"heck",
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "clap_lex"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9"
[[package]]
name = "clipboard-win"
version = "5.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bde03770d3df201d4fb868f2c9c59e66a3e4e2bd06692a0fe701e7103c7e84d4"
dependencies = [
"error-code",
]
[[package]]
name = "colorchoice"
version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
[[package]] [[package]]
name = "countme" name = "countme"
version = "3.0.1" version = "3.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7704b5fdd17b18ae31c4c1da5a2e0305a2bf17b5249300a9ee9ed7b72114c636" checksum = "7704b5fdd17b18ae31c4c1da5a2e0305a2bf17b5249300a9ee9ed7b72114c636"
[[package]]
name = "cpufeatures"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280"
dependencies = [
"libc",
]
[[package]]
name = "crypto-common"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
dependencies = [
"generic-array",
"typenum",
]
[[package]]
name = "digest"
version = "0.10.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292"
dependencies = [
"block-buffer",
"crypto-common",
]
[[package]] [[package]]
name = "dissimilar" name = "dissimilar"
version = "1.0.9" version = "1.0.9"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59f8e79d1fbf76bdfbde321e902714bf6c49df88a7dda6fc682fc2979226962d" checksum = "59f8e79d1fbf76bdfbde321e902714bf6c49df88a7dda6fc682fc2979226962d"
[[package]]
name = "either"
version = "1.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "91622ff5e7162018101f2fea40d6ebf4a78bbe5a49736a2020649edf9693679e"
[[package]]
name = "endian-type"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "869b0adbda23651a9c5c0c3d270aac9fcb52e8622a8f2b17e57802d7791962f2"
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "error-code"
version = "3.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dea2df4cf52843e0452895c455a1a2cfbb842a1e7329671acf418fdc53ed4c59"
[[package]] [[package]]
name = "expect-test" name = "expect-test"
version = "1.5.0" version = "1.5.0"
@@ -24,12 +326,139 @@ dependencies = [
"once_cell", "once_cell",
] ]
[[package]]
name = "find-msvc-tools"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582"
[[package]]
name = "foldhash"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2"
[[package]]
name = "generic-array"
version = "0.14.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a"
dependencies = [
"typenum",
"version_check",
]
[[package]] [[package]]
name = "hashbrown" name = "hashbrown"
version = "0.14.5" version = "0.14.5"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1" checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1"
[[package]]
name = "hashbrown"
version = "0.15.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1"
dependencies = [
"allocator-api2",
"equivalent",
"foldhash",
]
[[package]]
name = "hashbrown"
version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
[[package]]
name = "heck"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "hex"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70"
[[package]]
name = "home"
version = "0.5.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cc627f471c528ff0c4a49e1d5e60450c8f6461dd6d10ba9dcd3a61d3dff7728d"
dependencies = [
"windows-sys",
]
[[package]]
name = "humansize"
version = "2.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6cb51c9a029ddc91b07a787f1d86b53ccfa49b0e86688c946ebe8d3555685dd7"
dependencies = [
"libm",
]
[[package]]
name = "indexmap"
version = "2.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d466e9454f08e4a911e14806c24e16fba1b4c121d1ea474396f396069cf949d9"
dependencies = [
"equivalent",
"hashbrown 0.17.1",
]
[[package]]
name = "is_terminal_polyfill"
version = "1.70.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695"
[[package]]
name = "itertools"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b1c173a5686ce8bfa551b3563d0c2170bf24ca44da99c7ca4bfdab5418c3fe57"
dependencies = [
"either",
]
[[package]]
name = "libc"
version = "0.2.186"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66"
[[package]]
name = "libm"
version = "0.2.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6d2cec3eae94f9f509c767b45932f1ada8350c4bdb85af2fcab4a3c14807981"
[[package]]
name = "licxxbridge"
version = "0.0.0"
dependencies = [
"clap",
"zngur",
]
[[package]]
name = "lix"
version = "0.0.0"
dependencies = [
"lix-doc",
"pkg-config",
"regex",
"rootcause",
"rustyline",
"rustyline-derive",
"zngur",
]
[[package]] [[package]]
name = "lix-doc" name = "lix-doc"
version = "0.0.1" version = "0.0.1"
@@ -40,8 +469,87 @@ dependencies = [
] ]
[[package]] [[package]]
name = "lixutil-rs" name = "log"
version = "0.0.0" version = "0.4.33"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "memchr"
version = "2.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "88904434abc2901f197fe8cc55f0445e7ded921dba5911dad2e2b39b48e663c4"
[[package]]
name = "mime"
version = "0.3.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6877bb514081ee2a7ff5ef9de3281f14a4dd4bceac4c09388074a6b5df8a139a"
[[package]]
name = "mime_guess"
version = "2.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f7c44f8e672c00fe5308fa235f821cb4198414e1c77935c1ab6948d3fd78550e"
dependencies = [
"mime",
"unicase",
]
[[package]]
name = "minimal-lexical"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a"
[[package]]
name = "nibble_vec"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "77a5d83df9f36fe23f0c3648c6bbb8b0298bb5f1939c8f2704431371f4b84d43"
dependencies = [
"smallvec",
]
[[package]]
name = "nix"
version = "0.31.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf20d2fde8ff38632c426f1165ed7436270b44f199fc55284c38276f9db47c3d"
dependencies = [
"bitflags",
"cfg-if",
"cfg_aliases",
"libc",
]
[[package]]
name = "nom"
version = "7.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a"
dependencies = [
"memchr",
"minimal-lexical",
]
[[package]]
name = "num-traits"
version = "0.2.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841"
dependencies = [
"autocfg",
]
[[package]]
name = "object"
version = "0.37.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ff76201f031d8863c38aa7f905eca4f53abbfa15f609db4277d44cd8938f33fe"
dependencies = [
"memchr",
]
[[package]] [[package]]
name = "once_cell" name = "once_cell"
@@ -49,6 +557,108 @@ version = "1.19.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3fdb12b2476b595f9358c5161aa467c2438859caa136dec86c26fdd2efe17b92" checksum = "3fdb12b2476b595f9358c5161aa467c2438859caa136dec86c26fdd2efe17b92"
[[package]]
name = "once_cell_polyfill"
version = "1.70.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
[[package]]
name = "percent-encoding"
version = "2.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pkg-config"
version = "0.3.33"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "19f132c84eca552bf34cab8ec81f1c1dcc229b811638f9d283dceabe58c5569e"
[[package]]
name = "proc-macro2"
version = "1.0.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934"
dependencies = [
"unicode-ident",
]
[[package]]
name = "psm"
version = "0.1.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "645dbe486e346d9b5de3ef16ede18c26e6c70ad97418f4874b8b1889d6e761ea"
dependencies = [
"ar_archive_writer",
"cc",
]
[[package]]
name = "quote"
version = "1.0.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
dependencies = [
"proc-macro2",
]
[[package]]
name = "radix_trie"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3b4431027dcd37fc2a73ef740b5f233aa805897935b8bce0195e41bbf9a3289a"
dependencies = [
"endian-type",
"nibble_vec",
]
[[package]]
name = "regex"
version = "1.12.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f1292b7759ae1cb9ec195452d1390a074f0cd8541ab7a5a8c31cd6db45d4a6ba"
dependencies = [
"aho-corasick",
"memchr",
"regex-automata 0.4.14",
"regex-syntax 0.8.11",
]
[[package]]
name = "regex-automata"
version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59b23e92ee4318893fa3fe3e6fb365258efbfe6ac6ab30f090cdcbb7aa37efa9"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax 0.7.5",
]
[[package]]
name = "regex-automata"
version = "0.4.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e1dd4122fc1595e8162618945476892eefca7b88c52820e74af6262213cae8f"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax 0.8.11",
]
[[package]]
name = "regex-syntax"
version = "0.7.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dbb5fb1acd8a1a18b3dd5be62d25485eb770e05afb408a9627d14d451bae12da"
[[package]]
name = "regex-syntax"
version = "0.8.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
[[package]] [[package]]
name = "rnix" name = "rnix"
version = "0.12.0" version = "0.12.0"
@@ -58,6 +668,28 @@ dependencies = [
"rowan", "rowan",
] ]
[[package]]
name = "rootcause"
version = "0.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b660d9968fae12f4e691f2b2be5d9a3a6de875300c682e8d2cb89a618dd60875"
dependencies = [
"hashbrown 0.17.1",
"indexmap",
"rootcause-internals",
"rustc-hash 2.1.3",
"triomphe",
]
[[package]]
name = "rootcause-internals"
version = "0.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0184f6fcff3b58b7c963aee6e3cc915c04331aa6eef974f79d7d44d21e246c24"
dependencies = [
"triomphe",
]
[[package]] [[package]]
name = "rowan" name = "rowan"
version = "0.15.16" version = "0.15.16"
@@ -65,8 +697,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0a542b0253fa46e632d27a1dc5cf7b930de4df8659dc6e720b647fc72147ae3d" checksum = "0a542b0253fa46e632d27a1dc5cf7b930de4df8659dc6e720b647fc72147ae3d"
dependencies = [ dependencies = [
"countme", "countme",
"hashbrown", "hashbrown 0.14.5",
"rustc-hash", "rustc-hash 1.1.0",
"text-size", "text-size",
] ]
@@ -76,8 +708,250 @@ version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2" checksum = "08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2"
[[package]]
name = "rustc-hash"
version = "2.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6b1e7f9a428571be2dc5bc0505c13fb6bf936822b894ec87abf8a08a4e51742d"
[[package]]
name = "rustyline"
version = "18.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "53f6a737db68eb1a8ccff86b584b2fc13eca6a7bb6f78ebc7c529547e3ab9684"
dependencies = [
"bitflags",
"cfg-if",
"clipboard-win",
"home",
"libc",
"log",
"memchr",
"nix",
"radix_trie",
"unicode-segmentation",
"unicode-width 0.2.2",
"utf8parse",
"windows-sys",
]
[[package]]
name = "rustyline-derive"
version = "0.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "64e5587417a3c4e16a4415e8d7d07f80998ed835ade621d19dfbe9fbe3205b0f"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "serde"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde_core"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "sha2"
version = "0.10.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283"
dependencies = [
"cfg-if",
"cpufeatures",
"digest",
]
[[package]]
name = "shlex"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
[[package]]
name = "smallvec"
version = "1.15.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ed6a63f02c8539c91a8685a86f4099661ba3da017932f6ebbea6de3f0fa7c90"
[[package]]
name = "stacker"
version = "0.1.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "640c8cdd92b6b12f5bcb1803ca3bbf5ab96e5e6b6b96b9ab77dabe9e880b3190"
dependencies = [
"cc",
"cfg-if",
"libc",
"psm",
"windows-sys",
]
[[package]]
name = "strsim"
version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f"
[[package]]
name = "syn"
version = "2.0.118"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1b9ae57f904213ebb649ce6895b8a66c66f0203b9319718f69a5612a065b1422"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]] [[package]]
name = "text-size" name = "text-size"
version = "1.1.1" version = "1.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f18aa187839b2bdb1ad2fa35ead8c4c2976b64e4363c386d45ac0f7ee85c9233" checksum = "f18aa187839b2bdb1ad2fa35ead8c4c2976b64e4363c386d45ac0f7ee85c9233"
[[package]]
name = "triomphe"
version = "0.1.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b40688ea6389c8171614b25491f71d4a27946e0c7ce2da1c6de27e25abf1a0ae"
[[package]]
name = "typenum"
version = "1.20.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20"
[[package]]
name = "unicase"
version = "2.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dbc4bc3a9f746d862c45cb89d705aa10f187bb96c76001afab07a0d35ce60142"
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "unicode-segmentation"
version = "1.13.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c6f5d3c3b1bf09027a88a6bc961fc00497d651009560b5463668dc81b0fa87a8"
[[package]]
name = "unicode-width"
version = "0.1.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7dd6e30e90baa6f72411720665d41d89b9a3d039dc45b8faea1ddd07f617f6af"
[[package]]
name = "unicode-width"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4ac048d71ede7ee76d585517add45da530660ef4390e49b098733c6e897f254"
[[package]]
name = "utf8parse"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
[[package]]
name = "version_check"
version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
[[package]]
name = "windows-link"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-sys"
version = "0.61.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
dependencies = [
"windows-link",
]
[[package]]
name = "yansi"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09041cd90cf85f7f8b2df60c646f853b7f535ce68f85244eb6731cf89fa498ec"
[[package]]
name = "zngur"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7fc912d12934b4d04aabc52c14db6fc88ae8aa5a47902f27e8759c2de254723f"
dependencies = [
"zngur-generator",
]
[[package]]
name = "zngur-def"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f27e49a62db537cea43a6c122ded7eda99c4ac0ca1d7cfd74f2bf6a6c88712d"
dependencies = [
"indexmap",
"itertools",
]
[[package]]
name = "zngur-generator"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5ffad8c6994c477023aba7613bdf17c470003bbd191846d3096e84ffecf86d4c"
dependencies = [
"askama",
"hex",
"indexmap",
"itertools",
"sha2",
"zngur-def",
"zngur-parser",
]
[[package]]
name = "zngur-parser"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8d66c1b85ca6eab9576df5de31758cb1f9046b4ce47b25f45dabcb0fc7ef8789"
dependencies = [
"ariadne",
"chumsky",
"itertools",
"zngur-def",
]
Cargo.toml
+23 -1
View File
@@ -1,6 +1,28 @@
[workspace] [workspace]
resolver = "2" resolver = "2"
members = ["lix/lix-doc", "lix/libutil"] members = [
"lix/lix-doc",
"lix/lix-rs",
"tools/licxxbridge",
]
[workspace.package] [workspace.package]
edition = "2021" edition = "2021"
[workspace.dependencies]
clap = "4"
regex = "1.12.4"
rootcause = "0.13.0"
rustyline = "18"
rustyline-derive = "0.12"
syn = "2.0"
zngur = "0.10"
pkg-config = "0.3.33"
[profile.dev]
opt-level = 1
[profile.release]
debug = "full"
debug-assertions = true
overflow-checks = true
bench/bench.py
+2 -1
View File
@@ -10,6 +10,7 @@ import platform
import shlex import shlex
import textwrap import textwrap
import dataclasses import dataclasses
from pathlib import Path
flake_args = ["--extra-experimental-features", "nix-command flakes"] flake_args = ["--extra-experimental-features", "nix-command flakes"]
cases = { cases = {
@@ -18,7 +19,7 @@ cases = {
*flake_args, *flake_args,
"search", "search",
"--no-eval-cache", "--no-eval-cache",
"github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870", f"path:{Path('./bench/nixpkgs/').readlink()}",
"hello", "hello",
], ],
"rebuild": lambda build: [ "rebuild": lambda build: [
contrib/plugins/meson.build
+7 -13
View File
@@ -1,19 +1,13 @@
# Darwin: don't link liblix* into plugins (host process provides them at runtime).
# Explicitly link curl so it binds to Nix-store libcurl, not /usr/lib/libcurl.
if is_darwin
plugin_deps = [
liblix.partial_dependency(includes : true, compile_args : true),
curl,
]
else
plugin_deps = [liblix, curl]
endif
plugin_mtls_store = shared_module( plugin_mtls_store = shared_module(
'plugin_mtls_store', 'plugin_mtls_store',
'plugin_mtls_store.cc', 'plugin_mtls_store.cc',
dependencies : plugin_deps, # don't link liblix* into plugins (host process provides them at runtime).
# Explicitly link curl so it binds to Nix-store libcurl, not /usr/lib/libcurl.
dependencies : [
liblix.partial_dependency(includes : true, compile_args : true),
curl,
],
install : false, install : false,
build_by_default : true, build_by_default : true,
link_args : is_darwin ? shared_module_link_args : strict_shared_module_link_args, link_args : plugin_link_args,
) )
doc/manual/anchors.jq
+1 -2
View File
@@ -24,8 +24,7 @@ def map_contents_recursively(transformer):
def process_command: def process_command:
.[0] as $context | .[0] as $context |
.[1] as $body | .[1] as $body |
# XXX FUTURE: drop sections once mdBook is at 0.5.0 or above in nixpkgs $body | .items |= map(map_contents_recursively(if $context.renderer == "html" then transform_anchors_html else transform_anchors_strip end))
$body | (.items? // .sections) |= map(map_contents_recursively(if $context.renderer == "html" then transform_anchors_html else transform_anchors_strip end))
; ;
process_command process_command
doc/manual/book.toml
+3 -4
View File
@@ -31,12 +31,11 @@ command = "jq --from-file anchors.jq"
[output.markdown] [output.markdown]
# XXX FUTURE: may be reenabled once mdBook 0.5.0 or above and matching mdbook-linkchecker are in nixpkgs [output.linkcheck2]
#[output.linkcheck]
# no Internet during the build (in the sandbox) # no Internet during the build (in the sandbox)
#follow-web-links = false follow-web-links = false
# mdbook-linkcheck does not understand [foo]{#bar} style links, resulting in # mdbook-linkcheck does not understand [foo]{#bar} style links, resulting in
# excessive "Potential incomplete link" warnings. No other kind of warning was # excessive "Potential incomplete link" warnings. No other kind of warning was
# produced at the time of writing. # produced at the time of writing.
#warning-policy = "ignore" warning-policy = "ignore"
doc/manual/change-authors.yml
+30
View File
@@ -48,6 +48,11 @@ artemist:
display_name: Artemis Tosini display_name: Artemis Tosini
forgejo: artemist forgejo: artemist
astreaprtcl:
display_name: Astreaprtcl
forgejo: astreaprtcl
github: astreaprtcl
bb010g: bb010g:
display_name: Dusk Banks display_name: Dusk Banks
forgejo: bb010g forgejo: bb010g
@@ -57,6 +62,10 @@ blitz:
display_name: Julian Stecklina display_name: Julian Stecklina
github: blitz github: blitz
blokyk:
display_name: blokyk
github: blokyk
cole-h: cole-h:
display_name: Cole Helbling display_name: Cole Helbling
github: cole-h github: cole-h
@@ -73,6 +82,9 @@ detroyejr:
display_name: Jonathan De Troye display_name: Jonathan De Troye
github: detroyejr github: detroyejr
edef:
github: edef1c
edolstra: edolstra:
display_name: Eelco Dolstra display_name: Eelco Dolstra
github: edolstra github: edolstra
@@ -138,6 +150,9 @@ kasimeka:
forgejo: janw4ld forgejo: janw4ld
github: kasimeka github: kasimeka
keysmashes:
github: keysmashes
kfears: kfears:
display_name: KFears display_name: KFears
forgejo: kfearsoff forgejo: kfearsoff
@@ -241,6 +256,11 @@ raito:
forgejo: raito forgejo: raito
github: RaitoBezarius github: RaitoBezarius
rkjnsn:
display_name: Erik Jensen
forgejo: rkjnsn
github: rkjnsn
roberth: roberth:
display_name: Robert Hensing display_name: Robert Hensing
github: roberth github: roberth
@@ -249,9 +269,16 @@ rootile:
display_name: rootile (Rutile) display_name: rootile (Rutile)
forgejo: rootile forgejo: rootile
sandydoo:
github: sandydoo
seppel3210: seppel3210:
github: Seppel3210 github: Seppel3210
sterni:
forgejo: sterni
github: sternenseemann
stevalkr: stevalkr:
github: stevalkr github: stevalkr
@@ -309,5 +336,8 @@ yorickvp:
yshui: yshui:
github: yshui github: yshui
ysndr:
github: ysndr
zimbatm: zimbatm:
github: zimbatm github: zimbatm
doc/manual/rl-next/arg-behavior.md
+10
View File
@@ -0,0 +1,10 @@
---
synopsis: "allow setting nested attributes via `--arg`/`--argstr`"
cls: [5338]
category: "Features"
credits: [ma27]
issues: [fj#496]
---
Passing `--arg config.allowUnfree true` to e.g. `nix-build` now results in `config` with value
`{ allowUnfree = true; }` passed to the expression.
doc/manual/rl-next/check-netrc-or-ca-file.md
+9
View File
@@ -0,0 +1,9 @@
---
synopsis: "check for missing ca-file or netrc-file if one is specified"
cls: [5646]
category: "Improvements"
credits: [astreaprtcl]
issues: [fj#1106]
---
If the settings `ssl-cert-file` or `netrc-file` have been set by the user, check if those files actually exist and fail if they are missing.
doc/manual/rl-next/empty-attr-paths.md
+10
View File
@@ -0,0 +1,10 @@
---
synopsis: "libexpr: allow empty attr-names in parseAttrPath if they are quoted"
cls: [5375]
category: "Miscellany"
credits: [ma27]
---
Empty strings are now allowed in attribute paths as consumed by e.g. `nix-build`.
I.e. `nix-build -A 'foo."".bar'` works now.
The quotes are necessary, i.e. `nix-build -A foo..bar` will throw an error.
doc/manual/rl-next/fetch-empty-hash.md
+10
View File
@@ -0,0 +1,10 @@
---
synopsis: "don't treat tarball fetches with empty or zero hash as locked"
cls: []
category: "Fixes"
credits: [horrors]
issues: [fj#1233]
---
Lix no longer treats tarball fetches with empty or zero hashes as locked.
All such fetches are now also affected by `tarball-ttl` as a consequence.
doc/manual/rl-next/floor-ceil-corrupt-integers.md
+13
View File
@@ -0,0 +1,13 @@
---
synopsis: builtins.floor/builtins.ceil handle out-of-range inputs correctly
issues: [nix#12899]
cls: [3923]
prs: [nix#13013]
category: "Breaking Changes"
credits: [jade, nan-git, rootile]
---
Previously, `builtins.floor` and `builtins.ceil` always cast the input into a floating point value before running the operation and casting the floating point result back into an integer.
No checks were made for precision loss in either coercing integer inputs or converting the output to an integer (and in fact in the latter case, invoked undefined behaviour).
Now, Lix checks for precision loss on integer input (to avoid a silent eval semantics change if we were to simply pass it through as-is) and on integer output.
If your code fails to evaluate after this change, use `--extra-deprecated-features floor-ceil-corrupt-integers`.
doc/manual/rl-next/libexpr-fix-break.md
+14
View File
@@ -0,0 +1,14 @@
---
synopsis: "builtins.break doesn't break expression anymore"
issues: [1165]
cls: [5422]
category: "Fixes"
credits: [blokyk]
---
Wrapping an expression in `builtins.break` used to break some builtins like
`map` and the `is*` functions, which could modify the execution path of code
inadvertently, made debugging nix harder than it already is, and in some cases
even crashed the interpreter. Now, using `break` should be completely
transparent to whatever function receives it as an input, preventing the
above-mentioned issues.
doc/manual/rl-next/libexpr-fix-stdout.md
+9
View File
@@ -0,0 +1,9 @@
---
synopsis: "flake config warnings are now printed to stderr"
issues: [1155]
cls: [5379]
category: "Fixes"
credits: [lheckemann]
---
The settings listed in a flake-config confirmation prompt are now printed to stderr rather than stdout, which allows `nix print-dev-env` to emit valid bash again even in the presence of untrusted settings.
doc/manual/rl-next/libfetchers-lock-cache-input.md
+21
View File
@@ -0,0 +1,21 @@
---
synopsis: "Use a lock when fetching inputs"
issues: [1122]
cls: [5438]
category: "Fixes"
credits: [lheckemann]
---
Up to now, attempting to fetch the same git input from multiple processes
concurrently when the input is not yet cached presented multiple issues:
- If the input was not already present, it would unnecessarily be fetched
multiple times;
- Access to the fetcher cache database was contentious, and could lead to
evaluation or flake locking failing unnecessary because the fetcher cache
was locked.
We now acquire a lock on a path based on a hash of the input specification
before accessing the fetcher db, reducing contention significantly, and
preventing more than one process from fetching the same path at the same time.
doc/manual/rl-next/mimalloc.md
+16
View File
@@ -0,0 +1,16 @@
---
synopsis: "Use mimalloc for faster evaluation"
cls: [5645]
category: Features
credits: [getchoo, lovesegfault]
---
Lix now links with [mimalloc](https://github.com/microsoft/mimalloc),
replacing the system's default `malloc()` for all non-GC allocations.
This yields a **5–12% wall-clock improvement** on evaluation workloads,
ranging from `nix-instantiate hello` to `nix-env -qa` and full NixOS
configurations.
The allocator can be disabled at build time with `-Dmimalloc=disabled`,
or by passing the `useMimalloc = false` override to the `lix` package.
doc/manual/rl-next/minimum-lowdown-1.4.md
+10
View File
@@ -0,0 +1,10 @@
---
synopsis: "Lix now requires lowdown 1.4.0 or later"
issues: []
cls: [5374]
category: Packaging
credits: [sterni]
---
Support for linking against `lowdown < 1.4.0` has been removed from Lix since
all supported Nixpkgs channels distribute lowdown 2.0.4 or later.
doc/manual/rl-next/nej-apply-flag.md
+12
View File
@@ -0,0 +1,12 @@
---
synopsis: "nix-eval-jobs support `--apply` flag"
cls: [5748]
category: "Features"
credits: [isabelroses,mic92,ysndr]
issues: [fj#1214]
---
`nix-eval-jobs` now supports the `--apply` flag. With this you can apply the
provided function to the each derivation, the result of this function will then
be serialized as a JSON value and stored inside `"extraValue"` key of the json
line output.
doc/manual/rl-next/nix-copy-closure-include-outputs.md
+10
View File
@@ -0,0 +1,10 @@
---
synopsis: "Fix `nix-copy-closure --include-outputs`"
issues: [gh#5105]
cls: [5588]
category: "Fixes"
credits: [rkjnsn]
---
The `--include-outputs` flag for `nix-copy-closure` now works as intended.
Previously, the option was accepted but silently ignored.
doc/manual/rl-next/nix-doctor.md
+14
View File
@@ -0,0 +1,14 @@
---
synopsis: "Improve nix doctor"
cls: [5316, 5317, 5318, 5319, 5320, 5768, 5829]
category: Features
credits: [rootile, raito]
---
The `nix doctor` diagnosics interface now provides a lot more useful information including, but not limited to:
- General system information (OS, Hardware etc)
- Nix Information like Sandbox, Version, Store, State and other directories
- Flake registry
- Search path Information
- Nixpkgs provenance
- Remote builder configuration (including remote connection)
- fix crash when having relative Paths in PATH
doc/manual/rl-next/nix-path-shadow-error.md
+11
View File
@@ -0,0 +1,11 @@
---
synopsis: "Shadowing internal files through the Nix search path is now an error"
issues: [998]
cls: [4632, 5370]
category: "Breaking Changes"
credits: [thubrecht, jade, horrors]
---
As Lix uses the path `<nix/fetchurl.nix>` for bootstrapping purposes, the ability to shadow it by adding `nix=/some/path` (or `/other/path` that contains a `nix` directory) to the search path is not desirable.
Lix 2.95 deprecated this behavior with a warning, Lix 2.96 now turns it into a hard error if the `nix-path-shadow` deprecated feature isn't enabled. This deprecated feature is slated to be removed in Lix 2.98.
doc/manual/rl-next/no-max-connections.md
+11
View File
@@ -0,0 +1,11 @@
---
synopsis: "Remove `max-connections` store parameters for `ssh://` and `ssh-ng://` stores"
cls: []
category: Miscellany
credits: [horrors]
---
The `max-connections` parameter was undocumented, untested, and (in the case of `ssh`) even ignored
entirely for remote builds. During a survey of public nixos configurations we have found *two* uses
of `max-connections` for `ssh-ng`, and none at all for `ssh`. Since it is so rarely used but brings
significant internal complexity that hinders improvements we have decided to remove these features.
doc/manual/rl-next/repl-relative-st.md
+18
View File
@@ -0,0 +1,18 @@
---
synopsis: "Allow moving between stack frames relative to current debugger frame"
issues: [1156]
cls: [5411]
category: "Improvements"
credits: [blokyk]
---
Debugging functional programs often involve switching between a bunch of stack
frames to get the full context of what's happening and who's calling who.
Before this change, going up or down the stack in the nix debugger with `:st`
meant remembering the absolute index of each stack frame, instead of their
positions relative to one another; this got tiring *fast*.
Now, you can prepend `:st`'s argument with a + or - sign to indicate you want to
move relative to the current stack frame. For example, typing `:st +3` when you
were on frame `10` will go frame `13`; vice-versa, typing `:st -4` on frame `6`
will go to frame `2`.
doc/manual/rl-next/repl-reverse-bt.md
+17
View File
@@ -0,0 +1,17 @@
---
synopsis: "Print REPL backtraces in more convenient order"
issues: []
cls: [5491]
category: "Improvements"
credits: [blokyk]
---
When using the debugger, stack traces printed with the `:bt` command were
previously printed in reverse order compared to most other situations where they
appeared: the current stack frame would be printed at the very top, with the
most outer frame at the bottom, meaning that you'd have to scroll up to get a
sense of where you are.
With this change, the stack frames are printed such that the most relevant ones
are immediatly visible at the bottom, just like other traces in lix (e.g.
ones caused by errors).
doc/manual/rl-next/repl-st-errors.md
+14
View File
@@ -0,0 +1,14 @@
---
synopsis: "invalid arguments to :st now print an error"
cls: [5386]
category: "Improvements"
credits: [blokyk]
---
When using the debugger, the `:st` command used to traverse the call stack would
silently fail and put the debugger in an invalid state if the argument given to
it wasn't a valid stack frame index.
This change adds an error message warning the user if the given index wasn't a
valid frame (telling them the range of valid indices), as well as if it wasn't
even a valid integer to begin with.
doc/manual/rl-next/rustyline.md
+12
View File
@@ -0,0 +1,12 @@
---
synopsis: "REPL now uses rustyline"
cls: [5703]
category: "Improvements"
credits: [horrors]
issues: []
---
The REPL now uses [rustyline](https://github.com/kkawakam/rustyline) for input processing instead
of editline. This comes with some improvements to REPL behavior: wrapping lines no longer confuse
the line editor, unicode is fully supported, pasting multiline expressions is noew possible, even
undo commands are now available! We plan to improve the REPL further using these newfound powers.
doc/manual/rl-next/structuredattrs-url-guesswork.md
+11
View File
@@ -0,0 +1,11 @@
---
synopsis: "Hash mismatch diagnostics now work with `structuredAttrs`"
issues: [fj#1175]
cls: [5441]
category: Fixes
credits: [keysmashes]
---
Nixpkgs fetchers like `fetchurl` now use `structuredAttrs`, which broke the
hash mismatch diagnostics added in Lix 2.91. This has been fixed and the likely
URL is now shown again.
doc/manual/rl-next/trivial-flakes.md
+18
View File
@@ -0,0 +1,18 @@
---
synopsis: "Changes to `flake.nix` validation"
cls: [5523]
category: "Breaking Changes"
credits: [piegames, Qyriad, horrors]
issues: [gh#4945]
---
Flakes try to keep their inputs and metadata "simple", to make sure no unbounded computation may happen when calling e.g. `nix flake show`.
Those checks were haphazard, a maintenance burden, and also easily circumventable.
Lix has now replaced all the old checks by a simple rule: **No function calls outside of `outputs`.**
This is easier to reason about than the previous set of inconsistent rules, and crucially now also allows syntax features that users felt like they *should* have worked in the past, like let bindings.
However, some warts still remain for now: Some syntax constructs like `-1` internally desugar to `__sub 0 1`, which is a function call and thus remains forbidden.
This will be rectified as soon as the deprecation period of the respective anti-features has been completed.
This change is **breaking** in the sense that flakes which are written with the newly allowed language features will not evaluate with an older Lix version which still uses the old, more restrictive checks.
Crucially, this also affects **all transitive dependants** of such Flakes.
doc/manual/rl-next/unsigned-overflow-vuln.md
+28
View File
@@ -0,0 +1,28 @@
---
synopsis: "Fix unsigned overflow leading to out-of-band write in the NAR parser"
cls: [5554]
category: "Fixes"
credits: [horrors, raito, edef, sandydoo]
issues: []
---
The NAR parser contained an unsigned integer overflow that could be used by an
attacker to write arbitrary data to an unknown memory location and possibly
achieve code execution. A successful attack on the system-wide Lix daemon
could lead to privilege escalation to root. Any process that involves NAR
serialization could trigger this issue, including (but not limited to)
- local user interaction, whether the users are trusted or untrusted
- malicious substituters sending malformed NARs
- remote builders sending malformed build results
- remote daemons sending malformed inputs when requesting remote builds
Successful attacks using this bug require ASLR weakening of some sort, whether
by architecture constraints (e.g. on 32 bit systems, where little randomization
is possible) or system configuration (e.g. low ASLR entropy when loading
libraries), and millions of attempts. Local attacks can be mounted in less than
an hour. Remote builds typically require a fresh SSH connection for each build
and are thus less susceptible. Only one attempt can be made by substituters for
every build using substituters, they are thus not a likely vector for attacks.
At the time of writing, MITRE has not assigned this a CVE yet.
doc/manual/rl-next/user-traces.md
+23
View File
@@ -0,0 +1,23 @@
---
synopsis: "Always print frames from `addErrorContext` in error traces"
cls: [5847]
category: "Improvements"
credits: [blokyk]
issues: []
---
The [`builtins.addErrorContext`](@docroot@/language/builtins.md#builtins-addErrorContext)
function allows an author to add artificial stack frames with custom messages to
help end-users understand the context of an error and the path the code took to
get there, without having to read and understand the original source code. A
particularly notable user of this is the Nixpkgs module system, which adds
custom frames detailing what option it's evaluating or which definition it's
looking at.
However, previously, these frames would end up treated just as any other,
meaning they would most often not be visible without `--show-trace`; yet, using
`--show-trace`, they would be drowned out in the noise of the hundreds of other
frames, rendering them just as unusable.
With this change, these frames are now unconditionally shown, even without
`--show-trace`, which makes basic error traces much more informative.
doc/manual/src/command-ref/opt-common.md
+6
View File
@@ -177,6 +177,12 @@ Most commands in Lix accept the following command-line options:
You can override this using `--arg`, e.g., `nix-env --install --attr pkgname --arg system \"i686-freebsd\"`. You can override this using `--arg`, e.g., `nix-env --install --attr pkgname --arg system \"i686-freebsd\"`.
(Note that since the argument is a Nix string literal, you have to escape the quotes.) (Note that since the argument is a Nix string literal, you have to escape the quotes.)
Additionally, dots are interpreted as attribute-path separators.
I.e. `nix-instantiate '<nixpkgs>' -A hello-unfree --arg config.allowUnfree true` will result in an argument `config` with value `{ allowUnfree = true; }` being passed to `<nixpkgs>`.
Please note that merging of different arguments is rejected.
I.e. `--arg config '{ cudaSupport = true; }' --arg config.allowUnfree true` will not work whereas `--arg config.cudaSupport true --arg config.allowUnfree true` is accepted.
- <span id="opt-argstr">[`--argstr`](#opt-argstr)</span> *name* *value* - <span id="opt-argstr">[`--argstr`](#opt-argstr)</span> *name* *value*
This option is like `--arg`, only the value is not a Nix expression but a string. This option is like `--arg`, only the value is not a Nix expression but a string.
doc/manual/src/contributing/hacking.md
+53 -17
View File
@@ -51,28 +51,64 @@ $ nix-shell -A native-clangStdenvPackages
### Building from the development shell ### Building from the development shell
Run a clean build and test with `just clean setup build install test`. We have a [justfile](https://just.systems/) for extra convenient building.
It defaults to using `./build` as the build directory, and `$out` (`./outputs/out`) as the install directory.
For most cases, you can clean-build, install, and run the tests with:
```bash
$ just setup --wipe && just test
```
> **Note**
>
> The `--wipe` argument to `meson setup` conveniently works whether you have an existing build directory or not.
>
> However, it is *mostly*, but not *exactly* equivalent to deleting the build directory first.
> In particular, previously specified `-D` build options are **preserved** with `--wipe` (for some reason).
> For example, if you fetch and checkout a new version of Lix, and that new version *removes* a Meson build option from `./meson.options`, *and* a previous invocation in that build directory explicitly set that option, then `meson setup --wipe build` will error, complaining about the unknown option.
> For these cases, `just clean` will give you a well-and-truly-this-time-for-real clean build.
Because the integration tests require installation to work, `just test` automatically also calls `just install`, and Meson helpfully will automatically build any targets that need building when trying to install them.
You can override the build directory or install directory by setting the justfile [variables](https://just.systems/man/en/setting-variables-from-the-command-line.html) `outdir` and `builddir` on the command-line:
```bash
$ just builddir=build-before-bisect outdir=out-before-bisect setup
$ just builddir=build-before-bisect test
```
You'll have to set `builddir` for every target, but `outdir` only needs to be set for `setup`.
You can also run the unit tests and integration tests separately: You can also run the unit tests and integration tests separately:
```bash ```bash
$ just setup build test-unit $ just setup
$ just install test-integration $ just test-unit
$ just test-integration
``` ```
Many justfile aliases have a `-custom` variant which pass extra arguments to `meson`. Most justfile targets forward all further arguments to the underlying Meson invocation.
For example, to work on both Lix and nix-eval-jobs you can run: For example, to work on both Lix and nix-eval-jobs you can run:
``` ```bash
$ just setup-custom -Dnix-eval-jobs=enabled $ just setup -Dnix-eval-jobs=enabled
$ # or
$ mesonFlags=-Dnix-eval-jobs=enabled just setup
``` ```
Note that only targets which don't accept extra arguments can be used when Note that only targets which *don't* accept extra arguments can have other targets following them.
running multiple targets at once; `just setup build` is fine, but `just `just clean setup` is equivalent to `just clean && just setup`, but `just build test` runs the `build` target with the argument `test`.
setup-custom build` is an error. The `test` target is usually the last one to This means that if you want to, for example, build with lower parallelism, and then test, you will have to do something like this:
run, so it always accepts extra arguments.
```bash
$ just build -j4
$ just test
```
Finally, the rewrite of the integration test suite, functional2, also has its own justfile target which allows passing extra arguments to pytest.
For example, to collect and list all functional2 tests without running them, you can pass pytest's `--collect-only` argument:
```bash
$ just test-functional2 --collect-only
```
You can also build Lix manually: You can also build Lix manually:
@@ -147,13 +183,13 @@ The development shell on Linux uses LLD by default for faster link times.
This is set using `mesonFlags`, so to override it, you can simplify re-specify the linker to Meson: This is set using `mesonFlags`, so to override it, you can simplify re-specify the linker to Meson:
```bash ```bash
$ just setup-custom -Dc_link_args=-fuse-ld=ld -Dcpp_link_args=-fuse-ld=ld $ just setup -Dc_link_args=-fuse-ld=ld -Dcpp_link_args=-fuse-ld=ld
``` ```
While using LLD, you may find it helpful to use ThinLTO for even further improvements to link times for incremental builds: While using LLD, you may find it helpful to use ThinLTO for even further improvements to link times for incremental builds:
```bash ```bash
$ just setup-custom -Db_lto=true -Db_lto_mode=thin -Db_thinlto_cache=true $ just setup -Db_lto=true -Db_lto_mode=thin -Db_thinlto_cache=true
``` ```
## Sending changes to Gerrit for review {#sending-to-gerrit} ## Sending changes to Gerrit for review {#sending-to-gerrit}
@@ -164,7 +200,7 @@ Our instance is at <https://gerrit.lix.systems>.
There's much more information about how to use Gerrit in the [wiki section on Gerrit][wiki-gerrit] including how to use Jujutsu, how to use the UI and more. There's much more information about how to use Gerrit in the [wiki section on Gerrit][wiki-gerrit] including how to use Jujutsu, how to use the UI and more.
The Snix project also has some Gerrit information [in their contributing docs][snix-gerrit]. The Snix project also has some Gerrit information [in their contributing docs][snix-gerrit].
[wiki-gerrit]: https://wiki.lix.systems/books/lix-contributors/chapter/gerrit [wiki-gerrit]: https://wiki.lix.systems/books/contributing/chapter/intro-to-gerrit
[snix-gerrit]: https://snix.dev/docs/guides/contributing/ [snix-gerrit]: https://snix.dev/docs/guides/contributing/
The gist is that once you have your SSH key and git remote set up, you can send commits for review with: The gist is that once you have your SSH key and git remote set up, you can send commits for review with:
@@ -190,7 +226,7 @@ The `Code-Review+2` from before will stick around through trivial rebases so no
We use Buildkite for our CI, usually you will not have to interact directly with it other than reviewing any errors it produces, which are linked from Gerrit. We use Buildkite for our CI, usually you will not have to interact directly with it other than reviewing any errors it produces, which are linked from Gerrit.
However in certain cases a CI run will fail due to transient issues not related to your code and you will need to rerun it by hand. However in certain cases a CI run will fail due to transient issues not related to your code and you will need to rerun it by hand.
You can log in to the CI via [SSO](https://buildkite.com/sso/lix-project). On your job you can then hit the "Retry failed" button to rerun it, normally you will not have a repeat of the transient issue. You can log in to the CI via [SSO](https://buildkite.com/sso/afnix). On your job you can then hit the "Retry failed" button to rerun it, normally you will not have a repeat of the transient issue.
If the build still fails on CI issues or all builds are failing this should be reported via [Zulip on #T-infra](https://zulip.lix.systems/#narrow/channel/7-T-infra) or [Matrix on #dev](https://matrix.to/#/%23dev%3Alix.systems?via=lix.systems). If the build still fails on CI issues or all builds are failing this should be reported via [Zulip on #T-infra](https://zulip.lix.systems/#narrow/channel/7-T-infra) or [Matrix on #dev](https://matrix.to/#/%23dev%3Alix.systems?via=lix.systems).
## Building Lix with `nix` ## Building Lix with `nix`
@@ -530,7 +566,7 @@ Then add the new file there, and don't forget to register it in the appropriate
The following metadata properties are supported for builtin functions: The following metadata properties are supported for builtin functions:
* `name` (required): the language-facing name (as a member of the `builtins` attribute set) of the function. * `name` (required): the language-facing name (as a member of the `builtins` attribute set) of the function.
* `implementation` (optional): a C++ expression specifying the implementation of the builtin. * `implementation` (optional): a C++ expression specifying the implementation of the builtin.
It must be a function of signature `void(EvalState &, PosIdx, Value * *, Value &)`. It must be a function of signature `Value(EvalState &, PosIdx, Value * *)`.
If not specified, defaults to `prim_${name}`. If not specified, defaults to `prim_${name}`.
* `renameInGlobalScope` (optional): whether the definition should be "hidden" in the global scope by prefixing its name with two underscores. * `renameInGlobalScope` (optional): whether the definition should be "hidden" in the global scope by prefixing its name with two underscores.
If not specified, defaults to `true`. If not specified, defaults to `true`.
doc/manual/src/installation/prerequisites-source.md
-4
View File
@@ -50,10 +50,6 @@ The most current alternative to this section is to read `package.nix` and see wh
- The `boost` library of version 1.66.0 or higher. It can be obtained - The `boost` library of version 1.66.0 or higher. It can be obtained
from the official web site <https://www.boost.org/>. from the official web site <https://www.boost.org/>.
- The `editline` library of version 1.14.0 or higher. It can be
obtained from the its repository
<https://github.com/troglobit/editline>.
- Recent versions of Bison and Flex to build the parser. (This is - Recent versions of Bison and Flex to build the parser. (This is
because Nix needs GLR support in Bison and reentrancy support in because Nix needs GLR support in Bison and reentrancy support in
Flex.) For Bison, you need version 2.6, which can be obtained from Flex.) For Bison, you need version 2.6, which can be obtained from
doc/manual/src/language/derivations.md
+6
View File
@@ -17,6 +17,12 @@ the attributes of which specify the inputs of the build.
string. This is used as a symbolic name for the package by string. This is used as a symbolic name for the package by
`nix-env`, and it is appended to the output paths of the derivation. `nix-env`, and it is appended to the output paths of the derivation.
> **Note**
>
> Names can only contain alphanumerical characters (0-9, a-z, A-Z)
> as well as `+`, `-`, `.`, `_`, `?` and `=`. Names must be neither
> `.` nor `..`, and must not start with `.-` or `..-`.
- There must be an attribute named [`builder`]{#attr-builder} that identifies the - There must be an attribute named [`builder`]{#attr-builder} that identifies the
program that is executed to perform the build. It can be either a program that is executed to perform the build. It can be either a
derivation or a source (a local file reference, e.g., derivation or a source (a local file reference, e.g.,
doc/manual/src/language/values.md
+1 -1
View File
@@ -164,7 +164,7 @@ Note that lists are only lazy in values, and they are strict in length.
An attribute set is a collection of name-value-pairs (called *attributes*) enclosed in curly brackets (`{ }`). An attribute set is a collection of name-value-pairs (called *attributes*) enclosed in curly brackets (`{ }`).
An attribute name can be an identifier or a [string](#type-string). An attribute name can be an identifier or a [double-quoted string](#type-string).
An identifier must start with a letter (`a-z`, `A-Z`) or underscore (`_`), and can otherwise contain letters (`a-z`, `A-Z`), numbers (`0-9`), underscores (`_`), apostrophes (`'`), or dashes (`-`). An identifier must start with a letter (`a-z`, `A-Z`) or underscore (`_`), and can otherwise contain letters (`a-z`, `A-Z`), numbers (`0-9`), underscores (`_`), apostrophes (`'`), or dashes (`-`).
> *name* = *identifier* | *string* \ > *name* = *identifier* | *string* \
doc/manual/src/quick-start.md
+1 -1
View File
@@ -5,7 +5,7 @@
FIXME(Lix): This chapter is quite outdated with respect to recommended practices in 2024 and needs updating. FIXME(Lix): This chapter is quite outdated with respect to recommended practices in 2024 and needs updating.
The commands in here will work, however, and the installation section is up to date. The commands in here will work, however, and the installation section is up to date.
For more updated guidance, see the links on <https://lix.systems/resources/> For more updated guidance, see the links on <https://wiki.lix.systems/books/lix-users/page/nix-resources>
</div> </div>
doc/manual/substitute.py
-5
View File
@@ -70,11 +70,6 @@ def do_include(content: str, relative_md_path: Path, source_root: Path, search_p
def recursive_replace(data, book_root, search_path): def recursive_replace(data, book_root, search_path):
match data: match data:
# XXX FUTURE: drop sections once mdBook is at 0.5.0 or above in nixpkgs
case {'sections': sections}:
return data | dict(
sections = [recursive_replace(section, book_root, search_path) for section in sections],
)
case {'items': items}: case {'items': items}:
return data | dict( return data | dict(
items = [recursive_replace(item, book_root, search_path) for item in items], items = [recursive_replace(item, book_root, search_path) for item in items],
flake.lock
Generated
+4 -4
View File
@@ -106,16 +106,16 @@
}, },
"nixpkgs_2": { "nixpkgs_2": {
"locked": { "locked": {
"lastModified": 1773082486, "lastModified": 1783770249,
"narHash": "sha256-TKUDrM0nKUo5s/b8jhjXa2prcu5KU5Cck3HBTRLDjfo=", "narHash": "sha256-K8pGvFito5dp9T0+clr60q+bJPGEskK75aAJ39w7HBM=",
"owner": "NixOS", "owner": "NixOS",
"repo": "nixpkgs", "repo": "nixpkgs",
"rev": "7f8b8875bdb38a70c7b5ceb9ba6a6a8d69859e16", "rev": "62463162b3ce92919f19ada41a70a0d943a08da8",
"type": "github" "type": "github"
}, },
"original": { "original": {
"owner": "NixOS", "owner": "NixOS",
"ref": "nixos-25.11-small", "ref": "nixos-26.05-small",
"repo": "nixpkgs", "repo": "nixpkgs",
"type": "github" "type": "github"
} }
flake.nix
+56 -388
View File
@@ -2,7 +2,7 @@
description = "Lix: A modern, delicious implementation of the Nix package manager"; description = "Lix: A modern, delicious implementation of the Nix package manager";
inputs = { inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.11-small"; nixpkgs.url = "github:NixOS/nixpkgs/nixos-26.05-small";
nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2"; nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
# Required because Nix 2.18 is not in Nixpkgs ≥ 25.05 anymore. # Required because Nix 2.18 is not in Nixpkgs ≥ 25.05 anymore.
@@ -42,6 +42,7 @@
let let
inherit (nixpkgs) lib; inherit (nixpkgs) lib;
lixSrc = self;
# This notice gets echoed as a dev shell hook, and can be turned off with # This notice gets echoed as a dev shell hook, and can be turned off with
# `touch .nocontribmsg` # `touch .nocontribmsg`
@@ -76,192 +77,35 @@
(Run `touch .nocontribmsg` to hide this message.) (Run `touch .nocontribmsg` to hide this message.)
''; '';
versionJson = builtins.fromJSON (builtins.readFile ./version.json); scope = import ./nix-support/build/inputs.nix {
officialRelease = versionJson.official_release; inherit
lib
nixpkgs
nix_2_18
nix2container
lixSrc
nixpkgs-regression
;
};
# Set to true to build the release notes for the next release. inherit (scope)
buildUnreleasedNotes = true; crossSystems
darwinSystems
forAllStdenvs
forAllSystems
forAvailableSystems
linux64BitSystems
nixpkgsFor
overlayFor
systems
versionSuffix
;
versionSuffix = inherit (scope.callPackage ./nix-support/build/outputs.nix { })
if officialRelease then packages
"" ciArtifacts
else tests
"pre${ ;
builtins.substring 0 8 (self.lastModifiedDate or self.lastModified or "19700101")
}-dev_${self.shortRev or "dirty"}";
linux32BitSystems = [ "i686-linux" ];
linux64BitSystems = [
"x86_64-linux"
"aarch64-linux"
];
linuxSystems = linux32BitSystems ++ linux64BitSystems;
darwinSystems = [
"x86_64-darwin"
"aarch64-darwin"
];
nonDarwinSystems = linuxSystems;
systems = linuxSystems ++ darwinSystems;
# If you add something here, please update the list in doc/manual/src/contributing/hacking.md.
# Thanks~
crossSystems = [
"armv6l-linux"
"armv7l-linux"
"riscv64-linux"
"aarch64-linux"
"x86_64-freebsd"
# FIXME: broken dev shell due to python
# "x86_64-netbsd"
];
stdenvs = [
# see assertion in package.nix why these two are disabled
# "stdenv"
# "gccStdenv"
"clangStdenv"
"libcxxStdenv"
"ccacheStdenv"
];
forAllSystems = lib.genAttrs systems;
# Same as forAllSystems, but removes nulls, in case something is broken
# on that system.
forAvailableSystems =
f: lib.filterAttrs (name: value: value != null && value != { }) (forAllSystems f);
forAllCrossSystems = lib.genAttrs crossSystems;
forAllStdenvs =
f:
lib.listToAttrs (
map (stdenvName: {
name = "${stdenvName}Packages";
value = f stdenvName;
}) stdenvs
)
// {
# TODO delete this and reënable gcc stdenvs once gcc compiles kj coros correctly
stdenvPackages = f "clangStdenv";
};
# Memoize nixpkgs for different platforms for efficiency.
nixpkgsFor = forAllSystems (
system:
let
make-pkgs =
crossSystem: stdenv:
import nixpkgs {
localSystem = {
inherit system;
};
crossSystem = if crossSystem == null then null else { system = crossSystem; };
overlays = [ (overlayFor (p: p.${stdenv})) ];
};
stdenvs = forAllStdenvs (make-pkgs null);
native = stdenvs.stdenvPackages;
in
{
inherit stdenvs native;
static = native.pkgsStatic;
cross = forAllCrossSystems (crossSystem: make-pkgs crossSystem "clangStdenv");
}
);
overlayFor =
getStdenv: final: prev:
let
currentStdenv = getStdenv final;
in
{
nixStable = prev.nix;
nixVersions = prev.nixVersions // {
# Nix 2.18 has been removed from Nixpkgs ≥ 25.05, so we need to reintroduce it ourselves for our tests.
nix_2_18 =
nix_2_18.outputs.packages.${currentStdenv.hostPlatform.system}.default.overrideAttrs
(_: {
pname = "nix";
});
};
# Forward from the previous stage as we don’t want it to pick the lowdown override
nixUnstable = prev.nixUnstable;
check-headers = final.buildPackages.callPackage ./maintainers/check-headers.nix { };
check-syscalls = final.buildPackages.callPackage ./maintainers/check-syscalls.nix { };
default-busybox-sandbox-shell = final.busybox.override {
useMusl = true;
enableStatic = true;
enableMinimal = true;
extraConfig = ''
CONFIG_FEATURE_FANCY_ECHO y
CONFIG_FEATURE_SH_MATH y
CONFIG_FEATURE_SH_MATH_64 y
CONFIG_ASH y
CONFIG_ASH_OPTIMIZE_FOR_SIZE y
CONFIG_ASH_ALIAS y
CONFIG_ASH_BASH_COMPAT y
CONFIG_ASH_CMDCMD y
CONFIG_ASH_ECHO y
CONFIG_ASH_GETOPTS y
CONFIG_ASH_INTERNAL_GLOB y
CONFIG_ASH_JOB_CONTROL y
CONFIG_ASH_PRINTF y
CONFIG_ASH_TEST y
'';
};
nix = final.callPackage ./package.nix {
inherit versionSuffix officialRelease;
stdenv = currentStdenv;
busybox-sandbox-shell = final.busybox-sandbox-shell or final.default-busybox-sandbox-shell;
};
lix-clang-tidy = final.callPackage ./subprojects/lix-clang-tidy { };
nix-eval-jobs = final.callPackage ./subprojects/nix-eval-jobs {
stdenv = currentStdenv;
srcDir = ./subprojects/nix-eval-jobs;
};
# HACK: We need nix-prefetch-git for fetchCargoVendor for Rust stuff,
# so it can't use Lix, or we infrec:
# lix -> Rust stuff -> fetchCargoVendor -> nix-prefetch-git -> nix (lix)
# This will eventually become a problem upstream, but until then,
# apply some duct tape and pray.
nix-prefetch-git =
if (lib.functionArgs prev.nix-prefetch-git.override) ? "nix" then
prev.nix-prefetch-git.override { nix = prev.nix; }
else
prev.nix-prefetch-git;
# Export the patched version of boehmgc that Lix uses into the overlay
# for consumers of this flake.
boehmgc-nix = final.nix.passthru.boehmgc-nix;
# And same thing for our build-release-notes package.
build-release-notes = final.nix.passthru.build-release-notes;
lowdown =
assert lib.versionAtLeast prev.lowdown.version "2.0.0";
prev.lowdown;
capnproto = prev.capnproto.overrideAttrs (old: {
patches =
old.patches or [ ]
++ [
# backport of https://github.com/capnproto/capnproto/pull/1810
./misc/capnproto-promise-nodiscard.patch
]
++ lib.optionals (lib.versionOlder old.version "1.2.0") [
# backport of https://github.com/capnproto/capnproto/pull/2296
./misc/capnproto-monotonic-clocks-are-a-lie.patch
];
});
};
in in
{ {
# for repl debugging # for repl debugging
@@ -271,169 +115,13 @@
# 'nix.perl-bindings' packages. # 'nix.perl-bindings' packages.
overlays.default = overlayFor (p: p.clangStdenv); overlays.default = overlayFor (p: p.clangStdenv);
hydraJobs = { hydraJobs = ciArtifacts // {
# Binary package for various platforms.
build = forAllSystems (system: self.packages.${system}.nix);
# Building Lix twice in CI is expensive, but we can catch a lot of static
# build regressions by at least making sure it evals and configures.
configure-static = lib.genAttrs linux64BitSystems (
system:
self.packages.${system}.nix-static.overrideAttrs {
dontBuild = true;
installPhase = ''
runHook preInstall
echo "configure-static complete. exiting with success"
mkdir -p "$out"
exit 0
'';
}
);
devShell = forAllSystems (system: { devShell = forAllSystems (system: {
default = self.devShells.${system}.default; default = self.devShells.${system}.default;
clang = self.devShells.${system}.native-clangStdenvPackages; clang = self.devShells.${system}.native-clangStdenvPackages;
}); });
rl-next = forAllSystems ( inherit tests;
system:
let
rl-next-check =
name: dir:
let
pkgs = nixpkgsFor.${system}.native;
in
pkgs.buildPackages.runCommand "test-${name}-release-notes" { } ''
LANG=C.UTF-8 ${lib.getExe pkgs.build-release-notes} --change-authors ${./doc/manual/change-authors.yml} ${dir} >$out
'';
in
{
user = rl-next-check "rl-next" ./doc/manual/rl-next;
}
);
# Completion tests for the Nix REPL.
repl-completion = forAllSystems (
system: nixpkgsFor.${system}.native.callPackage ./tests/repl-completion.nix { }
);
# Perl bindings for various platforms.
perlBindings = forAllSystems (system: nixpkgsFor.${system}.native.nix.passthru.perl-bindings);
# nix-eval-jobs can be built against this Lix.
nix-eval-jobs = forAllSystems (system: nixpkgsFor.${system}.native.nix-eval-jobs);
# Binary tarball for various platforms, containing a Nix store
# with the closure of 'nix' package.
binaryTarball = forAllSystems (system: nixpkgsFor.${system}.native.nix.passthru.binaryTarball);
# docker image with Lix inside
dockerImage = lib.genAttrs linux64BitSystems (system: self.packages.${system}.dockerImage);
# API docs for Nix's unstable internal C++ interfaces.
internal-api-docs =
let
nixpkgs = nixpkgsFor.x86_64-linux.native;
inherit (nixpkgs) pkgs;
nix = pkgs.callPackage ./package.nix {
inherit versionSuffix officialRelease buildUnreleasedNotes;
inherit (pkgs) build-release-notes;
# Required since we don't support gcc stdenv
stdenv = pkgs.clangStdenv;
internalApiDocs = true;
busybox-sandbox-shell = pkgs.busybox-sandbox-shell;
};
in
nix.overrideAttrs (prev: {
# This Hydra job is just for the internal API docs.
# We don't need the build artifacts here.
dontBuild = true;
doCheck = false;
doInstallCheck = false;
});
# System tests.
tests =
import ./tests/nixos {
inherit
self
lib
nixpkgs
nixpkgsFor
;
}
// {
# the n-e-j test suite is unusably slow in darwin ci. disbled until anywho fixes this.
nix-eval-jobs = (lib.genAttrs nonDarwinSystems) (
system: self.packages.${system}.nix-eval-jobs.tests.nix-eval-jobs
);
# This is x86_64-linux only, just because we have significantly
# cheaper x86_64-linux compute in CI.
# It is clangStdenv because clang's sanitizers are nicer.
asanBuild = self.packages.x86_64-linux.nix-clangStdenv.override {
# Improve caching of non-code changes by not changing the
# derivation name every single time, since this will never be seen
# by users anyway.
versionSuffix = "";
sanitize = [
"address"
"undefined"
];
# it is very hard to make *every* CI build use this option such
# that we don't wind up building Lix twice, so we do it here where
# we are already doing so.
werror = true;
};
# Although this might be nicer to do with pre-commit, that would
# require adding 12MB of nodejs to the dev shell, whereas building it
# in CI with Nix avoids that at a cost of slower feedback on rarely
# touched files.
jsSyntaxCheck =
let
nixpkgs = nixpkgsFor.x86_64-linux.native;
inherit (nixpkgs) pkgs;
docSources = lib.fileset.toSource {
root = ./doc;
fileset = lib.fileset.fileFilter (f: f.hasExt "js") ./doc;
};
in
pkgs.runCommand "js-syntax-check" { } ''
find ${docSources} -type f -print -exec ${pkgs.nodejs-slim}/bin/node --check '{}' ';'
touch $out
'';
# clang-tidy run against the Lix codebase using the Lix clang-tidy plugin
clang-tidy = forAllSystems (
system:
let
pkgs = nixpkgsFor.${system}.native;
in
pkgs.callPackage ./package.nix {
# Required since we don't support gcc stdenv
stdenv = pkgs.clangStdenv;
versionSuffix = "";
lintInsteadOfBuild = true;
}
);
# Make sure that nix-env still produces the exact same result
# on a particular version of Nixpkgs.
evalNixpkgs = nixpkgsFor.x86_64-linux.native.callPackage ./tests/nixpkgs/eval.nix {
inherit nixpkgs-regression;
};
nixpkgsLibTests = forAllSystems (
system:
nixpkgsFor.${system}.native.callPackage ./tests/nixpkgs/lib.nix {
inherit nixpkgs system;
inherit (self.packages.${system}) nix;
}
);
};
pre-commit = forAvailableSystems ( pre-commit = forAvailableSystems (
system: system:
@@ -493,45 +181,7 @@
} }
); );
packages = forAllSystems ( inherit packages;
system:
rec {
inherit (nixpkgsFor.${system}.native) nix;
default = nix;
inherit (nixpkgsFor.${system}.native) lix-clang-tidy nix-eval-jobs;
}
// (
lib.optionalAttrs (builtins.elem system linux64BitSystems) {
# python doesn't work in static builds as of 2025-06-27
nix-static = nixpkgsFor.${system}.static.nix.overrideAttrs (_: {
doCheck = false;
});
dockerImage =
let
pkgs = nixpkgsFor.${system}.native;
nix2container' = import nix2container { inherit pkgs; };
in
import ./docker.nix {
inherit pkgs;
nix2container = nix2container'.nix2container;
tag = pkgs.nix.version;
};
}
// builtins.listToAttrs (
map (crossSystem: {
name = "nix-${crossSystem}";
value = nixpkgsFor.${system}.cross.${crossSystem}.nix;
}) crossSystems
)
// builtins.listToAttrs (
map (stdenvName: {
name = "nix-${stdenvName}";
value = nixpkgsFor.${system}.stdenvs."${stdenvName}Packages".nix;
}) stdenvs
)
)
);
devShells = devShells =
let let
@@ -564,12 +214,30 @@
in in
(makeShells "native" nixpkgsFor.${system}.native) (makeShells "native" nixpkgsFor.${system}.native)
// (makeShells "static" nixpkgsFor.${system}.static) // (makeShells "static" nixpkgsFor.${system}.static)
// (forAllCrossSystems ( // (lib.listToAttrs (
crossSystem: # Provide e.g., both '.#native-aarch64-linux` and `.#static-aarch64-linux`,
let # for each cross-system.
pkgs = nixpkgsFor.${system}.cross.${crossSystem}; # "native" feels like a misnomer here since it's literally cross compiling,
in # but at least it's consistent with the native/static dichotomy we've set up.
makeShell pkgs pkgs.clangStdenv lib.concatMap (
crossSystem:
let
pkgs = nixpkgsFor.${system}.cross.${crossSystem};
inherit (pkgs) pkgsStatic;
native = makeShell pkgs pkgs.clangStdenv;
static = makeShell pkgsStatic pkgsStatic.clangStdenv;
in
[
{
name = "native-${crossSystem}";
value = native;
}
{
name = "static-${crossSystem}";
value = static;
}
]
) crossSystems
)) ))
// { // {
default = self.devShells.${system}.native-clangStdenvPackages; default = self.devShells.${system}.native-clangStdenvPackages;
justfile
+41 -25
View File
@@ -1,51 +1,67 @@
# https://just.systems/man/en/ # https://just.systems/man/en/
#
# Take a look at ./doc/manual/src/contributing/hacking.md for a detailed
# explanation on how to use this file!
# Pin the shell to bash (anything sufficiently POSIX-y would do)
# HACK: We use https://github.com/casey/just#positional-arguments
# and `"@$"` to forward arguments to the inner commands.
# The reason we require this is that `{{ OPTIONS }}` does not escape any values,
# and thus requires one additional level of escaping when running `just` commands with e.g. spaces in them.
# just provides no good solution to this problem, so we have to rely on its forwarding of arguments and shell semantics.
set shell := ["bash", "-uc"]
outdir := x"${out:-$PWD/outputs/out}"
builddir := "build"
# List all available targets # List all available targets
list: list:
just --list just --list
# Clean build artifacts # Clean build artifacts and outputs.
clean: clean:
rm -rf build rm -rf {{ quote(builddir) }}/* {{ quote(builddir) }}/.* {{ quote(outdir) }}/* {{ quote(outdir) }}/.*
cargo clean
# Prepare meson for building with extra options # Prepare meson for building.
setup-custom *OPTIONS: [positional-arguments]
meson setup build --prefix="$PWD/outputs/out" $mesonFlags {{ OPTIONS }} setup *OPTIONS:
meson setup {{ builddir }} --reconfigure --prefix="{{outdir}}" $mesonFlags "$@"
# Prepare meson for building
setup: (setup-custom)
# Build lix with extra options # Build lix with extra options
build-custom *OPTIONS: [positional-arguments]
meson compile -C build {{ OPTIONS }} build *OPTIONS:
meson compile -C {{ builddir }} "$@"
# Build lix
build: (build-custom)
alias compile := build alias compile := build
# Install lix for local development with extra options # `meson install` will automatically build anything that needs to be built to install it.
install-custom *OPTIONS: (build-custom OPTIONS) [doc("Install Lix for local development")]
meson install -C build [positional-arguments]
install *OPTIONS:
meson install --quiet -C {{ builddir }} "$@"
# Install lix for local development # Run all tests tests (installs first).
install: (install-custom) [positional-arguments]
test *OPTIONS: (install)
# Run tests (usually requires `install`) with extra options meson test -C {{ builddir }} --print-errorlogs --max-lines 10000 "$@"
test *OPTIONS:
meson test -C build --print-errorlogs --max-lines 10000 {{ OPTIONS }}
# Run unit tests only # Run unit tests only
test-unit *OPTIONS: (test "--suite" "check") test-unit *OPTIONS: (test "--suite" "check")
# Run integration tests only # Run integration tests only
test-integration *OPTIONS: install (test "--suite" "installcheck") test-integration *OPTIONS: (test "--suite" "installcheck" OPTIONS)
# Run functional2 tests using pytest directly, allowing for additional arguments to be passed to pytest e.g. for more granular test selection # Run functional2 tests using pytest directly, allowing for additional arguments to be passed to pytest e.g. for more granular test selection
[positional-arguments]
test-functional2 *OPTIONS: test-functional2 *OPTIONS:
cd tests/functional2 && python -m pytest -v {{ OPTIONS }} cd tests/functional2 && python -m pytest -v "$@"
alias clang-tidy := lint # special target for cargo because meson cannot be convinced to not mangle cargo test output,
# and getting properly colored test output any other way also doesn't look all that possible.
[positional-arguments]
test-rs *OPTIONS:
meson test -C {{ builddir }} --interactive lix-rs-tests "$@"
# Lint with `clang-tidy` # Lint with `clang-tidy`
lint: lint:
lix/code-generation/build_builtins.py
+1 -1
View File
@@ -52,7 +52,7 @@ class Builtin:
</dt> </dt>
<dd> <dd>
{indent(self.documentation, " " * 3)} {indent(self.documentation, " " * 3)}
{ {
f"This function is only available if the [{self.experimental_feature}](@docroot@/contributing/experimental-features.md#xp-feature-{self.experimental_feature}) experimental feature is enabled." f"This function is only available if the [{self.experimental_feature}](@docroot@/contributing/experimental-features.md#xp-feature-{self.experimental_feature}) experimental feature is enabled."
lix/code-generation/build_extra_features.py
+1 -1
View File
@@ -52,7 +52,7 @@ class ExtraFeature:
{ {
"\n ".join( "\n ".join(
[ [
f"- {event.date}, {event.release}: {event.message} [{", ".join([f'[CL {cl}](https://git.lix.systems/c/lix/+/{cl})' for cl in event.cls])}]" f"- {event.date}, {event.release}: {event.message} [{", ".join([f'[CL {cl}](https://gerrit.lix.systems/c/lix/+/{cl})' for cl in event.cls])}]"
for event in self.timeline for event in self.timeline
] ]
) )
lix/code-generation/build_settings.py
+1 -1
View File
@@ -90,7 +90,7 @@ class Setting:
aliases = [f"`{item}`" for item in self.aliases] aliases = [f"`{item}`" for item in self.aliases]
description = dedent(f""" description = dedent(f"""
{indent(indentation, self.documentation)} {indent(indentation, self.documentation)}
{indent(indentation, PLATFORM_WARNING.format(platforms=str(platforms)[1:-1])) if self.platforms else ""} {indent(indentation, PLATFORM_WARNING.format(platforms=str(platforms)[1:-1])) if self.platforms else ""}
{indent(indentation, XP_WARNING.format(feature=self.experimental_feature, name=self.name)) if self.experimental_feature is not None else ""} {indent(indentation, XP_WARNING.format(feature=self.experimental_feature, name=self.name)) if self.experimental_feature is not None else ""}
lix/legacy/build-remote.cc
+32 -112
View File
@@ -5,6 +5,7 @@
#include "lix/libutil/file-descriptor.hh" #include "lix/libutil/file-descriptor.hh"
#include "lix/libutil/logging-rpc.hh" #include "lix/libutil/logging-rpc.hh"
#include "lix/libutil/logging.hh" #include "lix/libutil/logging.hh"
#include "lix/libutil/result.hh"
#include "lix/libutil/rpc.hh" #include "lix/libutil/rpc.hh"
#include "lix/libutil/types-rpc.hh" // IWYU pragma: keep #include "lix/libutil/types-rpc.hh" // IWYU pragma: keep
#include "lix/libutil/types.hh" #include "lix/libutil/types.hh"
@@ -50,7 +51,7 @@ struct Instance final : rpc::build_remote::HookInstance::Server
kj::Promise<void> init(InitContext context) override; kj::Promise<void> init(InitContext context) override;
kj::Promise<void> buildImpl(BuildContext context); kj::Promise<Result<void>> buildImpl(BuildContext context);
kj::Promise<void> build(BuildContext context) override; kj::Promise<void> build(BuildContext context) override;
}; };
} }
@@ -96,7 +97,7 @@ static std::tuple<bool, Machine *, AutoCloseFD> selectBestMachine(
uint64_t bestLoad = 0; uint64_t bestLoad = 0;
for (auto & m : machines) { for (auto & m : machines) {
debug("considering building on remote machine '%s'", m.storeUri); debug("considering building on remote machine '%s'", m.name);
if (m.enabled && m.systemSupported(neededSystem) && m.allSupported(requiredFeatures) if (m.enabled && m.systemSupported(neededSystem) && m.allSupported(requiredFeatures)
&& m.mandatoryMet(requiredFeatures)) && m.mandatoryMet(requiredFeatures))
@@ -184,67 +185,6 @@ struct BuilderConnection
AutoCloseFD slotLock; AutoCloseFD slotLock;
std::shared_ptr<Store> sshStore; std::shared_ptr<Store> sshStore;
std::string storeUri; std::string storeUri;
Pipe logPipe;
// start the thread that reads ssh stderr and turns it into log items.
// this future *must* outlive sshStore, otherwise it will never finish
kj::Promise<Result<void>> startLogThread(std::string buildDescription, std::string drvPath)
try {
if (!logPipe.readSide) {
co_return result::success();
}
logPipe.writeSide.close();
// NOTE this is very similar to handleBuilderOutput in DerivationGoal, but unlike
// the derivation goal we do not need to handle EIO from a pty here. we also have
// no timeouts or limits to keep track of, which makes deduplication less useful.
auto act = logger->startActivity(
lvlInfo, actBuild, buildDescription, Logger::Fields{drvPath, storeUri, 1, 1}
);
std::map<ActivityId, Activity> activities;
auto reader = AIO().lowLevelProvider.wrapInputFd(logPipe.readSide.get());
LogLineSplitter splitter;
auto flushLine = [&](const std::string & line) {
if (const auto state =
handleJSONLogMessage(line, act, activities, "the derivation builder"))
{
return *state;
} else {
return act.result(resBuildLogLine, line);
}
};
auto buf = kj::heapArray<char>(4096);
while (true) {
const auto got = co_await reader->tryRead(buf.begin(), 1, buf.size());
if (got == 0) {
break;
}
std::string_view data{buf.begin(), got};
while (!data.empty()) {
if (auto line = splitter.feed(data)) {
if (flushLine(*line) == Logger::BufferState::NeedsFlush) {
TRY_AWAIT(act.getLogger().flush());
}
}
}
}
if (auto line = splitter.finish(); !line.empty()) {
(void) flushLine(line);
TRY_AWAIT(act.getLogger().flush());
}
co_return result::success();
} catch (...) {
co_return result::current_exception();
}
}; };
struct AcceptedBuild final : rpc::build_remote::HookInstance::AcceptedBuild::Server struct AcceptedBuild final : rpc::build_remote::HookInstance::AcceptedBuild::Server
@@ -261,7 +201,7 @@ struct AcceptedBuild final : rpc::build_remote::HookInstance::AcceptedBuild::Ser
{ {
} }
kj::Promise<void> runImpl(RunContext context); kj::Promise<Result<void>> runImpl(RunContext context);
kj::Promise<void> run(RunContext context) override; kj::Promise<void> run(RunContext context) override;
}; };
@@ -326,26 +266,15 @@ try {
lock.reset(); lock.reset();
std::shared_ptr<Store> sshStore; std::shared_ptr<Store> sshStore;
Pipe logPipe;
try { try {
auto act = logger->startActivity( auto act =
lvlTalkative, actUnknown, fmt("connecting to '%s'", bestMachine->storeUri) logger->startActivity(lvlTalkative, actUnknown, fmt("connecting to '%s'", bestMachine->name));
);
std::tie(sshStore, logPipe) = TRY_AWAIT(bestMachine->openStore()); sshStore = TRY_AWAIT(bestMachine->openStore());
TRY_AWAIT(sshStore->connect()); co_return BuilderConnection{std::move(bestSlotLock), sshStore, bestMachine->storeUri};
co_return BuilderConnection{
std::move(bestSlotLock), sshStore, bestMachine->storeUri, std::move(logPipe)
};
} catch (std::exception & e) { // NOLINT(lix-foreign-exceptions) } catch (std::exception & e) { // NOLINT(lix-foreign-exceptions)
std::string msg = logPipe.readSide ? chomp(drainFD(logPipe.readSide.get(), false)) : ""; printError("cannot build on '%s': %s", bestMachine->name, e.what());
printError(
"cannot build on '%s': %s%s",
bestMachine->storeUri,
e.what(),
msg.empty() ? "" : ": " + msg
);
bestMachine->enabled = false; bestMachine->enabled = false;
} }
} }
@@ -366,7 +295,7 @@ static int main_build_remote(AsyncIoRoot & aio, std::string programName, Strings
if (argv.size() != 1) if (argv.size() != 1)
throw UsageError("called without required arguments"); throw UsageError("called without required arguments");
verbosity = (Verbosity) std::stoll(argv.front()); setVerbosity((Verbosity) std::stoll(argv.front()));
auto conn = aio.kj.lowLevelProvider->wrapUnixSocketFd(1); auto conn = aio.kj.lowLevelProvider->wrapUnixSocketFd(1);
capnp::TwoPartyServer srv(kj::heap<Instance>()); capnp::TwoPartyServer srv(kj::heap<Instance>());
@@ -395,16 +324,14 @@ kj::Promise<void> Instance::init(InitContext context)
initPlugins(); initPlugins();
initialized = true; initialized = true;
context.getResults().initResult().setGood();
} catch (...) { } catch (...) {
RPC_FILL(context.getResults(), initResult, std::current_exception()); rpc::rethrow_as_rpc_error();
} }
return kj::READY_NOW; return kj::READY_NOW;
} }
kj::Promise<void> Instance::buildImpl(BuildContext context) kj::Promise<Result<void>> Instance::buildImpl(BuildContext context)
try { try {
if (!initialized) { if (!initialized) {
throw Error("build hook not fully initialized"); throw Error("build hook not fully initialized");
@@ -431,8 +358,8 @@ try {
debug("got %d remote builders", machines.size()); debug("got %d remote builders", machines.size());
if (machines.empty()) { if (machines.empty()) {
context.getResults().initResult().initGood().setDeclinePermanently(); context.getResults().initResult().setDeclinePermanently();
co_return; co_return result::success();
} }
auto amWilling = context.getParams().getAmWilling(); auto amWilling = context.getParams().getAmWilling();
@@ -448,21 +375,23 @@ try {
if (auto immediateResponse = std::get_if<BuildRejected>(&result)) { if (auto immediateResponse = std::get_if<BuildRejected>(&result)) {
switch (*immediateResponse) { switch (*immediateResponse) {
case BuildRejected::Temporarily: case BuildRejected::Temporarily:
context.getResults().initResult().initGood().setPostpone(); context.getResults().initResult().setPostpone();
co_return; co_return result::success();
case BuildRejected::Permanently: case BuildRejected::Permanently:
context.getResults().initResult().initGood().setDecline(); context.getResults().initResult().setDecline();
co_return; co_return result::success();
} }
} }
auto builder = std::get_if<BuilderConnection>(&result); auto builder = std::get_if<BuilderConnection>(&result);
assert(builder); assert(builder);
auto ac = context.getResults().initResult().initGood().initAccept(); auto ac = context.getResults().initResult().initAccept();
ac.setMachine(kj::heap<AcceptedBuild>(store, drvPath, std::move(*builder))); ac.setMachine(kj::heap<AcceptedBuild>(store, drvPath, std::move(*builder)));
co_return result::success();
} catch (...) { } catch (...) {
RPC_FILL(context.getResults(), initResult, std::current_exception()); co_return result::current_exception();
} }
kj::Promise<void> Instance::build(BuildContext context) kj::Promise<void> Instance::build(BuildContext context)
@@ -471,11 +400,12 @@ try {
throw Error("build hooks can only accept a single job"); throw Error("build hooks can only accept a single job");
} }
used = true; // lock out other rpc calls during processing used = true; // lock out other rpc calls during processing
co_await buildImpl(context); auto result = co_await buildImpl(context);
TRY_AWAIT(logger->flush()); TRY_AWAIT(logger->flush());
used = context.getResults().getResult().getGood().isAccept(); used = result.has_value() && context.getResults().getResult().isAccept();
result.value();
} catch (...) { } catch (...) {
RPC_FILL(context.getResults(), getResult, std::current_exception()); rpc::rethrow_as_rpc_error();
} }
kj::Promise<void> AcceptedBuild::run(RunContext context) kj::Promise<void> AcceptedBuild::run(RunContext context)
@@ -493,23 +423,17 @@ kj::Promise<void> AcceptedBuild::run(RunContext context)
throw Error("build hooks builds are single-use items"); throw Error("build hooks builds are single-use items");
} }
used = true; used = true;
co_await runImpl(context); auto result = co_await runImpl(context);
TRY_AWAIT(logger->flush()); TRY_AWAIT(logger->flush());
result.value();
} catch (...) { } catch (...) {
RPC_FILL(context.getResults(), getResult, std::current_exception()); rpc::rethrow_as_rpc_error();
} }
} }
kj::Promise<void> AcceptedBuild::runImpl(RunContext context) kj::Promise<Result<void>> AcceptedBuild::runImpl(RunContext context)
{ {
try { try {
auto logHandler = builder.startLogThread(
fmt("%s on '%s'",
rpc::to<std::string_view>(context.getParams().getDescription()),
builder.storeUri),
store->printStorePath(drvPath)
);
auto & sshStore = builder.sshStore; auto & sshStore = builder.sshStore;
auto & storeUri = builder.storeUri; auto & storeUri = builder.storeUri;
@@ -619,13 +543,9 @@ kj::Promise<void> AcceptedBuild::runImpl(RunContext context)
); );
} }
// drop store connection, let log handler process any remaining input co_return result::success();
builder.sshStore = nullptr;
TRY_AWAIT(logHandler);
context.getResults().initResult().setGood();
} catch (...) { } catch (...) {
RPC_FILL(context.getResults(), initResult, std::current_exception()); co_return result::current_exception();
} }
} }
lix/legacy/builtin-builder.cc
+1 -1
View File
@@ -35,7 +35,7 @@ static int main_builtin_builder(AsyncIoRoot & aio, std::string programName, Stri
}; };
if (auto val = string2Int<int>(getArg("verbosity"))) { if (auto val = string2Int<int>(getArg("verbosity"))) {
verbosity = verbosityFromIntClamped(*val); setVerbosity(verbosityFromIntClamped(*val));
} else { } else {
throw Error("expected a verbosity argument"); throw Error("expected a verbosity argument");
} }
lix/legacy/nix-build.cc
+1 -1
View File
@@ -213,7 +213,7 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
auto autoArgsWithInNixShell = autoArgs; auto autoArgsWithInNixShell = autoArgs;
if (runEnv) { if (runEnv) {
auto newArgs = evaluator->buildBindings(autoArgsWithInNixShell->size() + 1); auto newArgs = evaluator->buildBindings(autoArgsWithInNixShell->size() + 1);
newArgs.alloc("inNixShell") = {NewValueAs::boolean, true}; newArgs.insert("inNixShell", {NewValueAs::boolean, true});
for (auto & i : *autoArgs) newArgs.insert(i); for (auto & i : *autoArgs) newArgs.insert(i);
autoArgsWithInNixShell = newArgs.finish(); autoArgsWithInNixShell = newArgs.finish();
} }
lix/legacy/nix-copy-closure.cc
+1 -1
View File
@@ -55,7 +55,7 @@ static int main_nix_copy_closure(AsyncIoRoot & aio, std::string programName, Str
for (auto & path : storePaths) for (auto & path : storePaths)
storePaths2.insert(from->followLinksToStorePath(path)); storePaths2.insert(from->followLinksToStorePath(path));
aio.blockOn(copyClosure(*from, *to, storePaths2, NoRepair, NoCheckSigs, useSubstitutes)); aio.blockOn(copyClosure(*from, *to, storePaths2, NoRepair, NoCheckSigs, useSubstitutes, includeOutputs));
return 0; return 0;
} }
lix/legacy/nix-env.cc
+2 -3
View File
@@ -154,8 +154,7 @@ static void getAllExprs(Evaluator & state,
Value vArg = {NewValueAs::string, path2.canonical().abs()}; Value vArg = {NewValueAs::string, path2.canonical().abs()};
if (seen.size() == maxAttrs) if (seen.size() == maxAttrs)
throw Error("too many Nix expressions in directory '%1%'", path); throw Error("too many Nix expressions in directory '%1%'", path);
attrs.alloc(attrName attrs.insert(attrName, {NewValueAs::app, state.mem, state.builtins.get("import"), vArg});
) = {NewValueAs::app, state.mem, state.builtins.get("import"), vArg};
} }
else if (st.type == InputAccessor::tDirectory) else if (st.type == InputAccessor::tDirectory)
/* `path2' is a directory (with no default.nix in it); /* `path2' is a directory (with no default.nix in it);
@@ -180,7 +179,7 @@ static Value loadSourceExpr(EvalState & state, const SourcePath & path_)
directory). */ directory). */
else if (st.type == InputAccessor::tDirectory) { else if (st.type == InputAccessor::tDirectory) {
auto attrs = state.ctx.buildBindings(maxAttrs); auto attrs = state.ctx.buildBindings(maxAttrs);
attrs.alloc("_combineChannels") = Value::EMPTY_LIST; attrs.insert("_combineChannels", Value::EMPTY_LIST);
StringSet seen; StringSet seen;
getAllExprs(state.ctx, path, seen, attrs); getAllExprs(state.ctx, path, seen, attrs);
return {NewValueAs::attrs, attrs}; return {NewValueAs::attrs, attrs};
lix/legacy/nix-instantiate.cc
+1 -5
View File
@@ -42,11 +42,7 @@ void processExpr(EvalState & state, const Strings & attrPaths,
NixStringContext context; NixStringContext context;
if (evalOnly) { if (evalOnly) {
Value vRes; Value vRes = autoArgs.empty() ? v : state.autoCallFunction(autoArgs, v, noPos);
if (autoArgs.empty())
vRes = v;
else
vRes = state.autoCallFunction(autoArgs, v, noPos);
if (output == okRaw) if (output == okRaw)
std::cout << *state.coerceToString(noPos, vRes, context, "while generating the nix-instantiate output", StringCoercionMode::Strict); std::cout << *state.coerceToString(noPos, vRes, context, "while generating the nix-instantiate output", StringCoercionMode::Strict);
// We intentionally don't output a newline here. The default PS1 for Bash in NixOS starts with a newline // We intentionally don't output a newline here. The default PS1 for Bash in NixOS starts with a newline
lix/legacy/nix-store.cc
+1 -1
View File
@@ -948,7 +948,7 @@ opServe(std::shared_ptr<Store> store, AsyncIoRoot & aio, Strings opFlags, String
auto getBuildSettings = [&]() { auto getBuildSettings = [&]() {
// FIXME: changing options here doesn't work if we're // FIXME: changing options here doesn't work if we're
// building through the daemon. // building through the daemon.
verbosity = lvlError; setVerbosity(lvlError);
settings.keepLog.override(false); settings.keepLog.override(false);
settings.useSubstitutes.override(false); settings.useSubstitutes.override(false);
settings.maxSilentTime.override(readNum<unsigned>(in)); settings.maxSilentTime.override(readNum<unsigned>(in));
lix/legacy/user-env.cc
+18 -17
View File
@@ -46,30 +46,31 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
auto attrs = state.ctx.buildBindings(7 + outputs.size()); auto attrs = state.ctx.buildBindings(7 + outputs.size());
attrs.alloc(state.ctx.symbols.sym_type) = {NewValueAs::string, "derivation"}; attrs.insert(state.ctx.symbols.sym_type, {NewValueAs::string, "derivation"});
attrs.alloc(state.ctx.symbols.sym_name) = {NewValueAs::string, i.queryName(state)}; attrs.insert(state.ctx.symbols.sym_name, {NewValueAs::string, i.queryName(state)});
auto system = i.querySystem(state); auto system = i.querySystem(state);
if (!system.empty()) if (!system.empty())
attrs.alloc(state.ctx.symbols.sym_system) = {NewValueAs::string, system}; attrs.insert(state.ctx.symbols.sym_system, {NewValueAs::string, system});
attrs.alloc(state.ctx.symbols.sym_outPath) = { attrs.insert(
NewValueAs::string, state.ctx.store->printStorePath(i.queryOutPath(state)) state.ctx.symbols.sym_outPath,
}; {NewValueAs::string, state.ctx.store->printStorePath(i.queryOutPath(state))}
);
if (drvPath) if (drvPath)
attrs.alloc(state.ctx.symbols.sym_drvPath) = { attrs.insert(
NewValueAs::string, state.ctx.store->printStorePath(*drvPath) state.ctx.symbols.sym_drvPath, {NewValueAs::string, state.ctx.store->printStorePath(*drvPath)}
}; );
// Copy each output meant for installation. // Copy each output meant for installation.
auto & vOutputs = attrs.alloc(state.ctx.symbols.sym_outputs);
auto outputsList = state.ctx.mem.newList(outputs.size()); auto outputsList = state.ctx.mem.newList(outputs.size());
vOutputs = {NewValueAs::list, outputsList}; attrs.insert(state.ctx.symbols.sym_outputs, {NewValueAs::list, outputsList});
for (const auto & [m, j] : enumerate(outputs)) { for (const auto & [m, j] : enumerate(outputs)) {
outputsList->elems[m] = {NewValueAs::string, j.first}; outputsList->elems[m] = {NewValueAs::string, j.first};
auto outputAttrs = state.ctx.buildBindings(2); auto outputAttrs = state.ctx.buildBindings(2);
outputAttrs.alloc(state.ctx.symbols.sym_outPath) = { outputAttrs.insert(
NewValueAs::string, state.ctx.store->printStorePath(*j.second) state.ctx.symbols.sym_outPath,
}; {NewValueAs::string, state.ctx.store->printStorePath(*j.second)}
attrs.alloc(j.first) = {NewValueAs::attrs, outputAttrs}; );
attrs.insert(j.first, {NewValueAs::attrs, outputAttrs});
/* This is only necessary when installing store paths, e.g., /* This is only necessary when installing store paths, e.g.,
`nix-env -i /nix/store/abcd...-foo'. */ `nix-env -i /nix/store/abcd...-foo'. */
@@ -87,7 +88,7 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
meta.insert(state.ctx.symbols.create(j), *v); meta.insert(state.ctx.symbols.create(j), *v);
} }
attrs.alloc(state.ctx.symbols.sym_meta) = {NewValueAs::attrs, meta}; attrs.insert(state.ctx.symbols.sym_meta, {NewValueAs::attrs, meta});
manifest->elems[n++] = {NewValueAs::attrs, attrs}; manifest->elems[n++] = {NewValueAs::attrs, attrs};
@@ -111,7 +112,7 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
/* Construct a Nix expression that calls the user environment /* Construct a Nix expression that calls the user environment
builder with the manifest as argument. */ builder with the manifest as argument. */
auto attrs = state.ctx.buildBindings(3); auto attrs = state.ctx.buildBindings(3);
state.ctx.paths.mkStorePathString(manifestFile, attrs.alloc("manifest")); attrs.insert("manifest", state.ctx.paths.mkStorePathString(manifestFile));
attrs.insert(state.ctx.symbols.create("derivations"), vManifest); attrs.insert(state.ctx.symbols.create("derivations"), vManifest);
Value args = {NewValueAs::attrs, attrs}; Value args = {NewValueAs::attrs, attrs};
lix/libcmd/common-eval-args.cc
+87 -39
View File
@@ -1,3 +1,6 @@
#include "libexpr/value.hh"
#include "libutil/strings.hh"
#include "lix/libexpr/attr-path.hh"
#include "lix/libexpr/eval-settings.hh" #include "lix/libexpr/eval-settings.hh"
#include "lix/libcmd/common-eval-args.hh" #include "lix/libcmd/common-eval-args.hh"
#include "lix/libmain/shared.hh" #include "lix/libmain/shared.hh"
@@ -10,31 +13,10 @@
#include "lix/libcmd/command.hh" #include "lix/libcmd/command.hh"
#include "lix/libutil/async.hh" #include "lix/libutil/async.hh"
#include "lix/libutil/error.hh" #include "lix/libutil/error.hh"
#include "lix/libutil/regex.hh" #include <deque>
#include <regex>
namespace nix { namespace nix {
static std::regex const identifierRegex = regex::parse("^[A-Za-z_][A-Za-z0-9_'-]*$");
static void checkValidNixIdentifier(const std::string & name)
{
std::smatch match;
if (!std::regex_match(name, match, identifierRegex)) {
throw UsageError(
"This invocation specifies a value for argument '%s' "
"which isn't a valid Nix identifier. "
"The project is dropping support for this so that it's possible to make e.g. "
"'%s' evaluating to '%s' in the future. "
"If you depend on this behavior, please reach out in "
"<https://git.lix.systems/lix-project/lix/issues/496> so we can discuss your use-case.",
name,
"--arg config.allowUnfree true",
"{ config.allowUnfree = true; }"
);
}
}
MixEvalArgs::MixEvalArgs() MixEvalArgs::MixEvalArgs()
{ {
addFlag( addFlag(
@@ -42,10 +24,7 @@ MixEvalArgs::MixEvalArgs()
.description = "Pass the value *expr* as the argument *name* to Nix functions.", .description = "Pass the value *expr* as the argument *name* to Nix functions.",
.category = category, .category = category,
.labels = {"name", "expr"}, .labels = {"name", "expr"},
.handler = {[&](std::string name, std::string expr) { .handler = {[&](std::string name, std::string expr) { autoArgs[name] = ExprArgument(expr); }}}
checkValidNixIdentifier(name);
autoArgs[name] = 'E' + expr;
}}}
); );
addFlag({ addFlag({
@@ -53,10 +32,7 @@ MixEvalArgs::MixEvalArgs()
.description = "Pass the string *string* as the argument *name* to Nix functions.", .description = "Pass the string *string* as the argument *name* to Nix functions.",
.category = category, .category = category,
.labels = {"name", "string"}, .labels = {"name", "string"},
.handler = {[&](std::string name, std::string s) { .handler = {[&](std::string name, std::string s) { autoArgs[name] = StringArgument(s); }},
checkValidNixIdentifier(name);
autoArgs[name] = 'S' + s;
}},
}); });
addFlag({ addFlag({
@@ -179,18 +155,90 @@ MixEvalArgs::MixEvalArgs()
}); });
} }
struct AutoArgsContainer
{
std::map<Symbol, std::variant<Value, AutoArgsContainer>> data;
Bindings * toBindings(Evaluator & state)
{
auto bb = state.buildBindings(data.size());
for (auto & [sym, v] : data) {
bb.insert(
sym,
std::visit(
overloaded{
[&](Value & v) { return v; },
[&](AutoArgsContainer & aac) -> Value {
return {NewValueAs::attrs, aac.toBindings(state)};
}
},
v
)
);
}
return bb.finish();
}
};
static void addAutoArgRecursive(
AutoArgsContainer & container,
Evaluator & state,
std::vector<std::string> && path,
Value & val,
const std::string_view pathStr
)
{
auto * data = &container.data;
auto size = path.size();
for (auto [i, pathCmp] : enumerate(path)) {
auto next = state.symbols.create(pathCmp);
auto entry = data->find(next);
if (entry == data->end()) {
if (i == size - 1) {
(*data)[next] = val;
} else {
(*data)[next] = AutoArgsContainer{};
data = &std::get<AutoArgsContainer>((*data)[next]).data;
}
} else {
std::visit(
overloaded{
[&](Value & v) {
throw Error(
"Cannot set %s via --arg/--argstr when it's the path-extension of another "
"auto-argument!",
pathStr
);
},
[&](AutoArgsContainer & v) { data = &v.data; }
},
entry->second
);
}
}
}
Bindings * MixEvalArgs::getAutoArgs(Evaluator & state) Bindings * MixEvalArgs::getAutoArgs(Evaluator & state)
{ {
auto res = state.buildBindings(autoArgs.size()); AutoArgsContainer aac;
for (auto & i : autoArgs) { for (auto & [name, value] : autoArgs) {
Value v; Value v = std::visit(
if (i.second[0] == 'E') overloaded{
v = state.evalLazily(state.parseExprFromString(i.second.substr(1), CanonPath::fromCwd())); [&](StringArgument & str) -> Value { return {NewValueAs::string, (std::string_view) str.value}; },
else [&](ExprArgument & e) -> Value {
v = {NewValueAs::string, ((std::string_view) i.second).substr(1)}; return state.evalLazily(state.parseExprFromString(e.expr, CanonPath::fromCwd()));
res.insert(state.symbols.create(i.first), v); }
},
value
);
addAutoArgRecursive(aac, state, parseAttrPath(name, false), v, name);
} }
return res.finish();
return aac.toBindings(state);
} }
kj::Promise<Result<EvalPaths::PathResult<SourcePath, ThrownError>>> kj::Promise<Result<EvalPaths::PathResult<SourcePath, ThrownError>>>
lix/libcmd/common-eval-args.hh
+10 -1
View File
@@ -14,6 +14,15 @@ class EvalState;
class Bindings; class Bindings;
struct SourcePath; struct SourcePath;
struct StringArgument
{
std::string value;
};
struct ExprArgument
{
std::string expr;
};
struct MixEvalArgs : virtual Args, virtual MixRepair struct MixEvalArgs : virtual Args, virtual MixRepair
{ {
static constexpr auto category = "Common evaluation options"; static constexpr auto category = "Common evaluation options";
@@ -27,7 +36,7 @@ struct MixEvalArgs : virtual Args, virtual MixRepair
std::optional<std::string> evalStoreUrl; std::optional<std::string> evalStoreUrl;
private: private:
std::map<std::string, std::string> autoArgs; std::map<std::string, std::variant<StringArgument, ExprArgument>> autoArgs;
}; };
/** @brief Resolve an argument that is generally a file, but could be something that /** @brief Resolve an argument that is generally a file, but could be something that
lix/libcmd/installables.cc
+11 -11
View File
@@ -446,18 +446,18 @@ Installables SourceExprCommand::parseInstallables(
throw UsageError("'--file' and '--expr' are exclusive"); throw UsageError("'--file' and '--expr' are exclusive");
auto evaluator = getEvaluator(); auto evaluator = getEvaluator();
Value vFile;
if (file == "-") { Value vFile = [&](NeverAsync = {}) {
auto & e = evaluator->parseStdin(); if (file == "-") {
vFile = state.eval(e); auto & e = evaluator->parseStdin();
} return state.eval(e);
else if (file) } else if (file) {
vFile = state.evalFile(state.aio.blockOn(lookupFileArg(*evaluator, *file)).unwrap()); return state.evalFile(state.aio.blockOn(lookupFileArg(*evaluator, *file)).unwrap());
else { } else {
auto & e = evaluator->parseExprFromString(*expr, CanonPath::fromCwd()); auto & e = evaluator->parseExprFromString(*expr, CanonPath::fromCwd());
vFile = state.eval(e); return state.eval(e);
} }
}();
for (auto & s : ss) { for (auto & s : ss) {
auto [prefix, extendedOutputsSpec] = ExtendedOutputsSpec::parse(s); auto [prefix, extendedOutputsSpec] = ExtendedOutputsSpec::parse(s);
lix/libcmd/markdown.cc
+4 -8
View File
@@ -55,7 +55,6 @@ std::string renderMarkdownToTerminal(std::string_view markdown, StandardOutputSt
struct lowdown_opts opts{ struct lowdown_opts opts{
.type = LOWDOWN_TERM, .type = LOWDOWN_TERM,
#ifdef LOWDOWN_SEPARATE_TERM_OPTS
.term = .term =
{ {
.cols = lowdown_cols, .cols = lowdown_cols,
@@ -65,16 +64,13 @@ std::string renderMarkdownToTerminal(std::string_view markdown, StandardOutputSt
.vmargin = 0, .vmargin = 0,
.centre = 0, .centre = 0,
}, },
// maxdepth needs to be part of the ifdefs to match declaration order
.maxdepth = 20, .maxdepth = 20,
#else
.maxdepth = 20,
.cols = lowdown_cols,
.hmargin = 0,
.vmargin = 0,
#endif /* LOWDOWN_SEPARATE_TERM_OPTS */
.feat = LOWDOWN_COMMONMARK | LOWDOWN_FENCED | LOWDOWN_DEFLIST | LOWDOWN_TABLES, .feat = LOWDOWN_COMMONMARK | LOWDOWN_FENCED | LOWDOWN_DEFLIST | LOWDOWN_TABLES,
#ifdef LOWDOWN_CONSOLIDATED_OFLAGS
.oflags = LOWDOWN_NOLINK,
#else
.oflags = LOWDOWN_TERM_NOLINK, .oflags = LOWDOWN_TERM_NOLINK,
#endif /* LOWDOWN_CONSOLIDATED_OFLAGS */
}; };
if (!shouldANSI(fileno)) { if (!shouldANSI(fileno)) {
opts.oflags |= LOWDOWN_TERM_NOANSI; opts.oflags |= LOWDOWN_TERM_NOANSI;
lix/libcmd/repl-interacter.cc
+60 -215
View File
@@ -1,251 +1,101 @@
#include "lix/libutil/c-calls.hh" #include "libutil/fmt.hh"
#include "libutil/terminal.hh"
#include "lix/libutil/error.hh" #include "lix/libutil/error.hh"
#include "lix/libutil/file-system.hh" #include "lix/libutil/file-system.hh"
#include "lix/libutil/logging.hh" #include "lix/libutil/logging.hh"
#include <csignal> #include "lix/lix-rs/main.gen.hh"
#include <cstdio> #include "lix/lix-rs/utils.hh"
#include <iostream> #include <iostream>
#include <memory>
#include <string> #include <string>
#include <string_view>
#include <cerrno>
// editline < 1.15.2 don't wrap their API for C++ usage
// (added in https://github.com/troglobit/editline/commit/91398ceb3427b730995357e9d120539fb9bb7461).
// This results in linker errors due to to name-mangling of editline C symbols.
// For compatibility with these versions, we wrap the API here
// (wrapping multiple times on newer versions is no problem).
extern "C" {
#include <editline.h>
}
#include "lix/libutil/finally.hh"
#include "lix/libutil/strings.hh"
#include "lix/libcmd/repl-interacter.hh" #include "lix/libcmd/repl-interacter.hh"
namespace rust {
Vec<String> Impl<lix::repl::CxxCompleter, lix::repl::ReplCompleter>::complete(
Ref<lix::repl::CxxCompleter> self, Ref<Str> input
)
try {
auto s = to_std_string(input);
auto result = std::vec::Vec<std::string::String>::new_();
for (auto & possible : self.cpp().completePrefix(s)) {
result.push(rust::to_string(possible));
}
return result;
} catch (...) {
// the completer should have logged anything interesting.
return std::vec::Vec<std::string::String>::new_();
}
}
namespace nix { namespace nix {
namespace { ReadlineLikeInteracter::ReadlineLikeInteracter(std::string historyFile) : historyFile(historyFile) {}
// Used to communicate to NixRepl::getLine whether a signal occurred in ::readline.
volatile sig_atomic_t g_signal_received = 0;
void sigintHandler(int signo) void ReadlineLikeInteracter::init(detail::ReplCompleterMixin * repl)
{ {
g_signal_received = signo;
}
static detail::ReplCompleterMixin * curRepl; // ugly
/**
* @return a null-terminated list of completions as expected by `el_print_columns`
*/
char ** copyCompletions(const StringSet& possible)
{
int ac = 0;
char ** vp = nullptr;
auto check = [&](auto * p) {
if (!p) {
if (vp) {
while (--ac >= 0)
free(vp[ac]);
free(static_cast<void *>(vp));
}
throw Error("allocation failure");
}
return p;
};
vp = check(static_cast<char **>(malloc(possible.size() * sizeof(char *))));
for (auto & p : possible)
vp[ac++] = check(strdup(p.c_str()));
return vp;
}
// Instead of using the readline-provided prefix, do our own tokenization
// to avoid the default behavior of treating dots/quotes as word boundaries.
// See the definition of SEPS for what it treats as a boundary:
// https://github.com/troglobit/editline/blob/caf4b3c0ce3b0785791198b11de6f3134e9f05d8/src/editline.c
std::string getLastTokenBeforeCursor()
{
std::string_view line{rl_line_buffer, static_cast<size_t>(rl_point)};
auto tokens = tokenizeString<std::vector<std::string>>(
line,
// Same as editline's SEPS, except for double and single quotes:
"#$&()*:;<=>?[\\]^`{|}~\n\t "
);
if (tokens.empty()) {
return "";
}
return tokens.back();
}
// Sometimes inserting text or listing possible completions has a side effect
// of hiding the text after the cursor (even though it remains in the buffer).
// This helper just refreshes the display while keeping the cursor in place.
//
// Inserting text also sometimes moves the whole buffer down one line, usually
// if the cursor is inside a quoted attr name. I'm not sure why (vs unquoted)
// but it still seems to work pretty well and is just a visual artifact.
el_status_t redisplay()
{
int cursorPos = rl_point;
rl_refresh_line(0, 0);
rl_point = cursorPos;
return (rl_point == rl_end) ? CSstay : CSmove;
}
};
static el_status_t doCompletion() {
auto s = getLastTokenBeforeCursor();
auto possible = curRepl->completePrefix(s);
if (possible.empty()) {
return el_ring_bell();
}
if (possible.size() == 1) {
const auto completion = *possible.cbegin();
if (completion.size() > s.size()) {
rl_insert_text(requireCString(completion.substr(s.size())));
return redisplay();
}
return el_ring_bell();
}
auto checkAllHaveSameAt = [&](size_t pos) {
auto & first = *possible.begin();
for (auto & p : possible) {
if (p.size() <= pos || p[pos] != first[pos]) {
return false;
}
}
return true;
};
size_t start = s.size();
size_t len = 0;
while (checkAllHaveSameAt(start + len)) {
++len;
}
if (len > 0) {
auto commonPrefix = possible.begin()->substr(start, len);
rl_insert_text(requireCString(commonPrefix));
el_ring_bell();
return redisplay();
}
char** columns = copyCompletions(possible);
el_print_columns(possible.size(), columns);
return redisplay();
}
ReadlineLikeInteracter::Guard ReadlineLikeInteracter::init(detail::ReplCompleterMixin * repl)
{
// Allow nix-repl specific settings in .inputrc
rl_readline_name = "nix-repl";
try { try {
createDirs(dirOf(historyFile)); createDirs(dirOf(historyFile));
} catch (SysError & e) { } catch (SysError & e) {
logWarning(e.info()); logWarning(e.info());
} }
el_hist_size = 1000;
read_history(requireCString(historyFile)); auto rl = repl::Rustyline::new_(rust::to_string(historyFile).as_str(), *repl);
auto oldRepl = curRepl; match_result(
curRepl = repl; std::move(rl),
Guard restoreRepl([oldRepl] { curRepl = oldRepl; }); [&](repl::Rustyline ok) { this->rl = std::make_unique<repl::Rustyline>(std::move(ok)); },
// editline does its own escaping of completions, so we rebind tab [&](rust::Box<rust::Dyn<rust::std::error::Error>> err) {
// to our own completion function to skip that and do nix escaping throw Error("%s", Uncolored(to_std_string(err.to_string())));
// instead of shell escaping. }
el_bind_key(CTL('I'), doCompletion); );
return restoreRepl;
} }
static constexpr const char * promptForType(ReplPromptType promptType) static rust::Ref<rust::Str> promptForType(ReplPromptType promptType)
{ {
switch (promptType) { switch (promptType) {
case ReplPromptType::ReplPrompt: case ReplPromptType::ReplPrompt:
return "nix-repl> "; return "nix-repl> "_rs;
case ReplPromptType::ContinuationPrompt: case ReplPromptType::ContinuationPrompt:
return " "; return " "_rs;
} }
assert(false); assert(false);
} }
bool ReadlineLikeInteracter::getLine(std::string & input, ReplPromptType promptType) bool ReadlineLikeInteracter::getLine(std::string & input, ReplPromptType promptType)
{ {
struct sigaction act, old; auto s = rl->ask(promptForType(promptType));
sigset_t savedSignalMask, set;
auto setupSignals = [&]() { // rustyline temporarily sets a SIGWINCH handler
act.sa_handler = sigintHandler; KJ_DEFER(invalidateWindowSize());
sigfillset(&act.sa_mask);
act.sa_flags = 0;
if (sigaction(SIGINT, &act, &old))
throw SysError("installing handler for SIGINT");
sigemptyset(&set); return match_result(
sigaddset(&set, SIGINT); std::move(s),
if (sigprocmask(SIG_UNBLOCK, &set, &savedSignalMask)) [&](rust::String ok) {
throw SysError("unblocking SIGINT"); input += to_std_string(ok);
}; input += '\n';
auto restoreSignals = [&]() { return true;
if (sigprocmask(SIG_SETMASK, &savedSignalMask, nullptr)) },
throw SysError("restoring signals"); [&](rust::rustyline::error::ReadlineError err) {
if (err.matches_Interrupted()) {
input.clear();
return true;
}
if (sigaction(SIGINT, &old, 0)) if (err.matches_Eof()) {
throw SysError("restoring handler for SIGINT"); return false;
}; }
setupSignals(); throw Error("%s", Uncolored(to_std_string(err.into().to_string())));
char * s = readline(promptForType(promptType)); // NOLINT(lix-unsafe-c-calls) }
Finally doFree([&]() { free(s); }); );
restoreSignals();
if (g_signal_received) {
g_signal_received = 0;
input.clear();
return true;
}
if (!s)
return false;
this->writeHistory();
input += s;
input += '\n';
return true;
} }
void ReadlineLikeInteracter::writeHistory() void ReadlineLikeInteracter::writeHistory()
{ {
int ret = write_history(requireCString(historyFile)); if (rl) {
int writeHistErr = errno; rl->write_history();
if (ret == 0) {
return;
} }
// If the open fails, editline returns EOF. If the close fails, editline
// forwards the return value of fclose(), which is EOF on error.
// readline however, returns the errno.
// So if we didn't get exactly EOF, then consider the return value the error
// code; otherwise use the errno we saved above.
// https://github.com/troglobit/editline/issues/66
if (ret != EOF) {
writeHistErr = ret;
}
// In any of these cases, we should explicitly ignore the error, but log
// them so the user isn't confused why their history is getting eaten.
std::string_view const errMsg(std::strerror(writeHistErr));
printTaggedWarning("ignoring error writing repl history to %s: %s", this->historyFile, errMsg);
} }
ReadlineLikeInteracter::~ReadlineLikeInteracter() ReadlineLikeInteracter::~ReadlineLikeInteracter()
@@ -253,11 +103,6 @@ ReadlineLikeInteracter::~ReadlineLikeInteracter()
this->writeHistory(); this->writeHistory();
} }
AutomationInteracter::Guard AutomationInteracter::init(detail::ReplCompleterMixin *)
{
return Guard([] {});
}
// ASCII ENQ character // ASCII ENQ character
constexpr const char * automationPrompt = "\x05"; constexpr const char * automationPrompt = "\x05";
lix/libcmd/repl-interacter.hh
+11 -11
View File
@@ -1,11 +1,14 @@
#pragma once #pragma once
/// @file /// @file
#include "lix/libutil/finally.hh"
#include "lix/libutil/types.hh" #include "lix/libutil/types.hh"
#include <functional> #include <memory>
#include <string> #include <string>
namespace rust::lix::repl {
struct Rustyline;
}
namespace nix { namespace nix {
namespace detail { namespace detail {
@@ -24,9 +27,7 @@ enum class ReplPromptType {
class ReplInteracter class ReplInteracter
{ {
public: public:
using Guard = Finally<std::function<void()>>; virtual void init(detail::ReplCompleterMixin * repl) {}
virtual Guard init(detail::ReplCompleterMixin * repl) = 0;
/** Returns a boolean of whether the interacter got EOF */ /** Returns a boolean of whether the interacter got EOF */
virtual bool getLine(std::string & input, ReplPromptType promptType) = 0; virtual bool getLine(std::string & input, ReplPromptType promptType) = 0;
virtual ~ReplInteracter(){}; virtual ~ReplInteracter(){};
@@ -35,12 +36,12 @@ public:
class ReadlineLikeInteracter final : public ReplInteracter class ReadlineLikeInteracter final : public ReplInteracter
{ {
std::string historyFile; std::string historyFile;
std::unique_ptr<rust::lix::repl::Rustyline> rl;
public: public:
ReadlineLikeInteracter(std::string historyFile) ReadlineLikeInteracter(std::string historyFile);
: historyFile(historyFile)
{ virtual void init(detail::ReplCompleterMixin * repl) override;
}
virtual Guard init(detail::ReplCompleterMixin * repl) override;
virtual bool getLine(std::string & input, ReplPromptType promptType) override; virtual bool getLine(std::string & input, ReplPromptType promptType) override;
/** Writes the current history to the history file. /** Writes the current history to the history file.
* *
@@ -54,7 +55,6 @@ class AutomationInteracter final : public ReplInteracter
{ {
public: public:
AutomationInteracter() = default; AutomationInteracter() = default;
virtual Guard init(detail::ReplCompleterMixin * repl) override;
virtual bool getLine(std::string & input, ReplPromptType promptType) override; virtual bool getLine(std::string & input, ReplPromptType promptType) override;
virtual ~AutomationInteracter() override = default; virtual ~AutomationInteracter() override = default;
}; };
lix/libcmd/repl.cc
+125 -42
View File
@@ -1,10 +1,12 @@
#include <algorithm>
#include <cstdio> #include <cstdio>
#include <editline.h>
#include <iostream> #include <iostream>
#include <cstdlib> #include <cstdlib>
#include <cstring> #include <cstring>
#include <optional>
#include <string_view> #include <string_view>
#include "libutil/logging.hh"
#include "lix/libexpr/value.hh" #include "lix/libexpr/value.hh"
#include "lix/libutil/box_ptr.hh" #include "lix/libutil/box_ptr.hh"
#include "lix/libcmd/repl-interacter.hh" #include "lix/libcmd/repl-interacter.hh"
@@ -377,7 +379,7 @@ ReplExitStatus NixRepl::mainLoop()
std::list<ReplLoadable> loadables = std::exchange(loaded, {}); std::list<ReplLoadable> loadables = std::exchange(loaded, {});
loadFiles(loadables); loadFiles(loadables);
auto _guard = interacter->init(static_cast<detail::ReplCompleterMixin *>(this)); interacter->init(static_cast<detail::ReplCompleterMixin *>(this));
/* Stop the progress bar because it interferes with the display of /* Stop the progress bar because it interferes with the display of
the repl. */ the repl. */
@@ -438,15 +440,10 @@ StringSet NixRepl::completePrefix(const std::string &prefix)
{ {
StringSet completions; StringSet completions;
// We should only complete colon commands if there's a colon at the beginning, if (prefix.starts_with(':')) {
// but editline (for... whatever reason) doesn't *give* us the colon in the
// completion callback. If the user types :rel<TAB>, `prefix` will only be `rel`.
// Luckily, editline provides a global variable for its current buffer, so we can
// check for the presence of a colon there.
if (rl_line_buffer != nullptr && rl_line_buffer[0] == ':') {
for (auto const & [colonCmd, cmd] : registeredCommands) { for (auto const & [colonCmd, cmd] : registeredCommands) {
if ((!cmd->attributes.debugModeOnly || inDebugger()) && colonCmd.starts_with(prefix)) { if ((!cmd->attributes.debugModeOnly || inDebugger()) && colonCmd.starts_with(prefix.substr(1))) {
completions.insert(std::string(colonCmd)); completions.insert(":" + colonCmd);
} }
} }
@@ -696,44 +693,135 @@ void NixRepl::initDebugBuiltinCommands()
addCommand( addCommand(
"backtrace", "backtrace",
[](NixRepl & repl, const std::string & _arg) { [](NixRepl & repl, const std::string & _arg) {
auto traces = repl.evaluator.debug->traces(); auto tracesGenerator = repl.evaluator.debug->traces();
for (const auto & [idx, i] : enumerate(traces)) {
std::cout << "\n" << ANSI_BLUE << idx << ANSI_NORMAL << ": "; // since we want to print the stack trace in reverse order,
showDebugTrace(std::cout, repl.evaluator.positions, *i); // we have to first traverse all frames and accumulate them
// in a list (in which we store each new trace at the /beginning/)
std::list<std::pair<size_t, const DebugTrace *>> reversedTraces;
for (const auto trace : tracesGenerator) {
// because the original traces are indexed from 0 upto N,
// this gives us their original index
auto idx = reversedTraces.size();
reversedTraces.push_front({idx, trace});
}
for (const auto & [traceIdx, trace] : reversedTraces) {
std::cout << "\n" << ANSI_BLUE << traceIdx << ANSI_NORMAL << ": ";
showDebugTrace(std::cout, repl.evaluator.positions, *trace);
} }
return ProcessLineResult::PromptAgain; return ProcessLineResult::PromptAgain;
}, },
{.aliases = {"bt"}, {.aliases = {"bt"}, .debugModeOnly = true, .help = "Show trace stack", .section = "Debug mode"}
.debugModeOnly = true,
.help = "Show trace stack",
.section = "Debug mode"}
); );
addCommand( addCommand(
"show-trace", "show-trace",
// this command has a bit of nuance to its function and error states.
// it can either:
// 1. be called without any argument
// -> just display the current stack frame (still have to walk up the stack :/)
// 2. be called with an absolute index
// -> try to go to that frame
// -> if it doesn't exist, print an "arg out of range" error
// 3. be called with a relative index
// -> if the final offset is in-bounds, go to that frame
// -> otherwise: clamp the index, i.e. go to 0/$max instead of out-of-bounds
//
// because the collection of frames is lazy and isn't a random-access list,
// we need to iterate the whole stack for most of these if we want to have
// good error messages; this is the biggest reason why this function is so
// long/complex compared to its role
//
[](NixRepl & repl, const std::string & arg) { [](NixRepl & repl, const std::string & arg) {
try { auto setTrace = [&](size_t traceIdx, const DebugTrace * trace) {
repl.debugTraceIndex = stoi(arg); repl.debugTraceIndex = traceIdx;
} catch (...) { std::cout << "\n" << ANSI_BLUE << traceIdx << ANSI_NORMAL << ": ";
showDebugTrace(std::cout, repl.evaluator.positions, *trace);
std::cout << std::endl;
printEnvBindings(repl.state, trace->expr, trace->env);
repl.loadDebugTraceEnv(*trace);
};
// tries to find a trace at a given index.
// - if it is found, it returns the requested trace, along with its
// index, which will be *the same* as requested
// - otherwise, it returns the last (=outermost) trace, along with
// its index, which will be *different* than the one requested
auto tryFindTrace = [&](size_t traceIdx) -> std::pair<size_t, const DebugTrace *> {
size_t lastIndex = 0;
const DebugTrace * lastTrace;
auto traces = repl.evaluator.debug->traces();
for (const auto & [idx, i] : enumerate(traces)) {
lastTrace = i;
lastIndex = idx;
if (idx == traceIdx) {
return std::pair(idx, i);
}
}
return std::pair(lastIndex, lastTrace);
};
bool isRelativeIdx = false;
int requestedTraceIdx;
if (arg.length() == 0) {
// if there's no argument, just re-print the current frame
requestedTraceIdx = repl.debugTraceIndex;
} else {
std::optional<int> maybeIdx = string2Int<int>(arg);
if (!maybeIdx) {
throw Error("argument '%s' is not a valid integer", arg);
}
isRelativeIdx = arg.starts_with('+') || arg.starts_with('-');
requestedTraceIdx =
isRelativeIdx ? maybeIdx.value() + repl.debugTraceIndex : maybeIdx.value();
} }
auto traces = repl.evaluator.debug->traces(); auto [actualTraceIdx, trace] = tryFindTrace((size_t) requestedTraceIdx);
for (const auto & [idx, i] : enumerate(traces)) {
if (idx == repl.debugTraceIndex) { // if we *did* find the frame we wanted originally, all is well
std::cout << "\n" << ANSI_BLUE << idx << ANSI_NORMAL << ": "; // in the world and we can just load it and exit
showDebugTrace(std::cout, repl.evaluator.positions, *i); if (actualTraceIdx == (size_t) requestedTraceIdx) {
std::cout << std::endl; setTrace(actualTraceIdx, trace);
printEnvBindings(repl.state, i->expr, i->env); return ProcessLineResult::PromptAgain;
repl.loadDebugTraceEnv(*i); }
break;
} // if we couldn't immediately find the requested trace on the "happy path", then either:
// a) it was an absolute index but didn't exist
// -> print a specific error showing the exact valid range
if (!isRelativeIdx) {
throw Error(
"stack index must be between %ld and %ld (inclusive), but was %ld",
0,
actualTraceIdx, // tryFindTrace sets *idx to the final (max) frame index if it fails
requestedTraceIdx
);
}
// b) it was a relative index
// -> clamp the index to the bounds and print a warning
if (requestedTraceIdx < 0) {
// just load frame 0 but print a warning about the bounds
std::tie(actualTraceIdx, trace) = tryFindTrace(0);
setTrace(actualTraceIdx, trace);
printTaggedWarning("stopped at stack frame %ld, cannot go any deeper", 0);
return ProcessLineResult::PromptAgain;
} else {
// (if we're here, then requestedTraceIdx > $max, since tryFindTrace failed)
// load the max frame (that `tryFindFrame` kindly already got for us),
// but print a warning that we can't go any further
setTrace(actualTraceIdx, trace);
printTaggedWarning("stopped at stack frame %ld, cannot go any higher", actualTraceIdx);
return ProcessLineResult::PromptAgain;
} }
return ProcessLineResult::PromptAgain;
}, },
{.aliases = {"st"}, {.aliases = {"st"},
.debugModeOnly = true, .debugModeOnly = true,
.help = "Show current trace. If an integer is provided, this switches to that stack " .help = "Show current trace. If an integer is provided, this switches to that stack "
"beforehand.", "beforehand. If the integer has an explicit + or - sign, it is treated as "
"relative to the current stack index.",
.section = "Debug mode", .section = "Debug mode",
.positionalArgsSpecifiers = {{.placeholderText = "integer index", .optional = true}}} .positionalArgsSpecifiers = {{.placeholderText = "integer index", .optional = true}}}
); );
@@ -867,8 +955,7 @@ void NixRepl::initBuiltinCommands()
[](NixRepl & repl, const std::string & arg) { [](NixRepl & repl, const std::string & arg) {
Value v = repl.evalString(arg); Value v = repl.evalString(arg);
Value f = repl.evalString( Value f = repl.evalString(
R""("drv: (import <nixpkgs> {}).runCommand "shell") "" R"(drv: (import <nixpkgs> {}).runCommand "shell" { buildInputs = [ drv ]; } "")"
R""({ buildInputs = [ drv ]; } "")""
); );
Value result = repl.state.callFunction(f, v, PosIdx()); Value result = repl.state.callFunction(f, v, PosIdx());
@@ -1318,12 +1405,10 @@ void NixRepl::loadFlake(const std::string & flakeRefS)
.kind = ReplLoadKind::Flake, .kind = ReplLoadKind::Flake,
}; };
Value v;
try { try {
loaded.remove(loadable); loaded.remove(loadable);
loaded.push_back(loadable); loaded.push_back(loadable);
v = flake::callFlake( Value v = flake::callFlake(
state, state,
flake::lockFlake( flake::lockFlake(
state, state,
@@ -1451,7 +1536,6 @@ Value NixRepl::getReplOverlaysEvalFunction()
} }
auto evalReplInitFilesPath = CanonPath::root + "repl-overlays.nix"; auto evalReplInitFilesPath = CanonPath::root + "repl-overlays.nix";
*replOverlaysEvalFunction = Value{};
auto code = auto code =
#include "repl-overlays.nix.gen.hh" #include "repl-overlays.nix.gen.hh"
; ;
@@ -1461,16 +1545,15 @@ Value NixRepl::getReplOverlaysEvalFunction()
evaluator.builtins.staticEnv evaluator.builtins.staticEnv
); );
**replOverlaysEvalFunction = state.eval(expr); *replOverlaysEvalFunction = state.eval(expr);
return **replOverlaysEvalFunction; return **replOverlaysEvalFunction;
} }
Value NixRepl::replOverlays() Value NixRepl::replOverlays()
{ {
Value replInits;
auto replInitStorage = evaluator.mem.newList(evalSettings.replOverlays.get().size()); auto replInitStorage = evaluator.mem.newList(evalSettings.replOverlays.get().size());
replInits = {NewValueAs::list, replInitStorage}; Value replInits = {NewValueAs::list, replInitStorage};
size_t i = 0; size_t i = 0;
for (auto path : evalSettings.replOverlays.get()) { for (auto path : evalSettings.replOverlays.get()) {
lix/libexec/kill-user.cc
+3
View File
@@ -36,9 +36,12 @@ int helperMain(const char * name, std::span<char *> args) noexcept
other things, determines if kill(-1, signo) affects the other things, determines if kill(-1, signo) affects the
calling process. In the OSX libc, it's set to true, calling process. In the OSX libc, it's set to true,
which means "follow POSIX", which we don't want here */ which means "follow POSIX", which we don't want here */
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wdeprecated-declarations"
if (syscall(SYS_kill, -1, SIGKILL, false) == 0) { if (syscall(SYS_kill, -1, SIGKILL, false) == 0) {
break; break;
} }
#pragma clang diagnostic pop
#else #else
if (kill(-1, SIGKILL) == 0) { if (kill(-1, SIGKILL) == 0) {
break; break;
lix/libexec/launch-builder-linux.cc
+12 -7
View File
@@ -3,6 +3,7 @@
#include "lix/libutil/rpc.hh" #include "lix/libutil/rpc.hh"
#include <cassert> #include <cassert>
#include <csignal> #include <csignal>
#include <fcntl.h>
#include <filesystem> #include <filesystem>
#include <format> #include <format>
#include <kj/io.h> #include <kj/io.h>
@@ -58,7 +59,7 @@ static void setPersonality(std::string_view system)
bool pathExists(const fs::path & path) bool pathExists(const fs::path & path)
{ {
return fs::exists(fs::symlink_status(path)); return fs::exists(fs::symlink_status(path)); // NOLINT(lix-foreign-exceptions): caught by main
} }
void bindPath(const fs::path & source, const fs::path & target, bool optional = false) void bindPath(const fs::path & source, const fs::path & target, bool optional = false)
@@ -71,7 +72,7 @@ void bindPath(const fs::path & source, const fs::path & target, bool optional =
} }
}; };
auto st = fs::symlink_status(source); auto st = fs::symlink_status(source); // NOLINT(lix-foreign-exceptions): caught by main
if (st.type() == fs::file_type::not_found) { if (st.type() == fs::file_type::not_found) {
if (optional) { if (optional) {
return; return;
@@ -81,14 +82,14 @@ void bindPath(const fs::path & source, const fs::path & target, bool optional =
} }
if (st.type() == fs::file_type::directory) { if (st.type() == fs::file_type::directory) {
fs::create_directories(target); fs::create_directories(target); // NOLINT(lix-foreign-exceptions): caught by main
bindMount(); bindMount();
} else if (st.type() == fs::file_type::symlink) { } else if (st.type() == fs::file_type::symlink) {
// Symlinks can (apparently) not be bind-mounted, so just copy it // Symlinks can (apparently) not be bind-mounted, so just copy it
fs::create_directories(target.parent_path()); fs::create_directories(target.parent_path()); // NOLINT(lix-foreign-exceptions): caught by main
fs::copy_symlink(source, target); fs::copy_symlink(source, target); // NOLINT(lix-foreign-exceptions): caught by main
} else { } else {
fs::create_directories(target.parent_path()); fs::create_directories(target.parent_path()); // NOLINT(lix-foreign-exceptions): caught by main
if (kj::AutoCloseFd file{open(target.c_str(), O_RDWR | O_CREAT, 0644)}; file == nullptr) { if (kj::AutoCloseFd file{open(target.c_str(), O_RDWR | O_CREAT, 0644)}; file == nullptr) {
throw SysError("could not create %s", target); throw SysError("could not create %s", target);
} }
@@ -224,7 +225,11 @@ bool prepareChildSetup(build::Request::Reader request)
}; };
const fs::path dst = chrootRootDir / target.relative_path(); const fs::path dst = chrootRootDir / target.relative_path();
fs::create_directories(dst.parent_path()); fs::create_directories(dst.parent_path());
writeFile(dst, std::string_view((const char *) sh, sizeof(sh))); kj::AutoCloseFd fd(open(dst.c_str(), O_RDWR | O_CREAT, 0755));
if (fd == nullptr) {
throw SysError("cannot create sandbox shell");
}
writeFull(fd.get(), std::string_view((const char *) sh, sizeof(sh)));
fs::permissions(dst, fs::perms(0555)); fs::permissions(dst, fs::perms(0555));
} else } else
#endif #endif
lix/libexec/unix-bind-connect.cc
+20 -4
View File
@@ -2,7 +2,17 @@
#include <sys/socket.h> #include <sys/socket.h>
#include <sys/un.h> #include <sys/un.h>
LIBEXEC_HELPER(4) LIBEXEC_HELPER(5)
static int resultFd = -1;
static void sendResult(int result)
{
// NOTE posix says pipe writes smaller than PIPE_BUF must be atomic, so this either
// succeeds or fails (pipe bufs of four bytes make no sense at all for our systems)
int error = result;
DIE_UNLESS_SYS("writing result", ::write(resultFd, &error, sizeof(error)));
}
int helperMain(const char *, std::span<char *> args) noexcept int helperMain(const char *, std::span<char *> args) noexcept
{ {
@@ -10,8 +20,12 @@ int helperMain(const char *, std::span<char *> args) noexcept
std::string_view method = args[1]; std::string_view method = args[1];
const auto dir = args[2]; const auto dir = args[2];
const auto name = args[3]; const auto name = args[3];
resultFd = argToInt<int>("result-fd", args[4]);
DIE_UNLESS_SYS("chdir", chdir(dir)); if (chdir(dir)) {
sendResult(errno);
return 0;
}
struct sockaddr_un addr; struct sockaddr_un addr;
addr.sun_family = AF_UNIX; addr.sun_family = AF_UNIX;
@@ -23,9 +37,11 @@ int helperMain(const char *, std::span<char *> args) noexcept
} }
if (method == "bind") { if (method == "bind") {
DIE_UNLESS_SYS("bind", bind(socket, reinterpret_cast<struct sockaddr *>(&addr), sizeof(addr))); auto result = bind(socket, reinterpret_cast<struct sockaddr *>(&addr), sizeof(addr)) ? errno : 0;
sendResult(result);
} else if (method == "connect") { } else if (method == "connect") {
DIE_UNLESS_SYS("connect", connect(socket, reinterpret_cast<struct sockaddr *>(&addr), sizeof(addr))); auto result = connect(socket, reinterpret_cast<struct sockaddr *>(&addr), sizeof(addr)) ? errno : 0;
sendResult(result);
} else { } else {
die(std::format("invalid method %s", method)); die(std::format("invalid method %s", method));
} }
lix/libexpr/attr-path.cc
+30 -13
View File
@@ -7,8 +7,7 @@
namespace nix { namespace nix {
std::vector<std::string> parseAttrPath(std::string_view const s, bool allowRhsTrailingDot)
std::vector<std::string> parseAttrPath(std::string_view const s)
{ {
std::vector<std::string> res; std::vector<std::string> res;
std::string cur; std::string cur;
@@ -16,6 +15,22 @@ std::vector<std::string> parseAttrPath(std::string_view const s)
auto i = s.begin(); auto i = s.begin();
while (i != s.end()) { while (i != s.end()) {
if (*i == '.') { if (*i == '.') {
if (!haveData) {
if (res.empty()) {
throw ParseError(
"Leading dot in attribute selection path '%1%' is not allowed! If the attribute name "
"is an empty string, use '\"\".foo.bar'",
s
);
} else {
throw ParseError(
"consecutive dots not allowed in selection path '%1%', use 'foo.\"\".bar' to denote "
"an "
"empty attribute name",
s
);
}
}
res.push_back(cur); res.push_back(cur);
haveData = false; haveData = false;
cur.clear(); cur.clear();
@@ -36,7 +51,11 @@ std::vector<std::string> parseAttrPath(std::string_view const s)
} }
++i; ++i;
} }
if (haveData) res.push_back(cur); if (haveData) {
res.push_back(cur);
} else if (!allowRhsTrailingDot) {
throw ParseError("Trailing dot on the right-hand side of path expr '%1%' is not allowed!", s);
};
return res; return res;
} }
@@ -90,9 +109,6 @@ findAlongAttrPath(EvalState & state, const std::string & attrPath, Bindings & au
according to what is specified in the attrPath. */ according to what is specified in the attrPath. */
if (!attrIndex) { if (!attrIndex) {
if (attr.empty())
throw Error("empty attribute name in selection path '%1%'", attrPath);
if (v.type() != nAttrs) { if (v.type() != nAttrs) {
auto pathPart = auto pathPart =
std::vector<std::string>(tokens.begin(), tokens.begin() + attrPathIdx); std::vector<std::string>(tokens.begin(), tokens.begin() + attrPathIdx);
@@ -163,13 +179,14 @@ findAlongAttrPath(EvalState & state, const std::string & attrPath, Bindings & au
std::pair<SourcePath, uint32_t> findPackageFilename(EvalState & state, Value & v, std::string what) std::pair<SourcePath, uint32_t> findPackageFilename(EvalState & state, Value & v, std::string what)
{ {
Value v2; Value v2 = [&]() {
try { try {
auto dummyArgs = state.ctx.mem.allocBindings(0); auto dummyArgs = state.ctx.mem.allocBindings(0);
v2 = findAlongAttrPath(state, "meta.position", *dummyArgs, v).first; return findAlongAttrPath(state, "meta.position", *dummyArgs, v).first;
} catch (Error &) { } catch (Error &) {
throw NoPositionInfo("package '%s' has no source location information", what); throw NoPositionInfo("package '%s' has no source location information", what);
} }
}();
// FIXME: is it possible to extract the Pos object instead of doing this // FIXME: is it possible to extract the Pos object instead of doing this
// toString + parsing? // toString + parsing?
lix/libexpr/attr-path.hh
+1 -1
View File
@@ -25,7 +25,7 @@ std::pair<SourcePath, uint32_t> findPackageFilename(EvalState & state, Value & v
* Such an attr path is a dot-separated sequence of attribute names, which are possibly quoted. * Such an attr path is a dot-separated sequence of attribute names, which are possibly quoted.
* No escaping is performed; attribute names containing double quotes are unrepresentable. * No escaping is performed; attribute names containing double quotes are unrepresentable.
*/ */
std::vector<std::string> parseAttrPath(std::string_view const s); std::vector<std::string> parseAttrPath(std::string_view const s, bool allowRhsTrailingDot = true);
/** /**
* Converts an attr path from a list of strings into a string once more. * Converts an attr path from a list of strings into a string once more.
lix/libexpr/attr-set.cc
+2 -11
View File
@@ -23,20 +23,11 @@ Bindings * EvalMemory::allocBindings(size_t capacity)
return new (allocBytes(sizeof(Bindings) + sizeof(Attr) * capacity)) Bindings(); return new (allocBytes(sizeof(Bindings) + sizeof(Attr) * capacity)) Bindings();
} }
void BindingsBuilder::insert(std::string_view name, Value value, PosIdx pos)
Value & BindingsBuilder::alloc(Symbol name, PosIdx pos)
{ {
bindings->push_back(Attr(name, {}, pos)); return insert(symbols.create(name), value, pos);
return (bindings->end() - 1)->value;
} }
Value & BindingsBuilder::alloc(std::string_view name, PosIdx pos)
{
return alloc(symbols.create(name), pos);
}
void Bindings::sort() void Bindings::sort()
{ {
if (size_) std::sort(begin(), end()); if (size_) std::sort(begin(), end());
lix/libexpr/attr-set.hh
+7 -7
View File
@@ -25,7 +25,8 @@ struct Attr
PosIdx pos; PosIdx pos;
mutable Value value; mutable Value value;
Attr(Symbol name, Value value, PosIdx pos = noPos) : name(name), pos(pos), value(value) {} Attr(Symbol name, Value value, PosIdx pos = noPos) : name(name), pos(pos), value(value) {}
Attr() { }; [[deprecated]]
Attr() {};
bool operator < (const Attr & a) const bool operator < (const Attr & a) const
{ {
return name < a.name; return name < a.name;
@@ -72,8 +73,9 @@ public:
const Attr * get(Symbol name) const Attr * get(Symbol name)
{ {
Attr key(name, {}); iterator i = std::lower_bound(begin(), end(), name, [](const Attr & value, const Symbol & compare) {
iterator i = std::lower_bound(begin(), end(), key); return value.name < compare;
});
if (i != end() && i->name == name) return &*i; if (i != end() && i->name == name) return &*i;
return nullptr; return nullptr;
} }
@@ -140,6 +142,8 @@ public:
insert(Attr(name, value, pos)); insert(Attr(name, value, pos));
} }
void insert(std::string_view name, Value value, PosIdx pos = noPos);
void insert(const Attr & attr) void insert(const Attr & attr)
{ {
push_back(attr); push_back(attr);
@@ -151,10 +155,6 @@ public:
bindings->push_back(attr); bindings->push_back(attr);
} }
Value & alloc(Symbol name, PosIdx pos = noPos);
Value & alloc(std::string_view name, PosIdx pos = noPos);
[[nodiscard("must use created bindings")]] [[nodiscard("must use created bindings")]]
Bindings * finish() Bindings * finish()
{ {
lix/libexpr/builtins/addErrorContext.md
+2 -58
View File
@@ -23,70 +23,14 @@ countDown 2
Then, evaluating the file will give the following stack trace: Then, evaluating the file will give the following stack trace:
```console ```console
$ nix-instantiate --show-trace err.nix $ nix-instantiate err.nix
error: error:
… from call site
at /home/plop/git.lix.systems/lix-project/lix/err.nix:9:1:
8| in
9| countDown 2
| ^
10|
… while calling 'countDown'
at /home/plop/git.lix.systems/lix-project/lix/err.nix:3:5:
2| countDown =
3| n:
| ^
4| if n == 0 then
… while calling the 'addErrorContext' builtin
at /home/plop/git.lix.systems/lix-project/lix/err.nix:7:7:
6| else
7| builtins.addErrorContext "while counting down; n = ${toString n}" ("x" + countDown (n - 1));
| ^
8| in
… while counting down; n = 2 … while counting down; n = 2
… from call site
at /home/plop/git.lix.systems/lix-project/lix/err.nix:7:80:
6| else
7| builtins.addErrorContext "while counting down; n = ${toString n}" ("x" + countDown (n - 1));
| ^
8| in
… while calling 'countDown'
at /home/plop/git.lix.systems/lix-project/lix/err.nix:3:5:
2| countDown =
3| n:
| ^
4| if n == 0 then
… while calling the 'addErrorContext' builtin
at /home/plop/git.lix.systems/lix-project/lix/err.nix:7:7:
6| else
7| builtins.addErrorContext "while counting down; n = ${toString n}" ("x" + countDown (n - 1));
| ^
8| in
… while counting down; n = 1 … while counting down; n = 1
… from call site
at /home/plop/git.lix.systems/lix-project/lix/err.nix:7:80:
6| else
7| builtins.addErrorContext "while counting down; n = ${toString n}" ("x" + countDown (n - 1));
| ^
8| in
… while calling 'countDown'
at /home/plop/git.lix.systems/lix-project/lix/err.nix:3:5:
2| countDown =
3| n:
| ^
4| if n == 0 then
… caused by explicit throw … caused by explicit throw
at /home/plop/git.lix.systems/lix-project/lix/err.nix:5:7: at err.nix:5:7:
4| if n == 0 then 4| if n == 0 then
5| throw "kaboom" 5| throw "kaboom"
| ^ | ^
lix/libexpr/builtins/ceil.md
+6 -5
View File
@@ -1,9 +1,10 @@
--- ---
name: ceil name: ceil
args: [double] args: [number]
--- ---
Converts an IEEE-754 double-precision floating-point number (*double*) to Returns an integer value containing the smallest integer greater than or equal to *number* (which is either a floating-point or integer value).
the next higher integer.
If the datatype is neither an integer nor a "float", an evaluation error will be If the result is out of range for the integer type, such as NaN, infinity, or a number with magnitude out of range, Lix throws an evaluation error.
thrown.
Lix currently throws an evaluation error for some *integer* inputs between 2\*\*52 and 2\*\*63 - 1 as those previously experienced floating-point precision loss due to a Nix bug (https://github.com/NixOS/nix/issues/12899).
In a future release, such integers will be passed through.
lix/libexpr/builtins/floor.md
+6 -5
View File
@@ -1,9 +1,10 @@
--- ---
name: floor name: floor
args: [double] args: [number]
--- ---
Converts an IEEE-754 double-precision floating-point number (*double*) to Returns an integer value containing the largest integer less than or equal to *number* (which is either a float or integer).
the next lower integer.
If the datatype is neither an integer nor a "float", an evaluation error will be If the result is out of range for the integer type, such as NaN, infinity, or a number with magnitude out of range, Lix throws an evaluation error.
thrown.
Lix currently throws an evaluation error for some *integer* inputs between 2\*\*52 and 2\*\*63 - 1 as those previously experienced floating-point precision loss due to a Nix bug (https://github.com/NixOS/nix/issues/12899).
In a future release, such integers will be passed through.
lix/libexpr/builtins/path.md
+2 -1
View File
@@ -10,7 +10,8 @@ present in *args*. All are optional except `path`:
- name\ - name\
The name of the path when added to the store. This can used to The name of the path when added to the store. This can used to
reference paths that have nix-illegal characters in their names, reference paths that have
[nix-illegal characters in their names](./derivations.md),
like `@`. like `@`.
- filter\ - filter\
lix/libexpr/builtins/scopedImport.md
+1 -1
View File
@@ -1,6 +1,6 @@
--- ---
name: scopedImport name: scopedImport
implementation: "[](EvalState & state, Value ** args, Value & v) { import(state, *args[1], args[0], v); }" implementation: "[](EvalState & state, Value ** args) -> Value { return import(state, *args[1], args[0]); }"
args: [scope, path] args: [scope, path]
renameInGlobalScope: false renameInGlobalScope: false
--- ---
lix/libexpr/eval-cache.hh
+1 -1
View File
@@ -53,7 +53,7 @@ public:
ref<AttrCursor> getRoot(); ref<AttrCursor> getRoot();
}; };
enum AttrType { enum AttrType : uint8_t {
Placeholder = 0, Placeholder = 0,
FullAttrs = 1, FullAttrs = 1,
String = 2, String = 2,
lix/libexpr/eval-expr.cc
+78 -89
View File
@@ -1,4 +1,5 @@
#include "eval.hh" #include "eval.hh"
#include "libexpr/nixexpr.hh"
#include "primops.hh" #include "primops.hh"
#include "gc-small-vector.hh" #include "gc-small-vector.hh"
@@ -10,14 +11,19 @@
namespace nix { namespace nix {
Value Expr::makeThunk(Evaluator & ctx, Env & env)
{
ctx.stats.nrThunks++;
return {NewValueAs::thunk, ctx.mem, env, *this};
}
/* Create a thunk for the delayed computation of the given expression /* Create a thunk for the delayed computation of the given expression
in the given environment. But if the expression is a variable, in the given environment. But if the expression is a variable,
then look it up right away. This significantly reduces the number then look it up right away. This significantly reduces the number
of thunks allocated. */ of thunks allocated. */
Value Expr::maybeThunk(EvalState & state, Env & env) Value Expr::maybeThunk(EvalState & state, Env & env)
{ {
state.ctx.stats.nrThunks++; return makeThunk(state.ctx, env);
return {NewValueAs::thunk, state.ctx.mem, env, *this};
} }
Value ExprVar::maybeThunk(EvalState & state, Env & env) Value ExprVar::maybeThunk(EvalState & state, Env & env)
@@ -99,18 +105,10 @@ Value ExprSet::eval(EvalState & state, Env & env)
in the original environment. */ in the original environment. */
Displacement displ = 0; Displacement displ = 0;
for (auto & i : attrs) { for (auto & i : attrs) {
Value vAttr; Env & thunkEnv = *i.second.chooseByKind(&env2, &env, inheritEnv);
if (hasOverrides && i.second.kind != ExprAttrs::AttrDef::Kind::Inherited) { Value vAttr = hasOverrides && i.second.kind != ExprAttrs::AttrDef::Kind::Inherited
vAttr = { ? i.second.e->makeThunk(state.ctx, thunkEnv)
NewValueAs::thunk, : i.second.e->maybeThunk(state, thunkEnv);
state.ctx.mem,
*i.second.chooseByKind(&env2, &env, inheritEnv),
*i.second.e
};
state.ctx.stats.nrThunks++;
} else {
vAttr = i.second.e->maybeThunk(state, *i.second.chooseByKind(&env2, &env, inheritEnv));
}
env2.values[displ++] = vAttr; env2.values[displ++] = vAttr;
v.attrs()->push_back(Attr(i.first, vAttr, i.second.pos)); v.attrs()->push_back(Attr(i.first, vAttr, i.second.pos));
} }
@@ -162,18 +160,18 @@ Value ExprSet::eval(EvalState & state, Env & env)
* http://github.com/NixOS/nix/issues/7012. Any accesses to the output attrset will thus infrec. * http://github.com/NixOS/nix/issues/7012. Any accesses to the output attrset will thus infrec.
*/ */
Value vBackup = v; Value vBackup = v;
Value nameVal; Symbol nameSym;
{ {
KJ_DEFER(v = vBackup); KJ_DEFER(v = vBackup);
v = Value{NewValueAs::blackhole}; v = Value{NewValueAs::blackhole};
nameVal = i.nameExpr->eval(state, *dynamicEnv); Value nameVal = i.nameExpr->eval(state, *dynamicEnv);
state.forceValue(nameVal, i.pos); state.forceValue(nameVal, i.pos);
if (nameVal.type() == nNull) { if (nameVal.type() == nNull) {
continue; continue;
} }
state.forceStringNoCtx(nameVal, i.pos, "while evaluating the name of a dynamic attribute"); state.forceStringNoCtx(nameVal, i.pos, "while evaluating the name of a dynamic attribute");
nameSym = state.ctx.symbols.create(nameVal.str());
} }
auto nameSym = state.ctx.symbols.create(nameVal.str());
auto j = v.attrs()->get(nameSym); auto j = v.attrs()->get(nameSym);
if (j) { if (j) {
state.ctx.errors state.ctx.errors
@@ -187,7 +185,29 @@ Value ExprSet::eval(EvalState & state, Env & env)
.debugThrow(); .debugThrow();
} }
i.valueExpr->setName(nameSym); // clang-format off
/* This line is so wrong that it is best kept in here with the documentation why it is wrong,
* lest some naive soul may add it once again some year in the future.
* See the following witness as to why it is wrong:
*
* nix-repl> fun = (name: { ${name} = x: x; }) # This function creates a dynamic attribute with a variable name
* Added fun.
* nix-repl> revSeq = x: y: builtins.seq x (builtins.seq y x) # evaluate x, then y in sequence, then return x
* Added revSeq.
* nix-repl> fun "foo" # The code seemingly works
* { foo = «lambda foo @ «string»:1:26»; }
* nix-repl> fun "bar" #
* { bar = «lambda bar @ «string»:1:26»; }
* nix-repl> revSeq (fun "foo") (fun "bar") # Until it doesn't
* { foo = «lambda bar @ «string»:1:26»; }
*
* What happened? Expressions are AST bound, therefore all lambdas share the same Expr and thus *the same name*.
* Using `setName` here updates the name of *all* lambdas from that expression, not just of the value at hand.
* And this is why all expressions must be treated as immutable after parsing.
*/
/* i.valueExpr->setName(nameSym); */
// clang-format on
/* Keep sorted order so find can catch duplicates */ /* Keep sorted order so find can catch duplicates */
v.attrs()->push_back(Attr(nameSym, i.valueExpr->maybeThunk(state, *dynamicEnv), i.pos)); v.attrs()->push_back(Attr(nameSym, i.valueExpr->maybeThunk(state, *dynamicEnv), i.pos));
v.attrs()->sort(); // FIXME: inefficient v.attrs()->sort(); // FIXME: inefficient
@@ -329,57 +349,21 @@ Value ExprOpUpdate::eval(EvalState & state, Env & env)
Value v2 = e2->eval(state, env); Value v2 = e2->eval(state, env);
state.checkAttrs(v2, env, *e2); state.checkAttrs(v2, env, *e2);
state.ctx.stats.nrOpUpdates++; return state.updateAttrs(v1, v2);
if (v1.attrs()->size() == 0) {
return v2;
}
if (v2.attrs()->size() == 0) {
return v1;
}
auto attrs = state.ctx.buildBindings(v1.attrs()->size() + v2.attrs()->size());
/* Merge the sets, preferring values from the second set. Make
sure to keep the resulting vector in sorted order. */
Bindings::iterator i = v1.attrs()->begin();
Bindings::iterator j = v2.attrs()->begin();
while (i != v1.attrs()->end() && j != v2.attrs()->end()) {
if (i->name == j->name) {
attrs.insert(*j);
++i;
++j;
} else if (i->name < j->name) {
attrs.insert(*i++);
} else {
attrs.insert(*j++);
}
}
while (i != v1.attrs()->end()) {
attrs.insert(*i++);
}
while (j != v2.attrs()->end()) {
attrs.insert(*j++);
}
Value v = {NewValueAs::attrs, attrs.alreadySorted()};
state.ctx.stats.nrOpUpdateValuesCopied += v.attrs()->size();
return v;
} }
Value ExprOpConcatLists::eval(EvalState & state, Env & env) Value ExprOpConcatLists::eval(EvalState & state, Env & env)
{ {
state.ctx.stats.nrListConcats++; state.ctx.stats.nrListConcats++;
/* We don't call into `concatLists` as that loses the position information of the expressions. */ /* We could simply call into `state.concatLists`, but that would add a redundant trace to our errors,
* and to fix that we would need to make the error on it and `forceList` optional, and *sigh*
*/
Value v1 = e1->eval(state, env); Value v1 = e1->eval(state, env);
state.checkList(v1, env, *e1); state.checkList(v1, env, *this); // Pass in `this` instead of `e1` to make the error point to the `++`
Value v2 = e2->eval(state, env); Value v2 = e2->eval(state, env);
state.checkList(v2, env, *e2); state.checkList(v2, env, *this); // Pass in `this` instead of `e2` to make the error point to the `++`
size_t l1 = v1.listSize(), l2 = v2.listSize(), len = l1 + l2; size_t l1 = v1.listSize(), l2 = v2.listSize(), len = l1 + l2;
@@ -429,12 +413,12 @@ Value ExprConcatStrings::eval(EvalState & state, Env & env)
}; };
// List of returned strings. References to these Values must NOT be persisted. // List of returned strings. References to these Values must NOT be persisted.
SmallTemporaryValueVector<conservativeStackReservation> values(es.size()); SmallTemporaryValueVector<conservativeStackReservation> values;
Value * vTmpP = values.data(); values.reserve(es.size());
for (auto & [i_pos, i] : es) { for (auto & [i_pos, i] : es) {
Value & vTmp = *vTmpP++; values.push_back(i->eval(state, env));
vTmp = i->eval(state, env); Value & vTmp = values.back();
/* If the first element is a path, then the result will also /* If the first element is a path, then the result will also
be a path, we don't copy anything (yet - that's done later, be a path, we don't copy anything (yet - that's done later,
@@ -452,7 +436,7 @@ Value ExprConcatStrings::eval(EvalState & state, Env & env)
} else { } else {
state.ctx.errors state.ctx.errors
.make<EvalError>("integer overflow in adding %1% + %2%", n, vTmp.integer()) .make<EvalError>("integer overflow in adding %1% + %2%", n, vTmp.integer())
.atPos(i_pos) .atPos(isInterpolation ? i_pos : pos)
.debugThrow(); .debugThrow();
} }
} else if (vTmp.type() == nFloat) { } else if (vTmp.type() == nFloat) {
@@ -462,7 +446,7 @@ Value ExprConcatStrings::eval(EvalState & state, Env & env)
nf += vTmp.fpoint(); nf += vTmp.fpoint();
} else { } else {
state.ctx.errors.make<EvalError>("cannot add %1% to an integer", showType(vTmp)) state.ctx.errors.make<EvalError>("cannot add %1% to an integer", showType(vTmp))
.atPos(i_pos) .atPos(isInterpolation ? i_pos : pos)
.withFrame(env, *this) .withFrame(env, *this)
.debugThrow(); .debugThrow();
} }
@@ -473,7 +457,7 @@ Value ExprConcatStrings::eval(EvalState & state, Env & env)
nf += vTmp.fpoint(); nf += vTmp.fpoint();
} else { } else {
state.ctx.errors.make<EvalError>("cannot add %1% to a float", showType(vTmp)) state.ctx.errors.make<EvalError>("cannot add %1% to a float", showType(vTmp))
.atPos(i_pos) .atPos(isInterpolation ? i_pos : pos)
.withFrame(env, *this) .withFrame(env, *this)
.debugThrow(); .debugThrow();
} }
@@ -489,13 +473,18 @@ Value ExprConcatStrings::eval(EvalState & state, Env & env)
: StringCoercionMode::Strict; : StringCoercionMode::Strict;
/* skip canonization of first path, which would only be not /* skip canonization of first path, which would only be not
canonized in the first place if it's coming from a ./${foo} type canonized in the first place if it's coming from a ./${foo} type
path */ path */
auto part = state.coerceToString( auto part = state.coerceToString(
i_pos, isInterpolation ? i_pos : pos,
vTmp, vTmp,
context, context,
"while evaluating a path segment", (isInterpolation && firstType == nPath)
? "while evaluating a path interpolation"
: (isInterpolation ? "while evaluating a string interpolation"
// TODO: to the person who eventually cleans up all of this mess,
// please turn this into "cannot concatenate $type to a $type" instead.
: "while concatenating"),
coercionMode, coercionMode,
firstType == nString, firstType == nString,
!first !first
@@ -527,9 +516,7 @@ Value ExprConcatStrings::eval(EvalState & state, Env & env)
Value ExprPos::eval(EvalState & state, Env & env) Value ExprPos::eval(EvalState & state, Env & env)
{ {
Value v; return state.mkPos(pos);
state.mkPos(v, pos);
return v;
} }
Value ExprBlackHole::eval(EvalState & state, Env & env) Value ExprBlackHole::eval(EvalState & state, Env & env)
@@ -619,19 +606,20 @@ Value ExprSelect::eval(EvalState & state, Env & env)
// Position for the current selector in this select chain. // Position for the current selector in this select chain.
PosIdx posCurrentSyntax; PosIdx posCurrentSyntax;
Value baseSelectee; Value baseSelectee = [&]() {
try { try {
// Evaluate the original thing we're selecting on. // Evaluate the original thing we're selecting on.
baseSelectee = e->eval(state, env); return e->eval(state, env);
} catch (Error & e) { } catch (Error & e) {
// clang-format off // clang-format off
e.addTrace(state.ctx.positions[getPos()], HintFmt( e.addTrace(state.ctx.positions[getPos()], HintFmt(
"while evaluating an expression to select '%s' on it", "while evaluating an expression to select '%s' on it",
showAttrPath(state.ctx.symbols, attrPath) showAttrPath(state.ctx.symbols, attrPath)
)); ));
// clang-format on // clang-format on
throw; throw;
} }
}();
try { try {
// With the original selectee evaluated, we'll walk the selection path starting // With the original selectee evaluated, we'll walk the selection path starting
@@ -717,9 +705,10 @@ Value ExprCall::eval(EvalState & state, Env & env)
// 5: under 10 // 5: under 10
// This excluded attrset lambdas (`{...}:`). Contributions of mixed lambdas appears insignificant at ~150 // This excluded attrset lambdas (`{...}:`). Contributions of mixed lambdas appears insignificant at ~150
// total. // total.
SmallValueVector<4> vArgs(args.size()); SmallValueVector<4> vArgs;
vArgs.reserve(args.size());
for (size_t i = 0; i < args.size(); ++i) { for (size_t i = 0; i < args.size(); ++i) {
vArgs[i] = args[i]->maybeThunk(state, env); vArgs.push_back(args[i]->maybeThunk(state, env));
} }
return state.callFunction(vFun, vArgs, pos); return state.callFunction(vFun, vArgs, pos);
lix/libexpr/eval-inline.hh
-1
View File
@@ -6,7 +6,6 @@
#include "lix/libexpr/eval-error.hh" #include "lix/libexpr/eval-error.hh"
#include "lix/libexpr/gc-alloc.hh" #include "lix/libexpr/gc-alloc.hh"
#include "value.hh" #include "value.hh"
#include <cstdint>
namespace nix { namespace nix {
lix/libexpr/eval.cc
+107 -79
View File
@@ -297,30 +297,29 @@ EvalPaths::EvalPaths(
} }
} }
#if LIX_MAJOR >= 3 || (LIX_MAJOR == 2 && LIX_MINOR >= 96) #if LIX_MAJOR >= 3 || (LIX_MAJOR == 2 && LIX_MINOR >= 98)
#warning \ #warning \
"The feature nix-path-shadow was deprecated in 2.95 with a warning, which needs to be turned into an error in 2.96" "The feature nix-path-shadow was deprecated in 2.95 with a warning, error in 2.96, and we should consider removing the bypass in 2.98"
#endif #endif
if (!featureSettings.isEnabled(DeprecatedFeature::NixPathShadow)) { if (!featureSettings.isEnabled(DeprecatedFeature::NixPathShadow)) {
for (auto & [prefix, path] : searchPath_.elements) { for (auto & [prefix, path] : searchPath_.elements) {
// Match on the 'nix' prefix // Match on the 'nix' prefix
if (prefix.s == "nix") { if (prefix.s == "nix") {
logWarning( throw EvalError(HintFmt(
{.msg = HintFmt( "The prefix '%s' is reserved for internal use by Lix in the Nix search "
"The prefix '%s' is reserved for internal use by Lix in the Nix search " "path, its usage is deprecated and will be forbidden in the future.\n"
"path, its usage is deprecated and will be forbidden in the future.\n" "Use %s to silence this error.\n"
"Use %s to silence this warning.\n" "This is due to adding '%s=%s' in the Nix search path, either through the "
"This is due to adding '%s=%s' in the Nix search path, either through the " "environment variable '%s' or by passing the flag %s to the nix "
"environment variable '%s' or by passing the flag %s to the nix " "invocation.",
"invocation.", "nix",
"nix", "--extra-deprecated-features nix-path-shadow",
"--extra-deprecated-features nix-path-shadow", prefix.s,
prefix.s, path.s,
path.s, "NIX_PATH",
"NIX_PATH", "-I"
"-I" ));
)}
);
} else } else
// Match prefixless paths that contain a `nix` directory // Match prefixless paths that contain a `nix` directory
if (auto res = if (auto res =
@@ -333,21 +332,19 @@ EvalPaths::EvalPaths(
}); });
})) }))
{ {
logWarning( throw EvalError(HintFmt(
{.msg = HintFmt( "Shadowing '%s' by configuring the nix-path is deprecated and "
"Shadowing '%s' by configuring the nix-path is deprecated and " "will be forbidden in the future.\n"
"will be forbidden in the future.\n" "Use %s to silence this error.\n"
"Use %s to silence this warning.\n" "This is due to adding '%s' to the nix-path without a prefix, "
"This is due to adding '%s' to the nix-path without a prefix, " "either by passing the flag '-I %s' to the nix invocation or by "
"either by passing the flag '-I %s' to the nix invocation or by " "adding this path to the environment variable '%s'.",
"adding this path to the environment variable '%s'.", "<nix/...>",
"<nix/...>", "--extra-deprecated-features nix-path-shadow",
"--extra-deprecated-features nix-path-shadow", path.s,
path.s, path.s,
path.s, "NIX_PATH"
"NIX_PATH" ));
)}
);
} }
} }
} }
@@ -437,11 +434,11 @@ void EvalPaths::allowPath(const StorePath & storePath)
allowPath(store->toRealPath(storePath)); allowPath(store->toRealPath(storePath));
} }
void EvalPaths::allowAndSetStorePathString(const StorePath & storePath, Value & v) Value EvalPaths::allowAndSetStorePathString(const StorePath & storePath)
{ {
allowPath(storePath); allowPath(storePath);
mkStorePathString(storePath, v); return mkStorePathString(storePath);
} }
CheckedSourcePath EvalPaths::checkSourcePath(const SourcePath & path_) CheckedSourcePath EvalPaths::checkSourcePath(const SourcePath & path_)
@@ -736,13 +733,13 @@ void mapStaticEnvBindings(const SymbolTable & st, const StaticEnv & se, const En
// add 'with' bindings. // add 'with' bindings.
Bindings::iterator j = env.values[0].attrs()->begin(); Bindings::iterator j = env.values[0].attrs()->begin();
while (j != env.values[0].attrs()->end()) { while (j != env.values[0].attrs()->end()) {
vm[std::string(st[j->name])] = j->value; vm.insert_or_assign(std::string(st[j->name]), j->value);
++j; ++j;
} }
} else { } else {
// iterate through staticenv bindings and add them. // iterate through staticenv bindings and add them.
for (auto & i : se.vars) for (auto & i : se.vars)
vm[std::string(st[i.first])] = env.values[i.second]; vm.insert_or_assign(std::string(st[i.first]), env.values[i.second]);
} }
} }
} }
@@ -850,26 +847,26 @@ Value::List * EvalMemory::newList(size_t size)
Value Evaluator::evalLazily(Expr & e) Value Evaluator::evalLazily(Expr & e)
{ {
stats.nrThunks++; return e.makeThunk(*this, builtins.env);
return {NewValueAs::thunk, mem, builtins.env, e};
} }
void EvalState::mkPos(Value & v, PosIdx p) Value EvalState::mkPos(PosIdx p)
{ {
auto origin = ctx.positions.originOf(p); auto origin = ctx.positions.originOf(p);
if (auto path = std::get_if<CheckedSourcePath>(&origin)) { if (auto path = std::get_if<CheckedSourcePath>(&origin)) {
auto attrs = ctx.buildBindings(3); auto attrs = ctx.buildBindings(3);
attrs.alloc(ctx.symbols.sym_file) = {NewValueAs::string, path->to_string()}; attrs.insert(ctx.symbols.sym_file, {NewValueAs::string, path->to_string()});
makePositionThunks(*this, p, attrs.alloc(ctx.symbols.sym_line), attrs.alloc(ctx.symbols.sym_column)); auto [line, col] = makePositionThunks(*this, p);
v = {NewValueAs::attrs, attrs}; attrs.insert(ctx.symbols.sym_line, line);
attrs.insert(ctx.symbols.sym_column, col);
return {NewValueAs::attrs, attrs};
} else } else
v.mkNull(); return Value::VNULL;
} }
Value EvalPaths::mkStorePathString(const StorePath & p)
void EvalPaths::mkStorePathString(const StorePath & p, Value & v)
{ {
v = { return {
NewValueAs::string, NewValueAs::string,
store->printStorePath(p), store->printStorePath(p),
NixStringContext{ NixStringContext{
@@ -885,13 +882,9 @@ std::string EvalState::mkOutputStringRaw(
return ctx.store->printStorePath(staticOutputPath); return ctx.store->printStorePath(staticOutputPath);
} }
Value EvalState::mkOutputString(const SingleDerivedPath::Built & b, const StorePath & staticOutputPath)
void EvalState::mkOutputString(
Value & value,
const SingleDerivedPath::Built & b,
const StorePath & staticOutputPath)
{ {
value = {NewValueAs::string, mkOutputStringRaw(staticOutputPath), NixStringContext{b}}; return {NewValueAs::string, mkOutputStringRaw(staticOutputPath), NixStringContext{b}};
} }
@@ -914,19 +907,6 @@ std::string EvalState::mkSingleDerivedPathStringRaw(
} }
void EvalState::mkSingleDerivedPathString(
const SingleDerivedPath & p,
Value & v)
{
v = {
NewValueAs::string,
mkSingleDerivedPathStringRaw(p),
NixStringContext{
std::visit([](auto && v) -> NixStringContextElem { return v; }, p),
}
};
}
struct CachedEvalFile struct CachedEvalFile
{ {
Value result; Value result;
@@ -1138,8 +1118,9 @@ Env & AttrsPattern::match(
Value EvalState::callFunction(Value & fun, std::span<Value> args, const PosIdx pos) Value EvalState::callFunction(Value & fun, std::span<Value> args, const PosIdx pos)
{ {
if (callDepth > evalSettings.maxCallDepth) if (callDepth >= evalSettings.maxCallDepth) {
ctx.errors.make<EvalError>("stack overflow; max-call-depth exceeded").atPos(pos).debugThrow(); ctx.errors.make<EvalError>("stack overflow; max-call-depth exceeded").atPos(pos).debugThrow();
}
MaintainCount _level(callDepth); MaintainCount _level(callDepth);
auto trace = evalSettings.traceFunctionCalls auto trace = evalSettings.traceFunctionCalls
@@ -1208,18 +1189,24 @@ Value EvalState::callFunction(Value & fun, std::span<Value> args, const PosIdx p
for (unsigned i = 0; i < argsLeft; i++) { for (unsigned i = 0; i < argsLeft; i++) {
pargs[i] = &args[i]; pargs[i] = &args[i];
} }
fn->fun(*this, pargs.data(), vCur); vCur = fn->fun(*this, pargs.data());
} catch (ThrownError & e) { } catch (ThrownError & e) {
// Distinguish between an error that simply happened while "throw" // Distinguish between an error that simply happened while "throw"
// was being evaluated and an explicit thrown error. // was being evaluated and an explicit thrown error.
if (fn->name == "throw") { if (fn->name == "throw" && !e.hasTrace()) {
e.addTrace(ctx.positions[pos], "caused by explicit %s", "throw"); e.addTrace(ctx.positions[pos], "caused by explicit %s", "throw");
} else { }
// otherwise, print a trace of this builtin, as long as it isn't
// a 'addErrorContext' call (which would just create noise)
else if (fn->name != "addErrorContext")
{
e.addTrace(ctx.positions[pos], "while calling the '%s' builtin", fn->name); e.addTrace(ctx.positions[pos], "while calling the '%s' builtin", fn->name);
} }
throw; throw;
} catch (Error & e) { } catch (Error & e) {
e.addTrace(ctx.positions[pos], "while calling the '%1%' builtin", fn->name); if (fn->name != "addErrorContext") {
e.addTrace(ctx.positions[pos], "while calling the '%1%' builtin", fn->name);
}
throw; throw;
} }
@@ -1263,9 +1250,11 @@ Value EvalState::callFunction(Value & fun, std::span<Value> args, const PosIdx p
// 1. Unify this and above code. Heavily redundant. // 1. Unify this and above code. Heavily redundant.
// 2. Create a fake env (arg1, arg2, etc.) and a fake expr (arg1: arg2: etc: builtins.name arg1 arg2 etc) // 2. Create a fake env (arg1, arg2, etc.) and a fake expr (arg1: arg2: etc: builtins.name arg1 arg2 etc)
// so the debugger allows to inspect the wrong parameters passed to the builtin. // so the debugger allows to inspect the wrong parameters passed to the builtin.
fn->fun(*this, vArgs.data(), vCur); vCur = fn->fun(*this, vArgs.data());
} catch (Error & e) { } catch (Error & e) {
e.addTrace(ctx.positions[pos], "while calling the '%1%' builtin", fn->name); if (fn->name != "addErrorContext") {
e.addTrace(ctx.positions[pos], "while calling the '%1%' builtin", fn->name);
}
throw; throw;
} }
} }
@@ -1364,9 +1353,49 @@ https://docs.lix.systems/manual/lix/stable/language/constructs.html#functions)",
return callFunction(fun, vAttrs, pos); return callFunction(fun, vAttrs, pos);
} }
void EvalState::concatLists( Value EvalState::updateAttrs(const Value & v1, const Value & v2)
Value & v, std::span<Value> lists, const PosIdx pos, std::string_view errorCtx {
) ctx.stats.nrOpUpdates++;
if (v1.attrs()->size() == 0) {
return v2;
}
if (v2.attrs()->size() == 0) {
return v1;
}
auto attrs = ctx.buildBindings(v1.attrs()->size() + v2.attrs()->size());
/* Merge the sets, preferring values from the second set. Make
sure to keep the resulting vector in sorted order. */
Bindings::iterator i = v1.attrs()->begin();
Bindings::iterator j = v2.attrs()->begin();
while (i != v1.attrs()->end() && j != v2.attrs()->end()) {
if (i->name == j->name) {
attrs.insert(*j);
++i;
++j;
} else if (i->name < j->name) {
attrs.insert(*i++);
} else {
attrs.insert(*j++);
}
}
while (i != v1.attrs()->end()) {
attrs.insert(*i++);
}
while (j != v2.attrs()->end()) {
attrs.insert(*j++);
}
Value v = {NewValueAs::attrs, attrs.alreadySorted()};
ctx.stats.nrOpUpdateValuesCopied += v.attrs()->size();
return v;
}
Value EvalState::concatLists(std::span<Value> lists, const PosIdx pos, std::string_view errorCtx)
{ {
ctx.stats.nrListConcats++; ctx.stats.nrListConcats++;
@@ -1382,12 +1411,10 @@ void EvalState::concatLists(
} }
if (nonEmpty && len == nonEmpty->listSize()) { if (nonEmpty && len == nonEmpty->listSize()) {
v = *nonEmpty; return *nonEmpty;
return;
} }
auto list = ctx.mem.newList(len); auto list = ctx.mem.newList(len);
v = {NewValueAs::list, list};
auto out = list->elems; auto out = list->elems;
for (size_t n = 0, pos = 0; n < lists.size(); ++n) { for (size_t n = 0, pos = 0; n < lists.size(); ++n) {
auto l = lists[n].listSize(); auto l = lists[n].listSize();
@@ -1396,6 +1423,7 @@ void EvalState::concatLists(
} }
pos += l; pos += l;
} }
return {NewValueAs::list, list};
} }
// always force this to be separate, otherwise forceValue may inline it and take // always force this to be separate, otherwise forceValue may inline it and take
lix/libexpr/eval.hh
+6 -20
View File
@@ -370,7 +370,7 @@ public:
/** /**
* Allow access to a store path and return it as a string. * Allow access to a store path and return it as a string.
*/ */
void allowAndSetStorePathString(const StorePath & storePath, Value & v); Value allowAndSetStorePathString(const StorePath & storePath);
/** /**
* Check whether access to a path is allowed and throw an error if * Check whether access to a path is allowed and throw an error if
@@ -449,7 +449,7 @@ public:
* The string is the printed store path with a context containing a * The string is the printed store path with a context containing a
* single `NixStringContextElem::Opaque` element of that store path. * single `NixStringContextElem::Opaque` element of that store path.
*/ */
void mkStorePathString(const StorePath & storePath, Value & v); Value mkStorePathString(const StorePath & storePath);
}; };
struct EvalStatistics struct EvalStatistics
@@ -800,7 +800,7 @@ public:
*/ */
Value autoCallFunction(Bindings & args, Value & fun, PosIdx pos); Value autoCallFunction(Bindings & args, Value & fun, PosIdx pos);
void mkPos(Value & v, PosIdx pos); Value mkPos(PosIdx pos);
/** /**
* Create a string representing a `SingleDerivedPath::Built`. * Create a string representing a `SingleDerivedPath::Built`.
@@ -809,30 +809,16 @@ public:
* single `NixStringContextElem::Built` element of the drv path and * single `NixStringContextElem::Built` element of the drv path and
* output name. * output name.
* *
* @param value Value we are settings
*
* @param b the drv whose output we are making a string for, and the * @param b the drv whose output we are making a string for, and the
* output * output
* *
* @param staticOutputPath Output path for that string. * @param staticOutputPath Output path for that string.
* Will be printed to form string. * Will be printed to form string.
*/ */
void mkOutputString( Value mkOutputString(const SingleDerivedPath::Built & b, const StorePath & staticOutputPath);
Value & value,
const SingleDerivedPath::Built & b,
const StorePath & staticOutputPath);
/** Value updateAttrs(const Value & v1, const Value & v2);
* Create a string representing a `SingleDerivedPath`. Value concatLists(std::span<Value> lists, const PosIdx pos, std::string_view errorCtx);
*
* A combination of `mkStorePathString` and `mkOutputString`.
*/
void mkSingleDerivedPathString(
const SingleDerivedPath & p,
Value & v);
void
concatLists(Value & v, std::span<Value> lists, const PosIdx pos, std::string_view errorCtx);
private: private:
lix/libexpr/extra-primops.hh
+15 -16
View File
@@ -8,25 +8,24 @@ namespace nix {
class EvalState; class EvalState;
struct Value; struct Value;
void prim_addDrvOutputDependencies(EvalState & state, Value * * args, Value & v); Value prim_addDrvOutputDependencies(EvalState & state, Value ** args);
void prim_fetchTree(EvalState & state, Value * * args, Value & v); Value prim_fetchTree(EvalState & state, Value ** args);
void prim_fetchGit(EvalState & state, Value * * args, Value & v); Value prim_fetchGit(EvalState & state, Value ** args);
void prim_fetchMercurial(EvalState & state, Value ** args, Value & v); Value prim_fetchMercurial(EvalState & state, Value ** args);
void prim_fetchTarball(EvalState & state, Value * * args, Value & v); Value prim_fetchTarball(EvalState & state, Value ** args);
void prim_fetchurl(EvalState & state, Value * * args, Value & v); Value prim_fetchurl(EvalState & state, Value ** args);
void prim_fromTOML(EvalState & state, Value * * args, Value & v); Value prim_fromTOML(EvalState & state, Value ** args);
void prim_appendContext(EvalState & state, Value ** args, Value & v); Value prim_appendContext(EvalState & state, Value ** args);
void prim_getContext(EvalState & state, Value * * args, Value & v); Value prim_getContext(EvalState & state, Value ** args);
void prim_hasContext(EvalState & state, Value * * args, Value & v); Value prim_hasContext(EvalState & state, Value ** args);
void prim_unsafeDiscardOutputDependency(EvalState & state, Value * * args, Value & v); Value prim_unsafeDiscardOutputDependency(EvalState & state, Value ** args);
void prim_unsafeDiscardStringContext(EvalState & state, Value ** args, Value & v); Value prim_unsafeDiscardStringContext(EvalState & state, Value ** args);
namespace flake { namespace flake {
void prim_flakeRefToString(EvalState & state, Value * * args, Value & v); Value prim_flakeRefToString(EvalState & state, Value ** args);
void prim_getFlake(EvalState & state, Value * * args, Value & v); Value prim_getFlake(EvalState & state, Value ** args);
void prim_parseFlakeRef(EvalState & state, Value * * args, Value & v); Value prim_parseFlakeRef(EvalState & state, Value ** args);
} }
} }
lix/libexpr/flake/config.cc
+126 -68
View File
@@ -4,6 +4,7 @@
#include "lix/libutil/json.hh" #include "lix/libutil/json.hh"
#include "lix/libutil/users.hh" #include "lix/libutil/users.hh"
#include "lix/libfetchers/fetch-settings.hh" #include "lix/libfetchers/fetch-settings.hh"
#include <cctype>
namespace nix::flake { namespace nix::flake {
@@ -30,80 +31,53 @@ static void writeTrustedList(const TrustedList & trustedList)
writeFile(path, JSON(trustedList).dump()); writeFile(path, JSON(trustedList).dump());
} }
static bool batchAskForSetting( static bool
bool & negativeTrustOverride, askForEachSetting(TrustedList & trustedList, std::map<std::string, std::string> & untrustedSettings)
TrustedList & trustedList,
std::map<std::string, std::string> & untrustedSettings)
{ {
printWarning("The following settings require your decision:"); // clang-format off
for (const auto & [name, valueS] : untrustedSettings) { constexpr auto prompt =
// FIXME: filter ANSI escapes, newlines, \r, etc. "[" ANSI_BOLD "y" ANSI_NORMAL "]es for now/"
logger->cout("- %s = %s", name, valueS); "[" ANSI_BOLD "N" ANSI_NORMAL "]o for now/"
} "[" ANSI_BOLD "a" ANSI_NORMAL "]lways allow";
// clang-format on
auto reply = logger
->ask(
fmt("Do you want to allow configuration settings to be applied?\nThis may allow the "
"flake to gain root, see the nix.conf manual page (" ANSI_BOLD "y" ANSI_NORMAL
"es for now/" ANSI_BOLD "A" ANSI_NORMAL "llow always/" ANSI_BOLD "n" ANSI_NORMAL
"o/" ANSI_BOLD "N" ANSI_NORMAL "o to all) ")
)
.value_or('n');
if (reply == 'N') {
printWarning("Rejecting all untrusted nix.conf entries");
printTaggedWarning(
"you can set '%s' to '%b' to automatically reject configuration options supplied by "
"flakes",
"accept-flake-config",
false
);
negativeTrustOverride = true;
return false;
}
if (reply == 'y' || reply == 'A') {
auto alwaysAllow = reply == 'A';
for (const auto & [name, valueS] : untrustedSettings) {
if (alwaysAllow) {
trustedList[name][valueS] = true;
}
globalConfig.set(name, valueS);
}
if (alwaysAllow) {
writeTrustedList(trustedList);
}
return true;
} else {
printTaggedWarning(
"you can set '%s' to '%b' to automatically reject configuration options supplied "
"by flakes",
"accept-flake-config",
false
);
}
auto didTrustedListChange = false; auto didTrustedListChange = false;
int acceptedCount = 0;
for (const auto & [name, valueS] : untrustedSettings) { for (const auto & [name, valueS] : untrustedSettings) {
auto individualReply = logger auto reply =
->ask( logger
fmt("Do you want to allow setting '%s = %s'? (" ANSI_BOLD ->ask(
"y" ANSI_NORMAL "es for now/" ANSI_BOLD "A" ANSI_NORMAL fmt("Do you want to allow setting '" ANSI_MAGENTA "%s = %s" ANSI_NORMAL "'? (%s) ",
"llow always/" ANSI_BOLD "n" ANSI_NORMAL "o for now) ", name,
name, valueS,
valueS) prompt)
) )
.value_or('n'); .value_or("n");
if (individualReply == 'y' || individualReply == 'A') { reply = toLower(reply);
if (individualReply == 'A') {
trustedList[name][valueS] = true; static const std::string yes = "yes for now";
didTrustedListChange = true; static const std::string no = "no for now";
static const std::string always = "always allow";
while (true) {
if (no.starts_with(reply)) {
break;
} }
globalConfig.set(name, valueS); if (yes.starts_with(reply) || always.starts_with(reply)) {
if (reply[0] == 'a') {
trustedList[name][valueS] = true;
didTrustedListChange = true;
}
globalConfig.set(name, valueS);
acceptedCount++;
break;
}
// if the reply wasn't a prefix of any answer, ask the user again
reply = logger->ask(fmt("Couldn't understand reply.\n%s: ", prompt)).value_or("n");
} }
} }
@@ -111,7 +85,91 @@ static bool batchAskForSetting(
writeTrustedList(trustedList); writeTrustedList(trustedList);
} }
return false; // return false if *none* of the settings were accepted
return acceptedCount > 0;
}
static bool batchAskForSetting(
bool & negativeTrustOverride,
TrustedList & trustedList,
std::map<std::string, std::string> & untrustedSettings)
{
std::string warning("The following settings require your decision:");
for (const auto & [name, valueS] : untrustedSettings) {
// FIXME: filter ANSI escapes, newlines, \r, etc.
warning += fmt("\n- %s = %s", name, valueS);
}
printWarning("%s", warning);
// clang-format off
constexpr auto globalPrompt =
"[" ANSI_BOLD "y" ANSI_NORMAL "]es for now/"
"[" ANSI_BOLD "n" ANSI_NORMAL "]o for now/"
"[" ANSI_BOLD "a" ANSI_NORMAL "]lways allow/"
"[" ANSI_BOLD "I" ANSI_NORMAL "]ndividually review";
// clang-format on
auto reply =
logger
->ask(
fmt("Do you want to allow these configuration settings to be applied?\n" ANSI_BOLD
"This may allow the flake to gain root" ANSI_NORMAL ", see the nix.conf manual page.\n"
"(%s) ",
globalPrompt)
)
.value_or("I"); // if the answer is empty (or there is no interactive prompt),
// just default to reviewing each individually
reply = toLower(reply);
static const std::string yes = "yes for now";
static const std::string no = "no for now";
static const std::string always = "always allow";
static const std::string review = "individually review";
// when interactive, loops and reprompts until the reply is one of y/n/a/i (or any prefix of the answers)
while (true) {
if (no.starts_with(reply)) {
printWarning("Rejecting all untrusted nix.conf entries");
printTaggedWarning(
"you can set '%s' to '%b' to automatically reject configuration options supplied by "
"flakes",
"accept-flake-config",
false
);
negativeTrustOverride = true;
return false;
}
if (yes.starts_with(reply) || always.starts_with(reply)) {
auto alwaysAllow = reply[0] == 'a';
for (const auto & [name, valueS] : untrustedSettings) {
if (alwaysAllow) {
trustedList[name][valueS] = true;
}
globalConfig.set(name, valueS);
}
if (alwaysAllow) {
printTaggedWarning(
"adding these configuration settings to the trusted list at %s, "
"edit it if you want to remove them in the future",
trustedListPath()
);
writeTrustedList(trustedList);
}
return true;
}
if (review.starts_with(reply)) {
return askForEachSetting(trustedList, untrustedSettings);
}
// if the reply wasn't a prefix of any, ask the user again
reply = logger->ask(fmt("Couldn't understand reply.\n%s: ", globalPrompt)).value_or("n");
}
} }
void ConfigFile::apply() void ConfigFile::apply()
lix/libexpr/flake/flake.cc
+39 -31
View File
@@ -78,10 +78,21 @@ try {
co_return result::current_exception(); co_return result::current_exception();
} }
/** Force a value that cannot contain any function calls */
static void forceTrivialValue(EvalState & state, Value & value, const PosIdx pos) static void forceTrivialValue(EvalState & state, Value & value, const PosIdx pos)
{ {
if (value.isThunk() && value.isTrivial()) /* /piegames sighs at the settings API */
state.forceValue(value, pos); auto prevOverridden = evalSettings.maxCallDepth.overridden;
auto prevMaxCallDepth = evalSettings.maxCallDepth;
evalSettings.maxCallDepth.override(0);
/* /piegames sighs at C++ */
KJ_DEFER({
evalSettings.maxCallDepth.override(prevMaxCallDepth);
evalSettings.maxCallDepth.overridden = prevOverridden;
});
state.forceValue(value, pos);
} }
@@ -327,11 +338,17 @@ static Flake getFlake(
Expr & flakeExpr = state.ctx.parseExprFromFile(resolvedFlakeFile); Expr & flakeExpr = state.ctx.parseExprFromFile(resolvedFlakeFile);
// Enforce that 'flake.nix' is a direct attrset, not a computation. // Enforce that 'flake.nix' is a direct attrset, not a computation.
if (!flakeExpr.try_cast<ExprAttrs>()) { // We do this by disallowing any function calls (maxCallDepth 0) and checking that the resulting value is
state.ctx.errors.make<EvalError>("file '%s' must be an attribute set", resolvedFlakeFile).debugThrow(); // an attrset.
} // The logic is already implemented in `forceTrivialValue`, but that takes a value instead of an Expr, so
// we wrap the expression in a thunk to be able to call it.
Value vInfo = flakeExpr.maybeThunk(state, state.ctx.builtins.env);
forceTrivialValue(state, vInfo, flakeExpr.getPos());
Value vInfo = state.eval(flakeExpr); if (vInfo.type() != nAttrs) {
state.ctx.errors.make<EvalError>("file '%s' must be an attribute set", resolvedFlakeFile)
.debugThrow();
}
if (auto description = vInfo.attrs()->get(state.ctx.symbols.sym_description)) { if (auto description = vInfo.attrs()->get(state.ctx.symbols.sym_description)) {
expectType(state, nString, description->value, description->pos); expectType(state, nString, description->value, description->pos);
@@ -942,10 +959,7 @@ LockedFlake lockFlake(
Value callFlake(EvalState & state, const LockedFlake & lockedFlake) Value callFlake(EvalState & state, const LockedFlake & lockedFlake)
{ {
Value vLocks; Value vLocks = {NewValueAs::string, lockedFlake.lockFile.to_string()};
Value vRootSubdir;
vLocks = {NewValueAs::string, lockedFlake.lockFile.to_string()};
Value vRootSrc = emitTreeAttrs( Value vRootSrc = emitTreeAttrs(
state.ctx, state.ctx,
@@ -955,14 +969,13 @@ Value callFlake(EvalState & state, const LockedFlake & lockedFlake)
lockedFlake.flake.forceDirty lockedFlake.flake.forceDirty
); );
vRootSubdir = {NewValueAs::string, lockedFlake.flake.lockedRef.subdir}; Value vRootSubdir = {NewValueAs::string, lockedFlake.flake.lockedRef.subdir};
if (!state.ctx.caches.vCallFlake) { if (!state.ctx.caches.vCallFlake) {
state.ctx.caches.vCallFlake = allocRootValue({}); state.ctx.caches.vCallFlake = allocRootValue(state.eval(state.ctx.parseExprFromString(
*state.ctx.caches.vCallFlake = state.eval(state.ctx.parseExprFromString(
#include "call-flake.nix.gen.hh" #include "call-flake.nix.gen.hh"
, CanonPath::root , CanonPath::root
)); )));
} }
Value vTmp1 = state.callFunction(*state.ctx.caches.vCallFlake, vLocks, noPos); Value vTmp1 = state.callFunction(*state.ctx.caches.vCallFlake, vLocks, noPos);
@@ -970,14 +983,14 @@ Value callFlake(EvalState & state, const LockedFlake & lockedFlake)
return state.callFunction(vTmp2, vRootSubdir, noPos); return state.callFunction(vTmp2, vRootSubdir, noPos);
} }
void prim_getFlake(EvalState & state, Value * * args, Value & v) Value prim_getFlake(EvalState & state, Value ** args)
{ {
std::string flakeRefS(state.forceStringNoCtx(*args[0], noPos, "while evaluating the argument passed to builtins.getFlake")); std::string flakeRefS(state.forceStringNoCtx(*args[0], noPos, "while evaluating the argument passed to builtins.getFlake"));
auto flakeRef = parseFlakeRef(flakeRefS, {}, true); auto flakeRef = parseFlakeRef(flakeRefS, {}, true);
if (evalSettings.pureEval && !flakeRef.input.isLocked()) if (evalSettings.pureEval && !flakeRef.input.isLocked())
throw Error("cannot call 'getFlake' on unlocked flake reference '%s' (use --impure to override)", flakeRefS); throw Error("cannot call 'getFlake' on unlocked flake reference '%s' (use --impure to override)", flakeRefS);
v = callFlake( return callFlake(
state, state,
lockFlake( lockFlake(
state, state,
@@ -992,10 +1005,7 @@ void prim_getFlake(EvalState & state, Value * * args, Value & v)
); );
} }
void prim_parseFlakeRef( Value prim_parseFlakeRef(EvalState & state, Value ** args)
EvalState & state,
Value * * args,
Value & v)
{ {
std::string flakeRefS(state.forceStringNoCtx(*args[0], noPos, std::string flakeRefS(state.forceStringNoCtx(*args[0], noPos,
"while evaluating the argument passed to builtins.parseFlakeRef")); "while evaluating the argument passed to builtins.parseFlakeRef"));
@@ -1003,28 +1013,26 @@ void prim_parseFlakeRef(
auto binds = state.ctx.buildBindings(attrs.size()); auto binds = state.ctx.buildBindings(attrs.size());
for (const auto & [key, value] : attrs) { for (const auto & [key, value] : attrs) {
auto s = state.ctx.symbols.create(key); auto s = state.ctx.symbols.create(key);
auto & vv = binds.alloc(s); Value vv = std::visit(
std::visit(
overloaded{ overloaded{
[&vv](const std::string & value) { vv = {NewValueAs::string, value}; }, [](const std::string & value) -> Value { return {NewValueAs::string, value}; },
[&vv](const uint64_t & value) { vv = {NewValueAs::integer, NixInt::Inner(value)}; }, [](const uint64_t & value) -> Value { return {NewValueAs::integer, NixInt::Inner(value)}; },
[&vv](const Explicit<bool> & value) { vv = {NewValueAs::boolean, value.t}; } [](const Explicit<bool> & value) -> Value { return {NewValueAs::boolean, value.t}; }
}, },
value value
); );
binds.insert(s, vv);
} }
v = {NewValueAs::attrs, binds}; return {NewValueAs::attrs, binds};
} }
void prim_flakeRefToString( Value prim_flakeRefToString(EvalState & state, Value ** args)
EvalState & state,
Value * * args,
Value & v)
{ {
state.forceAttrs(*args[0], noPos, state.forceAttrs(*args[0], noPos,
"while evaluating the argument passed to builtins.flakeRefToString"); "while evaluating the argument passed to builtins.flakeRefToString");
fetchers::Attrs attrs; fetchers::Attrs attrs;
for (const auto & attr : *args[0]->attrs()) { for (const auto & attr : *args[0]->attrs()) {
state.forceValue(attr.value, noPos);
auto t = attr.value.type(); auto t = attr.value.type();
if (t == nInt) { if (t == nInt) {
auto intValue = attr.value.integer().value; auto intValue = attr.value.integer().value;
@@ -1051,7 +1059,7 @@ void prim_flakeRefToString(
} }
} }
auto flakeRef = FlakeRef::fromAttrs(attrs); auto flakeRef = FlakeRef::fromAttrs(attrs);
v = {NewValueAs::string, flakeRef.to_string()}; return {NewValueAs::string, flakeRef.to_string()};
} }
} }
lix/libexpr/flake/flakeref.cc
-20
View File
@@ -67,16 +67,6 @@ FlakeRef parseFlakeRef(
return flakeRef; return flakeRef;
} }
std::optional<FlakeRef> maybeParseFlakeRef(
const std::string & url, const std::optional<Path> & baseDir)
{
try {
return parseFlakeRef(url, baseDir);
} catch (Error &) {
return {};
}
}
std::pair<FlakeRef, std::string> parseFlakeRefWithFragment( std::pair<FlakeRef, std::string> parseFlakeRefWithFragment(
const std::string & url, const std::string & url,
const std::optional<Path> & baseDir, const std::optional<Path> & baseDir,
@@ -234,16 +224,6 @@ std::pair<FlakeRef, std::string> parseFlakeRefWithFragment(
} }
} }
std::optional<std::pair<FlakeRef, std::string>> maybeParseFlakeRefWithFragment(
const std::string & url, const std::optional<Path> & baseDir)
{
try {
return parseFlakeRefWithFragment(url, baseDir);
} catch (Error & e) {
return {};
}
}
FlakeRef FlakeRef::fromAttrs(const fetchers::Attrs & attrs) FlakeRef FlakeRef::fromAttrs(const fetchers::Attrs & attrs)
{ {
auto attrs2(attrs); auto attrs2(attrs);
lix/libexpr/flake/flakeref.hh
-6
View File
@@ -70,18 +70,12 @@ FlakeRef parseFlakeRef(
bool allowMissing = false, bool allowMissing = false,
bool isFlake = true); bool isFlake = true);
std::optional<FlakeRef> maybeParseFlake(
const std::string & url, const std::optional<Path> & baseDir = {});
std::pair<FlakeRef, std::string> parseFlakeRefWithFragment( std::pair<FlakeRef, std::string> parseFlakeRefWithFragment(
const std::string & url, const std::string & url,
const std::optional<Path> & baseDir = {}, const std::optional<Path> & baseDir = {},
bool allowMissing = false, bool allowMissing = false,
bool isFlake = true); bool isFlake = true);
std::optional<std::pair<FlakeRef, std::string>> maybeParseFlakeRefWithFragment(
const std::string & url, const std::optional<Path> & baseDir = {});
std::tuple<FlakeRef, std::string, ExtendedOutputsSpec> parseFlakeRefWithFragmentAndExtendedOutputsSpec( std::tuple<FlakeRef, std::string, ExtendedOutputsSpec> parseFlakeRefWithFragmentAndExtendedOutputsSpec(
const std::string & url, const std::string & url,
const std::optional<Path> & baseDir = {}, const std::optional<Path> & baseDir = {},
lix/libexpr/json-to-value.cc
+83 -89
View File
@@ -1,115 +1,75 @@
#include "lix/libexpr/json-to-value.hh" #include "lix/libexpr/json-to-value.hh"
#include "gc-alloc.hh" #include "gc-alloc.hh"
#include "libutil/types.hh"
#include "lix/libexpr/value.hh" #include "lix/libexpr/value.hh"
#include "lix/libexpr/eval.hh" #include "lix/libexpr/eval.hh"
#include "lix/libutil/json.hh" #include "lix/libutil/json.hh"
#include <limits> #include <limits>
#include <variant>
namespace nix { namespace nix {
// for more information, refer to // for more information, refer to
// https://github.com/nlohmann/json/blob/master/include/nlohmann/detail/input/json_sax.hpp // https://github.com/nlohmann/json/blob/master/include/nlohmann/detail/input/json_sax.hpp
class JSONSax : nlohmann::json_sax<JSON> { class JSONSax : nlohmann::json_sax<JSON> {
class JSONState { struct WIPObject
protected: {
std::unique_ptr<JSONState> parent; GcMap<Symbol, Value> entries;
RootValue v; Symbol nextKey;
public:
virtual std::unique_ptr<JSONState> resolve(EvalState &)
{
assert(false && "tried to close toplevel json parser state");
}
explicit JSONState(std::unique_ptr<JSONState> && p) : parent(std::move(p)) {}
JSONState() = default;
JSONState(JSONState & p) = delete;
Value & value()
{
if (!v) {
v = allocRootValue({});
}
return *v;
}
virtual ~JSONState() {}
virtual void add() {}
}; };
using WIPArray = GcVector<Value>;
class JSONObjectState : public JSONState { // A stack of all of the currently opened and not yet closed JSON objects and arrays at the current moment
using JSONState::JSONState; // of parsing
GcMap<Symbol, Value> attrs; std::vector<std::variant<WIPObject, WIPArray>> stack;
Symbol _key;
std::unique_ptr<JSONState> resolve(EvalState & state) override
{
auto attrs2 = state.ctx.buildBindings(attrs.size());
for (auto & i : attrs)
attrs2.insert(i.first, i.second);
parent->value() = {NewValueAs::attrs, attrs2.alreadySorted()};
return std::move(parent);
}
void add() override
{
attrs.insert_or_assign(_key, value());
v = nullptr;
}
public:
void key(string_t & name, EvalState & state)
{
_key = state.ctx.symbols.create(name);
}
};
class JSONListState : public JSONState { std::optional<Value> final_value;
GcVector<Value> values;
std::unique_ptr<JSONState> resolve(EvalState & state) override void addValue(Value v)
{ {
auto list = state.ctx.mem.newList(values.size()); if (stack.size() == 0) {
parent->value() = {NewValueAs::list, list}; assert(!final_value);
for (size_t n = 0; n < values.size(); ++n) { final_value = v;
list->elems[n] = values[n]; return;
}
return std::move(parent);
} }
void add() override std::visit(
{ overloaded{
values.push_back(*v); [&](WIPObject & currentObject) {
v = nullptr; currentObject.entries.insert_or_assign(currentObject.nextKey, v);
} currentObject.nextKey = {}; // clear the current symbol
public: },
JSONListState(std::unique_ptr<JSONState> && p, std::size_t reserve) : JSONState(std::move(p)) [&](WIPArray & arr) { arr.push_back(v); }
{ },
values.reserve(reserve); stack.back()
} );
}; }
EvalState & state; EvalState & state;
std::unique_ptr<JSONState> rs;
public: public:
JSONSax(EvalState & state) : state(state), rs(new JSONState()) {}; JSONSax(EvalState & state) : state(state) {};
Value result() Value result()
{ {
return rs->value(); return final_value.value();
} }
bool null() override bool null() override
{ {
rs->value().mkNull(); addValue(Value::VNULL);
rs->add();
return true; return true;
} }
bool boolean(bool val) override bool boolean(bool val) override
{ {
rs->value() = {NewValueAs::boolean, val}; addValue({NewValueAs::boolean, val});
rs->add();
return true; return true;
} }
bool number_integer(number_integer_t val) override bool number_integer(number_integer_t val) override
{ {
rs->value() = {NewValueAs::integer, val}; addValue({NewValueAs::integer, val});
rs->add();
return true; return true;
} }
@@ -121,22 +81,19 @@ public:
return number_float(static_cast<number_float_t>(val_), ""); return number_float(static_cast<number_float_t>(val_), "");
} }
NixInt::Inner val = val_; NixInt::Inner val = val_;
rs->value() = {NewValueAs::integer, val}; addValue({NewValueAs::integer, val});
rs->add();
return true; return true;
} }
bool number_float(number_float_t val, const string_t & s) override bool number_float(number_float_t val, const string_t & s) override
{ {
rs->value() = {NewValueAs::floating, val}; addValue({NewValueAs::floating, val});
rs->add();
return true; return true;
} }
bool string(string_t & val) override bool string(string_t & val) override
{ {
rs->value() = {NewValueAs::string, val}; addValue({NewValueAs::string, val});
rs->add();
return true; return true;
} }
@@ -151,29 +108,66 @@ public:
bool start_object(std::size_t len) override bool start_object(std::size_t len) override
{ {
rs = std::make_unique<JSONObjectState>(std::move(rs)); stack.emplace_back(std::in_place_type<WIPObject>);
return true; return true;
} }
bool key(string_t & name) override bool key(string_t & name) override
{ {
dynamic_cast<JSONObjectState*>(rs.get())->key(name, state); auto & back = stack.back();
std::visit(
overloaded{
[&](WIPObject & frame) {
assert(!frame.nextKey);
frame.nextKey = state.ctx.symbols.create(name);
},
[](const WIPArray &) { assert(false && "tried adding a json key to an array value??"); }
},
back
);
return true; return true;
} }
bool end_object() override { bool end_object() override {
rs = rs->resolve(state); auto back = std::move(stack.back());
rs->add(); stack.pop_back();
std::visit(
overloaded{
[&](const WIPObject & frame) {
auto attrs2 = state.ctx.buildBindings(frame.entries.size());
for (auto & i : frame.entries) {
attrs2.insert(i.first, i.second);
}
addValue({NewValueAs::attrs, attrs2.alreadySorted()});
},
[](const WIPArray &) { assert(false && "tried to close a JSON object while in an array"); }
},
back
);
return true; return true;
} }
bool end_array() override { bool end_array() override {
return end_object(); auto back = std::move(stack.back());
stack.pop_back();
std::visit(
overloaded{
[](const WIPObject &) { assert(false && "tried to close a JSON array while in an object"); },
[&](const WIPArray & values) {
auto list = state.ctx.mem.newList(values.size());
std::ranges::copy(values, list->elems);
addValue({NewValueAs::list, list});
}
},
back
);
return true;
} }
bool start_array(size_t len) override { bool start_array(size_t len) override {
rs = std::make_unique<JSONListState>(std::move(rs), auto v = WIPArray{};
len != std::numeric_limits<size_t>::max() ? len : 128); v.reserve(len != std::numeric_limits<size_t>::max() ? len : 128);
stack.push_back(std::move(v));
return true; return true;
} }
@@ -184,13 +178,13 @@ public:
} }
}; };
void parseJSON(EvalState & state, const std::string_view & s_, Value & v) Value parseJSON(EvalState & state, const std::string_view & s_)
{ {
JSONSax parser(state); JSONSax parser(state);
bool res = JSON::sax_parse(s_, &parser); bool res = JSON::sax_parse(s_, &parser);
if (!res) if (!res)
throw JSONParseError("Invalid JSON Value"); throw JSONParseError("Invalid JSON Value");
v = parser.result(); return parser.result();
} }
} }
lix/libexpr/json-to-value.hh
+1 -2
View File
@@ -12,6 +12,5 @@ struct Value;
MakeError(JSONParseError, Error); MakeError(JSONParseError, Error);
void parseJSON(EvalState & state, const std::string_view & s, Value & v); Value parseJSON(EvalState & state, const std::string_view & s);
} }
lix/libexpr/nixexpr.cc
-18
View File
@@ -99,13 +99,6 @@ JSON ExprVar::toJSON(const SymbolTable & symbols) const
return {{"_type", "ExprVar"}, {"value", stringToJSON(symbols[name])}}; return {{"_type", "ExprVar"}, {"value", stringToJSON(symbols[name])}};
} }
JSON ExprInheritFrom::toJSON(SymbolTable const & symbols) const
{
return {
{"_type", "ExprInheritFrom"}
};
}
JSON ExprSelect::toJSON(const SymbolTable & symbols) const JSON ExprSelect::toJSON(const SymbolTable & symbols) const
{ {
JSON out = { JSON out = {
@@ -747,17 +740,6 @@ void ExprLambda::setName(Symbol name)
body->setName(name); body->setName(name);
} }
std::string ExprLambda::showNamePos(const EvalState & state) const
{
std::string id(name
? concatStrings("'", state.ctx.symbols[name], "'")
: "anonymous function");
return fmt("%1% at %2%", id, state.ctx.positions[pos]);
}
/* Position table. */ /* Position table. */
Pos PosTable::operator[](PosIdx p) const Pos PosTable::operator[](PosIdx p) const
lix/libexpr/nixexpr.hh
+46 -21
View File
@@ -126,10 +126,15 @@ public:
std::unique_ptr<Expr> parsed, Evaluator & es, const std::shared_ptr<const StaticEnv> & env std::unique_ptr<Expr> parsed, Evaluator & es, const std::shared_ptr<const StaticEnv> & env
); );
virtual JSON toJSON(const SymbolTable & symbols) const; virtual JSON toJSON(const SymbolTable & symbols) const = 0;
virtual void accept(ExprVisitor & ev, std::unique_ptr<Expr> & ptr) = 0; virtual void accept(ExprVisitor & ev, std::unique_ptr<Expr> & ptr) = 0;
virtual Value eval(EvalState & state, Env & env); virtual Value eval(EvalState & state, Env & env);
Value makeThunk(Evaluator & ctx, Env & env);
virtual Value maybeThunk(EvalState & state, Env & env); virtual Value maybeThunk(EvalState & state, Env & env);
/* Lambdas have a name associated with them, when they are declared in a binding:
* `identity = x: x` will print the resulting value as `«lambda identity @ «string»:1:14»`.
* This is set in the parser. After parsing, all expressions are immutable.
*/
virtual void setName(Symbol name); virtual void setName(Symbol name);
PosIdx getPos() const { return pos; } PosIdx getPos() const { return pos; }
@@ -175,7 +180,7 @@ struct ExprLiteral : Expr
{ {
protected: protected:
Value v; Value v;
ExprLiteral(const PosIdx pos) : Expr(pos) {}; ExprLiteral(const PosIdx pos, Value v) : Expr(pos), v(v) {};
public: public:
Value maybeThunk(EvalState & state, Env & env) override; Value maybeThunk(EvalState & state, Env & env) override;
JSON toJSON(const SymbolTable & symbols) const override; JSON toJSON(const SymbolTable & symbols) const override;
@@ -183,34 +188,46 @@ public:
void accept(ExprVisitor & ev, std::unique_ptr<Expr> & ptr) override { ev.visit(*this, ptr); } void accept(ExprVisitor & ev, std::unique_ptr<Expr> & ptr) override { ev.visit(*this, ptr); }
}; };
struct ExprInt : ExprLiteral struct ExprInt : private std::tuple<Value::Int>, ExprLiteral
{ {
Value::Int i; ExprInt(const PosIdx pos, NixInt n)
ExprInt(const PosIdx pos, NixInt n) : ExprLiteral(pos), i{{Value::Acb::tInt}, n} : tuple({{Value::Acb::tInt}, n})
, ExprLiteral(
pos,
Value::isTaggableInteger(n) ? Value{NewValueAs::integer, n} : Value(std::get<Value::Int>(*this))
)
{ {
v = Value::isTaggableInteger(n) ? Value{NewValueAs::integer, n} : Value(i);
} }
ExprInt(const PosIdx pos, NixInt::Inner n) : ExprInt(pos, NixInt(n)) {} ExprInt(const PosIdx pos, NixInt::Inner n) : ExprInt(pos, NixInt(n)) {}
}; };
struct ExprFloat : ExprLiteral struct ExprFloat : private std::tuple<Value::Float>, ExprLiteral
{ {
Value::Float f;
ExprFloat(const PosIdx pos, NewValueAs::floating_t, double f) ExprFloat(const PosIdx pos, NewValueAs::floating_t, double f)
: ExprLiteral(pos) : tuple({{Value::Acb::tFloat}, f})
, f{{Value::Acb::tFloat}, f} , ExprLiteral(pos, Value(std::get<Value::Float>(*this)))
{ {
v = Value(this->f);
} }
}; };
struct ExprString : ExprLiteral struct ExprStringBase
{ {
std::unique_ptr<Value::Str, Value::Str::Deleter> contents; std::unique_ptr<Value::Str, Value::Str::Deleter> contents;
Value::String strcb{.content = contents.get(), .context = nullptr}; Value::String strcb;
ExprString(const PosIdx pos, std::string s) : ExprLiteral(pos), contents(Value::Str::copy(s)) protected:
ExprStringBase(std::string_view s, const char ** context = nullptr)
: contents(Value::Str::copy(s))
, strcb{.content = contents.get(), .context = context}
{
}
};
struct ExprString : private ExprStringBase, ExprLiteral
{
ExprString(const PosIdx pos, std::string s)
: ExprStringBase(s)
, ExprLiteral(pos, Value{NewValueAs::string, &strcb})
{ {
v = {NewValueAs::string, &strcb};
} }
std::string_view str() const std::string_view str() const
@@ -219,11 +236,11 @@ struct ExprString : ExprLiteral
} }
}; };
struct ExprPath : ExprLiteral struct ExprPath : private ExprStringBase, ExprLiteral
{ {
std::unique_ptr<Value::Str, Value::Str::Deleter> contents; ExprPath(const PosIdx pos, std::string s)
Value::String strcb{.content = contents.get(), .context = Value::String::path}; : ExprStringBase(s, Value::String::path)
ExprPath(const PosIdx pos, std::string s) : ExprLiteral(pos), contents(Value::Str::copy(s)) , ExprLiteral(pos, Value{NewValueAs::path, &strcb})
{ {
v = Value{NewValueAs::path, &strcb}; v = Value{NewValueAs::path, &strcb};
} }
@@ -286,7 +303,11 @@ struct ExprInheritFrom : Expr
{ {
} }
JSON toJSON(const SymbolTable & symbols) const override; JSON toJSON(const SymbolTable & symbols) const override
{
abort();
}
Value eval(EvalState & state, Env & env) override; Value eval(EvalState & state, Env & env) override;
void accept(ExprVisitor & ev, std::unique_ptr<Expr> & ptr) override { ev.visit(*this, ptr); } void accept(ExprVisitor & ev, std::unique_ptr<Expr> & ptr) override { ev.visit(*this, ptr); }
}; };
@@ -501,7 +522,6 @@ struct ExprLambda : Expr
{ {
} }
void setName(Symbol name) override; void setName(Symbol name) override;
std::string showNamePos(const EvalState & state) const;
/** Returns the name of the lambda, /** Returns the name of the lambda,
* or "anonymous lambda" if it doesn't have one. * or "anonymous lambda" if it doesn't have one.
@@ -640,6 +660,11 @@ struct ExprPos : Expr
/* only used to mark thunks as black holes. */ /* only used to mark thunks as black holes. */
struct ExprBlackHole : Expr struct ExprBlackHole : Expr
{ {
JSON toJSON(const SymbolTable & symbols) const override
{
abort();
}
Value eval(EvalState & state, Env & env) override; Value eval(EvalState & state, Env & env) override;
void accept(ExprVisitor & ev, std::unique_ptr<Expr> & ptr) override { ev.visit(*this, ptr); } void accept(ExprVisitor & ev, std::unique_ptr<Expr> & ptr) override { ev.visit(*this, ptr); }
}; };
lix/libexpr/parser/state.hh
+12 -2
View File
@@ -133,14 +133,24 @@ inline void State::badEscapeFound(const PosIdx pos, char found, bool isIndented)
{ {
auto escape = std::string(isIndented ? "''\\" : "\\"); auto escape = std::string(isIndented ? "''\\" : "\\");
auto interpolEscape = std::string(isIndented ? "''${" : "\\${"); auto interpolEscape = std::string(isIndented ? "''${" : "\\${");
auto backslashChar = std::string("\\") + found;
HintFmt dedicatedInsert = isIndented
? HintFmt("you can simply write it as %s in the string", backslashChar)
: HintFmt("you need to escape the %s itself: %s", escape, escape + escape + found);
HintFmt msg = HintFmt( HintFmt msg = HintFmt(
"%s is an ill-defined escape. You can drop the %s and simply write %s instead. " "%s is an ill-defined escape sequence. In Nix, it simply means %s, therefore the %s is redundant and "
"should be removed. If the intent of the string was to mean %s instead (e.g. in a regex), %s. "
"Use %s to silence this warning.", "Use %s to silence this warning.",
escape + found, escape + found,
escape,
found, found,
escape,
backslashChar,
Uncolored(dedicatedInsert.str()),
"--extra-deprecated-features broken-string-escape" "--extra-deprecated-features broken-string-escape"
); );
/* Special case some common escapes to provide better messages */ /* Special case some common escapes to provide better messages */
if (found == '$' || found == '{') { if (found == '$' || found == '{') {
/* Someone possibly tried to escape an interpolation but used the wrong sequence. /* Someone possibly tried to escape an interpolation but used the wrong sequence.
lix/libexpr/primops.cc
+425 -338
View File
File diff suppressed because it is too large Load Diff
lix/libexpr/primops.hh
+3 -4
View File
@@ -48,13 +48,12 @@ public:
/** /**
* Load a ValueInitializer from a DSO and return whatever it initializes * Load a ValueInitializer from a DSO and return whatever it initializes
*/ */
void prim_importNative(EvalState & state, Value * * args, Value & v); Value prim_importNative(EvalState & state, Value ** args);
/** /**
* Execute a program and parse its output * Execute a program and parse its output
*/ */
void prim_exec(EvalState & state, Value * * args, Value & v); Value prim_exec(EvalState & state, Value ** args);
void makePositionThunks(EvalState & state, const PosIdx pos, Value & line, Value & column);
std::tuple<Value, Value> makePositionThunks(EvalState & state, const PosIdx pos);
} }
lix/libexpr/primops/context.cc
+16 -19
View File
@@ -7,22 +7,21 @@
namespace nix { namespace nix {
void prim_unsafeDiscardStringContext(EvalState & state, Value ** args, Value & v) Value prim_unsafeDiscardStringContext(EvalState & state, Value ** args)
{ {
NixStringContext context; NixStringContext context;
auto s = state.coerceToString(noPos, *args[0], context, "while evaluating the argument passed to builtins.unsafeDiscardStringContext"); auto s = state.coerceToString(noPos, *args[0], context, "while evaluating the argument passed to builtins.unsafeDiscardStringContext");
v = {NewValueAs::string, *s}; return {NewValueAs::string, *s};
} }
void prim_hasContext(EvalState & state, Value * * args, Value & v) Value prim_hasContext(EvalState & state, Value ** args)
{ {
NixStringContext context; NixStringContext context;
state.forceString(*args[0], context, noPos, "while evaluating the argument passed to builtins.hasContext"); state.forceString(*args[0], context, noPos, "while evaluating the argument passed to builtins.hasContext");
v = {NewValueAs::boolean, !context.empty()}; return {NewValueAs::boolean, !context.empty()};
} }
Value prim_unsafeDiscardOutputDependency(EvalState & state, Value ** args)
void prim_unsafeDiscardOutputDependency(EvalState & state, Value * * args, Value & v)
{ {
NixStringContext context; NixStringContext context;
auto s = state.coerceToString(noPos, *args[0], context, "while evaluating the argument passed to builtins.unsafeDiscardOutputDependency"); auto s = state.coerceToString(noPos, *args[0], context, "while evaluating the argument passed to builtins.unsafeDiscardOutputDependency");
@@ -39,11 +38,10 @@ void prim_unsafeDiscardOutputDependency(EvalState & state, Value * * args, Value
} }
} }
v = {NewValueAs::string, *s, context2}; return {NewValueAs::string, *s, context2};
} }
Value prim_addDrvOutputDependencies(EvalState & state, Value ** args)
void prim_addDrvOutputDependencies(EvalState & state, Value * * args, Value & v)
{ {
NixStringContext context; NixStringContext context;
auto s = state.coerceToString(noPos, *args[0], context, "while evaluating the argument passed to builtins.addDrvOutputDependencies"); auto s = state.coerceToString(noPos, *args[0], context, "while evaluating the argument passed to builtins.addDrvOutputDependencies");
@@ -82,7 +80,7 @@ void prim_addDrvOutputDependencies(EvalState & state, Value * * args, Value & v)
}, context.begin()->raw) }), }, context.begin()->raw) }),
}; };
v = {NewValueAs::string, *s, context2}; return {NewValueAs::string, *s, context2};
} }
@@ -105,7 +103,7 @@ void prim_addDrvOutputDependencies(EvalState & state, Value * * args, Value & v)
Note that for a given path any combination of the above attributes Note that for a given path any combination of the above attributes
may be present. may be present.
*/ */
void prim_getContext(EvalState & state, Value * * args, Value & v) Value prim_getContext(EvalState & state, Value ** args)
{ {
struct ContextInfo { struct ContextInfo {
bool path = false; bool path = false;
@@ -136,20 +134,19 @@ void prim_getContext(EvalState & state, Value * * args, Value & v)
for (const auto & info : contextInfos) { for (const auto & info : contextInfos) {
auto infoAttrs = state.ctx.buildBindings(3); auto infoAttrs = state.ctx.buildBindings(3);
if (info.second.path) if (info.second.path)
infoAttrs.alloc(state.ctx.symbols.sym_path) = {NewValueAs::boolean, true}; infoAttrs.insert(state.ctx.symbols.sym_path, {NewValueAs::boolean, true});
if (info.second.allOutputs) if (info.second.allOutputs)
infoAttrs.alloc(sAllOutputs) = {NewValueAs::boolean, true}; infoAttrs.insert(sAllOutputs, {NewValueAs::boolean, true});
if (!info.second.outputs.empty()) { if (!info.second.outputs.empty()) {
auto & outputsVal = infoAttrs.alloc(state.ctx.symbols.sym_outputs);
auto content = state.ctx.mem.newList(info.second.outputs.size()); auto content = state.ctx.mem.newList(info.second.outputs.size());
outputsVal = {NewValueAs::list, content}; infoAttrs.insert(state.ctx.symbols.sym_outputs, {NewValueAs::list, content});
for (const auto & [i, output] : enumerate(info.second.outputs)) for (const auto & [i, output] : enumerate(info.second.outputs))
content->elems[i] = {NewValueAs::string, output}; content->elems[i] = {NewValueAs::string, output};
} }
attrs.alloc(state.ctx.store->printStorePath(info.first)) = {NewValueAs::attrs, infoAttrs}; attrs.insert(state.ctx.store->printStorePath(info.first), {NewValueAs::attrs, infoAttrs});
} }
v = {NewValueAs::attrs, attrs}; return {NewValueAs::attrs, attrs};
} }
@@ -158,7 +155,7 @@ void prim_getContext(EvalState & state, Value * * args, Value & v)
See the commentary above unsafeGetContext for details of the See the commentary above unsafeGetContext for details of the
context representation. context representation.
*/ */
void prim_appendContext(EvalState & state, Value ** args, Value & v) Value prim_appendContext(EvalState & state, Value ** args)
{ {
NixStringContext context; NixStringContext context;
auto orig = state.forceString(*args[0], context, noPos, "while evaluating the first argument passed to builtins.appendContext"); auto orig = state.forceString(*args[0], context, noPos, "while evaluating the first argument passed to builtins.appendContext");
@@ -232,6 +229,6 @@ void prim_appendContext(EvalState & state, Value ** args, Value & v)
} }
} }
v = {NewValueAs::string, orig, context}; return {NewValueAs::string, orig, context};
} }
} }
lix/libexpr/primops/fetchMercurial.cc
+7 -7
View File
@@ -5,7 +5,7 @@
namespace nix { namespace nix {
void prim_fetchMercurial(EvalState & state, Value ** args, Value & v) Value prim_fetchMercurial(EvalState & state, Value ** args)
{ {
std::string url; std::string url;
std::optional<Hash> rev; std::optional<Hash> rev;
@@ -88,18 +88,18 @@ void prim_fetchMercurial(EvalState & state, Value ** args, Value & v)
auto [tree, input2] = state.aio.blockOn(input.fetch(state.ctx.store)); auto [tree, input2] = state.aio.blockOn(input.fetch(state.ctx.store));
auto attrs2 = state.ctx.buildBindings(8); auto attrs2 = state.ctx.buildBindings(8);
state.ctx.paths.mkStorePathString(tree.storePath, attrs2.alloc(state.ctx.symbols.sym_outPath)); attrs2.insert(state.ctx.symbols.sym_outPath, state.ctx.paths.mkStorePathString(tree.storePath));
if (input2.getRef()) if (input2.getRef())
attrs2.alloc("branch") = {NewValueAs::string, *input2.getRef()}; attrs2.insert("branch", {NewValueAs::string, *input2.getRef()});
// Backward compatibility: set 'rev' to // Backward compatibility: set 'rev' to
// 0000000000000000000000000000000000000000 for a dirty tree. // 0000000000000000000000000000000000000000 for a dirty tree.
auto rev2 = input2.getRev().value_or(Hash(HashType::SHA1)); auto rev2 = input2.getRev().value_or(Hash(HashType::SHA1));
attrs2.alloc("rev") = {NewValueAs::string, rev2.gitRev()}; attrs2.insert("rev", {NewValueAs::string, rev2.gitRev()});
attrs2.alloc("shortRev") = {NewValueAs::string, rev2.gitRev().substr(0, 12)}; attrs2.insert("shortRev", {NewValueAs::string, rev2.gitRev().substr(0, 12)});
if (auto revCount = input2.getRevCount()) if (auto revCount = input2.getRevCount())
attrs2.alloc("revCount") = {NewValueAs::integer, NixInt::Inner(*revCount)}; attrs2.insert("revCount", {NewValueAs::integer, NixInt::Inner(*revCount)});
v = {NewValueAs::attrs, attrs2};
state.ctx.paths.allowPath(tree.storePath); state.ctx.paths.allowPath(tree.storePath);
return {NewValueAs::attrs, attrs2};
} }
} }
lix/libexpr/primops/fetchTree.cc
+49 -39
View File
@@ -26,49 +26,51 @@ Value emitTreeAttrs(
auto attrs = state.buildBindings(10); auto attrs = state.buildBindings(10);
state.paths.mkStorePathString(tree.storePath, attrs.alloc(state.symbols.sym_outPath)); attrs.insert(state.symbols.sym_outPath, state.paths.mkStorePathString(tree.storePath));
// FIXME: support arbitrary input attributes. // FIXME: support arbitrary input attributes.
auto narHash = input.getNarHash(); auto narHash = input.getNarHash();
assert(narHash); assert(narHash);
attrs.alloc("narHash") = {NewValueAs::string, narHash->to_string()}; attrs.insert("narHash", {NewValueAs::string, narHash->to_string()});
if (input.getType() == "git") if (input.getType() == "git")
attrs.alloc("submodules") = { attrs.insert(
NewValueAs::boolean, fetchers::maybeGetBoolAttr(input.attrs, "submodules").value_or(false) "submodules",
}; {NewValueAs::boolean, fetchers::maybeGetBoolAttr(input.attrs, "submodules").value_or(false)}
);
if (!forceDirty) { if (!forceDirty) {
if (auto rev = input.getRev()) { if (auto rev = input.getRev()) {
attrs.alloc("rev") = {NewValueAs::string, rev->gitRev()}; attrs.insert("rev", {NewValueAs::string, rev->gitRev()});
attrs.alloc("shortRev") = {NewValueAs::string, rev->gitShortRev()}; attrs.insert("shortRev", {NewValueAs::string, rev->gitShortRev()});
} else if (emptyRevFallback) { } else if (emptyRevFallback) {
// Backwards compat for `builtins.fetchGit`: dirty repos return an empty sha1 as rev // Backwards compat for `builtins.fetchGit`: dirty repos return an empty sha1 as rev
auto emptyHash = Hash(HashType::SHA1); auto emptyHash = Hash(HashType::SHA1);
attrs.alloc("rev") = {NewValueAs::string, emptyHash.gitRev()}; attrs.insert("rev", {NewValueAs::string, emptyHash.gitRev()});
attrs.alloc("shortRev") = {NewValueAs::string, emptyHash.gitShortRev()}; attrs.insert("shortRev", {NewValueAs::string, emptyHash.gitShortRev()});
} }
if (auto revCount = input.getRevCount()) if (auto revCount = input.getRevCount())
attrs.alloc("revCount") = {NewValueAs::integer, NixInt::Inner(*revCount)}; attrs.insert("revCount", {NewValueAs::integer, NixInt::Inner(*revCount)});
else if (emptyRevFallback) else if (emptyRevFallback)
attrs.alloc("revCount") = {NewValueAs::integer, 0}; attrs.insert("revCount", {NewValueAs::integer, 0});
} }
if (auto dirtyRev = fetchers::maybeGetStrAttr(input.attrs, "dirtyRev")) { if (auto dirtyRev = fetchers::maybeGetStrAttr(input.attrs, "dirtyRev")) {
attrs.alloc("dirtyRev") = {NewValueAs::string, *dirtyRev}; attrs.insert("dirtyRev", {NewValueAs::string, *dirtyRev});
attrs.alloc("dirtyShortRev") = { attrs.insert(
NewValueAs::string, *fetchers::maybeGetStrAttr(input.attrs, "dirtyShortRev") "dirtyShortRev", {NewValueAs::string, *fetchers::maybeGetStrAttr(input.attrs, "dirtyShortRev")}
}; );
} }
if (auto lastModified = input.getLastModified()) { if (auto lastModified = input.getLastModified()) {
attrs.alloc("lastModified") = {NewValueAs::integer, *lastModified}; attrs.insert("lastModified", {NewValueAs::integer, *lastModified});
attrs.alloc("lastModifiedDate") = { attrs.insert(
NewValueAs::string, fmt("%s", std::put_time(std::gmtime(&*lastModified), "%Y%m%d%H%M%S")) "lastModifiedDate",
}; {NewValueAs::string, fmt("%s", std::put_time(std::gmtime(&*lastModified), "%Y%m%d%H%M%S"))}
);
} }
return {NewValueAs::attrs, attrs}; return {NewValueAs::attrs, attrs};
@@ -106,14 +108,14 @@ struct FetchTreeParams {
bool allowNameArgument = false; bool allowNameArgument = false;
}; };
static void fetchTree( static Value fetchTree(
EvalState & state, EvalState & state,
const PosIdx pos, const PosIdx pos,
Value * * args, Value ** args,
Value & v,
std::optional<std::string> type, std::optional<std::string> type,
const FetchTreeParams & params = FetchTreeParams{} const FetchTreeParams & params = FetchTreeParams{}
) { )
{
fetchers::Input input; fetchers::Input input;
NixStringContext context; NixStringContext context;
@@ -223,16 +225,17 @@ static void fetchTree(
state.ctx.paths.allowPath(tree.storePath); state.ctx.paths.allowPath(tree.storePath);
v = emitTreeAttrs(state.ctx, tree, input2, params.emptyRevFallback, false); return emitTreeAttrs(state.ctx, tree, input2, params.emptyRevFallback, false);
} }
void prim_fetchTree(EvalState & state, Value * * args, Value & v) Value prim_fetchTree(EvalState & state, Value ** args)
{ {
fetchTree(state, noPos, args, v, std::nullopt, FetchTreeParams { .allowNameArgument = false }); return fetchTree(state, noPos, args, std::nullopt, FetchTreeParams{.allowNameArgument = false});
} }
static void fetch(EvalState & state, const PosIdx pos, Value * * args, Value & v, static Value fetch(
const std::string & who, bool unpack, std::string name) EvalState & state, const PosIdx pos, Value ** args, const std::string & who, bool unpack, std::string name
)
{ {
std::optional<std::string> url; std::optional<std::string> url;
std::optional<Hash> expectedHash; std::optional<Hash> expectedHash;
@@ -297,8 +300,7 @@ static void fetch(EvalState & state, const PosIdx pos, Value * * args, Value & v
}); });
if (state.aio.blockOn(state.ctx.store->isValidPath(expectedPath))) { if (state.aio.blockOn(state.ctx.store->isValidPath(expectedPath))) {
state.ctx.paths.allowAndSetStorePathString(expectedPath, v); return state.ctx.paths.allowAndSetStorePathString(expectedPath);
return;
} }
} }
@@ -306,10 +308,16 @@ static void fetch(EvalState & state, const PosIdx pos, Value * * args, Value & v
// https://github.com/NixOS/nix/issues/4313 // https://github.com/NixOS/nix/issues/4313
auto storePath = unpack auto storePath = unpack
? state.aio ? state.aio
.blockOn(fetchers::downloadTarball(state.ctx.store, *url, name, (bool) expectedHash)) .blockOn(
fetchers::downloadTarball(
state.ctx.store, *url, name, expectedHash != Hash(HashType::SHA256)
)
)
.tree.storePath .tree.storePath
: state.aio : state.aio
.blockOn(fetchers::downloadFile(state.ctx.store, *url, name, (bool) expectedHash)) .blockOn(
fetchers::downloadFile(state.ctx.store, *url, name, expectedHash != Hash(HashType::SHA256))
)
.storePath; .storePath;
if (expectedHash) { if (expectedHash) {
@@ -329,22 +337,24 @@ static void fetch(EvalState & state, const PosIdx pos, Value * * args, Value & v
} }
} }
state.ctx.paths.allowAndSetStorePathString(storePath, v); return state.ctx.paths.allowAndSetStorePathString(storePath);
} }
void prim_fetchurl(EvalState & state, Value * * args, Value & v) Value prim_fetchurl(EvalState & state, Value ** args)
{ {
fetch(state, noPos, args, v, "fetchurl", false, ""); return fetch(state, noPos, args, "fetchurl", false, "");
} }
void prim_fetchTarball(EvalState & state, Value * * args, Value & v) Value prim_fetchTarball(EvalState & state, Value ** args)
{ {
fetch(state, noPos, args, v, "fetchTarball", true, "source"); return fetch(state, noPos, args, "fetchTarball", true, "source");
} }
void prim_fetchGit(EvalState & state, Value * * args, Value & v) Value prim_fetchGit(EvalState & state, Value ** args)
{ {
fetchTree(state, noPos, args, v, "git", FetchTreeParams { .emptyRevFallback = true, .allowNameArgument = true }); return fetchTree(
state, noPos, args, "git", FetchTreeParams{.emptyRevFallback = true, .allowNameArgument = true}
);
} }
} }
lix/libexpr/primops/fromTOML.cc
+15 -24
View File
@@ -7,7 +7,7 @@
namespace nix { namespace nix {
void prim_fromTOML(EvalState & state, Value ** args, Value & val) Value prim_fromTOML(EvalState & state, Value ** args)
{ {
auto toml = state.forceStringNoCtx( auto toml = state.forceStringNoCtx(
*args[0], noPos, "while evaluating the argument passed to builtins.fromTOML" *args[0], noPos, "while evaluating the argument passed to builtins.fromTOML"
@@ -15,62 +15,53 @@ void prim_fromTOML(EvalState & state, Value ** args, Value & val)
std::istringstream tomlStream(std::string{toml}); std::istringstream tomlStream(std::string{toml});
auto visit = [&](this const auto & self, Value & v, toml::value t) -> void { auto visit = [&](this const auto & self, toml::value t) -> Value {
switch (t.type()) { switch (t.type()) {
case toml::value_t::table: { case toml::value_t::table: {
auto table = toml::get<toml::table>(t); auto table = toml::get<toml::table>(t);
auto attrs = state.ctx.buildBindings(table.size()); auto attrs = state.ctx.buildBindings(table.size());
for (auto & elem : table) { for (auto & elem : table) {
self(attrs.alloc(elem.first), elem.second); attrs.insert(elem.first, self(elem.second));
} }
v = {NewValueAs::attrs, attrs}; return {NewValueAs::attrs, attrs};
} break; }
case toml::value_t::array: { case toml::value_t::array: {
auto array = toml::get<std::vector<toml::value>>(t); auto array = toml::get<std::vector<toml::value>>(t);
size_t size = array.size(); size_t size = array.size();
auto list = state.ctx.mem.newList(size); auto list = state.ctx.mem.newList(size);
v = {NewValueAs::list, list};
for (size_t i = 0; i < size; ++i) { for (size_t i = 0; i < size; ++i) {
self(list->elems[i], array[i]); list->elems[i] = self(array[i]);
} }
} break; return {NewValueAs::list, list};
}
case toml::value_t::boolean: case toml::value_t::boolean:
v = {NewValueAs::boolean, toml::get<bool>(t)}; return {NewValueAs::boolean, toml::get<bool>(t)};
break;
case toml::value_t::integer: case toml::value_t::integer:
v = {NewValueAs::integer, toml::get<int64_t>(t)}; return {NewValueAs::integer, toml::get<int64_t>(t)};
break;
case toml::value_t::floating: case toml::value_t::floating:
v = {NewValueAs::floating, toml::get<NixFloat>(t)}; return {NewValueAs::floating, toml::get<NixFloat>(t)};
break;
case toml::value_t::string: case toml::value_t::string:
v = {NewValueAs::string, toml::get<std::string>(t)}; return {NewValueAs::string, toml::get<std::string>(t)};
break;
case toml::value_t::local_datetime: case toml::value_t::local_datetime:
case toml::value_t::offset_datetime: case toml::value_t::offset_datetime:
case toml::value_t::local_date: case toml::value_t::local_date:
case toml::value_t::local_time: case toml::value_t::local_time:
// NOLINTNEXTLINE(lix-foreign-exceptions) // NOLINTNEXTLINE(lix-foreign-exceptions)
throw std::runtime_error("Dates and times are not supported"); throw std::runtime_error("Dates and times are not supported");
break;
case toml::value_t::empty: case toml::value_t::empty:
v.mkNull(); return Value::VNULL;
break;
} }
}; };
try { try {
visit( return visit(
val,
toml::parse( toml::parse(
tomlStream, tomlStream,
"fromTOML", /* the "filename" */ "fromTOML", /* the "filename" */
toml::spec::v( toml::spec::v(1, 0, 0) // Be explicit that we are parsing TOML 1.0.0 without extensions
1, 0, 0
) // Be explicit that we are parsing TOML 1.0.0 without extensions
) )
); );
} catch (std::exception & e) { // NOLINT(lix-foreign-exceptions) // TODO: toml::syntax_error } catch (std::exception & e) { // NOLINT(lix-foreign-exceptions) // TODO: toml::syntax_error
lix/libexpr/settings/repl-overlays.md
+3 -3
View File
@@ -17,10 +17,10 @@ Each file is called with three arguments:
except that it's available in except that it's available in
[`pure-eval`](@docroot@/command-ref/conf-file.html#conf-pure-eval) [`pure-eval`](@docroot@/command-ref/conf-file.html#conf-pure-eval)
mode). mode).
2. The top-level bindings produced by the previous `repl-overlays` 2. The final top-level bindings produced by calling all
value (or the default top-level bindings).
3. The final top-level bindings produced by calling all
`repl-overlays`. `repl-overlays`.
3. The top-level bindings produced by the previous `repl-overlays`
value (or the default top-level bindings).
For example, the following file would alias `pkgs` to For example, the following file would alias `pkgs` to
`legacyPackages.${info.currentSystem}` (if that attribute is defined): `legacyPackages.${info.currentSystem}` (if that attribute is defined):
lix/libexpr/value.cc
+2 -9
View File
@@ -12,8 +12,10 @@ namespace nix
static const Value::List emptyListData{.size = 0}; static const Value::List emptyListData{.size = 0};
Value Value::EMPTY_LIST{Value::list_t{}, &emptyListData}; Value Value::EMPTY_LIST{Value::list_t{}, &emptyListData};
Value Value::EMPTY_SET{attrs_t{}, &Bindings::EMPTY};
const Value::Null Value::NULL_ACB = {{Value::Acb::tNull}}; const Value::Null Value::NULL_ACB = {{Value::Acb::tNull}};
Value Value::VNULL{null_t{}};
static_assert(alignof(Value::String) >= Value::TAG_ALIGN); static_assert(alignof(Value::String) >= Value::TAG_ALIGN);
static_assert(alignof(Bindings) >= Value::TAG_ALIGN); static_assert(alignof(Bindings) >= Value::TAG_ALIGN);
@@ -51,15 +53,6 @@ void Value::print(EvalState & state, std::ostream & str, PrintOptions options)
printValue(state, str, *this, options); printValue(state, str, *this, options);
} }
bool Value::isTrivial() const
{
return internalType() != tApp
&& (internalType() != tThunk
|| (thunk().expr->try_cast<ExprSet>()
&& static_cast<ExprSet *>(thunk().expr)->dynamicAttrs.empty())
|| thunk().expr->try_cast<ExprLambda>() || thunk().expr->try_cast<ExprList>());
}
Value::Value(string_t, Str * s, const NixStringContext & context) Value::Value(string_t, Str * s, const NixStringContext & context)
: Value(NewValueAs::string, s, copyContext(context)) : Value(NewValueAs::string, s, copyContext(context))
{ {
lix/libexpr/value.hh
+6 -18
View File
@@ -32,7 +32,7 @@ struct Value;
/** /**
* Function that implements a primop. * Function that implements a primop.
*/ */
using PrimOpImpl = void(EvalState & state, Value ** args, Value & v); using PrimOpImpl = Value(EvalState & state, Value ** args);
/** /**
* Info about a primitive operation, and its implementation * Info about a primitive operation, and its implementation
@@ -284,6 +284,7 @@ private:
InternalType internalType() const InternalType internalType() const
{ {
// NOLINTNEXTLINE(lix-cast-to-non-fixed-enum): TAG_MASK ensures it's in range
return InternalType(raw & TAG_MASK); return InternalType(raw & TAG_MASK);
} }
@@ -356,6 +357,8 @@ public:
* Empty list constant. * Empty list constant.
*/ */
static Value EMPTY_LIST; static Value EMPTY_LIST;
static Value EMPTY_SET;
static Value VNULL;
struct String; struct String;
struct Acb; struct Acb;
@@ -721,6 +724,8 @@ public:
Type type() const Type type() const
{ {
// TAG_MASK == 7, max enumerator is 5, so all possible values are in range
// NOLINTNEXTLINE(lix-cast-to-non-fixed-enum)
return Type(raw & TAG_MASK); return Type(raw & TAG_MASK);
} }
}; };
@@ -764,16 +769,6 @@ public:
*/ */
inline ValueType type(bool invalidIsThunk = false) const; inline ValueType type(bool invalidIsThunk = false) const;
inline void mkNull()
{
*this = {NewValueAs::null};
}
inline void mkExternal(ExternalValueBase * e)
{
*this = {NewValueAs::external, *e};
}
bool isList() const bool isList() const
{ {
return internalType() == tList; return internalType() == tList;
@@ -783,13 +778,6 @@ public:
size_t listSize() const; size_t listSize() const;
/**
* Check whether forcing this value requires a trivial amount of
* computation. In particular, function applications are
* non-trivial.
*/
bool isTrivial() const;
auto listItems() const auto listItems() const
{ {
struct ListIterable struct ListIterable
lix/libfetchers/cache.cc
+24
View File
@@ -1,4 +1,6 @@
#include "lix/libfetchers/cache.hh" #include "lix/libfetchers/cache.hh"
#include "libstore/pathlocks.hh"
#include "libutil/hash.hh"
#include "lix/libstore/sqlite.hh" #include "lix/libstore/sqlite.hh"
#include "lix/libutil/async.hh" #include "lix/libutil/async.hh"
#include "lix/libutil/sync.hh" #include "lix/libutil/sync.hh"
@@ -90,6 +92,28 @@ struct CacheImpl : Cache
co_return result::current_exception(); co_return result::current_exception();
} }
kj::Promise<Result<std::variant<std::pair<Attrs, StorePath>, PathLock>>> lookupOrLock(ref<Store> store, const Attrs & inAttrs) override try {
// In order to avoid fetching the same input multiple times
// concurrently, we first acquire a lock based on the input attributes.
auto hashResult = hashString(HashType::SHA256, attrsToJSON(inAttrs).dump());
auto lockPath = getCacheDir() + "/nix/fetcher-lock-" + hashResult.to_string(HashFormat::Base32, false);
auto pathLock = TRY_AWAIT(lockPathAsync(lockPath));
// Once this lock is acquired, we check if the input is already present
// in the cache (which requires locking the cache db)
auto lookedUp = TRY_AWAIT(lookup(store, inAttrs));
// We return either the cache hit or the lock; this allows fetchers to
// keep a lock on fetching the input in question if it's not already in
// the cache, and otherwise frees the cache db up again.
std::variant<std::pair<Attrs, StorePath>, PathLock> result(std::move(pathLock));
if (lookedUp) {
// Reassigning the variant frees the lock
result = *lookedUp;
}
co_return result;
} catch (...) {
co_return result::current_exception();
}
kj::Promise<Result<std::optional<LookupResult>>> lookupExpired( kj::Promise<Result<std::optional<LookupResult>>> lookupExpired(
ref<Store> store, ref<Store> store,
const Attrs & inAttrs) override const Attrs & inAttrs) override
lix/libfetchers/cache.hh
+5
View File
@@ -1,6 +1,7 @@
#pragma once #pragma once
///@file ///@file
#include "libstore/pathlocks.hh"
#include "lix/libfetchers/fetchers.hh" #include "lix/libfetchers/fetchers.hh"
#include "lix/libstore/path.hh" #include "lix/libstore/path.hh"
@@ -21,6 +22,10 @@ struct Cache
ref<Store> store, ref<Store> store,
const Attrs & inAttrs) = 0; const Attrs & inAttrs) = 0;
virtual kj::Promise<Result<std::variant<std::pair<Attrs, StorePath>, PathLock>>> lookupOrLock(
ref<Store> store,
const Attrs & inAttrs) = 0;
struct LookupResult struct LookupResult
{ {
bool expired = false; bool expired = false;
lix/libfetchers/git.cc
+43 -16
View File
@@ -1,3 +1,4 @@
#include "libfetchers/attrs.hh"
#include "lix/libutil/archive.hh" #include "lix/libutil/archive.hh"
#include "lix/libutil/async-io.hh" #include "lix/libutil/async-io.hh"
#include "lix/libutil/async.hh" #include "lix/libutil/async.hh"
@@ -15,9 +16,10 @@
#include "lix/libutil/url-parts.hh" #include "lix/libutil/url-parts.hh"
#include "lix/libstore/pathlocks.hh" #include "lix/libstore/pathlocks.hh"
#include "lix/libutil/users.hh" #include "lix/libutil/users.hh"
#include "lix/libutil/git.hh"
#include "lix/libutil/logging.hh" #include "lix/libutil/logging.hh"
#include "lix/libutil/finally.hh" #include "lix/libutil/finally.hh"
#include "lix/lix-rs/main.gen.hh"
#include "lix/lix-rs/utils.hh"
#include "lix/libfetchers/fetch-settings.hh" #include "lix/libfetchers/fetch-settings.hh"
@@ -80,17 +82,17 @@ try {
)); ));
std::string_view line = output; std::string_view line = output;
line = line.substr(0, line.find("\n")); auto line_rs = rust::to_string(line.substr(0, line.find("\n")));
if (const auto parseResult = git::parseLsRemoteLine(line)) { if (const auto parseResult =
switch (parseResult->kind) { to_std(rust::lix::fetchers::git::LsRemoteRefLine::try_from(line_rs.as_str()).ok()))
case git::LsRemoteRefLine::Kind::Symbolic: {
debug("resolved HEAD ref '%s' for repo '%s'", parseResult->target, path); auto target = to_std_string(parseResult->target.as_str());
break; if (parseResult->kind.matches_Object()) {
case git::LsRemoteRefLine::Kind::Object: debug("resolved HEAD rev '%s' for repo '%s'", target, path);
debug("resolved HEAD rev '%s' for repo '%s'", parseResult->target, path); } else {
break; debug("resolved HEAD ref '%s' for repo '%s'", target, path);
} }
co_return parseResult->target; co_return to_std_string(parseResult->target.as_str());
} }
co_return std::nullopt; co_return std::nullopt;
} catch (ExecError &) { } catch (ExecError &) {
@@ -605,9 +607,18 @@ struct GitInputScheme : InputScheme
return {std::move(storePath), input}; return {std::move(storePath), input};
}; };
// If the input isn't present in the fetcher cache, we get a lock for
// fetching the given input. We want to hold this lock until we're done
// fetching, so we keep it in this scope.
std::optional<PathLock> fetchLock;
if (input.getRev()) { if (input.getRev()) {
if (auto res = TRY_AWAIT(getCache()->lookup(store, getLockedAttrs()))) auto resOrLock = TRY_AWAIT(getCache()->lookupOrLock(store, getLockedAttrs()));
if (auto res = std::get_if<std::pair<Attrs, StorePath>>(&resOrLock))
co_return makeResult(res->first, std::move(res->second)); co_return makeResult(res->first, std::move(res->second));
auto lock = std::get_if<PathLock>(&resOrLock);
assert(lock);
fetchLock = std::move(*lock);
} }
auto [isLocal, actualUrl_] = getActualUrl(input); auto [isLocal, actualUrl_] = getActualUrl(input);
@@ -676,7 +687,10 @@ struct GitInputScheme : InputScheme
} }
} }
if (auto res = TRY_AWAIT(getCache()->lookup(store, unlockedAttrs))) { // If the input isn't in the cache, we get a lock for fetching it.
// Make sure to keep this until we're done fetching!
auto cached = TRY_AWAIT(getCache()->lookupOrLock(store, unlockedAttrs));
if (auto res = std::get_if<std::pair<Attrs, StorePath>>(&cached)) {
auto rev2 = Hash::parseAny(getStrAttr(res->first, "rev"), HashType::SHA1); auto rev2 = Hash::parseAny(getStrAttr(res->first, "rev"), HashType::SHA1);
if (!input.getRev() || input.getRev() == rev2) { if (!input.getRev() || input.getRev() == rev2) {
input.attrs.insert_or_assign("rev", rev2.gitRev()); input.attrs.insert_or_assign("rev", rev2.gitRev());
@@ -858,8 +872,21 @@ struct GitInputScheme : InputScheme
/* Now that we know the ref, check again whether we have it in /* Now that we know the ref, check again whether we have it in
the store. */ the store. */
if (auto res = TRY_AWAIT(getCache()->lookup(store, getLockedAttrs()))) // If we already have a lock for fetching this path, we can't use lookupOrLock because it would deadlock.
co_return makeResult(res->first, std::move(res->second)); if (fetchLock) {
if (auto res = TRY_AWAIT(getCache()->lookup(store, getLockedAttrs()))) {
co_return makeResult(res->first, std::move(res->second));
}
} else {
// If we don't have the lock, we need to acquire it by using lookupOrLock.
auto resultOrLock = TRY_AWAIT(getCache()->lookupOrLock(store, getLockedAttrs()));
if (auto res = std::get_if<std::pair<Attrs, StorePath>>(&resultOrLock)) {
co_return makeResult(res->first, std::move(res->second));
}
auto lock = std::get_if<PathLock>(&resultOrLock);
assert(lock);
fetchLock = std::move(*lock);
}
Path tmpDir = createTempDir(); Path tmpDir = createTempDir();
AutoDelete delTmpDir(tmpDir, true); AutoDelete delTmpDir(tmpDir, true);
@@ -974,7 +1001,7 @@ struct GitInputScheme : InputScheme
}); });
Finally const _wait([&] { proc.waitAndCheck(); }); Finally const _wait([&] { proc.waitAndCheck(); });
TRY_AWAIT(unpackTarfile(*proc.getStdout(), tmpDir)); TRY_AWAIT(unpackTarfile(_input.toURLString(), *proc.getStdout(), tmpDir));
} }
auto storePath = TRY_AWAIT( auto storePath = TRY_AWAIT(
lix/libfetchers/github.cc
+12 -7
View File
@@ -7,10 +7,11 @@
#include "lix/libutil/result.hh" #include "lix/libutil/result.hh"
#include "lix/libutil/types.hh" #include "lix/libutil/types.hh"
#include "lix/libutil/url-parts.hh" #include "lix/libutil/url-parts.hh"
#include "lix/libutil/git.hh"
#include "lix/libutil/json.hh" #include "lix/libutil/json.hh"
#include "lix/libfetchers/fetchers.hh" #include "lix/libfetchers/fetchers.hh"
#include "lix/libfetchers/fetch-settings.hh" #include "lix/libfetchers/fetch-settings.hh"
#include "lix/lix-rs/main.gen.hh"
#include "lix/lix-rs/utils.hh"
#include <optional> #include <optional>
#include <fstream> #include <fstream>
@@ -486,15 +487,16 @@ struct SourceHutInputScheme : GitArchiveInputScheme
std::string line; std::string line;
getline(is, line); getline(is, line);
auto remoteLine = git::parseLsRemoteLine(line); auto remoteLine = to_std(
rust::lix::fetchers::git::LsRemoteRefLine::try_from(rust::to_string(line).as_str()).ok()
);
if (!remoteLine) { if (!remoteLine) {
throw BadURL("in '%d', couldn't resolve HEAD ref '%d'", input.to_string(), ref); throw BadURL("in '%d', couldn't resolve HEAD ref '%d'", input.to_string(), ref);
} }
refUri = remoteLine->target; refUri = to_std_string(remoteLine->target.as_str());
} else { } else {
refUri = fmt("refs/(heads|tags)/%s", ref); refUri = fmt("refs/(heads|tags)/%s", ref);
} }
std::regex refRegex = regex::parse(refUri);
auto file = store->toRealPath( auto file = store->toRealPath(
TRY_AWAIT(downloadFile(store, fmt("%s/info/refs", base_url), "source", false, headers)) TRY_AWAIT(downloadFile(store, fmt("%s/info/refs", base_url), "source", false, headers))
@@ -505,9 +507,12 @@ struct SourceHutInputScheme : GitArchiveInputScheme
std::string line; std::string line;
std::optional<std::string> id; std::optional<std::string> id;
while(!id && getline(is, line)) { while(!id && getline(is, line)) {
auto parsedLine = git::parseLsRemoteLine(line); auto parsedLine = to_std(
if (parsedLine && parsedLine->reference && std::regex_match(*parsedLine->reference, refRegex)) rust::lix::fetchers::git::LsRemoteRefLine::try_from(rust::to_string(line).as_str()).ok()
id = parsedLine->target; );
if (parsedLine && (*parsedLine).matches_ref_uri(rust::to_string(refUri).as_str())) {
id = to_std_string(parsedLine->target.as_str());
}
} }
if(!id) if(!id)

Some files were not shown because too many files have changed in this diff Show More