meson fixed the bugs that prevented using the sanitizer args, but using
such a fixed meson will provide broken flags to the subprojects. *sigh*
Change-Id: I0ab54351b19a3879e9a97daed1621dc5f730d376
it's used very rarely, on ssh it's pretty much broken, and on ssh-ng
it's standing squarely in the way of the rpc transition. we have not
found significant use of this feature in public configs (with all of
*two* repositories using this store parameter), so why even keep it?
Change-Id: Ifcafdf794d815001a1a55a771e5823010e5b174b
there's no reason a store should not open a connection during init if it
needs to open any connection to function. delaying connection setup like
this makes openStore less deterministic and graceful fallback impossible
this caused test failures where the old behaviour was required to ensure
test output stability. because of course something like that must happen
Change-Id: I946adddee026f1f4c74b4699d730b4d1ac9a5072
build-remote should not be the only thing that ever reads logs from
remote store connections. ssh-ng is better off here because it does
not transport its logs on the ssh stderr channel, but ssh *does* do
this. neither have any ssh logs shown after connection setup except
when using an ssh:// url as a remote builder, which is not markedly
helpful for debugging anything at all. let's have both kinds of ssh
store classes handle logs themselves to alleviate all of that. this
also improves the utility of remote builder checks in `nix doctor`.
Change-Id: I23947a9666bc07561eac341578ed45993f1b9839
adding logs to a daemon store can fail before the entire request has
been read, leaving non-command data in socket buffers. if the daemon
does not terminate immediately after the failure it will try to read
the remaining data as commands, which is very likely to not work and
cause unbounded memory allocation instead. this doesn't give clients
an attack vector they did not have before though, they could've just
as well sent malformed commands without a bad preceding AddBuildLog.
Change-Id: I86c22890eace19164d932cbd342ac9f52cee6531
we don't support range-based iteration yet because it's a huge hassle to
make c++ and rust iteration styles meet in any reasonable way. this is a
good start to make rust collections *actually* usable though, so here we
go. `into_iter` is not explicitly supported, but can be used regardless.
Change-Id: Iebb76c9409059bc81623e5be81ac2656b1ec5138
frames manually added with `addErrorContext` are generally a lot more
useful/informative to the average user than other frames, esp. in
the module system, which can create much better error messages than
we can.
however, before this change, frames from addErrorContext were truncated
by default if they weren't in the first 3 frames, so they were basically
useless (with `--show-trace`, you're dredging through 250 frames of
module shenanigans just to spot one singular line).
this change also removes the frames for _the call to_ `addErrorContext`,
which is just pure noise.
the way this change is done hopefully leaves a bit of space for future
similar changes to error printing, by introducing a new `TraceKind` enum
that can be used to categorize traces (i haven't done that in this CL
because that would be a pretty herculean task, given all the calls to
`BaseError::addTrace` in the codebase, and we probably want to be
careful about what categories we choose). the actual printing code could
definitely be improved tho... (e.g. by iterating twice through the trace
stack instead to first pick out the most important traces and _then_
printing less "important" traces if there's space left)
Change-Id: I52acc52f231991a9f2309d9cecae362397c6888c
this makes using the zero-copy pipes a lot more ergonomic. error
handling is unfortunately not trivial and duplicating it for all
pipe users would thus be a bad idea. we're also not oblivious to
the fact that this is a `sourceToSink`, but it's async this time
around. (at least we don't need terrible stackful coroutines..?)
Change-Id: I1ba59f27183988ad68e7f88d102935d005690f43
rpc will often need to transfer data received on push-bashed interfaces
to consumers that expect input stream sources, which are pull-based. we
want to avoid copies for performance reasons (since kj overhead as kind
of on the high side for us already), so we'll use a shared-buffer class
that behaves much like a mutex. we *don't* use mutexes because ours are
cross-thread-capable and thus require syscalls for all wakeups they do.
Change-Id: I3b14925f5d9f5e07ea2cafdf00a88a64f79e4742
Found ... by edef's harness. There was more dead code it found, which I
didn't address because it felt less obviously deletable:
- `AutoDestroyCgroup::path()` / `::delegation()`
- `TeeSource`, `LambdaSink`
- `S3BinaryCacheStore::getS3Stats()` (maybe Hydra? idk)
- `fetchers::getBoolAttr()` (technically deletable but it would be
inconsistent with the others)
Change-Id: I913d2afd61e24136fb506389fc33a1016a6a6964
sometimes we need to disambiguate based on the kind of the thing we are
filling due to how the c++ type works. in templates some constraints on
type resolution are lifted. sometimes we can make the compiler help us.
Co-authored-by: piegames <git@piegames.de>
Change-Id: Id9549764813a862b8d64959157fed258709eb6ac
This is purely aesthetically bothering me, but [] inserting stuff into
the map is such a C++ coded design decision and we don't intend to
mutate it here so we probably should not do it.
Change-Id: Idf37c51175b031fb396e8fd53dc8fe6f2039fb55
Many issues stems from `<nixpkgs>` not resolving anywhere or having the
wrong version, let's make it awfully obvious again here.
Towards #230.
Change-Id: If630e9616566e77143f028cdcfe0b7f00a1d486d
Signed-off-by: Raito Bezarius <raito@lix.systems>
without maxprocesses we can utilize core-rich systems better, and test
timeouts are reportedly to be too low to run on small systems as well.
fixes#890
Change-Id: I89386b89fcd69ef4bf77ecf0a49c0b85e1f17c3d
this fixes a large portion of tests currently marked no_daemon. most of
them only needed to set some trusted settings, which is easily done now
Change-Id: Id5a5ee94951cdc92bddd2264c738ca4f98980c8b
Some code-paths expect e.g. `nixpkgs.rev` to exist.
This change makes sure this always exists. This is done on purpose
within this let-in dance rather than in the default such that
`nix-build release.nix --arg nixpkgs ../nixpkgs` also benefits from this
fix.
Change-Id: I94f1f901823d3df433e3c44ae380bb2f67eef70e
attrpathsSuperset got renamed to preEval in nixpkgs[1] breaking Hydra's
evaluation. Updating nixpkgs to make sure this is consistent now, no
matter if flake-inputs or release.nix is being used.
[1] See commit 19a31658dc39324c9acf8d81198cd7137bdc1e92.
Change-Id: I69ab3f894534b1d89b626ddd4feda8ea502b69b3
single-user builds have a writable home directory in the build sandbox,
and the test suite does not like cargo writing anything into there. the
impact of this seems to be localized to f1 (at least on linux), but not
letting cargo write into the homedir is easy enough to do. nixpkgs also
does something much like this when compiling rustc, so we're not alone.
(it also seems that nixpkgs intended to do this for all rustc users but
broke it at some point in the past. some packages set CARGO_HOME in the
same way we do, possibly because they've had exactly the same problems)
fixes#1251
Change-Id: Ie17a0677ef09bb076ef6295bb590a0895924434e
we need to cache the current terminal size for progress bar reasons, but
we don't want to catch SIGWINCH to update the current terminal size from
a dedicated thread for repl reasons. a SIGWINCH handler function is much
easier to square with these requirements than communication with another
thread to have it change its signal mask, and since any races in handler
code affect only progress bar output and only very rarely (if ever) we'd
better chose the simplest approach. the progress bar could set a handler
of its own for this purpose, but we would much rather replace it instead
fixes#1246
Change-Id: I814d9aaf1b6fbb6a8cefc5af675a3aa372549dc8
this is a bit of a hack, but since zngur cannot handle multiple trait
implementations per type yet we will have to commit to singles types.
Change-Id: I60e7b96bbeaa9fb87cf43662d4a9a5d44116bf47
note: only two of the tests actually needed migrating, as the others
were already covered (e.g. by lang)
Change-Id: I30abb2bd728ae2b144ad76566096a216aed3b8f8
mimalloc is a compact general purpose allocator from Microsoft. It
consistently outperforms glibc's `malloc()` in allocation-heavy
workloads, such as Lix's evaluator
It's currently only linked in the main `nix` executable, as Boehm's GC
uses its own allocator. Other allocations that *do* go through glibc's
`malloc()` are still much faster, though
Benchmarked on x86_64-linux against Nixpkgs `bd07873`:
| attribute | thunks | lix@`1396012` | mimalloc | uplift |
|-------------------|---------|---------------|----------|--------|
| hello | 206444 | 0.726s | 0.620s | 1.17x |
| chromium | 1177382 | 2.273s | 2.136s | 1.06x |
| firefox-unwrapped | 1394797 | 2.521s | 2.378s | 1.06x |
| texliveFull | 3186440 | 4.992s | 4.672s | 1.07x |
| nixosTests.gnome | 7905808 | 6.115s | 5.723s | 1.07x |
Based-on: https://github.com/NixOS/nix/pull/15596
Co-authored-by: Bernardo Meurer Costa <beme@anthropic.com>
Change-Id: I3ad92eacc075efeaf3d9f7730ada7b29835536a4
Some issues are caused by users confused about their ambient search
paths, let's analyze the current configuration and make it awfully
obvious what is going on.
Towards #230.
Change-Id: I953ef3feaefc56cee68623c5aaa4d568ba1fd034
Signed-off-by: Raito Bezarius <raito@lix.systems>
We try to access to the Flake registry, if we are failing, it's hosed.
Otherwise, let's record some facts we can reuse later.
Towards #230.
Change-Id: I80dd8272d5c27f7f7ff01052e3487a0496ee29fc
Signed-off-by: Raito Bezarius <raito@lix.systems>
Based on `nix-info` and `nix --version` output.
Contributes towards #230.
Change-Id: I13bb9608b733e10037c6816456b5312e712c18ad
Signed-off-by: Raito Bezarius <raito@lix.systems>
Based on `nix-info` output and experience (LOCALE issues).
Change-Id: I350fff9592e7acc8fba0babe67c9fe1bf94a5781
Signed-off-by: Raito Bezarius <raito@lix.systems>
This is some code that is specific to systemd to fetch hostnamectl
information via the JSON flag.
This will be used for `nix doctor`.
Errors will show up similar to this:
```
❯ ./build/lix/nix/nix doctor
[INFO] Nix system type: 'x86_64-linux'
warning: could not get host information via hostnamectl: error: Expected JSON value to be of type 'string' but it is of type 'number'
```
and makes them non-fatal.
Change-Id: Idd39f29626e770b1cd3896e8eb8545416c74b041
Signed-off-by: Raito Bezarius <raito@lix.systems>
treating a file with an empty or zero hash as locked is not helpful.
these are placeholdes for "hash is not known", thus treating them as
a valid lock makes them completely useless (and confusing to users).
fixes#1233
Change-Id: If42b47281e6973fc86662b973db69e26f6346f5a
25.11 is dead, and so is mdbook 0.4
Also reenable linkcheck which was doing nothing this whole time lmao
Change-Id: I4a8b9c763b881de840d6ef1b1d85bf19941f4386
These uses of std::move are taking data out of a repeatedly-used state
object, à la Rust's std::mem::take(). But move-constructing or
move-assigning from an object does not guarantee that the moved-from
object is equivalent to a default-constructed one, or even that the
moved-from object will continue to function like a non-moved-from object
of that type (only that the object remains "valid").
Assuming there's no differing custom allocator shenanigans at play, both
libstdc++ and libc++ do leave moved-from std::map objects (and probably
others) in an empty, well-behaved, default-constructed state, at the
time of this writing. However the correct tool here is std::exchange().
Change-Id: I7cf76762a1b05a542b53fcd74be100646a6a6964
This way, other parts of the codebase can use it.
Change-Id: I58673a6a56362692980426a6c8f70f5c6aff24d4
Signed-off-by: Raito Bezarius <raito@lix.systems>
Since d0190cff6f, the `justfile` uses
`$out` from the environment as an argument to `--prefix` in its `meson`
invocation.
The `$out` variable is... A bit spicy to use since its semantics change
depending on whether `nix develop` or `nix-shell` is used.
In a `nix develop` shell, a directory *relative to PWD* (and not
relative to the Nix expression) is exported. In a `nix-shell`, the
`$out` used comes from the derivation.
The alternative to this change is to make the `justfile` use a variable
name that would not conflict with a `nix-shell` and `nix develop` shell.
Since it wants to use that value for the `outdir` variable, this would
be the name I would pick first.
I chose to instead work around the issue within the `nix-shell` since
this would be breaking the existing interface. As tenuous as it is.
Change-Id: I64cd5fd85345457e14880cb133e9ddafd55a25da
rust-analyzer binaries that don't come from nix (such as the one
shipped with vscode's rust-analyzer extension by default) need the
`RUST_SRC_PATH` env var to be the path to rustc's sources, but this
wasn't set inside the devshell. this change explicitely adds it to
the environment. (trying to add pkgs.rustLibSrc and rustcSrc to
the shell's packages unfortunately does nothing, because they are
just pure source packages, nothing more.)
(the reason this probably didn't come up for other devs is because
1. a lot of lix devs use other editors that probably just use the
binary from the path, and the devshell installs pkgs.rust-analyzer,
which is patched to always have `RUST_SRC_PATH` set, and 2. because
pkgs.vscode-extensions.rust-lang.rust-analyzer is also patched to use
pkgs.rust-analyzer by default instead of the bundled one, so installing
the extension using nix instead of from the marketplace would also hide
this problem.)
Change-Id: Ia27daf2a88756311ac5485ea0af44aae242241a9
results don't allow for streaming or pipelining. transporting errors in
exception texts is *bad*, but it's still better than not having streams
that actually work. this isn't a great situation, but with capnp that's
pretty much the best we can do. still beats the other rpc frameworks :/
Change-Id: I2abf17bc5ea5de2baeea44ff4d7b2f4db33e98dd
using result types in capnp is fraught. while it makes some sense for
interfaces that need neither streaming nor pipelining and can provide
much better error fidelity there it's also fundamentally incompatible
with those that *do* need streaming or pipelining: streams will never
be stopped early unless an exception flies, and pipelines cannot look
through result types. likely the best thing we can do is to encode an
error for transport in the capnp/kj exception description strings. :(
Change-Id: Icb7d16238fa9a7aaf92c00363f7be4076ac02a61
This is a hack, but this is the best we can do with Just unfortunately.
Maybe rewriting this file into a python script wouldn't be the worst
idea at this point …
Change-Id: I32d089f6b49165faed99fb592d068dbb8a7e48e6
This is the real deal. The new tests have surfaced one bug in the test
suite itself, and one in the RPC legacy protocol wrapper.
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: Id75b671c10b45655d3627abd88563e2d41719446
All tests should have been clear here, but alas
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: Id642be1f6e0fb714b844ee5348dc096866cf3906
This commit enables the parametrization for "legacy-combined" protocol.
All failing tests with that have been either fixed or disabled. Notably:
- A couple of tests (e.g. involving builders) could be made to work with
a non-local store, but this would require some refactoring to testlib in
order to make certain configuration settings generic over local and
remote operations. We've disabled those for now, in order to make
progress
- The store tests all run locally only because of their nature
- The flakes tests also all run locally only because my energy for
fixing them is limited (and it's a *lot* of test failures, with probably
little overall benefit in terms of test coverage)
Change-Id: I56fa249a64f7c17c952f688ec89e9687f2a13f12
this takes the form of a couple of hideous macros for now to get started
without worrying about semantics of rust logging and tracing crates, all
of which are in some way incompatible with the current lix expectations.
we do want to move to using those eventually, but that will have to be a
task for later. as long as only small amounts of lix code live in crates
we don't have to worry much anyway, and by the time we get around to any
Activity beimg ported we'll hopefully have a better understanding of the
requirements on either side. we'll reëvaluate the log infra around then.
Change-Id: I4aec476a486f56d0972294358cbf0b6810ab9394
This is a second attempt at https://gerrit.lix.systems/c/lix/+/5516
instead of letting meson try to be cargo we'll just have cargo at home.
this requires some contortions to link everything together due to quite
a few meson deficiencies, but at the end we get to pretend that rust is
just c++ painted orange. we'll use cxx to bring real interop back soon.
also fixes#1230
also reverts 10845bfe63
co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: Ib1fda843fa80d818705d5a65ec9054216a6a6964
This first geralizes the `nix` fixture with `pytest_generate_tests` to
iterate over all protocols (unless they are marked with `no_daemon`),
though for now the list of protocols is set to be empty. The lang tests
are all tagged with `no_daemon` because they are mostly pure and running
them multiple times would be wasteful.
Co-authored-by: rootile <lix@rootile.de>
Co-authored-by: piegames <git@piegames.de>
Change-Id: Ib407edb420ba4bf434cacf9563a71f5ae6fa8eef
This is a bit of a trip.
Commit 63b0a01d75 moved config
initialization out of static initializers.
In doing so, an oversight was made with the new
`GlobalConfig::registerGlobalConfig` calls. Previously all global
configuration would be handled well before `loadConfFile` was called.
The change placed `registerGlobalConfig` in `main` *after*
`initNix`. The `initNix` function is what calls `initLibStore`, which in
turn callse `loadConfFile`.
So, in effect, the configuration for `daemonAuthorizationSettings` and
`nixDevelopSettings` are not actually loaded.
Change-Id: I10dba2c98e032f910942ee0e0ca6bb39ef60edc2
Full warning was:
warning: Dependency of package 'lix-shell-env' uses a nested list in attribute 'nativeBuildInputs'.
This is deprecated as of Nixpkgs release 26.05, and support will be removed in a future nixpkgs release.
Change-Id: I6b6c7425e1b03d3238ea21138cb3f2cf6a6a6964
they're not doing anything, and they're slightly in the way of reworking
the rust build infra. we'll add them back once that rework is completed.
Change-Id: Idde8ba3585fd08b89b4b4298349f8f58a29a9661
blokyk on Matrix found that is not only no longer needed but now
actually breaks the very thing it sought to fix
Change-Id: I39cc0df611a6d81c763063c9fd2f0f426a6a6964
When removing the f1 repl characterization test suit, these files were
kept on accident, despite them being dead code now
Change-Id: I13b2c5a96004787c26fe7209e4cb3aeedd8ada15
This reverts commit 0d6b372c19.
Reason for revert: In the time between this getting +2 and it being merged, all repl tests and its syntax have been migrated to f2, making these changes null and to dead code
Change-Id: I00f5237bf3f2ca3a60cc1584f67fb67196805e7e
Gosh. Why was there never a README in here explaining what it does
without requiring people to read C++? There was a half-assed grammar in
a comment, but it was very half-assed, so let's replace it with a
better-written one.
I'm documenting this so that neither I nor anyone else have to read C++
while rewriting the parser in Python.
See: https://git.lix.systems/lix-project/lix/issues/1196
Change-Id: Ia34ea7b02f109eb7753833e1c03d4b5d1d3c3b75
When extracting a global pyproject.toml from the f2 one, some f2
specific things slipped through and ended up in the global one.
This commit moves those parts back where they belong
Change-Id: I95706fabec487410717fb362c95bfee900cf5b8c
if the netrcFile setting has been changed tell curl that the file is required, so
fetching fails if the file does not exist
if the caFile setting has been specified, check if the file exists and throw an
error if is it non-existent
Refs: #1106
Change-Id: Icb9330a7a715175d35b9ed894ed945f0fd4d7061
So apparently we can still use the repl if the input file fails, as long
as we have the debugger enabled.
This Commit removes the "only one output block for should_fails"
requirement to allow this behavior
Change-Id: I02568c8d7e4305ab6414953306650bbe13c6f4bc
curl does not support this configuration. we can't easily change the
config setup to mirror how curl works, but we *can* print a warning.
fixes#1223
Change-Id: Iaa1583275d9cc9e5eed5552210db183a8e0628c8
the case-sensitiveness of the "accept all options" prompt (and the fact
it silently defaults to no) was confusing a few users, so by rewording
it slightly we can avoid that case-sensitiveness and make the prompt
clearer. for consistency, i also rewrote the individual accept prompt to
make it case-insensitive.
at reviewers' requests, i also reworked the code so it re-prompts the
user on invalid answers.
Change-Id: Ie786c0f26a0aa752b22d5a5e8af1b2c7ca2846b5
lix.pc currently lists several optional libraries (such as libcpuid and
libseccomp) as required dependencies. Update the pkg-config generation
step to only list optional dependencies which are used by the Lix build.
Change-Id: Iceea7ecc9421b5b10fc08edbe95a063a4e39cea1
what the actual fuck. swallowing all errors *except* EXDEV is not okay.
renames do not do this, so moves should not do it either. luckily we do
not use moveFile anywhere except the store path registration code. this
may or may not have caused problems in the past. probably nobody knows.
Change-Id: I2b0255a5703983cbd129abc3219c11ac7171fd12
catch and rewrap std::fs errors in some meaningful way, otherwise lix
can crash with foreign exception aborts if filesystem failures happen
that aren't already being caught. renames in particular can fail with
incomprehensible errors in even pretty normal circumstances elsewise.
fixes#1219, fixes#995
Change-Id: I0b1e5696d300aac3422e43cffc17a2426803d64e
I'm not sure why this wasn't an issue before, but the generation of
jobset constituents in Hydra trips over this. The Hydra-convention is to
have the `system`-attribute last anyways, not in-between.
Since this is a single attribute, get rid of it for the sake of
simplicity.
Change-Id: I80ef1b92ed158c8712ee899534ce82130614f338
This is a Hydra jobset definition to be used to warm up a cache with Lix
versions built against several nixpkgs versions. This is only happening
after a bunch of tests have passed.
Change-Id: Id0b0e50c08581f3f377c7c793d2fd83a6007999d
If e.g. i686-linux is not in the list of systems and thus no attribute in `nixpkgsFor`
exists (e.g. on hydra.afnix.fr), don't try to instantiate it.
Change-Id: I118e2af5289ad7bd95ce7f1e6b0728afb6f0321f
...expect for the dev-shell: while it doesn't seem reasonable to me to
even build dev-shells in post-merge CI, this should still remain part of
the pre-merge checks.
Change-Id: I5cb9b4cf53484f884d694770ed94d0dc0862117e
Step II to move the code out of flake.nix. Done in a second intermediate
commit to make it easier to review.
Change-Id: I13fb386f2e9b87b1d28c366e9b7f0d2dbe03315c
That way we can use it in non-flake contexts down the chain.
We don't override the `nixpkgs` anyways since this is a test artifact
that needs an old nixpkgs input, so this isn't a concern here anyways.
Change-Id: If2aa8f7171a445e8321ef449026e206cc5a0fe3c
The upcoming change factors out a lot of code to make it reusable for a
"legacy" Hydra project that we use to build a matrix of (Lix x nixpkgs)
to have a warm cache for everyone.
The Hydra project will not use flakes to have some inputs overridable,
so the change is needed. In non-flake mode we don't have a notion of
"self", so rename it to `lixSrc` such that the next commit is purely
about moving stuff without changing any aspect.
Current stock Hydra doesn't expose `lastModified` though, so right now
that is a cache-miss. There's a pending Hydra PR fixing that[1].
[1] https://git.lix.systems/lix-project/hydra/pulls/82
Change-Id: I59d70b8b874f576c782217e7bca099eda4c7add9
The reference to the flake is used in a single place only to get the
`doc` output of Lix. Remove it so that we can also instantiate the tests
in a legacy Hydra jobset (happening further down the chain).
Change-Id: Ie3eccf42d67c81fcf991342097783e5067d512bc
Noticed on my quest to make Lix buildable against multiple nixpkgs' to
provide pre-warmed caches for the flake on AFNix's Hydra.
`pkgs.systemd` has a new patch policy[1] that only allows patches for
hotfixes and fundamental incompatibilities. As such, the patch that
allows nspawn workloads to start without a `/usr/bin` was removed with the
recommendation to create that stub yourself.
[1] https://github.com/NixOS/nixpkgs/pull/488508
Change-Id: Iaa35ae3b8971867f12daeac5f94958c20a7d9233
The option has been broken since Nix 2.4. The flag was accepted, but not
used. This change plumbs it through to computeFSClosure.
Change-Id: Id6adee8ea8a6a4f457b24b650660f319e3daa2b2
This makes it easier to build lix on a different file system from the
source repo, e.g. symlinking ./build to a directory in /tmp.
The ./outputs directory is included for completeness as another
significant source of generated artifacts.
Change-Id: I1bb4c21c700beacb4882a1be473dd31353edd177
this logs unpackTarfile calls as an `actUnknown` activity, which
unfortunately doesn't report any progress for now. for the reasoning
behind this choice, see the previous iterations of this CL (cl/5590)
as well as the attached comments (the crux of the conversion is in DMs,
unfortunately). in short:
- i could't make a new ActivityType because that would be a breaking
change to the daemon protocol, including with previous lix versions
- i can't use actFileTransfer because that would show up as a download,
which would be confusing for both users and tools
- i can't use actCopyPath /or/ actCopyPaths because that would break
NAR/substitution accounting when doing remote builds
also, ArchiveDecompressionSource, which is apparently used for
decompressing logs etc, does its own custom decompression without using
unpackTarfile, so it is not affected by this. from what i can tell,
unpackTarfile is only used for:
- fetchtarball fetcher
- git fetcher
- channel unpacking
- nix-prefetch-url
Fixes#1213
Change-Id: I68f3be520681d8889292353dd41ab19a60c5eeea
By default, when one uses `in` assertions, upon failure newlines in any
of the two strings is escaped, and everything is printed in a single
line.
This commit allows one to pass `-vv` to print out the left and righthand
side unescaped, with each line of the output being its own line,
improving readability a lot when dealing with multiline outputs.
Change-Id: Iaf67a20fce57d375e2c62a8e8a0957b14eec26ef
It's software archaeology time.
- In 2019 (Nix 2.4), Eelco added `checkOverlays`, which strictly checked
that the overlay must be a function of two arguments, one called `final`
and the other `prev`. (dc3f52a144,
gh#3573)
- In 2024, NofairKing opened an issue about checking the exact name
being a silly idea (gh#10516). Instead of trying to make the check more
sensible, the first fix attempt simply expanded the check to allow
`previous` instead of `prev` (the author's preferred name for the
attribute, supposedly; gh#10553). After some discussion, instead a PR
got merged which simply removed the `prev` check altogether, while
leaving the `final` argument check intact for some reason
(ad65a50a94a97bf1f1a1902f43542d28a2e8206b, gh#10572).
- In 2025, over at Lix and oblivious to the latest change over at
CppNix, I came across this stupid check while trying to improve the
abstractions of the AST in Nixexpr. I spent (wasted, retrospect)
considerable amounts of time removing the exact name checks while also
improving the check overall (making it check for more than two
arguments, and also generally improved error message).
(0928d4d87a,
Id4244171123dd8a228be71ce9f04d8e9f647c111)
- Fast forward to 2026, where I run into this becursed piece of code
once again, and once again during some Lixexpr cleanups. Now that I have
seen how carlessly the issue was handled at CppNix, I can't be arsed to
give a flying fuck anymore. Out the code goes. Bye.
With this commit, all casts on `Expr` subclasses outside of libexpr
itself have been removed. No more violation of abstraction boundaries.
Good riddance.
Change-Id: I939968bb01d461764cfa0f4ea7152b4fcf1acf93
pour the foundations for rpc, and let them set. this is very much
unstable and must be opted into with explicit store uris (e.g. by
setting `NIX_REMOTE=daemon?protocol=any`). the daemon sockets are
not enabled by default and must be enabled with the `rpc-sockets`
experimental feature. we will not advertise this just yet because
in the current state it one has to be *very* dedicated to the rpc
cause to deploy this, but once we have some more bits migrated we
may want to add release notes and officially as for beta testing.
Co-Authored-By: piegames <git@piegames.de>
Change-Id: I85a96ccb700b91190c1eb37154bcc6ae1c03401a
splitting it off lets us reuse it (and all the legacy protocol handling
it encapsulates) in the rpc world as well by just tunneling wire bytes.
doing this allows incremental migration of to a new rpc protocol, until
we are finally ready to call the rpc protocol ready and remove the link
to the legacy protocol altogether (and maybe merge this function back).
Change-Id: I8f2a8795c4a9d713ebbbbfd02e9eec5271a2b7a9
this adds the protocol registry. we also extend the systemd unit setup
to allow adding new protocols without much fussing, and we move deamon
instances into protocol-dependent scopes. this will make monitoring or
resource limiting of individual protocol daemon instances much easier.
Change-Id: Ic9da08eb074868b6e57385075c5502b081e8461f
we'll use this to create a registry of protocols the system understands.
also use this struct during connection setup to make that simpler later.
Change-Id: Ifa481fea7ea2efa2a1f1be4d81076a9e022d24f9
inherited classes to stack of variants
Previously the code represented the stack of unclosed arrays and
objects as a linked list of virtual objects which seemed unnecessary.
There is a closed very small (3) number of possibilities, and one of
them only exists at the top level. A vector of variants seems a lot
simpler, and as a bonus, more performant.
Change-Id: I6bc429d682d22f69ca91dc0518e6be646a6a6964
string data shares a buffer with the binary string length field. size
calculations for string read buffers always include the length field;
sufficiently large length fields can cause these calculations to wrap.
a malicious nar could use this for OOB writes in the daemon (as root).
since we use strings only as tags for archive members and for symlinks
with their OS-dependent length limits we can simply limit string size.
1 MiB should be sufficient for all symlinks, and tags are always tiny.
Change-Id: I89fb05f73c1dbeda45d91244aba4cd526a3d83e1
These parameters are now created on https://s3.afnix.fr.
Change-Id: I96b6fd913429ee46d04c412cb141edd288665ced
Signed-off-by: Raito Bezarius <raito@lix.systems>
this is not read in code paths that are called repeatedly, and the only
reader can never see it being false. we also question the wisdom of the
flag in the first place; why should opening a connection fail because a
previous connection did? conditions may have improved a lot since then!
Change-Id: Ib7c219e6239432d99a22b10024847218469f1cd4
This isn't really relevant for a setting of 10k, but now that we want to
set it to 0 for Flakes it makes a difference as to whether or not one
top-level function call is allowed or not. (It shouldn't be)
Change-Id: I6bbf99826af0289c232b9d45d172a378b81abe4b
Caught by edef's harness. I rewrote the structured build log tests in
f2 and added a test for this one.
Before: it failed the build altogether:
```
building '/tmp/jade/pytest-of-jade/pytest-77/test_invalid_fields_files0_0/nix/store/skhjs5zs082nqiakj69qz32gyywz0v1a-unusual-loggi
ng.drv'...
error: unsupported JSON type 7
```
Now the output is as expected:
```
warning: Unable to handle a JSON message from the derivation builder: error: unsupported log field type number
@nix {"action": "start", "fields": [1.5], "id": 2, "type": 1, "level": 1, "text": "abc"}
```
Change-Id: Idbd28c5a4d4cf15f27207f83b730e7946a6a6964
This is a minor adjustment to 2b22dae1ba /
I3e9d7c1c7a6599a8e68302448bbb961d051002b7
Basically having the error point to the operand instead of the operator
kind of makes sense, but it would require error spans to *truly* make
sense, and in the meantime maintaining the logic is more hassle than it
is worth.
Change-Id: Idac98dd77a0f4a6cb386cb74e0d4eb5fd2f503e5
"while evaluating a path segment" was just plain wrong
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I827374635df2887d17a8ee8b1755c078bf0b506b
It's useful to obtain a fully assembled command and then wrap it to
execute into something else, e.g. gdb, strace or fakeroot.
Change-Id: Ida5928fd001925549eab89edfbf0facada7ef7f1
We choose to throw an error in the invalid-input space where we *can*
produce a valid (but wrong) result to allow ourselves to, mirroring
CppNix, change the semantics to not corrupt it at all in the future,
while having a middle state where it is an error.
This is a largely-rewritten version of
https://github.com/NixOS/nix/pull/13013.
Co-authored-by: rootile <lix@rootile.de>
Fixes: https://github.com/NixOS/nix/issues/12899
Upstream-PR: https://github.com/NixOS/nix/pull/13013
Change-Id: I6a6a6964cdd1a88714952e80c660d1fa57d1f2d8
This was found by edef's harness. Unfortunately this entire module has
no coverage, so I don't really want to write a test for it immediately
as I'd have to harness it (though we have an http server in f2!).
(I think it will be much more fun and rewarding to do so once we get
coverage up and running).
Filed a bug about the missing docs for this too:
https://git.lix.systems/lix-project/lix/issues/1190
Test plan:
```nix
let pkgs = import <nixpkgs> {}; in {
foo = pkgs.fetchurl {
url = "https://jade.fyi/robots.txt";
name = "blah";
hash = "";
};
}
```
Before:
```
$ nix-prefetch-url ./foo.nix -A foo
fetching path input 'path:/nix/store/knf5m0famajw2bzbsg72b052iy1ynpck-source'
path is '/nix/store/5rcgj95as1ywr463rva6bky18088ij4x-robots.txt'
1cr11i0ih4m6kkpks9fxil71wg5k2crikah1rfjgy607qzgy8vad
```
After (note correct path name):
```
$ nix-prefetch-url ./foo.nix -A foo
fetching path input 'path:/nix/store/knf5m0famajw2bzbsg72b052iy1ynpck-source'
path is '/nix/store/hc9drd1l3vki3bbaacwhi8j1dr9907vd-blah'
1cr11i0ih4m6kkpks9fxil71wg5k2crikah1rfjgy607qzgy8vad
```
Change-Id: I6d9da1c54c6a398a0a169ff01f3f3dce6a6a6964
before this change, the innermost frame was all the way at the top,
which means that in very nested code you'd have to scroll though
dozens or hundreds of frames before even seeing where you are, because
the last lines (which are immediately visible when the command returns)
would be the nix code's entry point and other outer frames instead,
which is rarely relevant.
this CL reverses this order, so that the innermost frames are the last
ones printed, and thus are immediately visible. (note that this is
already how errors are printed by nix in other contexts anyway, the
debugger's :bt is the only thing that prints the trace in "forward"
order.)
because `DebugState.traces()` use the homegrown `libutil::Generator`
instead of an stdlib container, we have to do that reversing ourselves,
in this case by just storing every element in an `std::list` (in reverse
order) and then traversing it. this is feels kinda dirty, but i don't
know any other way, and it's not exactly performance-critical.
Change-Id: I9f23e40e57f72a251d65a335b3bba3c3f77b935d
it seems like the repl_characterization tests have not been actually
running correctly for a while, since tests from more than 2 years ago
that should have failed have been in the test suite and OK'd thus far.
the reason why some tests passed when they shouldn't have is because
the AST nodes used by the characterization parser didn't have a correct
comparison operator, so the tests would only fail when one of the output
line was of the wrong *kind*, but not if it had the wrong *value*. this
was fixed by simply adding a default comparator to the TextNode type as
well as every derived class.
i've taken the occasion to fix the tests that were failing now that we
are actually being strict about them. most of these are pretty boring
changes like indentation, but if possible i'd like to get an explicit
ACK from at least one or two people on the core team that these changes
are okay.
for reference, here is the zulip thread that lead to this CL:
https://zulip.lix.systems/#narrow/channel/8-Testing-.2F-functional2/topic/.E2.9C.94.20repl-characterization.20tests.20not.20failing/near/10276
ps: i had to turn clang-format off around the macro for text node
declarations because it re-formatted the entire macro, causing a lot
of noising and producing uglier code; if this isn't okay or if there's
a better alternative, please let me know!
Change-Id: I1d6d92eb3e9cb705ecfeccd7b9294d6cfd7fe25b
I missed some cases in https://gerrit.lix.systems/c/lix/+/5490, which is
not surprising given that I didn't have a lint for it at the time.
Some of these have JSON deserializers, which is kind of scary (in that
it was UB). Probably no impact, again, due to the lack of
-fstrict-enums, but still, yikes!
Change-Id: I895a8bfdd9972d8ade9fa6f5aa30845d6a6a6964
This started by noticing that the coverage report generated by cl/5489
showed that the error message for when allow-import-from-derivation is
disabled appeared to be not covered by any tests. Since the existing
tests for IFD were in functional, which is deprecated, I migrated those
into functional2 and added a test for erroring when IFD is disabled.
Change-Id: I565674de3dd693e023ec99d501adc4706a6a6964
as of cl/5456 prim_importNative default constructs a `Value` in order to
pass as an out parameter to a ValueInitializer function. Default
construction is deprecated, but there isn't a meaningful value to
initialize it to instead. After cl/5357 this will give a deprecation
warning (which in an asanBuild will be an error), so this commit
suppresses that warning for that `Value` construction.
Change-Id: Ie567a6e81b5753672001320e9fc816c56a6a6964
The existing function is used with good reason as explained by its
comment. This suppresses the warning it creates.
Change-Id: I58b0de3f5854266fa77b6a06b7821a636a6a6964
Motivated by needing to remove a Value default construction from
`ExprVar::maybeThunk`, so part of #744
Change-Id: Ia12b5b019e49055ebd7ffa0cc39a59c06a6a6964
Found by edef's fuzzing. We::jade don't think this is security-relevant
or likely to be mistreated by a compiler, but UB is bad.
https://eel.is/c++draft/expr.static.cast#8
> If the enumeration type does not have a fixed underlying type, the value is unchanged if the original value is within the range of the enumeration values ([dcl.enum]), and otherwise, the behavior is undefined
Notably, the range is defined as the smallest bitfield type that could hold all the values, not any real type which exists: https://eel.is/c++draft/dcl.enum#8.
This is basically a footgun, and I'm writing a clang-tidy check to forbid casting to such types. But first I needed to write clang-tidy testing infrastructure: https://gerrit.lix.systems/c/lix/+/5493.
Change-Id: Ieaaa0fe2a92fd9f24f60761e312741186a6a6964
notably the bad-store test previously tested *bash*, not lix. we also
need to extend the list of non-fatal errors, but that's probably fine
Change-Id: I983f4adc8047fe800323099de4037711869ee547
this should improve error messages when connecting to long socket paths,
and make it at all possible to detect ENOTDIR from connect helpers. this
will be needed to handle daemon sockets that live in directories we have
not previously considered like the protocol name directories we need for
rpc purposes, all of which would have resulted in hard errors previously
Change-Id: I4e16cf09fedd07c93cfdf7b0f151106984507315
The link should be to Gerrit, not Forgejo.
Issue introduced in: https://gerrit.lix.systems/c/lix/+/5005
Change-Id: Ibf93f114303f59959621a265196e6eda29b315f9
I did a pass through all files, fixing bits that have annoyed me here
and there based on vibes. Roughly:
- Replaced some out values like bools or strings with lists where
appropriate. Those tests were likely older than the value printer
- Added some parser tests where it makes sense
- Gently touched some formatting woes, while trying to not cause too
much diff noise
- Removed some dead `with (import ./lib.nix);` code
Change-Id: I8c40b2110f0b7799f68ae38ba61f049c5f1f6ee8
Related to #1162
Currently if JSON with duplicate object keys is passed into `builtins.toJSON`,
it will silently drop all but the last value, keeping only the last.
This may be surprising, but as this has been consistent reliable behavior that
users might depend on, we should test for it.
Change-Id: Icc2adefabb161530e7cbfa7330919bab6a6a6964
Without this, the primop chokes on any thunks on attributes passed in
the attrset. It even is the reason why the test contained `builtins.seq`
to work around this. Supposedly, this might have been an intentional
restriction and changing this might break things in ways I cannot forsee
due to not knowing much about Flakes, however the status quo is equally
broken:
- The error message looks like an internal error and not like some
explicitly forbidden invariant violation.
- Seemingly simple syntax literals like "-1" compile to "__sub 0 1" and
thus create a thunk which then fails, which is utterly confusing ("why
does 1 work but not -1?")
- This is a stark violation of the principle of least surprise.
- Thunking relies on maybeThunk and thunk inlining optimizations, thus
not forcing thunks turns operational details of the evaluator into
language-observable behavior. That's bad.
I am changing this now regardless of the risk of breakage, because the
bytecode evaluator will have different thunk inlining optimizations and
thus inevitably cause mismatches in behavior anyways.
Change-Id: Ifc45c4d2900e40822383670b28e4e50ab8af317a
Although the fact that some characters are illegal in derivation names
was referenced in some parts of the docs, the actual list was not
documented anywhere, and the only source of truth seems to be the
[store source code](lix/libstore/path.cc).
This adds an admonition in the `derivation` docs that specifies which
characters and strings are actually allowed (especially since some
of them can be somewhat surprising, like the string "..-foo" not
being a valid name because of `..`).
Change-Id: Ibc4b4a141e262c21306ce9c0392f92cf0abe610a
the docs say that the arguments for an overlay is info/prev/final, but
it's actually info/final/prev. this fixes the docs both in the nix.conf
manual as well as in the `nix repl` help text
Change-Id: I3de5c122e7e77cc3f4e4550e3741d6a6e906ea20
That way we can automatically push the static build (compressed as
tarball) into the AFNix S3 bucket and have it updated on each update to
main.
Change-Id: I1002727ab12c0ab6b95c8d37ae4620996607d7b7
This applies the suggested change in #1179, replacing a dead link with a
link to the content's new location. Closes#1179.
Change-Id: I543e3308516218e224e495b8cf14a7c06a6a6964
This is a small tweak to the logic added in cl/1511 to detect explicit
throws when printing stack traces. Now when deciding whether to print
"caused by explicit throw", it checks not only that the error is a
ThrowError and that we are in a throw, but also that the ThrowError was
thrown by *this* throw, and not by another while in the process of
evaluating this throw's operand.
It turns this:
```
let
set = {
inner = throw "nested throw";
};
in
throw set.inner
error:
… caused by explicit throw
at /pwd/in.nix:6:1:
5| in
6| throw set.inner
| ^
7|
… while evaluating the attribute 'inner'
at /pwd/in.nix:3:5:
2| set = {
3| inner = throw "nested throw";
| ^
4| };
… caused by explicit throw
at /pwd/in.nix:3:13:
2| set = {
3| inner = throw "nested throw";
| ^
4| };
error: nested throw
```
into this:
```
error:
… while calling the 'throw' builtin
at /pwd/in.nix:6:1:
5| in
6| throw set.inner
| ^
7|
… while evaluating the attribute 'inner'
at /pwd/in.nix:3:5:
2| set = {
3| inner = throw "nested throw";
| ^
4| };
… caused by explicit throw
at /pwd/in.nix:3:13:
2| set = {
3| inner = throw "nested throw";
| ^
4| };
error: nested throw
```
Notice the difference in the top context frame. Before it incorrectly
attributed the throw error to the both throws instead of just the one
that actually threw.
Change-Id: If3b0b8311f1ae2ff1471e260fe59d9166a6a6964
Admittedly, I do not understand what the comment means:
> As soon as Nixpkgs updates to >= 3.0.0, change to lowdown_2_0!
But this should work fine for nixpkgs versions providing lowdown 2.x or
3.x, and not cause rebuilds of the nixpkgs Lix/Nix derivations.
Change-Id: I6fb9e775335b4dbc23add76b98106970956e19fb
When index-debug-info is enabled and the store path being copied is a
regular file (not a directory), std::get_if<nar_index::Directory>
returns nullptr since the NAR root is a File variant. The loop then
immediately dereferences buildIdDir->contents on the null pointer,
causing a segfault.
Add a null check at the top of the loop to break early when the NAR
root is not a directory.
Change-Id: I3a6e792b84cc12c837ecaddf4fee889e1bcb6397
A significant part of #1136.
This is a rather large cl, because all of the primops need to be changed
together.
Change-Id: I7d92698d5344bd6186ee4fa47f5c21966a6a6964
I just copied and slightly edited the `man nix.conf(5)` docs about
`repl-overlays` to the nix repl help text (but cut out the more
complicated example and redirecting to the nix.conf docs for more
info).
Fixes#303
Change-Id: I85efaef606d8779ac66fe72cd3947d663f33fa44
This input was added in cl/2884, but was unused even then. The
`linuxPackages` within `buildPackages` is used instead.
Change-Id: I71c522ef683aa098eac0b356b22007ba6a6a6964
Calls to JSONState::add() always followed an assignment to
JSONState::value(), resulting in the new value going through
JSONState::v for no good reason. The only use of `v` outside of a
pointless stepping stone for `add` was to store the final top-level
Value before it is read out by JSONSax::result(), so it really only
makes sense for the top level JSONState to contain Value field.
Change-Id: I1758c7b770eb4b0c122e501b764dd42b6a6a6964
We would've had `build-lowdown_2_0.aarch64-linux.aarch64-linux`
otherwise. This is a bit of a problem because my way of generating a
list of constituents for Hydra's per-architecture[1] release job stops
recursing once it encounters a system.
The alternative would be to switch `hydraJobs` to `<system>.x.y` or
switching Hydra to "legacy" jobsets. For the latter I'd prefer to do the
same for Buildkite such that we don't have diverging things to build
depending on pre/post-merge CI.
[1] per-architecute because the trusted AFNix builders don't support all
the architectures we support in Lix and with Flakes there's no way
of parameterizing the list of supported systems.
Change-Id: I33f31260caf86ed5bb0f728770ca3cf1c00adf31
Closes#496
When running
nix-build ../nixpkgs --arg config.allowUnfree true -A hello-unfree
the package `hello-unfree` is now built rather than getting an
eval-error rejecting instantiation. This is because `config.allowUnfree`
is now interpreted as nested attribute-set declaration, similar to how
it's done in `nix repl`.
To prevent sudden breakage, this behavior was carefully deprecated with
Nix throwing an error if the identifier for `--arg` is not a pure
identifier, but an expression as above.
Any kind of merging is rejected. I.e. doing
nix-build ../nixpkgs --arg config '{cudaSupport = true;}' --arg config.allowUnfree true
is prohibited. That way we don't have to think about merge semantics for
cases like this (or even worse `--arg config 'rec { ... }'`). Another
nice side-effect of this is that we don't need to create an EvalState to
force the values and implement merging.
Change-Id: I8b560883a4468a3f32f915764b08f5fdd8fe71bb
While it doesn't make sense to have `foo..bar`, the attribute-path
`foo."".bar` is valid and shouldn't throw.
Change-Id: Ifcddaad6233c6ba8f17cb5c953c2101d276dfeb6
This is a little more elegant and easier to reason about than prefixing
strings with whatever type the rest of the string is.
Change-Id: I7769535303dcb9f67b79e89bef162beec990e2a0
previously, `builtins.break` didn't force its argument, resulting in
a value wrapped with `break` being opaque to most builtins if not
also wrapped with `seq`. see [lix-project/lix#1165] for more details
on what this can break.
this tiny fix just adds a call to `forceValue` inside `prim_break`,
but unfortunately this "breaks" a few existing tests because it
changes the call stack; those tests' golden outputs have been adjusted
without modifying their intended purpose.
Fixes#1165
[lix-project/lix#1165]: https://git.lix.systems/lix-project/lix/issues/1165
Change-Id: I5fe4ee3ff28b38aaf924125b8978130812e58fef
This job is used to indicate that all relevant Hydra jobs of an
architecture have built. The idea is to build some CD mechanism on
AFNix's Hydra to e.g. auto-update the nightly manual.
See https://hydra.afnix.fr/jobset/lix/demo for the current setup.
Change-Id: I41724c5884a068bbe41407ab30f8edf8e4914001
See [lix-project/lix#1156], but basically currently the `:st <n>`
debugger command doesn't allow any negative indices, and putting a plus
sign in front of the arg doesn't change anything; thus, we can exploit
that "design space" to allow users to move between different stack
frames easily, by simply prepending their arg with a +/- sign.
The actual behavior is little more nuanced when you account for errors:
as suggested by @pennae (thanks! :), when the user inputs an offset that
would result in an invalid frame index, the debugger instead clamps it
to the closest bound (i.e. 0 for negative offsets, $maxFrame for
positive ones) and just prints a warning.
Fixes#1156
[lix-project/lix#1156]: https://git.lix.systems/lix-project/lix/issues/1156
Change-Id: I02a0cdb6aaebbdb0515308880a3bf9c0d2fcd25e
The default constructor of Attr default constructs a Value, which is
itself deprecated. Therefore the default constructor of Attr must
itself be either deprecated in turn or removed. The default can't be
trivially deleted because Bindings::EMPTY depends on the default
constructor of Bindings which depends on Attr's default constructor, so
I'm settling for deprecating it for now.
Part of work towards #744
Change-Id: Ie34b08788780615c5478a0354122530b6a6a6964
This makes the removal vector a vector of Symbols instead of Attrs, and
uses a custom Compare to still be able to std::set_difference them.
std::ranges::set_difference **should** be the perfect function for this,
but because for some reason it spuriously requires
`std::indirectly_copyable<I2, Out>`, I can't use it here. This
defficiency has bee recognized before [here](https://github.com/cplusplus/papers/issues/1021),
but no one has driven the fix forward.
Part of #744
Change-Id: I6d2c016ea41e033bf38836f859541b506a6a6964
The manual for the builtins and nix.conf currently has inconsistent
indentation, which causes some of the descriptions to end up being
partially treatedas code blocks in markdown (and thus the manual).
This was simply caused by the template string for the docs having
too much indentation before the description is inserted, so this fixes
that 16-bytes mistake.
Change-Id: Ia264e3b1abb20430109029d07a2d2b0a1a726bd4
currently, the `:st <n>` command in the debugger will simply silently
fail if the argument cannot be converted to an integer, or if it falls
outside the range of valid stack indices. this isn't too big of problem,
but it can be nicer to tell the user something went wrong, rather than
not give them any output and having them guess (esp. in the second case).
this commit adds two errors, one for each case:
1. the argument is not actually an integer, or is outside INT_MIN/MAX
-> "argument '%arg' is not a valid integer"
2. the argument is an integer outside the range of stack traces
-> "stack index must be between 0 and %max_frame, but was %arg"
Change-Id: I8109feeede79a9ad3db9ee7dc95d37e7dd19741a
NixOS 25.05 which distributed lowdown < 1.4 has been EOL for a bit now,
25.11 ships lowdown 2.0.4.
Dropping support means we can tweak the lowdown options for terminal
output which have been added in 1.4.0 without having diverging behavior
in possible builds of Lix.
Change-Id: Icae97cf5e9e680766b8a6f4e85514f4c4625d1dd
lowdown 3.0.0 merged some flags into one to save on bits and did not add
any aliases for backward compatibility.
As with the changes for lowdown >= 1.4, we define a preprocessor flag to
gate the changes on and add a job to CI to ensure that lowdown < 3.0
keeps working (which is used by NixOS 25.11).
Unfortunately, we need to jump through some hoops to prevent nix and lix
from upstream Nixpkgs from being rebuilt due to a changed lowdown. Since
both implementation's packaging in Nixpkgs has their own package set /
fix point now, we can't simply inherit them from `prev` since they will
always be (re-)computed from the `final` fix point. As a consequence,
we need to expose our changed lowdown version at a non-default attribute
or break the builds of Nixpkgs derivations we test against.
Change-Id: I20a3e2fdaa05906f032ff66911c42867557fdd11
base classes
`ExprLiteral::v` needs to be initialized in ExprLiteral's constructor,
but at the time that it gets initialized subclass fields don't yet exist
so it can't reference them. Previously, `v` was default constructed and
then later assigned a proper value, but this creates a problem when
attempting to remove all default constructions of `Value` from the
codebase. This commit moves the backing fields of each of the subclasses
into new base classes, which exist just to make sure they get
initialized before the `ExprLiteral` base class.
Work towards #744
Change-Id: Ic6d24cab474460b113f2fbcc8d92ad266a6a6964
ahead of time
Rather than creating fixed size vectors of default constructed Values
before assigning to those elements, reserve the desired capacity and
then push created values onto the vector. This avoids default
constructing any Values.
Part of fixing #744
Change-Id: I36eff4275a893b181eaf3ce145b1ee446a6a6964
Progress towards #744
`alloc` default constructed a `Value` which is a problem because the
defaut constructor of `Value` is deprecated
Change-Id: I789cba20bd98728758395080a3a9cf6e6a6a6964
- launch-builder-linux.cc was missing an include for musl
and used function that are not defined in the launchers
- musl caches pids used for raise, breaking sandbox setup
- the mtls contrib plugin won't build, didn't try fixing,
static builds can't really use plugins reliably anyway.
Change-Id: I5ab1664e45ea977e5bcf05e41d825e6014e62146
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh
Change-Id: Ia7a63807681389cf5ccb9d07539b880697a88be5
This avoids a large list of CLs when a user can discover all the CLs by
browsing the chain.
Change-Id: I1890eb87e8bf6f930d3d00c3f7156690bb8de068
Signed-off-by: Raito Bezarius <raito@lix.systems>
This might do more copies than are necessary. I think that's fine.
Sorry, :reload not reloading has been bothering the shit out of us.
Fixes#1115.
Change-Id: I7f4caca3dad39696ce66ce57b1a520836a6a6964
So far, we had both `fj#` and `lix#` issue references in our release
notes, due to supporting both in the md files.
This commit unifies those in order for all references to be consistent.
Change-Id: I67c75784b460bfe4ebf214c2f6ac105cc60a7bad
kj can and will throw exceptions immediately without wrapping them as
promise results, and we must take care of these as well to not crash.
fixes#1150
Change-Id: Ifa29d0f3da4a6490f3e45d4c04a8fa49eaa20f43
mdbook-linkcheck was dropped from nixpkgs and replaced with mdbook-linkcheck2,
which is not entirely a drop-in replacement, but drop-in enough for us.
Change-Id: Iab03efab89ebae70811e0de3ea7cf0a9a059a9e4
Since commit ac64c727b5, during launch of the
builder it is attempted to raise all capabilities into the ambient set.
Specifically, what "all" means here is determined by the Linux API headers Lix
was built against.
Occasionally, new capabilities are added in the Linux kernel, leading to
PR_CAP_AMBIENT_RAISE failing with EINVAL on the newly added capabilities if the
API headers are from after the change but the running kernel is from before.
Similarly to how capset already silently ignores nonexistent capabilities, we
ignore this error so the builder doesn't fail to launch unnecessarily.
Unfortunately it is very hard to test for this situation currently, since the
last time a capability was added was CAP_CHECKPOINT_RESTORE in Linux 5.9, and
all kernel versions in nixpkgs are newer than that.
Change-Id: Ibeb2f0757729b877bd3ca9f02e1aa4536a6a6964
not doing this has lix crash if stdio forwarding fails for odd reasons,
such as ssh failing to set up a connection due to transient dns errors.
cf #1148
Change-Id: I543b7444236e69d3c13b47fb6c6eb37c65b96763
This is intended to be a softer version of
`allow-import-from-derivation`, allowing for users to only flag when IFD
is encountered rather than error out
Based-on: https://github.com/NixOS/nix/pull/13279
Co-authored-by: gustavderdrache <alex.ford@determinate.systems>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Change-Id: I879a444e3a24b4e82a8cea3a5a94834c78c3924a
`builtins.storePath` allows you to use "external" knowledge of present
store paths in your store to be used in your expressions.
This is sometimes used to avoid double wrapping store paths, for example
when dealing with nixpkgs sources (`builtins.storePath pkgs.path` is a
frequent pattern).
Unfortunately, using your external knowledge of present store paths is
technically impure because Flakes is about not depending on the *STATE*
of your store, no matter what you know about it or what you do to ensure
that your knowledge is accurate.
That being said, `builtins.storePath` is already possible in the
fragment of "pure Nixlang" via a clever use of `builtins.appendContext`.
Precisely:
```
storePath = path:
let path' = builtins.unsafeDiscardStringContext path;
in
# NOTE: merging the context set in all generality is impossible
because getContext on a pure path doesn't work.
builtins.appendContext path' { ${path'} = { path = true; }; };
```
works in pure contexts.
Because of this, we deem that storePath should be allowed in pure
contexts and users do not have to come up with a `storePath` polyfill.
Fixes#402.
Change-Id: I5fff750a4feed082745431835db451f945249cfb
Signed-off-by: Raito Bezarius <raito@lix.systems>
nix3 CLI never honored that setting.
When we update a nix3 profile element with new built store paths for
which we know the deriver (the derivation path which built it), we add
the derivation paths to the profile element to prevent garbage
collection the same way the nix2 CLI logic does.
Fixes#1095.
Change-Id: Icc236d174c5ce5ffe042d1c8e1968f6cb5b50359
Signed-off-by: Raito Bezarius <raito@lix.systems>
Instead of showing untrusted configuration items one by one, collect all
of them into one batched ask with 4 options:
- apply the configuration wholesale **for now** (yes)
- apply the configuration wholesale and trust it permanently (allow
permanently)
- selectively choose subset (no)
- rejecting the configuration wholesale (no to all)
Note that we do not offer options to reject permanently a configuration,
that's what `--no-accept-flake-config` is for.
Individual prompts gain a new option to distinguish "yes for now" and
"yes forever".
Fixes#682.
Change-Id: I2492717ac277af33ea4ac9b1a11bd7d6d1bfe0db
In messages printed by the evaluator, we use showType(.) to say things
like "expected X, got Y" where X, Y can be string, string with context,
etc.
After changes in the Lix data model (pointer tagging and friends), paths
got squished inside a internal string type.
We missed updating `showType` to take this fact into account which
caused an unfortunate:
`error: expected a string but found a string with context:
/nix/store/wjf8rdpp63rhyasbzc7zfms1agf7pwyd-source`
while using a path on `builtins.getContext`
A test is added.
Change-Id: Ifb0129ead2ea1973fc17dc0133f8a694995536b1
Signed-off-by: Raito Bezarius <raito@lix.systems>
This partitions a string into two parts based on the first occurrence of
the separator.
This function handles the case when the separator never occur in the
target string by returning an optional<T> in the second argument of the
pair.
Change-Id: I9ad44375dbaa137a4d21d2408bc2ffa34fab62a7
Signed-off-by: Raito Bezarius <raito@lix.systems>
Parsing and completing enumerations on the CLI was usually done by hand.
With this enum trait, the parsing and completion is automated away.
Change-Id: Icd49729564355d4c79e2c8050ce7e8b9d8e7eda3
Signed-off-by: Raito Bezarius <raito@lix.systems>
36c4d6f592 introduced it commented. Zero
explanation why.
horrors suggested to drop it entirely because it hide the options in
categories but they are pretty common and we should make as much visible
as possible.
Change-Id: I09894b5ba0f2b2cce5869136914289fed034fce3
Signed-off-by: Raito Bezarius <raito@lix.systems>
When `max-jobs = 0`, our async semaphores have null capacity and
therefore no slot token can be acquired.
In addition, a derivation with `preferLocalBuild = true` with `max-jobs
= 0` which is not a built-in derivation is not considered suitable for
being possible to build.
Our documentation in `max-jobs` writes:
> `0` is useful when using remote builders to prevent any local builds
(except for preferLocalBuild derivation attribute which executes locally
regardless).
A valid interpretation is that `preferLocalBuild` derivations (all of
them) will always execute even if `max-jobs = 0`.
This CL achieves this promise by enabling `preferLocalBuild = true`
derivations to draw in priority capacity from a `preferLocalBuild`-only
pool and then fall back to the build pool (`maxBuildJobs`).
If there's non-zero max-jobs, we make the `preferLocalBuild`-only pool
empty.
If there's zero max-jobs, we make the `preferLocalBuild`-only pool
non-empty (capacity: 1).
Remark 1: if `max-jobs = 0`, no non-`preferLocalBuild = true`
derivations can be built without a remote builder and `preferLocalBuild
= true` derivations can be built one at a time if `extra-local-jobs` is
not overridden.
Remark 2: if `max-jobs != 0`, no `preferLocalBuild = true` derivations
can bypass capacity limits, i.e. `max-jobs` unless `extra-local-jobs` is
overridden.
Fixes#855.
Change-Id: I00a0acac7375a881aec6281ebb13a4530b26e460
Signed-off-by: Raito Bezarius <raito@lix.systems>
This is going to be used in the extra-local-job future setting.
Change-Id: Ied76f32b8fa97cb71691d76167103001aa14fa01
Signed-off-by: Raito Bezarius <raito@lix.systems>
When ref/rev are very similar looking, e.g. a revision SHA1, the logic
to reconstruct whether we had a ref or rev initially cannot work.
This regression was introduced by
I2c72d5a32e406a7ca308e271730bd0af01c5d18b which passed the information
as an attribute field called `refOrRev`, instead, we pass `ref` or `rev`
but not `refOrRev`.
We apply the same post-processing logic to distinguish rev/ref in
path-style flake references: `github:owner/repo/$refOrRev`.
Fixes#520#841.
Change-Id: I2b1985cd5f351f43fea84232c1972715e2408fb1
Signed-off-by: Raito Bezarius <raito@lix.systems>
In structured attrs derivations, __json is used to serialize the `env`
attribute into a JSON object.
If the derivation enables structured attributes AND specify its own
`__json`, one of the field will take precedence (the reader can have fun
by guessing which one).
To reduce underspecification, we disallow `__json` in structured
attributes derivations.
Fixes#380.
Change-Id: I51c3b2af1ff9449471ba81d4c72df6a7a263eef7
Signed-off-by: Raito Bezarius <raito@lix.systems>
`nix.full_path()` is the *built* path, not the *installed* path. in the
sandbox this point to a location that contains *only* `nix`, not any of
the old-cli symlinks. any test that needs them in path would thus fail.
Change-Id: Ia651db8d1151a84f8d1b52e8993f6db61b0fe104
Without the `--git-dir` git will attempt to figure out which the git directory is on its own.
This causes issues when the `GIT_DIR` envvar is set in the environment for other reasons (such as being part of a `git rebase`).
Providing this parameter overrides the environment and thus makes sure the revs are always coming from the intended repository.
Fixes: #1135
Change-Id: Ibd43cfd8e0278463c0e1ccf86779982947a6038a
Signed-off-by: benaryorg <binary@benary.org>
To stop spurious warnings, we will stop this bizarre behavior of setting
empty values to impure environment variables.
On the warning side, we verify that the presence of the environment
variable.
The corresponding test is updated to "there is a warning" to "there is
no warning".
Change-Id: I12f5c6445ef00a83d269488d7aed0b0f61aeec12
All impure environment variables are initialized as part of `initEnv()`
to an empty value if they are unset.
Whenever `NIX_SSL_CERT_FILE` is set, this cause a spurious warning.
Let's document it here as a test before we fix it.
Change-Id: Iccba8ccb487d9cc25490e940ff09c1c5a13ae097
Signed-off-by: Raito Bezarius <raito@lix.systems>
CA tests have grown quite complicated and are hard to debug and
manoeuver for new needs.
They are a prime target for F2 migration and they were extremely
pleasant to write.
Most functionality is preserved or improved with additional behaviors
asserted on warnings and the full set of reports.
Change-Id: I70cd52abbe5ff13f56744b966ab00962ad3a49d6
Signed-off-by: Raito Bezarius <raito@lix.systems>
Instead of taking in a final argument `Value &` out parameter which it
writes to, it now returns its result
Change-Id: Iab6bc3a3ac6a4b17c6d31115a766a6ea6a6a6964
we already report errors that happen during reads via exceptions
rethrown from read. reporting errors during destruction too is a
source of confusion for users since these errors are not usually
relevant and just the decompressor thread reporting short reads.
shorts reads during shutdown are not an error, they're expected.
fixes#1138
Change-Id: I7983ef0ed7722460f1d683bc8171a454b3d06e90
I could instead add the right braces or manually ensure the
XMLOpenElement destructor is run at the right time. Or I could just
write what I actually want to happen.
Fixes#1137.
Change-Id: Ie1f157de1632571fb0c21e26d8e3cae56a6a6964
This is a rebase of CL 1626 after the performance problems that blocked
that change are no longer an issue.
Change-Id: I4a17f7606c5fdc9a924933a860eb401f6a6a6964
The pseudo-constructor `Value::mkFloat` would previously be called on an
default-constructed (under-initialized) `Value` to create a properly
initialized `Value` that represents a float. This change removes mkFloat
and constructs float `Value`s directly.
Change-Id: I48ae3a836842ce5e5eda6323404ba7576a6a6964
The pseudo-constructor `Value::mkStringMove` would previously be called
on a default-constructed (under-initialized) `Value` to create a
properly initialized `Value` that represents a string, reusing an
existing GC allocated Str. This change turns that method into a proper
constructor
Change-Id: I8d7a97e2afce231a04ecf56fdd6d22d06a6a6964
it sometimes happens that one forgets to add `.run().ok()` to a command,
leading to things not actually being tested, potentially even slipping
though code reviews!
This commit adds a check at fixture teardown time, checking that all
commands have been run. Sadly, due to when this check happens, we cannot
add a test checking for this behavior
Change-Id: If3df2c20c267b9f03ae0ca099b77c57e2310f335
this includes tests that fail teardown when *skipped*, which confuses
the tap parser by reporting more tests than advertised in the header.
not sure what to do about this though, and it is better than nothing.
Change-Id: I4fc6badacbfa3b3d5f5f4a4ad76b177379bb4a30
pytest-tap didn't get it right in combination with xdist. we need to
replace the reporter *completely*, which is just incredibly weird :/
Change-Id: I2e35b9dc8ea4b24563b9b3306312778781eb674e
The pseudo-constructor `Value::mkPath` would previously be called on an
default-constructed (under-initialized) `Value` to create a properly
initialized `Value` that represents a path. This change
removes mkPath and constructs path `Value`s directly.
Change-Id: I9021de1ff59490828f9fec6866e083996a6a6964
The pseudo-constructor `Value::mkAttrs` would previously be called on an
default-constructed (under-initialized) `Value` to create a properly
initialized `Value` that represents an attribute set. This change
removes mkAttrs and constructs attr `Value`s directly.
Change-Id: I11cd801eefecd454a7a5b6229d6770d06a6a6964
This is not ca-derivations. We don't care about self-references or any of the
rewriting or stuff like that; if you want something like that, write
your user code so it figures out where it is.
The reason I want to do this is for integrating other build systems with
Lix: if you're importing something built *outside the store* with a
non-Nix build system, it makes no sense to put it in an input-addressed
path since it's not possible to come up with a derivation for it.
But you need *some* hash, so the output hashing is the most sensible option.
It is also nice because of not needing trusted user privileges to import
it, since the hash describes the exact contents of the output path.
We use this feature (implemented externally to Lix; this CL eliminates
that step) at Mercury to integrate buck2 with Nix on the output side: we
import things to the Nix store as ca paths with references.
These can then be consumed by Nix language with:
```
# Hack from https://git.lix.systems/lix-project/lix/issues/402#issuecomment-5889
path:
builtins.appendContext path {
${path} = {
path = true;
};
}
```
Test plan:
```
$ nix store add-path --references-list xx-refs-list ./README.md
/nix/store/szcwnm13d9gmxx8fly0bz82l42jgysw6-README.md
$ cat xx-refs-list
/nix/store/g9hhwjbkdrw0wnbd3axfs4icyb37nr6b-bash-interactive-5.3p3
$ nix path-info --json /nix/store/szcwnm13d9gmxx8fly0bz82l42jgysw6-README.md | jq .
[
{
"ca": "fixed:r:sha256:1pj59fy7zcrn949ry9kgv4ba17h2sy3z2is469abj912cvbl14n3",
"narHash": "sha256-w5JA12YiJLlUMkRH8YfXAp6gFtlvJp8TSTazf7xLRd4=",
"narSize": 1384,
"path": "/nix/store/szcwnm13d9gmxx8fly0bz82l42jgysw6-README.md",
"references": [
"/nix/store/g9hhwjbkdrw0wnbd3axfs4icyb37nr6b-bash-interactive-5.3p3"
],
"registrationTime": 1771266146,
"valid": true
}
]
```
Change-Id: I005a03003dfc24108e018e599dbe0b5d6a6a6964
this requires some hacks to make error reports visible, but hey. it's
better than no progress reports at all, and successful runs no longer
spew huge amounts of useless log output onto the developer's terminal
Change-Id: I9e4766b2f825a4ec451e117eb4609db65f328785
Fixes path resolution issue with meson-embedded bash script when
`build` is a symlink, or a custom build directory is passed as a
argument to `just` or `meson`.
Change-Id: I67b13c524d196241bb9a8a3b8ad2c17c2c094ed8
not all operations that involve workers need to build. for example nix
copy --from calls instantiate thir worker with the *origin* store, not
the store they copy *into*. for binary cache stores this is not likely
to ever work if cgroups are enabled and the copy is run on normal user
credentials, even though no cgroups will ever be needed to copy things
fixes#1088
Change-Id: I065e29e1a1d0f58d81823609ef0701ff82cdd1d5
Affects `github`, `gitlab` and `sourcehut` fetchers.
Previously this was an assertion, which crashed the evaluator
with SIGABRT.
Fixes: #1133
Change-Id: Ia9bb8dd29ac8b9c97bf048827f62c5076a6a6964
overridden-ness can be reset, and is usually reset such that setting
value taken from a config file are not considered as overriden. when
launching builtin builders we *do* want to send config file settings
changes to the builder, so we'll need one more getter method for it.
Change-Id: I861538a469121c77ebc1898a276439e6b756797d
diverted stores are only necessary when the logical store paths of
objects matter for the test itself, such as for derivation hashes,
substitution from golden sample nars, or actual tests of the store
diversion functionality. all other tests can use undiverted stores
to run, especially since only linux can build in diverted a store.
Change-Id: I62f0907bdef9961609af22b610195fcec54c1e57
When the destructor of PathSubstitutionGoal is run, this happens in a
sync context and can cause starvation of all ongoing IO w.r.t. to other
substitutions, including our own substitution.
While there's only a decompressor thread per stream, the other side of
the IO runs on the event loop.
In order to fix this, it is sufficient to remove the thread indirection
and inline the async code.
Fixes#1126. Great thanks to horrors' patience.
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I3eb37bc37d156f0f5528364e568fdaa2ced58011
Signed-off-by: Raito Bezarius <raito@lix.systems>
Looks like too much stuff is breaking on this, so let's make it only a
warning for now to ease transition
Change-Id: I52d50ceb1fe1fbe9f4e28d0aabf3537f7e4d52d1
Explicitly catch common errors (trying to escape line breaks, badly
escaping interpolations) to provide better messages and user guidance
for these cases.
Change-Id: I3dd1b2ad3bca33be393e65be5e72f4fb9544a46a
StreamFdMessageReader reads message data *lazily*. if you don't access
all segments of the message before you close the underlying stream you
may find yourselv reading from something *very* different, and in this
case that something is The Void™. this causes reads to fail, writes on
the other side to fail to match, and finally our build launch to fail.
this does not lead to happy outcomes, so we will copy the full message
into a fresh new buffer *before* we try to access any of its contents.
fixes#1118
all-analysis-by: deprekated <kate@lix.systems>
Change-Id: I105540831fde855817194e9e539acf177f54a6f4
This is the long awaited refactor of the NixSettings.
It allows one to set, unset and update any and all settings with a neat
and easy-to-use interface
closes#846
Change-Id: Id4cfb5f853cc1168b506a1f6f405076f3a7cab65
if a substituter is entirely offline and cannot be queries at all we
should not be failing if other substituters are configured. likewise
if a substituter goes offline after querying but before we try using
it we should attempt to fetch that path from some other substituter.
ideally we'd treat all substituters as a single entity instead of as
one store each, then have that single entity take care of fallbacks,
retries, error reporting, etc. that requires larger rewrites though.
fixes#1061
Change-Id: I9d8fc0544ff380bf017256e8fcc82823dc634f10
we don't use std::cerr often enough to want this, and since cerr flushes
after every operation anyway it never did anything useful to begin with.
Change-Id: Ia54be340826da5073e9a1786c463555f4f0e491c
kj exceptions get wrapped as Error instead of passed through as is.
luckily this also means that we can add context to them very easily
Change-Id: Icedab6c016f4434447dd38ba14138c102fc6149a
oops. we have to use init* here, not get*. get returns a discarding
builder for something that was not inited previously, but the linux
cause worked anyway because the first member is created by default!
Change-Id: I40f8a12a04eef2f4e3a80d1537ac9b975490a027
this allows us great flexibility in how children are launched (since the
actual launching is done by a separate executable), makes fork no longer
needed in the core codebase (outside of runProgram, anyway), and we even
get to use linux vfork to its full potential to decrease the launch cost
of sandboxes to a constant factor (previously it was O(#drvs + #deps) of
the build graph, which obviously goes to n² quickly if you are unlucky.)
Change-Id: I66e2d1b20242dc24d708666ef325fb8725bd9296
we can have a ProcessGroup for a pid that *should* be a process group
but hasn't gotten around to setting its pgid yet. in such cases we do
want to be killing the thing anyway, not shoot into the void and hope
the right thing falls over. so far this has not been a problem due to
a mixture of just not having done this and being slow enough to work.
Change-Id: I4e0e54513252d8e18256b9286b819bfa957d70dc
if we move this out of libstore and don't want to pull in all of libutil
with its myriad side effects std::filesystem is our best option, by far.
most notably we don't replace pathExists because std::filesystem::exists
behaves like `stat()` on symlinks, not like the `lstat()` as we require.
Change-Id: I1e488418dcabb33f2ebb73d8c3d1b43528aa51f1
since we're using tmpDirInSandbox as the working directory for the new
process we also rename it accordingly. buildUser likewise turns into a
different Credentials type because exposing user lock state seems odd.
Change-Id: Id4a1a6eb733f774c893f373b91c2a271a4b84185
setting the signal is not enough, we must also check that the process we
expect to be parent to actually *is* our parent, not another process (eg
init if the daemon exited). we also have to set the death signal *after*
all set[ug]id calls, otherwise it will be cleared again by such changes.
Change-Id: I4e8c9102ea407576ed85b3203c8bb9bfb56762de
it's only used this once, and it accesses a (cached) global variable for
the filter. we want to move all sandbox setup state into a single object
soon, moving filter setup will make it a little bit easier to deal with.
Change-Id: I234d92d5ca044a16644b70bd303bfb7956c120f0
when starting builders we want the inner processes to run with a blank
slate. if some signals are masked for any reason the builder processes
may attempt to send signals to each other that are never delivered; we
avoid this by unmasking all singals. since every build tree also has a
session and process group of its own we are not in danger of sending a
signal to a builder by accident from any source, so unmasking is okay.
Change-Id: I90720ed2bd44502ffb6d2bb848c05369809abeba
we don't need to mess with this rlimit for e.g. the daemon. increasing
the limit later should always be safe since we don't allocate (or map)
much before constructing the eval states that ultimately fill our heap
and could thus make stack expansion impossible after some time passes.
Change-Id: Ieafda537fbc99a6a7f83a093a981e7df947da437
this will be needed in other places in the future, and splitting it out
also clarifies the surrounding code. the keep-caps dance launchPasta is
doing doesn't have to be moved since it is only needed to allow setuid.
Change-Id: I6baaa138c2b1bca9626971ed3266c1a971a63acc
the devshell no longer worked for us due to the recent deprecations :D
let's update pre-commit too because while it doesn't fail it does warn
Change-Id: Ic2060c82e4e97bb7a96cebd29097abefabdfe733
it's ancient, it does not do what it was added for (improving
determinism), and it probably hasn't been needed for anything
in a very long time. we can probably get away with killing it
Change-Id: I66f5c841964d9ad62726cb2dae8d6fc42095399f
mostly because this will make it easier to change the data source for
them, and especially to change to data source to something immutable.
Change-Id: Ic5f1a55d02d4c2cc928c3f8b308d07167efb467b
closing them as soon as we're done ensure that the fd table doesn't
balloon in size, which in turn lets us build larger graphs. it also
makes forking processes cheaper due to less fd table copy overhead.
Change-Id: I233d1b7358ffe1b05680f845cd99de38b179f449
In Nixpkgs, there are several strings like "\d\.\d" which attempt to be
a regex but are just literally "d.d". The escaping rules are silly and
we should warn our users about that.
Co-authored-by: Commentator2.0 <lix@crystal-cavern.systems>
Change-Id: I779b0757358fc9adc34dc140e1670b83abc93b67
The indentation stripping semantics of strings are fairly bad and have a
few gotchas where they behave unintuitively. But the good news is, that
these cases are easy to catch and can be avoided.
This commit adds a warning in the parser when such strings are detected.
Unfortunately Nixpkgs uses this kind of a lot, so we won't be able to
actually enable this warning for a while to come.
Co-authored-by: Commentator2.0 <lix@crystal-cavern.systems>
Change-Id: I3b3b68c2eee4cd70959d3f4ca643cb6caf3a2217
Unfortunately, the previous code only detected superficial __overrides
and did not catch cases like `rec { __overrides.foo = 2; }`.
Change-Id: Ic38bdef1a6a3bdea91915aef44447f4ecc238259
Changed the writing style of the descriptions, expanded with more
examples and rationale, and added the new timline metadata in the
frontmatter.
Change-Id: I218389e3504fc21f4eb45a927e77a41a1a70d4f5
The code within the code-generation had tons of code-duplicates and was
overall quite meh to read, understand and expand.
This commit refactors the code-generation to make it more readable and
comprehensible, while also unifying its usage a bit more
Change-Id: I3a5df8b7d8d9b6c76e02ef47dfb151c7dab7d0ab
lix, where modularity is made up and the patterns don't matter. merging
everything to match actual behavior of the system makes using rust much
easier because we'll have only *one* library to link, not half a dozen.
Change-Id: Ie3cb1db659798511128716402497697320bd0484
starting pasta as soon as we have all namespaces it must be in available
lets it start up while we finish creating the sandbox. this may speed up
sandbox launches somewhat, but likely not enough to show up in practice.
Change-Id: Id6724cbdc48b99284cd7ba7b56c98829d74557c5
mostly as a prototype for other process launching optimizations, but
also to move linux-only bits out of runProgram2. we want to stop the
fork+exec dance we do now and migrate to posix_spawn, but linux will
always need an amount special handling for launching build children.
Change-Id: Ied55dc0de95c05b52aaca923be2e3c1d101f47e0
we'll use this to run programs from a vforked child instead of using
runProgram. ideally we would like to have runProgram be as efficient
as possible, but our mount namespace handling makes this harder than
is worth dealing with right now. the linux derivation goal is a good
place to prototype functionality like this, especially since we need
fine control over credentials and file descriptor tables of children
Change-Id: Ibc356613ae10b06ea81de9575611948f5ee353b6
this was racy previously; a builder may have spawned processes before we
added it to a cgroup. these processes would not be registered correctly.
Change-Id: If60a64a798ee806d5701590a83d1015f55d7b867
this halves sandbox launch overhead, reducing the build time for 3000
trivial runCommand derivations on our machine from 80 seconds to 44s.
as a nice side effect we also get better error message propagation in
some cases, most notably setgroups failing when run in lix sandboxes.
Change-Id: Ia7c50a844915d9b8a20475b90b2d0179fd2fff34
creating user namespaces early means we don't need pipes to signal that
a namespace has been created and set up. we'll also create the netns at
the same time because they're privilege-bound to the userns, and having
both available will let us launch pasta earlier to have it ready sooner
Change-Id: Ie66fe2ab7f50a520fb81dd90944384e168d94476
permissions and ownership will be the same since the sandbox process
will not have changed uid/gid or umask by the time it gets around to
writing these files. all of these files contain some information the
host must provide, be it contents or configuration info of some kind
Change-Id: I0b57cb0699fd29e798ccf330dff99f571f3c0692
this way we can use it anywhere without fear of it being uninitialized.
only the linux platform bit uses this anyway, so we will move it there.
Change-Id: I35e207eec91daa8aa327c4f8b36c0dc8e703a7c4
`xucred` already contain `cr_ngroups` and `cr_groups`, we just have to
use it properly.
Change-Id: Ic29b4c551f9d93e86e9ed2bef20889d27d5438f4
Signed-off-by: Raito Bezarius <raito@lix.systems>
Two things happens here:
- we consider supplementary groups that are known for authentication,
fixes#968.
- we check supplementary groups if they are our build users group and
throw an error if they are (new behavior).
Nonetheless, we did not remove the ability for `matchUser` to recurse
and check for groups that the user may have but the connection might not
have communicated as part of their groups, i.e. if a process reduces its
list of groups via a call to setgroups, it will still be authorized.
This will come in another commit.
The authorization NixOS test has been extended with a store ping test
running in systemd with DynamicUser=true *AND* a supplementary group in
allowed-users.
Co-authored-by: Tom Hubrecht <github@mail.hubrecht.ovh>
Change-Id: I25b2b8304d66a04651cea523b5585a5d15ceebe8
Signed-off-by: Raito Bezarius <raito@lix.systems>
On Linux, SO_PEERGROUPS is an option introduced in 2017, that is, in
Linux 4.13.
Thankfully, Lix targets 5.10+ kernels minimum.
It was chosen to allocate 128 gids by default and ramp up (2x) as needed
rather than allocate a full 65k of integers as it seems wasteful. I bet
the time to the 16 allocations should incur at most an additional
millisecond on a modern system, don't quote me on that though.
This is preparation to enable ability for the daemon to vet based on
supplementary groups.
Related to #968.
Suggested-by: alois31
Suggested-by: eldritch horrors
Change-Id: I26d698327db5d174bf70ca25b0afede132bd9169
Signed-off-by: Raito Bezarius <raito@lix.systems>
Rather than a bunch of booleans, use optional<T> as a way to convey that
the identifier is known or not.
This is preparatory work for SO_PEERGROUP usage.
Change-Id: I0e3ce69e4103f9df5a2161c695bfc650123c1549
Signed-off-by: Raito Bezarius <raito@lix.systems>
we can already handle this case (by forbidding any ~/... paths in option
values), and since builtin builders are now run via exec we may not have
a home dir set, or even available, for them to resolve. loading a config
file in this situation will fail with an error even if the config is ok.
fixes#1109
Change-Id: I3491b31558871faa34766462747e3c15d1eccc64
xattrs are revealing some unfortunate brittleness in real world
derivations that are getting -EINVAL errors while executing their test
code.
The reason for this is that Linux checks [1] UID delegations for xattrs
writes.
In the Lix sandbox, except if you enabled a uid-range feature, you have
exactly 3 UIDs: root, nixbld and nobody.
If your test code makes use of UIDs which have not been delegated, you
will receive an EINVAL on your operation. Test code is not resilient
with respect to the number of available UIDs in their namespace.
To avoid further issues for end users who are running into spurious
derivation build failures, we forbid xattrs again for now.
For more information about the plans, please consult or chime in [2].
Fixes#1105. Reopens#838. Fixes#1103.
[1]: https://elixir.bootlin.com/linux/v6.18.6/source/fs/posix_acl.c#L257
[2]: https://zulip.lix.systems/#narrow/channel/9-Store/topic/disablement.20of.20xattrs/with/5295
Change-Id: I864066b34cd8319d2271fac1b179cb4f950d836e
Signed-off-by: Raito Bezarius <raito@lix.systems>
this will let us configure more than one socket to connect/bind to,
which in turn lets us use posix acls on sockets for access control.
we will also need something like this for the final rpc transition.
Change-Id: I9c39f14906e9bf809055ab5c94bf687745b4f69e
now that builtin builders are regular executables we no longer need to
treat them specially during sandbox launch itself, only while we build
the command line and environment for the sandboxed process. we are not
far from being able to extract platform-dependent sandbox launch code,
ideally moving all of it into (much more replaceable) libexec helpers.
Change-Id: I9b7041314683c56cd70eec9b1b4eae6de228883f
this means that builtinFetchurl runs in a real process now, and thus we
no longer need its workarounds for running in a forked process. forking
dropped the signal handler thread and broke the curl state via sharing,
neither of which happens any more now. we can run fetchurl builtins and
their actions straight from the main thread of our builder now, and the
temporary files and settings overrides we did are now also unnecessary.
Change-Id: I738171bc120ffcd541b7ff1424fed7924c2cdc1d
this will let us migrate the fork+do_stuff combo of builtin builders we
have today to fork+exec of this new command. we use a subcommand rather
than a simple libexec helper because we would like to avoid linking all
of lix into the simple libexec helpers if possible. this is only hookup
for the builders, none of the buiders are migrated here to ease review.
Change-Id: I9358f1f3fee6ca640c81a7bd865128ae0d0e35a5
call them with their individual parameters instead. this will make it
much easier to turn them into executables we can exec in the sandbox.
Change-Id: Ib49cf44715b2a480fecadabd21216c69cd730ef5
not having the optionals set when the builder runs (e.g. because FOD
hashes were not configured correctly) could cause assertion failures
in the builder process. while this should usually not happen we *do*
want to print a useful error message here instead of a crash report.
Change-Id: I81fb1d45fcbf660d0d9a7f0c0a12c38379ecd4c6
See: cl/4840
When importing Python modules, we include `functional2` in the module
path, like this:
from functional2.testlib.fixtures.env import ManagedEnv
This means that python expects to see a file like
`functional2/testlib/fixtures/env.py`. We run `pytest` from `tests/` in
the `justfile` and have `tests/functional2/__init__.py` so `pytest` in
`meson` is able to find these imports.
However, language servers generally consider the `pyproject.toml` to be
the project root, so (e.g.) `pyright` is unable to follow any of the
`functional2` imports, leading to lots of spurious errors.
In cl/4840 I moved `tests/functional2/pyproject.toml` to
`tests/pyproject.toml`, which worked but was considered aesthetically
unappealing.
This diff is much larger but it's a more elegant solution.
Change-Id: I2983c7b87f88f59a4e3521451a9f5acd6a6a6964
like diff hooks before the build hooks also use runProgram features that
aren't present in posix_spawn. just like diff hooks we do not expect the
build hook processes to be so fast that launch overhead matters somehow.
Change-Id: If3c33dbd7d2ac20c95886e06d24feda312946c78
use a wrapper type instead. whether something should be treated as a
single process or a group is a type decision more than a runtime one
Change-Id: I966e998c19e74be64ddd72ab11af809d001f61b5
diff hooks use uid/gid switch functionality that is otherwise only
needed for linux sandbox setup and unsupported by posix_spawn. not
doing these switches inside lix core code may let us move to using
posix_spawn for most process launching in the future, and for diff
hooks the added overhead of a wrapper program really does not hurt
at all. diff hooks are expected to be expensive in terms of output
size, process launch overhead is not likely to even be noticeable.
Change-Id: Ifa4b3eedef237632db3eb88d10e6469acae01f9e
it's kind of weird to not allow diff-hooks in single user mode unless
that user is root. maybe that's why we have no tests for them? we can
at least add a test that diff hooks are run at all when we expect it.
Change-Id: I54d623b5416acda1c205cc062b0f3a33c9f4aaa7
nothing except the linux sandbox actually *needs* this for correctness.
the linux sandbox only uses it to ensure that builder uids are freed up
when the sandbox is torn down, and even *then* it only works as we need
it to with PID namespaces enabled since the parent death signals is not
inherited across fork, but when pid1 of a pid namespace dies the kernel
also kills all namespace members. in all other cases this flag does not
help us that much because it actively prevents child processes cleaning
up after themselves, possibly leaving more trash around than otherwise.
Change-Id: I9ae0a9e91833a91d1011375dda402ac6c0a7ef6f
Adds an example plugin implementing an mTLS-enabled binary cache store
(https+mtls:// scheme) using client certificates for authentication.
Darwin fix: don't link liblix* into plugins (host resolves symbols at
runtime via dynamic_lookup). Explicitly link curl so it binds to
Nix-store libcurl, not /usr/lib/libcurl. This prevents the plugin's
curl_easy_setopt calls from operating on the wrong libcurl instance.
Test portability: BSD sed -i wrapper, OpenSSL -sha256 for cert signing,
redirect test server output to log file.
Change-Id: I652b987d3ac45e31df50ff4ba1f523294438c2b6
this really doesn't have to be here, it doesn't help very much. doing it
in the parent is cheap enough to not care and sandbox setup is not async
yet *anyway*, so we would not even notice if the old way was any faster.
Change-Id: I5a3a99af0fa5928e9a42f9c6589d98ff38b8c775
this will make it easier to turn build sandbox processes into helpers.
the point they were at was effectivly unsandboxed except for a few fds
that were redirected by commonExecveingChildInit, which only made logs
of any errors that much harder to convey from the child to the parent.
Change-Id: I67006eb33e1e13311bb8d14e6a0c3d5e6baf0c13
it doesn't do anything anywhere else, and personality setup is not a
critical operation (all it does is change which arch uname returns).
Change-Id: I39d7fcc4916e6882e49191d2294f9b7ee0dbbcd0
there's no need to do it in the fork, we're not changing security
domains before opening the pty anyway. we do change who owns this
pty device, but since we change it via chown we can just *not* do
that before we have an open fd to the pty. in practice this isn't
even necessary because the daemon runs as root, but if we ever do
run the daemon as an unprivileged user we'd need this to be split
Change-Id: I35264ab2954c7ba2c9c24c927366d64acada6772
this also requires moving namespace support information into Worker and
out of function-scope static variables, otherwise we can't use async IO
for the libexec helper output. we could set the fd to blocking for just
one CL and extract the Worker changes into another that the reverts the
blocking fd usage, but that seems not warranted for the scope of these.
Change-Id: I6996fab1ae74693d50cefb6a6a9c21d61dada1d9
Those functions have been recently refactored to return a formatted
`string` or take an `std::ostream` and only do the formatting have been
renamed to match what they're actually doing.
Change-Id: I3fe32fbe8723c2d93226370b8dd297f16a6a6964
this is mostly a test and example for the libexec helper infrastructure,
but it also lets us simplify pager launching until we we can more easily
handle executable-not-found errors the launch fallbacks would cause when
using runProgram2 instead of fork. ideally we'd use `posix_spawn` later.
fixes#1104
Change-Id: Ia33cc12e8a9d60ffad6f5c055bb1b8b596810e64
the new libexec directory is not available as a setting like other
directories (e.g. binDir) are since we consider libexec helpers to
be very internal. repointing them is like repointing a .so file we
dynamically link to; it can work, but needs much more preparation.
Change-Id: I40e64be0b32276f2864c0f2eb0b998d4c8ce7c88
we don't need the full sandbox setup helpers for this: mount namespaces
do not need to be kept, loggers are not needed, and redirections can be
done by runProgram2. once the build hook is removed we will not run the
communication bits from a different process anyway, this prepares that.
Change-Id: I95d28f7c2c25e43ccd82b448d270403ce4f28852
we do not need explict redirection management now, and dieWithParent
doesn't need an override either. we'd much prefer to kill ssh if the
process in charge exits; even ssh multiplexers are not fazed by this
Change-Id: I81e28b7605df73c887878ea4716228c7ad0f5c6f
the redirections the tests did were once not possible with runProgram2,
but they have been for a while now. we should use them instead of fork.
Change-Id: Ia422c8941ed04a9403dee68bfe938d68952253da
returning the pid as a pid_t instead of as a Pid raii wrapper is only
convenient for daemon use (where it'll eventually go away). using the
released pid correctly in other places is harder without the wrapper.
Change-Id: Ib42a2f357d2f0849beabd015b321bfff31334eda
Fixes#323
Let's now all go an a little rant about spaghetti code...
The result of this code is that the missing paths are not printed
anymore. The basic issue was that the parent class of this command is
StorePathsCommand, which inherits from BuiltPathsCommand, and their
purpose is to work on path that are clearly in the store, building them
if needed (and ofc telling the user about what's missing), the sequance
of calls is:
- BuiltPathsCommand::run(ref<Store> store, Installables && installables)
- Installable::toBuiltPaths( *getEvaluator()->begin(aio()),
getEvalStore(), store, realiseMode, operateOn, installables);
where operateOn is Output by default, realiseMode is Derivation, so
the only thing that can be built are the derivations for the required
installables
- Installable::build(state, evalStore, store, mode, installables)
- Installable::build2(state, evalStore, store, mode, installables,
bMode)
And that final call has the following:
```
switch (mode) {
case Realise::Nothing:
case Realise::Derivation:
state.aio.blockOn(printMissing(store, pathsToBuild, lvlError));
```
So there were two options, hack a new spaghetti in the existing
spaghetti code, or condense all those calls that are actually useless in
our case because they mostly transform a list of installables into a map
from installables to their BuiltPath which are then iterated to retrieve
the final outputs, whereas it is possible to directly get the required
paths in a much more efficient manner and without printing unrequired
stuff through a multitude of intertwined function calls by simply
replacing one method that was previously inherited from the grandparent
class
Change-Id: I1d2baaef5a099cd98b63b5346f2613914c6cd2ac
it's no longer needed now that withPager exists. also fix a
logger-never-resumed bug that never showed up in the world.
Change-Id: I7311a50896f5291364320ec4c88506dc3bb11d4c
RunPager is weird and confusing in that it replaces what stdout *is*
depending on environmental conditions. this has not caused problems,
but it's easy to imagine situations in which it would (eg if the stl
decided to capture the stdout fd by duplicating it). using a wrapper
for this also makes clear *what* actually goes into the pager; while
the previous contract was semi-reasonable it was also very implicit,
and with the proliferation of functions we had that printed directly
to stdout it would have been easy to send wrong output to the pager.
using a wrapper also makes process management much easier because we
do not have to rely on destructors to always produce correct output.
Change-Id: Ifd3760940af1ec719fe856158c913cbb9a5bf270
this has been unused since ab3ce1cc13,
which is now almost ten years ago. surely it was just forgotten then
Change-Id: Ib93b52d4ce5f19629432c5f91ff426582c57f99b
cancelling a get() promise could leave the pool with allegedly in use
resource handles that do not actually exist. this would cause asserts
in ~Pool to fire (and crash the entire program due to bad accounting)
probably fixes#1041
Change-Id: Ic61ab9b658f02f71e8be7577b8a1115600846f86
merge all pkgconfig-found libraries into a single one and redirect all
existing lib files to the merged variant. the clean separation between
libraries is largely fictional anyway because the main config block is
in libstore and cannot reasonably be moved out, or because libutil has
assumptions built into its behavior that libmain later establishes, or
because libexpr messes with global boehmgc configuration. nearly every
kind of layering violation imaginable can be found *some*where in lix,
and not pretending that this is not the case will let us clean up some
of them. this will also allow us to merge these libraries at the meson
level which is itself necessary to make rust code easier to integrate.
Change-Id: I172f8a08d5d90ded0a0a99fd9e0fa5d804db00a7
xattrs scrubbing was gated behind Linux platforms, but Lix can safely
assume that POSIX APIs are available.
macOS implements these APIs modulo some creative ideas.
Fixes#1008.
Change-Id: I1566df4923bf24092d3fd32ccf6a96e4256e87b2
Signed-off-by: Raito Bezarius <raito@lix.systems>
Fixes#838.
xattrs were historically disabled in 2017, since then, NFS v4.2 gained
support for xattrs and almost all filesystems support xattrs. In
addition, moving the build directory to a xattrs-supporting filesystem
is always an option. Programs may exploit xattrs (including ACLs) during
their build process and testing phase, to better serve these programs,
we will remove this limitation.
Lix will use POSIX ACLs for its UNIX domain sockets in the future and be
one of these programs which will run tests making use of xattrs, while
not writing any xattrs in the derivation outputs themselves.
xattrs are still scrubbed from derivation outputs so it is not possible
to obtain an advantage by writing a security-related xattr to a well
chosen file.
Tests were added to test these scenarios on Linux. Darwin is carefully
excluded because of #1008 and #1090, solving that is welcome. In the
meantime, they are marked xfail.
Change-Id: Ia3255eeb8442e83db4f10dcb5a51cbc368a2550d
Signed-off-by: Raito Bezarius <raito@lix.systems>
one would expect the lang tests to be selectable (using `-k`) using
`eval_okay` `eval_fail` etc though this was not the case so far.
This commit renames the functions to reflect the expected names
Change-Id: I4cd340c093d774feeb4d96820d28e49c665b159f
I don't know how this was missed or no one really complained about it,
but perl-bindings was using CppNix if nix was not overlaid to our own
Lix.
If you overlay Lix manually, the resulting Lix's perl-bindings cannot
really refer to Lix itself, therefore, it's CppNix.
Change-Id: Ie5ce95564cb7fb5444acbea6124619c5121e1576
Signed-off-by: Raito Bezarius <raito@lix.systems>
during the value rewrite we accidentally broke extension of incomplete
primop application. this only shows up when binding on incomplete call
to a primop to a name, binding an incomplete call to *that* to another
name, and then finally calling the second binding with enough args for
a complete primop application. since this only shows up when calling a
primop with three or more args it took a while to surface. we have few
builtins that match this: foldl', replaceStrings, and substring. these
are not used incompletely in this manner very often, so this lingered.
fixes#1102
Change-Id: I218dffc14ae876efc86a86c7eb6c895e2405201c
these ended up referring to different directories and would break during
source reorganization (or even just including them in different places).
Change-Id: I91e8db04e3d3565efd5920982ed5bc59d2c27058
RegisterCommand was renamed in b30556f8f1
and de-static-ified in 4d9fe67430. adding
new settings was never possible with static instances since settings do
need an existing Config instance during construction; GlobalConfig must
be used to add settings. it is not possible to change existing configs.
Change-Id: I26af8abc375ba5883fce060ffefc710d6dfbca4c
Since lix-doc is no longer a separate library, the pkg-config files were broken
for static builds. Adjust them to refer to the new monocrate library instead.
Actually, since the monocrate library is static in all cases it needs to be
linked explicitly even when everything else is linked dynamically. I assume
this only doesn't break the dynamic nix-eval-jobs because it doesn't actually
use anything from it.
Change-Id: Icf29938688f4fa87425ff346a3105ef56a6a6964
Use a span representing the underlying hash, this allows comparing based
on those spans. Other minor tweaks to the header file.
Change-Id: Ie05960439e60500109083c06d347d6cdf41bfda8
When --check is ran with no output, this is seen as having *SOME*
invalid outputs, actually, ALL of them are invalid here.
Instead of reporting a confusing error, let's consider ALL of outputs
being invalid to be no output at all available and advise the user to
rerun the build without --check.
If there's *some* (>0) invalid outputs, let's print them exactly to give
a chance to the user to remediate this.
Fixes#485.
Change-Id: I00955ef9ea4f129e2c98d68c73b1e981f90278a0
Signed-off-by: Raito Bezarius <raito@lix.systems>
Yeah uh, I don't think `$shell` is ever set in the functional1
environment. Pretty sure this has been expanding to `''` THIS WHOLE
TIME.
Also `SHELL` is `/sbin/nologin` in that builder. idk why so I'm just
passing `shell` unconditionally in functional's `mkDerivation` helper.
Maybe next we should put `set -euo pipefail` into all these test
builders?
Change-Id: Ie5fb85305932c03a901ba84ffcce48f16a6a6964
macos-builder02 in particular has been having problems for a while now
that no one could reproduce. We believe we have finally found the cause:
missing shebangs, not just in the tests themselves but *also* in the
inline scripts `nix-shell` itself writes.
Fixes#1042.
I believe this will also fix#1093.
See also: https://github.com/NixOS/nix/pull/14778
Change-Id: I54d04a770b8e78484815db88a8bc88776a6a6964
When `:load`-ing a file that doesn't result in a successful outcome, the
"loaded" file is removed again from the list of loaded files.
This avoids having it be used as part of `:reload` which will probably
not result in a positive outcome.
Instead, the user is expected to retry `:load` before issuing a
`:reload`.
Fixes#50.
Change-Id: I85d025cf39f79e8e02117628bffa20c81917dc3f
Signed-off-by: Raito Bezarius <raito@lix.systems>
If `:reload` is used with no file loaded, the environment is completely
cleared.
That's a surprising behavior and one of the root cause behind #50.
Environment clear is only performed if there's files to reload.
Change-Id: Icb014f490a83aaa4d758dbf37822db16d5bb65c1
Signed-off-by: Raito Bezarius <raito@lix.systems>
Only successful loads counts towards the list of loaded flake
references.
Change-Id: I04128f90e57ef8352183d72dc6def7a09d52173a
Signed-off-by: Raito Bezarius <raito@lix.systems>
libutil config.{hh,cc} and libstore/globals.{hh,cc} contain so many
interrelated things. Hopefully we can separate them out a little bit for
clarity.
Change-Id: Ib78fc46fe305d13aada8886e2e168d446a6a6964
This makes that the default for all builds... which I'm not sure is the
right option, but it is also very cheap to do so, and adding an entire
other Lix build just for static libraries (not to be confused with
static Lix) would be prohibitively expensive.
Change-Id: Id173c77cfa62886724378ca599d21c8c6a6a6964
`build_env` is assumed to be a certain trivial environment, e.g. a
singleton search path.
An environment in general is either empty, contain one or more search
paths separated by a colon.
It seems that the intent was to set the `build_shell` first as a
prepended path in the managed environment then extend via the
`build_env` parts.
This fixes a usecase when the `BUILD_TEST_ENV` is non-trivial.
Change-Id: If5b8ab976d867a57ce0b8d29255f64695e30a8b2
Signed-off-by: Raito Bezarius <raito@lix.systems>
In an attempt to slowly split up and clean up `eval.cc`
The `Expr::eval` functions contain the most important logic when needing
to touch something on the evaluator, and having them spread across the
3kloc eval.cc file was really annoying. This should help, though more
cleanups are yet to come
Change-Id: I2c2d15a024ab1ec3fc5591f628d6768341a1c8ab
This adds a way to control the deprecated features that need to be
enable while evaluating the specific nixpkgs version
Change-Id: Ia2b9a29f1966d4d5f32239a36ba8270330007da9
This avoids rebuilding Lix each time we need to modify the lib tests
(as flake.nix stays unchanged), and adds a way to control the deprecated
features that need to be enable while evaluating the nixpkgs lib
testsuite
Change-Id: I2baa1cf90effcaf9d53337c6a7a5b24d6297178c
Certain evaluation requires to pass impurity arguments, add `flags`
to add more arguments to the instantiation.
Change-Id: I538ca2ff12c07cc66e845b42f81baa87f11b8f0f
Signed-off-by: Raito Bezarius <raito@lix.systems>
Is there a situation where `home` can be None? This hinders the ability
to use it quickly without type gating in the test code itself.
Change-Id: Idd739ddad67e10d6f7a6aad283c0e3b7cd08106a
Signed-off-by: Raito Bezarius <raito@lix.systems>
Otherwise, this result in debug traces with a \n cutting the list of
controllers.
Change-Id: I8aba6379d335c4edab9d9ddb3fb411813d8dc426
Signed-off-by: Raito Bezarius <raito@lix.systems>
The prior use of the UID is uninteresting in general for consumers who
wants to learn things about _what_ the build is, not what is the UID of
the build user running it.
Achieving this with a place where a mapping between cgroup names and
build requests are maintained is non trivial because there's no "global"
daemon that keeps track of this information and such a daemon would
require a persistent location to track this.
Rather than solving that, we will just encode the derivation hash inside
the cgroup name. Userspace can simply pull this apart and query Nix to
obtain the original corresponding derivation for more information.
This provides userspace with a way to map cgroup names to pnames for
example. Going further, if you have a map between attribute paths and
derivation hashes, you can power a database of cgroup metrics per
attribute paths by combining all of that.
In addition to this rename, we adopt a systemd-ish convention for
template units (`@`).
As a result, userspace tooling that will scrape cgroup metrics will
discover a consistent path as long as they filter out the UID part,
especially in context of Id6c458aad30eaa08c3609ac8280a7dde8e8f3cf9
change.
This problem is tracked under #1073.
Change-Id: I238d0568a3e4b1ff3057781c0639528d666b4d37
Signed-off-by: Raito Bezarius <raito@lix.systems>
nlohmann has customization points for exception throws. we can use these
instead and wrap json exceptions at the source instead of playing a game
of whack-a-mole with json errors all over the tree. since nlohmann needs
macros set to achieve this we can no longer precompile its headers *and*
must forbid including it anywhere without the proper defines or ordering
of include directives will break lix with ODR violations, if we see them
at all before we get another json-related bug report. a new lint will be
in charge of ensuring this doesn't happen. we also re-allow direct calls
to nlohmann json parsing since error handling is no longer a problem. we
will keep the wrapper for more convenient error context handling though.
fixes#1092
Change-Id: I54ecc14f5bec5e2177729b41c3703216e76cc6a3
It's 2026 and the command handler was entirely manual!
We introduce a basic command handler framework for the REPL that handles
all the previous (implicit) features reasonably well.
This also builds up the infrastructure for nicer features.
Most important feature is that now we can let external plugins adds
commands to the REPL.
There's slight REPL changes:
- errors for debugger commands are more explicit.
- short-hand have a long-hand version.
- help shows command sorted by std::map natural key sorting (based on aliases)
while keeping the special syntaxes for the REPL language at the top.
- sections were introduced for Flakes commands.
- :st was merged into one command with an optional argument marker.
Change-Id: Ibd24f994491bf5d45d4b218fd84c0944ae5050fd
Signed-off-by: Raito Bezarius <raito@lix.systems>
Status quo: We have `force$Type` and `eval$Type`, both which first
produce a value and then do a type check. The type checking logic is not
consistently implemented, with lots of code duplication.
This change does:
- Introduce new `check*` functions which unify the logic (the error
handling unfortunately still needs some duplication for now)
- Make both `force*` and `eval*` use the `check*` function for the
actual type checking
- Inline and dismantle the `eval*` functions for being of little use and
little used. This makes the `ExprOp*::eval` implementations for binary
logic operators more verbose, but IMO that's a good thing: The
implementation now needs to be a lot more explicit about the
short-circuiting semantics, something which was previously hidden behind
the short-circuiting semantics of the C++ language, in a way that could
easily be overlooked and lead to confusion, which is something that
happened to me twice in a year.
- Changes `forceAttrs` and `forceList` to include the context in case
`forceValue` fails (compared to only when the type check fails). This
was done for code consistency, because I could not find any reason why
list and attrs had different semantics here than int, float and bool. So
far the visible change is minimal (see the diff on the err.exp), however
this needs vetting for potential performance regressions.
Change-Id: I33e5c706d46850c9e1126293ee01dab85ba07587
That struct has been annoying me since I've encountered it. There is no
reason for having to track all these symbols out of band in the state.
Change-Id: I5c2c0d8174af0a51f9b456cc3651d8203a406d09
The names were prefixed with `sym_` to allow faithfully representing
names that start with __. (Previously the underscores were omitted, at
the cost of readability for people less familiar with the intricate
details of Nix. Is `toString` literally `toString` or is it
`__toString`? etc.) As a bonus, this allows to easily grep for any
symbol through the entire code base to find where it is used. I've done
exactly that and grouped and annotated the symbols based on their usage.
Change-Id: I230fecd9b0b18e0984ac8111c1e6381f45c5abf2
I was pretty confused by this when I saw it, not realizing that `build`
meant `tests/functional2/build` and not the top-level Meson `build`
directory.
Change-Id: I4865f196e5bf029419c6b488e78cbfb46a6a6964
100 causes too much diff churn w.r.t. current code style
"120 cols would be nuisance for us because our two-column layout is
exactly 118 wide (not counting the line number margin)"
"Ours is exactly 118 too"
Change-Id: Ifc7f21003e1011a17a1d799afaee775aeb927fd5
This logic is used in the various build-related REPL commands and is
factored out to make it easier to write the handlers for each
build-related REPL commands.
Change-Id: Iaa18df489db75495b12924e9a76b3fff1975eb64
Signed-off-by: Raito Bezarius <raito@lix.systems>
The completions for debug commands is purposefully left out because it
is going to go away in the next changes.
Change-Id: Ibc78b7587de3f15857bb09a3a0732334693ad550
Signed-off-by: Raito Bezarius <raito@lix.systems>
cl/4796 introduced a bug where running `just test-functional2` would
hang forever on macOS but `meson test -C build --suite installcheck
functional2` would succeed because of a mismatch between the
`$BUILD_TEST_SHELL` environment variable and the `-Dbuild-test-shell`
build option set in `mesonFlags`.
We can move these definitions to `finalAttrs.buildTestShell` and similar
and use those values in _both_ `mesonFlags` and `env` in order to make
them consistent in all cases.
See: https://gerrit.lix.systems/c/lix/+/4796/comments/50def5b4_fa5671f2
Change-Id: If300bbe46d6269ace29b44156fb4a5196a6a6964
...and add both a static bash and busybox into the sandbox for building:
$ /nix/store/fz43jxs5qfg5vldzk38y2hmrcl58qk18-busybox-1.36.1/bin/sh -c 'declare -A foo'
/nix/store/fz43jxs5qfg5vldzk38y2hmrcl58qk18-busybox-1.36.1/bin/sh: declare: not found
The reason is that busybox's `sh` doesn't know about `declare` which
means it cannot build derivations with `__structuredAttrs = true;`.
Change-Id: Ie3ca431f862fd3e59dc649582704ae739ee5834a
Nearly each tests gets a fresh store (in the test root residing under TMPDIR),
which with default settings means 8 MiB of reserved space to be deleted by the
user in case of space shortages, which of course for a test suite is just
wasted. Stop doing that to considerably reduce the litter (experimentally, from
3.7 GiB to 67 MiB per run). The old functional test suite already did the same.
Change-Id: I912cdda6e796ea37cfee0b8fe0478e976a6a6964
this is only needed on macos, and only for tests that build anything. on
linux it can actually *break* stuff due to unfortunate interactions with
store url parsing, which is required to work for any remote build tests.
Change-Id: Ic6f7e090f15e129fc365d7edc56cdbc1a5686047
apparently this file slipped through the code review (the actual test
does exist, no worries) but this empty file and init py exist, sitting
here empty. This commit drops them as they serve no purpose
see commit 1b7ad3a7ad
Change-Id: I64c4e03cd3ceedf9e430ae014127966bd8e4269d
This avoids any complications that can arise from the environment
affecting evaluation of the help pages (which don't need to be calling
out to anything external anyways)
I don't think this is actively causing problems now - though it did in
Nix! - so this is largely preventative, but I don't really see a reason
why this was ever running in impure mode in the first place either
Based-on: https://github.com/NixOS/nix/pull/14086
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
Change-Id: Ice0c760562d70c09934249edc3b767473516d372
- test_check used an old version of an error message
- test_good2 did not build an unspecified dependency
- test_attribute_selection used a fixed drv path. we
can't keep this with the current setup because the
drv `system` makes the drv hash platform-dependent
fixes#1081
Co-Authored-By: kloenk <me@kloenk.dev>
Change-Id: Ia009740f8b5432e83467ea451a5ff53d5e141b3f
the cgroups experimental feature does not work properly without this
because we do not stop subdaemons when the main daemon is shut down.
systemd needs the assigned cgroups to be empty to restart the daemon
and thus cannot cleanly restart the daemon if any connections exist.
starting a fresh unit for each connection creates a new cgroup every
time instead of sharing any delegations and thus solves the problem.
fixes#1030
Change-Id: Id6c458aad30eaa08c3609ac8280a7dde8e8f3cf9
A friend reported that this is rebuilding, which doesn't make any sense
because nixpkgs also has a lix from nightly that surely has the same
dependency and thus it surely should be built on hydra, right?
Turns out no, since they were overriding the requiredSystemFeatures to
remove big-parallel. Silly, but we can match that and get cache hits.
Change-Id: I4b1c8e850c6c38b03881354e978f41a10592ec95
This bump nixpkgs input to 25.11 and fix the two warnings that were
coming with the update:
1. `pkgs.system` => `pkgs.stdenv.hostPlatform.system`
2. `runCommandNoCC` => `runCommand`
also:
1. remove cppnix 2.3 compat tests (cppnix 2.3 is officially dead now)
2. remove lowdown 1.3 compat tests (nixpkgs no longer carries it)
Co-authored-by: Tom Hubrecht <github@mail.hubrecht.ovh>
Change-Id: I78526b5d8992a6c63ecd7f0c7c1fa6346a6a6964
It only debatably belongs in libutil since libutil doesn't know what a
progress bar is but whatever.
It was either that or libstore. Since I'm about to make it a setting.
Change-Id: Iba59b0a190087b316ff3191a0ddd9ec36a6a6964
make the global_assets folder more readable by placing asset pack files
within a dedicated folder instead of building up a mess similar to f1
Change-Id: Ia2c16f38eb6da96e1e73584bd91391ee56acb410
This will probably get the implementation of the fixtures revised when
we land the new extraction code, but we are setting it up to be generic
against that.
The operator-> thing is kind of a crime. But it also makes the code
vastly more readable so it's impossible to say if it's bad or not.
Change-Id: Ia5aca69cefaa03cd533ad19d20d856ff7e76a546
Instead of manually reading the ref file,
which gives the tag object hash when ref is a tag,
shell out to git.
The logic for finding the ref file is left for cache TTL tracking.
Fixes: #1070
Change-Id: I490b1e62f83cf602c56232c6081a52166a6a6964
Those are new warnings that come with a more up-to-date llvm
Fixes#1066
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I305d0d810d12a5e8d31c7d89e0cdb3a82df71556
As far as I can tell, there's no harm to let know a nix-shell invocation
about NIX_LOG_FD being stderr.
Fixes#336.
Change-Id: Ifdb3591813251d4bc481158d28dc18e1489de72c
Signed-off-by: Raito Bezarius <raito@lix.systems>
As far as I can tell, there's no harm to let stdenv know that they can
write to file descriptor 2 which is stderr inside of a nix3-develop
shell.
Contributes towards #336.
Change-Id: I51dcbcbc19a1698a0d1255f9d943d9ebd15d115d
Signed-off-by: Raito Bezarius <raito@lix.systems>
Closes#1064
The culprit here is that `genGraphString` is only invoked with the
store-paths associated with the outputs of the derivation, so when
filling `dependents`, the `graph_data.find(p)` call would return the end
of the iterator when doing this for references to other store-paths.
As a result, the code wrote information behind the graph data-structure
causing a corruption. For me, this resulted in a SIGSEGV most of the
time and in a few cases in an uncaught `map::at`-exception as reported
by Niklas.
This patch changes two aspects of the original implementation:
* When filling `dependents` in the node-set, use `map.at()` instead of
`map.find()->second`. The latter doesn't make any sense and was the
cause of corrupting memory. The `at` would've made it far easier to
spot this in the first place.
* Filter out store-paths that don't belong to a different output of the
derivation when creating `outputGraph`. This variable is used on two
places, `genGraphString` and for topological sorting.
The latter already filters out store-paths from a different drv, so
this is happening now when creating the variable in the first place
such that `genGraphString` never ends up with corrupt data in the
first place. This is the actual bugfix.
Implemented a regression-test for this case to be sure.
Change-Id: Ie02144d89c32b0a776cb1ece0601d0229315ebc3
we'll need this to modify argv for socket-activated daemons. this is our
replacement for the old savedArgv mechanism that was unscoped and fucky.
Change-Id: Ie048eb8ea99f1c9cd627a051292c836c83197068
the parent daemon does not change any settings before starting a child,
so there's nothing we may want to change that is not already set by the
config file. this also doesn't prevent changes of the config file being
applied to daemons where we do not expect it since it'll only restore a
setting to the parents' value if the child also has an override for it.
Change-Id: Ic5a9ef13458c103ec9979cb187ba8d3ce5e1e719
Lix changed to C++23 in 4ea8c9d61¹, but the perl subproject wasn't
updated with it. This commit corrects that.
[1]: 4ea8c9d643
Change-Id: I5464138d206d87cf25762720b147487d6a6a6964
We offer the three usual options: verify, repair then delete and direct
people to report us corruption so we can fix if this is induced by us.
Fixes#447.
Change-Id: I0df61769d732d227333c206f312857c0593f7bce
Signed-off-by: Raito Bezarius <raito@lix.systems>
If the derivation does not start with D, do not return a simple
"expected string 'D'" error but a full error message.
This contributes towards #447.
Change-Id: Iee05f3918e4cc43e79f244ab2fd64a52cf2bb6d2
Signed-off-by: Raito Bezarius <raito@lix.systems>
Prior to I6a6a6964d2b5ad47ae5ea9eb11af9b6373ce2141 — `sudo nix
upgrade-nix` would perform direct store access.
This ensured a certain number of desireable properties for upgrading the
Lix binary itself.
We re-introduce direct store access for upgrading Lix binaries.
Fixes#1060.
Change-Id: I523c4d3023ed5fe9eff8fde9a266c56a0de47d8c
Signed-off-by: Raito Bezarius <raito@lix.systems>
Alternative to cl/4661 discussed in #1044.
It can be assumed that `$tmpdir/build-top` can be created safely without
any risk, this way, we don't need to reuse the random directory creation
primitive.
Fixes#1044.
Change-Id: Iec52477f3047fc40959b183c607312d5a40fc8c9
Signed-off-by: Raito Bezarius <raito@lix.systems>
Otherwise, loading Perl bindings fails early with[1]
undefined symbol: Perl_pad_sv at /nix/store/h2jsb5i4yfblr2f3ac2c7zpmlmj7zjym-perl-5.40.0/lib/perl5/5.40.0/XSLoader.pm line 94
Apparently, it's expected behavior by Perl that this symbol only exists
with `DEBUGGING` being set, hence it's used by the headers. However,
`pkgs.perl` from nixpkgs is apparently not built with `-DDEBUGGING`
causing this error.
Now, `NDEBUG` is manually unset after loading the Perl
headers rather than setting `DEBUGGING` causing the error mentioned
above.
I confirmed that this not only fixes the problem described above, but
running the Hydra tests with
diff --git a/perl/lib/Nix/Store.xs b/perl/lib/Nix/Store.xs
index dfdd64d28..14788266c 100644
--- a/perl/lib/Nix/Store.xs
+++ b/perl/lib/Nix/Store.xs
@@ -27,6 +27,7 @@ using namespace nix;
static AsyncIoRoot & aio()
{
+assert(false);
static thread_local AsyncIoRoot root;
return root;
}
still results in assertion errors.
Finally, added a small install-check that importing `Nix::Store` works
fine.
[1] https://git.lix.systems/lix-project/hydra/issues/69
Change-Id: I58521777eb0f94b766a9813aa4bbd06f9052bd35
As discussed in the F2 matrix channel, not being able to put the error
message inside of the thrown exception just results in one assigning an
additional variable to immediatly throw it. The original reason for
these rules to exist are that the message isn't printed twice, though
the line above the thrown exception is printed anyways in the
stacktrace, so we still get the error message twice but now with
additional work.
Hence this commit removes the said rules
Change-Id: I0b37c0b0861334703a5772c36353d31441f19dcd
So far, our ruff config was confined to the f2 package. This meant, that
when one added additional paths to the ruff formatter, those wouldn't
get the same rules applied as f2, resulting in inconsistent styling
thoughout the project.
Due to how configs are resolved, only the "closesed" pyproject toml is
considered. This means, we need to tell f2 to extend its configuration
with the base level one. Though no change is required for other parts of
the project, as long as they don't have their own pyproject.toml
Change-Id: I145c764e7b850194020b5560e1025f4aa80411ae
Nix daemon runs in a service context, usually with systemd.
If the Nix daemon unit adds `[Service] CacheDirectory=nix-daemon`, it should
successfully use `/var/cache/nix-daemon` automatically.
Instead, it uses `/root/.cache` right now, which is really bad.
By default, we add `[Service] CacheDirectory=nix` now which means that
caches are moved into `/var/cache/nix`.
Fixes#634.
Change-Id: I854b1045bfdce8f60110aea70bda1bf6657dfd7b
Signed-off-by: Raito Bezarius <raito@lix.systems>
Recently, in 54df89f601, support for mdbook 0.5
was introduced, including some logic to handle the `sections` -> `items`
rename. However, compatibility with 0.4's `sections` was only kept on the read
path, while writing 0.5's `items` unconditionally, which ends up in the bit
bucket on 0.4, effectively disabling substitution fully and leaving the include
directives in the final documentation. Restore writing into the `sections` when
they were there so that substitution works again.
Change-Id: Idd4d7653012660f3f7fc27f81f29b82d6a6a6964
Silly me forgot to add the dependency responsible for handling
`@pytest.mark.timeout` failsafes to kill a test if it takes too long,
which results in a warning when running f2.
This commit adds the required dependency and hence removes the warning
Change-Id: I281cad05bb0cf50208f72080cb2bfd8cbfb09d0c
Running the sandbox (as unprivileged user) requires all of user, mount and PID
namespacing. Previously, only the user namespacing support was checked.
Unfortunately, newer Ubuntu kernels impose restrictions on capabilities within
unprivileged user namespaces [1]; as observed by experimentation, in particular
(re-)mounting filesystems or changing mount propagation is prohibited even
inside a mount namespace, making it impossible to use the Lix sandbox. So any
tests requiring sandboxing should check for this breakage and be skipped if
detected.
`unshare --mount` performs the same problematic operations, so that's what gets
used for the test. PID namespaces are included as well for completeness, even
though currently no instance of breakage due to them specifically is known.
This fixes [2] for the functional test suite only.
[1] https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces
[2] https://git.lix.systems/lix-project/lix/issues/545
Change-Id: If95b527bf965ccb06371bbef7b6007666a6a6964
Builds using a diverted store strictly require sandboxing. Explicit disabling
by the user is already handled properly by force-enabling the sandbox anyway
(with a warning), but sandbox-fallback was still honored, predictably leading
to hilarious breakage (like the "confusing output" in [1]) when it actually
triggered. Cause a hard failure instead.
[1] https://git.lix.systems/lix-project/lix/issues/545
Change-Id: I61ea32c3bd8bca4e5ba278312b0a49326a6a6964
Builds using a diverted store strictly require sandboxing. It therefore makes
sense to automatically enable it even if otherwise explicitly disabled by the
user. However, they should still be informed that their choice is not being
respected.
Change-Id: Ia2cbdc24a4c55d8ecc49094e35e698e16a6a6964
Disabling build sandboxing is a significant change to build isolation. If this
is done automatically due to namespacing issues (and sandbox-fallback not being
disabled), show the user a loud warning without the need to opt into a lot of
other log spam.
Change-Id: I52daccdd6934873234223c798ca3dcff6a6a6964
This contains the depth of nix shells nesting.
Fixes#826
Co-authored-by: Qyriad <qyriad@qyriad.me>
Change-Id: If584c9d02730d6c857636dafdeab0c01f4ec8e0f
If the NIX_PATH shadows internal files, this will often break things,
hence we forbid it.
Fixes#998
Change-Id: I70e5d389532ada1c9f910c60281abe565e3ce6bb
Some of the tests depended on the evaluation order chosen internally by
nix-eval-jobs, which will break when that order changes. In practice the order
is not guaranteed, and already not deterministic with multiple workers.
Change-Id: I2e85d096f4439a8075a6f4a08d31c0f16a6a6964
GCC still ICEs on coroutines, while Clang works properly. Use the latter, just
like for Lix itself.
Change-Id: I802d6df7279550a5f5355e6c1ea0533c6a6a6964
Previously the collector-side implementation of the worker interaction protocol
was a bunch of spaghetti code. Split it apart at places where it makes sense to
be easier to follow.
Change-Id: If3cc6b6fbf289dd778856b40b55316c76a6a6964
As the worker processes are now properly spawned using fork-exec, they no
longer inherit the stack size from the non-main coordinator thread, but get a
main thread with reasonable stack size on their own. For this reason the
coordinator threads can use the default stack size, and turn std::thread can be
used as the only reason for the custom wrapper was to enlarge the stack.
Change-Id: I1192474885abe9b0625ac483840b6e1a6a6a6964
Similar to how 7b37d5ea6a switched the subdaemons
from fork-only to fork-exec, do the same for the nix-eval-jobs workers. This is
the promised change that causes the signal handler thread to appear.
Change-Id: I590a5eb6bddfb546a331a01191fd788b6a6a6964
The retry added in 4c5efd4548 has never worked
properly, as releasing the pid the first time causes it to be -1 and thus
waiting for a random child on retry. While there is a good chance that this
really catches the crashed worker, this is by no means guaranteed, and it could
just as well have caught different restarting worker. Wait synchronously
instead. The worst possible outcome is a hang instead of an error message if
the worker manages to close its pipe but then fail to exit, which is not
supposed to happen in the first place.
Change-Id: I4e4c3f532ec15c3118f6f77821ab820a6a6a6964
warning: embedding a directive within macro arguments has undefined behavior
????? CLANG? why does this take -pedantic to show??
Change-Id: I6f4a9f3db1b89448f9f5d00d82b551e46a6a6964
This change brought to you by my uninstalling go but forgetting to
remove ~/.local/opt/go/bin from my PATH.
Change-Id: I937675bab11ac7e0678c896cba78204b6a6a6964
Maintains compatibility with mdbook 0.4.x.
Includes comments for what to remove once 0.5.x is the only we care
about. Some other changes technically could be changed at that point,
but currently serve to enable universal support of 0.4.x and 0.5.x
Fixes#1051.
Change-Id: Ic5b405038d180bcd357bbd9e5716879e0c26e5f5
In the same vein as be18b7dc2¹.
Before:
error: syntax error, expecting ';'
After:
error: syntax error, expecting ';' to end binding
[1]: be18b7dc25
Change-Id: Iffbcb113d2b892a50c646c9875e970376a6a6964
Lix blocks SIGPIPE, and std::cout ignores EPIPE (actually it probably stashes
an error code in some flag that no one ever reads, with the same end result).
Consequently, nix-eval-jobs would waste resources by continuing to evaluate
even when the reader interested in the results has long gone away. Instead, use
writeToStdout, which throws an error on EPIPE, leading to the process
terminating as desired.
Reported-by: Winter <winter@winter.cafe>
Change-Id: I962c09bab582a8ed27dd41c01b1519876a6a6964
Right now, the worker will not receive interrupts, as it has no signal handler
thread due to the way it forks, but this will change soon. Prepare to handle
worker interruption properly, by letting the worker exit successfully in this
case and adjusting the coordinator to handle this exit gracefully.
Change-Id: I9f79670b0d2004e7e2d8bf36cf67108e6a6a6964
For some reason that eludes me, it used to only print the last attribute
instead of the full chain.
Change-Id: I2f5f6e85e2b1ab3ab20a9c99bf06d1e45a4d09bb
`e`, `ae`, `j`, `jAttrs` … holy fuck this function was so much worse
than it needed to be. This refactoring includes:
- Descriptive variable names where possible
- More code comments indicating what even is happening
- Shuffled the control flow around to be more linear (move early return
conditions up, etc.) and have less rightwards-drift, to increase
readability
- Extracted `mergeAttrs` function for future changes and readability
Change-Id: I7253b47ce4910fdc67aa7d6edff2f3c0bb8fa55b
In one of the cases it pointed to the first instead of the second
occurence, which is a bit confusing
Change-Id: Ie508d1f84feb434708804d45aa8a9c8b1e3c5f69
How long do we have the value printer already? It's time to stop
concatenating strings like it's 2005
Change-Id: I3f5074de2439a1ad78af94de877bb141bc9f1d82
When using `builtins.derivation`, the `outputs` list can be omitted, and
in that case, it defaults to `[ "out" ]`, set this default when building
the shell derivation for `nix develop`
Fixes: #556
Change-Id: I4dc77d060f30ee499e186ec919de30bd3e5c53bb
Commit 5dc847b47b introduced usage of
std::chrono::milliseconds, which prints with the unit, leading to its
duplication. Remove this duplication again. Specifically, the explicitly
printed one is kept, due to better typography.
Change-Id: I724ea6be6f2a1349fe799602c3429f096a6a6964
We have done it!
The functional/lang framework has fully been migrated to functional2 :D
closes: #856
Change-Id: I63ad8d7dbcd9b5267ca04af68df73b1ffa3d6461
After multiple cycles of deprecation, we now use the lix repositorty as
a default, when no prefix is defined
Change-Id: I29b445fee39fb20c0ff024e3b95402f459144b62
Added builtins.warn` which takes two arguments: a message that is
displayed as a warning during evaluation which must be a string and a
value that is returned from the expression.
The next commits add new settings to control the behavior of the new
builtin: `debugger-on-warn` allows the user to start the debugger and
`abort-on-warn` aborts evaluation with an error.
Unlike upstream, I chose not to mark evaluation warnings from
`builtins.warn` as distinct from other warnings because that breaks the
commonly expected logging format `level: message`.
Co-authored-by: Qyriad <qyriad@qyriad.me>
Upstream-PR: https://github.com/NixOS/nix/pull/10592
Fixes: https://git.lix.systems/lix-project/lix/issues/579
Change-Id: I8658c88e5c27952b65e8b9f5525a572e0680cc1f
There are currently 4 different ways to run garbage collection using
lix:
- `nix-collect-garbage`
- `nix-store --gc`
- `nix store gc`
- (using the daemon directly)
As they were written all at different times, their output varies (and is
broken in some case). This unifies the display of informations in the
following ways:
- The list of paths in the results is always printed (in the nix3 cli it
is hidden unless `-v` is passed)
- The number of paths in the result set is always displayed
- The size of deleted paths is only shown when actually deleting things
(as it would have been 0B in any case)
Fixes#905
Change-Id: I40d9ec7c6d76795f6c6dd30df196d1e855bdb9db
This also allows removing a static initialization of a primop.
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Change-Id: I74da68205630f658b239e4327bf5e27bfc1b91da
This is done at the start of the command call, which allows not building
expensive things if the arguments are incorrect anyway.
Fixes#687
Change-Id: Ic924fe2115cf760684c6fdf7987279e96ab00286
the newly added f1/lang tests is required for the f1/lang framework
dismanteling to happen in a separate commit
Change-Id: Ic419c515262294c51a46d1513daa7848e4b71405
While the err file should always be empty, we prefer not have (easily
avoidable) warnings in the log, in order for actual warnings to be
spotted more easily.
Additionally this way no additional changes are required in case they
make use of some depreacted features in the future.
Change-Id: Ie078e2a851b2035d839f6b95d4188e475eb96b2d
autoargs get their own folder and are not allowed with the other
function args tests, just like they deserve their own ring of hell
It shall be noted though that this test is particularly pointless, as it
merely tests the normal argument passing (which is fine), and not the
✨magic✨ autoargs actually provide
Change-Id: Ia2dc00754a13c02e4926200141f90bc797820a15
That one is a bit confusing, as I renamed the old `in` test to `in-2`
(file identities in Git when)
Change-Id: Id0e17de414dbc1b1b616d456175704f98657a16a
Call it a bug, call it a feature, but the old testing framework
ungracefully shits itself when it has no tests to run … in other words,
we finally migrated all the parse-fail tests 🎉
Change-Id: Ibd7f1c04d9a396a20f14361af7924f0074d1ac23
The second test has also been fixed. I'd do it in two commits if
functional1 wasn't such a PITA (guess why we're doing the migration …).
Basically, the introduced syntax got changed later on, but the test
never got updated, and back then tests didn't test the output so nobody
ever noticed it now failing for a wrong reason.
Change-Id: I5b66127d507c65676b8dd4a82d1e1f2857fef145
This really looks like one of the first tests ever written, it doesn't
seem to be actively testing anything interesting
Change-Id: I442840db932203c25da2d8400ada791f8aed04ce
Previously, `builtins.floor` and `builtins.ceil` always cast the input into a floating point value before running the operation and casting the floating point result back into an integer.
No checks were made for precision loss in either coercing integer inputs or converting the output to an integer (and in fact in the latter case, invoked undefined behaviour).
Now, Lix checks for precision loss on integer input (to avoid a silent eval semantics change if we were to simply pass it through as-is) and on integer output.
If your code fails to evaluate after this change, use `--extra-deprecated-features floor-ceil-corrupt-integers`.
synopsis:"flake config warnings are now printed to stderr"
issues:[1155]
cls:[5379]
category:"Fixes"
credits:[lheckemann]
---
The settings listed in a flake-config confirmation prompt are now printed to stderr rather than stdout, which allows `nix print-dev-env` to emit valid bash again even in the presence of untrusted settings.
synopsis:"Shadowing internal files through the Nix search path is now an error"
issues:[998]
cls:[4632,5370]
category:"Breaking Changes"
credits:[thubrecht, jade, horrors]
---
As Lix uses the path `<nix/fetchurl.nix>` for bootstrapping purposes, the ability to shadow it by adding `nix=/some/path` (or `/other/path` that contains a `nix` directory) to the search path is not desirable.
Lix 2.95 deprecated this behavior with a warning, Lix 2.96 now turns it into a hard error if the `nix-path-shadow` deprecated feature isn't enabled. This deprecated feature is slated to be removed in Lix 2.98.
Flakes try to keep their inputs and metadata "simple", to make sure no unbounded computation may happen when calling e.g. `nix flake show`.
Those checks were haphazard, a maintenance burden, and also easily circumventable.
Lix has now replaced all the old checks by a simple rule: **No function calls outside of `outputs`.**
This is easier to reason about than the previous set of inconsistent rules, and crucially now also allows syntax features that users felt like they *should* have worked in the past, like let bindings.
However, some warts still remain for now: Some syntax constructs like `-1` internally desugar to `__sub 0 1`, which is a function call and thus remains forbidden.
This will be rectified as soon as the deprecation period of the respective anti-features has been completed.
This change is **breaking** in the sense that flakes which are written with the newly allowed language features will not evaluate with an older Lix version which still uses the old, more restrictive checks.
Crucially, this also affects **all transitive dependants** of such Flakes.
> If you are building via the Lix daemon (default on Linux and macOS), it is the Lix daemon user account (that is, `root`) that should have SSH access to a user (not necessarily `root`) on the remote machine.
> If you are building via the Lix daemon (default on Linux and macOS), it is the Lix daemon user account (that is, `root`) that should have SSH access to a user (not necessarily `root`) on the remote machine.
>
>
> Furthermore, `root` needs to have the public host keys for the remote system in its `.ssh/known_hosts`.
> Furthermore, `root` needs to have the public host keys for the remote system in its `.ssh/known_hosts`.
> To add them to `known_hosts` for root, do `ssh-keyscan USER@HOST | sudo tee -a ~root/.ssh/known_hosts`.
> To add them to `known_hosts` for root, do `ssh-keyscan HOST | sudo tee -a ~root/.ssh/known_hosts`.
>
>
> If you can’t or don’t want to configure `root` to be able to access the remote machine, you can use a private Nix store instead by passing e.g. `--store ~/my-nix` when running a Nix command from the local machine.
> If you can’t or don’t want to configure `root` to be able to access the remote machine, you can use a private Nix store instead by passing e.g. `--store ~/my-nix` when running a Nix command from the local machine.
## Configuration
The list of remote machines can be specified on the command line or in
The list of remote machines can be specified on the command line or in
the Lix configuration file. The former is convenient for testing. For
the Lix configuration file. The former is convenient for testing.
example, the following command allows you to build a derivation for
Additionally, there are two supported formats to configure remote builders:
`x86_64-darwin` on a Linux machine:
The legacy, "space"-separated format and starting with Lix 2.95.0, a TOML.
- `uri`: Due to backward compatibility, the `ssh://` may be omitted for the store-uri.
- `ssh-public-host-key`: The key must be provided encoded in base64. Specifically calculated via `base64 -w0 /etc/ssh/ssh_host_ed25519_key.pub`
### Format detection
At first, the given configuration is being parsed syntactically as a toml.
If parsing fails and the given configuration contains a `"` the error is presented to the user, as those characters are necessary for TOML, but disallowed for the legacy format.
Otherwise, parsing is retried using the legacy format.
If non-syntactic errors are detected within the toml, the exception will always be shown to the user directly.
## Builder selection
The configuration(s) above specify several machines that can perform `i686-linux` builds.
However, `granite` will only do builds that have the attribute
```nix
requiredSystemFeatures = [ "benchmark" ];
```
or
```nix
requiredSystemFeatures = [ "benchmark" "kvm" ];
```
`diorite` cannot do builds that require `kvm`, but `andesite` does support
such builds. For regular builds, `diorite` will be preferred over
@@ -177,6 +177,12 @@ Most commands in Lix accept the following command-line options:
You can override this using `--arg`, e.g., `nix-env --install --attr pkgname --arg system \"i686-freebsd\"`.
You can override this using `--arg`, e.g., `nix-env --install --attr pkgname --arg system \"i686-freebsd\"`.
(Note that since the argument is a Nix string literal, you have to escape the quotes.)
(Note that since the argument is a Nix string literal, you have to escape the quotes.)
Additionally, dots are interpreted as attribute-path separators.
I.e. `nix-instantiate '<nixpkgs>' -A hello-unfree --arg config.allowUnfree true` will result in an argument `config` with value `{ allowUnfree = true; }` being passed to `<nixpkgs>`.
Please note that merging of different arguments is rejected.
I.e. `--arg config '{ cudaSupport = true; }' --arg config.allowUnfree true` will not work whereas `--arg config.cudaSupport true --arg config.allowUnfree true` is accepted.
@@ -19,7 +19,7 @@ This description is not normative, but a feature removal may roughly happen like
1. Add a warning when the feature is being used.
1. Add a warning when the feature is being used.
2. Disable the feature by default, putting it behind a deprecated feature flag.
2. Disable the feature by default, putting it behind a deprecated feature flag.
- If disabling the feature started out as an opt-in experimental feature, turn that experimental flag into a no-op or remove it entirely.
- If disabling the feature started out as an opt-in experimental feature, turn that experimental flag into a no-op or remove it entirely.
For example, `--extra-experimental-features=no-url-literals` becomes `--extra-deprecated-features=url-literals`.
For example, `--extra-experimental-featuresno-url-literals` becomes `--extra-deprecated-featuresurl-literals`.
3. Decide on a time frame for how long that feature will still be supported for backwards compatibility, and clearly communicate that in the error messages.
3. Decide on a time frame for how long that feature will still be supported for backwards compatibility, and clearly communicate that in the error messages.
- Sometimes, automatic migration to alternatives is possible, and such should be provided if possible
- Sometimes, automatic migration to alternatives is possible, and such should be provided if possible
- At least one NixOS release cycle should be the minimum
- At least one NixOS release cycle should be the minimum
@@ -51,28 +51,64 @@ $ nix-shell -A native-clangStdenvPackages
### Building from the development shell
### Building from the development shell
Run a clean build and test with `just clean setup buildinstall test`.
We have a [justfile](https://just.systems/) for extra convenient building.
It defaults to using `./build` as the build directory, and `$out` (`./outputs/out`) as the install directory.
For most cases, you can clean-build, install, and run the tests with:
```bash
$ just setup --wipe && just test
```
> **Note**
>
> The `--wipe` argument to `meson setup` conveniently works whether you have an existing build directory or not.
>
> However, it is *mostly*, but not *exactly* equivalent to deleting the build directory first.
> In particular, previously specified `-D` build options are **preserved** with `--wipe` (for some reason).
> For example, if you fetch and checkout a new version of Lix, and that new version *removes* a Meson build option from `./meson.options`, *and* a previous invocation in that build directory explicitly set that option, then `meson setup --wipe build` will error, complaining about the unknown option.
> For these cases, `just clean` will give you a well-and-truly-this-time-for-real clean build.
Because the integration tests require installation to work, `just test` automatically also calls `just install`, and Meson helpfully will automatically build any targets that need building when trying to install them.
You can override the build directory or install directory by setting the justfile [variables](https://just.systems/man/en/setting-variables-from-the-command-line.html) `outdir` and `builddir` on the command-line:
```bash
$ just builddir=build-before-bisect outdir=out-before-bisect setup
$ just builddir=build-before-bisect test
```
You'll have to set `builddir` for every target, but `outdir` only needs to be set for `setup`.
You can also run the unit tests and integration tests separately:
You can also run the unit tests and integration tests separately:
```bash
```bash
$ just setup build test-unit
$ just setup
$ just install test-integration
$ just test-unit
$ just test-integration
```
```
Many justfile aliases have a `-custom` variant which pass extra arguments to `meson`.
Most justfile targets forward all further arguments to the underlying Meson invocation.
For example, to work on both Lix and nix-eval-jobs you can run:
For example, to work on both Lix and nix-eval-jobs you can run:
```
```bash
$ just setup-custom -Dnix-eval-jobs=enabled
$ just setup -Dnix-eval-jobs=enabled
$ # or
$ mesonFlags=-Dnix-eval-jobs=enabled just setup
```
```
Note that only targets which don't accept extra arguments can be used when
Note that only targets which *don't* accept extra arguments can have other targets following them.
running multiple targets at once; `just setup build` is fine, but `just
`just clean setup` is equivalent to `just clean && just setup`, but `just build test` runs the `build` target with the argument `test`.
setup-custom build` is an error. The `test` target is usually the last one to
This means that if you want to, for example, build with lower parallelism, and then test, you will have to do something like this:
run, so it always accepts extra arguments.
```bash
$ just build -j4
$ just test
```
Finally, the rewrite of the integration test suite, functional2, also has its own justfile target which allows passing extra arguments to pytest.
For example, to collect and list all functional2 tests without running them, you can pass pytest's `--collect-only` argument:
```bash
$ just test-functional2 --collect-only
```
You can also build Lix manually:
You can also build Lix manually:
@@ -141,6 +177,21 @@ To inspect the canonical source of truth on what the state of the buildsystem co
$ meson introspect
$ meson introspect
```
```
#### LLD
The development shell on Linux uses LLD by default for faster link times.
This is set using `mesonFlags`, so to override it, you can simplify re-specify the linker to Meson:
```bash
$ just setup -Dc_link_args=-fuse-ld=ld -Dcpp_link_args=-fuse-ld=ld
```
While using LLD, you may find it helpful to use ThinLTO for even further improvements to link times for incremental builds:
```bash
$ just setup -Db_lto=true -Db_lto_mode=thin -Db_thinlto_cache=true
```
## Sending changes to Gerrit for review {#sending-to-gerrit}
## Sending changes to Gerrit for review {#sending-to-gerrit}
We use Gerrit for all our code review in Lix.
We use Gerrit for all our code review in Lix.
@@ -149,7 +200,7 @@ Our instance is at <https://gerrit.lix.systems>.
There's much more information about how to use Gerrit in the [wiki section on Gerrit][wiki-gerrit] including how to use Jujutsu, how to use the UI and more.
There's much more information about how to use Gerrit in the [wiki section on Gerrit][wiki-gerrit] including how to use Jujutsu, how to use the UI and more.
The Snix project also has some Gerrit information [in their contributing docs][snix-gerrit].
The Snix project also has some Gerrit information [in their contributing docs][snix-gerrit].
The gist is that once you have your SSH key and git remote set up, you can send commits for review with:
The gist is that once you have your SSH key and git remote set up, you can send commits for review with:
@@ -175,7 +226,7 @@ The `Code-Review+2` from before will stick around through trivial rebases so no
We use Buildkite for our CI, usually you will not have to interact directly with it other than reviewing any errors it produces, which are linked from Gerrit.
We use Buildkite for our CI, usually you will not have to interact directly with it other than reviewing any errors it produces, which are linked from Gerrit.
However in certain cases a CI run will fail due to transient issues not related to your code and you will need to rerun it by hand.
However in certain cases a CI run will fail due to transient issues not related to your code and you will need to rerun it by hand.
You can log in to the CI via [SSO](https://buildkite.com/sso/lix-project). On your job you can then hit the "Retry failed" button to rerun it, normally you will not have a repeat of the transient issue.
You can log in to the CI via [SSO](https://buildkite.com/sso/afnix). On your job you can then hit the "Retry failed" button to rerun it, normally you will not have a repeat of the transient issue.
If the build still fails on CI issues or all builds are failing this should be reported via [Zulip on #T-infra](https://zulip.lix.systems/#narrow/channel/7-T-infra) or [Matrix on #dev](https://matrix.to/#/%23dev%3Alix.systems?via=lix.systems).
If the build still fails on CI issues or all builds are failing this should be reported via [Zulip on #T-infra](https://zulip.lix.systems/#narrow/channel/7-T-infra) or [Matrix on #dev](https://matrix.to/#/%23dev%3Alix.systems?via=lix.systems).
## Building Lix with `nix`
## Building Lix with `nix`
@@ -515,7 +566,7 @@ Then add the new file there, and don't forget to register it in the appropriate
The following metadata properties are supported for builtin functions:
The following metadata properties are supported for builtin functions:
*`name` (required): the language-facing name (as a member of the `builtins` attribute set) of the function.
*`name` (required): the language-facing name (as a member of the `builtins` attribute set) of the function.
*`implementation` (optional): a C++ expression specifying the implementation of the builtin.
*`implementation` (optional): a C++ expression specifying the implementation of the builtin.
It must be a function of signature `void(EvalState &, PosIdx, Value * *, Value &)`.
It must be a function of signature `Value(EvalState &, PosIdx, Value * *)`.
If not specified, defaults to `prim_${name}`.
If not specified, defaults to `prim_${name}`.
*`renameInGlobalScope` (optional): whether the definition should be "hidden" in the global scope by prefixing its name with two underscores.
*`renameInGlobalScope` (optional): whether the definition should be "hidden" in the global scope by prefixing its name with two underscores.
@@ -383,7 +383,10 @@ I grepped `lix/` for `get[eE]nv\("` to find the mentions in Lix code.
Overrides compile-time configuration of various locations used by Lix. See `lix/libstore/globals.cc`.
Overrides compile-time configuration of various locations used by Lix. See `lix/libstore/globals.cc`.
**Expected value**: a directory
**Expected value**: a directory
- `NIX_DAEMON_SOCKET_PATH` (optional) - Overrides the daemon socket path from `$NIX_STATE_DIR/daemon-socket/socket`.
- `LIX_DAEMON_SOCKET_DIR` (optional) - Overrides the daemon socket directory from `$NIX_STATE_DIR/daemon-socket`.
**Expected value**: a directory
- `NIX_DAEMON_SOCKET_PATH` (optional) - Overrides the daemon socket path from `$NIX_STATE_DIR/daemon-socket/socket`. Ignored if `LIX_DAEMON_SOCKET_DIR` is set.
**Expected value**: path to a socket
**Expected value**: path to a socket
- `NIX_LOG_FD` (output) - An FD number for logs in `internal-json` format to be sent to.
- `NIX_LOG_FD` (output) - An FD number for logs in `internal-json` format to be sent to.
@@ -401,7 +404,6 @@ I grepped `lix/` for `get[eE]nv\("` to find the mentions in Lix code.
**Expected value**: the path to an executable shell
**Expected value**: the path to an executable shell
- `PRINT_PATH` - Undocumented. Used by `nix-prefetch-url` as an alternative form of `--print-path`. Why???
- `PRINT_PATH` - Undocumented. Used by `nix-prefetch-url` as an alternative form of `--print-path`. Why???
- `_NIX_IN_TEST` - If present with any value, makes `fetchClosure` accept file URLs in addition to HTTP ones. Why is this not `_NIX_FORCE_HTTP`??
Not used anywhere else.
Not used anywhere else.
- `NIX_ALLOW_EVAL` - Used by eval-cache tests to block evaluation if set to `0`.
- `NIX_ALLOW_EVAL` - Used by eval-cache tests to block evaluation if set to `0`.
@@ -164,7 +164,7 @@ Note that lists are only lazy in values, and they are strict in length.
An attribute set is a collection of name-value-pairs (called *attributes*) enclosed in curly brackets (`{ }`).
An attribute set is a collection of name-value-pairs (called *attributes*) enclosed in curly brackets (`{ }`).
An attribute name can be an identifier or a [string](#type-string).
An attribute name can be an identifier or a [double-quoted string](#type-string).
An identifier must start with a letter (`a-z`, `A-Z`) or underscore (`_`), and can otherwise contain letters (`a-z`, `A-Z`), numbers (`0-9`), underscores (`_`), apostrophes (`'`), or dashes (`-`).
An identifier must start with a letter (`a-z`, `A-Z`) or underscore (`_`), and can otherwise contain letters (`a-z`, `A-Z`), numbers (`0-9`), underscores (`_`), apostrophes (`'`), or dashes (`-`).
FIXME(Lix): This section does not document the most common modern practices in terms of avoiding channels, pinning, declarative software installation (see flakey-profile or home-manager or NixOS), or using flakes, etc.
FIXME(Lix): This section does not document the most common modern practices in terms of avoiding channels, pinning, declarative software installation (see flakey-profile or home-manager or NixOS), or using flakes, etc.
It is, however, likely correct at a technical level.
It is, however, likely correct at a technical level.
For more information on modern practices, see the [resources](https://lix.systems/resources) page on the Lix site.
For more information on modern practices, see the [resources](https://wiki.lix.systems/books/lix-users/page/nix-resources) page on the Lix site.
- Deprecate shadowing internal files through the Nix search path [lix#998](https://git.lix.systems/lix-project/lix/issues/998) [cl/4632](https://gerrit.lix.systems/c/lix/+/4632)
As Lix uses the path `<nix/fetchurl.nix>` for bootstrapping purposes, the ability to shadow it by adding `nix=/some/path` (or `/other/path` that contains a `nix` directory) to the search path is not desirable.
To alleviate potential issues, Lix now emits a warning when the Nix search path contains potential shadows for internal files, which will be changed to an error in a future release.
The warning can be disabled by enabling the deprecated feature `nix-path-shadow`.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- More deprecated features [cl/2092](https://gerrit.lix.systems/c/lix/+/2092) [cl/2310](https://gerrit.lix.systems/c/lix/+/2310) [cl/2311](https://gerrit.lix.systems/c/lix/+/2311) [cl/4638](https://gerrit.lix.systems/c/lix/+/4638) [cl/4652](https://gerrit.lix.systems/c/lix/+/4652) [cl/4764](https://gerrit.lix.systems/c/lix/+/4764)
This release cycle features a new batch of deprecated (anti-)features.
You can opt in into the old behavior with `--extra-deprecated-features` or any equivalent configuration option.
-`broken-string-indentation` indented strings (those starting with `''`) might produce unintended results due to how the whitespace stripping is done. Those cases will now warn the user.
-`broken-string-escape` "escaped" characters without a properly defined escape sequence evaluate to "themselves". This is in most cases unintended behaviour, both for writing regexes, and using legacy or uncommon escape sequences like `\f`. The user will now be warned, if those are present.
-`floating-without-zero` so far, one was able to declare a float using something like `.123`. This can cause confusion about accessing attributes. Floating point numbers must now always include the leading zero, i.e. `0.123`
-`rec-set-merges` Attribute sets like `{ foo = {}; foo.bar = 42;}` implicitly merge at parse time, however if one of them is marked as recursive but not the others then the recursive attribute may get lost (order-dependent). Therefore, merging attrs with mixed-`rec` is now forbidden.
-`rec-set-dynamic-attrs` Dynamic attributes have weird semantics in the presence of recursive attrsets (they evaluate *after* the rest of the set). This is now forbidden.
-`or-as-identifier``or` as an identifier has always been weird since the `or` (almost-)keyword has been introduced. We are deprecating the backcompat hacks from the early days of Nix in favor of making `or` a full and proper keyword.
-`tokens-no-whitespace` Function applications without space around the arguments like `0a`, `0.00.0` or `foo"1"2` are now forbidden. The same applies to list elements. The primary reason for this deprecation is to remove foot guns around surprising tokenization rules regarding number literals, but this will also free up some syntax for other purposes (e.g. `r""` strings) for reuse at some point in the future.
-`shadow-internal-symbols` has been expanded to also forbid shadowing `null`, `true` and `false`.
-`ancient-let` deprecation has been turned into a full parser error instead of a warning.
-`rec-set-overrides` deprecation has been turned into a full parser error instead of a warning.
Many thanks to [piegames](https://git.lix.systems/piegames), [rootile (Rutile)](https://git.lix.systems/rootile), and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Move `/root/.cache/nix` to `/var/cache/nix` by default [lix#634](https://git.lix.systems/lix-project/lix/issues/634) [cl/4671](https://gerrit.lix.systems/c/lix/+/4671)
By default, Lix attempts to locate a cache directory for its operations (such
as the narinfo cache) by checking the value of `$XDG_CACHE_DIR`.
However, since the Nix daemon is a system service, using `$XDG_CACHE_DIR` is
not typical in this context.
To address this, systemd provides a better solution. Specifically, when
`CacheDirectory=` is set in the `[Service]` section of a systemd unit, it
automatically sets the `$CACHE_DIRECTORY` environment variable and systemd will
manage that cache directory for us.
Now, our systemd unit includes `CacheDirectory=nix`, which sets the
`$CACHE_DIRECTORY` and takes precedence over `$XDG_CACHE_DIR`.
If the daemon is run under user units, systemd will automatically set
`$XDG_CACHE_DIR`.
If neither of these variables is set, Lix falls back to its default behavior.
By default, Lix will try to find a cache directory for its various operations
(e.g. narinfo cache) by looking into `$XDG_CACHE_DIR`.
In summary, what was stored in `/root/.cache/nix` is now moved to
`/var/cache/nix/nix`.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
The `fetch-closure` experimental feature has been removed.
Outside of allowing the user to import closure from binary cache,
`fetchClosure` also allowed you to do the following:
* rewrite non-CA path to CA
* reject non-CA paths at fetching time
* reject CA paths at fetching time
Some people are using those mechanism to prevent users from having to build any
package and force going via the declared cache or as a way to use ancient/old
software without paying the evaluation cost of a second nixpkgs.
Both use cases are somewhat of an antipattern in Nix semantics. If the user
cannot fetch a program directly via the substituter mechanism and fall back to
local build, this is a feature *and* a misconfiguration. If the user cannot build
certain derivations because they are too expensive, the build directives should
pass `-j0` or similar.
As for the second usecase, there's a different way to do it that also allows to
have a way to reproduce the paths that are hardcoded in that file, perform
`import (fetchurl "https://my-cache/${hashparts storepath}.drv")` rather, i.e.
an IFD to a possibly well known name. The backend can generate them on the fly
or once, and possess stable names.
Finally, as for the non-CA → CA features, Lix removed ca-derivations.
fetchClosure offers ca-derivations-like features which suffers from similar
shortcomings albeit lessened. It only follows that we should deprecate
and remove these capabilities.
Many thanks to [just1602](https://git.lix.systems/just1602) for this.
## Features
-`nix store add-path` now supports references [cl/5205](https://gerrit.lix.systems/c/lix/+/5205)
Lix supports two categories of hashes in store paths: input-addressed and output-addressed.
Currently, in Nix language, there is no way to produce output-addressed paths with references, as fixed-output derivations forbid references.
However, the Nix store actually *supports* references in output-addressed paths.
This is very useful for importing build products created outside of Lix that reference dependency store paths since such build products have no associated derivation so don't make any sense to input-address.
Previously, output-addressed paths with references could only be created by writing a custom client to the rather-baroque Nix daemon protocol; now it's available in the CLI.
Using `nix store add-path --references-list-json REFS_LIST_FILE SOME_PATH` with a JSON list of string store paths, you can now create such paths with the Lix CLI.
They may be consumed from Nix language using something like `builtins.storePath` or the following which also works in pure evaluation mode:
```nix
# Hack from https://git.lix.systems/lix-project/lix/issues/402#issuecomment-5889
path:
builtins.appendContext path {
${path} = {
path = true;
};
}
```
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Add `builtins.warn` for emitting warnings from Nix code [cl/2248](https://gerrit.lix.systems/c/lix/+/2248)
Lix now has a builtin function for emitting warnings.
Like `builtins.trace`, it takes two arguments: the message to emit, and the expression to return.
_Unlike_ `builtins.trace`, `builtins.warn` requires the first argument — the message — to be a string.
In the future we may extend `builtins.warn` to accept a more structured API.
To go along with this, we also have two new config settings:
- [`debugger-on-warn`](@docroot@/command-ref/conf-file.md#conf-debugger-on-warn), which, when used with `--debugger`, makes `builtins.warn` also function like [`builtins.break`](@docroot@/language/builtins.md#builtins-break).
- [`abort-on-warn`](@docroot@/command-ref/conf-file.md#conf-abort-on-warn), which aborts evaluation entirely after the warning is emitted.
Many thanks to [Emilia Bopp](https://git.lix.systems/milibopp) and [Qyriad](https://git.lix.systems/Qyriad) for this.
- `keep-env-derivations` is now supported for nix3 CLI (`nix profile`) [lix#1095](https://git.lix.systems/lix-project/lix/issues/1095) [cl/5332](https://gerrit.lix.systems/c/lix/+/5332)
The `keep-env-derivations` feature is now available for `nix profile`. This allows users to prevent the garbage collection of derivations used to install a profile, even when `keep-derivations = false` (set to `true` by default).
Previously, `nix-env` supported this feature, but `nix profile` **never** did. This caused issues when garbage collection removed the associated `.drv` files, which are required, for example, by vulnerability management tools (e.g. [vulnix](https://github.com/nix-community/vulnix)) for proper operation.
This issue has now been resolved.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Make `log-format` a setting [cl/4686](https://gerrit.lix.systems/c/lix/+/4686)
The [`--log-format` CLI option](@docroot@/command-ref/opt-common.md#opt-log-format) can now be set in [`nix.conf`](@docroot@/command-ref/conf-file.md#conf-log-format)!
For example, you can now persistently enable the `multiline-with-logs` log format [added in Lix 2.91](@docroot@/release-notes/rl-2.91.md) by adding the following to your `nix.conf`:
```conf
log-format = multiline-with-logs
```
Or the equivalent in a NixOS configuration:
```nix
{
nix.settings.log-format = "multiline-with-logs";
}
```
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- Allow remote builders to be configured using TOML [cl/4533](https://gerrit.lix.systems/c/lix/+/4533)
Lix now supports configuring remote builders using a TOML file instead of the old, very cursed and incomprehensible format.
This comes with not only a human-understandable file, but also with better messages and error reports on misconfiguration.
A more detailed Documentation can be found on the [distributed-builds](@docroot@/advanced-topics/distributed-builds.md) documentation page.
Many thanks to [rootile (Rutile)](https://git.lix.systems/rootile) and [Qyriad](https://git.lix.systems/Qyriad) for this.
- Emit warnings when encountering IFD with `warn-import-from-derivation` [nix#13279](https://github.com/NixOS/nix/pull/13279) [cl/3879](https://gerrit.lix.systems/c/lix/+/3879)
Instead of only being able to toggle the use of [Import from
Derivation](https://nix.dev/manual/nix/stable/language/import-from-derivation) with
`allow-import-from-derivation`, Lix is now able to warn users whenever IFD is encountered with
`warn-import-from-derivation`.
Many thanks to [Seth Flynn](https://git.lix.systems/getchoo), [gustavderdrache](https://github.com/gustavderdrache), and [Eelco Dolstra](https://github.com/edolstra) for this.
## Improvements
- Collect Flakes untrusted settings into one prompt [lix#682](https://git.lix.systems/lix-project/lix/issues/682) [cl/2921](https://gerrit.lix.systems/c/lix/+/2921)
When working with Flakes containing untrusted settings, a prompt is shown for each setting, asking whether to vet or approve it. This looks like:
```
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all)
```
In Flakes with a large number of settings to approve or reject, this process can become tedious as each option must be handled individually.
To address this, all untrusted settings are now consolidated into a single prompt: allowing for bulk acceptance permanently or not, rejection, or detailed review. For example:
### Scrutiny scenario
```console
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all) n
warning: you can set 'accept-flake-config' to 'false' to automatically reject configuration options supplied by flakes
Do you want to allow setting 'allow-dirty = false'? (yes for now/Allow always/no for now) y
Do you want to allow setting 'sandbox = false'? (yes for now/Allow always/no for now) n
```
### Reject everything scenario
```console
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all) N
Rejecting all untrusted nix.conf entries
warning: you can set 'accept-flake-config' to 'false' to automatically reject configuration options supplied by flakes
```
### Accept everything scenario
```console
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all) y
```
### Accept everything PERMANENTLY scenario
Note that accepting everything permanently will authorize these options for any
further operations.
The file containing this trust information is usually located in
`~/.local/share/nix/trusted-settings.json` and can be edited manually to revoke
this permission until Lix provides a first-class command for this manipulation.
```console
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all) A
```
Many thanks to [isabelroses](https://git.lix.systems/isabelroses), [Raito Bezarius](https://git.lix.systems/raito), and [eldritch horrors](https://git.lix.systems/pennae) for this.
- `--check` or `--rebuild` is clearer about a missing path [lix#485](https://git.lix.systems/lix-project/lix/issues/485)
Previously, when running Lix with --check or --rebuild, failures often surfaced
as an unhelpful error:
> "some outputs of '...' are not valid, so checking is not possible"
This message could mean two different things:
- The requested output paths don't exist at all, or,
- Some outputs exist but are not known to Lix
Lix cannot reliably distinguish these cases, so it treated them the same.
We've updated the error messages to clarify what Lix can determine: whether any
valid outputs (> 0) are present or whether no outputs are available.
When no valid outputs can be found, Lix will now suggest building the derivation
normally (without --check or --rebuild) before trying again.
When some valid outputs are present, Lix now reports which ones are valid,
shows the full list of known outputs, and also suggests building the derivation
normally.
In the future, Lix may automate this recovery step when it knows how to rebuild
the paths, but implementing that safely requires more extensive changes to the
codebase.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- `nix develop` no longer ignores the env variable `SSL_CERT_FILE` [cl/5042](https://gerrit.lix.systems/c/lix/+/5042)
Running `nix develop` and `nix print-dev-env` on shells that define the environment variable `SSL_CERT_FILE` now works correctly by exporting that variable inside the built shell.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- Linux sandbox launch overhead greatly reduced [cl/5030](https://gerrit.lix.systems/c/lix/+/5030) [cl/5073](https://gerrit.lix.systems/c/lix/+/5073) [cl/5074](https://gerrit.lix.systems/c/lix/+/5074)
Sandboxed builds are now much cheaper to launch on Linux, with constant management
overhead. This will mostly be noticeable when building derivation trees containing
many small derivations like nixpkgs' `writeFile` or `runCommand` with scripts that
exit quickly. In synthetic tests we have seen build times of 3000 small runCommand
drop from 80 seconds to 14 seconds, which is the most optimistic case in practice.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- mTLS store connections via a plugin [cl/3754](https://gerrit.lix.systems/c/lix/+/3754) [cl/3696](https://gerrit.lix.systems/c/lix/+/3696) [cl/3697](https://gerrit.lix.systems/c/lix/+/3697) [cl/3698](https://gerrit.lix.systems/c/lix/+/3698)
To support use cases requiring mutual TLS (mTLS) authentication when connecting
to remote Nix stores, e.g. private stores, we have introduced a **contributed**
mTLS plugin extending the Lix store interface.
This design follows an extensibility model which was brought up [by a proposal
This mTLS plugin serves as a concrete example of how store connection
mechanisms can be modularized through external plugins, without extending Lix
core. This idea can be generalized to integrate automatic certificate renewal
or advanced integrations with secrets engine or posture checks.
It enables custom TLS client certificates to be used for authenticating against
a remote store that enforces mTLS.
To use the plugin, configure Lix manually by setting in your `nix.conf`:
```
plugin-files = /a/path/to/libplugin_mtls_store.so
```
Currently, this must be done explicitly. In the future, Nixpkgs will provide a
mechanism to reference an up-to-date and curated set of plugins automatically.
Making plugins easily consumable outside of Nixpkgs (e.g., from external plugin
registries or binary distributions) remains an open question and will require
further design.
Contributed plugins come with significantly reduced **stability** and
**maintenance** guarantees compared to the Lix core. We encourage users who
depend on a given plugin to take on maintenance responsibilities and apply for
ownership within the Lix mono-repository. These plugins are subject to removal
at any time.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [eldritch horrors](https://git.lix.systems/pennae), [mic92](https://github.com/mic92), [vlaci](https://github.com/vlaci), and [nkk0](https://github.com/nkk0) for this.
- Add an indication of nix-shell nesting depth [lix#826](https://git.lix.systems/lix-project/lix/issues/826) [cl/4657](https://gerrit.lix.systems/c/lix/+/4657)
When in a nix shell (either via a `nix-shell` or a `nix develop` invocation), a variable `NIX_SHELL_LEVEL` is exported to indicate the nesting depth of nix shells.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- `nix store delete` can now unlink a GC root before deleting its closure [cl/4660](https://gerrit.lix.systems/c/lix/+/4660)
Ever build something, and then you want to delete it and whatever dependencies it downloaded?
Before you had to resolve the `result` symlink and copy it, then delete it, *then* `nix store delete --delete-closure --skip-live` on the path you copied.
Now you can just pass `--unlink` and the `result` symlink itself.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- `nix path-info` no longer lies to the user about fetching paths [lix#323](https://git.lix.systems/lix-project/lix/issues/323) [cl/4866](https://gerrit.lix.systems/c/lix/+/4866)
When running `nix path-info` with an installable that is not present in the store, Lix no longer
tells the user which paths are missing and that they will be fetched, as the documentation clearly
states that this command does not fetch missing paths.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- Derivations can now be printed in detail in `nix repl` [cl/3842](https://gerrit.lix.systems/c/lix/+/3842)
Traditionally derivations printed in the REPL would only print a formatted object
representing the path of the derivation file it refers to. This makes inspecting
the enhanced derivation attribute sets encountered from `mkDerivation` or similar
wrappers more difficult. Even the `:p`/`:print` command would not elaborate attribute sets
tagged as a derivation.
With this change you can now use `:p`/`:print` to directly inspect a derivation
by providing one as the top-level object. Derivation attribute sets will only be
printed two levels deep and internal derivation attrsets will remain in unexpanded
path form as before. `drvAttrs` will also be elided as these attributes are already
present in the top-level attribute set of the derivation. These heuristics provide
a balance between readability and functionality. When the `:p`/`:print` is omitted,
a bare derivation is printed in the path format as before.
Many thanks to [Lunaphied](https://git.lix.systems/Lunaphied) for this.
- Reject `__json` in structured attributes derivations [lix#380](https://git.lix.systems/lix-project/lix/issues/380) [cl/5286](https://gerrit.lix.systems/c/lix/+/5286)
In structured attributes derivations, `__json` is used internally to store the
JSON representation of the `env` attribute field that users can set.
Unfortunately, a user can set `__json` *and* enable structured attributes,
resulting in a broken derivation from a semantic point of view.
As no user can benefit from setting `__json` *and* enable structured attributes,
we disallow that possibility and throw an error from now on.
This is not seen as a breaking change because there's no user code that can
benefit from this behavior, hence, it's an improvement to user experience.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Shells support `$NIX_LOG_FD` now [lix#336](https://git.lix.systems/lix-project/lix/issues/336) [cl/4694](https://gerrit.lix.systems/c/lix/+/4694) [cl/4695](https://gerrit.lix.systems/c/lix/+/4695)
Lix's "debugging" shells (`nix3-develop` and `nix-shell`) now set the
`$NIX_LOG_FD` environment variable.
This means that [hook logging in
stdenv](https://github.com/NixOS/nixpkgs/pull/310387) appears while debugging
derivations via `nix3-develop` or `nix-shell`.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Supplementary groups are now supported for daemon authentication [lix#968](https://git.lix.systems/lix-project/lix/issues/968) [cl/5021](https://gerrit.lix.systems/c/lix/+/5021)
macOS, FreeBSD and Linux now support receiving supplementary groups during UNIX domain authentication to a Lix daemon.
This change is particularly beneficial for systemd units with `DynamicUser=true` that need to connect to a Lix daemon, using a `SupplementaryGroups=` allocated by systemd in the context of the process. This is desirable if you wish to harden Lix clients.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [Tom Hubrecht](https://git.lix.systems/tom-hubrecht), [alois31](https://git.lix.systems/alois31), and [eldritch horrors](https://git.lix.systems/pennae) for this.
## Fixes
- Nix shells' `$NIX_BUILD_TOP` are shorter [lix#1044](https://git.lix.systems/lix-project/lix/issues/1044) [cl/4663](https://gerrit.lix.systems/c/lix/+/4663)
Following the changes in 2.94.0 to shorten build directory paths, aimed at [resolving UNIX domain socket length issues](https://gerrit.lix.systems/c/lix/+/4168/13) and [improving nix-shell](https://git.lix.systems/lix-project/lix/issues/940), we inadvertently introduced an excessively long path for the `$NIX_BUILD_TOP` environment variable used by Nix shells (their effective temporary `/build` directory).
To fix this, we replaced the `build-top-$HASH` directory name with simply `build-top`, reducing these paths by at least 30 characters.
We also added a test to ensure that Nix shells do not introduce more than 50 extra characters relative to their base directory (e.g., `/tmp` when `$TMPDIR` is not set).
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Fix resolving of symlinks in flake paths [lix#106](https://git.lix.systems/lix-project/lix/issues/106) [lix#12286](https://git.lix.systems/lix-project/lix/pulls/12286) [cl/4783](https://gerrit.lix.systems/c/lix/+/4783)
Flake paths are now canonicalized to resolve symlinks. This ensures that when a flake is accessed via a symlink, paths are resolved relative to the target directory, not the symlink's location.
Many thanks to [stevalkr](https://github.com/stevalkr) and [xyenon](https://git.lix.systems/xyenon) for this.
- The REPL no longer considers failed loads for `:reload` [lix#50](https://git.lix.systems/lix-project/lix/issues/50) [cl/4864](https://gerrit.lix.systems/c/lix/+/4864) [cl/4865](https://gerrit.lix.systems/c/lix/+/4865) [cl/4700](https://gerrit.lix.systems/c/lix/+/4700) [cl/4889](https://gerrit.lix.systems/c/lix/+/4889)
The [REPL](@docroot@/command-ref/new-cli/nix3-repl.md) allows "loading" files, flakes, and expressions into the environment, with the commands `:load`/`:l`, `:load-flake`/`:lf`, and `:add`/`:a` respectively.
The results of those stay in the environment as-is even if their sources change, until the `:reload` command is used.
However `:reload` would re-perform *all* instances of `:l`/`:lf`/`:a`, meaning you would get things like this:
```nix
nix-repl> :l /tmp/texting.nix
error: getting status of '/tmp/texting.nix': No such file or directory
# oops, typo.
nix-repl> :l /tmp/testing.nix
# Do some stuff…
nix-repl> :reload
error: getting status of '/tmp/texting.nix': No such file or directory
```
This is pretty silly, but also *incredibly* annoying, as it would stop there and *not* reload the correct files anymore.
This effectively meant typoing any of the load commands would make `:reload` useless for the rest of the entire `nix repl` session!
This has been fixed, so now only *successful* loads count towards `:reload`.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [Qyriad](https://git.lix.systems/Qyriad) for this.
- Consistently use commit hash as rev when locking git inputs [cl/4762](https://gerrit.lix.systems/c/lix/+/4762)
Lix will now use commit hashes instead of tag object hashes in the `rev` field
when fetching git inputs by tag in `flake.lock` and `builtins.fetchTree` output.
Note that this means that Lix may change some `flake.lock` files on re-locking. Old `flake.lock` files still remain valid.
Many thanks to [goldstein](https://git.lix.systems/goldstein) for this.
## Development
- Functional lang migration [lix#856](https://git.lix.systems/lix-project/lix/issues/856) [cl/3213](https://gerrit.lix.systems/c/lix/+/3213)
We have done it! The functional/lang framework has now been fully migrated to functional2/lang.
This means: no more `just clean` and `just install` mess and whatever because one removed a test.
The lang test suite is also getting a face lift, with an improved folder structure and restructuring of many tests.
Only the first CL of the chain is provided but there's way more changes associated to this project.
Many thanks to [piegames](https://git.lix.systems/piegames) and [rootile (Rutile)](https://git.lix.systems/rootile) for this.
## Miscellany
- Warn instead of erroring when the final destination of a transfer changes in-flight [lix#1004](https://git.lix.systems/lix-project/lix/issues/1004) [cl/4641](https://gerrit.lix.systems/c/lix/+/4641)
Lix will now emit a warning during downloads where the final destination changes suddently mid-transfer instead of throwing an error.
This transfer behavior has been known to happen very rarely while fetching from some CDNs.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
f"This function is only available if the [{self.experimental_feature}](@docroot@/contributing/experimental-features.md#xp-feature-{self.experimental_feature}) experimental feature is enabled."
ifself.experimental_featureisnotNone
else""
}
</dd>
""")
defmain():
defmain():
importargparse
ap=get_argument_parser()
ap.add_argument(
ap=argparse.ArgumentParser()
"--experimental-features",help="Directory containing the experimental feature definitions"
ap.add_argument('--header',help='Path of the header to generate')
)
ap.add_argument('--docs',help='Path of the documentation file to generate')
ap.add_argument('--experimental-features',help='Directory containing the experimental feature definitions')
'''+(f'''This function is only available if the [{builtin.experimental_feature}](@docroot@/contributing/experimental-features.md#xp-feature-{builtin.experimental_feature}) experimental feature is enabled.
f' - [`{feature.name}`](@docroot@/contributing/{"deprecated"ifargs.deprecatedelse"experimental"}-features.md#{"dp"ifargs.deprecatedelse"xp"}-feature-{feature.name})\n')
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
eldritch horrors