Related to #1162
Currently if JSON with duplicate object keys is passed into `builtins.toJSON`,
it will silently drop all but the last value, keeping only the last.
This may be surprising, but as this has been consistent reliable behavior that
users might depend on, we should test for it.
Change-Id: Icc2adefabb161530e7cbfa7330919bab6a6a6964
Without this, the primop chokes on any thunks on attributes passed in
the attrset. It even is the reason why the test contained `builtins.seq`
to work around this. Supposedly, this might have been an intentional
restriction and changing this might break things in ways I cannot forsee
due to not knowing much about Flakes, however the status quo is equally
broken:
- The error message looks like an internal error and not like some
explicitly forbidden invariant violation.
- Seemingly simple syntax literals like "-1" compile to "__sub 0 1" and
thus create a thunk which then fails, which is utterly confusing ("why
does 1 work but not -1?")
- This is a stark violation of the principle of least surprise.
- Thunking relies on maybeThunk and thunk inlining optimizations, thus
not forcing thunks turns operational details of the evaluator into
language-observable behavior. That's bad.
I am changing this now regardless of the risk of breakage, because the
bytecode evaluator will have different thunk inlining optimizations and
thus inevitably cause mismatches in behavior anyways.
Change-Id: Ifc45c4d2900e40822383670b28e4e50ab8af317a
Although the fact that some characters are illegal in derivation names
was referenced in some parts of the docs, the actual list was not
documented anywhere, and the only source of truth seems to be the
[store source code](lix/libstore/path.cc).
This adds an admonition in the `derivation` docs that specifies which
characters and strings are actually allowed (especially since some
of them can be somewhat surprising, like the string "..-foo" not
being a valid name because of `..`).
Change-Id: Ibc4b4a141e262c21306ce9c0392f92cf0abe610a
the docs say that the arguments for an overlay is info/prev/final, but
it's actually info/final/prev. this fixes the docs both in the nix.conf
manual as well as in the `nix repl` help text
Change-Id: I3de5c122e7e77cc3f4e4550e3741d6a6e906ea20
That way we can automatically push the static build (compressed as
tarball) into the AFNix S3 bucket and have it updated on each update to
main.
Change-Id: I1002727ab12c0ab6b95c8d37ae4620996607d7b7
This applies the suggested change in #1179, replacing a dead link with a
link to the content's new location. Closes#1179.
Change-Id: I543e3308516218e224e495b8cf14a7c06a6a6964
This is a small tweak to the logic added in cl/1511 to detect explicit
throws when printing stack traces. Now when deciding whether to print
"caused by explicit throw", it checks not only that the error is a
ThrowError and that we are in a throw, but also that the ThrowError was
thrown by *this* throw, and not by another while in the process of
evaluating this throw's operand.
It turns this:
```
let
set = {
inner = throw "nested throw";
};
in
throw set.inner
error:
… caused by explicit throw
at /pwd/in.nix:6:1:
5| in
6| throw set.inner
| ^
7|
… while evaluating the attribute 'inner'
at /pwd/in.nix:3:5:
2| set = {
3| inner = throw "nested throw";
| ^
4| };
… caused by explicit throw
at /pwd/in.nix:3:13:
2| set = {
3| inner = throw "nested throw";
| ^
4| };
error: nested throw
```
into this:
```
error:
… while calling the 'throw' builtin
at /pwd/in.nix:6:1:
5| in
6| throw set.inner
| ^
7|
… while evaluating the attribute 'inner'
at /pwd/in.nix:3:5:
2| set = {
3| inner = throw "nested throw";
| ^
4| };
… caused by explicit throw
at /pwd/in.nix:3:13:
2| set = {
3| inner = throw "nested throw";
| ^
4| };
error: nested throw
```
Notice the difference in the top context frame. Before it incorrectly
attributed the throw error to the both throws instead of just the one
that actually threw.
Change-Id: If3b0b8311f1ae2ff1471e260fe59d9166a6a6964
Admittedly, I do not understand what the comment means:
> As soon as Nixpkgs updates to >= 3.0.0, change to lowdown_2_0!
But this should work fine for nixpkgs versions providing lowdown 2.x or
3.x, and not cause rebuilds of the nixpkgs Lix/Nix derivations.
Change-Id: I6fb9e775335b4dbc23add76b98106970956e19fb
When index-debug-info is enabled and the store path being copied is a
regular file (not a directory), std::get_if<nar_index::Directory>
returns nullptr since the NAR root is a File variant. The loop then
immediately dereferences buildIdDir->contents on the null pointer,
causing a segfault.
Add a null check at the top of the loop to break early when the NAR
root is not a directory.
Change-Id: I3a6e792b84cc12c837ecaddf4fee889e1bcb6397
A significant part of #1136.
This is a rather large cl, because all of the primops need to be changed
together.
Change-Id: I7d92698d5344bd6186ee4fa47f5c21966a6a6964
I just copied and slightly edited the `man nix.conf(5)` docs about
`repl-overlays` to the nix repl help text (but cut out the more
complicated example and redirecting to the nix.conf docs for more
info).
Fixes#303
Change-Id: I85efaef606d8779ac66fe72cd3947d663f33fa44
This input was added in cl/2884, but was unused even then. The
`linuxPackages` within `buildPackages` is used instead.
Change-Id: I71c522ef683aa098eac0b356b22007ba6a6a6964
Calls to JSONState::add() always followed an assignment to
JSONState::value(), resulting in the new value going through
JSONState::v for no good reason. The only use of `v` outside of a
pointless stepping stone for `add` was to store the final top-level
Value before it is read out by JSONSax::result(), so it really only
makes sense for the top level JSONState to contain Value field.
Change-Id: I1758c7b770eb4b0c122e501b764dd42b6a6a6964
We would've had `build-lowdown_2_0.aarch64-linux.aarch64-linux`
otherwise. This is a bit of a problem because my way of generating a
list of constituents for Hydra's per-architecture[1] release job stops
recursing once it encounters a system.
The alternative would be to switch `hydraJobs` to `<system>.x.y` or
switching Hydra to "legacy" jobsets. For the latter I'd prefer to do the
same for Buildkite such that we don't have diverging things to build
depending on pre/post-merge CI.
[1] per-architecute because the trusted AFNix builders don't support all
the architectures we support in Lix and with Flakes there's no way
of parameterizing the list of supported systems.
Change-Id: I33f31260caf86ed5bb0f728770ca3cf1c00adf31
Closes#496
When running
nix-build ../nixpkgs --arg config.allowUnfree true -A hello-unfree
the package `hello-unfree` is now built rather than getting an
eval-error rejecting instantiation. This is because `config.allowUnfree`
is now interpreted as nested attribute-set declaration, similar to how
it's done in `nix repl`.
To prevent sudden breakage, this behavior was carefully deprecated with
Nix throwing an error if the identifier for `--arg` is not a pure
identifier, but an expression as above.
Any kind of merging is rejected. I.e. doing
nix-build ../nixpkgs --arg config '{cudaSupport = true;}' --arg config.allowUnfree true
is prohibited. That way we don't have to think about merge semantics for
cases like this (or even worse `--arg config 'rec { ... }'`). Another
nice side-effect of this is that we don't need to create an EvalState to
force the values and implement merging.
Change-Id: I8b560883a4468a3f32f915764b08f5fdd8fe71bb
While it doesn't make sense to have `foo..bar`, the attribute-path
`foo."".bar` is valid and shouldn't throw.
Change-Id: Ifcddaad6233c6ba8f17cb5c953c2101d276dfeb6
This is a little more elegant and easier to reason about than prefixing
strings with whatever type the rest of the string is.
Change-Id: I7769535303dcb9f67b79e89bef162beec990e2a0
previously, `builtins.break` didn't force its argument, resulting in
a value wrapped with `break` being opaque to most builtins if not
also wrapped with `seq`. see [lix-project/lix#1165] for more details
on what this can break.
this tiny fix just adds a call to `forceValue` inside `prim_break`,
but unfortunately this "breaks" a few existing tests because it
changes the call stack; those tests' golden outputs have been adjusted
without modifying their intended purpose.
Fixes#1165
[lix-project/lix#1165]: https://git.lix.systems/lix-project/lix/issues/1165
Change-Id: I5fe4ee3ff28b38aaf924125b8978130812e58fef
This job is used to indicate that all relevant Hydra jobs of an
architecture have built. The idea is to build some CD mechanism on
AFNix's Hydra to e.g. auto-update the nightly manual.
See https://hydra.afnix.fr/jobset/lix/demo for the current setup.
Change-Id: I41724c5884a068bbe41407ab30f8edf8e4914001
See [lix-project/lix#1156], but basically currently the `:st <n>`
debugger command doesn't allow any negative indices, and putting a plus
sign in front of the arg doesn't change anything; thus, we can exploit
that "design space" to allow users to move between different stack
frames easily, by simply prepending their arg with a +/- sign.
The actual behavior is little more nuanced when you account for errors:
as suggested by @pennae (thanks! :), when the user inputs an offset that
would result in an invalid frame index, the debugger instead clamps it
to the closest bound (i.e. 0 for negative offsets, $maxFrame for
positive ones) and just prints a warning.
Fixes#1156
[lix-project/lix#1156]: https://git.lix.systems/lix-project/lix/issues/1156
Change-Id: I02a0cdb6aaebbdb0515308880a3bf9c0d2fcd25e
The default constructor of Attr default constructs a Value, which is
itself deprecated. Therefore the default constructor of Attr must
itself be either deprecated in turn or removed. The default can't be
trivially deleted because Bindings::EMPTY depends on the default
constructor of Bindings which depends on Attr's default constructor, so
I'm settling for deprecating it for now.
Part of work towards #744
Change-Id: Ie34b08788780615c5478a0354122530b6a6a6964
This makes the removal vector a vector of Symbols instead of Attrs, and
uses a custom Compare to still be able to std::set_difference them.
std::ranges::set_difference **should** be the perfect function for this,
but because for some reason it spuriously requires
`std::indirectly_copyable<I2, Out>`, I can't use it here. This
defficiency has bee recognized before [here](https://github.com/cplusplus/papers/issues/1021),
but no one has driven the fix forward.
Part of #744
Change-Id: I6d2c016ea41e033bf38836f859541b506a6a6964
The manual for the builtins and nix.conf currently has inconsistent
indentation, which causes some of the descriptions to end up being
partially treatedas code blocks in markdown (and thus the manual).
This was simply caused by the template string for the docs having
too much indentation before the description is inserted, so this fixes
that 16-bytes mistake.
Change-Id: Ia264e3b1abb20430109029d07a2d2b0a1a726bd4
currently, the `:st <n>` command in the debugger will simply silently
fail if the argument cannot be converted to an integer, or if it falls
outside the range of valid stack indices. this isn't too big of problem,
but it can be nicer to tell the user something went wrong, rather than
not give them any output and having them guess (esp. in the second case).
this commit adds two errors, one for each case:
1. the argument is not actually an integer, or is outside INT_MIN/MAX
-> "argument '%arg' is not a valid integer"
2. the argument is an integer outside the range of stack traces
-> "stack index must be between 0 and %max_frame, but was %arg"
Change-Id: I8109feeede79a9ad3db9ee7dc95d37e7dd19741a
NixOS 25.05 which distributed lowdown < 1.4 has been EOL for a bit now,
25.11 ships lowdown 2.0.4.
Dropping support means we can tweak the lowdown options for terminal
output which have been added in 1.4.0 without having diverging behavior
in possible builds of Lix.
Change-Id: Icae97cf5e9e680766b8a6f4e85514f4c4625d1dd
lowdown 3.0.0 merged some flags into one to save on bits and did not add
any aliases for backward compatibility.
As with the changes for lowdown >= 1.4, we define a preprocessor flag to
gate the changes on and add a job to CI to ensure that lowdown < 3.0
keeps working (which is used by NixOS 25.11).
Unfortunately, we need to jump through some hoops to prevent nix and lix
from upstream Nixpkgs from being rebuilt due to a changed lowdown. Since
both implementation's packaging in Nixpkgs has their own package set /
fix point now, we can't simply inherit them from `prev` since they will
always be (re-)computed from the `final` fix point. As a consequence,
we need to expose our changed lowdown version at a non-default attribute
or break the builds of Nixpkgs derivations we test against.
Change-Id: I20a3e2fdaa05906f032ff66911c42867557fdd11
base classes
`ExprLiteral::v` needs to be initialized in ExprLiteral's constructor,
but at the time that it gets initialized subclass fields don't yet exist
so it can't reference them. Previously, `v` was default constructed and
then later assigned a proper value, but this creates a problem when
attempting to remove all default constructions of `Value` from the
codebase. This commit moves the backing fields of each of the subclasses
into new base classes, which exist just to make sure they get
initialized before the `ExprLiteral` base class.
Work towards #744
Change-Id: Ic6d24cab474460b113f2fbcc8d92ad266a6a6964
ahead of time
Rather than creating fixed size vectors of default constructed Values
before assigning to those elements, reserve the desired capacity and
then push created values onto the vector. This avoids default
constructing any Values.
Part of fixing #744
Change-Id: I36eff4275a893b181eaf3ce145b1ee446a6a6964
Progress towards #744
`alloc` default constructed a `Value` which is a problem because the
defaut constructor of `Value` is deprecated
Change-Id: I789cba20bd98728758395080a3a9cf6e6a6a6964
- launch-builder-linux.cc was missing an include for musl
and used function that are not defined in the launchers
- musl caches pids used for raise, breaking sandbox setup
- the mtls contrib plugin won't build, didn't try fixing,
static builds can't really use plugins reliably anyway.
Change-Id: I5ab1664e45ea977e5bcf05e41d825e6014e62146
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh
Change-Id: Ia7a63807681389cf5ccb9d07539b880697a88be5
This avoids a large list of CLs when a user can discover all the CLs by
browsing the chain.
Change-Id: I1890eb87e8bf6f930d3d00c3f7156690bb8de068
Signed-off-by: Raito Bezarius <raito@lix.systems>
This might do more copies than are necessary. I think that's fine.
Sorry, :reload not reloading has been bothering the shit out of us.
Fixes#1115.
Change-Id: I7f4caca3dad39696ce66ce57b1a520836a6a6964
So far, we had both `fj#` and `lix#` issue references in our release
notes, due to supporting both in the md files.
This commit unifies those in order for all references to be consistent.
Change-Id: I67c75784b460bfe4ebf214c2f6ac105cc60a7bad
kj can and will throw exceptions immediately without wrapping them as
promise results, and we must take care of these as well to not crash.
fixes#1150
Change-Id: Ifa29d0f3da4a6490f3e45d4c04a8fa49eaa20f43
mdbook-linkcheck was dropped from nixpkgs and replaced with mdbook-linkcheck2,
which is not entirely a drop-in replacement, but drop-in enough for us.
Change-Id: Iab03efab89ebae70811e0de3ea7cf0a9a059a9e4
Since commit ac64c727b5, during launch of the
builder it is attempted to raise all capabilities into the ambient set.
Specifically, what "all" means here is determined by the Linux API headers Lix
was built against.
Occasionally, new capabilities are added in the Linux kernel, leading to
PR_CAP_AMBIENT_RAISE failing with EINVAL on the newly added capabilities if the
API headers are from after the change but the running kernel is from before.
Similarly to how capset already silently ignores nonexistent capabilities, we
ignore this error so the builder doesn't fail to launch unnecessarily.
Unfortunately it is very hard to test for this situation currently, since the
last time a capability was added was CAP_CHECKPOINT_RESTORE in Linux 5.9, and
all kernel versions in nixpkgs are newer than that.
Change-Id: Ibeb2f0757729b877bd3ca9f02e1aa4536a6a6964
not doing this has lix crash if stdio forwarding fails for odd reasons,
such as ssh failing to set up a connection due to transient dns errors.
cf #1148
Change-Id: I543b7444236e69d3c13b47fb6c6eb37c65b96763
This is intended to be a softer version of
`allow-import-from-derivation`, allowing for users to only flag when IFD
is encountered rather than error out
Based-on: https://github.com/NixOS/nix/pull/13279
Co-authored-by: gustavderdrache <alex.ford@determinate.systems>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Change-Id: I879a444e3a24b4e82a8cea3a5a94834c78c3924a
`builtins.storePath` allows you to use "external" knowledge of present
store paths in your store to be used in your expressions.
This is sometimes used to avoid double wrapping store paths, for example
when dealing with nixpkgs sources (`builtins.storePath pkgs.path` is a
frequent pattern).
Unfortunately, using your external knowledge of present store paths is
technically impure because Flakes is about not depending on the *STATE*
of your store, no matter what you know about it or what you do to ensure
that your knowledge is accurate.
That being said, `builtins.storePath` is already possible in the
fragment of "pure Nixlang" via a clever use of `builtins.appendContext`.
Precisely:
```
storePath = path:
let path' = builtins.unsafeDiscardStringContext path;
in
# NOTE: merging the context set in all generality is impossible
because getContext on a pure path doesn't work.
builtins.appendContext path' { ${path'} = { path = true; }; };
```
works in pure contexts.
Because of this, we deem that storePath should be allowed in pure
contexts and users do not have to come up with a `storePath` polyfill.
Fixes#402.
Change-Id: I5fff750a4feed082745431835db451f945249cfb
Signed-off-by: Raito Bezarius <raito@lix.systems>
nix3 CLI never honored that setting.
When we update a nix3 profile element with new built store paths for
which we know the deriver (the derivation path which built it), we add
the derivation paths to the profile element to prevent garbage
collection the same way the nix2 CLI logic does.
Fixes#1095.
Change-Id: Icc236d174c5ce5ffe042d1c8e1968f6cb5b50359
Signed-off-by: Raito Bezarius <raito@lix.systems>
Instead of showing untrusted configuration items one by one, collect all
of them into one batched ask with 4 options:
- apply the configuration wholesale **for now** (yes)
- apply the configuration wholesale and trust it permanently (allow
permanently)
- selectively choose subset (no)
- rejecting the configuration wholesale (no to all)
Note that we do not offer options to reject permanently a configuration,
that's what `--no-accept-flake-config` is for.
Individual prompts gain a new option to distinguish "yes for now" and
"yes forever".
Fixes#682.
Change-Id: I2492717ac277af33ea4ac9b1a11bd7d6d1bfe0db
In messages printed by the evaluator, we use showType(.) to say things
like "expected X, got Y" where X, Y can be string, string with context,
etc.
After changes in the Lix data model (pointer tagging and friends), paths
got squished inside a internal string type.
We missed updating `showType` to take this fact into account which
caused an unfortunate:
`error: expected a string but found a string with context:
/nix/store/wjf8rdpp63rhyasbzc7zfms1agf7pwyd-source`
while using a path on `builtins.getContext`
A test is added.
Change-Id: Ifb0129ead2ea1973fc17dc0133f8a694995536b1
Signed-off-by: Raito Bezarius <raito@lix.systems>
This partitions a string into two parts based on the first occurrence of
the separator.
This function handles the case when the separator never occur in the
target string by returning an optional<T> in the second argument of the
pair.
Change-Id: I9ad44375dbaa137a4d21d2408bc2ffa34fab62a7
Signed-off-by: Raito Bezarius <raito@lix.systems>
Parsing and completing enumerations on the CLI was usually done by hand.
With this enum trait, the parsing and completion is automated away.
Change-Id: Icd49729564355d4c79e2c8050ce7e8b9d8e7eda3
Signed-off-by: Raito Bezarius <raito@lix.systems>
36c4d6f592 introduced it commented. Zero
explanation why.
horrors suggested to drop it entirely because it hide the options in
categories but they are pretty common and we should make as much visible
as possible.
Change-Id: I09894b5ba0f2b2cce5869136914289fed034fce3
Signed-off-by: Raito Bezarius <raito@lix.systems>
When `max-jobs = 0`, our async semaphores have null capacity and
therefore no slot token can be acquired.
In addition, a derivation with `preferLocalBuild = true` with `max-jobs
= 0` which is not a built-in derivation is not considered suitable for
being possible to build.
Our documentation in `max-jobs` writes:
> `0` is useful when using remote builders to prevent any local builds
(except for preferLocalBuild derivation attribute which executes locally
regardless).
A valid interpretation is that `preferLocalBuild` derivations (all of
them) will always execute even if `max-jobs = 0`.
This CL achieves this promise by enabling `preferLocalBuild = true`
derivations to draw in priority capacity from a `preferLocalBuild`-only
pool and then fall back to the build pool (`maxBuildJobs`).
If there's non-zero max-jobs, we make the `preferLocalBuild`-only pool
empty.
If there's zero max-jobs, we make the `preferLocalBuild`-only pool
non-empty (capacity: 1).
Remark 1: if `max-jobs = 0`, no non-`preferLocalBuild = true`
derivations can be built without a remote builder and `preferLocalBuild
= true` derivations can be built one at a time if `extra-local-jobs` is
not overridden.
Remark 2: if `max-jobs != 0`, no `preferLocalBuild = true` derivations
can bypass capacity limits, i.e. `max-jobs` unless `extra-local-jobs` is
overridden.
Fixes#855.
Change-Id: I00a0acac7375a881aec6281ebb13a4530b26e460
Signed-off-by: Raito Bezarius <raito@lix.systems>
This is going to be used in the extra-local-job future setting.
Change-Id: Ied76f32b8fa97cb71691d76167103001aa14fa01
Signed-off-by: Raito Bezarius <raito@lix.systems>
When ref/rev are very similar looking, e.g. a revision SHA1, the logic
to reconstruct whether we had a ref or rev initially cannot work.
This regression was introduced by
I2c72d5a32e406a7ca308e271730bd0af01c5d18b which passed the information
as an attribute field called `refOrRev`, instead, we pass `ref` or `rev`
but not `refOrRev`.
We apply the same post-processing logic to distinguish rev/ref in
path-style flake references: `github:owner/repo/$refOrRev`.
Fixes#520#841.
Change-Id: I2b1985cd5f351f43fea84232c1972715e2408fb1
Signed-off-by: Raito Bezarius <raito@lix.systems>
In structured attrs derivations, __json is used to serialize the `env`
attribute into a JSON object.
If the derivation enables structured attributes AND specify its own
`__json`, one of the field will take precedence (the reader can have fun
by guessing which one).
To reduce underspecification, we disallow `__json` in structured
attributes derivations.
Fixes#380.
Change-Id: I51c3b2af1ff9449471ba81d4c72df6a7a263eef7
Signed-off-by: Raito Bezarius <raito@lix.systems>
`nix.full_path()` is the *built* path, not the *installed* path. in the
sandbox this point to a location that contains *only* `nix`, not any of
the old-cli symlinks. any test that needs them in path would thus fail.
Change-Id: Ia651db8d1151a84f8d1b52e8993f6db61b0fe104
Without the `--git-dir` git will attempt to figure out which the git directory is on its own.
This causes issues when the `GIT_DIR` envvar is set in the environment for other reasons (such as being part of a `git rebase`).
Providing this parameter overrides the environment and thus makes sure the revs are always coming from the intended repository.
Fixes: #1135
Change-Id: Ibd43cfd8e0278463c0e1ccf86779982947a6038a
Signed-off-by: benaryorg <binary@benary.org>
To stop spurious warnings, we will stop this bizarre behavior of setting
empty values to impure environment variables.
On the warning side, we verify that the presence of the environment
variable.
The corresponding test is updated to "there is a warning" to "there is
no warning".
Change-Id: I12f5c6445ef00a83d269488d7aed0b0f61aeec12
All impure environment variables are initialized as part of `initEnv()`
to an empty value if they are unset.
Whenever `NIX_SSL_CERT_FILE` is set, this cause a spurious warning.
Let's document it here as a test before we fix it.
Change-Id: Iccba8ccb487d9cc25490e940ff09c1c5a13ae097
Signed-off-by: Raito Bezarius <raito@lix.systems>
CA tests have grown quite complicated and are hard to debug and
manoeuver for new needs.
They are a prime target for F2 migration and they were extremely
pleasant to write.
Most functionality is preserved or improved with additional behaviors
asserted on warnings and the full set of reports.
Change-Id: I70cd52abbe5ff13f56744b966ab00962ad3a49d6
Signed-off-by: Raito Bezarius <raito@lix.systems>
Instead of taking in a final argument `Value &` out parameter which it
writes to, it now returns its result
Change-Id: Iab6bc3a3ac6a4b17c6d31115a766a6ea6a6a6964
we already report errors that happen during reads via exceptions
rethrown from read. reporting errors during destruction too is a
source of confusion for users since these errors are not usually
relevant and just the decompressor thread reporting short reads.
shorts reads during shutdown are not an error, they're expected.
fixes#1138
Change-Id: I7983ef0ed7722460f1d683bc8171a454b3d06e90
I could instead add the right braces or manually ensure the
XMLOpenElement destructor is run at the right time. Or I could just
write what I actually want to happen.
Fixes#1137.
Change-Id: Ie1f157de1632571fb0c21e26d8e3cae56a6a6964
This is a rebase of CL 1626 after the performance problems that blocked
that change are no longer an issue.
Change-Id: I4a17f7606c5fdc9a924933a860eb401f6a6a6964
The pseudo-constructor `Value::mkFloat` would previously be called on an
default-constructed (under-initialized) `Value` to create a properly
initialized `Value` that represents a float. This change removes mkFloat
and constructs float `Value`s directly.
Change-Id: I48ae3a836842ce5e5eda6323404ba7576a6a6964
The pseudo-constructor `Value::mkStringMove` would previously be called
on a default-constructed (under-initialized) `Value` to create a
properly initialized `Value` that represents a string, reusing an
existing GC allocated Str. This change turns that method into a proper
constructor
Change-Id: I8d7a97e2afce231a04ecf56fdd6d22d06a6a6964
it sometimes happens that one forgets to add `.run().ok()` to a command,
leading to things not actually being tested, potentially even slipping
though code reviews!
This commit adds a check at fixture teardown time, checking that all
commands have been run. Sadly, due to when this check happens, we cannot
add a test checking for this behavior
Change-Id: If3df2c20c267b9f03ae0ca099b77c57e2310f335
this includes tests that fail teardown when *skipped*, which confuses
the tap parser by reporting more tests than advertised in the header.
not sure what to do about this though, and it is better than nothing.
Change-Id: I4fc6badacbfa3b3d5f5f4a4ad76b177379bb4a30
pytest-tap didn't get it right in combination with xdist. we need to
replace the reporter *completely*, which is just incredibly weird :/
Change-Id: I2e35b9dc8ea4b24563b9b3306312778781eb674e
The pseudo-constructor `Value::mkPath` would previously be called on an
default-constructed (under-initialized) `Value` to create a properly
initialized `Value` that represents a path. This change
removes mkPath and constructs path `Value`s directly.
Change-Id: I9021de1ff59490828f9fec6866e083996a6a6964
The pseudo-constructor `Value::mkAttrs` would previously be called on an
default-constructed (under-initialized) `Value` to create a properly
initialized `Value` that represents an attribute set. This change
removes mkAttrs and constructs attr `Value`s directly.
Change-Id: I11cd801eefecd454a7a5b6229d6770d06a6a6964
This is not ca-derivations. We don't care about self-references or any of the
rewriting or stuff like that; if you want something like that, write
your user code so it figures out where it is.
The reason I want to do this is for integrating other build systems with
Lix: if you're importing something built *outside the store* with a
non-Nix build system, it makes no sense to put it in an input-addressed
path since it's not possible to come up with a derivation for it.
But you need *some* hash, so the output hashing is the most sensible option.
It is also nice because of not needing trusted user privileges to import
it, since the hash describes the exact contents of the output path.
We use this feature (implemented externally to Lix; this CL eliminates
that step) at Mercury to integrate buck2 with Nix on the output side: we
import things to the Nix store as ca paths with references.
These can then be consumed by Nix language with:
```
# Hack from https://git.lix.systems/lix-project/lix/issues/402#issuecomment-5889
path:
builtins.appendContext path {
${path} = {
path = true;
};
}
```
Test plan:
```
$ nix store add-path --references-list xx-refs-list ./README.md
/nix/store/szcwnm13d9gmxx8fly0bz82l42jgysw6-README.md
$ cat xx-refs-list
/nix/store/g9hhwjbkdrw0wnbd3axfs4icyb37nr6b-bash-interactive-5.3p3
$ nix path-info --json /nix/store/szcwnm13d9gmxx8fly0bz82l42jgysw6-README.md | jq .
[
{
"ca": "fixed:r:sha256:1pj59fy7zcrn949ry9kgv4ba17h2sy3z2is469abj912cvbl14n3",
"narHash": "sha256-w5JA12YiJLlUMkRH8YfXAp6gFtlvJp8TSTazf7xLRd4=",
"narSize": 1384,
"path": "/nix/store/szcwnm13d9gmxx8fly0bz82l42jgysw6-README.md",
"references": [
"/nix/store/g9hhwjbkdrw0wnbd3axfs4icyb37nr6b-bash-interactive-5.3p3"
],
"registrationTime": 1771266146,
"valid": true
}
]
```
Change-Id: I005a03003dfc24108e018e599dbe0b5d6a6a6964
this requires some hacks to make error reports visible, but hey. it's
better than no progress reports at all, and successful runs no longer
spew huge amounts of useless log output onto the developer's terminal
Change-Id: I9e4766b2f825a4ec451e117eb4609db65f328785
Fixes path resolution issue with meson-embedded bash script when
`build` is a symlink, or a custom build directory is passed as a
argument to `just` or `meson`.
Change-Id: I67b13c524d196241bb9a8a3b8ad2c17c2c094ed8
not all operations that involve workers need to build. for example nix
copy --from calls instantiate thir worker with the *origin* store, not
the store they copy *into*. for binary cache stores this is not likely
to ever work if cgroups are enabled and the copy is run on normal user
credentials, even though no cgroups will ever be needed to copy things
fixes#1088
Change-Id: I065e29e1a1d0f58d81823609ef0701ff82cdd1d5
Affects `github`, `gitlab` and `sourcehut` fetchers.
Previously this was an assertion, which crashed the evaluator
with SIGABRT.
Fixes: #1133
Change-Id: Ia9bb8dd29ac8b9c97bf048827f62c5076a6a6964
overridden-ness can be reset, and is usually reset such that setting
value taken from a config file are not considered as overriden. when
launching builtin builders we *do* want to send config file settings
changes to the builder, so we'll need one more getter method for it.
Change-Id: I861538a469121c77ebc1898a276439e6b756797d
diverted stores are only necessary when the logical store paths of
objects matter for the test itself, such as for derivation hashes,
substitution from golden sample nars, or actual tests of the store
diversion functionality. all other tests can use undiverted stores
to run, especially since only linux can build in diverted a store.
Change-Id: I62f0907bdef9961609af22b610195fcec54c1e57
When the destructor of PathSubstitutionGoal is run, this happens in a
sync context and can cause starvation of all ongoing IO w.r.t. to other
substitutions, including our own substitution.
While there's only a decompressor thread per stream, the other side of
the IO runs on the event loop.
In order to fix this, it is sufficient to remove the thread indirection
and inline the async code.
Fixes#1126. Great thanks to horrors' patience.
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I3eb37bc37d156f0f5528364e568fdaa2ced58011
Signed-off-by: Raito Bezarius <raito@lix.systems>
Looks like too much stuff is breaking on this, so let's make it only a
warning for now to ease transition
Change-Id: I52d50ceb1fe1fbe9f4e28d0aabf3537f7e4d52d1
Explicitly catch common errors (trying to escape line breaks, badly
escaping interpolations) to provide better messages and user guidance
for these cases.
Change-Id: I3dd1b2ad3bca33be393e65be5e72f4fb9544a46a
StreamFdMessageReader reads message data *lazily*. if you don't access
all segments of the message before you close the underlying stream you
may find yourselv reading from something *very* different, and in this
case that something is The Void™. this causes reads to fail, writes on
the other side to fail to match, and finally our build launch to fail.
this does not lead to happy outcomes, so we will copy the full message
into a fresh new buffer *before* we try to access any of its contents.
fixes#1118
all-analysis-by: deprekated <kate@lix.systems>
Change-Id: I105540831fde855817194e9e539acf177f54a6f4
This is the long awaited refactor of the NixSettings.
It allows one to set, unset and update any and all settings with a neat
and easy-to-use interface
closes#846
Change-Id: Id4cfb5f853cc1168b506a1f6f405076f3a7cab65
if a substituter is entirely offline and cannot be queries at all we
should not be failing if other substituters are configured. likewise
if a substituter goes offline after querying but before we try using
it we should attempt to fetch that path from some other substituter.
ideally we'd treat all substituters as a single entity instead of as
one store each, then have that single entity take care of fallbacks,
retries, error reporting, etc. that requires larger rewrites though.
fixes#1061
Change-Id: I9d8fc0544ff380bf017256e8fcc82823dc634f10
we don't use std::cerr often enough to want this, and since cerr flushes
after every operation anyway it never did anything useful to begin with.
Change-Id: Ia54be340826da5073e9a1786c463555f4f0e491c
kj exceptions get wrapped as Error instead of passed through as is.
luckily this also means that we can add context to them very easily
Change-Id: Icedab6c016f4434447dd38ba14138c102fc6149a
oops. we have to use init* here, not get*. get returns a discarding
builder for something that was not inited previously, but the linux
cause worked anyway because the first member is created by default!
Change-Id: I40f8a12a04eef2f4e3a80d1537ac9b975490a027
this allows us great flexibility in how children are launched (since the
actual launching is done by a separate executable), makes fork no longer
needed in the core codebase (outside of runProgram, anyway), and we even
get to use linux vfork to its full potential to decrease the launch cost
of sandboxes to a constant factor (previously it was O(#drvs + #deps) of
the build graph, which obviously goes to n² quickly if you are unlucky.)
Change-Id: I66e2d1b20242dc24d708666ef325fb8725bd9296
we can have a ProcessGroup for a pid that *should* be a process group
but hasn't gotten around to setting its pgid yet. in such cases we do
want to be killing the thing anyway, not shoot into the void and hope
the right thing falls over. so far this has not been a problem due to
a mixture of just not having done this and being slow enough to work.
Change-Id: I4e0e54513252d8e18256b9286b819bfa957d70dc
if we move this out of libstore and don't want to pull in all of libutil
with its myriad side effects std::filesystem is our best option, by far.
most notably we don't replace pathExists because std::filesystem::exists
behaves like `stat()` on symlinks, not like the `lstat()` as we require.
Change-Id: I1e488418dcabb33f2ebb73d8c3d1b43528aa51f1
since we're using tmpDirInSandbox as the working directory for the new
process we also rename it accordingly. buildUser likewise turns into a
different Credentials type because exposing user lock state seems odd.
Change-Id: Id4a1a6eb733f774c893f373b91c2a271a4b84185
setting the signal is not enough, we must also check that the process we
expect to be parent to actually *is* our parent, not another process (eg
init if the daemon exited). we also have to set the death signal *after*
all set[ug]id calls, otherwise it will be cleared again by such changes.
Change-Id: I4e8c9102ea407576ed85b3203c8bb9bfb56762de
it's only used this once, and it accesses a (cached) global variable for
the filter. we want to move all sandbox setup state into a single object
soon, moving filter setup will make it a little bit easier to deal with.
Change-Id: I234d92d5ca044a16644b70bd303bfb7956c120f0
when starting builders we want the inner processes to run with a blank
slate. if some signals are masked for any reason the builder processes
may attempt to send signals to each other that are never delivered; we
avoid this by unmasking all singals. since every build tree also has a
session and process group of its own we are not in danger of sending a
signal to a builder by accident from any source, so unmasking is okay.
Change-Id: I90720ed2bd44502ffb6d2bb848c05369809abeba
we don't need to mess with this rlimit for e.g. the daemon. increasing
the limit later should always be safe since we don't allocate (or map)
much before constructing the eval states that ultimately fill our heap
and could thus make stack expansion impossible after some time passes.
Change-Id: Ieafda537fbc99a6a7f83a093a981e7df947da437
this will be needed in other places in the future, and splitting it out
also clarifies the surrounding code. the keep-caps dance launchPasta is
doing doesn't have to be moved since it is only needed to allow setuid.
Change-Id: I6baaa138c2b1bca9626971ed3266c1a971a63acc
the devshell no longer worked for us due to the recent deprecations :D
let's update pre-commit too because while it doesn't fail it does warn
Change-Id: Ic2060c82e4e97bb7a96cebd29097abefabdfe733
it's ancient, it does not do what it was added for (improving
determinism), and it probably hasn't been needed for anything
in a very long time. we can probably get away with killing it
Change-Id: I66f5c841964d9ad62726cb2dae8d6fc42095399f
mostly because this will make it easier to change the data source for
them, and especially to change to data source to something immutable.
Change-Id: Ic5f1a55d02d4c2cc928c3f8b308d07167efb467b
closing them as soon as we're done ensure that the fd table doesn't
balloon in size, which in turn lets us build larger graphs. it also
makes forking processes cheaper due to less fd table copy overhead.
Change-Id: I233d1b7358ffe1b05680f845cd99de38b179f449
In Nixpkgs, there are several strings like "\d\.\d" which attempt to be
a regex but are just literally "d.d". The escaping rules are silly and
we should warn our users about that.
Co-authored-by: Commentator2.0 <lix@crystal-cavern.systems>
Change-Id: I779b0757358fc9adc34dc140e1670b83abc93b67
The indentation stripping semantics of strings are fairly bad and have a
few gotchas where they behave unintuitively. But the good news is, that
these cases are easy to catch and can be avoided.
This commit adds a warning in the parser when such strings are detected.
Unfortunately Nixpkgs uses this kind of a lot, so we won't be able to
actually enable this warning for a while to come.
Co-authored-by: Commentator2.0 <lix@crystal-cavern.systems>
Change-Id: I3b3b68c2eee4cd70959d3f4ca643cb6caf3a2217
Unfortunately, the previous code only detected superficial __overrides
and did not catch cases like `rec { __overrides.foo = 2; }`.
Change-Id: Ic38bdef1a6a3bdea91915aef44447f4ecc238259
Changed the writing style of the descriptions, expanded with more
examples and rationale, and added the new timline metadata in the
frontmatter.
Change-Id: I218389e3504fc21f4eb45a927e77a41a1a70d4f5
The code within the code-generation had tons of code-duplicates and was
overall quite meh to read, understand and expand.
This commit refactors the code-generation to make it more readable and
comprehensible, while also unifying its usage a bit more
Change-Id: I3a5df8b7d8d9b6c76e02ef47dfb151c7dab7d0ab
lix, where modularity is made up and the patterns don't matter. merging
everything to match actual behavior of the system makes using rust much
easier because we'll have only *one* library to link, not half a dozen.
Change-Id: Ie3cb1db659798511128716402497697320bd0484
starting pasta as soon as we have all namespaces it must be in available
lets it start up while we finish creating the sandbox. this may speed up
sandbox launches somewhat, but likely not enough to show up in practice.
Change-Id: Id6724cbdc48b99284cd7ba7b56c98829d74557c5
mostly as a prototype for other process launching optimizations, but
also to move linux-only bits out of runProgram2. we want to stop the
fork+exec dance we do now and migrate to posix_spawn, but linux will
always need an amount special handling for launching build children.
Change-Id: Ied55dc0de95c05b52aaca923be2e3c1d101f47e0
we'll use this to run programs from a vforked child instead of using
runProgram. ideally we would like to have runProgram be as efficient
as possible, but our mount namespace handling makes this harder than
is worth dealing with right now. the linux derivation goal is a good
place to prototype functionality like this, especially since we need
fine control over credentials and file descriptor tables of children
Change-Id: Ibc356613ae10b06ea81de9575611948f5ee353b6
this was racy previously; a builder may have spawned processes before we
added it to a cgroup. these processes would not be registered correctly.
Change-Id: If60a64a798ee806d5701590a83d1015f55d7b867
this halves sandbox launch overhead, reducing the build time for 3000
trivial runCommand derivations on our machine from 80 seconds to 44s.
as a nice side effect we also get better error message propagation in
some cases, most notably setgroups failing when run in lix sandboxes.
Change-Id: Ia7c50a844915d9b8a20475b90b2d0179fd2fff34
creating user namespaces early means we don't need pipes to signal that
a namespace has been created and set up. we'll also create the netns at
the same time because they're privilege-bound to the userns, and having
both available will let us launch pasta earlier to have it ready sooner
Change-Id: Ie66fe2ab7f50a520fb81dd90944384e168d94476
permissions and ownership will be the same since the sandbox process
will not have changed uid/gid or umask by the time it gets around to
writing these files. all of these files contain some information the
host must provide, be it contents or configuration info of some kind
Change-Id: I0b57cb0699fd29e798ccf330dff99f571f3c0692
this way we can use it anywhere without fear of it being uninitialized.
only the linux platform bit uses this anyway, so we will move it there.
Change-Id: I35e207eec91daa8aa327c4f8b36c0dc8e703a7c4
`xucred` already contain `cr_ngroups` and `cr_groups`, we just have to
use it properly.
Change-Id: Ic29b4c551f9d93e86e9ed2bef20889d27d5438f4
Signed-off-by: Raito Bezarius <raito@lix.systems>
Two things happens here:
- we consider supplementary groups that are known for authentication,
fixes#968.
- we check supplementary groups if they are our build users group and
throw an error if they are (new behavior).
Nonetheless, we did not remove the ability for `matchUser` to recurse
and check for groups that the user may have but the connection might not
have communicated as part of their groups, i.e. if a process reduces its
list of groups via a call to setgroups, it will still be authorized.
This will come in another commit.
The authorization NixOS test has been extended with a store ping test
running in systemd with DynamicUser=true *AND* a supplementary group in
allowed-users.
Co-authored-by: Tom Hubrecht <github@mail.hubrecht.ovh>
Change-Id: I25b2b8304d66a04651cea523b5585a5d15ceebe8
Signed-off-by: Raito Bezarius <raito@lix.systems>
On Linux, SO_PEERGROUPS is an option introduced in 2017, that is, in
Linux 4.13.
Thankfully, Lix targets 5.10+ kernels minimum.
It was chosen to allocate 128 gids by default and ramp up (2x) as needed
rather than allocate a full 65k of integers as it seems wasteful. I bet
the time to the 16 allocations should incur at most an additional
millisecond on a modern system, don't quote me on that though.
This is preparation to enable ability for the daemon to vet based on
supplementary groups.
Related to #968.
Suggested-by: alois31
Suggested-by: eldritch horrors
Change-Id: I26d698327db5d174bf70ca25b0afede132bd9169
Signed-off-by: Raito Bezarius <raito@lix.systems>
Rather than a bunch of booleans, use optional<T> as a way to convey that
the identifier is known or not.
This is preparatory work for SO_PEERGROUP usage.
Change-Id: I0e3ce69e4103f9df5a2161c695bfc650123c1549
Signed-off-by: Raito Bezarius <raito@lix.systems>
we can already handle this case (by forbidding any ~/... paths in option
values), and since builtin builders are now run via exec we may not have
a home dir set, or even available, for them to resolve. loading a config
file in this situation will fail with an error even if the config is ok.
fixes#1109
Change-Id: I3491b31558871faa34766462747e3c15d1eccc64
xattrs are revealing some unfortunate brittleness in real world
derivations that are getting -EINVAL errors while executing their test
code.
The reason for this is that Linux checks [1] UID delegations for xattrs
writes.
In the Lix sandbox, except if you enabled a uid-range feature, you have
exactly 3 UIDs: root, nixbld and nobody.
If your test code makes use of UIDs which have not been delegated, you
will receive an EINVAL on your operation. Test code is not resilient
with respect to the number of available UIDs in their namespace.
To avoid further issues for end users who are running into spurious
derivation build failures, we forbid xattrs again for now.
For more information about the plans, please consult or chime in [2].
Fixes#1105. Reopens#838. Fixes#1103.
[1]: https://elixir.bootlin.com/linux/v6.18.6/source/fs/posix_acl.c#L257
[2]: https://zulip.lix.systems/#narrow/channel/9-Store/topic/disablement.20of.20xattrs/with/5295
Change-Id: I864066b34cd8319d2271fac1b179cb4f950d836e
Signed-off-by: Raito Bezarius <raito@lix.systems>
this will let us configure more than one socket to connect/bind to,
which in turn lets us use posix acls on sockets for access control.
we will also need something like this for the final rpc transition.
Change-Id: I9c39f14906e9bf809055ab5c94bf687745b4f69e
now that builtin builders are regular executables we no longer need to
treat them specially during sandbox launch itself, only while we build
the command line and environment for the sandboxed process. we are not
far from being able to extract platform-dependent sandbox launch code,
ideally moving all of it into (much more replaceable) libexec helpers.
Change-Id: I9b7041314683c56cd70eec9b1b4eae6de228883f
this means that builtinFetchurl runs in a real process now, and thus we
no longer need its workarounds for running in a forked process. forking
dropped the signal handler thread and broke the curl state via sharing,
neither of which happens any more now. we can run fetchurl builtins and
their actions straight from the main thread of our builder now, and the
temporary files and settings overrides we did are now also unnecessary.
Change-Id: I738171bc120ffcd541b7ff1424fed7924c2cdc1d
this will let us migrate the fork+do_stuff combo of builtin builders we
have today to fork+exec of this new command. we use a subcommand rather
than a simple libexec helper because we would like to avoid linking all
of lix into the simple libexec helpers if possible. this is only hookup
for the builders, none of the buiders are migrated here to ease review.
Change-Id: I9358f1f3fee6ca640c81a7bd865128ae0d0e35a5
call them with their individual parameters instead. this will make it
much easier to turn them into executables we can exec in the sandbox.
Change-Id: Ib49cf44715b2a480fecadabd21216c69cd730ef5
not having the optionals set when the builder runs (e.g. because FOD
hashes were not configured correctly) could cause assertion failures
in the builder process. while this should usually not happen we *do*
want to print a useful error message here instead of a crash report.
Change-Id: I81fb1d45fcbf660d0d9a7f0c0a12c38379ecd4c6
See: cl/4840
When importing Python modules, we include `functional2` in the module
path, like this:
from functional2.testlib.fixtures.env import ManagedEnv
This means that python expects to see a file like
`functional2/testlib/fixtures/env.py`. We run `pytest` from `tests/` in
the `justfile` and have `tests/functional2/__init__.py` so `pytest` in
`meson` is able to find these imports.
However, language servers generally consider the `pyproject.toml` to be
the project root, so (e.g.) `pyright` is unable to follow any of the
`functional2` imports, leading to lots of spurious errors.
In cl/4840 I moved `tests/functional2/pyproject.toml` to
`tests/pyproject.toml`, which worked but was considered aesthetically
unappealing.
This diff is much larger but it's a more elegant solution.
Change-Id: I2983c7b87f88f59a4e3521451a9f5acd6a6a6964
like diff hooks before the build hooks also use runProgram features that
aren't present in posix_spawn. just like diff hooks we do not expect the
build hook processes to be so fast that launch overhead matters somehow.
Change-Id: If3c33dbd7d2ac20c95886e06d24feda312946c78
use a wrapper type instead. whether something should be treated as a
single process or a group is a type decision more than a runtime one
Change-Id: I966e998c19e74be64ddd72ab11af809d001f61b5
diff hooks use uid/gid switch functionality that is otherwise only
needed for linux sandbox setup and unsupported by posix_spawn. not
doing these switches inside lix core code may let us move to using
posix_spawn for most process launching in the future, and for diff
hooks the added overhead of a wrapper program really does not hurt
at all. diff hooks are expected to be expensive in terms of output
size, process launch overhead is not likely to even be noticeable.
Change-Id: Ifa4b3eedef237632db3eb88d10e6469acae01f9e
it's kind of weird to not allow diff-hooks in single user mode unless
that user is root. maybe that's why we have no tests for them? we can
at least add a test that diff hooks are run at all when we expect it.
Change-Id: I54d623b5416acda1c205cc062b0f3a33c9f4aaa7
nothing except the linux sandbox actually *needs* this for correctness.
the linux sandbox only uses it to ensure that builder uids are freed up
when the sandbox is torn down, and even *then* it only works as we need
it to with PID namespaces enabled since the parent death signals is not
inherited across fork, but when pid1 of a pid namespace dies the kernel
also kills all namespace members. in all other cases this flag does not
help us that much because it actively prevents child processes cleaning
up after themselves, possibly leaving more trash around than otherwise.
Change-Id: I9ae0a9e91833a91d1011375dda402ac6c0a7ef6f
Adds an example plugin implementing an mTLS-enabled binary cache store
(https+mtls:// scheme) using client certificates for authentication.
Darwin fix: don't link liblix* into plugins (host resolves symbols at
runtime via dynamic_lookup). Explicitly link curl so it binds to
Nix-store libcurl, not /usr/lib/libcurl. This prevents the plugin's
curl_easy_setopt calls from operating on the wrong libcurl instance.
Test portability: BSD sed -i wrapper, OpenSSL -sha256 for cert signing,
redirect test server output to log file.
Change-Id: I652b987d3ac45e31df50ff4ba1f523294438c2b6
this really doesn't have to be here, it doesn't help very much. doing it
in the parent is cheap enough to not care and sandbox setup is not async
yet *anyway*, so we would not even notice if the old way was any faster.
Change-Id: I5a3a99af0fa5928e9a42f9c6589d98ff38b8c775
this will make it easier to turn build sandbox processes into helpers.
the point they were at was effectivly unsandboxed except for a few fds
that were redirected by commonExecveingChildInit, which only made logs
of any errors that much harder to convey from the child to the parent.
Change-Id: I67006eb33e1e13311bb8d14e6a0c3d5e6baf0c13
it doesn't do anything anywhere else, and personality setup is not a
critical operation (all it does is change which arch uname returns).
Change-Id: I39d7fcc4916e6882e49191d2294f9b7ee0dbbcd0
there's no need to do it in the fork, we're not changing security
domains before opening the pty anyway. we do change who owns this
pty device, but since we change it via chown we can just *not* do
that before we have an open fd to the pty. in practice this isn't
even necessary because the daemon runs as root, but if we ever do
run the daemon as an unprivileged user we'd need this to be split
Change-Id: I35264ab2954c7ba2c9c24c927366d64acada6772
this also requires moving namespace support information into Worker and
out of function-scope static variables, otherwise we can't use async IO
for the libexec helper output. we could set the fd to blocking for just
one CL and extract the Worker changes into another that the reverts the
blocking fd usage, but that seems not warranted for the scope of these.
Change-Id: I6996fab1ae74693d50cefb6a6a9c21d61dada1d9
Those functions have been recently refactored to return a formatted
`string` or take an `std::ostream` and only do the formatting have been
renamed to match what they're actually doing.
Change-Id: I3fe32fbe8723c2d93226370b8dd297f16a6a6964
this is mostly a test and example for the libexec helper infrastructure,
but it also lets us simplify pager launching until we we can more easily
handle executable-not-found errors the launch fallbacks would cause when
using runProgram2 instead of fork. ideally we'd use `posix_spawn` later.
fixes#1104
Change-Id: Ia33cc12e8a9d60ffad6f5c055bb1b8b596810e64
the new libexec directory is not available as a setting like other
directories (e.g. binDir) are since we consider libexec helpers to
be very internal. repointing them is like repointing a .so file we
dynamically link to; it can work, but needs much more preparation.
Change-Id: I40e64be0b32276f2864c0f2eb0b998d4c8ce7c88
we don't need the full sandbox setup helpers for this: mount namespaces
do not need to be kept, loggers are not needed, and redirections can be
done by runProgram2. once the build hook is removed we will not run the
communication bits from a different process anyway, this prepares that.
Change-Id: I95d28f7c2c25e43ccd82b448d270403ce4f28852
we do not need explict redirection management now, and dieWithParent
doesn't need an override either. we'd much prefer to kill ssh if the
process in charge exits; even ssh multiplexers are not fazed by this
Change-Id: I81e28b7605df73c887878ea4716228c7ad0f5c6f
the redirections the tests did were once not possible with runProgram2,
but they have been for a while now. we should use them instead of fork.
Change-Id: Ia422c8941ed04a9403dee68bfe938d68952253da
returning the pid as a pid_t instead of as a Pid raii wrapper is only
convenient for daemon use (where it'll eventually go away). using the
released pid correctly in other places is harder without the wrapper.
Change-Id: Ib42a2f357d2f0849beabd015b321bfff31334eda
Fixes#323
Let's now all go an a little rant about spaghetti code...
The result of this code is that the missing paths are not printed
anymore. The basic issue was that the parent class of this command is
StorePathsCommand, which inherits from BuiltPathsCommand, and their
purpose is to work on path that are clearly in the store, building them
if needed (and ofc telling the user about what's missing), the sequance
of calls is:
- BuiltPathsCommand::run(ref<Store> store, Installables && installables)
- Installable::toBuiltPaths( *getEvaluator()->begin(aio()),
getEvalStore(), store, realiseMode, operateOn, installables);
where operateOn is Output by default, realiseMode is Derivation, so
the only thing that can be built are the derivations for the required
installables
- Installable::build(state, evalStore, store, mode, installables)
- Installable::build2(state, evalStore, store, mode, installables,
bMode)
And that final call has the following:
```
switch (mode) {
case Realise::Nothing:
case Realise::Derivation:
state.aio.blockOn(printMissing(store, pathsToBuild, lvlError));
```
So there were two options, hack a new spaghetti in the existing
spaghetti code, or condense all those calls that are actually useless in
our case because they mostly transform a list of installables into a map
from installables to their BuiltPath which are then iterated to retrieve
the final outputs, whereas it is possible to directly get the required
paths in a much more efficient manner and without printing unrequired
stuff through a multitude of intertwined function calls by simply
replacing one method that was previously inherited from the grandparent
class
Change-Id: I1d2baaef5a099cd98b63b5346f2613914c6cd2ac
it's no longer needed now that withPager exists. also fix a
logger-never-resumed bug that never showed up in the world.
Change-Id: I7311a50896f5291364320ec4c88506dc3bb11d4c
RunPager is weird and confusing in that it replaces what stdout *is*
depending on environmental conditions. this has not caused problems,
but it's easy to imagine situations in which it would (eg if the stl
decided to capture the stdout fd by duplicating it). using a wrapper
for this also makes clear *what* actually goes into the pager; while
the previous contract was semi-reasonable it was also very implicit,
and with the proliferation of functions we had that printed directly
to stdout it would have been easy to send wrong output to the pager.
using a wrapper also makes process management much easier because we
do not have to rely on destructors to always produce correct output.
Change-Id: Ifd3760940af1ec719fe856158c913cbb9a5bf270
this has been unused since ab3ce1cc13,
which is now almost ten years ago. surely it was just forgotten then
Change-Id: Ib93b52d4ce5f19629432c5f91ff426582c57f99b
cancelling a get() promise could leave the pool with allegedly in use
resource handles that do not actually exist. this would cause asserts
in ~Pool to fire (and crash the entire program due to bad accounting)
probably fixes#1041
Change-Id: Ic61ab9b658f02f71e8be7577b8a1115600846f86
merge all pkgconfig-found libraries into a single one and redirect all
existing lib files to the merged variant. the clean separation between
libraries is largely fictional anyway because the main config block is
in libstore and cannot reasonably be moved out, or because libutil has
assumptions built into its behavior that libmain later establishes, or
because libexpr messes with global boehmgc configuration. nearly every
kind of layering violation imaginable can be found *some*where in lix,
and not pretending that this is not the case will let us clean up some
of them. this will also allow us to merge these libraries at the meson
level which is itself necessary to make rust code easier to integrate.
Change-Id: I172f8a08d5d90ded0a0a99fd9e0fa5d804db00a7
xattrs scrubbing was gated behind Linux platforms, but Lix can safely
assume that POSIX APIs are available.
macOS implements these APIs modulo some creative ideas.
Fixes#1008.
Change-Id: I1566df4923bf24092d3fd32ccf6a96e4256e87b2
Signed-off-by: Raito Bezarius <raito@lix.systems>
Fixes#838.
xattrs were historically disabled in 2017, since then, NFS v4.2 gained
support for xattrs and almost all filesystems support xattrs. In
addition, moving the build directory to a xattrs-supporting filesystem
is always an option. Programs may exploit xattrs (including ACLs) during
their build process and testing phase, to better serve these programs,
we will remove this limitation.
Lix will use POSIX ACLs for its UNIX domain sockets in the future and be
one of these programs which will run tests making use of xattrs, while
not writing any xattrs in the derivation outputs themselves.
xattrs are still scrubbed from derivation outputs so it is not possible
to obtain an advantage by writing a security-related xattr to a well
chosen file.
Tests were added to test these scenarios on Linux. Darwin is carefully
excluded because of #1008 and #1090, solving that is welcome. In the
meantime, they are marked xfail.
Change-Id: Ia3255eeb8442e83db4f10dcb5a51cbc368a2550d
Signed-off-by: Raito Bezarius <raito@lix.systems>
one would expect the lang tests to be selectable (using `-k`) using
`eval_okay` `eval_fail` etc though this was not the case so far.
This commit renames the functions to reflect the expected names
Change-Id: I4cd340c093d774feeb4d96820d28e49c665b159f
I don't know how this was missed or no one really complained about it,
but perl-bindings was using CppNix if nix was not overlaid to our own
Lix.
If you overlay Lix manually, the resulting Lix's perl-bindings cannot
really refer to Lix itself, therefore, it's CppNix.
Change-Id: Ie5ce95564cb7fb5444acbea6124619c5121e1576
Signed-off-by: Raito Bezarius <raito@lix.systems>
during the value rewrite we accidentally broke extension of incomplete
primop application. this only shows up when binding on incomplete call
to a primop to a name, binding an incomplete call to *that* to another
name, and then finally calling the second binding with enough args for
a complete primop application. since this only shows up when calling a
primop with three or more args it took a while to surface. we have few
builtins that match this: foldl', replaceStrings, and substring. these
are not used incompletely in this manner very often, so this lingered.
fixes#1102
Change-Id: I218dffc14ae876efc86a86c7eb6c895e2405201c
these ended up referring to different directories and would break during
source reorganization (or even just including them in different places).
Change-Id: I91e8db04e3d3565efd5920982ed5bc59d2c27058
RegisterCommand was renamed in b30556f8f1
and de-static-ified in 4d9fe67430. adding
new settings was never possible with static instances since settings do
need an existing Config instance during construction; GlobalConfig must
be used to add settings. it is not possible to change existing configs.
Change-Id: I26af8abc375ba5883fce060ffefc710d6dfbca4c
Since lix-doc is no longer a separate library, the pkg-config files were broken
for static builds. Adjust them to refer to the new monocrate library instead.
Actually, since the monocrate library is static in all cases it needs to be
linked explicitly even when everything else is linked dynamically. I assume
this only doesn't break the dynamic nix-eval-jobs because it doesn't actually
use anything from it.
Change-Id: Icf29938688f4fa87425ff346a3105ef56a6a6964
Use a span representing the underlying hash, this allows comparing based
on those spans. Other minor tweaks to the header file.
Change-Id: Ie05960439e60500109083c06d347d6cdf41bfda8
When --check is ran with no output, this is seen as having *SOME*
invalid outputs, actually, ALL of them are invalid here.
Instead of reporting a confusing error, let's consider ALL of outputs
being invalid to be no output at all available and advise the user to
rerun the build without --check.
If there's *some* (>0) invalid outputs, let's print them exactly to give
a chance to the user to remediate this.
Fixes#485.
Change-Id: I00955ef9ea4f129e2c98d68c73b1e981f90278a0
Signed-off-by: Raito Bezarius <raito@lix.systems>
Yeah uh, I don't think `$shell` is ever set in the functional1
environment. Pretty sure this has been expanding to `''` THIS WHOLE
TIME.
Also `SHELL` is `/sbin/nologin` in that builder. idk why so I'm just
passing `shell` unconditionally in functional's `mkDerivation` helper.
Maybe next we should put `set -euo pipefail` into all these test
builders?
Change-Id: Ie5fb85305932c03a901ba84ffcce48f16a6a6964
macos-builder02 in particular has been having problems for a while now
that no one could reproduce. We believe we have finally found the cause:
missing shebangs, not just in the tests themselves but *also* in the
inline scripts `nix-shell` itself writes.
Fixes#1042.
I believe this will also fix#1093.
See also: https://github.com/NixOS/nix/pull/14778
Change-Id: I54d04a770b8e78484815db88a8bc88776a6a6964
When `:load`-ing a file that doesn't result in a successful outcome, the
"loaded" file is removed again from the list of loaded files.
This avoids having it be used as part of `:reload` which will probably
not result in a positive outcome.
Instead, the user is expected to retry `:load` before issuing a
`:reload`.
Fixes#50.
Change-Id: I85d025cf39f79e8e02117628bffa20c81917dc3f
Signed-off-by: Raito Bezarius <raito@lix.systems>
If `:reload` is used with no file loaded, the environment is completely
cleared.
That's a surprising behavior and one of the root cause behind #50.
Environment clear is only performed if there's files to reload.
Change-Id: Icb014f490a83aaa4d758dbf37822db16d5bb65c1
Signed-off-by: Raito Bezarius <raito@lix.systems>
Only successful loads counts towards the list of loaded flake
references.
Change-Id: I04128f90e57ef8352183d72dc6def7a09d52173a
Signed-off-by: Raito Bezarius <raito@lix.systems>
libutil config.{hh,cc} and libstore/globals.{hh,cc} contain so many
interrelated things. Hopefully we can separate them out a little bit for
clarity.
Change-Id: Ib78fc46fe305d13aada8886e2e168d446a6a6964
This makes that the default for all builds... which I'm not sure is the
right option, but it is also very cheap to do so, and adding an entire
other Lix build just for static libraries (not to be confused with
static Lix) would be prohibitively expensive.
Change-Id: Id173c77cfa62886724378ca599d21c8c6a6a6964
`build_env` is assumed to be a certain trivial environment, e.g. a
singleton search path.
An environment in general is either empty, contain one or more search
paths separated by a colon.
It seems that the intent was to set the `build_shell` first as a
prepended path in the managed environment then extend via the
`build_env` parts.
This fixes a usecase when the `BUILD_TEST_ENV` is non-trivial.
Change-Id: If5b8ab976d867a57ce0b8d29255f64695e30a8b2
Signed-off-by: Raito Bezarius <raito@lix.systems>
In an attempt to slowly split up and clean up `eval.cc`
The `Expr::eval` functions contain the most important logic when needing
to touch something on the evaluator, and having them spread across the
3kloc eval.cc file was really annoying. This should help, though more
cleanups are yet to come
Change-Id: I2c2d15a024ab1ec3fc5591f628d6768341a1c8ab
This adds a way to control the deprecated features that need to be
enable while evaluating the specific nixpkgs version
Change-Id: Ia2b9a29f1966d4d5f32239a36ba8270330007da9
This avoids rebuilding Lix each time we need to modify the lib tests
(as flake.nix stays unchanged), and adds a way to control the deprecated
features that need to be enable while evaluating the nixpkgs lib
testsuite
Change-Id: I2baa1cf90effcaf9d53337c6a7a5b24d6297178c
Certain evaluation requires to pass impurity arguments, add `flags`
to add more arguments to the instantiation.
Change-Id: I538ca2ff12c07cc66e845b42f81baa87f11b8f0f
Signed-off-by: Raito Bezarius <raito@lix.systems>
Is there a situation where `home` can be None? This hinders the ability
to use it quickly without type gating in the test code itself.
Change-Id: Idd739ddad67e10d6f7a6aad283c0e3b7cd08106a
Signed-off-by: Raito Bezarius <raito@lix.systems>
Otherwise, this result in debug traces with a \n cutting the list of
controllers.
Change-Id: I8aba6379d335c4edab9d9ddb3fb411813d8dc426
Signed-off-by: Raito Bezarius <raito@lix.systems>
The prior use of the UID is uninteresting in general for consumers who
wants to learn things about _what_ the build is, not what is the UID of
the build user running it.
Achieving this with a place where a mapping between cgroup names and
build requests are maintained is non trivial because there's no "global"
daemon that keeps track of this information and such a daemon would
require a persistent location to track this.
Rather than solving that, we will just encode the derivation hash inside
the cgroup name. Userspace can simply pull this apart and query Nix to
obtain the original corresponding derivation for more information.
This provides userspace with a way to map cgroup names to pnames for
example. Going further, if you have a map between attribute paths and
derivation hashes, you can power a database of cgroup metrics per
attribute paths by combining all of that.
In addition to this rename, we adopt a systemd-ish convention for
template units (`@`).
As a result, userspace tooling that will scrape cgroup metrics will
discover a consistent path as long as they filter out the UID part,
especially in context of Id6c458aad30eaa08c3609ac8280a7dde8e8f3cf9
change.
This problem is tracked under #1073.
Change-Id: I238d0568a3e4b1ff3057781c0639528d666b4d37
Signed-off-by: Raito Bezarius <raito@lix.systems>
nlohmann has customization points for exception throws. we can use these
instead and wrap json exceptions at the source instead of playing a game
of whack-a-mole with json errors all over the tree. since nlohmann needs
macros set to achieve this we can no longer precompile its headers *and*
must forbid including it anywhere without the proper defines or ordering
of include directives will break lix with ODR violations, if we see them
at all before we get another json-related bug report. a new lint will be
in charge of ensuring this doesn't happen. we also re-allow direct calls
to nlohmann json parsing since error handling is no longer a problem. we
will keep the wrapper for more convenient error context handling though.
fixes#1092
Change-Id: I54ecc14f5bec5e2177729b41c3703216e76cc6a3
It's 2026 and the command handler was entirely manual!
We introduce a basic command handler framework for the REPL that handles
all the previous (implicit) features reasonably well.
This also builds up the infrastructure for nicer features.
Most important feature is that now we can let external plugins adds
commands to the REPL.
There's slight REPL changes:
- errors for debugger commands are more explicit.
- short-hand have a long-hand version.
- help shows command sorted by std::map natural key sorting (based on aliases)
while keeping the special syntaxes for the REPL language at the top.
- sections were introduced for Flakes commands.
- :st was merged into one command with an optional argument marker.
Change-Id: Ibd24f994491bf5d45d4b218fd84c0944ae5050fd
Signed-off-by: Raito Bezarius <raito@lix.systems>
Status quo: We have `force$Type` and `eval$Type`, both which first
produce a value and then do a type check. The type checking logic is not
consistently implemented, with lots of code duplication.
This change does:
- Introduce new `check*` functions which unify the logic (the error
handling unfortunately still needs some duplication for now)
- Make both `force*` and `eval*` use the `check*` function for the
actual type checking
- Inline and dismantle the `eval*` functions for being of little use and
little used. This makes the `ExprOp*::eval` implementations for binary
logic operators more verbose, but IMO that's a good thing: The
implementation now needs to be a lot more explicit about the
short-circuiting semantics, something which was previously hidden behind
the short-circuiting semantics of the C++ language, in a way that could
easily be overlooked and lead to confusion, which is something that
happened to me twice in a year.
- Changes `forceAttrs` and `forceList` to include the context in case
`forceValue` fails (compared to only when the type check fails). This
was done for code consistency, because I could not find any reason why
list and attrs had different semantics here than int, float and bool. So
far the visible change is minimal (see the diff on the err.exp), however
this needs vetting for potential performance regressions.
Change-Id: I33e5c706d46850c9e1126293ee01dab85ba07587
That struct has been annoying me since I've encountered it. There is no
reason for having to track all these symbols out of band in the state.
Change-Id: I5c2c0d8174af0a51f9b456cc3651d8203a406d09
The names were prefixed with `sym_` to allow faithfully representing
names that start with __. (Previously the underscores were omitted, at
the cost of readability for people less familiar with the intricate
details of Nix. Is `toString` literally `toString` or is it
`__toString`? etc.) As a bonus, this allows to easily grep for any
symbol through the entire code base to find where it is used. I've done
exactly that and grouped and annotated the symbols based on their usage.
Change-Id: I230fecd9b0b18e0984ac8111c1e6381f45c5abf2
I was pretty confused by this when I saw it, not realizing that `build`
meant `tests/functional2/build` and not the top-level Meson `build`
directory.
Change-Id: I4865f196e5bf029419c6b488e78cbfb46a6a6964
100 causes too much diff churn w.r.t. current code style
"120 cols would be nuisance for us because our two-column layout is
exactly 118 wide (not counting the line number margin)"
"Ours is exactly 118 too"
Change-Id: Ifc7f21003e1011a17a1d799afaee775aeb927fd5
This logic is used in the various build-related REPL commands and is
factored out to make it easier to write the handlers for each
build-related REPL commands.
Change-Id: Iaa18df489db75495b12924e9a76b3fff1975eb64
Signed-off-by: Raito Bezarius <raito@lix.systems>
The completions for debug commands is purposefully left out because it
is going to go away in the next changes.
Change-Id: Ibc78b7587de3f15857bb09a3a0732334693ad550
Signed-off-by: Raito Bezarius <raito@lix.systems>
cl/4796 introduced a bug where running `just test-functional2` would
hang forever on macOS but `meson test -C build --suite installcheck
functional2` would succeed because of a mismatch between the
`$BUILD_TEST_SHELL` environment variable and the `-Dbuild-test-shell`
build option set in `mesonFlags`.
We can move these definitions to `finalAttrs.buildTestShell` and similar
and use those values in _both_ `mesonFlags` and `env` in order to make
them consistent in all cases.
See: https://gerrit.lix.systems/c/lix/+/4796/comments/50def5b4_fa5671f2
Change-Id: If300bbe46d6269ace29b44156fb4a5196a6a6964
...and add both a static bash and busybox into the sandbox for building:
$ /nix/store/fz43jxs5qfg5vldzk38y2hmrcl58qk18-busybox-1.36.1/bin/sh -c 'declare -A foo'
/nix/store/fz43jxs5qfg5vldzk38y2hmrcl58qk18-busybox-1.36.1/bin/sh: declare: not found
The reason is that busybox's `sh` doesn't know about `declare` which
means it cannot build derivations with `__structuredAttrs = true;`.
Change-Id: Ie3ca431f862fd3e59dc649582704ae739ee5834a
Nearly each tests gets a fresh store (in the test root residing under TMPDIR),
which with default settings means 8 MiB of reserved space to be deleted by the
user in case of space shortages, which of course for a test suite is just
wasted. Stop doing that to considerably reduce the litter (experimentally, from
3.7 GiB to 67 MiB per run). The old functional test suite already did the same.
Change-Id: I912cdda6e796ea37cfee0b8fe0478e976a6a6964
this is only needed on macos, and only for tests that build anything. on
linux it can actually *break* stuff due to unfortunate interactions with
store url parsing, which is required to work for any remote build tests.
Change-Id: Ic6f7e090f15e129fc365d7edc56cdbc1a5686047
apparently this file slipped through the code review (the actual test
does exist, no worries) but this empty file and init py exist, sitting
here empty. This commit drops them as they serve no purpose
see commit 1b7ad3a7ad
Change-Id: I64c4e03cd3ceedf9e430ae014127966bd8e4269d
This avoids any complications that can arise from the environment
affecting evaluation of the help pages (which don't need to be calling
out to anything external anyways)
I don't think this is actively causing problems now - though it did in
Nix! - so this is largely preventative, but I don't really see a reason
why this was ever running in impure mode in the first place either
Based-on: https://github.com/NixOS/nix/pull/14086
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
Change-Id: Ice0c760562d70c09934249edc3b767473516d372
- test_check used an old version of an error message
- test_good2 did not build an unspecified dependency
- test_attribute_selection used a fixed drv path. we
can't keep this with the current setup because the
drv `system` makes the drv hash platform-dependent
fixes#1081
Co-Authored-By: kloenk <me@kloenk.dev>
Change-Id: Ia009740f8b5432e83467ea451a5ff53d5e141b3f
the cgroups experimental feature does not work properly without this
because we do not stop subdaemons when the main daemon is shut down.
systemd needs the assigned cgroups to be empty to restart the daemon
and thus cannot cleanly restart the daemon if any connections exist.
starting a fresh unit for each connection creates a new cgroup every
time instead of sharing any delegations and thus solves the problem.
fixes#1030
Change-Id: Id6c458aad30eaa08c3609ac8280a7dde8e8f3cf9
A friend reported that this is rebuilding, which doesn't make any sense
because nixpkgs also has a lix from nightly that surely has the same
dependency and thus it surely should be built on hydra, right?
Turns out no, since they were overriding the requiredSystemFeatures to
remove big-parallel. Silly, but we can match that and get cache hits.
Change-Id: I4b1c8e850c6c38b03881354e978f41a10592ec95
This bump nixpkgs input to 25.11 and fix the two warnings that were
coming with the update:
1. `pkgs.system` => `pkgs.stdenv.hostPlatform.system`
2. `runCommandNoCC` => `runCommand`
also:
1. remove cppnix 2.3 compat tests (cppnix 2.3 is officially dead now)
2. remove lowdown 1.3 compat tests (nixpkgs no longer carries it)
Co-authored-by: Tom Hubrecht <github@mail.hubrecht.ovh>
Change-Id: I78526b5d8992a6c63ecd7f0c7c1fa6346a6a6964
It only debatably belongs in libutil since libutil doesn't know what a
progress bar is but whatever.
It was either that or libstore. Since I'm about to make it a setting.
Change-Id: Iba59b0a190087b316ff3191a0ddd9ec36a6a6964
make the global_assets folder more readable by placing asset pack files
within a dedicated folder instead of building up a mess similar to f1
Change-Id: Ia2c16f38eb6da96e1e73584bd91391ee56acb410
This will probably get the implementation of the fixtures revised when
we land the new extraction code, but we are setting it up to be generic
against that.
The operator-> thing is kind of a crime. But it also makes the code
vastly more readable so it's impossible to say if it's bad or not.
Change-Id: Ia5aca69cefaa03cd533ad19d20d856ff7e76a546
Instead of manually reading the ref file,
which gives the tag object hash when ref is a tag,
shell out to git.
The logic for finding the ref file is left for cache TTL tracking.
Fixes: #1070
Change-Id: I490b1e62f83cf602c56232c6081a52166a6a6964
Those are new warnings that come with a more up-to-date llvm
Fixes#1066
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I305d0d810d12a5e8d31c7d89e0cdb3a82df71556
As far as I can tell, there's no harm to let know a nix-shell invocation
about NIX_LOG_FD being stderr.
Fixes#336.
Change-Id: Ifdb3591813251d4bc481158d28dc18e1489de72c
Signed-off-by: Raito Bezarius <raito@lix.systems>
As far as I can tell, there's no harm to let stdenv know that they can
write to file descriptor 2 which is stderr inside of a nix3-develop
shell.
Contributes towards #336.
Change-Id: I51dcbcbc19a1698a0d1255f9d943d9ebd15d115d
Signed-off-by: Raito Bezarius <raito@lix.systems>
Closes#1064
The culprit here is that `genGraphString` is only invoked with the
store-paths associated with the outputs of the derivation, so when
filling `dependents`, the `graph_data.find(p)` call would return the end
of the iterator when doing this for references to other store-paths.
As a result, the code wrote information behind the graph data-structure
causing a corruption. For me, this resulted in a SIGSEGV most of the
time and in a few cases in an uncaught `map::at`-exception as reported
by Niklas.
This patch changes two aspects of the original implementation:
* When filling `dependents` in the node-set, use `map.at()` instead of
`map.find()->second`. The latter doesn't make any sense and was the
cause of corrupting memory. The `at` would've made it far easier to
spot this in the first place.
* Filter out store-paths that don't belong to a different output of the
derivation when creating `outputGraph`. This variable is used on two
places, `genGraphString` and for topological sorting.
The latter already filters out store-paths from a different drv, so
this is happening now when creating the variable in the first place
such that `genGraphString` never ends up with corrupt data in the
first place. This is the actual bugfix.
Implemented a regression-test for this case to be sure.
Change-Id: Ie02144d89c32b0a776cb1ece0601d0229315ebc3
we'll need this to modify argv for socket-activated daemons. this is our
replacement for the old savedArgv mechanism that was unscoped and fucky.
Change-Id: Ie048eb8ea99f1c9cd627a051292c836c83197068
the parent daemon does not change any settings before starting a child,
so there's nothing we may want to change that is not already set by the
config file. this also doesn't prevent changes of the config file being
applied to daemons where we do not expect it since it'll only restore a
setting to the parents' value if the child also has an override for it.
Change-Id: Ic5a9ef13458c103ec9979cb187ba8d3ce5e1e719
Lix changed to C++23 in 4ea8c9d61¹, but the perl subproject wasn't
updated with it. This commit corrects that.
[1]: 4ea8c9d643
Change-Id: I5464138d206d87cf25762720b147487d6a6a6964
We offer the three usual options: verify, repair then delete and direct
people to report us corruption so we can fix if this is induced by us.
Fixes#447.
Change-Id: I0df61769d732d227333c206f312857c0593f7bce
Signed-off-by: Raito Bezarius <raito@lix.systems>
If the derivation does not start with D, do not return a simple
"expected string 'D'" error but a full error message.
This contributes towards #447.
Change-Id: Iee05f3918e4cc43e79f244ab2fd64a52cf2bb6d2
Signed-off-by: Raito Bezarius <raito@lix.systems>
Prior to I6a6a6964d2b5ad47ae5ea9eb11af9b6373ce2141 — `sudo nix
upgrade-nix` would perform direct store access.
This ensured a certain number of desireable properties for upgrading the
Lix binary itself.
We re-introduce direct store access for upgrading Lix binaries.
Fixes#1060.
Change-Id: I523c4d3023ed5fe9eff8fde9a266c56a0de47d8c
Signed-off-by: Raito Bezarius <raito@lix.systems>
Alternative to cl/4661 discussed in #1044.
It can be assumed that `$tmpdir/build-top` can be created safely without
any risk, this way, we don't need to reuse the random directory creation
primitive.
Fixes#1044.
Change-Id: Iec52477f3047fc40959b183c607312d5a40fc8c9
Signed-off-by: Raito Bezarius <raito@lix.systems>
Otherwise, loading Perl bindings fails early with[1]
undefined symbol: Perl_pad_sv at /nix/store/h2jsb5i4yfblr2f3ac2c7zpmlmj7zjym-perl-5.40.0/lib/perl5/5.40.0/XSLoader.pm line 94
Apparently, it's expected behavior by Perl that this symbol only exists
with `DEBUGGING` being set, hence it's used by the headers. However,
`pkgs.perl` from nixpkgs is apparently not built with `-DDEBUGGING`
causing this error.
Now, `NDEBUG` is manually unset after loading the Perl
headers rather than setting `DEBUGGING` causing the error mentioned
above.
I confirmed that this not only fixes the problem described above, but
running the Hydra tests with
diff --git a/perl/lib/Nix/Store.xs b/perl/lib/Nix/Store.xs
index dfdd64d28..14788266c 100644
--- a/perl/lib/Nix/Store.xs
+++ b/perl/lib/Nix/Store.xs
@@ -27,6 +27,7 @@ using namespace nix;
static AsyncIoRoot & aio()
{
+assert(false);
static thread_local AsyncIoRoot root;
return root;
}
still results in assertion errors.
Finally, added a small install-check that importing `Nix::Store` works
fine.
[1] https://git.lix.systems/lix-project/hydra/issues/69
Change-Id: I58521777eb0f94b766a9813aa4bbd06f9052bd35
As discussed in the F2 matrix channel, not being able to put the error
message inside of the thrown exception just results in one assigning an
additional variable to immediatly throw it. The original reason for
these rules to exist are that the message isn't printed twice, though
the line above the thrown exception is printed anyways in the
stacktrace, so we still get the error message twice but now with
additional work.
Hence this commit removes the said rules
Change-Id: I0b37c0b0861334703a5772c36353d31441f19dcd
So far, our ruff config was confined to the f2 package. This meant, that
when one added additional paths to the ruff formatter, those wouldn't
get the same rules applied as f2, resulting in inconsistent styling
thoughout the project.
Due to how configs are resolved, only the "closesed" pyproject toml is
considered. This means, we need to tell f2 to extend its configuration
with the base level one. Though no change is required for other parts of
the project, as long as they don't have their own pyproject.toml
Change-Id: I145c764e7b850194020b5560e1025f4aa80411ae
Nix daemon runs in a service context, usually with systemd.
If the Nix daemon unit adds `[Service] CacheDirectory=nix-daemon`, it should
successfully use `/var/cache/nix-daemon` automatically.
Instead, it uses `/root/.cache` right now, which is really bad.
By default, we add `[Service] CacheDirectory=nix` now which means that
caches are moved into `/var/cache/nix`.
Fixes#634.
Change-Id: I854b1045bfdce8f60110aea70bda1bf6657dfd7b
Signed-off-by: Raito Bezarius <raito@lix.systems>
Recently, in 54df89f601, support for mdbook 0.5
was introduced, including some logic to handle the `sections` -> `items`
rename. However, compatibility with 0.4's `sections` was only kept on the read
path, while writing 0.5's `items` unconditionally, which ends up in the bit
bucket on 0.4, effectively disabling substitution fully and leaving the include
directives in the final documentation. Restore writing into the `sections` when
they were there so that substitution works again.
Change-Id: Idd4d7653012660f3f7fc27f81f29b82d6a6a6964
Silly me forgot to add the dependency responsible for handling
`@pytest.mark.timeout` failsafes to kill a test if it takes too long,
which results in a warning when running f2.
This commit adds the required dependency and hence removes the warning
Change-Id: I281cad05bb0cf50208f72080cb2bfd8cbfb09d0c
Running the sandbox (as unprivileged user) requires all of user, mount and PID
namespacing. Previously, only the user namespacing support was checked.
Unfortunately, newer Ubuntu kernels impose restrictions on capabilities within
unprivileged user namespaces [1]; as observed by experimentation, in particular
(re-)mounting filesystems or changing mount propagation is prohibited even
inside a mount namespace, making it impossible to use the Lix sandbox. So any
tests requiring sandboxing should check for this breakage and be skipped if
detected.
`unshare --mount` performs the same problematic operations, so that's what gets
used for the test. PID namespaces are included as well for completeness, even
though currently no instance of breakage due to them specifically is known.
This fixes [2] for the functional test suite only.
[1] https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces
[2] https://git.lix.systems/lix-project/lix/issues/545
Change-Id: If95b527bf965ccb06371bbef7b6007666a6a6964
Builds using a diverted store strictly require sandboxing. Explicit disabling
by the user is already handled properly by force-enabling the sandbox anyway
(with a warning), but sandbox-fallback was still honored, predictably leading
to hilarious breakage (like the "confusing output" in [1]) when it actually
triggered. Cause a hard failure instead.
[1] https://git.lix.systems/lix-project/lix/issues/545
Change-Id: I61ea32c3bd8bca4e5ba278312b0a49326a6a6964
Builds using a diverted store strictly require sandboxing. It therefore makes
sense to automatically enable it even if otherwise explicitly disabled by the
user. However, they should still be informed that their choice is not being
respected.
Change-Id: Ia2cbdc24a4c55d8ecc49094e35e698e16a6a6964
Disabling build sandboxing is a significant change to build isolation. If this
is done automatically due to namespacing issues (and sandbox-fallback not being
disabled), show the user a loud warning without the need to opt into a lot of
other log spam.
Change-Id: I52daccdd6934873234223c798ca3dcff6a6a6964
This contains the depth of nix shells nesting.
Fixes#826
Co-authored-by: Qyriad <qyriad@qyriad.me>
Change-Id: If584c9d02730d6c857636dafdeab0c01f4ec8e0f
If the NIX_PATH shadows internal files, this will often break things,
hence we forbid it.
Fixes#998
Change-Id: I70e5d389532ada1c9f910c60281abe565e3ce6bb
Some of the tests depended on the evaluation order chosen internally by
nix-eval-jobs, which will break when that order changes. In practice the order
is not guaranteed, and already not deterministic with multiple workers.
Change-Id: I2e85d096f4439a8075a6f4a08d31c0f16a6a6964
GCC still ICEs on coroutines, while Clang works properly. Use the latter, just
like for Lix itself.
Change-Id: I802d6df7279550a5f5355e6c1ea0533c6a6a6964
Previously the collector-side implementation of the worker interaction protocol
was a bunch of spaghetti code. Split it apart at places where it makes sense to
be easier to follow.
Change-Id: If3cc6b6fbf289dd778856b40b55316c76a6a6964
As the worker processes are now properly spawned using fork-exec, they no
longer inherit the stack size from the non-main coordinator thread, but get a
main thread with reasonable stack size on their own. For this reason the
coordinator threads can use the default stack size, and turn std::thread can be
used as the only reason for the custom wrapper was to enlarge the stack.
Change-Id: I1192474885abe9b0625ac483840b6e1a6a6a6964
Similar to how 7b37d5ea6a switched the subdaemons
from fork-only to fork-exec, do the same for the nix-eval-jobs workers. This is
the promised change that causes the signal handler thread to appear.
Change-Id: I590a5eb6bddfb546a331a01191fd788b6a6a6964
The retry added in 4c5efd4548 has never worked
properly, as releasing the pid the first time causes it to be -1 and thus
waiting for a random child on retry. While there is a good chance that this
really catches the crashed worker, this is by no means guaranteed, and it could
just as well have caught different restarting worker. Wait synchronously
instead. The worst possible outcome is a hang instead of an error message if
the worker manages to close its pipe but then fail to exit, which is not
supposed to happen in the first place.
Change-Id: I4e4c3f532ec15c3118f6f77821ab820a6a6a6964
warning: embedding a directive within macro arguments has undefined behavior
????? CLANG? why does this take -pedantic to show??
Change-Id: I6f4a9f3db1b89448f9f5d00d82b551e46a6a6964
This change brought to you by my uninstalling go but forgetting to
remove ~/.local/opt/go/bin from my PATH.
Change-Id: I937675bab11ac7e0678c896cba78204b6a6a6964
Maintains compatibility with mdbook 0.4.x.
Includes comments for what to remove once 0.5.x is the only we care
about. Some other changes technically could be changed at that point,
but currently serve to enable universal support of 0.4.x and 0.5.x
Fixes#1051.
Change-Id: Ic5b405038d180bcd357bbd9e5716879e0c26e5f5
In the same vein as be18b7dc2¹.
Before:
error: syntax error, expecting ';'
After:
error: syntax error, expecting ';' to end binding
[1]: be18b7dc25
Change-Id: Iffbcb113d2b892a50c646c9875e970376a6a6964
Lix blocks SIGPIPE, and std::cout ignores EPIPE (actually it probably stashes
an error code in some flag that no one ever reads, with the same end result).
Consequently, nix-eval-jobs would waste resources by continuing to evaluate
even when the reader interested in the results has long gone away. Instead, use
writeToStdout, which throws an error on EPIPE, leading to the process
terminating as desired.
Reported-by: Winter <winter@winter.cafe>
Change-Id: I962c09bab582a8ed27dd41c01b1519876a6a6964
Right now, the worker will not receive interrupts, as it has no signal handler
thread due to the way it forks, but this will change soon. Prepare to handle
worker interruption properly, by letting the worker exit successfully in this
case and adjusting the coordinator to handle this exit gracefully.
Change-Id: I9f79670b0d2004e7e2d8bf36cf67108e6a6a6964
For some reason that eludes me, it used to only print the last attribute
instead of the full chain.
Change-Id: I2f5f6e85e2b1ab3ab20a9c99bf06d1e45a4d09bb
`e`, `ae`, `j`, `jAttrs` … holy fuck this function was so much worse
than it needed to be. This refactoring includes:
- Descriptive variable names where possible
- More code comments indicating what even is happening
- Shuffled the control flow around to be more linear (move early return
conditions up, etc.) and have less rightwards-drift, to increase
readability
- Extracted `mergeAttrs` function for future changes and readability
Change-Id: I7253b47ce4910fdc67aa7d6edff2f3c0bb8fa55b
In one of the cases it pointed to the first instead of the second
occurence, which is a bit confusing
Change-Id: Ie508d1f84feb434708804d45aa8a9c8b1e3c5f69
How long do we have the value printer already? It's time to stop
concatenating strings like it's 2005
Change-Id: I3f5074de2439a1ad78af94de877bb141bc9f1d82
When using `builtins.derivation`, the `outputs` list can be omitted, and
in that case, it defaults to `[ "out" ]`, set this default when building
the shell derivation for `nix develop`
Fixes: #556
Change-Id: I4dc77d060f30ee499e186ec919de30bd3e5c53bb
Commit 5dc847b47b introduced usage of
std::chrono::milliseconds, which prints with the unit, leading to its
duplication. Remove this duplication again. Specifically, the explicitly
printed one is kept, due to better typography.
Change-Id: I724ea6be6f2a1349fe799602c3429f096a6a6964
We have done it!
The functional/lang framework has fully been migrated to functional2 :D
closes: #856
Change-Id: I63ad8d7dbcd9b5267ca04af68df73b1ffa3d6461
After multiple cycles of deprecation, we now use the lix repositorty as
a default, when no prefix is defined
Change-Id: I29b445fee39fb20c0ff024e3b95402f459144b62
Added builtins.warn` which takes two arguments: a message that is
displayed as a warning during evaluation which must be a string and a
value that is returned from the expression.
The next commits add new settings to control the behavior of the new
builtin: `debugger-on-warn` allows the user to start the debugger and
`abort-on-warn` aborts evaluation with an error.
Unlike upstream, I chose not to mark evaluation warnings from
`builtins.warn` as distinct from other warnings because that breaks the
commonly expected logging format `level: message`.
Co-authored-by: Qyriad <qyriad@qyriad.me>
Upstream-PR: https://github.com/NixOS/nix/pull/10592
Fixes: https://git.lix.systems/lix-project/lix/issues/579
Change-Id: I8658c88e5c27952b65e8b9f5525a572e0680cc1f
There are currently 4 different ways to run garbage collection using
lix:
- `nix-collect-garbage`
- `nix-store --gc`
- `nix store gc`
- (using the daemon directly)
As they were written all at different times, their output varies (and is
broken in some case). This unifies the display of informations in the
following ways:
- The list of paths in the results is always printed (in the nix3 cli it
is hidden unless `-v` is passed)
- The number of paths in the result set is always displayed
- The size of deleted paths is only shown when actually deleting things
(as it would have been 0B in any case)
Fixes#905
Change-Id: I40d9ec7c6d76795f6c6dd30df196d1e855bdb9db
This also allows removing a static initialization of a primop.
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Change-Id: I74da68205630f658b239e4327bf5e27bfc1b91da
This is done at the start of the command call, which allows not building
expensive things if the arguments are incorrect anyway.
Fixes#687
Change-Id: Ic924fe2115cf760684c6fdf7987279e96ab00286
the newly added f1/lang tests is required for the f1/lang framework
dismanteling to happen in a separate commit
Change-Id: Ic419c515262294c51a46d1513daa7848e4b71405
While the err file should always be empty, we prefer not have (easily
avoidable) warnings in the log, in order for actual warnings to be
spotted more easily.
Additionally this way no additional changes are required in case they
make use of some depreacted features in the future.
Change-Id: Ie078e2a851b2035d839f6b95d4188e475eb96b2d
autoargs get their own folder and are not allowed with the other
function args tests, just like they deserve their own ring of hell
It shall be noted though that this test is particularly pointless, as it
merely tests the normal argument passing (which is fine), and not the
✨magic✨ autoargs actually provide
Change-Id: Ia2dc00754a13c02e4926200141f90bc797820a15
That one is a bit confusing, as I renamed the old `in` test to `in-2`
(file identities in Git when)
Change-Id: Id0e17de414dbc1b1b616d456175704f98657a16a
Call it a bug, call it a feature, but the old testing framework
ungracefully shits itself when it has no tests to run … in other words,
we finally migrated all the parse-fail tests 🎉
Change-Id: Ibd7f1c04d9a396a20f14361af7924f0074d1ac23
The second test has also been fixed. I'd do it in two commits if
functional1 wasn't such a PITA (guess why we're doing the migration …).
Basically, the introduced syntax got changed later on, but the test
never got updated, and back then tests didn't test the output so nobody
ever noticed it now failing for a wrong reason.
Change-Id: I5b66127d507c65676b8dd4a82d1e1f2857fef145
This really looks like one of the first tests ever written, it doesn't
seem to be actively testing anything interesting
Change-Id: I442840db932203c25da2d8400ada791f8aed04ce
synopsis:"flake config warnings are now printed to stderr"
issues:[1155]
cls:[5379]
category:"Fixes"
credits:[lheckemann]
---
The settings listed in a flake-config confirmation prompt are now printed to stderr rather than stdout, which allows `nix print-dev-env` to emit valid bash again even in the presence of untrusted settings.
synopsis:"Shadowing internal files through the Nix search path is now an error"
issues:[998]
cls:[4632,5370]
category:"Breaking Changes"
credits:[thubrecht, jade, horrors]
---
As Lix uses the path `<nix/fetchurl.nix>` for bootstrapping purposes, the ability to shadow it by adding `nix=/some/path` (or `/other/path` that contains a `nix` directory) to the search path is not desirable.
Lix 2.95 deprecated this behavior with a warning, Lix 2.96 now turns it into a hard error if the `nix-path-shadow` deprecated feature isn't enabled. This deprecated feature is slated to be removed in Lix 2.98.
> If you are building via the Lix daemon (default on Linux and macOS), it is the Lix daemon user account (that is, `root`) that should have SSH access to a user (not necessarily `root`) on the remote machine.
>
> Furthermore, `root` needs to have the public host keys for the remote system in its `.ssh/known_hosts`.
> To add them to `known_hosts` for root, do `ssh-keyscan USER@HOST | sudo tee -a ~root/.ssh/known_hosts`.
> To add them to `known_hosts` for root, do `ssh-keyscan HOST | sudo tee -a ~root/.ssh/known_hosts`.
>
> If you can’t or don’t want to configure `root` to be able to access the remote machine, you can use a private Nix store instead by passing e.g. `--store ~/my-nix` when running a Nix command from the local machine.
## Configuration
The list of remote machines can be specified on the command line or in
the Lix configuration file. The former is convenient for testing. For
example, the following command allows you to build a derivation for
- `uri`: Due to backward compatibility, the `ssh://` may be omitted for the store-uri.
- `ssh-public-host-key`: The key must be provided encoded in base64. Specifically calculated via `base64 -w0 /etc/ssh/ssh_host_ed25519_key.pub`
### Format detection
At first, the given configuration is being parsed syntactically as a toml.
If parsing fails and the given configuration contains a `"` the error is presented to the user, as those characters are necessary for TOML, but disallowed for the legacy format.
Otherwise, parsing is retried using the legacy format.
If non-syntactic errors are detected within the toml, the exception will always be shown to the user directly.
## Builder selection
The configuration(s) above specify several machines that can perform `i686-linux` builds.
However, `granite` will only do builds that have the attribute
```nix
requiredSystemFeatures = [ "benchmark" ];
```
or
```nix
requiredSystemFeatures = [ "benchmark" "kvm" ];
```
`diorite` cannot do builds that require `kvm`, but `andesite` does support
such builds. For regular builds, `diorite` will be preferred over
@@ -177,6 +177,12 @@ Most commands in Lix accept the following command-line options:
You can override this using `--arg`, e.g., `nix-env --install --attr pkgname --arg system \"i686-freebsd\"`.
(Note that since the argument is a Nix string literal, you have to escape the quotes.)
Additionally, dots are interpreted as attribute-path separators.
I.e. `nix-instantiate '<nixpkgs>' -A hello-unfree --arg config.allowUnfree true` will result in an argument `config` with value `{ allowUnfree = true; }` being passed to `<nixpkgs>`.
Please note that merging of different arguments is rejected.
I.e. `--arg config '{ cudaSupport = true; }' --arg config.allowUnfree true` will not work whereas `--arg config.cudaSupport true --arg config.allowUnfree true` is accepted.
@@ -19,7 +19,7 @@ This description is not normative, but a feature removal may roughly happen like
1. Add a warning when the feature is being used.
2. Disable the feature by default, putting it behind a deprecated feature flag.
- If disabling the feature started out as an opt-in experimental feature, turn that experimental flag into a no-op or remove it entirely.
For example, `--extra-experimental-features=no-url-literals` becomes `--extra-deprecated-features=url-literals`.
For example, `--extra-experimental-featuresno-url-literals` becomes `--extra-deprecated-featuresurl-literals`.
3. Decide on a time frame for how long that feature will still be supported for backwards compatibility, and clearly communicate that in the error messages.
- Sometimes, automatic migration to alternatives is possible, and such should be provided if possible
- At least one NixOS release cycle should be the minimum
@@ -51,28 +51,64 @@ $ nix-shell -A native-clangStdenvPackages
### Building from the development shell
Run a clean build and test with `just clean setup buildinstall test`.
We have a [justfile](just.systems) for extra convenient building.
It defaults to using `./build` as the build directory, and `$out` (`./outputs/out`) as the install directory.
For most cases, you can clean-build, install, and run the tests with:
```bash
$ just setup --wipe && just test
```
> **Note**
>
> The `--wipe` argument to `meson setup` conveniently works whether you have an existing build directory or not.
>
> However, it is *mostly*, but not *exactly* equivalent to deleting the build directory first.
> In particular, previously specified `-D` build options are **preserved** with `--wipe` (for some reason).
> For example, if you fetch and checkout a new version of Lix, and that new version *removes* a Meson build option from `./meson.options`, *and* a previous invocation in that build directory explicitly set that option, then `meson setup --wipe build` will error, complaining about the unknown option.
> For these cases, `just clean` will give you a well-and-truly-this-time-for-real clean build.
Because the integration tests require installation to work, `just test` automatically also calls `just install`, and Meson helpfully will automatically build any targets that need building when trying to install them.
You can override the build directory or install directory by setting the justfile [variables](https://just.systems/man/en/setting-variables-from-the-command-line.html) `outdir` and `builddir` on the command-line:
```bash
$ just builddir=build-before-bisect outdir=out-before-bisect setup
$ just builddir=build-before-bisect test
```
You'll have to set `builddir` for every target, but `outdir` only needs to be set for `setup`.
You can also run the unit tests and integration tests separately:
```bash
$ just setup build test-unit
$ just install test-integration
$ just setup
$ just test-unit
$ just test-integration
```
Many justfile aliases have a `-custom` variant which pass extra arguments to `meson`.
Most justfile targets forward all further arguments to the underlying Meson invocation.
For example, to work on both Lix and nix-eval-jobs you can run:
```
$ just setup-custom -Dnix-eval-jobs=enabled
$ # or
$ mesonFlags=-Dnix-eval-jobs=enabled just setup
```bash
$ just setup -Dnix-eval-jobs=enabled
```
Note that only targets which don't accept extra arguments can be used when
running multiple targets at once; `just setup build` is fine, but `just
setup-custom build` is an error. The `test` target is usually the last one to
run, so it always accepts extra arguments.
Note that only targets which *don't* accept extra arguments can have other targets following them.
`just clean setup` is equivalent to `just clean && just setup`, but `just build test` runs the `build` target with the argument `test`.
This means that if you want to, for example, build with lower parallelism, and then test, you will have to do something like this:
```bash
$ just build -j4
$ just test
```
Finally, the rewrite of the integration test suite, functional2, also has its own justfile target which allows passing extra arguments to pytest.
For example, to collect and list all functional2 tests without running them, you can pass pytest's `--collect-only` argument:
```bash
$ just test-functional2 --collect-only
```
You can also build Lix manually:
@@ -141,6 +177,21 @@ To inspect the canonical source of truth on what the state of the buildsystem co
$ meson introspect
```
#### LLD
The development shell on Linux uses LLD by default for faster link times.
This is set using `mesonFlags`, so to override it, you can simplify re-specify the linker to Meson:
```bash
$ just setup-custom -Dc_link_args=-fuse-ld=ld -Dcpp_link_args=-fuse-ld=ld
```
While using LLD, you may find it helpful to use ThinLTO for even further improvements to link times for incremental builds:
```bash
$ just setup-custom -Db_lto=true -Db_lto_mode=thin -Db_thinlto_cache=true
```
## Sending changes to Gerrit for review {#sending-to-gerrit}
We use Gerrit for all our code review in Lix.
@@ -393,6 +444,18 @@ You can build it yourself:
# xdg-open ./result/coverage/index.html
```
Or, in a dev shell, set `-Dcoverage=true` when running `meson setup`.
Coverage data goes into `build/profraw` when you run executables in the dev shell.
Then, run `ninja -C build coverage-report` to produce an HTML report of coverage in `build/coverage/index.html` alongside a LLVM `.lcov` file.
> [!NOTE]
> We use the [llvm source-based coverage], which has better precision than using clang with gcov, which is debuginfo based (but likely worse performance, which is fine).
>
> It should be noted that Meson [allegedly has coverage support][meson-coverage], but it only supports gcov-style coverage, so we don't use it.
@@ -383,7 +383,10 @@ I grepped `lix/` for `get[eE]nv\("` to find the mentions in Lix code.
Overrides compile-time configuration of various locations used by Lix. See `lix/libstore/globals.cc`.
**Expected value**: a directory
- `NIX_DAEMON_SOCKET_PATH` (optional) - Overrides the daemon socket path from `$NIX_STATE_DIR/daemon-socket/socket`.
- `LIX_DAEMON_SOCKET_DIR` (optional) - Overrides the daemon socket directory from `$NIX_STATE_DIR/daemon-socket`.
**Expected value**: a directory
- `NIX_DAEMON_SOCKET_PATH` (optional) - Overrides the daemon socket path from `$NIX_STATE_DIR/daemon-socket/socket`. Ignored if `LIX_DAEMON_SOCKET_DIR` is set.
**Expected value**: path to a socket
- `NIX_LOG_FD` (output) - An FD number for logs in `internal-json` format to be sent to.
@@ -401,7 +404,6 @@ I grepped `lix/` for `get[eE]nv\("` to find the mentions in Lix code.
**Expected value**: the path to an executable shell
- `PRINT_PATH` - Undocumented. Used by `nix-prefetch-url` as an alternative form of `--print-path`. Why???
- `_NIX_IN_TEST` - If present with any value, makes `fetchClosure` accept file URLs in addition to HTTP ones. Why is this not `_NIX_FORCE_HTTP`??
Not used anywhere else.
- `NIX_ALLOW_EVAL` - Used by eval-cache tests to block evaluation if set to `0`.
FIXME(Lix): This section does not document the most common modern practices in terms of avoiding channels, pinning, declarative software installation (see flakey-profile or home-manager or NixOS), or using flakes, etc.
It is, however, likely correct at a technical level.
For more information on modern practices, see the [resources](https://lix.systems/resources) page on the Lix site.
For more information on modern practices, see the [resources](https://wiki.lix.systems/books/lix-users/page/nix-resources) page on the Lix site.
- Fix unsigned overflow leading to out-of-band write in the NAR parser [cl/5553](https://gerrit.lix.systems/c/lix/+/5553)
The NAR parser contained an unsigned integer overflow that could be used by an
attacker to write arbitrary data to an unknown memory location and possibly
achieve code execution. A successful attack on the system-wide Lix daemon
could lead to privilege escalation to root. Any process that involves NAR
serialization could trigger this issue, including (but not limited to)
- local user interaction, whether the users are trusted or untrusted
- malicious substituters sending malformed NARs
- remote builders sending malformed build results
- remote daemons sending malformed inputs when requesting remote builds
Successful attacks using this bug require ASLR weakening of some sort, whether
by architecture constraints (e.g. on 32 bit systems, where little randomization
is possible) or system configuration (e.g. low ASLR entropy when loading
libraries), and millions of attempts. Local attacks can be mounted in less than
an hour. Remote builds typically require a fresh SSH connection for each build
and are thus less susceptible. Only one attempt can be made by substituters for
every build using substituters, they are thus not a likely vector for attacks.
At the time of writing, MITRE has not assigned this a CVE yet.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae), [Raito Bezarius](https://git.lix.systems/raito), [edef](https://github.com/edef1c), and [sandydoo](https://github.com/sandydoo) for this.
- Deprecate shadowing internal files through the Nix search path [lix#998](https://git.lix.systems/lix-project/lix/issues/998) [cl/4632](https://gerrit.lix.systems/c/lix/+/4632)
As Lix uses the path `<nix/fetchurl.nix>` for bootstrapping purposes, the ability to shadow it by adding `nix=/some/path` (or `/other/path` that contains a `nix` directory) to the search path is not desirable.
To alleviate potential issues, Lix now emits a warning when the Nix search path contains potential shadows for internal files, which will be changed to an error in a future release.
The warning can be disabled by enabling the deprecated feature `nix-path-shadow`.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- More deprecated features [cl/2092](https://gerrit.lix.systems/c/lix/+/2092) [cl/2310](https://gerrit.lix.systems/c/lix/+/2310) [cl/2311](https://gerrit.lix.systems/c/lix/+/2311) [cl/4638](https://gerrit.lix.systems/c/lix/+/4638) [cl/4652](https://gerrit.lix.systems/c/lix/+/4652) [cl/4764](https://gerrit.lix.systems/c/lix/+/4764)
This release cycle features a new batch of deprecated (anti-)features.
You can opt in into the old behavior with `--extra-deprecated-features` or any equivalent configuration option.
-`broken-string-indentation` indented strings (those starting with `''`) might produce unintended results due to how the whitespace stripping is done. Those cases will now warn the user.
-`broken-string-escape` "escaped" characters without a properly defined escape sequence evaluate to "themselves". This is in most cases unintended behaviour, both for writing regexes, and using legacy or uncommon escape sequences like `\f`. The user will now be warned, if those are present.
-`floating-without-zero` so far, one was able to declare a float using something like `.123`. This can cause confusion about accessing attributes. Floating point numbers must now always include the leading zero, i.e. `0.123`
-`rec-set-merges` Attribute sets like `{ foo = {}; foo.bar = 42;}` implicitly merge at parse time, however if one of them is marked as recursive but not the others then the recursive attribute may get lost (order-dependent). Therefore, merging attrs with mixed-`rec` is now forbidden.
-`rec-set-dynamic-attrs` Dynamic attributes have weird semantics in the presence of recursive attrsets (they evaluate *after* the rest of the set). This is now forbidden.
-`or-as-identifier``or` as an identifier has always been weird since the `or` (almost-)keyword has been introduced. We are deprecating the backcompat hacks from the early days of Nix in favor of making `or` a full and proper keyword.
-`tokens-no-whitespace` Function applications without space around the arguments like `0a`, `0.00.0` or `foo"1"2` are now forbidden. The same applies to list elements. The primary reason for this deprecation is to remove foot guns around surprising tokenization rules regarding number literals, but this will also free up some syntax for other purposes (e.g. `r""` strings) for reuse at some point in the future.
-`shadow-internal-symbols` has been expanded to also forbid shadowing `null`, `true` and `false`.
-`ancient-let` deprecation has been turned into a full parser error instead of a warning.
-`rec-set-overrides` deprecation has been turned into a full parser error instead of a warning.
Many thanks to [piegames](https://git.lix.systems/piegames), [rootile (Rutile)](https://git.lix.systems/rootile), and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Move `/root/.cache/nix` to `/var/cache/nix` by default [lix#634](https://git.lix.systems/lix-project/lix/issues/634) [cl/4671](https://gerrit.lix.systems/c/lix/+/4671)
By default, Lix attempts to locate a cache directory for its operations (such
as the narinfo cache) by checking the value of `$XDG_CACHE_DIR`.
However, since the Nix daemon is a system service, using `$XDG_CACHE_DIR` is
not typical in this context.
To address this, systemd provides a better solution. Specifically, when
`CacheDirectory=` is set in the `[Service]` section of a systemd unit, it
automatically sets the `$CACHE_DIRECTORY` environment variable and systemd will
manage that cache directory for us.
Now, our systemd unit includes `CacheDirectory=nix`, which sets the
`$CACHE_DIRECTORY` and takes precedence over `$XDG_CACHE_DIR`.
If the daemon is run under user units, systemd will automatically set
`$XDG_CACHE_DIR`.
If neither of these variables is set, Lix falls back to its default behavior.
By default, Lix will try to find a cache directory for its various operations
(e.g. narinfo cache) by looking into `$XDG_CACHE_DIR`.
In summary, what was stored in `/root/.cache/nix` is now moved to
`/var/cache/nix/nix`.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
The `fetch-closure` experimental feature has been removed.
Outside of allowing the user to import closure from binary cache,
`fetchClosure` also allowed you to do the following:
* rewrite non-CA path to CA
* reject non-CA paths at fetching time
* reject CA paths at fetching time
Some people are using those mechanism to prevent users from having to build any
package and force going via the declared cache or as a way to use ancient/old
software without paying the evaluation cost of a second nixpkgs.
Both use cases are somewhat of an antipattern in Nix semantics. If the user
cannot fetch a program directly via the substituter mechanism and fall back to
local build, this is a feature *and* a misconfiguration. If the user cannot build
certain derivations because they are too expensive, the build directives should
pass `-j0` or similar.
As for the second usecase, there's a different way to do it that also allows to
have a way to reproduce the paths that are hardcoded in that file, perform
`import (fetchurl "https://my-cache/${hashparts storepath}.drv")` rather, i.e.
an IFD to a possibly well known name. The backend can generate them on the fly
or once, and possess stable names.
Finally, as for the non-CA → CA features, Lix removed ca-derivations.
fetchClosure offers ca-derivations-like features which suffers from similar
shortcomings albeit lessened. It only follows that we should deprecate
and remove these capabilities.
Many thanks to [just1602](https://git.lix.systems/just1602) for this.
## Features
-`nix store add-path` now supports references [cl/5205](https://gerrit.lix.systems/c/lix/+/5205)
Lix supports two categories of hashes in store paths: input-addressed and output-addressed.
Currently, in Nix language, there is no way to produce output-addressed paths with references, as fixed-output derivations forbid references.
However, the Nix store actually *supports* references in output-addressed paths.
This is very useful for importing build products created outside of Lix that reference dependency store paths since such build products have no associated derivation so don't make any sense to input-address.
Previously, output-addressed paths with references could only be created by writing a custom client to the rather-baroque Nix daemon protocol; now it's available in the CLI.
Using `nix store add-path --references-list-json REFS_LIST_FILE SOME_PATH` with a JSON list of string store paths, you can now create such paths with the Lix CLI.
They may be consumed from Nix language using something like `builtins.storePath` or the following which also works in pure evaluation mode:
```nix
# Hack from https://git.lix.systems/lix-project/lix/issues/402#issuecomment-5889
path:
builtins.appendContext path {
${path} = {
path = true;
};
}
```
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Add `builtins.warn` for emitting warnings from Nix code [cl/2248](https://gerrit.lix.systems/c/lix/+/2248)
Lix now has a builtin function for emitting warnings.
Like `builtins.trace`, it takes two arguments: the message to emit, and the expression to return.
_Unlike_ `builtins.trace`, `builtins.warn` requires the first argument — the message — to be a string.
In the future we may extend `builtins.warn` to accept a more structured API.
To go along with this, we also have two new config settings:
- [`debugger-on-warn`](@docroot@/command-ref/conf-file.md#conf-debugger-on-warn), which, when used with `--debugger`, makes `builtins.warn` also function like [`builtins.break`](@docroot@/language/builtins.md#builtins-break).
- [`abort-on-warn`](@docroot@/command-ref/conf-file.md#conf-abort-on-warn), which aborts evaluation entirely after the warning is emitted.
Many thanks to [Emilia Bopp](https://git.lix.systems/milibopp) and [Qyriad](https://git.lix.systems/Qyriad) for this.
- `keep-env-derivations` is now supported for nix3 CLI (`nix profile`) [lix#1095](https://git.lix.systems/lix-project/lix/issues/1095) [cl/5332](https://gerrit.lix.systems/c/lix/+/5332)
The `keep-env-derivations` feature is now available for `nix profile`. This allows users to prevent the garbage collection of derivations used to install a profile, even when `keep-derivations = false` (set to `true` by default).
Previously, `nix-env` supported this feature, but `nix profile` **never** did. This caused issues when garbage collection removed the associated `.drv` files, which are required, for example, by vulnerability management tools (e.g. [vulnix](https://github.com/nix-community/vulnix)) for proper operation.
This issue has now been resolved.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Make `log-format` a setting [cl/4686](https://gerrit.lix.systems/c/lix/+/4686)
The [`--log-format` CLI option](@docroot@/command-ref/opt-common.md#opt-log-format) can now be set in [`nix.conf`](@docroot@/command-ref/conf-file.md#conf-log-format)!
For example, you can now persistently enable the `multiline-with-logs` log format [added in Lix 2.91](@docroot@/release-notes/rl-2.91.md) by adding the following to your `nix.conf`:
```conf
log-format = multiline-with-logs
```
Or the equivalent in a NixOS configuration:
```nix
{
nix.settings.log-format = "multiline-with-logs";
}
```
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- Allow remote builders to be configured using TOML [cl/4533](https://gerrit.lix.systems/c/lix/+/4533)
Lix now supports configuring remote builders using a TOML file instead of the old, very cursed and incomprehensible format.
This comes with not only a human-understandable file, but also with better messages and error reports on misconfiguration.
A more detailed Documentation can be found on the [distributed-builds](@docroot@/advanced-topics/distributed-builds.md) documentation page.
Many thanks to [rootile (Rutile)](https://git.lix.systems/rootile) and [Qyriad](https://git.lix.systems/Qyriad) for this.
- Emit warnings when encountering IFD with `warn-import-from-derivation` [nix#13279](https://github.com/NixOS/nix/pull/13279) [cl/3879](https://gerrit.lix.systems/c/lix/+/3879)
Instead of only being able to toggle the use of [Import from
Derivation](https://nix.dev/manual/nix/stable/language/import-from-derivation) with
`allow-import-from-derivation`, Lix is now able to warn users whenever IFD is encountered with
`warn-import-from-derivation`.
Many thanks to [Seth Flynn](https://git.lix.systems/getchoo), [gustavderdrache](https://github.com/gustavderdrache), and [Eelco Dolstra](https://github.com/edolstra) for this.
## Improvements
- Collect Flakes untrusted settings into one prompt [lix#682](https://git.lix.systems/lix-project/lix/issues/682) [cl/2921](https://gerrit.lix.systems/c/lix/+/2921)
When working with Flakes containing untrusted settings, a prompt is shown for each setting, asking whether to vet or approve it. This looks like:
```
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all)
```
In Flakes with a large number of settings to approve or reject, this process can become tedious as each option must be handled individually.
To address this, all untrusted settings are now consolidated into a single prompt: allowing for bulk acceptance permanently or not, rejection, or detailed review. For example:
### Scrutiny scenario
```console
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all) n
warning: you can set 'accept-flake-config' to 'false' to automatically reject configuration options supplied by flakes
Do you want to allow setting 'allow-dirty = false'? (yes for now/Allow always/no for now) y
Do you want to allow setting 'sandbox = false'? (yes for now/Allow always/no for now) n
```
### Reject everything scenario
```console
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all) N
Rejecting all untrusted nix.conf entries
warning: you can set 'accept-flake-config' to 'false' to automatically reject configuration options supplied by flakes
```
### Accept everything scenario
```console
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all) y
```
### Accept everything PERMANENTLY scenario
Note that accepting everything permanently will authorize these options for any
further operations.
The file containing this trust information is usually located in
`~/.local/share/nix/trusted-settings.json` and can be edited manually to revoke
this permission until Lix provides a first-class command for this manipulation.
```console
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all) A
```
Many thanks to [isabelroses](https://git.lix.systems/isabelroses), [Raito Bezarius](https://git.lix.systems/raito), and [eldritch horrors](https://git.lix.systems/pennae) for this.
- `--check` or `--rebuild` is clearer about a missing path [lix#485](https://git.lix.systems/lix-project/lix/issues/485)
Previously, when running Lix with --check or --rebuild, failures often surfaced
as an unhelpful error:
> "some outputs of '...' are not valid, so checking is not possible"
This message could mean two different things:
- The requested output paths don't exist at all, or,
- Some outputs exist but are not known to Lix
Lix cannot reliably distinguish these cases, so it treated them the same.
We've updated the error messages to clarify what Lix can determine: whether any
valid outputs (> 0) are present or whether no outputs are available.
When no valid outputs can be found, Lix will now suggest building the derivation
normally (without --check or --rebuild) before trying again.
When some valid outputs are present, Lix now reports which ones are valid,
shows the full list of known outputs, and also suggests building the derivation
normally.
In the future, Lix may automate this recovery step when it knows how to rebuild
the paths, but implementing that safely requires more extensive changes to the
codebase.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- `nix develop` no longer ignores the env variable `SSL_CERT_FILE` [cl/5042](https://gerrit.lix.systems/c/lix/+/5042)
Running `nix develop` and `nix print-dev-env` on shells that define the environment variable `SSL_CERT_FILE` now works correctly by exporting that variable inside the built shell.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- Linux sandbox launch overhead greatly reduced [cl/5030](https://gerrit.lix.systems/c/lix/+/5030) [cl/5073](https://gerrit.lix.systems/c/lix/+/5073) [cl/5074](https://gerrit.lix.systems/c/lix/+/5074)
Sandboxed builds are now much cheaper to launch on Linux, with constant management
overhead. This will mostly be noticeable when building derivation trees containing
many small derivations like nixpkgs' `writeFile` or `runCommand` with scripts that
exit quickly. In synthetic tests we have seen build times of 3000 small runCommand
drop from 80 seconds to 14 seconds, which is the most optimistic case in practice.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- mTLS store connections via a plugin [cl/3754](https://gerrit.lix.systems/c/lix/+/3754) [cl/3696](https://gerrit.lix.systems/c/lix/+/3696) [cl/3697](https://gerrit.lix.systems/c/lix/+/3697) [cl/3698](https://gerrit.lix.systems/c/lix/+/3698)
To support use cases requiring mutual TLS (mTLS) authentication when connecting
to remote Nix stores, e.g. private stores, we have introduced a **contributed**
mTLS plugin extending the Lix store interface.
This design follows an extensibility model which was brought up [by a proposal
This mTLS plugin serves as a concrete example of how store connection
mechanisms can be modularized through external plugins, without extending Lix
core. This idea can be generalized to integrate automatic certificate renewal
or advanced integrations with secrets engine or posture checks.
It enables custom TLS client certificates to be used for authenticating against
a remote store that enforces mTLS.
To use the plugin, configure Lix manually by setting in your `nix.conf`:
```
plugin-files = /a/path/to/libplugin_mtls_store.so
```
Currently, this must be done explicitly. In the future, Nixpkgs will provide a
mechanism to reference an up-to-date and curated set of plugins automatically.
Making plugins easily consumable outside of Nixpkgs (e.g., from external plugin
registries or binary distributions) remains an open question and will require
further design.
Contributed plugins come with significantly reduced **stability** and
**maintenance** guarantees compared to the Lix core. We encourage users who
depend on a given plugin to take on maintenance responsibilities and apply for
ownership within the Lix mono-repository. These plugins are subject to removal
at any time.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [eldritch horrors](https://git.lix.systems/pennae), [mic92](https://github.com/mic92), [vlaci](https://github.com/vlaci), and [nkk0](https://github.com/nkk0) for this.
- Add an indication of nix-shell nesting depth [lix#826](https://git.lix.systems/lix-project/lix/issues/826) [cl/4657](https://gerrit.lix.systems/c/lix/+/4657)
When in a nix shell (either via a `nix-shell` or a `nix develop` invocation), a variable `NIX_SHELL_LEVEL` is exported to indicate the nesting depth of nix shells.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- `nix store delete` can now unlink a GC root before deleting its closure [cl/4660](https://gerrit.lix.systems/c/lix/+/4660)
Ever build something, and then you want to delete it and whatever dependencies it downloaded?
Before you had to resolve the `result` symlink and copy it, then delete it, *then* `nix store delete --delete-closure --skip-live` on the path you copied.
Now you can just pass `--unlink` and the `result` symlink itself.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- `nix path-info` no longer lies to the user about fetching paths [lix#323](https://git.lix.systems/lix-project/lix/issues/323) [cl/4866](https://gerrit.lix.systems/c/lix/+/4866)
When running `nix path-info` with an installable that is not present in the store, Lix no longer
tells the user which paths are missing and that they will be fetched, as the documentation clearly
states that this command does not fetch missing paths.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- Derivations can now be printed in detail in `nix repl` [cl/3842](https://gerrit.lix.systems/c/lix/+/3842)
Traditionally derivations printed in the REPL would only print a formatted object
representing the path of the derivation file it refers to. This makes inspecting
the enhanced derivation attribute sets encountered from `mkDerivation` or similar
wrappers more difficult. Even the `:p`/`:print` command would not elaborate attribute sets
tagged as a derivation.
With this change you can now use `:p`/`:print` to directly inspect a derivation
by providing one as the top-level object. Derivation attribute sets will only be
printed two levels deep and internal derivation attrsets will remain in unexpanded
path form as before. `drvAttrs` will also be elided as these attributes are already
present in the top-level attribute set of the derivation. These heuristics provide
a balance between readability and functionality. When the `:p`/`:print` is omitted,
a bare derivation is printed in the path format as before.
Many thanks to [Lunaphied](https://git.lix.systems/Lunaphied) for this.
- Reject `__json` in structured attributes derivations [lix#380](https://git.lix.systems/lix-project/lix/issues/380) [cl/5286](https://gerrit.lix.systems/c/lix/+/5286)
In structured attributes derivations, `__json` is used internally to store the
JSON representation of the `env` attribute field that users can set.
Unfortunately, a user can set `__json` *and* enable structured attributes,
resulting in a broken derivation from a semantic point of view.
As no user can benefit from setting `__json` *and* enable structured attributes,
we disallow that possibility and throw an error from now on.
This is not seen as a breaking change because there's no user code that can
benefit from this behavior, hence, it's an improvement to user experience.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Shells support `$NIX_LOG_FD` now [lix#336](https://git.lix.systems/lix-project/lix/issues/336) [cl/4694](https://gerrit.lix.systems/c/lix/+/4694) [cl/4695](https://gerrit.lix.systems/c/lix/+/4695)
Lix's "debugging" shells (`nix3-develop` and `nix-shell`) now set the
`$NIX_LOG_FD` environment variable.
This means that [hook logging in
stdenv](https://github.com/NixOS/nixpkgs/pull/310387) appears while debugging
derivations via `nix3-develop` or `nix-shell`.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Supplementary groups are now supported for daemon authentication [lix#968](https://git.lix.systems/lix-project/lix/issues/968) [cl/5021](https://gerrit.lix.systems/c/lix/+/5021)
macOS, FreeBSD and Linux now support receiving supplementary groups during UNIX domain authentication to a Lix daemon.
This change is particularly beneficial for systemd units with `DynamicUser=true` that need to connect to a Lix daemon, using a `SupplementaryGroups=` allocated by systemd in the context of the process. This is desirable if you wish to harden Lix clients.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [Tom Hubrecht](https://git.lix.systems/tom-hubrecht), [alois31](https://git.lix.systems/alois31), and [eldritch horrors](https://git.lix.systems/pennae) for this.
## Fixes
- Nix shells' `$NIX_BUILD_TOP` are shorter [lix#1044](https://git.lix.systems/lix-project/lix/issues/1044) [cl/4663](https://gerrit.lix.systems/c/lix/+/4663)
Following the changes in 2.94.0 to shorten build directory paths, aimed at [resolving UNIX domain socket length issues](https://gerrit.lix.systems/c/lix/+/4168/13) and [improving nix-shell](https://git.lix.systems/lix-project/lix/issues/940), we inadvertently introduced an excessively long path for the `$NIX_BUILD_TOP` environment variable used by Nix shells (their effective temporary `/build` directory).
To fix this, we replaced the `build-top-$HASH` directory name with simply `build-top`, reducing these paths by at least 30 characters.
We also added a test to ensure that Nix shells do not introduce more than 50 extra characters relative to their base directory (e.g., `/tmp` when `$TMPDIR` is not set).
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Fix resolving of symlinks in flake paths [lix#106](https://git.lix.systems/lix-project/lix/issues/106) [lix#12286](https://git.lix.systems/lix-project/lix/pulls/12286) [cl/4783](https://gerrit.lix.systems/c/lix/+/4783)
Flake paths are now canonicalized to resolve symlinks. This ensures that when a flake is accessed via a symlink, paths are resolved relative to the target directory, not the symlink's location.
Many thanks to [stevalkr](https://github.com/stevalkr) and [xyenon](https://git.lix.systems/xyenon) for this.
- The REPL no longer considers failed loads for `:reload` [lix#50](https://git.lix.systems/lix-project/lix/issues/50) [cl/4864](https://gerrit.lix.systems/c/lix/+/4864) [cl/4865](https://gerrit.lix.systems/c/lix/+/4865) [cl/4700](https://gerrit.lix.systems/c/lix/+/4700) [cl/4889](https://gerrit.lix.systems/c/lix/+/4889)
The [REPL](@docroot@/command-ref/new-cli/nix3-repl.md) allows "loading" files, flakes, and expressions into the environment, with the commands `:load`/`:l`, `:load-flake`/`:lf`, and `:add`/`:a` respectively.
The results of those stay in the environment as-is even if their sources change, until the `:reload` command is used.
However `:reload` would re-perform *all* instances of `:l`/`:lf`/`:a`, meaning you would get things like this:
```nix
nix-repl> :l /tmp/texting.nix
error: getting status of '/tmp/texting.nix': No such file or directory
# oops, typo.
nix-repl> :l /tmp/testing.nix
# Do some stuff…
nix-repl> :reload
error: getting status of '/tmp/texting.nix': No such file or directory
```
This is pretty silly, but also *incredibly* annoying, as it would stop there and *not* reload the correct files anymore.
This effectively meant typoing any of the load commands would make `:reload` useless for the rest of the entire `nix repl` session!
This has been fixed, so now only *successful* loads count towards `:reload`.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [Qyriad](https://git.lix.systems/Qyriad) for this.
- Consistently use commit hash as rev when locking git inputs [cl/4762](https://gerrit.lix.systems/c/lix/+/4762)
Lix will now use commit hashes instead of tag object hashes in the `rev` field
when fetching git inputs by tag in `flake.lock` and `builtins.fetchTree` output.
Note that this means that Lix may change some `flake.lock` files on re-locking. Old `flake.lock` files still remain valid.
Many thanks to [goldstein](https://git.lix.systems/goldstein) for this.
## Development
- Functional lang migration [lix#856](https://git.lix.systems/lix-project/lix/issues/856) [cl/3213](https://gerrit.lix.systems/c/lix/+/3213)
We have done it! The functional/lang framework has now been fully migrated to functional2/lang.
This means: no more `just clean` and `just install` mess and whatever because one removed a test.
The lang test suite is also getting a face lift, with an improved folder structure and restructuring of many tests.
Only the first CL of the chain is provided but there's way more changes associated to this project.
Many thanks to [piegames](https://git.lix.systems/piegames) and [rootile (Rutile)](https://git.lix.systems/rootile) for this.
## Miscellany
- Warn instead of erroring when the final destination of a transfer changes in-flight [lix#1004](https://git.lix.systems/lix-project/lix/issues/1004) [cl/4641](https://gerrit.lix.systems/c/lix/+/4641)
Lix will now emit a warning during downloads where the final destination changes suddently mid-transfer instead of throwing an error.
This transfer behavior has been known to happen very rarely while fetching from some CDNs.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
f"This function is only available if the [{self.experimental_feature}](@docroot@/contributing/experimental-features.md#xp-feature-{self.experimental_feature}) experimental feature is enabled."
ifself.experimental_featureisnotNone
else""
}
</dd>
""")
defmain():
importargparse
ap=argparse.ArgumentParser()
ap.add_argument('--header',help='Path of the header to generate')
ap.add_argument('--docs',help='Path of the documentation file to generate')
ap.add_argument('--experimental-features',help='Directory containing the experimental feature definitions')
'''+(f'''This function is only available if the [{builtin.experimental_feature}](@docroot@/contributing/experimental-features.md#xp-feature-{builtin.experimental_feature}) experimental feature is enabled.
f' - [`{feature.name}`](@docroot@/contributing/{"deprecated"ifargs.deprecatedelse"experimental"}-features.md#{"dp"ifargs.deprecatedelse"xp"}-feature-{feature.name})\n')
This is the simplest invocation, and it does not require the user of the expression to configure [`trusted-public-keys`](@docroot@/command-ref/conf-file.md#conf-trusted-public-keys) to ensure their authenticity.
If your store path is [input addressed](@docroot@/glossary.md#gloss-input-addressed-store-object) instead of content addressed, consider the other two invocations.
**Fetch any store path and rewrite it to a fully content-addressed store path**
It is possible to fetch an [input-addressed store path](@docroot@/glossary.md#gloss-input-addressed-store-object) and return it as is.
However, this is the least preferred way of invoking `fetchClosure`, because it requires that the input-addressed paths are trusted by the Lix configuration.
**`builtins.storePath`**
`fetchClosure` is similar to [`builtins.storePath`](#builtins-storePath) in that it allows you to use a previously built store path in a Nix expression.
However, `fetchClosure` is more reproducible because it specifies a binary cache from which the path can be fetched.
Also, using content-addressed store paths does not require users to configure [`trusted-public-keys`](@docroot@/command-ref/conf-file.md#conf-trusted-public-keys) to ensure their authenticity.
@@ -10,7 +10,8 @@ present in *args*. All are optional except `path`:
- name\
The name of the path when added to the store. This can used to
reference paths that have nix-illegal characters in their names,
reference paths that have
[nix-illegal characters in their names](./derivations.md),
like `@`.
- filter\
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Jade Lovelace