This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh
Change-Id: I28226b8a91530d6d261454d21f2541b25389daff
string data shares a buffer with the binary string length field. size
calculations for string read buffers always include the length field;
sufficiently large length fields can cause these calculations to wrap.
a malicious nar could use this for OOB writes in the daemon (as root).
since we use strings only as tags for archive members and for symlinks
with their OS-dependent length limits we can simply limit string size.
1 MiB should be sufficient for all symlinks, and tags are always tiny.
Change-Id: I89fb05f73c1dbeda45d91244aba4cd526a3d83e1
I don't understand how `ssh -l lix-releng` is supposed to work if it
doesn't say which host to target.
Change-Id: I791f3f3f49ecd5884c9e86b5d3b617fc139e031f
Signed-off-by: Raito Bezarius <raito@lix.systems>
These parameters are now created on https://s3.afnix.fr.
Change-Id: I96b6fd913429ee46d04c412cb141edd288665ced
Signed-off-by: Raito Bezarius <raito@lix.systems>
When index-debug-info is enabled and the store path being copied is a
regular file (not a directory), std::get_if<nar_index::Directory>
returns nullptr since the NAR root is a File variant. The loop then
immediately dereferences buildIdDir->contents on the null pointer,
causing a segfault.
Add a null check at the top of the loop to break early when the NAR
root is not a directory.
Change-Id: I3a6e792b84cc12c837ecaddf4fee889e1bcb6397
(cherry picked from commit 6c7ccc2588)
The culprit was a space added at the wrong place, which messed with c++
raw strings.
Fixes#1178
Change-Id: Ic1e09cb7215d9a6dd2d13fd92242649b0e1fcd13
(cherry picked from commit cbeb4fcd69)
lowdown 3.0.0 merged some flags into one to save on bits and did not add
any aliases for backward compatibility.
As with the changes for lowdown >= 1.4, we define a preprocessor flag to
gate the changes on and add a job to CI to ensure that lowdown < 3.0
keeps working (which is used by NixOS 25.11).
Unfortunately, we need to jump through some hoops to prevent nix and lix
from upstream Nixpkgs from being rebuilt due to a changed lowdown. Since
both implementation's packaging in Nixpkgs has their own package set /
fix point now, we can't simply inherit them from `prev` since they will
always be (re-)computed from the `final` fix point. As a consequence,
we need to expose our changed lowdown version at a non-default attribute
or break the builds of Nixpkgs derivations we test against.
Change-Id: I20a3e2fdaa05906f032ff66911c42867557fdd11
(cherry picked from commit af0390c27b)
This causes a build error with lto, and according to Jade is not
strictly needed anymore.
Change-Id: I41e53a57f40711061effe08f78545011a4b51754
(cherry picked from commit 96db7c79cf)
The manual for the builtins and nix.conf currently has inconsistent
indentation, which causes some of the descriptions to end up being
partially treatedas code blocks in markdown (and thus the manual).
This was simply caused by the template string for the docs having
too much indentation before the description is inserted, so this fixes
that 16-bytes mistake.
Change-Id: Ia264e3b1abb20430109029d07a2d2b0a1a726bd4
(cherry picked from commit 8294cd534b)
- launch-builder-linux.cc was missing an include for musl
and used function that are not defined in the launchers
- musl caches pids used for raise, breaking sandbox setup
- the mtls contrib plugin won't build, didn't try fixing,
static builds can't really use plugins reliably anyway.
Change-Id: I5ab1664e45ea977e5bcf05e41d825e6014e62146
(cherry picked from commit 022e43aa7f)
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh
Change-Id: Ia7a63807681389cf5ccb9d07539b880697a88be5
This avoids a large list of CLs when a user can discover all the CLs by
browsing the chain.
Change-Id: I1890eb87e8bf6f930d3d00c3f7156690bb8de068
Signed-off-by: Raito Bezarius <raito@lix.systems>
This might do more copies than are necessary. I think that's fine.
Sorry, :reload not reloading has been bothering the shit out of us.
Fixes#1115.
Change-Id: I7f4caca3dad39696ce66ce57b1a520836a6a6964
So far, we had both `fj#` and `lix#` issue references in our release
notes, due to supporting both in the md files.
This commit unifies those in order for all references to be consistent.
Change-Id: I67c75784b460bfe4ebf214c2f6ac105cc60a7bad
kj can and will throw exceptions immediately without wrapping them as
promise results, and we must take care of these as well to not crash.
fixes#1150
Change-Id: Ifa29d0f3da4a6490f3e45d4c04a8fa49eaa20f43
mdbook-linkcheck was dropped from nixpkgs and replaced with mdbook-linkcheck2,
which is not entirely a drop-in replacement, but drop-in enough for us.
Change-Id: Iab03efab89ebae70811e0de3ea7cf0a9a059a9e4
Since commit ac64c727b5, during launch of the
builder it is attempted to raise all capabilities into the ambient set.
Specifically, what "all" means here is determined by the Linux API headers Lix
was built against.
Occasionally, new capabilities are added in the Linux kernel, leading to
PR_CAP_AMBIENT_RAISE failing with EINVAL on the newly added capabilities if the
API headers are from after the change but the running kernel is from before.
Similarly to how capset already silently ignores nonexistent capabilities, we
ignore this error so the builder doesn't fail to launch unnecessarily.
Unfortunately it is very hard to test for this situation currently, since the
last time a capability was added was CAP_CHECKPOINT_RESTORE in Linux 5.9, and
all kernel versions in nixpkgs are newer than that.
Change-Id: Ibeb2f0757729b877bd3ca9f02e1aa4536a6a6964
not doing this has lix crash if stdio forwarding fails for odd reasons,
such as ssh failing to set up a connection due to transient dns errors.
cf #1148
Change-Id: I543b7444236e69d3c13b47fb6c6eb37c65b96763
This is intended to be a softer version of
`allow-import-from-derivation`, allowing for users to only flag when IFD
is encountered rather than error out
Based-on: https://github.com/NixOS/nix/pull/13279
Co-authored-by: gustavderdrache <alex.ford@determinate.systems>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Change-Id: I879a444e3a24b4e82a8cea3a5a94834c78c3924a
`builtins.storePath` allows you to use "external" knowledge of present
store paths in your store to be used in your expressions.
This is sometimes used to avoid double wrapping store paths, for example
when dealing with nixpkgs sources (`builtins.storePath pkgs.path` is a
frequent pattern).
Unfortunately, using your external knowledge of present store paths is
technically impure because Flakes is about not depending on the *STATE*
of your store, no matter what you know about it or what you do to ensure
that your knowledge is accurate.
That being said, `builtins.storePath` is already possible in the
fragment of "pure Nixlang" via a clever use of `builtins.appendContext`.
Precisely:
```
storePath = path:
let path' = builtins.unsafeDiscardStringContext path;
in
# NOTE: merging the context set in all generality is impossible
because getContext on a pure path doesn't work.
builtins.appendContext path' { ${path'} = { path = true; }; };
```
works in pure contexts.
Because of this, we deem that storePath should be allowed in pure
contexts and users do not have to come up with a `storePath` polyfill.
Fixes#402.
Change-Id: I5fff750a4feed082745431835db451f945249cfb
Signed-off-by: Raito Bezarius <raito@lix.systems>
nix3 CLI never honored that setting.
When we update a nix3 profile element with new built store paths for
which we know the deriver (the derivation path which built it), we add
the derivation paths to the profile element to prevent garbage
collection the same way the nix2 CLI logic does.
Fixes#1095.
Change-Id: Icc236d174c5ce5ffe042d1c8e1968f6cb5b50359
Signed-off-by: Raito Bezarius <raito@lix.systems>
Instead of showing untrusted configuration items one by one, collect all
of them into one batched ask with 4 options:
- apply the configuration wholesale **for now** (yes)
- apply the configuration wholesale and trust it permanently (allow
permanently)
- selectively choose subset (no)
- rejecting the configuration wholesale (no to all)
Note that we do not offer options to reject permanently a configuration,
that's what `--no-accept-flake-config` is for.
Individual prompts gain a new option to distinguish "yes for now" and
"yes forever".
Fixes#682.
Change-Id: I2492717ac277af33ea4ac9b1a11bd7d6d1bfe0db
In messages printed by the evaluator, we use showType(.) to say things
like "expected X, got Y" where X, Y can be string, string with context,
etc.
After changes in the Lix data model (pointer tagging and friends), paths
got squished inside a internal string type.
We missed updating `showType` to take this fact into account which
caused an unfortunate:
`error: expected a string but found a string with context:
/nix/store/wjf8rdpp63rhyasbzc7zfms1agf7pwyd-source`
while using a path on `builtins.getContext`
A test is added.
Change-Id: Ifb0129ead2ea1973fc17dc0133f8a694995536b1
Signed-off-by: Raito Bezarius <raito@lix.systems>
This partitions a string into two parts based on the first occurrence of
the separator.
This function handles the case when the separator never occur in the
target string by returning an optional<T> in the second argument of the
pair.
Change-Id: I9ad44375dbaa137a4d21d2408bc2ffa34fab62a7
Signed-off-by: Raito Bezarius <raito@lix.systems>
Parsing and completing enumerations on the CLI was usually done by hand.
With this enum trait, the parsing and completion is automated away.
Change-Id: Icd49729564355d4c79e2c8050ce7e8b9d8e7eda3
Signed-off-by: Raito Bezarius <raito@lix.systems>
36c4d6f592 introduced it commented. Zero
explanation why.
horrors suggested to drop it entirely because it hide the options in
categories but they are pretty common and we should make as much visible
as possible.
Change-Id: I09894b5ba0f2b2cce5869136914289fed034fce3
Signed-off-by: Raito Bezarius <raito@lix.systems>
When `max-jobs = 0`, our async semaphores have null capacity and
therefore no slot token can be acquired.
In addition, a derivation with `preferLocalBuild = true` with `max-jobs
= 0` which is not a built-in derivation is not considered suitable for
being possible to build.
Our documentation in `max-jobs` writes:
> `0` is useful when using remote builders to prevent any local builds
(except for preferLocalBuild derivation attribute which executes locally
regardless).
A valid interpretation is that `preferLocalBuild` derivations (all of
them) will always execute even if `max-jobs = 0`.
This CL achieves this promise by enabling `preferLocalBuild = true`
derivations to draw in priority capacity from a `preferLocalBuild`-only
pool and then fall back to the build pool (`maxBuildJobs`).
If there's non-zero max-jobs, we make the `preferLocalBuild`-only pool
empty.
If there's zero max-jobs, we make the `preferLocalBuild`-only pool
non-empty (capacity: 1).
Remark 1: if `max-jobs = 0`, no non-`preferLocalBuild = true`
derivations can be built without a remote builder and `preferLocalBuild
= true` derivations can be built one at a time if `extra-local-jobs` is
not overridden.
Remark 2: if `max-jobs != 0`, no `preferLocalBuild = true` derivations
can bypass capacity limits, i.e. `max-jobs` unless `extra-local-jobs` is
overridden.
Fixes#855.
Change-Id: I00a0acac7375a881aec6281ebb13a4530b26e460
Signed-off-by: Raito Bezarius <raito@lix.systems>
This is going to be used in the extra-local-job future setting.
Change-Id: Ied76f32b8fa97cb71691d76167103001aa14fa01
Signed-off-by: Raito Bezarius <raito@lix.systems>
When ref/rev are very similar looking, e.g. a revision SHA1, the logic
to reconstruct whether we had a ref or rev initially cannot work.
This regression was introduced by
I2c72d5a32e406a7ca308e271730bd0af01c5d18b which passed the information
as an attribute field called `refOrRev`, instead, we pass `ref` or `rev`
but not `refOrRev`.
We apply the same post-processing logic to distinguish rev/ref in
path-style flake references: `github:owner/repo/$refOrRev`.
Fixes#520#841.
Change-Id: I2b1985cd5f351f43fea84232c1972715e2408fb1
Signed-off-by: Raito Bezarius <raito@lix.systems>
In structured attrs derivations, __json is used to serialize the `env`
attribute into a JSON object.
If the derivation enables structured attributes AND specify its own
`__json`, one of the field will take precedence (the reader can have fun
by guessing which one).
To reduce underspecification, we disallow `__json` in structured
attributes derivations.
Fixes#380.
Change-Id: I51c3b2af1ff9449471ba81d4c72df6a7a263eef7
Signed-off-by: Raito Bezarius <raito@lix.systems>
`nix.full_path()` is the *built* path, not the *installed* path. in the
sandbox this point to a location that contains *only* `nix`, not any of
the old-cli symlinks. any test that needs them in path would thus fail.
Change-Id: Ia651db8d1151a84f8d1b52e8993f6db61b0fe104
Without the `--git-dir` git will attempt to figure out which the git directory is on its own.
This causes issues when the `GIT_DIR` envvar is set in the environment for other reasons (such as being part of a `git rebase`).
Providing this parameter overrides the environment and thus makes sure the revs are always coming from the intended repository.
Fixes: #1135
Change-Id: Ibd43cfd8e0278463c0e1ccf86779982947a6038a
Signed-off-by: benaryorg <binary@benary.org>
To stop spurious warnings, we will stop this bizarre behavior of setting
empty values to impure environment variables.
On the warning side, we verify that the presence of the environment
variable.
The corresponding test is updated to "there is a warning" to "there is
no warning".
Change-Id: I12f5c6445ef00a83d269488d7aed0b0f61aeec12
All impure environment variables are initialized as part of `initEnv()`
to an empty value if they are unset.
Whenever `NIX_SSL_CERT_FILE` is set, this cause a spurious warning.
Let's document it here as a test before we fix it.
Change-Id: Iccba8ccb487d9cc25490e940ff09c1c5a13ae097
Signed-off-by: Raito Bezarius <raito@lix.systems>
CA tests have grown quite complicated and are hard to debug and
manoeuver for new needs.
They are a prime target for F2 migration and they were extremely
pleasant to write.
Most functionality is preserved or improved with additional behaviors
asserted on warnings and the full set of reports.
Change-Id: I70cd52abbe5ff13f56744b966ab00962ad3a49d6
Signed-off-by: Raito Bezarius <raito@lix.systems>
Instead of taking in a final argument `Value &` out parameter which it
writes to, it now returns its result
Change-Id: Iab6bc3a3ac6a4b17c6d31115a766a6ea6a6a6964
we already report errors that happen during reads via exceptions
rethrown from read. reporting errors during destruction too is a
source of confusion for users since these errors are not usually
relevant and just the decompressor thread reporting short reads.
shorts reads during shutdown are not an error, they're expected.
fixes#1138
Change-Id: I7983ef0ed7722460f1d683bc8171a454b3d06e90
I could instead add the right braces or manually ensure the
XMLOpenElement destructor is run at the right time. Or I could just
write what I actually want to happen.
Fixes#1137.
Change-Id: Ie1f157de1632571fb0c21e26d8e3cae56a6a6964
This is a rebase of CL 1626 after the performance problems that blocked
that change are no longer an issue.
Change-Id: I4a17f7606c5fdc9a924933a860eb401f6a6a6964
The pseudo-constructor `Value::mkFloat` would previously be called on an
default-constructed (under-initialized) `Value` to create a properly
initialized `Value` that represents a float. This change removes mkFloat
and constructs float `Value`s directly.
Change-Id: I48ae3a836842ce5e5eda6323404ba7576a6a6964
The pseudo-constructor `Value::mkStringMove` would previously be called
on a default-constructed (under-initialized) `Value` to create a
properly initialized `Value` that represents a string, reusing an
existing GC allocated Str. This change turns that method into a proper
constructor
Change-Id: I8d7a97e2afce231a04ecf56fdd6d22d06a6a6964
it sometimes happens that one forgets to add `.run().ok()` to a command,
leading to things not actually being tested, potentially even slipping
though code reviews!
This commit adds a check at fixture teardown time, checking that all
commands have been run. Sadly, due to when this check happens, we cannot
add a test checking for this behavior
Change-Id: If3df2c20c267b9f03ae0ca099b77c57e2310f335
this includes tests that fail teardown when *skipped*, which confuses
the tap parser by reporting more tests than advertised in the header.
not sure what to do about this though, and it is better than nothing.
Change-Id: I4fc6badacbfa3b3d5f5f4a4ad76b177379bb4a30
pytest-tap didn't get it right in combination with xdist. we need to
replace the reporter *completely*, which is just incredibly weird :/
Change-Id: I2e35b9dc8ea4b24563b9b3306312778781eb674e
The pseudo-constructor `Value::mkPath` would previously be called on an
default-constructed (under-initialized) `Value` to create a properly
initialized `Value` that represents a path. This change
removes mkPath and constructs path `Value`s directly.
Change-Id: I9021de1ff59490828f9fec6866e083996a6a6964
The pseudo-constructor `Value::mkAttrs` would previously be called on an
default-constructed (under-initialized) `Value` to create a properly
initialized `Value` that represents an attribute set. This change
removes mkAttrs and constructs attr `Value`s directly.
Change-Id: I11cd801eefecd454a7a5b6229d6770d06a6a6964
This is not ca-derivations. We don't care about self-references or any of the
rewriting or stuff like that; if you want something like that, write
your user code so it figures out where it is.
The reason I want to do this is for integrating other build systems with
Lix: if you're importing something built *outside the store* with a
non-Nix build system, it makes no sense to put it in an input-addressed
path since it's not possible to come up with a derivation for it.
But you need *some* hash, so the output hashing is the most sensible option.
It is also nice because of not needing trusted user privileges to import
it, since the hash describes the exact contents of the output path.
We use this feature (implemented externally to Lix; this CL eliminates
that step) at Mercury to integrate buck2 with Nix on the output side: we
import things to the Nix store as ca paths with references.
These can then be consumed by Nix language with:
```
# Hack from https://git.lix.systems/lix-project/lix/issues/402#issuecomment-5889
path:
builtins.appendContext path {
${path} = {
path = true;
};
}
```
Test plan:
```
$ nix store add-path --references-list xx-refs-list ./README.md
/nix/store/szcwnm13d9gmxx8fly0bz82l42jgysw6-README.md
$ cat xx-refs-list
/nix/store/g9hhwjbkdrw0wnbd3axfs4icyb37nr6b-bash-interactive-5.3p3
$ nix path-info --json /nix/store/szcwnm13d9gmxx8fly0bz82l42jgysw6-README.md | jq .
[
{
"ca": "fixed:r:sha256:1pj59fy7zcrn949ry9kgv4ba17h2sy3z2is469abj912cvbl14n3",
"narHash": "sha256-w5JA12YiJLlUMkRH8YfXAp6gFtlvJp8TSTazf7xLRd4=",
"narSize": 1384,
"path": "/nix/store/szcwnm13d9gmxx8fly0bz82l42jgysw6-README.md",
"references": [
"/nix/store/g9hhwjbkdrw0wnbd3axfs4icyb37nr6b-bash-interactive-5.3p3"
],
"registrationTime": 1771266146,
"valid": true
}
]
```
Change-Id: I005a03003dfc24108e018e599dbe0b5d6a6a6964
this requires some hacks to make error reports visible, but hey. it's
better than no progress reports at all, and successful runs no longer
spew huge amounts of useless log output onto the developer's terminal
Change-Id: I9e4766b2f825a4ec451e117eb4609db65f328785
Fixes path resolution issue with meson-embedded bash script when
`build` is a symlink, or a custom build directory is passed as a
argument to `just` or `meson`.
Change-Id: I67b13c524d196241bb9a8a3b8ad2c17c2c094ed8
not all operations that involve workers need to build. for example nix
copy --from calls instantiate thir worker with the *origin* store, not
the store they copy *into*. for binary cache stores this is not likely
to ever work if cgroups are enabled and the copy is run on normal user
credentials, even though no cgroups will ever be needed to copy things
fixes#1088
Change-Id: I065e29e1a1d0f58d81823609ef0701ff82cdd1d5
Affects `github`, `gitlab` and `sourcehut` fetchers.
Previously this was an assertion, which crashed the evaluator
with SIGABRT.
Fixes: #1133
Change-Id: Ia9bb8dd29ac8b9c97bf048827f62c5076a6a6964
overridden-ness can be reset, and is usually reset such that setting
value taken from a config file are not considered as overriden. when
launching builtin builders we *do* want to send config file settings
changes to the builder, so we'll need one more getter method for it.
Change-Id: I861538a469121c77ebc1898a276439e6b756797d
diverted stores are only necessary when the logical store paths of
objects matter for the test itself, such as for derivation hashes,
substitution from golden sample nars, or actual tests of the store
diversion functionality. all other tests can use undiverted stores
to run, especially since only linux can build in diverted a store.
Change-Id: I62f0907bdef9961609af22b610195fcec54c1e57
When the destructor of PathSubstitutionGoal is run, this happens in a
sync context and can cause starvation of all ongoing IO w.r.t. to other
substitutions, including our own substitution.
While there's only a decompressor thread per stream, the other side of
the IO runs on the event loop.
In order to fix this, it is sufficient to remove the thread indirection
and inline the async code.
Fixes#1126. Great thanks to horrors' patience.
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I3eb37bc37d156f0f5528364e568fdaa2ced58011
Signed-off-by: Raito Bezarius <raito@lix.systems>
Looks like too much stuff is breaking on this, so let's make it only a
warning for now to ease transition
Change-Id: I52d50ceb1fe1fbe9f4e28d0aabf3537f7e4d52d1
Explicitly catch common errors (trying to escape line breaks, badly
escaping interpolations) to provide better messages and user guidance
for these cases.
Change-Id: I3dd1b2ad3bca33be393e65be5e72f4fb9544a46a
StreamFdMessageReader reads message data *lazily*. if you don't access
all segments of the message before you close the underlying stream you
may find yourselv reading from something *very* different, and in this
case that something is The Void™. this causes reads to fail, writes on
the other side to fail to match, and finally our build launch to fail.
this does not lead to happy outcomes, so we will copy the full message
into a fresh new buffer *before* we try to access any of its contents.
fixes#1118
all-analysis-by: deprekated <kate@lix.systems>
Change-Id: I105540831fde855817194e9e539acf177f54a6f4
This is the long awaited refactor of the NixSettings.
It allows one to set, unset and update any and all settings with a neat
and easy-to-use interface
closes#846
Change-Id: Id4cfb5f853cc1168b506a1f6f405076f3a7cab65
if a substituter is entirely offline and cannot be queries at all we
should not be failing if other substituters are configured. likewise
if a substituter goes offline after querying but before we try using
it we should attempt to fetch that path from some other substituter.
ideally we'd treat all substituters as a single entity instead of as
one store each, then have that single entity take care of fallbacks,
retries, error reporting, etc. that requires larger rewrites though.
fixes#1061
Change-Id: I9d8fc0544ff380bf017256e8fcc82823dc634f10
we don't use std::cerr often enough to want this, and since cerr flushes
after every operation anyway it never did anything useful to begin with.
Change-Id: Ia54be340826da5073e9a1786c463555f4f0e491c
kj exceptions get wrapped as Error instead of passed through as is.
luckily this also means that we can add context to them very easily
Change-Id: Icedab6c016f4434447dd38ba14138c102fc6149a
oops. we have to use init* here, not get*. get returns a discarding
builder for something that was not inited previously, but the linux
cause worked anyway because the first member is created by default!
Change-Id: I40f8a12a04eef2f4e3a80d1537ac9b975490a027
this allows us great flexibility in how children are launched (since the
actual launching is done by a separate executable), makes fork no longer
needed in the core codebase (outside of runProgram, anyway), and we even
get to use linux vfork to its full potential to decrease the launch cost
of sandboxes to a constant factor (previously it was O(#drvs + #deps) of
the build graph, which obviously goes to n² quickly if you are unlucky.)
Change-Id: I66e2d1b20242dc24d708666ef325fb8725bd9296
we can have a ProcessGroup for a pid that *should* be a process group
but hasn't gotten around to setting its pgid yet. in such cases we do
want to be killing the thing anyway, not shoot into the void and hope
the right thing falls over. so far this has not been a problem due to
a mixture of just not having done this and being slow enough to work.
Change-Id: I4e0e54513252d8e18256b9286b819bfa957d70dc
if we move this out of libstore and don't want to pull in all of libutil
with its myriad side effects std::filesystem is our best option, by far.
most notably we don't replace pathExists because std::filesystem::exists
behaves like `stat()` on symlinks, not like the `lstat()` as we require.
Change-Id: I1e488418dcabb33f2ebb73d8c3d1b43528aa51f1
since we're using tmpDirInSandbox as the working directory for the new
process we also rename it accordingly. buildUser likewise turns into a
different Credentials type because exposing user lock state seems odd.
Change-Id: Id4a1a6eb733f774c893f373b91c2a271a4b84185
setting the signal is not enough, we must also check that the process we
expect to be parent to actually *is* our parent, not another process (eg
init if the daemon exited). we also have to set the death signal *after*
all set[ug]id calls, otherwise it will be cleared again by such changes.
Change-Id: I4e8c9102ea407576ed85b3203c8bb9bfb56762de
it's only used this once, and it accesses a (cached) global variable for
the filter. we want to move all sandbox setup state into a single object
soon, moving filter setup will make it a little bit easier to deal with.
Change-Id: I234d92d5ca044a16644b70bd303bfb7956c120f0
when starting builders we want the inner processes to run with a blank
slate. if some signals are masked for any reason the builder processes
may attempt to send signals to each other that are never delivered; we
avoid this by unmasking all singals. since every build tree also has a
session and process group of its own we are not in danger of sending a
signal to a builder by accident from any source, so unmasking is okay.
Change-Id: I90720ed2bd44502ffb6d2bb848c05369809abeba
we don't need to mess with this rlimit for e.g. the daemon. increasing
the limit later should always be safe since we don't allocate (or map)
much before constructing the eval states that ultimately fill our heap
and could thus make stack expansion impossible after some time passes.
Change-Id: Ieafda537fbc99a6a7f83a093a981e7df947da437
this will be needed in other places in the future, and splitting it out
also clarifies the surrounding code. the keep-caps dance launchPasta is
doing doesn't have to be moved since it is only needed to allow setuid.
Change-Id: I6baaa138c2b1bca9626971ed3266c1a971a63acc
the devshell no longer worked for us due to the recent deprecations :D
let's update pre-commit too because while it doesn't fail it does warn
Change-Id: Ic2060c82e4e97bb7a96cebd29097abefabdfe733
it's ancient, it does not do what it was added for (improving
determinism), and it probably hasn't been needed for anything
in a very long time. we can probably get away with killing it
Change-Id: I66f5c841964d9ad62726cb2dae8d6fc42095399f
mostly because this will make it easier to change the data source for
them, and especially to change to data source to something immutable.
Change-Id: Ic5f1a55d02d4c2cc928c3f8b308d07167efb467b
closing them as soon as we're done ensure that the fd table doesn't
balloon in size, which in turn lets us build larger graphs. it also
makes forking processes cheaper due to less fd table copy overhead.
Change-Id: I233d1b7358ffe1b05680f845cd99de38b179f449
In Nixpkgs, there are several strings like "\d\.\d" which attempt to be
a regex but are just literally "d.d". The escaping rules are silly and
we should warn our users about that.
Co-authored-by: Commentator2.0 <lix@crystal-cavern.systems>
Change-Id: I779b0757358fc9adc34dc140e1670b83abc93b67
The indentation stripping semantics of strings are fairly bad and have a
few gotchas where they behave unintuitively. But the good news is, that
these cases are easy to catch and can be avoided.
This commit adds a warning in the parser when such strings are detected.
Unfortunately Nixpkgs uses this kind of a lot, so we won't be able to
actually enable this warning for a while to come.
Co-authored-by: Commentator2.0 <lix@crystal-cavern.systems>
Change-Id: I3b3b68c2eee4cd70959d3f4ca643cb6caf3a2217
Unfortunately, the previous code only detected superficial __overrides
and did not catch cases like `rec { __overrides.foo = 2; }`.
Change-Id: Ic38bdef1a6a3bdea91915aef44447f4ecc238259
Changed the writing style of the descriptions, expanded with more
examples and rationale, and added the new timline metadata in the
frontmatter.
Change-Id: I218389e3504fc21f4eb45a927e77a41a1a70d4f5
The code within the code-generation had tons of code-duplicates and was
overall quite meh to read, understand and expand.
This commit refactors the code-generation to make it more readable and
comprehensible, while also unifying its usage a bit more
Change-Id: I3a5df8b7d8d9b6c76e02ef47dfb151c7dab7d0ab
lix, where modularity is made up and the patterns don't matter. merging
everything to match actual behavior of the system makes using rust much
easier because we'll have only *one* library to link, not half a dozen.
Change-Id: Ie3cb1db659798511128716402497697320bd0484
starting pasta as soon as we have all namespaces it must be in available
lets it start up while we finish creating the sandbox. this may speed up
sandbox launches somewhat, but likely not enough to show up in practice.
Change-Id: Id6724cbdc48b99284cd7ba7b56c98829d74557c5
mostly as a prototype for other process launching optimizations, but
also to move linux-only bits out of runProgram2. we want to stop the
fork+exec dance we do now and migrate to posix_spawn, but linux will
always need an amount special handling for launching build children.
Change-Id: Ied55dc0de95c05b52aaca923be2e3c1d101f47e0
we'll use this to run programs from a vforked child instead of using
runProgram. ideally we would like to have runProgram be as efficient
as possible, but our mount namespace handling makes this harder than
is worth dealing with right now. the linux derivation goal is a good
place to prototype functionality like this, especially since we need
fine control over credentials and file descriptor tables of children
Change-Id: Ibc356613ae10b06ea81de9575611948f5ee353b6
this was racy previously; a builder may have spawned processes before we
added it to a cgroup. these processes would not be registered correctly.
Change-Id: If60a64a798ee806d5701590a83d1015f55d7b867
this halves sandbox launch overhead, reducing the build time for 3000
trivial runCommand derivations on our machine from 80 seconds to 44s.
as a nice side effect we also get better error message propagation in
some cases, most notably setgroups failing when run in lix sandboxes.
Change-Id: Ia7c50a844915d9b8a20475b90b2d0179fd2fff34
creating user namespaces early means we don't need pipes to signal that
a namespace has been created and set up. we'll also create the netns at
the same time because they're privilege-bound to the userns, and having
both available will let us launch pasta earlier to have it ready sooner
Change-Id: Ie66fe2ab7f50a520fb81dd90944384e168d94476
permissions and ownership will be the same since the sandbox process
will not have changed uid/gid or umask by the time it gets around to
writing these files. all of these files contain some information the
host must provide, be it contents or configuration info of some kind
Change-Id: I0b57cb0699fd29e798ccf330dff99f571f3c0692
this way we can use it anywhere without fear of it being uninitialized.
only the linux platform bit uses this anyway, so we will move it there.
Change-Id: I35e207eec91daa8aa327c4f8b36c0dc8e703a7c4
`xucred` already contain `cr_ngroups` and `cr_groups`, we just have to
use it properly.
Change-Id: Ic29b4c551f9d93e86e9ed2bef20889d27d5438f4
Signed-off-by: Raito Bezarius <raito@lix.systems>
Two things happens here:
- we consider supplementary groups that are known for authentication,
fixes#968.
- we check supplementary groups if they are our build users group and
throw an error if they are (new behavior).
Nonetheless, we did not remove the ability for `matchUser` to recurse
and check for groups that the user may have but the connection might not
have communicated as part of their groups, i.e. if a process reduces its
list of groups via a call to setgroups, it will still be authorized.
This will come in another commit.
The authorization NixOS test has been extended with a store ping test
running in systemd with DynamicUser=true *AND* a supplementary group in
allowed-users.
Co-authored-by: Tom Hubrecht <github@mail.hubrecht.ovh>
Change-Id: I25b2b8304d66a04651cea523b5585a5d15ceebe8
Signed-off-by: Raito Bezarius <raito@lix.systems>
On Linux, SO_PEERGROUPS is an option introduced in 2017, that is, in
Linux 4.13.
Thankfully, Lix targets 5.10+ kernels minimum.
It was chosen to allocate 128 gids by default and ramp up (2x) as needed
rather than allocate a full 65k of integers as it seems wasteful. I bet
the time to the 16 allocations should incur at most an additional
millisecond on a modern system, don't quote me on that though.
This is preparation to enable ability for the daemon to vet based on
supplementary groups.
Related to #968.
Suggested-by: alois31
Suggested-by: eldritch horrors
Change-Id: I26d698327db5d174bf70ca25b0afede132bd9169
Signed-off-by: Raito Bezarius <raito@lix.systems>
Rather than a bunch of booleans, use optional<T> as a way to convey that
the identifier is known or not.
This is preparatory work for SO_PEERGROUP usage.
Change-Id: I0e3ce69e4103f9df5a2161c695bfc650123c1549
Signed-off-by: Raito Bezarius <raito@lix.systems>
we can already handle this case (by forbidding any ~/... paths in option
values), and since builtin builders are now run via exec we may not have
a home dir set, or even available, for them to resolve. loading a config
file in this situation will fail with an error even if the config is ok.
fixes#1109
Change-Id: I3491b31558871faa34766462747e3c15d1eccc64
xattrs are revealing some unfortunate brittleness in real world
derivations that are getting -EINVAL errors while executing their test
code.
The reason for this is that Linux checks [1] UID delegations for xattrs
writes.
In the Lix sandbox, except if you enabled a uid-range feature, you have
exactly 3 UIDs: root, nixbld and nobody.
If your test code makes use of UIDs which have not been delegated, you
will receive an EINVAL on your operation. Test code is not resilient
with respect to the number of available UIDs in their namespace.
To avoid further issues for end users who are running into spurious
derivation build failures, we forbid xattrs again for now.
For more information about the plans, please consult or chime in [2].
Fixes#1105. Reopens#838. Fixes#1103.
[1]: https://elixir.bootlin.com/linux/v6.18.6/source/fs/posix_acl.c#L257
[2]: https://zulip.lix.systems/#narrow/channel/9-Store/topic/disablement.20of.20xattrs/with/5295
Change-Id: I864066b34cd8319d2271fac1b179cb4f950d836e
Signed-off-by: Raito Bezarius <raito@lix.systems>
this will let us configure more than one socket to connect/bind to,
which in turn lets us use posix acls on sockets for access control.
we will also need something like this for the final rpc transition.
Change-Id: I9c39f14906e9bf809055ab5c94bf687745b4f69e
now that builtin builders are regular executables we no longer need to
treat them specially during sandbox launch itself, only while we build
the command line and environment for the sandboxed process. we are not
far from being able to extract platform-dependent sandbox launch code,
ideally moving all of it into (much more replaceable) libexec helpers.
Change-Id: I9b7041314683c56cd70eec9b1b4eae6de228883f
this means that builtinFetchurl runs in a real process now, and thus we
no longer need its workarounds for running in a forked process. forking
dropped the signal handler thread and broke the curl state via sharing,
neither of which happens any more now. we can run fetchurl builtins and
their actions straight from the main thread of our builder now, and the
temporary files and settings overrides we did are now also unnecessary.
Change-Id: I738171bc120ffcd541b7ff1424fed7924c2cdc1d
this will let us migrate the fork+do_stuff combo of builtin builders we
have today to fork+exec of this new command. we use a subcommand rather
than a simple libexec helper because we would like to avoid linking all
of lix into the simple libexec helpers if possible. this is only hookup
for the builders, none of the buiders are migrated here to ease review.
Change-Id: I9358f1f3fee6ca640c81a7bd865128ae0d0e35a5
call them with their individual parameters instead. this will make it
much easier to turn them into executables we can exec in the sandbox.
Change-Id: Ib49cf44715b2a480fecadabd21216c69cd730ef5
not having the optionals set when the builder runs (e.g. because FOD
hashes were not configured correctly) could cause assertion failures
in the builder process. while this should usually not happen we *do*
want to print a useful error message here instead of a crash report.
Change-Id: I81fb1d45fcbf660d0d9a7f0c0a12c38379ecd4c6
See: cl/4840
When importing Python modules, we include `functional2` in the module
path, like this:
from functional2.testlib.fixtures.env import ManagedEnv
This means that python expects to see a file like
`functional2/testlib/fixtures/env.py`. We run `pytest` from `tests/` in
the `justfile` and have `tests/functional2/__init__.py` so `pytest` in
`meson` is able to find these imports.
However, language servers generally consider the `pyproject.toml` to be
the project root, so (e.g.) `pyright` is unable to follow any of the
`functional2` imports, leading to lots of spurious errors.
In cl/4840 I moved `tests/functional2/pyproject.toml` to
`tests/pyproject.toml`, which worked but was considered aesthetically
unappealing.
This diff is much larger but it's a more elegant solution.
Change-Id: I2983c7b87f88f59a4e3521451a9f5acd6a6a6964
like diff hooks before the build hooks also use runProgram features that
aren't present in posix_spawn. just like diff hooks we do not expect the
build hook processes to be so fast that launch overhead matters somehow.
Change-Id: If3c33dbd7d2ac20c95886e06d24feda312946c78
use a wrapper type instead. whether something should be treated as a
single process or a group is a type decision more than a runtime one
Change-Id: I966e998c19e74be64ddd72ab11af809d001f61b5
diff hooks use uid/gid switch functionality that is otherwise only
needed for linux sandbox setup and unsupported by posix_spawn. not
doing these switches inside lix core code may let us move to using
posix_spawn for most process launching in the future, and for diff
hooks the added overhead of a wrapper program really does not hurt
at all. diff hooks are expected to be expensive in terms of output
size, process launch overhead is not likely to even be noticeable.
Change-Id: Ifa4b3eedef237632db3eb88d10e6469acae01f9e
it's kind of weird to not allow diff-hooks in single user mode unless
that user is root. maybe that's why we have no tests for them? we can
at least add a test that diff hooks are run at all when we expect it.
Change-Id: I54d623b5416acda1c205cc062b0f3a33c9f4aaa7
nothing except the linux sandbox actually *needs* this for correctness.
the linux sandbox only uses it to ensure that builder uids are freed up
when the sandbox is torn down, and even *then* it only works as we need
it to with PID namespaces enabled since the parent death signals is not
inherited across fork, but when pid1 of a pid namespace dies the kernel
also kills all namespace members. in all other cases this flag does not
help us that much because it actively prevents child processes cleaning
up after themselves, possibly leaving more trash around than otherwise.
Change-Id: I9ae0a9e91833a91d1011375dda402ac6c0a7ef6f
Adds an example plugin implementing an mTLS-enabled binary cache store
(https+mtls:// scheme) using client certificates for authentication.
Darwin fix: don't link liblix* into plugins (host resolves symbols at
runtime via dynamic_lookup). Explicitly link curl so it binds to
Nix-store libcurl, not /usr/lib/libcurl. This prevents the plugin's
curl_easy_setopt calls from operating on the wrong libcurl instance.
Test portability: BSD sed -i wrapper, OpenSSL -sha256 for cert signing,
redirect test server output to log file.
Change-Id: I652b987d3ac45e31df50ff4ba1f523294438c2b6
this really doesn't have to be here, it doesn't help very much. doing it
in the parent is cheap enough to not care and sandbox setup is not async
yet *anyway*, so we would not even notice if the old way was any faster.
Change-Id: I5a3a99af0fa5928e9a42f9c6589d98ff38b8c775
this will make it easier to turn build sandbox processes into helpers.
the point they were at was effectivly unsandboxed except for a few fds
that were redirected by commonExecveingChildInit, which only made logs
of any errors that much harder to convey from the child to the parent.
Change-Id: I67006eb33e1e13311bb8d14e6a0c3d5e6baf0c13
it doesn't do anything anywhere else, and personality setup is not a
critical operation (all it does is change which arch uname returns).
Change-Id: I39d7fcc4916e6882e49191d2294f9b7ee0dbbcd0
there's no need to do it in the fork, we're not changing security
domains before opening the pty anyway. we do change who owns this
pty device, but since we change it via chown we can just *not* do
that before we have an open fd to the pty. in practice this isn't
even necessary because the daemon runs as root, but if we ever do
run the daemon as an unprivileged user we'd need this to be split
Change-Id: I35264ab2954c7ba2c9c24c927366d64acada6772
this also requires moving namespace support information into Worker and
out of function-scope static variables, otherwise we can't use async IO
for the libexec helper output. we could set the fd to blocking for just
one CL and extract the Worker changes into another that the reverts the
blocking fd usage, but that seems not warranted for the scope of these.
Change-Id: I6996fab1ae74693d50cefb6a6a9c21d61dada1d9
Those functions have been recently refactored to return a formatted
`string` or take an `std::ostream` and only do the formatting have been
renamed to match what they're actually doing.
Change-Id: I3fe32fbe8723c2d93226370b8dd297f16a6a6964
this is mostly a test and example for the libexec helper infrastructure,
but it also lets us simplify pager launching until we we can more easily
handle executable-not-found errors the launch fallbacks would cause when
using runProgram2 instead of fork. ideally we'd use `posix_spawn` later.
fixes#1104
Change-Id: Ia33cc12e8a9d60ffad6f5c055bb1b8b596810e64
the new libexec directory is not available as a setting like other
directories (e.g. binDir) are since we consider libexec helpers to
be very internal. repointing them is like repointing a .so file we
dynamically link to; it can work, but needs much more preparation.
Change-Id: I40e64be0b32276f2864c0f2eb0b998d4c8ce7c88
we don't need the full sandbox setup helpers for this: mount namespaces
do not need to be kept, loggers are not needed, and redirections can be
done by runProgram2. once the build hook is removed we will not run the
communication bits from a different process anyway, this prepares that.
Change-Id: I95d28f7c2c25e43ccd82b448d270403ce4f28852
we do not need explict redirection management now, and dieWithParent
doesn't need an override either. we'd much prefer to kill ssh if the
process in charge exits; even ssh multiplexers are not fazed by this
Change-Id: I81e28b7605df73c887878ea4716228c7ad0f5c6f
the redirections the tests did were once not possible with runProgram2,
but they have been for a while now. we should use them instead of fork.
Change-Id: Ia422c8941ed04a9403dee68bfe938d68952253da
returning the pid as a pid_t instead of as a Pid raii wrapper is only
convenient for daemon use (where it'll eventually go away). using the
released pid correctly in other places is harder without the wrapper.
Change-Id: Ib42a2f357d2f0849beabd015b321bfff31334eda
Fixes#323
Let's now all go an a little rant about spaghetti code...
The result of this code is that the missing paths are not printed
anymore. The basic issue was that the parent class of this command is
StorePathsCommand, which inherits from BuiltPathsCommand, and their
purpose is to work on path that are clearly in the store, building them
if needed (and ofc telling the user about what's missing), the sequance
of calls is:
- BuiltPathsCommand::run(ref<Store> store, Installables && installables)
- Installable::toBuiltPaths( *getEvaluator()->begin(aio()),
getEvalStore(), store, realiseMode, operateOn, installables);
where operateOn is Output by default, realiseMode is Derivation, so
the only thing that can be built are the derivations for the required
installables
- Installable::build(state, evalStore, store, mode, installables)
- Installable::build2(state, evalStore, store, mode, installables,
bMode)
And that final call has the following:
```
switch (mode) {
case Realise::Nothing:
case Realise::Derivation:
state.aio.blockOn(printMissing(store, pathsToBuild, lvlError));
```
So there were two options, hack a new spaghetti in the existing
spaghetti code, or condense all those calls that are actually useless in
our case because they mostly transform a list of installables into a map
from installables to their BuiltPath which are then iterated to retrieve
the final outputs, whereas it is possible to directly get the required
paths in a much more efficient manner and without printing unrequired
stuff through a multitude of intertwined function calls by simply
replacing one method that was previously inherited from the grandparent
class
Change-Id: I1d2baaef5a099cd98b63b5346f2613914c6cd2ac
it's no longer needed now that withPager exists. also fix a
logger-never-resumed bug that never showed up in the world.
Change-Id: I7311a50896f5291364320ec4c88506dc3bb11d4c
RunPager is weird and confusing in that it replaces what stdout *is*
depending on environmental conditions. this has not caused problems,
but it's easy to imagine situations in which it would (eg if the stl
decided to capture the stdout fd by duplicating it). using a wrapper
for this also makes clear *what* actually goes into the pager; while
the previous contract was semi-reasonable it was also very implicit,
and with the proliferation of functions we had that printed directly
to stdout it would have been easy to send wrong output to the pager.
using a wrapper also makes process management much easier because we
do not have to rely on destructors to always produce correct output.
Change-Id: Ifd3760940af1ec719fe856158c913cbb9a5bf270
this has been unused since ab3ce1cc13,
which is now almost ten years ago. surely it was just forgotten then
Change-Id: Ib93b52d4ce5f19629432c5f91ff426582c57f99b
cancelling a get() promise could leave the pool with allegedly in use
resource handles that do not actually exist. this would cause asserts
in ~Pool to fire (and crash the entire program due to bad accounting)
probably fixes#1041
Change-Id: Ic61ab9b658f02f71e8be7577b8a1115600846f86
merge all pkgconfig-found libraries into a single one and redirect all
existing lib files to the merged variant. the clean separation between
libraries is largely fictional anyway because the main config block is
in libstore and cannot reasonably be moved out, or because libutil has
assumptions built into its behavior that libmain later establishes, or
because libexpr messes with global boehmgc configuration. nearly every
kind of layering violation imaginable can be found *some*where in lix,
and not pretending that this is not the case will let us clean up some
of them. this will also allow us to merge these libraries at the meson
level which is itself necessary to make rust code easier to integrate.
Change-Id: I172f8a08d5d90ded0a0a99fd9e0fa5d804db00a7
xattrs scrubbing was gated behind Linux platforms, but Lix can safely
assume that POSIX APIs are available.
macOS implements these APIs modulo some creative ideas.
Fixes#1008.
Change-Id: I1566df4923bf24092d3fd32ccf6a96e4256e87b2
Signed-off-by: Raito Bezarius <raito@lix.systems>
Fixes#838.
xattrs were historically disabled in 2017, since then, NFS v4.2 gained
support for xattrs and almost all filesystems support xattrs. In
addition, moving the build directory to a xattrs-supporting filesystem
is always an option. Programs may exploit xattrs (including ACLs) during
their build process and testing phase, to better serve these programs,
we will remove this limitation.
Lix will use POSIX ACLs for its UNIX domain sockets in the future and be
one of these programs which will run tests making use of xattrs, while
not writing any xattrs in the derivation outputs themselves.
xattrs are still scrubbed from derivation outputs so it is not possible
to obtain an advantage by writing a security-related xattr to a well
chosen file.
Tests were added to test these scenarios on Linux. Darwin is carefully
excluded because of #1008 and #1090, solving that is welcome. In the
meantime, they are marked xfail.
Change-Id: Ia3255eeb8442e83db4f10dcb5a51cbc368a2550d
Signed-off-by: Raito Bezarius <raito@lix.systems>
one would expect the lang tests to be selectable (using `-k`) using
`eval_okay` `eval_fail` etc though this was not the case so far.
This commit renames the functions to reflect the expected names
Change-Id: I4cd340c093d774feeb4d96820d28e49c665b159f
I don't know how this was missed or no one really complained about it,
but perl-bindings was using CppNix if nix was not overlaid to our own
Lix.
If you overlay Lix manually, the resulting Lix's perl-bindings cannot
really refer to Lix itself, therefore, it's CppNix.
Change-Id: Ie5ce95564cb7fb5444acbea6124619c5121e1576
Signed-off-by: Raito Bezarius <raito@lix.systems>
during the value rewrite we accidentally broke extension of incomplete
primop application. this only shows up when binding on incomplete call
to a primop to a name, binding an incomplete call to *that* to another
name, and then finally calling the second binding with enough args for
a complete primop application. since this only shows up when calling a
primop with three or more args it took a while to surface. we have few
builtins that match this: foldl', replaceStrings, and substring. these
are not used incompletely in this manner very often, so this lingered.
fixes#1102
Change-Id: I218dffc14ae876efc86a86c7eb6c895e2405201c
these ended up referring to different directories and would break during
source reorganization (or even just including them in different places).
Change-Id: I91e8db04e3d3565efd5920982ed5bc59d2c27058
RegisterCommand was renamed in b30556f8f1
and de-static-ified in 4d9fe67430. adding
new settings was never possible with static instances since settings do
need an existing Config instance during construction; GlobalConfig must
be used to add settings. it is not possible to change existing configs.
Change-Id: I26af8abc375ba5883fce060ffefc710d6dfbca4c
Since lix-doc is no longer a separate library, the pkg-config files were broken
for static builds. Adjust them to refer to the new monocrate library instead.
Actually, since the monocrate library is static in all cases it needs to be
linked explicitly even when everything else is linked dynamically. I assume
this only doesn't break the dynamic nix-eval-jobs because it doesn't actually
use anything from it.
Change-Id: Icf29938688f4fa87425ff346a3105ef56a6a6964
Use a span representing the underlying hash, this allows comparing based
on those spans. Other minor tweaks to the header file.
Change-Id: Ie05960439e60500109083c06d347d6cdf41bfda8
When --check is ran with no output, this is seen as having *SOME*
invalid outputs, actually, ALL of them are invalid here.
Instead of reporting a confusing error, let's consider ALL of outputs
being invalid to be no output at all available and advise the user to
rerun the build without --check.
If there's *some* (>0) invalid outputs, let's print them exactly to give
a chance to the user to remediate this.
Fixes#485.
Change-Id: I00955ef9ea4f129e2c98d68c73b1e981f90278a0
Signed-off-by: Raito Bezarius <raito@lix.systems>
Yeah uh, I don't think `$shell` is ever set in the functional1
environment. Pretty sure this has been expanding to `''` THIS WHOLE
TIME.
Also `SHELL` is `/sbin/nologin` in that builder. idk why so I'm just
passing `shell` unconditionally in functional's `mkDerivation` helper.
Maybe next we should put `set -euo pipefail` into all these test
builders?
Change-Id: Ie5fb85305932c03a901ba84ffcce48f16a6a6964
macos-builder02 in particular has been having problems for a while now
that no one could reproduce. We believe we have finally found the cause:
missing shebangs, not just in the tests themselves but *also* in the
inline scripts `nix-shell` itself writes.
Fixes#1042.
I believe this will also fix#1093.
See also: https://github.com/NixOS/nix/pull/14778
Change-Id: I54d04a770b8e78484815db88a8bc88776a6a6964
When `:load`-ing a file that doesn't result in a successful outcome, the
"loaded" file is removed again from the list of loaded files.
This avoids having it be used as part of `:reload` which will probably
not result in a positive outcome.
Instead, the user is expected to retry `:load` before issuing a
`:reload`.
Fixes#50.
Change-Id: I85d025cf39f79e8e02117628bffa20c81917dc3f
Signed-off-by: Raito Bezarius <raito@lix.systems>
If `:reload` is used with no file loaded, the environment is completely
cleared.
That's a surprising behavior and one of the root cause behind #50.
Environment clear is only performed if there's files to reload.
Change-Id: Icb014f490a83aaa4d758dbf37822db16d5bb65c1
Signed-off-by: Raito Bezarius <raito@lix.systems>
Only successful loads counts towards the list of loaded flake
references.
Change-Id: I04128f90e57ef8352183d72dc6def7a09d52173a
Signed-off-by: Raito Bezarius <raito@lix.systems>
libutil config.{hh,cc} and libstore/globals.{hh,cc} contain so many
interrelated things. Hopefully we can separate them out a little bit for
clarity.
Change-Id: Ib78fc46fe305d13aada8886e2e168d446a6a6964
This makes that the default for all builds... which I'm not sure is the
right option, but it is also very cheap to do so, and adding an entire
other Lix build just for static libraries (not to be confused with
static Lix) would be prohibitively expensive.
Change-Id: Id173c77cfa62886724378ca599d21c8c6a6a6964
`build_env` is assumed to be a certain trivial environment, e.g. a
singleton search path.
An environment in general is either empty, contain one or more search
paths separated by a colon.
It seems that the intent was to set the `build_shell` first as a
prepended path in the managed environment then extend via the
`build_env` parts.
This fixes a usecase when the `BUILD_TEST_ENV` is non-trivial.
Change-Id: If5b8ab976d867a57ce0b8d29255f64695e30a8b2
Signed-off-by: Raito Bezarius <raito@lix.systems>
In an attempt to slowly split up and clean up `eval.cc`
The `Expr::eval` functions contain the most important logic when needing
to touch something on the evaluator, and having them spread across the
3kloc eval.cc file was really annoying. This should help, though more
cleanups are yet to come
Change-Id: I2c2d15a024ab1ec3fc5591f628d6768341a1c8ab
This adds a way to control the deprecated features that need to be
enable while evaluating the specific nixpkgs version
Change-Id: Ia2b9a29f1966d4d5f32239a36ba8270330007da9
This avoids rebuilding Lix each time we need to modify the lib tests
(as flake.nix stays unchanged), and adds a way to control the deprecated
features that need to be enable while evaluating the nixpkgs lib
testsuite
Change-Id: I2baa1cf90effcaf9d53337c6a7a5b24d6297178c
Certain evaluation requires to pass impurity arguments, add `flags`
to add more arguments to the instantiation.
Change-Id: I538ca2ff12c07cc66e845b42f81baa87f11b8f0f
Signed-off-by: Raito Bezarius <raito@lix.systems>
Is there a situation where `home` can be None? This hinders the ability
to use it quickly without type gating in the test code itself.
Change-Id: Idd739ddad67e10d6f7a6aad283c0e3b7cd08106a
Signed-off-by: Raito Bezarius <raito@lix.systems>
Otherwise, this result in debug traces with a \n cutting the list of
controllers.
Change-Id: I8aba6379d335c4edab9d9ddb3fb411813d8dc426
Signed-off-by: Raito Bezarius <raito@lix.systems>
The prior use of the UID is uninteresting in general for consumers who
wants to learn things about _what_ the build is, not what is the UID of
the build user running it.
Achieving this with a place where a mapping between cgroup names and
build requests are maintained is non trivial because there's no "global"
daemon that keeps track of this information and such a daemon would
require a persistent location to track this.
Rather than solving that, we will just encode the derivation hash inside
the cgroup name. Userspace can simply pull this apart and query Nix to
obtain the original corresponding derivation for more information.
This provides userspace with a way to map cgroup names to pnames for
example. Going further, if you have a map between attribute paths and
derivation hashes, you can power a database of cgroup metrics per
attribute paths by combining all of that.
In addition to this rename, we adopt a systemd-ish convention for
template units (`@`).
As a result, userspace tooling that will scrape cgroup metrics will
discover a consistent path as long as they filter out the UID part,
especially in context of Id6c458aad30eaa08c3609ac8280a7dde8e8f3cf9
change.
This problem is tracked under #1073.
Change-Id: I238d0568a3e4b1ff3057781c0639528d666b4d37
Signed-off-by: Raito Bezarius <raito@lix.systems>
nlohmann has customization points for exception throws. we can use these
instead and wrap json exceptions at the source instead of playing a game
of whack-a-mole with json errors all over the tree. since nlohmann needs
macros set to achieve this we can no longer precompile its headers *and*
must forbid including it anywhere without the proper defines or ordering
of include directives will break lix with ODR violations, if we see them
at all before we get another json-related bug report. a new lint will be
in charge of ensuring this doesn't happen. we also re-allow direct calls
to nlohmann json parsing since error handling is no longer a problem. we
will keep the wrapper for more convenient error context handling though.
fixes#1092
Change-Id: I54ecc14f5bec5e2177729b41c3703216e76cc6a3
It's 2026 and the command handler was entirely manual!
We introduce a basic command handler framework for the REPL that handles
all the previous (implicit) features reasonably well.
This also builds up the infrastructure for nicer features.
Most important feature is that now we can let external plugins adds
commands to the REPL.
There's slight REPL changes:
- errors for debugger commands are more explicit.
- short-hand have a long-hand version.
- help shows command sorted by std::map natural key sorting (based on aliases)
while keeping the special syntaxes for the REPL language at the top.
- sections were introduced for Flakes commands.
- :st was merged into one command with an optional argument marker.
Change-Id: Ibd24f994491bf5d45d4b218fd84c0944ae5050fd
Signed-off-by: Raito Bezarius <raito@lix.systems>
Status quo: We have `force$Type` and `eval$Type`, both which first
produce a value and then do a type check. The type checking logic is not
consistently implemented, with lots of code duplication.
This change does:
- Introduce new `check*` functions which unify the logic (the error
handling unfortunately still needs some duplication for now)
- Make both `force*` and `eval*` use the `check*` function for the
actual type checking
- Inline and dismantle the `eval*` functions for being of little use and
little used. This makes the `ExprOp*::eval` implementations for binary
logic operators more verbose, but IMO that's a good thing: The
implementation now needs to be a lot more explicit about the
short-circuiting semantics, something which was previously hidden behind
the short-circuiting semantics of the C++ language, in a way that could
easily be overlooked and lead to confusion, which is something that
happened to me twice in a year.
- Changes `forceAttrs` and `forceList` to include the context in case
`forceValue` fails (compared to only when the type check fails). This
was done for code consistency, because I could not find any reason why
list and attrs had different semantics here than int, float and bool. So
far the visible change is minimal (see the diff on the err.exp), however
this needs vetting for potential performance regressions.
Change-Id: I33e5c706d46850c9e1126293ee01dab85ba07587
That struct has been annoying me since I've encountered it. There is no
reason for having to track all these symbols out of band in the state.
Change-Id: I5c2c0d8174af0a51f9b456cc3651d8203a406d09
The names were prefixed with `sym_` to allow faithfully representing
names that start with __. (Previously the underscores were omitted, at
the cost of readability for people less familiar with the intricate
details of Nix. Is `toString` literally `toString` or is it
`__toString`? etc.) As a bonus, this allows to easily grep for any
symbol through the entire code base to find where it is used. I've done
exactly that and grouped and annotated the symbols based on their usage.
Change-Id: I230fecd9b0b18e0984ac8111c1e6381f45c5abf2
I was pretty confused by this when I saw it, not realizing that `build`
meant `tests/functional2/build` and not the top-level Meson `build`
directory.
Change-Id: I4865f196e5bf029419c6b488e78cbfb46a6a6964
100 causes too much diff churn w.r.t. current code style
"120 cols would be nuisance for us because our two-column layout is
exactly 118 wide (not counting the line number margin)"
"Ours is exactly 118 too"
Change-Id: Ifc7f21003e1011a17a1d799afaee775aeb927fd5
This logic is used in the various build-related REPL commands and is
factored out to make it easier to write the handlers for each
build-related REPL commands.
Change-Id: Iaa18df489db75495b12924e9a76b3fff1975eb64
Signed-off-by: Raito Bezarius <raito@lix.systems>
The completions for debug commands is purposefully left out because it
is going to go away in the next changes.
Change-Id: Ibc78b7587de3f15857bb09a3a0732334693ad550
Signed-off-by: Raito Bezarius <raito@lix.systems>
cl/4796 introduced a bug where running `just test-functional2` would
hang forever on macOS but `meson test -C build --suite installcheck
functional2` would succeed because of a mismatch between the
`$BUILD_TEST_SHELL` environment variable and the `-Dbuild-test-shell`
build option set in `mesonFlags`.
We can move these definitions to `finalAttrs.buildTestShell` and similar
and use those values in _both_ `mesonFlags` and `env` in order to make
them consistent in all cases.
See: https://gerrit.lix.systems/c/lix/+/4796/comments/50def5b4_fa5671f2
Change-Id: If300bbe46d6269ace29b44156fb4a5196a6a6964
...and add both a static bash and busybox into the sandbox for building:
$ /nix/store/fz43jxs5qfg5vldzk38y2hmrcl58qk18-busybox-1.36.1/bin/sh -c 'declare -A foo'
/nix/store/fz43jxs5qfg5vldzk38y2hmrcl58qk18-busybox-1.36.1/bin/sh: declare: not found
The reason is that busybox's `sh` doesn't know about `declare` which
means it cannot build derivations with `__structuredAttrs = true;`.
Change-Id: Ie3ca431f862fd3e59dc649582704ae739ee5834a
Nearly each tests gets a fresh store (in the test root residing under TMPDIR),
which with default settings means 8 MiB of reserved space to be deleted by the
user in case of space shortages, which of course for a test suite is just
wasted. Stop doing that to considerably reduce the litter (experimentally, from
3.7 GiB to 67 MiB per run). The old functional test suite already did the same.
Change-Id: I912cdda6e796ea37cfee0b8fe0478e976a6a6964
this is only needed on macos, and only for tests that build anything. on
linux it can actually *break* stuff due to unfortunate interactions with
store url parsing, which is required to work for any remote build tests.
Change-Id: Ic6f7e090f15e129fc365d7edc56cdbc1a5686047
apparently this file slipped through the code review (the actual test
does exist, no worries) but this empty file and init py exist, sitting
here empty. This commit drops them as they serve no purpose
see commit 1b7ad3a7ad
Change-Id: I64c4e03cd3ceedf9e430ae014127966bd8e4269d
This avoids any complications that can arise from the environment
affecting evaluation of the help pages (which don't need to be calling
out to anything external anyways)
I don't think this is actively causing problems now - though it did in
Nix! - so this is largely preventative, but I don't really see a reason
why this was ever running in impure mode in the first place either
Based-on: https://github.com/NixOS/nix/pull/14086
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
Change-Id: Ice0c760562d70c09934249edc3b767473516d372
- test_check used an old version of an error message
- test_good2 did not build an unspecified dependency
- test_attribute_selection used a fixed drv path. we
can't keep this with the current setup because the
drv `system` makes the drv hash platform-dependent
fixes#1081
Co-Authored-By: kloenk <me@kloenk.dev>
Change-Id: Ia009740f8b5432e83467ea451a5ff53d5e141b3f
the cgroups experimental feature does not work properly without this
because we do not stop subdaemons when the main daemon is shut down.
systemd needs the assigned cgroups to be empty to restart the daemon
and thus cannot cleanly restart the daemon if any connections exist.
starting a fresh unit for each connection creates a new cgroup every
time instead of sharing any delegations and thus solves the problem.
fixes#1030
Change-Id: Id6c458aad30eaa08c3609ac8280a7dde8e8f3cf9
A friend reported that this is rebuilding, which doesn't make any sense
because nixpkgs also has a lix from nightly that surely has the same
dependency and thus it surely should be built on hydra, right?
Turns out no, since they were overriding the requiredSystemFeatures to
remove big-parallel. Silly, but we can match that and get cache hits.
Change-Id: I4b1c8e850c6c38b03881354e978f41a10592ec95
This bump nixpkgs input to 25.11 and fix the two warnings that were
coming with the update:
1. `pkgs.system` => `pkgs.stdenv.hostPlatform.system`
2. `runCommandNoCC` => `runCommand`
also:
1. remove cppnix 2.3 compat tests (cppnix 2.3 is officially dead now)
2. remove lowdown 1.3 compat tests (nixpkgs no longer carries it)
Co-authored-by: Tom Hubrecht <github@mail.hubrecht.ovh>
Change-Id: I78526b5d8992a6c63ecd7f0c7c1fa6346a6a6964
It only debatably belongs in libutil since libutil doesn't know what a
progress bar is but whatever.
It was either that or libstore. Since I'm about to make it a setting.
Change-Id: Iba59b0a190087b316ff3191a0ddd9ec36a6a6964
make the global_assets folder more readable by placing asset pack files
within a dedicated folder instead of building up a mess similar to f1
Change-Id: Ia2c16f38eb6da96e1e73584bd91391ee56acb410
This will probably get the implementation of the fixtures revised when
we land the new extraction code, but we are setting it up to be generic
against that.
The operator-> thing is kind of a crime. But it also makes the code
vastly more readable so it's impossible to say if it's bad or not.
Change-Id: Ia5aca69cefaa03cd533ad19d20d856ff7e76a546
Instead of manually reading the ref file,
which gives the tag object hash when ref is a tag,
shell out to git.
The logic for finding the ref file is left for cache TTL tracking.
Fixes: #1070
Change-Id: I490b1e62f83cf602c56232c6081a52166a6a6964
Those are new warnings that come with a more up-to-date llvm
Fixes#1066
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I305d0d810d12a5e8d31c7d89e0cdb3a82df71556
As far as I can tell, there's no harm to let know a nix-shell invocation
about NIX_LOG_FD being stderr.
Fixes#336.
Change-Id: Ifdb3591813251d4bc481158d28dc18e1489de72c
Signed-off-by: Raito Bezarius <raito@lix.systems>
As far as I can tell, there's no harm to let stdenv know that they can
write to file descriptor 2 which is stderr inside of a nix3-develop
shell.
Contributes towards #336.
Change-Id: I51dcbcbc19a1698a0d1255f9d943d9ebd15d115d
Signed-off-by: Raito Bezarius <raito@lix.systems>
Closes#1064
The culprit here is that `genGraphString` is only invoked with the
store-paths associated with the outputs of the derivation, so when
filling `dependents`, the `graph_data.find(p)` call would return the end
of the iterator when doing this for references to other store-paths.
As a result, the code wrote information behind the graph data-structure
causing a corruption. For me, this resulted in a SIGSEGV most of the
time and in a few cases in an uncaught `map::at`-exception as reported
by Niklas.
This patch changes two aspects of the original implementation:
* When filling `dependents` in the node-set, use `map.at()` instead of
`map.find()->second`. The latter doesn't make any sense and was the
cause of corrupting memory. The `at` would've made it far easier to
spot this in the first place.
* Filter out store-paths that don't belong to a different output of the
derivation when creating `outputGraph`. This variable is used on two
places, `genGraphString` and for topological sorting.
The latter already filters out store-paths from a different drv, so
this is happening now when creating the variable in the first place
such that `genGraphString` never ends up with corrupt data in the
first place. This is the actual bugfix.
Implemented a regression-test for this case to be sure.
Change-Id: Ie02144d89c32b0a776cb1ece0601d0229315ebc3
we'll need this to modify argv for socket-activated daemons. this is our
replacement for the old savedArgv mechanism that was unscoped and fucky.
Change-Id: Ie048eb8ea99f1c9cd627a051292c836c83197068
the parent daemon does not change any settings before starting a child,
so there's nothing we may want to change that is not already set by the
config file. this also doesn't prevent changes of the config file being
applied to daemons where we do not expect it since it'll only restore a
setting to the parents' value if the child also has an override for it.
Change-Id: Ic5a9ef13458c103ec9979cb187ba8d3ce5e1e719
Lix changed to C++23 in 4ea8c9d61¹, but the perl subproject wasn't
updated with it. This commit corrects that.
[1]: 4ea8c9d643
Change-Id: I5464138d206d87cf25762720b147487d6a6a6964
We offer the three usual options: verify, repair then delete and direct
people to report us corruption so we can fix if this is induced by us.
Fixes#447.
Change-Id: I0df61769d732d227333c206f312857c0593f7bce
Signed-off-by: Raito Bezarius <raito@lix.systems>
If the derivation does not start with D, do not return a simple
"expected string 'D'" error but a full error message.
This contributes towards #447.
Change-Id: Iee05f3918e4cc43e79f244ab2fd64a52cf2bb6d2
Signed-off-by: Raito Bezarius <raito@lix.systems>
Prior to I6a6a6964d2b5ad47ae5ea9eb11af9b6373ce2141 — `sudo nix
upgrade-nix` would perform direct store access.
This ensured a certain number of desireable properties for upgrading the
Lix binary itself.
We re-introduce direct store access for upgrading Lix binaries.
Fixes#1060.
Change-Id: I523c4d3023ed5fe9eff8fde9a266c56a0de47d8c
Signed-off-by: Raito Bezarius <raito@lix.systems>
Alternative to cl/4661 discussed in #1044.
It can be assumed that `$tmpdir/build-top` can be created safely without
any risk, this way, we don't need to reuse the random directory creation
primitive.
Fixes#1044.
Change-Id: Iec52477f3047fc40959b183c607312d5a40fc8c9
Signed-off-by: Raito Bezarius <raito@lix.systems>
Otherwise, loading Perl bindings fails early with[1]
undefined symbol: Perl_pad_sv at /nix/store/h2jsb5i4yfblr2f3ac2c7zpmlmj7zjym-perl-5.40.0/lib/perl5/5.40.0/XSLoader.pm line 94
Apparently, it's expected behavior by Perl that this symbol only exists
with `DEBUGGING` being set, hence it's used by the headers. However,
`pkgs.perl` from nixpkgs is apparently not built with `-DDEBUGGING`
causing this error.
Now, `NDEBUG` is manually unset after loading the Perl
headers rather than setting `DEBUGGING` causing the error mentioned
above.
I confirmed that this not only fixes the problem described above, but
running the Hydra tests with
diff --git a/perl/lib/Nix/Store.xs b/perl/lib/Nix/Store.xs
index dfdd64d28..14788266c 100644
--- a/perl/lib/Nix/Store.xs
+++ b/perl/lib/Nix/Store.xs
@@ -27,6 +27,7 @@ using namespace nix;
static AsyncIoRoot & aio()
{
+assert(false);
static thread_local AsyncIoRoot root;
return root;
}
still results in assertion errors.
Finally, added a small install-check that importing `Nix::Store` works
fine.
[1] https://git.lix.systems/lix-project/hydra/issues/69
Change-Id: I58521777eb0f94b766a9813aa4bbd06f9052bd35
As discussed in the F2 matrix channel, not being able to put the error
message inside of the thrown exception just results in one assigning an
additional variable to immediatly throw it. The original reason for
these rules to exist are that the message isn't printed twice, though
the line above the thrown exception is printed anyways in the
stacktrace, so we still get the error message twice but now with
additional work.
Hence this commit removes the said rules
Change-Id: I0b37c0b0861334703a5772c36353d31441f19dcd
So far, our ruff config was confined to the f2 package. This meant, that
when one added additional paths to the ruff formatter, those wouldn't
get the same rules applied as f2, resulting in inconsistent styling
thoughout the project.
Due to how configs are resolved, only the "closesed" pyproject toml is
considered. This means, we need to tell f2 to extend its configuration
with the base level one. Though no change is required for other parts of
the project, as long as they don't have their own pyproject.toml
Change-Id: I145c764e7b850194020b5560e1025f4aa80411ae
Nix daemon runs in a service context, usually with systemd.
If the Nix daemon unit adds `[Service] CacheDirectory=nix-daemon`, it should
successfully use `/var/cache/nix-daemon` automatically.
Instead, it uses `/root/.cache` right now, which is really bad.
By default, we add `[Service] CacheDirectory=nix` now which means that
caches are moved into `/var/cache/nix`.
Fixes#634.
Change-Id: I854b1045bfdce8f60110aea70bda1bf6657dfd7b
Signed-off-by: Raito Bezarius <raito@lix.systems>
Recently, in 54df89f601, support for mdbook 0.5
was introduced, including some logic to handle the `sections` -> `items`
rename. However, compatibility with 0.4's `sections` was only kept on the read
path, while writing 0.5's `items` unconditionally, which ends up in the bit
bucket on 0.4, effectively disabling substitution fully and leaving the include
directives in the final documentation. Restore writing into the `sections` when
they were there so that substitution works again.
Change-Id: Idd4d7653012660f3f7fc27f81f29b82d6a6a6964
Silly me forgot to add the dependency responsible for handling
`@pytest.mark.timeout` failsafes to kill a test if it takes too long,
which results in a warning when running f2.
This commit adds the required dependency and hence removes the warning
Change-Id: I281cad05bb0cf50208f72080cb2bfd8cbfb09d0c
Running the sandbox (as unprivileged user) requires all of user, mount and PID
namespacing. Previously, only the user namespacing support was checked.
Unfortunately, newer Ubuntu kernels impose restrictions on capabilities within
unprivileged user namespaces [1]; as observed by experimentation, in particular
(re-)mounting filesystems or changing mount propagation is prohibited even
inside a mount namespace, making it impossible to use the Lix sandbox. So any
tests requiring sandboxing should check for this breakage and be skipped if
detected.
`unshare --mount` performs the same problematic operations, so that's what gets
used for the test. PID namespaces are included as well for completeness, even
though currently no instance of breakage due to them specifically is known.
This fixes [2] for the functional test suite only.
[1] https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces
[2] https://git.lix.systems/lix-project/lix/issues/545
Change-Id: If95b527bf965ccb06371bbef7b6007666a6a6964
Builds using a diverted store strictly require sandboxing. Explicit disabling
by the user is already handled properly by force-enabling the sandbox anyway
(with a warning), but sandbox-fallback was still honored, predictably leading
to hilarious breakage (like the "confusing output" in [1]) when it actually
triggered. Cause a hard failure instead.
[1] https://git.lix.systems/lix-project/lix/issues/545
Change-Id: I61ea32c3bd8bca4e5ba278312b0a49326a6a6964
Builds using a diverted store strictly require sandboxing. It therefore makes
sense to automatically enable it even if otherwise explicitly disabled by the
user. However, they should still be informed that their choice is not being
respected.
Change-Id: Ia2cbdc24a4c55d8ecc49094e35e698e16a6a6964
Disabling build sandboxing is a significant change to build isolation. If this
is done automatically due to namespacing issues (and sandbox-fallback not being
disabled), show the user a loud warning without the need to opt into a lot of
other log spam.
Change-Id: I52daccdd6934873234223c798ca3dcff6a6a6964
This contains the depth of nix shells nesting.
Fixes#826
Co-authored-by: Qyriad <qyriad@qyriad.me>
Change-Id: If584c9d02730d6c857636dafdeab0c01f4ec8e0f
If the NIX_PATH shadows internal files, this will often break things,
hence we forbid it.
Fixes#998
Change-Id: I70e5d389532ada1c9f910c60281abe565e3ce6bb
Some of the tests depended on the evaluation order chosen internally by
nix-eval-jobs, which will break when that order changes. In practice the order
is not guaranteed, and already not deterministic with multiple workers.
Change-Id: I2e85d096f4439a8075a6f4a08d31c0f16a6a6964
GCC still ICEs on coroutines, while Clang works properly. Use the latter, just
like for Lix itself.
Change-Id: I802d6df7279550a5f5355e6c1ea0533c6a6a6964
Previously the collector-side implementation of the worker interaction protocol
was a bunch of spaghetti code. Split it apart at places where it makes sense to
be easier to follow.
Change-Id: If3cc6b6fbf289dd778856b40b55316c76a6a6964
As the worker processes are now properly spawned using fork-exec, they no
longer inherit the stack size from the non-main coordinator thread, but get a
main thread with reasonable stack size on their own. For this reason the
coordinator threads can use the default stack size, and turn std::thread can be
used as the only reason for the custom wrapper was to enlarge the stack.
Change-Id: I1192474885abe9b0625ac483840b6e1a6a6a6964
Similar to how 7b37d5ea6a switched the subdaemons
from fork-only to fork-exec, do the same for the nix-eval-jobs workers. This is
the promised change that causes the signal handler thread to appear.
Change-Id: I590a5eb6bddfb546a331a01191fd788b6a6a6964
The retry added in 4c5efd4548 has never worked
properly, as releasing the pid the first time causes it to be -1 and thus
waiting for a random child on retry. While there is a good chance that this
really catches the crashed worker, this is by no means guaranteed, and it could
just as well have caught different restarting worker. Wait synchronously
instead. The worst possible outcome is a hang instead of an error message if
the worker manages to close its pipe but then fail to exit, which is not
supposed to happen in the first place.
Change-Id: I4e4c3f532ec15c3118f6f77821ab820a6a6a6964
warning: embedding a directive within macro arguments has undefined behavior
????? CLANG? why does this take -pedantic to show??
Change-Id: I6f4a9f3db1b89448f9f5d00d82b551e46a6a6964
This change brought to you by my uninstalling go but forgetting to
remove ~/.local/opt/go/bin from my PATH.
Change-Id: I937675bab11ac7e0678c896cba78204b6a6a6964
Maintains compatibility with mdbook 0.4.x.
Includes comments for what to remove once 0.5.x is the only we care
about. Some other changes technically could be changed at that point,
but currently serve to enable universal support of 0.4.x and 0.5.x
Fixes#1051.
Change-Id: Ic5b405038d180bcd357bbd9e5716879e0c26e5f5
In the same vein as be18b7dc2¹.
Before:
error: syntax error, expecting ';'
After:
error: syntax error, expecting ';' to end binding
[1]: be18b7dc25
Change-Id: Iffbcb113d2b892a50c646c9875e970376a6a6964
Lix blocks SIGPIPE, and std::cout ignores EPIPE (actually it probably stashes
an error code in some flag that no one ever reads, with the same end result).
Consequently, nix-eval-jobs would waste resources by continuing to evaluate
even when the reader interested in the results has long gone away. Instead, use
writeToStdout, which throws an error on EPIPE, leading to the process
terminating as desired.
Reported-by: Winter <winter@winter.cafe>
Change-Id: I962c09bab582a8ed27dd41c01b1519876a6a6964
Right now, the worker will not receive interrupts, as it has no signal handler
thread due to the way it forks, but this will change soon. Prepare to handle
worker interruption properly, by letting the worker exit successfully in this
case and adjusting the coordinator to handle this exit gracefully.
Change-Id: I9f79670b0d2004e7e2d8bf36cf67108e6a6a6964
For some reason that eludes me, it used to only print the last attribute
instead of the full chain.
Change-Id: I2f5f6e85e2b1ab3ab20a9c99bf06d1e45a4d09bb
`e`, `ae`, `j`, `jAttrs` … holy fuck this function was so much worse
than it needed to be. This refactoring includes:
- Descriptive variable names where possible
- More code comments indicating what even is happening
- Shuffled the control flow around to be more linear (move early return
conditions up, etc.) and have less rightwards-drift, to increase
readability
- Extracted `mergeAttrs` function for future changes and readability
Change-Id: I7253b47ce4910fdc67aa7d6edff2f3c0bb8fa55b
In one of the cases it pointed to the first instead of the second
occurence, which is a bit confusing
Change-Id: Ie508d1f84feb434708804d45aa8a9c8b1e3c5f69
How long do we have the value printer already? It's time to stop
concatenating strings like it's 2005
Change-Id: I3f5074de2439a1ad78af94de877bb141bc9f1d82
When using `builtins.derivation`, the `outputs` list can be omitted, and
in that case, it defaults to `[ "out" ]`, set this default when building
the shell derivation for `nix develop`
Fixes: #556
Change-Id: I4dc77d060f30ee499e186ec919de30bd3e5c53bb
Commit 5dc847b47b introduced usage of
std::chrono::milliseconds, which prints with the unit, leading to its
duplication. Remove this duplication again. Specifically, the explicitly
printed one is kept, due to better typography.
Change-Id: I724ea6be6f2a1349fe799602c3429f096a6a6964
We have done it!
The functional/lang framework has fully been migrated to functional2 :D
closes: #856
Change-Id: I63ad8d7dbcd9b5267ca04af68df73b1ffa3d6461
After multiple cycles of deprecation, we now use the lix repositorty as
a default, when no prefix is defined
Change-Id: I29b445fee39fb20c0ff024e3b95402f459144b62
Added builtins.warn` which takes two arguments: a message that is
displayed as a warning during evaluation which must be a string and a
value that is returned from the expression.
The next commits add new settings to control the behavior of the new
builtin: `debugger-on-warn` allows the user to start the debugger and
`abort-on-warn` aborts evaluation with an error.
Unlike upstream, I chose not to mark evaluation warnings from
`builtins.warn` as distinct from other warnings because that breaks the
commonly expected logging format `level: message`.
Co-authored-by: Qyriad <qyriad@qyriad.me>
Upstream-PR: https://github.com/NixOS/nix/pull/10592
Fixes: https://git.lix.systems/lix-project/lix/issues/579
Change-Id: I8658c88e5c27952b65e8b9f5525a572e0680cc1f
There are currently 4 different ways to run garbage collection using
lix:
- `nix-collect-garbage`
- `nix-store --gc`
- `nix store gc`
- (using the daemon directly)
As they were written all at different times, their output varies (and is
broken in some case). This unifies the display of informations in the
following ways:
- The list of paths in the results is always printed (in the nix3 cli it
is hidden unless `-v` is passed)
- The number of paths in the result set is always displayed
- The size of deleted paths is only shown when actually deleting things
(as it would have been 0B in any case)
Fixes#905
Change-Id: I40d9ec7c6d76795f6c6dd30df196d1e855bdb9db
This also allows removing a static initialization of a primop.
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Change-Id: I74da68205630f658b239e4327bf5e27bfc1b91da
This is done at the start of the command call, which allows not building
expensive things if the arguments are incorrect anyway.
Fixes#687
Change-Id: Ic924fe2115cf760684c6fdf7987279e96ab00286
the newly added f1/lang tests is required for the f1/lang framework
dismanteling to happen in a separate commit
Change-Id: Ic419c515262294c51a46d1513daa7848e4b71405
While the err file should always be empty, we prefer not have (easily
avoidable) warnings in the log, in order for actual warnings to be
spotted more easily.
Additionally this way no additional changes are required in case they
make use of some depreacted features in the future.
Change-Id: Ie078e2a851b2035d839f6b95d4188e475eb96b2d
autoargs get their own folder and are not allowed with the other
function args tests, just like they deserve their own ring of hell
It shall be noted though that this test is particularly pointless, as it
merely tests the normal argument passing (which is fine), and not the
✨magic✨ autoargs actually provide
Change-Id: Ia2dc00754a13c02e4926200141f90bc797820a15
That one is a bit confusing, as I renamed the old `in` test to `in-2`
(file identities in Git when)
Change-Id: Id0e17de414dbc1b1b616d456175704f98657a16a
Call it a bug, call it a feature, but the old testing framework
ungracefully shits itself when it has no tests to run … in other words,
we finally migrated all the parse-fail tests 🎉
Change-Id: Ibd7f1c04d9a396a20f14361af7924f0074d1ac23
The second test has also been fixed. I'd do it in two commits if
functional1 wasn't such a PITA (guess why we're doing the migration …).
Basically, the introduced syntax got changed later on, but the test
never got updated, and back then tests didn't test the output so nobody
ever noticed it now failing for a wrong reason.
Change-Id: I5b66127d507c65676b8dd4a82d1e1f2857fef145
This really looks like one of the first tests ever written, it doesn't
seem to be actively testing anything interesting
Change-Id: I442840db932203c25da2d8400ada791f8aed04ce
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh
Change-Id: Ida5959473ec469dd5e5e977ebdc8aab6e694b0de
https://github.com/NixOS/nixpkgs/pull/451579 PR
enabled c-aresSupport for curl on darwin which ended up breaking
DNS resolution in FOD sandboxes:
```
nix-run> exporting https://tangled.org/@weethet.bsky.social/nix-run (rev 73d7bf6b58848fb8f42e3a69816e0847f041c689) into /nix/store/m4m951648wmipxgwrgsml9gzjwfpfhm7-nix-run-73d7bf6
nix-run> Initialized empty Git repository in /nix/store/m4m951648wmipxgwrgsml9gzjwfpfhm7-nix-run-73d7bf6/.git/
nix-run> fatal: unable to access 'https://tangled.org/@weethet.bsky.social/nix-run/': Could not resolve host: tangled.org (Could not contact DNS servers)
nix-run> fatal: unable to access 'https://tangled.org/@weethet.bsky.social/nix-run/': Could not resolve host: tangled.org (Could not contact DNS servers)
nix-run> fatal: unable to access 'https://tangled.org/@weethet.bsky.social/nix-run/': Could not resolve host: tangled.org (Could not contact DNS servers)
nix-run> Unable to checkout 73d7bf6b58848fb8f42e3a69816e0847f041c689 from https://tangled.org/@weethet.bsky.social/nix-run.
```
with these sandbox failures:
```
deny mach-lookup com.apple.SystemConfiguration.DNSConfiguration
deny file-read-metadata /private/etc/hosts
deny file-read-data /private/etc/hosts
```
We allow those so that DNS resolution work fine
Change-Id: I9102293691972feb085adf8e9b1ad915bb3a36ab
This adds the enablement code to support HTTP/3 if the user requests it.
We leave it disabled because h3 is not onpar with h2 performance.
Change-Id: I1fd3d4c97b972dcf36bccacc6c9a8290e22b31e0
Signed-off-by: Raito Bezarius <raito@lix.systems>
Channels have moved from `nixos.org/channels` to `cnannels.nixos.org`.
This udates all relevant links (excluding release notes) to use the new
canonical URLs and replaces HTTP with HTTPS.
Fixes#1031.
Change-Id: I212821c44ac5e482c8e9eaa415c7d8ee17ff8341
Signed-off-by: adam <me@adamperkowski.dev>
URL literals were deprecated in 278fddc317,
effectively stabilizing the experimental feature. Enabling it has no effect any
more. Remove it.
Change-Id: I8fbca03b1a2be6a8cddfce644043ac0a6a6a6964
We start this section with shortcomings of unsandboxed builds.
Fixes#1018.
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: Ieb17e4340beab0c1197951813ae602de453a3fd9
Signed-off-by: Raito Bezarius <raito@lix.systems>
This is unnecessary because Darwin builds already cannot create
`set{u,g}id` files due to the minimal sandbox policy, because we can
forbid messing with the top‐level build directory directly in the
sandbox rules, and because Darwin builds can trivially avoid isolation
through temporary directories right now anyway.
This does regress the build directory isolation for builds with
`sandbox = false`, but I can’t imagine that mattering given the
above. The sandbox change prepares us for a world where we close
off shared temporary directories for `sandbox = true` builds and try
harder to achieve proper isolation on Darwin, but probably doesn’t
have a meaningful security impact one way or another for now.
With this change, we get down to 41 byte build directory paths on
Darwin, a ~2–3½ byte improvement over the old status quo. We can
also restore the 0710 permissions on Linux.
Change-Id: I6a6a6964a681c0365241fe7234831db656b76799
The minimum build directory length on Darwin with default settings
when we were still using `/tmp` was 22 bytes. Deriving build directory
names from my local store, the median and mode were 43, the mean was
around 44½, and the maximum was 127.
The switch to `/nix/var/nix/builds` over `/tmp` added a 15 byte
penalty, and the additional `/b` directory added another 2.
Now that we use opaque build directory names, the length is 48, so
we’re still at a ~3½–5 byte penalty over the previous status
quo. This change brings us down to 43, matching the previous median.
Note that these calculations do not take into account the fact that
`/tmp` is a symbolic link to `/private/tmp` on Darwin. Anything
that was canonicalizing paths would have had an additional 8 byte
penalty in the previous status quo that is not applicable here,
so we may already be ahead even without this change. If the more
opaque directory name here is undesirable, then that factor could
potentially help us squeeze by without. Alternatively, in combination
with dropping the `/b` on Darwin we could use `/nix/var/nix/bld` or
similar, but I feel that the paths in general are sufficiently opaque
that it should be okay to go with the shorter option here. Given that
some projects already had to reduce filename lengths to avoid this
limit even before the recent changes, I think it is best to try and
improve on the previous status quo.
Note that `/nix/var/nix/builds` will unfortunately not be cleaned up
on Darwin. However, we don’t clean up the directories inside it on
Darwin anyway, so hopefully that’s okay for now?
Closes: #913
Change-Id: I6a6a6964bffce7194bcddcaefb4c4a37569c7df5
We always use the default temporary directory, because
`createUniqueDir` has an interface nice enough to use directly for
the few bespoke uses.
Change-Id: I6a6a696450b7c0a0bd76655632fb14d7c5e38199
This makes the paths more nondeterministic, but more reliably unique,
and lets us remove the retry loop.
Note that this adds random entropy to the build directory visible
inside derivations on Darwin and unsandboxed Linux. It was already
non‐deterministic in the presence of concurrent builds and similar,
but now we can reliably expect it to be different every time. On the
whole I think that’s a good thing, as it is impossible to ensure
a single consistent build directory and derivation outputs should
not depend on it.
Package reproducibility isn’t great on Darwin to begin with,
though, and the reproducibility bugs this will turn up in packages
will be more urgent to fix than when the build directory was mostly
consistent. A quick survey of my local store shows that many C, C++,
and Rust binaries contain build directory references, likely due to
use of `__FILE__` and its equivalents; non‐binary offenders include:
* Install logs included in the Rust and Cargo bootstrap compilers
* Example errors in the Rust documentation referencing build paths
* Configuration information installed with CPython itself
* Python 2 metadata from resholve’s closure
* Cython metadata
* Generated headers in Facebook libraries referencing source paths
* Generated CMake files in Facebook libraries referencing source paths
I haven’t built that much in this store since the last GC, so this is
probably only a small sample of the problems across the tree. These are
all instances of <https://reproducible-builds.org/docs/build-path/>,
though, and should probably just be treated as general reproducibility
bugs outside of contexts like the Linux sandbox where we can normalize
them away entirely.
I have implemented away build directory paths for C/C++, applied some
additional fixes for non‐`__FILE__`‐related issues in binaries
from ATF and LLVM, and fixed the derivation bug causing the CPython
3 issue, and will work on upstreaming these changes. Rust is working
on the problem upstream, with some temporary workarounds we can
potentially apply in Nixpkgs for now. The rest will require some
distributed effort.
Change-Id: I6a6a69645b4915c56c0fdef904aa81684e4136c6
This does change the behaviour when the global temporary directory
does not exist, but other uses of the global temporary directory are
already broken in that circumstance, and it should be fixed centrally
if the use case is considered desirable. The logic was not present
before the recent churn around build directories – it was added now
that Lix is taking ownership of the build directory in the store –
so this should not be a meaningful regression.
Change-Id: I6a6a6964e345ea6803226c5ad759e836de7cb0ed
There is now no risk of race conditions on a system with a functioning
entropy source, and the bespoke prefixes are either redundant to the
default or unnecessary.
Change-Id: I6a6a69641211c6bb979ea48ad30aecb1a53d03f0
Relax the constraints on keeping the exact same filename format to
provide a more robust source of entropy with a simpler interface
(as previously suggested by eldritch horrors). Using 128 bits of
OS‐provided entropy ensures global uniqueness and allows us to
skip any thought of gracefully handling the case where these files
already exist.
My microbenchmark that repeatedly constructed paths like this and
printed them out showed that this takes about 1.23× the time of
the previous implementation, both taking on the order of a couple
microseconds for one iteration. Since everything that uses it is doing
things more expensive than printing to standard output, the actual
performance delta is likely to be lost in the noise. If it somehow
becomes a bottleneck, it can be optimized without sacrificing the
guarantees by reading from the system RNG only to seed a thread‐local
CSPRNG like [ChaCha8Rand], but I think that’s very unlikely.
We also tweak the recommended way of creating a temporary file inside
a directory in anticipation of later changes, and rename the `suffix`
parameter to `prefix` (it’s a prefix to the random characters and
a suffix to the root, but this way is more consistent).
[ChaCha8Rand]: https://c2sp.org/chacha8rand
Change-Id: I6a6a69648502c746d13d8c3bd2768cbbf1b90466
base32Encode now takes std::span<std::byte>, with a base32EncodeStr
convenience wrapper which takes std::string_view.
Co-authored-by: Qyriad <qyriad@qyriad.me>
Change-Id: I6a6a6964f799dc84ecbfb55c7ca03a064cff71d9
the change to shareable thunks also removed a few cases of pointer
equality checks that allowed structures containing functions to be
considered equal to other sets containing the same functions, even
if the sets themselves were pointer-equal themselves. *so* busted.
Change-Id: If87fdab658f9037ce2a654f69a9e3da6ae2f53e5
we can't change the protocol to allow daemons to do this, and we should
not try to guess what the `auto` store uri means depending on whether a
command was run by root or not due to copious side effects and not even
being able to tell whether the `auto` store uri was given explicitly or
not. while `auto` may *technically* allow this via its naming we should
resist the urge to add a hack and fix the underlying protocols instead,
especially since repairPath should be a rare, superuser only operation.
fixes#888
Change-Id: I1b53245db226199f827a89a237a2ab9907c3f766
if curl fails during a transfer and said transfer is piped through
libarchive we see very strange errors (#1009). in such cases we'll
want to prioritize reporting the source errors to allow debugging.
Change-Id: I8cc231257eca5b749471ecd38df15071170decc2
The test would fail if the identity cannot be determined
(as is the case e.g. when using auto-allocate-uids under darwin).
Instead we use the uid when the user name is not available.
Fixes: #1025
Signed-off-by: hexchen <hexchen@lilwit.ch>
Change-Id: I12f35840b78f989b5fd09bbd44c833dc6a6a6964
This helps users to debug whether their current profile symlinks are
correctly set.
Expected outputs look like this:
```
❯ sudo ./outputs/out/bin/nix doctor
[snip]
[PASS] All profiles are gcroots.
[PASS] Client protocol matches store protocol.
[INFO] You are trusted by store uri: local
[FAIL] Error: current generation cannot be discovered for profile:
'/nix/var/nix/profiles/default'
```
```
❯ ./outputs/out/bin/nix doctor
[snip]
[PASS] All profiles are gcroots.
[PASS] Client protocol matches store protocol.
[INFO] You are trusted by store uri: daemon
[PASS] You have 28 generations for profile
'/nix/var/nix/profiles/per-user/raito/profile'
The current generation number is '290'
```
Change-Id: I50c69cbeac3291d668f4c2332803411579adc944
Signed-off-by: Raito Bezarius <raito@lix.systems>
Co-authored-by: Qyriad <qyriad@qyriad.me>
Inaccessible-but-extant PATH elements can't be executed anyway, so we
can skip them as if they didn't exist, instead of erroring like before.
Change-Id: I9288c3ecb6768171e62a3088122d98421558eb03
In d2696cdd¹, libstore was made to prioritize NIX_BIN_DIR/nix (over
/proc/self/exe) if it exists. But `pathExists()` throws an exception if
lstat() returns any error other than non-existence.
Because this check happens in libstore's Settings constructor for the
static-initialized global `nix::Settings`, this case would abort Lix
well before main().
[1]: d2696cdd1e
Change-Id: I62e44c8a33b28c234900df3e4790c1badefee358
So far it was impossible to put absolute paths into files, as one
wouldn't know whre the test would be placed.
This commit adds a new Fileish variant called `EnvTemplate` which uses
the given string as a template and replaces the `@ENV_VARIABLE_NAME@`
placeholders with the according value of the environment variable.
This way one can use `@HOME@` or `@TEST_ROOT@` to build absolute paths
Change-Id: I425cb6408dceb8d7f26d136ace4ac98b1ca31ec3
Since we have async, we are unleashed and therefore we started to cause
people to run into system limits like number of open files.
We introduce an async semaphore which uses max(25 % of max open files,
max number of cores) which should lead to 256 maximum concurrent copies.
We believe that the copy operation is the only one that can cause issues
given that the daemon already runs with obscene fd limits.
Fixes#1022.
Change-Id: Iec433d10d5c5003962ea749cf8e32bafb314f0d9
Signed-off-by: Raito Bezarius <raito@lix.systems>
In case of empty messages, it is good to print the raw error code.
Additionally, we print request IDs which can help users to reconcile
what happened with the service provider.
Change-Id: I4d83c011c1b7a5514e3d1b21123df38308279044
Signed-off-by: Raito Bezarius <raito@lix.systems>
Completion status can be updated in two ways:
- progress callback
- transfer status update callback (esp. important in multipart
transfers)
It's guaranteed that one of the two will provide progress for our
promise.
This fixes#945.
Change-Id: Iac9e92df34f82fbc1facae7a411ccabf1688da52
Signed-off-by: Raito Bezarius <raito@lix.systems>
This fixes problems with S3 implementations that have mandatory payload
signing, e.g. Garage v2, AWS S3 itself, etc.
The problem manifested itself when the AWS SDK threw some error codes 99
(NETWORK_CONNECTION) with no messages and retried until it finally came
to a moment where the state machinery decided to send a
`Transfer-Encoding` header in a `Content-Encoding`/`Content-Length`-set
request with signed headers (even though payload signing is disabled),
causing the server to reject the transfer and crash the copy.
I did not debug super far what went wrong in AWS SDK, but I can confirm
this change makes transfers possible to finish with Garage v2.
Change-Id: Icc9e6a9f2afb0d760cf2d1e27816decd385a1d85
Signed-off-by: Raito Bezarius <raito@lix.systems>
Calling `nix.nix` with an empty argument list, would crash on darwin due
to a Index Out Of Range Error, as the build="auto" functionallity would
try to access the first element of the provided arguments (second
element at the place of call) resulting in a crash
Change-Id: Ia678109808b3f75a30182114baa421cedaa8759b
libarchive's xz offers single threaded xz compression which is very slow
and provides ~10-20Mbps compression speed in addition to maxing a core.
In exchange, it achieves optimal compression ratios among all our
compression methods.
Nonetheless, xz prevent the saturation of 1Gbps+ connections and slow
down significantly decompression for end users. As these connections and
faster hardware is becoming prevalent for cache servers and clients, we
offer to default to zstd.
Lix is a "compress once, decompress many times" application. To avoid
incurring a high penalty to end users very sensitive to compress ratio
(very slow Internet connections), we dampen the consequences of
switching to zstd by increasing the default zstd level to 12.
On one example, xz will compress a 4.4GB file to 632MB, zstd on 12 will
compress it to 775MB, that is a ~18 % increase over the optimal xz
compression. zstd took 18 seconds to produce this file.
Increasing to level 14 leads to a 773MB file while taking 37s.
Increasing to level 16 leads to 735MB file while taking 66s.
Finally, xz took 77s, so a 50 % reduction in time taken to compress in
exchange of an increase of 18 % of the compressed size.
This change will reduce issues encountered in #945 but is probably not
the root cause.
References:
- https://discourse.nixos.org/t/switch-cache-nixos-org-to-zstd-to-fix-slow-nixos-updates-nix-downloads/23961
Change-Id: I7beda2bf2c1fed146dcb797b8f85dc290c486ab2
Signed-off-by: Raito Bezarius <raito@lix.systems>
if a promise for an async s3 operation is cancelled is inner fulfiller
will be destroyed, but the s3 operation may still be running. once the
operation finished and attempted to fulfill the promise we'd segfault.
Change-Id: I603c75d6cc39a1aee465b3ceee06b638f7f7f596
As we saw actual users running into them throughout the debugging of
#920 and #1014.
It's best to document and point end users to this so they can take
measures for themselves.
Change-Id: I7d08407f4354055bf65fc6dd7d1624c5a9304402
Signed-off-by: Raito Bezarius <raito@lix.systems>
Fixes#1014.
Address family detection is performed by Pasta, we influence Pasta into
enabling dual stack by listing both an IPv4 & IPv6.
Change-Id: I4121e0eae7d5185c287957106adc36edf3f82a40
Signed-off-by: Raito Bezarius <raito@lix.systems>
Instead of letting inner shell perform the cleanup, we will fork/execvp
and finish the work ourselves. Once an interruption arrives, we wait
again and return the exit status code of the inner shell.
This idea was suggested by eldritch horrors.
Fixes#1020.
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: Id38af7ac33874aa1c4b7ca9c6123e3cbd8c3cd8c
Signed-off-by: Raito Bezarius <raito@lix.systems>
This was forgotten which should have been part of 64b9247f.
Change-Id: I3cf83d8699f2c1963ce530bbea6a511075202a79
Signed-off-by: Raito Bezarius <raito@lix.systems>
Legacy commands "main function" will now return an exit code.
Fun fact: `main_nix_build` was the only one not returning integers
already.
Change-Id: Ia43a16c3c3fb9a670e8889aefc4ee9b6528a7df4
Signed-off-by: Raito Bezarius <raito@lix.systems>
This also prevent `legacy` to return nothing as it's void.
Change-Id: Ic797544a59b04b41ad9e1c46af4cad3a1cb6fdc8
Signed-off-by: Raito Bezarius <raito@lix.systems>
realistically only runProgram is ever called for interactive reasons,
and even those calls seem to be rather ill-advised in many cases. the
chance of multiple interactive processes interfering with each other,
whether for input or for output, must be very low to make this in any
way reasonable: if e.g. git calls ssh for multiple fetched inputs and
ssh requests passphrases for both we can otherwise not guarantee that
*any* input is routed correctly. misrouted output is merely annoying.
Change-Id: I794e3fdf0a3238cb9292003a89ac267f0de7a939
Eval errors will now print a simple, no frills chain of involved
derivations at the bottom. For example, trying to evaluate
`pkgs.xonsh.override { python3 = pkgs.python2; }` has the usual
Package ‘python-2.7.18.8’ in /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/pkgs/development/interpreters/python/cpython/2.7/default.nix:398 is marked as insecure, refusing to evaluate.
message, but now also includes the following:
note: trace involved the following derivations:
derivation 'xonsh-0.19.9'
derivation 'python2.7-xonsh-0.19.9'
derivation 'python2.7-setuptools-44.0.0'
To give the user information about why the erroring derivation was
involved in the first place.
We would like more structured information in the future, but this should
still be a significant improvement.
Change-Id: Icf6da52abd0a043cfb63943bf0b0c160c21ee59e
the wrapper is needed by transfer streams to restart a failed transfer
if desired. curlFileTransfer itself is more of a fancy handler for the
thread we're dedicating to curl io handling. the thread will stay with
the multi handle for now because quit handling needs to stay there. we
could have CurlMulti keep only a flag, but that does not help us much.
Change-Id: I99550f0bbb635b75898ca7260f08275df86050e3
we have anonymous namespaces and no other scoping requirements, we can
move it out. this will also make it easier to move *other* things out,
such as the transport streams and all the shared multi state handling.
Change-Id: Iafa1eaeb69e096e98c4667f0531b17d5fe7c78a6
waiting for them to finish leaves us prone to deadlocking, e.g. if some
active transfers is blocked and hasn't been destroyed before our thread
was asked to exit. aborting transfers is the safer option here; even if
we cause use after frees we'll at least have a crash, not hang forever.
Change-Id: I333dcced810e8a85e2ded1f377962a0176798098
If BUILD_TEST_SHELL is not set or empty, None or the empty string respectively
should not make it into the PATH. Ensure this property.
Change-Id: I4ce9b0c06c407b465308b63b9cb64e7d6a6a6964
this is still somewhat experimental and should be considered in flux. we
will have to nail down a logger interface once we start moving the store
protocol to rpc, but until we do that we can use build hooks to test it.
Change-Id: Id20cd346c9520f45871799c31b0af040adde56ef
extracting the size-specific log handling into a new sink deduplicates
the size-limiting code and lets us clean up the derivation goal a bit:
a number of log-related fields we kept only because previous ownership
semantics were a lot more ad-hoc can be moved into the new sink class.
Change-Id: Ie770a276597c84f9d928a9624c0f0de86adfe815
we'll want to remove handleRawChildStream from DerivationGoal, but the
timeout handling is still important. the wrapper is thus useful, but a
more generic way to pass it a promise to do the handling is necessary.
Change-Id: I55bf041d51e4f8a8b8291deae053423d06724329
we never need the connection or the rpc client directly after the hook
is constructed, only the hook instance rpc object. wrapping them using
kj primitives makes this more explicit and prevents destruction errors
Change-Id: I7e0b14d7e365c2e001273e04c0e741ba4afb12f2
this makes it possible to wait for the hook to exit in one place,
process its exit status for some purpose, and later process it in
another place for a different purpose. rpc will need this to shut
down a hook cleanly after the hook has completed its assignments.
Change-Id: I5af2d5aac4b02a0a200d720d0e6f0b5df5496aaf
only LocalDerivationGoal needs these, and once the child is done
running (or is begin torn down) we can just destroy all of them.
Change-Id: Id4d9f0105aaf33d6736c435bb891d3f9af8e7894
wrapping the wait in a timeout at the call site is insufficient because
a timeout would not relock the mutex and leave the raii object invalid.
Change-Id: I0be2aaf6b3c9966070fb0f1deaab048765fd7af3
it seems like pytest is updating the message printed when skipping a set
due to no parametrization
which leads to breakage on staging-next.
this commit changes the expected message to allow both old and new skip
message
Change-Id: I34d11ad84d7075911d8229fbc89889bb880a27c9
This partially reverts commit f68233ec43.
while more modern capnp in *does* have the build system changes, it
does *not* have the patches. and those patches are rather important
to us; the clock patch because our CI machiens don't behave the way
capnp expect and the nodiscard patch because it *is* a bugsquasher.
Change-Id: Ic273777f09fd8e5e7001f815db8998dae2dd7b88
The prospective callers of this should probably be doing something
smarter or more abstracted to begin with, but this is useful as an
incremental improvement for call sites with existing `makeTempPath`
logic in the face of filename length limits.
Change-Id: I6a6a69644292f5bbf984a1df90192e06c6022b53
LocalStore::optimisePath() was made asynchronous in 38b75b736¹, but we
lost warnings for not awaiting kj::Promise in f68233ec4², since our
capnproto no longer has the [[nodiscard]] patch.
[1]: 38b75b7367
[2]: f68233ec43
Change-Id: I2363396949f40dd43eae08cc76be9ede6b488cf7
this is important for rpc error reporting: if the peer disconnects we
want to throw a useful error that can e.g. let pending builds finish,
not kill the entire process because kj errors are non-lix exceptions.
Change-Id: I41ebe10ad7fd65d886238ce52eeca1fa243b1bb2
this removes another file descriptor from the build hook. we are now
using only rpc to communicate between build hook and daemon, setting
the stage for a future world in which we don't even need build hooks
Change-Id: I706d9d3f2a420abd42fc7762c205931a45d3c8c5
interpolation of the remote store name into activity text is now done in
the build hook, so we no longer need to send it back to the daemon. this
also reduces the dependency of the daemon on how *exactly* remote builds
are done a little bit, and maybe in time we can make these fully opaque.
Change-Id: Ie51c4f2cd107350fec4ad0a8ef83621ed86db6a2
only local builds can now have a builderOutFD, remote builds only log
via json streams. these two kinds of logs have different requirements
and each is only needed by its respective build kind. splitting these
apart thus makes sense, though ideally we'd also split DerivationGoal
into a RemoteDerivationGoal to clean up the rest, but that will wait.
Change-Id: Ib577537266d1160355ab9c44b4604ebda87a7d04
treat it like ssh-ng instead, and have the build hook do the translation
of ssh stdout to the json log message steam the derivaiton goal can use.
since the ssh-ng path also handles all store urls that aren't legacy ssh
we now have a single logging system that handles every remote build kind
equally, without requiring fd passing into the build hook. this is later
required to rpc-ify the log stream emitted by build hooks to the daemon.
Change-Id: Ifb522eb8a9745029050f16b1b3b3601a6ddac748
When the maximum memory size is chosen too large (relative to the free memory
usage), so that the evaluation workers don't fit into the free memory, they
will start being swapped out. Further increase in memory usage will not be
reflected in the RSS, so they will not exit, and instead fill up all swap too
until the system runs out of memory.
The BDW-GC keeps track of its heap size, including any parts of it that may be
swapped out. For this reason it does not suffer from a similar problem. While
not all memory usage by the evaluation worker is accounted for by the GC, a
simple affine model for the overhead works reasonably well in practice. The
specific parameter values were determined using nixpkgs evaluation, and appear
to work quite well also for somewhat different workloads (like the `hydraJobs`
of Lix, which include a bunch of NixOS system configurations). So this is what
gets used.
When Lix is configured without the BDW-GC, of course this approach cannot be
used. In this case the old strategy is retained.
Change-Id: I6cb5f0a9b4ceda9dd14be165dda108cd6a6a6964
Includes link to the Buildkite SSO login, which is currently not visible
anywhere else in documentation.
Change-Id: Ic3ec6412127b2ac0df7e6efc4ec6020cc9d8b34f
These are a footgun and are not acceptable in functional2 due to thread
safety, effects on other tests, etc.
Co-authored-by: Commentator2.0 <lix@crystal-cavern.systems>
Change-Id: I8d7285061eaa9bab27edd52f3646024c8cf605e5
When providing both a `.err.exp` and a `.out.exp` for a lang test
**not** containing a toml, the test was collected twice and hence threw
a duplicate id error.
This commit adresses this issue
Change-Id: Ia781d69e0f1db2809a30192d6cd0a98861e6cc32
this restriction was added in 4af2611bd1
and extended later. it makes little sense to allow *one* activity type
specifically and drop all others on the floor, especially since broken
builders can create transfer activities with ids that would be used by
other, "real" activities. the only thing achieved here is to drop logs
written in json format in the build sandbox, which does not help much.
we may revisit this restriction later during other log-related rework.
Change-Id: I8bda494083877b71a2f958470fa52380f6ec4968
it was split up like this only because the old worker system had no
promises. since we have them now we can use them for encapsulation.
Change-Id: Idbd523eff617d4c53c14b125ab3dfce4979cdab1
currently all loggers can always accept messages and never suggest
flushing buffers. in the future this may change, and at that point
we're already fully set up for it. local loggers should never keep
asynchronous (i.e. network-backed) buffers, disk buffers are fine.
networked loggers will require buffers and periodic flushes later.
Change-Id: Ide2114f5bc17f4a1d289c92ed4f9381a1d59dacf
this keeps the call stack flat even for very deep store directories.
moving progress reporting out of the method optimizing one path also
lets us move the inevitable IO wait and associated promise overhead.
Change-Id: I5083d88b05c247c7e19a21a646bcf3f62bfa3200
create activities from loggers themselves instead of passing the logger
as a constructor argument and allow direct construction of children, no
direct logger access needed. most call sites are not changed because we
still need to handle the "no parent" case, and the logger method can do
that more cleanly than a ternary at each site that creates an activity.
we may eventually want to create a root activity, which is cleaner too.
Change-Id: I295e056228dabb08a1316eba7973874784baa113
remotes can generate the same activity ids we might generate locally,
especially if the remote is accessed over the network. in that case a
pid collision is possible, and since activity ids are sequential it's
very possible to create colliding activity ids on both sides as well.
Change-Id: Id58074a41f5f7a59171b52818d1fb5a1beb4bf40
activities are scopes, and there's no reason scopes should not be able
to move around. this also means we can *create* them elsewhere without
also having to box them in some way, making parent relationships clear
Change-Id: I5df036e12ebd8270feb4dca1f23b6bee1f08e906
Currenlty `lix` and `nixos-module` have slightly diverging version
numbering. This commit fixes the lix side of things, in accordance to
the format proposed in
https://git.lix.systems/lix-project/lix/issues/585#issuecomment-7336
which allows for better use of `builtins.compareVersions`
fixes#585
Change-Id: I2072b701f9fadf780e7aab67b62fbcd2ec5df76d
this has no performance impact in any benchmarks we've run. nul bytes
are still used as implicit truncation points in many places all over:
rejecting them in all locations that treat them as a string end point
requires large changes such as using a proper path library everywhere
Change-Id: I936158bd435f6abf009a689adfbc24496262c578
This reverts commit 42691f0d94.
Reason for revert: darwin builds time out now, apparently due to newly appearing infinite loops in the test suite
Change-Id: I81a09efd53636a287f3095ac87ed8854b6e57922
The functional2 test suite was broken in the development shell for two mostly
separate reasons leading to "no such file or directory" errors:
* The `BUILD_TEST_SHELL` (already containing the correct path) would always be
set from the Meson option `build_test_shell`, even to the empty string if
that option is not set. Skip the overwrite in this case to make the
environment variable work again.
* Sandboxed builds would fail to find their builder `/bin/sh`, since in the
development shell busybox is not found, so no sandbox shell gets configured.
Use the shell from `BUILD_TEST_SHELL` instead. (How this does not break the
old functional tests remains a mystery.)
Change-Id: I6a6a696424e8caaef3f9b68e3738bfd58ea0b056
Local store make use of SQLite as a cache for output maps, they can
diverge with actual store reality.
Causing a crash at realization time.
We can just throw an error instead.
Change-Id: I943a4bddfd2461a34933e9ec079a6638ab58e5b1
Signed-off-by: Raito Bezarius <raito@lix.systems>
Fixes fj#940.
When running `nix-shell`, the `$NIX_BUILD_TOP` environment variable is
set to `$TMPDIR` or `/tmp`.
nixpkgs stdenv uses $NIX_BUILD_TOP to create `$NIX_BUILD_TOP/env-vars`
which contains all the environment variables set by stdenv. This is used
for debugging purposes in combination with `--keep-failed` to reload the
bash environment of a derivation.
`$TMPDIR` is often unset, therefore, `/tmp/env-vars` was constantly
being created. On a multi-user system or, when you run Lix as root, you
might create a `/tmp/env-vars` with different permission bits.
As a result, `nix-shell` can cease to function because that file will
fail creation for an unprivileged user for example.
fj#940 rightfully remark that the code is not consistent between
nix3-develop and nix-shell and it should be reworked.
Change-Id: Iddf15945385d8bd497b2800b37fee5e1f97689b7
Signed-off-by: Raito Bezarius <raito@lix.systems>
This avoids mangling debug gdb scripts and breaking them resulting in no
script loading at all.
Change-Id: Ib5e7034c37a1e41d1a1c9bd8442b8b8b8758e350
Signed-off-by: Raito Bezarius <raito@lix.systems>
UnsafeValueList held pointers from Value which were not necessarily GC
allocated, causing mayhem when evaluating something with genericClosure
(texlive environments).
We get rid of storing pointers and we let comparison take places on
const references.
We keep `gc_allocator<Value>` so that GC can scan things it allocates
inside the `res` list.
Co-authored-by: eldritch horrors <pennae@lix.systems>
Reported-by: qbit
Change-Id: I4fed3a3d9e18a2ef2d751f32d81801540e196f92
Signed-off-by: Raito Bezarius <raito@lix.systems>
thunk values are shareable, and we can represent invalid/uninitialized
values with a special bit pattern that makes no sense otherwise. there
is no need to keep allocating values on the heap, instead we can treat
values like reference-counted smart pointers to heap objects, which in
turn lets us save a lot of allocations and, ultimately, gc heap space.
compared to our baseline (main of 2025-09-27) we save 15%+ memory on a
system rebuild and 17% on nix search. eval time regresses by ~3% for a
system rebuild, while nix search is 7% faster. further optimization is
probably possible (but for now this will just have to be good enough).
Change-Id: Ib6c47acdbe2fac4f76a83c2269f16f30ef66b2e1
with thunk state being shareable we no longer need to worry about value
uniqueness, only about value lifetime. this means we can liberally drop
indirections and allocations, passing references instead of pointers or
using stack memory instead of gc-managed memory for some intermediates.
Change-Id: I2d48a6fd57a376d544bd9bd2d05e5420611986d1
there's no measurable performance gain in real-world testing to abusing
our list value storage like this. we haven't tested how much storage we
actually need on the stack to offset most of the temporary storage cost
and used 128 as a "good enough" value instead, reserving 1 kiB of stack
space on 64 bit platforms in a leaf function. this should do for a bit.
Change-Id: Ie98519b1da3e6fe685da88d1c44ffb4580fb592a
Value is already tagged. Value::Acb blocks are allocated for lambdas (so
we can fit the value tag into the three bits we have available), but the
current layout is rather wasteful for this purpose. the type bits can be
stored together with parts of pointers, which in the lambda case will be
the scope the lambda captures. the expr could also be used, but Env is a
gc-allocated item and thus guaranteed to be aligned properly for tagging
Change-Id: Ia685875387c7795bc4a00d73d1ce3cfea84e7297
this is a strong prerequisite for making values themselves copyable
without duplicating evaluation side effects. with this we can treat
`Value` the way we treated `Value *` to date and drop indirections.
Change-Id: I08f30d12697614a3ae7149615f6f1da83b13f52b
preserve type information of wrapped exception when constructing a
ForeignException. this requires using `std::current_exception` for
construction in all cases as passing the exception by reference to
any function erases dynamic type information: `throw` may copy the
thrown exception from its source expression; this only works as we
need it to if static and dynamic type of the expression are equal.
fixes#1000
Change-Id: I7167f298540de7c43c2566592bdb8d7a93c34d31
same as for null: we have few of them, they're statically allocated,
and they're not the largest contributor to the value population. not
storing them in Value itself frees up resources we *will* use later.
Change-Id: I521f9f243f48f56a78f7bffdf1dc1f0bc40a5c2d
we only need the one object for it. there's no need to waste precious
bits of the value internal type enum for this simple singleton datum.
Change-Id: Ie314b5bf429015e518798d9d65ad8ab2bb84a38e
floats are used very rarely, and our float support is bad enough to
strongly discourage using them on reproducibility grounds alone. we
can thus move them to more expensive storage without hurting folks.
Change-Id: I1086f612f85e294dd3fae4a2d334e09f52bbe4a8
external values very rarely appear during eval "normal" eval, and
creating them is pretty expensive. does *anything* even use them?
Change-Id: Id50fa3f76b7e1f551d550d99996a1ed5880b2531
despite not using allocation caches this does not have a statistically
significant performance impact, with less than 1% extra memory needed.
Change-Id: Ibe51a55ba986e471f217f3724977af17880fafff
using the same nodes as tApp is possible, and thanks to multi-arg app
nodes it can even be a bit faster than the linked lists used to date.
Change-Id: Idccb7c0b54c808e62da85d1c42ee09e6e92c4f7b
these behave like the old chains of app nodes, but they can store more
than one argument per node. for tApp values themselves this is not all
that useful, but if we could share tApp and tPrimOpApp backing storage
we could avoid creating and traversing the linked lists of values that
are currently needed to represent partially applied builtin functions.
Change-Id: I5a2a02d9733e1e0be5443459e2998d62fd3b9a5b
paths already are just strings with slightly magic semantics. the type
can mirror that at no perf cost, letting us drop one internalType tag.
Change-Id: I98acaa4fe3bedd28fc0841e1b81184d8dcddacc5
memory overhead is minimal and performance impact not measurable. once
we've done something like this for all value types that don't fit in a
single machine word we can cut a word from Value, offsetting the cost.
Change-Id: I9813bacd7e851957ad3426aed8f74033179a4212
we now use a single cache set for a number of sizes from one to eight
words. this also matches small attrsets, but perf impact seems small.
Change-Id: Icf16b329b98a20fcc9fe75e6395e148f0852c798
on its own this is not very useful, but having accessors for every value
kind is a prerequisite for doing smart things with Value than the union.
the net effect for now is only to add a few parentheses across the tree.
Change-Id: I88688ac09eb08495dad1eb221034ca540f094950
If you need to disable cgroups temporarily, remember that you can do
`NIX_CONF='include /etc/nix/nix.conf\nuse-cgroups = false' nix-build ...` or
`nix-build --no-use-cgroups ...`.
## What about other service managers than systemd?
systemd has a [documentation](https://systemd.io/CGROUP_DELEGATION/) on how to
handle cgroup delegation from service management perspective.
If your service manager adheres to systemd semantics, e.g. writing an extended
attribute `user.delegate=1` on the delegated cgroup tree directory and moving
the `nix-daemon` process inside a cgroup tree to respect the inner process
rule, then, the feature will work as well.
## Why is the cgroup feature still experimental?
While the cgroup feature unlocks many use cases, its behavior and integration (e.g. user experience), especially at scale on build farms or in multi-tenant environments, are not yet fully matured. There’s also potential for deeper systemd integration (e.g. using slices and scopes) that has not been fully explored.
To avoid locking in an unstable interface, we’re keeping the experimental flag until we have validated the feature across a broader range of scenarios, including but not limited to:
synopsis:Fix develop shells for derivations with escape codes
issues:[fj#991]
cls:[4154,4155]
category:Fixes
credits:[Qyriad]
---
ASCII control characters (including `\e`, used for ANSI escape codes) in derivation variables are now correctly escaped for `nix develop` and `nix print-dev-env`, instead of erroring.
The attribute set printer, such as is seen in `nix repl` or in type errors, now prints hyperlinks on each attribute name to its definition site if it is known.
Example: all of the attributes shown here are hyperlinks to the exact definition site of the attribute in question:
1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
2. The user lacks permissions for that path.
We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
**Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.
You're not alone. Thousands of Lix users suffer every day from excessive `builtins.toString` syndrome. It’s 2025, and we still have to cast integers to use them in strings.
To address this, Lix introduces the **`coerce-integers`** experimental feature. When enabled, interpolated integers within `"${...}"` are automatically coerced to strings. This allows writing:
To enable the feature, you need to add `coerce-integers` to your set of experimental features.
### Stabilization criteria
The `coerce-integers` feature is experimental and limited strictly to string interpolation (`"${...}"`). Before stabilization, the following must hold:
1.**Interpolation-only**
Coercion must not occur outside interpolation. Expressions like `"" + 42` must continue to fail.
2.**Expectation that no explicit cast are being observed**
Cases observing explicit coercion (e.g., via `tryEval` gadget or similar) are expected not to be load-bearing in actual production code.
### Timeline for stabilization
If the feature proves safe and is widely adopted across typical usage (e.g., actual configurations in the wild turning on the flag, non-trivial out-of-tree projects using it), the experimental flag will be removed **after six months of active use or two Lix releases**, whichever is longer.
This avoids locking the feature in experimental status indefinitely, as happened with Flakes, while allowing time for validation and ecosystem integration.
### What about coercing floats or more?
Coercion beyond integers -- such as for floats or other types -- is **not planned**, even under an experimental flag. Questions like "what is the canonical string representation of a float?" involve subtle and context-dependent trade-offs. Without a robust and principled mechanism to define and audit such behavior, introducing broader coercion risks setting unintended and hard-to-reverse precedents. The scope of `coerce-integers` is intentionally narrow and will remain so.
In terms of outlook, a proposal like https://git.lix.systems/lix-project/lix/issues/835 could pave the way for a better solution.
synopsis:Remove reliance on Bash for remote stores via SSH
issues:[fj#830, fj#805, fj#304]
cls:[3159]
category:"Fixes"
credits:[raito]
---
The pre-flight `echo started` handshake -- added years ago to catch race conditions -- has been removed.
After removal of connection sharing in Lix 2.93, it required a Bash-compatible shell and a standard `echo`, so it failed on:
* builders protected by `ForceCommand` wrappers (e.g. `nix-remote-build`),
* BusyBox / initrd images with no Bash,
* hosts using non-POSIX shells such as Nushell.
The race the probe once addressed was tied to SSH connection-sharing -- since connection-sharing code has already been removed, the probe is now pointless.
Real connection or protocol errors are now left to SSH/Nix to report directly.
This is technically a breaking change if you had scripts that relied on the literal "started" which needs to be updated to rely on other signals, e.g., exit codes.
synopsis:"repl-overlays now work in the debugger for flakes"
issues:[fj#777]
cls:[3398]
category:Fixes
credits:[jade]
---
Due to a bug, it was previously not possible to use the debugger on flakes with repl-overlays, or with pure evaluation in general:
```
$ nix repl --pure-eval
Lix 2.94.0-dev-pre20250617-87d99da
Type :? for help.
Loading 'repl-overlays'...
error: access to absolute path '/Users/jade/.config/nix/repl.nix' is forbidden in pure eval mode (use '--impure' to override)
```
This is now fixed.
The contents of the repl-overlays file itself (i.e. most typically the top level lambda in it) will be evaluated in impure mode.
It may be necessary to use `builtins.seq` to force the impure operations to happen first if one wants to do impure operations inside a repl-overlays file in pure evaluation mode.
synopsis:"`disallowedRequisites` now reports chains of disallowed requisites"
issues:[fj#334,fj#626,gh#10877]
category:Improvements
credits:[ma27,roberth]
---
When a build fails because of [`disallowedRequisites`](@docroot@/language/advanced-attributes.md#adv-attr-disallowedRequisites), the error message now includes the chain of references that led to the failure. This makes it easier to see in which derivations the chain can be broken, to resolve the problem.
Example:
```
$ nix-build -A hello
error: output '/nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2' is not allowed to refer to the following paths:
synopsis:"Lix libraries can now be linked statically"
issues:[fj#789]
cls:[3775,3778]
category:Fixes
credits:[alois31]
---
Previously the pkg-config files distributed with Lix were only suitable for dynamic linkage, causing "undefined reference to…" linker errors when trying to link statically.
Private dependency information has now been added to make static linkage work as expected without user intervention.
In addition, relevant static libraries are now prelinked to avoid strange failures due to missing static initializers.
In the Lix evaluator, **symbols** represent immutable strings, like those used
for attribute names.
In evaluator design, such strings are typically [**interned**](https://en.wikipedia.org/wiki/String_interning), stored uniquely
to save memory, and Lix inherits this approach from the original C++ codebase.
However, some builtins, like `builtins.attrNames`, must return a `Value` type
that can represent any Nix value (strings, integers, lists, etc.).
Before this change, these builtins would create lists of `Value` objects by
allocating them through the garbage collector, copying the symbol’s string
content each time.
This allocation is unnecessary if the interned symbols themselves also hold a
`Value` representation allocated outside the garbage collector, since these
live for the full duration of evaluation.
As a result, this reduces the number of allocations, leading to:
* A significant drop in maximum [resident set memory](https://en.wikipedia.org/wiki/Resident_set_size) (RSS), with some large-scale
tests showing up to 11% (about 500 MiB) savings in large colmena deployments.
* A slight decrease in CPU usage during Nix evaluations.
This change is inspired by https://github.com/NixOS/nix/pull/13258 but the approach is different.
**Note** : [`xokdvium`](https://github.com/xokdvium) is the rightful author of https://gerrit.lix.systems/c/lix/+/3300 and the credit was missed on our end during the development process. We are deeply sorry for this mistake.
> If you are building via the Lix daemon (default on Linux and macOS), it is the Lix daemon user account (that is, `root`) that should have SSH access to a user (not necessarily `root`) on the remote machine.
>
> Furthermore, `root` needs to have the public host keys for the remote system in its `.ssh/known_hosts`.
> To add them to `known_hosts` for root, do `ssh-keyscan USER@HOST | sudo tee -a ~root/.ssh/known_hosts`.
> To add them to `known_hosts` for root, do `ssh-keyscan HOST | sudo tee -a ~root/.ssh/known_hosts`.
>
> If you can’t or don’t want to configure `root` to be able to access the remote machine, you can use a private Nix store instead by passing e.g. `--store ~/my-nix` when running a Nix command from the local machine.
## Configuration
The list of remote machines can be specified on the command line or in
the Lix configuration file. The former is convenient for testing. For
example, the following command allows you to build a derivation for
- `uri`: Due to backward compatibility, the `ssh://` may be omitted for the store-uri.
- `ssh-public-host-key`: The key must be provided encoded in base64. Specifically calculated via `base64 -w0 /etc/ssh/ssh_host_ed25519_key.pub`
### Format detection
At first, the given configuration is being parsed syntactically as a toml.
If parsing fails and the given configuration contains a `"` the error is presented to the user, as those characters are necessary for TOML, but disallowed for the legacy format.
Otherwise, parsing is retried using the legacy format.
If non-syntactic errors are detected within the toml, the exception will always be shown to the user directly.
## Builder selection
The configuration(s) above specify several machines that can perform `i686-linux` builds.
However, `granite` will only do builds that have the attribute
```nix
requiredSystemFeatures = [ "benchmark" ];
```
or
```nix
requiredSystemFeatures = [ "benchmark" "kvm" ];
```
`diorite` cannot do builds that require `kvm`, but `andesite` does support
such builds. For regular builds, `diorite` will be preferred over
# [Pasta](https://passt.top/passt/about/): a network sandbox for fixed-output derivations
## Introduction
This section only applies to **Linux systems** as Pasta is a Linux-only measure.
Since [CVE-2025-46416](https://lix.systems/blog/2025-06-24-lix-cves/), the Lix project decided to adopt [Pasta](https://passt.top/passt/about/) for all fixed-output derivations, protecting against various attack vectors such as UNIX abstract domain sockets or more manipulation at the network layer from a malicious fixed-output derivation code.
Pasta acts as a translation layer between a layer-2 network interface and layer-4 sockets (TCP, UDP, ICMP/ICMPv6 echo) on the host. It requires no special privileges and can serve as a alternative to [SLiRP](https://en.wikipedia.org/wiki/Slirp) which was used [by Guix to mitigate the same problem](https://codeberg.org/guix/guix/commit/fb42611b8f27960304db5a1c0d33b8371dcde2a8).
## How to disable Pasta?
It's sufficient to pass `pasta-path = ""` in your `/etc/nix/nix.conf` or on the command line `--pasta-path ""` of a Lix invocation.
## Known issues surrounding Pasta
- Only the first DNS server in `/etc/resolv.conf` is considered: failover is not possible.
- [Reduced feature set compared to the Linux kernel](https://passt.top/passt/about/#features)
- [Performance overhead in multi-gigabits contexts and IMIX MTUs](https://passt.top/passt/about/#performance_1)
@@ -58,7 +58,7 @@ $ nix-build flake:nixpkgs -A firefox
$ nix-build flake:github:NixOS/nixpkgs/release-23.11 -A firefox
```
Finally, for legacy reasons, if a path starts with `channel:`, the rest of the argument is interpreted as the name of a *nixpkgs* channel tarball to fetch from `https://nixos.org/channels/$CHANNEL_NAME/nixexprs.tar.xz`.
Finally, for legacy reasons, if a path starts with `channel:`, the rest of the argument is interpreted as the name of a *nixpkgs* channel tarball to fetch from `https://channels.nixos.org/$CHANNEL_NAME/nixexprs.tar.xz`.
This is a **hard coded URL** pattern and is *not* related to the subscribed channels managed by the [nix-channel](./nix-channel.md) command.
> **Note**: any of the special syntaxes may always be disambiguated by prefixing the path.
Channels are a mechanism for referencing remote Nix expressions and conveniently retrieving their latest version.
The moving parts of channels are:
- The official channels listed at <https://nixos.org/channels>
- The official channels listed at <https://channels.nixos.org>
- The user-specific list of [subscribed channels](#subscribed-channels)
- The [downloaded channel contents](#channels)
- The [Nix expression search path](@docroot@/command-ref/conf-file.md#conf-nix-path), set with the [`-I` option](#opt-I) or the [`NIX_PATH` environment variable](#env-NIX_PATH)
@@ -77,9 +77,9 @@ This command has the following operations:
Subscribe to the Nixpkgs channel and run `hello` from the GNU Hello package:
@@ -19,7 +19,7 @@ This description is not normative, but a feature removal may roughly happen like
1. Add a warning when the feature is being used.
2. Disable the feature by default, putting it behind a deprecated feature flag.
- If disabling the feature started out as an opt-in experimental feature, turn that experimental flag into a no-op or remove it entirely.
For example, `--extra-experimental-features=no-url-literals` becomes `--extra-deprecated-features=url-literals`.
For example, `--extra-experimental-featuresno-url-literals` becomes `--extra-deprecated-featuresurl-literals`.
3. Decide on a time frame for how long that feature will still be supported for backwards compatibility, and clearly communicate that in the error messages.
- Sometimes, automatic migration to alternatives is possible, and such should be provided if possible
- At least one NixOS release cycle should be the minimum
@@ -141,6 +141,21 @@ To inspect the canonical source of truth on what the state of the buildsystem co
$ meson introspect
```
#### LLD
The development shell on Linux uses LLD by default for faster link times.
This is set using `mesonFlags`, so to override it, you can simplify re-specify the linker to Meson:
```bash
$ just setup-custom -Dc_link_args=-fuse-ld=ld -Dcpp_link_args=-fuse-ld=ld
```
While using LLD, you may find it helpful to use ThinLTO for even further improvements to link times for incremental builds:
```bash
$ just setup-custom -Db_lto=true -Db_lto_mode=thin -Db_thinlto_cache=true
```
## Sending changes to Gerrit for review {#sending-to-gerrit}
We use Gerrit for all our code review in Lix.
@@ -170,6 +185,14 @@ Once you get a `Code-Review+2` vote on your change, it's rebased on `main` and C
If the change appears as "Rebase Required", you need to rebase it on `main` locally or via the Gerrit UI and wait for `Verified+1` before the Submit button is made active
The `Code-Review+2` from before will stick around through trivial rebases so no need to re-request review for a mere rebase.
## Interacting with the CI, Buildkite
We use Buildkite for our CI, usually you will not have to interact directly with it other than reviewing any errors it produces, which are linked from Gerrit.
However in certain cases a CI run will fail due to transient issues not related to your code and you will need to rerun it by hand.
You can log in to the CI via [SSO](https://buildkite.com/sso/lix-project). On your job you can then hit the "Retry failed" button to rerun it, normally you will not have a repeat of the transient issue.
If the build still fails on CI issues or all builds are failing this should be reported via [Zulip on #T-infra](https://zulip.lix.systems/#narrow/channel/7-T-infra) or [Matrix on #dev](https://matrix.to/#/%23dev%3Alix.systems?via=lix.systems).
## Building Lix with `nix`
To build a release version of Lix for the current operating system and CPU architecture:
@@ -62,6 +62,12 @@ For `installcheck` specifically, first run `just install` before running the tes
Finer-grained filtering within a test suite is also possible using the [--gtest_filter](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) command-line option to a test suite executable, or the `GTEST_FILTER` environment variable.
### Inspecting failures
The test suite emits logs in `build/meson-logs/`; the full textual failure logs are in `build/meson-logs/testlog.txt`.
If you want a much nicer experience of viewing the logs in a structured manner, use `xunit-viewer --results build/meson-logs/testlog.junit.xml --server` to view them in a web browser.
### Unit test support libraries
There are headers and code which are not just used to test the library in question, but also downstream libraries.
@@ -377,7 +383,10 @@ I grepped `lix/` for `get[eE]nv\("` to find the mentions in Lix code.
Overrides compile-time configuration of various locations used by Lix. See `lix/libstore/globals.cc`.
**Expected value**: a directory
- `NIX_DAEMON_SOCKET_PATH` (optional) - Overrides the daemon socket path from `$NIX_STATE_DIR/daemon-socket/socket`.
- `LIX_DAEMON_SOCKET_DIR` (optional) - Overrides the daemon socket directory from `$NIX_STATE_DIR/daemon-socket`.
**Expected value**: a directory
- `NIX_DAEMON_SOCKET_PATH` (optional) - Overrides the daemon socket path from `$NIX_STATE_DIR/daemon-socket/socket`. Ignored if `LIX_DAEMON_SOCKET_DIR` is set.
**Expected value**: path to a socket
- `NIX_LOG_FD` (output) - An FD number for logs in `internal-json` format to be sent to.
@@ -395,7 +404,6 @@ I grepped `lix/` for `get[eE]nv\("` to find the mentions in Lix code.
**Expected value**: the path to an executable shell
- `PRINT_PATH` - Undocumented. Used by `nix-prefetch-url` as an alternative form of `--print-path`. Why???
- `_NIX_IN_TEST` - If present with any value, makes `fetchClosure` accept file URLs in addition to HTTP ones. Why is this not `_NIX_FORCE_HTTP`??
Not used anywhere else.
- `NIX_ALLOW_EVAL` - Used by eval-cache tests to block evaluation if set to `0`.
Lix in Docker is very sensitive to **functional** DNS resolution if you are running with [Pasta protections](../advanced-topics/pasta.md) which are enabled by default since Lix 2.93.0 on most distributions.
If you notice failure to download things, double check whether your **first** DNS entry in `/etc/resolv.conf` is functional.
Lix with [Pasta protections](../advanced-topics/pasta.md) does not support failing over the next entries.
In Lix, only builds done on Linux with `sandbox = true` and a functioning
`pasta-path` are isolated from the rest of the system, all other builds are
considered non-isolated to some degree.
For example, running Lix with [Pasta](@docroot@/advanced-topics/pasta.md)
disabled makes the host network visible to fixed-output derivations, reducing
isolation somewhat.
## Clean termination of non-isolated builds
Non-isolated builds may not terminate cleanly in all cases due to limitations in Lix's process management.
This occurs when a build keeps the build log file descriptor open past the end of the actual build. A common cause of this are background tasks that aren't properly terminated before the main build process exits, for example: HTTP servers run as part of a test suite.
See [issue #1018](https://git.lix.systems/lix-project/lix/issues/1018) for an example.
The only solution is to manually terminate leftover processes in your derivation, including during failure scenarios.
FIXME(Lix): This section does not document the most common modern practices in terms of avoiding channels, pinning, declarative software installation (see flakey-profile or home-manager or NixOS), or using flakes, etc.
It is, however, likely correct at a technical level.
For more information on modern practices, see the [resources](https://lix.systems/resources) page on the Lix site.
For more information on modern practices, see the [resources](https://wiki.lix.systems/books/lix-users/page/nix-resources) page on the Lix site.
</div>
@@ -41,7 +41,7 @@ install Lix. If this is not the case for some reason, you can add it
- Remove support for daemon protocols before 2.18 [fj#510](https://git.lix.systems/lix-project/lix/issues/510) [cl/3249](https://gerrit.lix.systems/c/lix/+/3249)
Support for daemon wire protocols belonging to Nix 2.17 or older have been
removed. This impacts clients connecting to the local daemon socket or any
remote builder configured using the `ssh-ng` protocol. Builders configured
with the `ssh` protocol are still accessible from clients such as Nix 2.3.
Additionally Lix will not be able to connect to an old daemon locally, and
remote build connections to old daemons is likewise limited to `ssh` urls.
We have decided to take this step because the old protocols are very badly
tested (if at all), maintenance overhead is high, and a number of problems
with their design makes it infeasible to remain backwards compatible while
we move Lix to a more modern RPC mechanism with better versioning support.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Remove impure derivations and dynamic derivations [fj#815](https://git.lix.systems/lix-project/lix/issues/815) [cl/3210](https://gerrit.lix.systems/c/lix/+/3210)
The `impure-derivations` and `dynamic-derivations` experimental feature have
been removed.
New impure or dynamic derivations cannot be created from this point forward, and
any such pre-existing store derivations canot be read or built any more.
Derivation outputs created by building such a derivation are still valid
until garbage collected; existing store derivations can only be garbage
collected.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- First argument to `--arg`/`--argstr` must be a valid Nix identifier [fj#496](https://git.lix.systems/lix-project/lix/issues/496)
The first argument to `--arg`/`--argstr` must be a valid Nix identifier, i.e.
`nix-build --arg config.allowUnfree true` is now rejected.
This is because that invocation is a false friend since it doesn't set
The idea is to change the behavior to the latter in the long-term. For that,
non-identifiers started giving a warning since 2.92 and are now rejected to give people
who depend on that a chance to notice and potentially weigh in on the discussion.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- New cgroup delegation model [fj#537](https://git.lix.systems/lix-project/lix/issues/537) [fj#77](https://git.lix.systems/lix-project/lix/issues/77) [cl/3230](https://gerrit.lix.systems/c/lix/+/3230)
Builds using cgroups (i.e. `use-cgroups = true` and the experimental feature
`cgroups`) now always delegate a cgroup tree to the sandbox.
Compared to the original C++ Nix project, our delegation includes the
`subtree_control` file as well, which means that the sandbox can disable
certain controllers in its own cgroup tree.
This is a breaking change because this requires the Nix daemon to run with an
already delegated cgroup tree by the service manager.
## How to setup the cgroup tree with systemd?
systemd offers knobs to perform the required setup using:
```
[Service]
Delegate=yes
DelegateSubtree=supervisor
```
These directives are now included in our systemd packaging.
## What about using Nix as root without connecting to the daemon?
Builds run as `root` without connecting to the daemon relying on the cgroup
feature are now broken, i.e.
```console
# nix-build --use-cgroups --sandbox ... # will not work
If you need to disable cgroups temporarily, remember that you can do
`NIX_CONF='include /etc/nix/nix.conf\nuse-cgroups = false' nix-build ...` or
`nix-build --no-use-cgroups ...`.
## What about other service managers than systemd?
systemd has a [documentation](https://systemd.io/CGROUP_DELEGATION/) on how to
handle cgroup delegation from service management perspective.
If your service manager adheres to systemd semantics, e.g. writing an extended
attribute `user.delegate=1` on the delegated cgroup tree directory and moving
the `nix-daemon` process inside a cgroup tree to respect the inner process
rule, then, the feature will work as well.
## Why is the cgroup feature still experimental?
While the cgroup feature unlocks many use cases, its behavior and integration (e.g. user experience), especially at scale on build farms or in multi-tenant environments, are not yet fully matured. There’s also potential for deeper systemd integration (e.g. using slices and scopes) that has not been fully explored.
To avoid locking in an unstable interface, we’re keeping the experimental flag until we have validated the feature across a broader range of scenarios, including but not limited to:
* Nix as root
* Hydra-style build farms
* Forgejo CI runners
* Shared remote builders
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [eldritch horrors](https://git.lix.systems/pennae), and [lheckemann](https://git.lix.systems/lheckemann) for this.
- Enable high compress ratio zstd compression by default for binary caches uploads [fj#945](https://git.lix.systems/lix-project/lix/issues/945) [cl/4503](https://gerrit.lix.systems/c/lix/+/4503)
The default compression method for binary cache uploads has been switched from
[`xz`](https://github.com/tukaani-project/xz) to
[`zstd`](https://github.com/facebook/zstd) to address performance and usability
issues related to modern hardware and high-speed connections.
## Why?
`xz` offers compression ratios but is single-threaded in our implementation and
very slow (~10-20 Mbps in our test), preventing full utilization of 100Mbps+
connections and significantly slowing decompression for end users.
Lix is a "compress once, decompress many" application: build farms can afford
to spend more time compressing to achieve a faster download transfer for the
end user. More importantly, it matters that all end users spend the least
amount of time decompressing.
## What about compression ratios?
`zstd` cannot achieve the same peaks as `xz`, nonetheless, `zstd` compression
level has been increased to level 12 by default to balance compression ratio
and performance.
## Synthetic test case data
* **xz** (default compression level) on a 4.4GB file: ~632MB (77s)
* **zstd** (level 12) on the same file: ~775MB (18s), 18% larger but 50% faster
* **zstd** (level 14): ~773MB (37s)
* **zstd** (level 16): ~735MB (66s)
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [Raito Bezarius](https://git.lix.systems/raito) for this.
- Repl debugger uses `--ignore-try` by default [lix#666](https://git.lix.systems/lix-project/lix/issues/666) [cl/3488](https://gerrit.lix.systems/c/lix/+/3488)
Previously, using the debugger meant that exceptions thrown in `builtins.tryEval` would trigger the debugger.
However, this caught nixpkgs initialization code, which is unhelpful in the majority of cases, so we changed the default.
We consider this a breaking change since eval results *will* change if strings
with embedded NUL bytes were used, but we also consider the old behavior to be
not intentional (seeing how inconsistent it was) but merely fallout from a old
and misguided implementation decision to be worked around, not actually fixed.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Fixed output derivations can be run using `pasta` network isolation [fj#285](https://git.lix.systems/lix-project/lix/issues/285) [cl/3452](https://gerrit.lix.systems/c/lix/+/3452)
Fixed output derivations traditionally run in the host network namespace.
On Linux this allows such derivations to communicate with other sandboxes
or the host using the abstract Unix domains socket namespace; this hasn't
been unproblematic in the past and has been used in two distinct exploits
to break out of the sandbox. For this reason fixed output derivations can
now run in a network namespace (provided by [`pasta`]), restricted to TCP
and UDP communication with the rest of the world. When enabled this could
be a breaking change and we classify it as such, even though we don't yet
enable or require such isolation by default. We may enforce this in later
releases of Lix once we have sufficient confidence that breakage is rare.
[`pasta`]: https://passt.top/
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [puck](https://git.lix.systems/puck) for this.
- Function equality semantics are more consistent, but still bad [cl/4556](https://gerrit.lix.systems/c/lix/+/4556) [cl/4244](https://gerrit.lix.systems/c/lix/+/4244)
Lix has inherited a historic misfeature from CppNix in the form of pointer
equality checks built into the `==` operator. These checks were originally
meant to optimize comparison for large sets, but they have the unfortunate
side effect of producing unexpected results when sets containing functions
are compared. **Lix 2.93 and earlier** behave as shown in the repl session
```
Lix 2.93.3
Type :? for help.
nix-repl> f = x: x
Added f.
nix-repl> f == f
false
nix-repl> let s.f = f; in s.f == s.f
false
nix-repl> # however!
{ inherit f; } == { inherit f; }
true
nix-repl> [ f ] == [ f ]
true
nix-repl> # and, in another twist:
[ f ] == map f [ f ]
false
```
Nixpkgs relies on sets containing functions being comparable, so we cannot
simply deprecate this behavior. Due to changes to the object model used by
Lix ***all* comparisons above now evaluate to `true`**. This is considered
a breaking change because eval results may differ, but we also consider it
minor because the optimization is unsound (c.f. `let l = [NaN]; in l == l`
evaluates to `true` even though floating point `NaN` is incomparable). Lix
intends to remove this optimization altogether in the future, but until we
can do that we instead make it slightly less broken to allow other, *real*
optimizations. Function equality comparison remains **undefined behavior**
and should not be relied upon in Nixlang code that intends to be portable.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- `nix eval --write-to` has been removed [fj#974](https://git.lix.systems/lix-project/lix/issues/974) [fj#227](https://git.lix.systems/lix-project/lix/issues/227) [cl/4045](https://gerrit.lix.systems/c/lix/+/4045)
`nix eval --write-to` has been removed since it was underspecified, not widely
useful, and prone to security-sensitive misbehaviors. The feature was added in
Nix 2.4 purely for internal use in the build system. According to our research
it hasn't found any use outside of some distribution packaging scripts. Please
use structured outputs formats (such as JSON) instead as they have better type
fidelity, don't conflate attributes with paths, and are useful to other tools.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Remove the `parse-toml-timestamps` experimental feature
The `parse-toml-timestamps` experimental feature has been removed.
This feature used in‐band signalling to mark timestamps, making it
impossible to unambiguously parse TOML documents. It also exposed
implementation‐defined behaviour in the TOML specification that
changed in the toml11 parser library.
Any interface for parsing TOML timestamps suitable for future
stabilization would necessarily involve breaking changes, and there
is no evidence this experimental feature is being relied upon in the
wild, so it has been removed.
Many thanks to [Emily](https://git.lix.systems/emilazy) for this.
* Add `http3 = true` in your Nix configuration for permanent activation.
To disable it, use `--no-http3`.
**Note**:
* `--no-http2 --http3` will still enable both HTTP/2 and HTTP/3.
* `--http2 --http3` will prioritize HTTP/3 and fall back to HTTP/2 (and then
HTTP/1.1).
These are current CLI limitations. In the future, we plan to replace `--httpX`
options with `--max-http-version [1,2,3]` for easier version selection in Lix
transfers.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Add hyperlinks in attr set printing [cl/3790](https://gerrit.lix.systems/c/lix/+/3790)
The attribute set printer, such as is seen in `nix repl` or in type errors, now prints hyperlinks on each attribute name to its definition site if it is known.
Example: all of the attributes shown here are hyperlinks to the exact definition site of the attribute in question:
You're not alone. Thousands of Lix users suffer every day from excessive `builtins.toString` syndrome. It’s 2025, and we still have to cast integers to use them in strings.
To address this, Lix introduces the **`coerce-integers`** experimental feature. When enabled, interpolated integers within `"${...}"` are automatically coerced to strings. This allows writing:
To enable the feature, you need to add `coerce-integers` to your set of experimental features.
### Stabilization criteria
The `coerce-integers` feature is experimental and limited strictly to string interpolation (`"${...}"`). Before stabilization, the following must hold:
1. **Interpolation-only**
Coercion must not occur outside interpolation. Expressions like `"" + 42` must continue to fail.
2. **Expectation that no explicit cast are being observed**
Cases observing explicit coercion (e.g., via `tryEval` gadget or similar) are expected not to be load-bearing in actual production code.
### Timeline for stabilization
If the feature proves safe and is widely adopted across typical usage (e.g., actual configurations in the wild turning on the flag, non-trivial out-of-tree projects using it), the experimental flag will be removed **after six months of active use or two Lix releases**, whichever is longer.
This avoids locking the feature in experimental status indefinitely, as happened with Flakes, while allowing time for validation and ecosystem integration.
### What about coercing floats or more?
Coercion beyond integers -- such as for floats or other types -- is **not planned**, even under an experimental flag. Questions like "what is the canonical string representation of a float?" involve subtle and context-dependent trade-offs. Without a robust and principled mechanism to define and audit such behavior, introducing broader coercion risks setting unintended and hard-to-reverse precedents. The scope of `coerce-integers` is intentionally narrow and will remain so.
In terms of outlook, a proposal like https://git.lix.systems/lix-project/lix/issues/835 could pave the way for a better solution.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [delroth](https://github.com/delroth), [eldritch horrors](https://git.lix.systems/pennae), and [winter](https://git.lix.systems/winter) for this.
- nix-eval-jobs: support `--no-instantiate` flag [fj#987](https://git.lix.systems/lix-project/lix/issues/987)
`nix-eval-jobs` now supports a flag called `--no-instantiate`. With this enabled,
no write operations on the eval store are performed. That means, only evaluation is
performed, but derivations (and their gcroots) aren't created.
Many thanks to [mic92](https://github.com/mic92) and [ma27](https://git.lix.systems/ma27) for this.
## Improvements
- Assess current profile generations pointers in `nix doctor` [cl/3108](https://gerrit.lix.systems/c/lix/+/3108)
Added a new check to `nix doctor` that verifies whether the current generation of
a Nix profile can be resolved. This helps users diagnose issues with broken or
misconfigured profile symlinks.
This helps determining if you have broken symlinks or misconfigured packaging.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Improved susbtituter query speed
The code used to query substituters for derivations has been rewritten slightly
to take advantage of our asynchronous runtime. Such queries run for every build
that could download from substituters and processes every derivation that isn't
yet present on the local system. Previously Lix would use `http-connections` to
limit query concurrency, even for modern caches that support HTTP/2 and have no
limit on how many queries can be run concurrently on one single connection. Lix
no longer does this, resulting in approximately 60% reduction in query time for
medium-sized closures (e.g. NixOS system closures) during testing, although the
exact number depends greatly on local network latency and generally improves as
latency increases. Unlike previously setting `http-connections` to `1` or other
low values no longer brings a massive penalty in query performance if the cache
in use by the querying system supports HTTP/2 (as e.g. `cache.nixos.org` does).
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
Hitting Control-C or sending `SIGINT` to Lix now prints an informational message
if it is still running after on second, the second Control-C/`SIGINT` terminates
Lix immediately without waiting for any shutdown code to finish running. Lix did
not treat the second such event differently from first in the past; this made it
impossible to easily terminate running Lix processes that got stuck in e.g. very
expensive Nixlang code that never interacted with the store. We now terminate as
soon as the user hits Control-C again without waiting any more, to much the same
effect as putting Lix into the background and killing it immediately afterwards.
This means you can now more conveniently break out of stuck Nixlang evaluations:
```
❯ nix-instantiate --eval --expr 'let f = n: if n == 0 then 0 else f (n - 1) + f (n - 1); in f 32'
^CStill shutting down. Press ^C again to abort all operations immediately.
^C
❌130 ❯
```
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- `--keep-failed` chowns the build directory to the user that request the build
Running a build with `--keep-failed` now chowns the temporary directory from the
builder user and group to the user that request the build if the build came from
a local user connected to the daemon. This makes inspecting failed derivations a
lot easier. On Linux the build directory made visible to the user will not be in
the same path as it was in the sandbox and continuing builds will usually break.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Better debuggability on fixed-output hash mismatches
Fixed-output derivation hash mismatch error messages will now include the path that was
produced unexpectedly, and this path will be registered as valid even if `--check`
(`nix-store`, `nix-build`) or `--rebuild` (`nix build`) was passed. This makes comparing
the expected path with the obtained path easier, and is useful for debugging when
upstreams modify previously-published releases or when changes in fixed-output
derivations' dependencies affect their output unexpectedly.
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Add --raw flag to `nix-instantiate --eval` for unescaped output [gh#12119](https://github.com/NixOS/nix/pull/12119) [cl/2886](https://gerrit.lix.systems/c/lix/+/2886)
The `nix-instantiate --eval` command now supports a `--raw` flag. When used,
the result must be coercible to a string (as with `${...}`) and is printed
verbatim, without quotes or escaping.
Many thanks to [Martin Fischer](https://github.com/not-my-profile), [infinisil](https://github.com/infinisil), and [Raito Bezarius](https://git.lix.systems/raito) for this.
- Allow `nix store ls` to read nar listings from binary cache stores. [cl/3225](https://gerrit.lix.systems/c/lix/+/3225)
The `nix store ls` command now supports reading `.ls` nar listings from binary cache stores.
If a listing is detected for the store path being queried, the nar is no longer downloaded.
These nar listings are available in binary cache stores where the `write-nar-listing` option is
enabled, such as cache.nixos.org.
Many thanks to [Victor Fuentes](https://git.lix.systems/vlinkz) for this.
- show tree with references that lead to an output cycle [fj#551](https://git.lix.systems/lix-project/lix/issues/551)
When Lix determines a cyclic dependency between several outputs of a derivation,
it now displays which files in which outputs lead to an output cycle:
```
error: cycle detected in build of '/nix/store/gc5h2whz3rylpf34n99nswvqgkjkigmy-demo.drv' in the references of output 'bar' from output 'foo'.
Shown below are the files inside the outputs leading to the cycle:
Please note that showing the files and its contents while displaying the cycles only works
on Linux.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- Lix now enables parallel marking in boehm-gc [fj#983](https://git.lix.systems/lix-project/lix/issues/983) [cl/3880](https://gerrit.lix.systems/c/lix/+/3880)
This brings a fairly modest performance improvement (~38% for `nixpkgs search hello`) to evaluation, especially in scenarios that necessitate larger heap sizes.
Many thanks to [Eelco Dolstra](https://github.com/edolstra) and [Seth Flynn](https://git.lix.systems/getchoo) for this.
- `disallowedRequisites` now reports chains of disallowed requisites [fj#334](https://git.lix.systems/lix-project/lix/issues/334) [fj#626](https://git.lix.systems/lix-project/lix/issues/626) [gh#10877](https://github.com/NixOS/nix/issues/10877)
When a build fails because of [`disallowedRequisites`](@docroot@/language/advanced-attributes.md#adv-attr-disallowedRequisites), the error message now includes the chain of references that led to the failure. This makes it easier to see in which derivations the chain can be broken, to resolve the problem.
Example:
```
$ nix-build -A hello
error: output '/nix/store/0b7k85gg5r28gb54px9nq7iv5986mns9-hello-2.12.2' is not allowed to refer to the following paths:
Many thanks to [ma27](https://git.lix.systems/ma27) and [Robert Hensing](https://github.com/roberth) for this.
- Stack traces now summarize involved derivations at the bottom [cl/4493](https://gerrit.lix.systems/c/lix/+/4493)
When evaluation errors and a stack trace is printed,
For example, if I add Nheko to a NixOS `environment.systemPackages` without adding `olm-3.2.16` `nixpkgs.config.permittedInsecurePackages`, then without `--show-trace`, I previously got this:
```
error:
… while calling the 'head' builtin
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/attrsets.nix:1701:13:
1700| if length values == 1 || pred here (elemAt values 1) (head values) then
1701| head values
| ^
1702| else
… while evaluating the attribute 'value'
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/modules.nix:1118:7:
1117| // {
1118| value = addErrorContext "while evaluating the option `${showOption loc}':" value;
| ^
1119| inherit (res.defsFinal') highestPrio;
(stack trace truncated; use '--show-trace' to show the full trace)
error: Package ‘olm-3.2.16’ in /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/pkgs/by-name/ol/olm/package.nix:37 is marked as insecure, refusing to evaluate.
< -snip the whole explanation about olm's CVEs- >
```
This doesn't tell me anything about where `olm-3.2.16` came from.
With `--show-trace`, there's 1155 lines to sift through, but does contain lines like "while evaluating derivation 'nheko-0.12.1'".
With this change, those lines are summarized and collected at the bottom, regardless of `--show-trace`:
```
error:
… while calling the 'head' builtin
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/attrsets.nix:1701:13:
1700| if length values == 1 || pred here (elemAt values 1) (head values) then
1701| head values
| ^
1702| else
… while evaluating the attribute 'value'
at /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/lib/modules.nix:1118:7:
1117| // {
1118| value = addErrorContext "while evaluating the option `${showOption loc}':" value;
| ^
1119| inherit (res.defsFinal') highestPrio;
(stack trace truncated; use '--show-trace' to show the full trace)
error: Package ‘olm-3.2.16’ in /nix/store/9v6qa656sq3xc58vkxslqy646p0ajj61-source/pkgs/by-name/ol/olm/package.nix:37 is marked as insecure, refusing to evaluate.
< -snip the whole explanation about olm's CVEs- >
note: trace involved the following derivations:
derivation 'etc'
derivation 'dbus-1'
derivation 'system-path'
derivation 'nheko-0.12.1'
derivation 'mtxclient-0.10.1'
```
Now we finally know that olm was evaluated because of Nheko, without sifting through *thousands* of lines of error message.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- Symbols reuses once-allocated Value to reduce garbage collected allocations [cl/3308](https://gerrit.lix.systems/c/lix/+/3308) [cl/3300](https://gerrit.lix.systems/c/lix/+/3300) [cl/3314](https://gerrit.lix.systems/c/lix/+/3314) [cl/3310](https://gerrit.lix.systems/c/lix/+/3310) [cl/3312](https://gerrit.lix.systems/c/lix/+/3312) [cl/3313](https://gerrit.lix.systems/c/lix/+/3313)
In the Lix evaluator, **symbols** represent immutable strings, like those used
for attribute names.
In evaluator design, such strings are typically [**interned**](https://en.wikipedia.org/wiki/String_interning), stored uniquely
to save memory, and Lix inherits this approach from the original C++ codebase.
However, some builtins, like `builtins.attrNames`, must return a `Value` type
that can represent any Nix value (strings, integers, lists, etc.).
Before this change, these builtins would create lists of `Value` objects by
allocating them through the garbage collector, copying the symbol’s string
content each time.
This allocation is unnecessary if the interned symbols themselves also hold a
`Value` representation allocated outside the garbage collector, since these
live for the full duration of evaluation.
As a result, this reduces the number of allocations, leading to:
* A significant drop in maximum [resident set memory](https://en.wikipedia.org/wiki/Resident_set_size) (RSS), with some large-scale
tests showing up to 11% (about 500 MiB) savings in large colmena deployments.
* A slight decrease in CPU usage during Nix evaluations.
This change is inspired by https://github.com/NixOS/nix/pull/13258 but the approach is different.
**Note** : [`xokdvium`](https://github.com/xokdvium) is the rightful author of https://gerrit.lix.systems/c/lix/+/3300 and the credit was missed on our end during the development process. We are deeply sorry for this mistake.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [eldritch horrors](https://git.lix.systems/pennae), [Tom Hubrecht](https://git.lix.systems/tom-hubrecht), [xokdvium](https://github.com/xokdvium), and [NaN-git](https://github.com/NaN-git) for this.
## Fixes
- `build-dir` no longer defaults to `temp-dir` [cl/3453](https://gerrit.lix.systems/c/lix/+/3453)
The directory in which temporary build directories are created no longer defaults
to the value of the `temp-dir` setting to avoid builders making their directories
world-accessible. This behavior has been used to escape the build sandbox and can
cause build impurities even when not used maliciously. We now default to `builds`
in `NIX_STATE_DIR` (which is `/nix/var/nix/b` in the default configuration).
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Global certificate authorities are copied inside the builder's environment [gh#12698](https://github.com/NixOS/nix/issues/12698) [fj#885](https://git.lix.systems/lix-project/lix/issues/885) [cl/3765](https://gerrit.lix.systems/c/lix/+/3765)
Previously, CA certificates were only installed at
`/etc/ssl/certs/ca-certificates.crt` for sandboxed builds on Linux.
This setup was insufficient in light of recent changes in `nixpkgs`, which now
enforce HTTPS usage for `fetchurl`, even for fixed-output derivations, to
mitigate confidentiality risks such as `netrc` or credentials leakage.
`nixpkgs` still make use of a special package called `cacerts` which contains a
copy of the CA certificates maintained by Nixpkgs and added as a reference for
TLS-enabled fetchers.
As a result, having a consistent and trusted certificate authority in all
builder environments is becoming more essential.
On `nix-darwin`, the `NIX_SSL_CERT_FILE` environment variable is always
explicitly defined, but it is ignored by the sandbox setup.
Simultaneously, Nix evaluates and propagates impure environment variables via
`lib.proxyImpureEnvVars`, meaning that if `NIX_SSL_CERT_FILE` is set (which
influences the default value for `ssl-cert-file`), it will be forwarded
unchanged into the builder environment.
However, on Linux, Nix also *copies* the CA file into the sandbox, creating a
discrepancy between the value of `NIX_SSL_CERT_FILE` and the actual trusted
certificate path used during the build.
This divergence caused confusion and was partially addressed by attempts to
whitelist the CA path in the Darwin sandbox (see cl/2906), but that approach
involved a non-trivial path canonicalization step and is not as general as this one.
To address this properly, we now emit a warning and override
`NIX_SSL_CERT_FILE` inside the builder, explicitly pointing it to the CA file
copied into the sandbox.
This eliminates ambiguity between `NIX_SSL_CERT_FILE`
and `ssl-cert-file`, ensuring consistent trust anchors across platforms.
This warning might become a hard error as we figure out what to do regarding
`lib.proxyImpureEnvVars` in nixpkgs.
The behavior has been verified across sandboxed and unsandboxed builds on both
Linux and Darwin.
As a consequence of this change, approximately 500KB of CA certificate data is
now unconditionally copied into the build directory for fixed-output
derivations.
While this ensures consistent trust verification without having to restart the
daemon after system upgrades, it may introduce a slight overhead in build
performance. At present, no optimizations have been implemented to avoid this
copy, but if this overhead proves noticeable in your workflows, please open an
issue so we can evaluate and possibly implement different strategies to render
trust anchors visible.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [Emily](https://git.lix.systems/emilazy) for this.
- libstore: exponential backoff for downloads [lix#932](https://git.lix.systems/lix-project/lix/issues/932) [cl/3856](https://gerrit.lix.systems/c/lix/+/3856)
The connection timeout when downloading from e.g. a binary cache is exponentially
increased per failure. The option `connect-timeout` is now an alias to `max-connect-timeout`
which is the maximum value for a timeout. The start value is controlled
by `initial-connect-timeout` which is `5` by default.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- Fix develop shells for derivations with escape codes [fj#991](https://git.lix.systems/lix-project/lix/issues/991) [cl/4154](https://gerrit.lix.systems/c/lix/+/4154) [cl/4155](https://gerrit.lix.systems/c/lix/+/4155)
ASCII control characters (including `\e`, used for ANSI escape codes) in derivation variables are now correctly escaped for `nix develop` and `nix print-dev-env`, instead of erroring.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
To find out why, use nix-store --query --roots and nix-store --query --referrers.
```
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
- Fallback to safe temp dir when build-dir is unwritable [fj#876](https://git.lix.systems/lix-project/lix/issues/876) [cl/3501](https://gerrit.lix.systems/c/lix/+/3501)
Non-daemon builds started failing with a permission error after introducing the `build-dir` option:
1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
2. The user lacks permissions for that path.
We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
**Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Parse overflowing JSON number literals as floating‐point [cl/3919](https://gerrit.lix.systems/c/lix/+/3919)
Previously, `builtins.fromJSON "-9223372036854775809"` would
return a floating‐point number, while `builtins.fromJSON
"9223372036854775808"` would cause an evaluation error. This was
introduced with the banning of integer overflow in Lix 2.91; previously
the latter would result in C++ undefined behaviour. These cases are
now treated consistently with JSON’s model of a single numeric type,
and JSON number literals that do not fit in a Nix‐language integer
will be parsed as floating‐point numbers.
Many thanks to [Emily](https://git.lix.systems/emilazy) for this.
- Fix handling of OSC codes in terminal output [fj#160](https://git.lix.systems/lix-project/lix/issues/160) [cl/3143](https://gerrit.lix.systems/c/lix/+/3143)
OSC codes in terminal output are now handled correctly, where OSC 8 (hyperlink) is preserved any
time color codes are allowed and all other OSC codes are stripped out. This applies not only to
output from build commands but also to rendered documentation in the REPL.
Many thanks to [lilyball](https://git.lix.systems/lilyball) for this.
- Fix nix develop for derivations that rejects dependencies with structured attrs [fj#997](https://git.lix.systems/lix-project/lix/issues/997) [cl/4182](https://gerrit.lix.systems/c/lix/+/4182)
For the sake of concision, we refer to `disallowedReferences` in what follows,
but all output checks were equally fixed:
`{dis,}allowed{References,Requisites}`.
Derivations can define *output checks* to reject unwanted dependencies, such as
interpreters like `bash` or compilers like `gcc`. This can be done in two ways:
* **Legacy style**: `disallowedReferences = [ ... ]` in the environment.
`nix-eval-jobs` doesn't clear the `NIX_PATH` from the environment anymore. This matches the behavior
of [upstream version `2.30`](https://github.com/nix-community/nix-eval-jobs/releases/tag/v2.30.0).
Many thanks to [ma27](https://git.lix.systems/ma27) and [mic92](https://github.com/mic92) for this.
- Remove reliance on Bash for remote stores via SSH [fj#830](https://git.lix.systems/lix-project/lix/issues/830) [fj#805](https://git.lix.systems/lix-project/lix/issues/805) [fj#304](https://git.lix.systems/lix-project/lix/issues/304) [cl/3159](https://gerrit.lix.systems/c/lix/+/3159)
The pre-flight `echo started` handshake -- added years ago to catch race conditions -- has been removed.
After removal of connection sharing in Lix 2.93, it required a Bash-compatible shell and a standard `echo`, so it failed on:
* builders protected by `ForceCommand` wrappers (e.g. `nix-remote-build`),
* BusyBox / initrd images with no Bash,
* hosts using non-POSIX shells such as Nushell.
The race the probe once addressed was tied to SSH connection-sharing -- since connection-sharing code has already been removed, the probe is now pointless.
Real connection or protocol errors are now left to SSH/Nix to report directly.
This is technically a breaking change if you had scripts that relied on the literal "started" which needs to be updated to rely on other signals, e.g., exit codes.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- repl-overlays now work in the debugger for flakes [fj#777](https://git.lix.systems/lix-project/lix/issues/777) [cl/3398](https://gerrit.lix.systems/c/lix/+/3398)
Due to a bug, it was previously not possible to use the debugger on flakes with repl-overlays, or with pure evaluation in general:
```
$ nix repl --pure-eval
Lix 2.94.0-dev-pre20250617-87d99da
Type :? for help.
Loading 'repl-overlays'...
error: access to absolute path '/Users/jade/.config/nix/repl.nix' is forbidden in pure eval mode (use '--impure' to override)
```
This is now fixed.
The contents of the repl-overlays file itself (i.e. most typically the top level lambda in it) will be evaluated in impure mode.
It may be necessary to use `builtins.seq` to force the impure operations to happen first if one wants to do impure operations inside a repl-overlays file in pure evaluation mode.
Many thanks to [jade](https://git.lix.systems/jade) for this.
- `nix-shell` default shell directory is not `/tmp` anymore for `$NIX_BUILD_TOP` [fj#940](https://git.lix.systems/lix-project/lix/issues/940)
Previously, Lix `nix-shell`s could exit non-zero status when `stdenv`'s `dumpVars` phase failed to write to `$NIX_BUILD_TOP/env-vars`, despite `dumpVars` being intended as a debugging aid.
This happens when `TMPDIR` is not set and defaults therefore to `/tmp`, resulting in a `/tmp/env-vars` global file that every `nix-shell` wants to write.
We fix this issue by reusing a pre-created, unique, and writable location, as the build top directory, avoiding shell exiting from write failures silently.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
When using e.g. [Snix's nar-bridge](https://snix.dev/docs/components/overview/#nar-bridge) via
an `http`-store, Lix would create cache entries with a wrong URL to the NAR when uploading
a store-path.
This caused hard build failures for Hydra.
Lix doesn't create these entries on upload anymore. Instead, it only removes negative cache entries.
The cache entry for a narinfo is now created the first time, Lix queries the cache
for the previously uploaded store-path again.
Many thanks to [ma27](https://git.lix.systems/ma27) for this.
- Lix libraries can now be linked statically [fj#789](https://git.lix.systems/lix-project/lix/issues/789) [cl/3775](https://gerrit.lix.systems/c/lix/+/3775) [cl/3778](https://gerrit.lix.systems/c/lix/+/3778)
Previously the pkg-config files distributed with Lix were only suitable for dynamic linkage, causing "undefined reference to…" linker errors when trying to link statically.
Private dependency information has now been added to make static linkage work as expected without user intervention.
In addition, relevant static libraries are now prelinked to avoid strange failures due to missing static initializers.
Many thanks to [alois31](https://git.lix.systems/alois31) for this.
- add description to zsh completions [fj#910](https://git.lix.systems/lix-project/lix/issues/910) [cl/3632](https://gerrit.lix.systems/c/lix/+/3632)
Emit descriptions when completing args in zsh completions. This uses the descriptions we already
provided in NIX\_GET\_COMPLETIONS.
Many thanks to [matthewbauer](https://github.com/matthewbauer) for this.
## Miscellany
- Deprecation of CA derivations, dynamic derivations, and impure derivations [fj#815](https://git.lix.systems/lix-project/lix/issues/815)
Content-addressed derivations are now deprecated and slated for removal in Lix 2.94.
We're doing this because the CA derivation system has been a known cause of problems
and inconsistencies, is unmaintained, habitually makes improving the store code very
difficult (or blocks such improvements outright), and is beset by a number of design
flaws that in our opinion cannot be fixed without a full reimplementation from zero.
Dynamic derivations and impure derivations are built on the CA derivation framework,
and owing to this they too are deprecated and slated for removal in another release.
- Fix upgrade-nix breaking its own access to the daemon [lix#1189](https://git.lix.systems/lix-project/lix/issues/1189) [lix#1207](https://git.lix.systems/lix-project/lix/issues/1207) [cl/5504](https://gerrit.lix.systems/c/lix/+/5504) [cl/5567](https://gerrit.lix.systems/c/lix/+/5567)
`nix upgrade-nix`, and the helper script `misc/upgrade-lix.sh` now pass `--store local` to all Nix commands, so the upgrade process can make changes to the daemon without breaking further steps in the upgrade.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
# Lix 2.95.2 (2026-05-04)
## Fixes
- Fix unsigned overflow leading to out-of-band write in the NAR parser [cl/5550](https://gerrit.lix.systems/c/lix/+/5550)
The NAR parser contained an unsigned integer overflow that could be used by an
attacker to write arbitrary data to an unknown memory location and possibly
achieve code execution. A successful attack on the system-wide Lix daemon
could lead to privilege escalation to root. Any process that involves NAR
serialization could trigger this issue, including (but not limited to)
- local user interaction, whether the users are trusted or untrusted
- malicious substituters sending malformed NARs
- remote builders sending malformed build results
- remote daemons sending malformed inputs when requesting remote builds
Successful attacks using this bug require ASLR weakening of some sort, whether
by architecture constraints (e.g. on 32 bit systems, where little randomization
is possible) or system configuration (e.g. low ASLR entropy when loading
libraries), and millions of attempts. Local attacks can be mounted in less than
an hour. Remote builds typically require a fresh SSH connection for each build
and are thus less susceptible. Only one attempt can be made by substituters for
every build using substituters, they are thus not a likely vector for attacks.
At the time of writing, MITRE has not assigned this a CVE yet.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae), [Raito Bezarius](https://git.lix.systems/raito), [edef](https://github.com/edef1c), and [sandydoo](https://github.com/sandydoo) for this.
Static builds using musl were broken in 2.95.0 and should work again now.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- flake config warnings are now printed to stderr [lix#1155](https://git.lix.systems/lix-project/lix/issues/1155) [cl/5379](https://gerrit.lix.systems/c/lix/+/5379)
The settings listed in a flake-config confirmation prompt are now printed to stderr rather than stdout, which allows `nix print-dev-env` to emit valid bash again even in the presence of untrusted settings.
Many thanks to [lheckemann](https://git.lix.systems/lheckemann) for this.
# Lix 2.95.0 (2026-03-13)
## Breaking Changes
- Deprecate shadowing internal files through the Nix search path [lix#998](https://git.lix.systems/lix-project/lix/issues/998) [cl/4632](https://gerrit.lix.systems/c/lix/+/4632)
As Lix uses the path `<nix/fetchurl.nix>` for bootstrapping purposes, the ability to shadow it by adding `nix=/some/path` (or `/other/path` that contains a `nix` directory) to the search path is not desirable.
To alleviate potential issues, Lix now emits a warning when the Nix search path contains potential shadows for internal files, which will be changed to an error in a future release.
The warning can be disabled by enabling the deprecated feature `nix-path-shadow`.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- More deprecated features [cl/2092](https://gerrit.lix.systems/c/lix/+/2092) [cl/2310](https://gerrit.lix.systems/c/lix/+/2310) [cl/2311](https://gerrit.lix.systems/c/lix/+/2311) [cl/4638](https://gerrit.lix.systems/c/lix/+/4638) [cl/4652](https://gerrit.lix.systems/c/lix/+/4652) [cl/4764](https://gerrit.lix.systems/c/lix/+/4764)
This release cycle features a new batch of deprecated (anti-)features.
You can opt in into the old behavior with `--extra-deprecated-features` or any equivalent configuration option.
-`broken-string-indentation` indented strings (those starting with `''`) might produce unintended results due to how the whitespace stripping is done. Those cases will now warn the user.
-`broken-string-escape` "escaped" characters without a properly defined escape sequence evaluate to "themselves". This is in most cases unintended behaviour, both for writing regexes, and using legacy or uncommon escape sequences like `\f`. The user will now be warned, if those are present.
-`floating-without-zero` so far, one was able to declare a float using something like `.123`. This can cause confusion about accessing attributes. Floating point numbers must now always include the leading zero, i.e. `0.123`
-`rec-set-merges` Attribute sets like `{ foo = {}; foo.bar = 42;}` implicitly merge at parse time, however if one of them is marked as recursive but not the others then the recursive attribute may get lost (order-dependent). Therefore, merging attrs with mixed-`rec` is now forbidden.
-`rec-set-dynamic-attrs` Dynamic attributes have weird semantics in the presence of recursive attrsets (they evaluate *after* the rest of the set). This is now forbidden.
-`or-as-identifier``or` as an identifier has always been weird since the `or` (almost-)keyword has been introduced. We are deprecating the backcompat hacks from the early days of Nix in favor of making `or` a full and proper keyword.
-`tokens-no-whitespace` Function applications without space around the arguments like `0a`, `0.00.0` or `foo"1"2` are now forbidden. The same applies to list elements. The primary reason for this deprecation is to remove foot guns around surprising tokenization rules regarding number literals, but this will also free up some syntax for other purposes (e.g. `r""` strings) for reuse at some point in the future.
-`shadow-internal-symbols` has been expanded to also forbid shadowing `null`, `true` and `false`.
-`ancient-let` deprecation has been turned into a full parser error instead of a warning.
-`rec-set-overrides` deprecation has been turned into a full parser error instead of a warning.
Many thanks to [piegames](https://git.lix.systems/piegames), [rootile (Rutile)](https://git.lix.systems/rootile), and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Move `/root/.cache/nix` to `/var/cache/nix` by default [lix#634](https://git.lix.systems/lix-project/lix/issues/634) [cl/4671](https://gerrit.lix.systems/c/lix/+/4671)
By default, Lix attempts to locate a cache directory for its operations (such
as the narinfo cache) by checking the value of `$XDG_CACHE_DIR`.
However, since the Nix daemon is a system service, using `$XDG_CACHE_DIR` is
not typical in this context.
To address this, systemd provides a better solution. Specifically, when
`CacheDirectory=` is set in the `[Service]` section of a systemd unit, it
automatically sets the `$CACHE_DIRECTORY` environment variable and systemd will
manage that cache directory for us.
Now, our systemd unit includes `CacheDirectory=nix`, which sets the
`$CACHE_DIRECTORY` and takes precedence over `$XDG_CACHE_DIR`.
If the daemon is run under user units, systemd will automatically set
`$XDG_CACHE_DIR`.
If neither of these variables is set, Lix falls back to its default behavior.
By default, Lix will try to find a cache directory for its various operations
(e.g. narinfo cache) by looking into `$XDG_CACHE_DIR`.
In summary, what was stored in `/root/.cache/nix` is now moved to
`/var/cache/nix/nix`.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
The `fetch-closure` experimental feature has been removed.
Outside of allowing the user to import closure from binary cache,
`fetchClosure` also allowed you to do the following:
* rewrite non-CA path to CA
* reject non-CA paths at fetching time
* reject CA paths at fetching time
Some people are using those mechanism to prevent users from having to build any
package and force going via the declared cache or as a way to use ancient/old
software without paying the evaluation cost of a second nixpkgs.
Both use cases are somewhat of an antipattern in Nix semantics. If the user
cannot fetch a program directly via the substituter mechanism and fall back to
local build, this is a feature *and* a misconfiguration. If the user cannot build
certain derivations because they are too expensive, the build directives should
pass `-j0` or similar.
As for the second usecase, there's a different way to do it that also allows to
have a way to reproduce the paths that are hardcoded in that file, perform
`import (fetchurl "https://my-cache/${hashparts storepath}.drv")` rather, i.e.
an IFD to a possibly well known name. The backend can generate them on the fly
or once, and possess stable names.
Finally, as for the non-CA → CA features, Lix removed ca-derivations.
fetchClosure offers ca-derivations-like features which suffers from similar
shortcomings albeit lessened. It only follows that we should deprecate
and remove these capabilities.
Many thanks to [just1602](https://git.lix.systems/just1602) for this.
## Features
-`nix store add-path` now supports references [cl/5205](https://gerrit.lix.systems/c/lix/+/5205)
Lix supports two categories of hashes in store paths: input-addressed and output-addressed.
Currently, in Nix language, there is no way to produce output-addressed paths with references, as fixed-output derivations forbid references.
However, the Nix store actually *supports* references in output-addressed paths.
This is very useful for importing build products created outside of Lix that reference dependency store paths since such build products have no associated derivation so don't make any sense to input-address.
Previously, output-addressed paths with references could only be created by writing a custom client to the rather-baroque Nix daemon protocol; now it's available in the CLI.
Using `nix store add-path --references-list-json REFS_LIST_FILE SOME_PATH` with a JSON list of string store paths, you can now create such paths with the Lix CLI.
They may be consumed from Nix language using something like `builtins.storePath` or the following which also works in pure evaluation mode:
```nix
# Hack from https://git.lix.systems/lix-project/lix/issues/402#issuecomment-5889
path:
builtins.appendContext path {
${path} = {
path = true;
};
}
```
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Add `builtins.warn` for emitting warnings from Nix code [cl/2248](https://gerrit.lix.systems/c/lix/+/2248)
Lix now has a builtin function for emitting warnings.
Like `builtins.trace`, it takes two arguments: the message to emit, and the expression to return.
_Unlike_ `builtins.trace`, `builtins.warn` requires the first argument — the message — to be a string.
In the future we may extend `builtins.warn` to accept a more structured API.
To go along with this, we also have two new config settings:
- [`debugger-on-warn`](@docroot@/command-ref/conf-file.md#conf-debugger-on-warn), which, when used with `--debugger`, makes `builtins.warn` also function like [`builtins.break`](@docroot@/language/builtins.md#builtins-break).
- [`abort-on-warn`](@docroot@/command-ref/conf-file.md#conf-abort-on-warn), which aborts evaluation entirely after the warning is emitted.
Many thanks to [Emilia Bopp](https://git.lix.systems/milibopp) and [Qyriad](https://git.lix.systems/Qyriad) for this.
- `keep-env-derivations` is now supported for nix3 CLI (`nix profile`) [lix#1095](https://git.lix.systems/lix-project/lix/issues/1095) [cl/5332](https://gerrit.lix.systems/c/lix/+/5332)
The `keep-env-derivations` feature is now available for `nix profile`. This allows users to prevent the garbage collection of derivations used to install a profile, even when `keep-derivations = false` (set to `true` by default).
Previously, `nix-env` supported this feature, but `nix profile` **never** did. This caused issues when garbage collection removed the associated `.drv` files, which are required, for example, by vulnerability management tools (e.g. [vulnix](https://github.com/nix-community/vulnix)) for proper operation.
This issue has now been resolved.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Make `log-format` a setting [cl/4686](https://gerrit.lix.systems/c/lix/+/4686)
The [`--log-format` CLI option](@docroot@/command-ref/opt-common.md#opt-log-format) can now be set in [`nix.conf`](@docroot@/command-ref/conf-file.md#conf-log-format)!
For example, you can now persistently enable the `multiline-with-logs` log format [added in Lix 2.91](@docroot@/release-notes/rl-2.91.md) by adding the following to your `nix.conf`:
```conf
log-format = multiline-with-logs
```
Or the equivalent in a NixOS configuration:
```nix
{
nix.settings.log-format = "multiline-with-logs";
}
```
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- Allow remote builders to be configured using TOML [cl/4533](https://gerrit.lix.systems/c/lix/+/4533)
Lix now supports configuring remote builders using a TOML file instead of the old, very cursed and incomprehensible format.
This comes with not only a human-understandable file, but also with better messages and error reports on misconfiguration.
A more detailed Documentation can be found on the [distributed-builds](@docroot@/advanced-topics/distributed-builds.md) documentation page.
Many thanks to [rootile (Rutile)](https://git.lix.systems/rootile) and [Qyriad](https://git.lix.systems/Qyriad) for this.
- Emit warnings when encountering IFD with `warn-import-from-derivation` [nix#13279](https://github.com/NixOS/nix/pull/13279) [cl/3879](https://gerrit.lix.systems/c/lix/+/3879)
Instead of only being able to toggle the use of [Import from
Derivation](https://nix.dev/manual/nix/stable/language/import-from-derivation) with
`allow-import-from-derivation`, Lix is now able to warn users whenever IFD is encountered with
`warn-import-from-derivation`.
Many thanks to [Seth Flynn](https://git.lix.systems/getchoo), [gustavderdrache](https://github.com/gustavderdrache), and [Eelco Dolstra](https://github.com/edolstra) for this.
## Improvements
- Collect Flakes untrusted settings into one prompt [lix#682](https://git.lix.systems/lix-project/lix/issues/682) [cl/2921](https://gerrit.lix.systems/c/lix/+/2921)
When working with Flakes containing untrusted settings, a prompt is shown for each setting, asking whether to vet or approve it. This looks like:
```
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all)
```
In Flakes with a large number of settings to approve or reject, this process can become tedious as each option must be handled individually.
To address this, all untrusted settings are now consolidated into a single prompt: allowing for bulk acceptance permanently or not, rejection, or detailed review. For example:
### Scrutiny scenario
```console
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all) n
warning: you can set 'accept-flake-config' to 'false' to automatically reject configuration options supplied by flakes
Do you want to allow setting 'allow-dirty = false'? (yes for now/Allow always/no for now) y
Do you want to allow setting 'sandbox = false'? (yes for now/Allow always/no for now) n
```
### Reject everything scenario
```console
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all) N
Rejecting all untrusted nix.conf entries
warning: you can set 'accept-flake-config' to 'false' to automatically reject configuration options supplied by flakes
```
### Accept everything scenario
```console
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all) y
```
### Accept everything PERMANENTLY scenario
Note that accepting everything permanently will authorize these options for any
further operations.
The file containing this trust information is usually located in
`~/.local/share/nix/trusted-settings.json` and can be edited manually to revoke
this permission until Lix provides a first-class command for this manipulation.
```console
❯ nix flake lock
warning: ignoring untrusted flake configuration setting 'allow-dirty', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
warning: ignoring untrusted flake configuration setting 'sandbox', pass '--accept-flake-config' to trust it (may allow the flake to gain root, see the nix.conf manual page)
The following settings require your decision:
- allow-dirty = false
- sandbox = false
Do you want to allow configuration settings to be applied?
This may allow the flake to gain root, see the nix.conf manual page (yes for now/Allow always/no/No to all) A
```
Many thanks to [isabelroses](https://git.lix.systems/isabelroses), [Raito Bezarius](https://git.lix.systems/raito), and [eldritch horrors](https://git.lix.systems/pennae) for this.
- `--check` or `--rebuild` is clearer about a missing path [lix#485](https://git.lix.systems/lix-project/lix/issues/485)
Previously, when running Lix with --check or --rebuild, failures often surfaced
as an unhelpful error:
> "some outputs of '...' are not valid, so checking is not possible"
This message could mean two different things:
- The requested output paths don't exist at all, or,
- Some outputs exist but are not known to Lix
Lix cannot reliably distinguish these cases, so it treated them the same.
We've updated the error messages to clarify what Lix can determine: whether any
valid outputs (> 0) are present or whether no outputs are available.
When no valid outputs can be found, Lix will now suggest building the derivation
normally (without --check or --rebuild) before trying again.
When some valid outputs are present, Lix now reports which ones are valid,
shows the full list of known outputs, and also suggests building the derivation
normally.
In the future, Lix may automate this recovery step when it knows how to rebuild
the paths, but implementing that safely requires more extensive changes to the
codebase.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- `nix develop` no longer ignores the env variable `SSL_CERT_FILE` [cl/5042](https://gerrit.lix.systems/c/lix/+/5042)
Running `nix develop` and `nix print-dev-env` on shells that define the environment variable `SSL_CERT_FILE` now works correctly by exporting that variable inside the built shell.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- Linux sandbox launch overhead greatly reduced [cl/5030](https://gerrit.lix.systems/c/lix/+/5030) [cl/5073](https://gerrit.lix.systems/c/lix/+/5073) [cl/5074](https://gerrit.lix.systems/c/lix/+/5074)
Sandboxed builds are now much cheaper to launch on Linux, with constant management
overhead. This will mostly be noticeable when building derivation trees containing
many small derivations like nixpkgs' `writeFile` or `runCommand` with scripts that
exit quickly. In synthetic tests we have seen build times of 3000 small runCommand
drop from 80 seconds to 14 seconds, which is the most optimistic case in practice.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- mTLS store connections via a plugin [cl/3754](https://gerrit.lix.systems/c/lix/+/3754) [cl/3696](https://gerrit.lix.systems/c/lix/+/3696) [cl/3697](https://gerrit.lix.systems/c/lix/+/3697) [cl/3698](https://gerrit.lix.systems/c/lix/+/3698)
To support use cases requiring mutual TLS (mTLS) authentication when connecting
to remote Nix stores, e.g. private stores, we have introduced a **contributed**
mTLS plugin extending the Lix store interface.
This design follows an extensibility model which was brought up [by a proposal
This mTLS plugin serves as a concrete example of how store connection
mechanisms can be modularized through external plugins, without extending Lix
core. This idea can be generalized to integrate automatic certificate renewal
or advanced integrations with secrets engine or posture checks.
It enables custom TLS client certificates to be used for authenticating against
a remote store that enforces mTLS.
To use the plugin, configure Lix manually by setting in your `nix.conf`:
```
plugin-files = /a/path/to/libplugin_mtls_store.so
```
Currently, this must be done explicitly. In the future, Nixpkgs will provide a
mechanism to reference an up-to-date and curated set of plugins automatically.
Making plugins easily consumable outside of Nixpkgs (e.g., from external plugin
registries or binary distributions) remains an open question and will require
further design.
Contributed plugins come with significantly reduced **stability** and
**maintenance** guarantees compared to the Lix core. We encourage users who
depend on a given plugin to take on maintenance responsibilities and apply for
ownership within the Lix mono-repository. These plugins are subject to removal
at any time.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [eldritch horrors](https://git.lix.systems/pennae), [mic92](https://github.com/mic92), [vlaci](https://github.com/vlaci), and [nkk0](https://github.com/nkk0) for this.
- Add an indication of nix-shell nesting depth [lix#826](https://git.lix.systems/lix-project/lix/issues/826) [cl/4657](https://gerrit.lix.systems/c/lix/+/4657)
When in a nix shell (either via a `nix-shell` or a `nix develop` invocation), a variable `NIX_SHELL_LEVEL` is exported to indicate the nesting depth of nix shells.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- `nix store delete` can now unlink a GC root before deleting its closure [cl/4660](https://gerrit.lix.systems/c/lix/+/4660)
Ever build something, and then you want to delete it and whatever dependencies it downloaded?
Before you had to resolve the `result` symlink and copy it, then delete it, *then* `nix store delete --delete-closure --skip-live` on the path you copied.
Now you can just pass `--unlink` and the `result` symlink itself.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- `nix path-info` no longer lies to the user about fetching paths [lix#323](https://git.lix.systems/lix-project/lix/issues/323) [cl/4866](https://gerrit.lix.systems/c/lix/+/4866)
When running `nix path-info` with an installable that is not present in the store, Lix no longer
tells the user which paths are missing and that they will be fetched, as the documentation clearly
states that this command does not fetch missing paths.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
- Derivations can now be printed in detail in `nix repl` [cl/3842](https://gerrit.lix.systems/c/lix/+/3842)
Traditionally derivations printed in the REPL would only print a formatted object
representing the path of the derivation file it refers to. This makes inspecting
the enhanced derivation attribute sets encountered from `mkDerivation` or similar
wrappers more difficult. Even the `:p`/`:print` command would not elaborate attribute sets
tagged as a derivation.
With this change you can now use `:p`/`:print` to directly inspect a derivation
by providing one as the top-level object. Derivation attribute sets will only be
printed two levels deep and internal derivation attrsets will remain in unexpanded
path form as before. `drvAttrs` will also be elided as these attributes are already
present in the top-level attribute set of the derivation. These heuristics provide
a balance between readability and functionality. When the `:p`/`:print` is omitted,
a bare derivation is printed in the path format as before.
Many thanks to [Lunaphied](https://git.lix.systems/Lunaphied) for this.
- Reject `__json` in structured attributes derivations [lix#380](https://git.lix.systems/lix-project/lix/issues/380) [cl/5286](https://gerrit.lix.systems/c/lix/+/5286)
In structured attributes derivations, `__json` is used internally to store the
JSON representation of the `env` attribute field that users can set.
Unfortunately, a user can set `__json` *and* enable structured attributes,
resulting in a broken derivation from a semantic point of view.
As no user can benefit from setting `__json` *and* enable structured attributes,
we disallow that possibility and throw an error from now on.
This is not seen as a breaking change because there's no user code that can
benefit from this behavior, hence, it's an improvement to user experience.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Shells support `$NIX_LOG_FD` now [lix#336](https://git.lix.systems/lix-project/lix/issues/336) [cl/4694](https://gerrit.lix.systems/c/lix/+/4694) [cl/4695](https://gerrit.lix.systems/c/lix/+/4695)
Lix's "debugging" shells (`nix3-develop` and `nix-shell`) now set the
`$NIX_LOG_FD` environment variable.
This means that [hook logging in
stdenv](https://github.com/NixOS/nixpkgs/pull/310387) appears while debugging
derivations via `nix3-develop` or `nix-shell`.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Supplementary groups are now supported for daemon authentication [lix#968](https://git.lix.systems/lix-project/lix/issues/968) [cl/5021](https://gerrit.lix.systems/c/lix/+/5021)
macOS, FreeBSD and Linux now support receiving supplementary groups during UNIX domain authentication to a Lix daemon.
This change is particularly beneficial for systemd units with `DynamicUser=true` that need to connect to a Lix daemon, using a `SupplementaryGroups=` allocated by systemd in the context of the process. This is desirable if you wish to harden Lix clients.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito), [Tom Hubrecht](https://git.lix.systems/tom-hubrecht), [alois31](https://git.lix.systems/alois31), and [eldritch horrors](https://git.lix.systems/pennae) for this.
## Fixes
- Nix shells' `$NIX_BUILD_TOP` are shorter [lix#1044](https://git.lix.systems/lix-project/lix/issues/1044) [cl/4663](https://gerrit.lix.systems/c/lix/+/4663)
Following the changes in 2.94.0 to shorten build directory paths, aimed at [resolving UNIX domain socket length issues](https://gerrit.lix.systems/c/lix/+/4168/13) and [improving nix-shell](https://git.lix.systems/lix-project/lix/issues/940), we inadvertently introduced an excessively long path for the `$NIX_BUILD_TOP` environment variable used by Nix shells (their effective temporary `/build` directory).
To fix this, we replaced the `build-top-$HASH` directory name with simply `build-top`, reducing these paths by at least 30 characters.
We also added a test to ensure that Nix shells do not introduce more than 50 extra characters relative to their base directory (e.g., `/tmp` when `$TMPDIR` is not set).
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Fix resolving of symlinks in flake paths [lix#106](https://git.lix.systems/lix-project/lix/issues/106) [lix#12286](https://git.lix.systems/lix-project/lix/pulls/12286) [cl/4783](https://gerrit.lix.systems/c/lix/+/4783)
Flake paths are now canonicalized to resolve symlinks. This ensures that when a flake is accessed via a symlink, paths are resolved relative to the target directory, not the symlink's location.
Many thanks to [stevalkr](https://github.com/stevalkr) and [xyenon](https://git.lix.systems/xyenon) for this.
- The REPL no longer considers failed loads for `:reload` [lix#50](https://git.lix.systems/lix-project/lix/issues/50) [cl/4864](https://gerrit.lix.systems/c/lix/+/4864) [cl/4865](https://gerrit.lix.systems/c/lix/+/4865) [cl/4700](https://gerrit.lix.systems/c/lix/+/4700) [cl/4889](https://gerrit.lix.systems/c/lix/+/4889)
The [REPL](@docroot@/command-ref/new-cli/nix3-repl.md) allows "loading" files, flakes, and expressions into the environment, with the commands `:load`/`:l`, `:load-flake`/`:lf`, and `:add`/`:a` respectively.
The results of those stay in the environment as-is even if their sources change, until the `:reload` command is used.
However `:reload` would re-perform *all* instances of `:l`/`:lf`/`:a`, meaning you would get things like this:
```nix
nix-repl> :l /tmp/texting.nix
error: getting status of '/tmp/texting.nix': No such file or directory
# oops, typo.
nix-repl> :l /tmp/testing.nix
# Do some stuff…
nix-repl> :reload
error: getting status of '/tmp/texting.nix': No such file or directory
```
This is pretty silly, but also *incredibly* annoying, as it would stop there and *not* reload the correct files anymore.
This effectively meant typoing any of the load commands would make `:reload` useless for the rest of the entire `nix repl` session!
This has been fixed, so now only *successful* loads count towards `:reload`.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [Qyriad](https://git.lix.systems/Qyriad) for this.
- Consistently use commit hash as rev when locking git inputs [cl/4762](https://gerrit.lix.systems/c/lix/+/4762)
Lix will now use commit hashes instead of tag object hashes in the `rev` field
when fetching git inputs by tag in `flake.lock` and `builtins.fetchTree` output.
Note that this means that Lix may change some `flake.lock` files on re-locking. Old `flake.lock` files still remain valid.
Many thanks to [goldstein](https://git.lix.systems/goldstein) for this.
## Development
- Functional lang migration [lix#856](https://git.lix.systems/lix-project/lix/issues/856) [cl/3213](https://gerrit.lix.systems/c/lix/+/3213)
We have done it! The functional/lang framework has now been fully migrated to functional2/lang.
This means: no more `just clean` and `just install` mess and whatever because one removed a test.
The lang test suite is also getting a face lift, with an improved folder structure and restructuring of many tests.
Only the first CL of the chain is provided but there's way more changes associated to this project.
Many thanks to [piegames](https://git.lix.systems/piegames) and [rootile (Rutile)](https://git.lix.systems/rootile) for this.
## Miscellany
- Warn instead of erroring when the final destination of a transfer changes in-flight [lix#1004](https://git.lix.systems/lix-project/lix/issues/1004) [cl/4641](https://gerrit.lix.systems/c/lix/+/4641)
Lix will now emit a warning during downloads where the final destination changes suddently mid-transfer instead of throwing an error.
This transfer behavior has been known to happen very rarely while fetching from some CDNs.
Many thanks to [Tom Hubrecht](https://git.lix.systems/tom-hubrecht) for this.
f"This function is only available if the [{self.experimental_feature}](@docroot@/contributing/experimental-features.md#xp-feature-{self.experimental_feature}) experimental feature is enabled."
ifself.experimental_featureisnotNone
else""
}
</dd>
""")
defmain():
importargparse
ap=argparse.ArgumentParser()
ap.add_argument('--header',help='Path of the header to generate')
ap.add_argument('--docs',help='Path of the documentation file to generate')
ap.add_argument('--experimental-features',help='Directory containing the experimental feature definitions')
'''+(f'''This function is only available if the [{builtin.experimental_feature}](@docroot@/contributing/experimental-features.md#xp-feature-{builtin.experimental_feature}) experimental feature is enabled.
f' - [`{feature.name}`](@docroot@/contributing/{"deprecated"ifargs.deprecatedelse"experimental"}-features.md#{"dp"ifargs.deprecatedelse"xp"}-feature-{feature.name})\n')
throwUsageError("--keep does not make sense without --ignore-environment");
for(constauto&var:unset)
unsetenv(var.c_str());
(void)sys::unsetenv(var);
}
}
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Raito Bezarius