Commit Graph
78 Commits
Author SHA1 Message Date
2c6542bd9c libstore/http: provide makeOptions hook
Some users may have arbitrary needs to connect to their store URIs, e.g.
mTLS authentication, Kerberos authentication, custom renewal using any
RPC mechanism of their preference and so on.

To avoid encoding all these patterns in Lix itself, we push the
configuration to the plugin boundaries and offer a hook for end users to
inherit from `HttpBinaryCacheStore` and provide new store schemes like
`https+mtls://my.very.secure.cache?tls-certificate=...&tls-key=...` or
`https+krb5://my.kerberos.enabled.cache`.

Co-authored-by: George Shammas <george@shamm.as>
Co-authored-by: eldritch horrors <pennae@lix.systems>
Signed-off-by: Raito Bezarius <raito@lix.systems>
Change-Id: I79f322b1a74632500fc79d53f5c920f9e43fd0c4
2025-07-23 13:17:28 +00:00
Raito Bezarius f8ccd9d572 libutil/async-io: augment read type safety w.r.t to EOFs
Usually, EOFs are represented by returning 0 in the `read` APIs, at
least, this is what read(2) dictate.

As clever creature, we may sum zeroes sometimes (advanced form:
`buf->added(got)`) and forego handling the EOF condition.

To avoid the bug that lurked in remote-store.cc and caused busy looping
if the remote end disconnects suddenly, we return
`Result<Option<size_t>>` forcing the caller to perform a specific
processing for the EOF situation.

The conversion did not raise any other offending code path.

Change-Id: I185fdcb77aa82d87ab0802d66ac37c1363657a73
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-07-23 10:35:37 +00:00
Raito Bezarius b225794503 libcmd: drop warning about interpretation of store paths
Fixes #897.

Change-Id: Ie4da11d0427d8c9f48cd6e979d80bd39474e8390
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-07-07 21:13:16 +00:00
Raito Bezarius df58bffc0f doc/rl-next: missed credit for the symbol reuse work
We missed xokdvium being author on cl/3300. This is something we
absolutely want to avoid.

We credit xokdvium in the RL and add a note on this problem.

Thanks to xokdvium for reaching out in private to us so we can repair
this mistake.

Change-Id: I094d0f95b6647104621d6b228e69a4529a300304
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-07-06 11:03:07 +00:00
Raito Bezarius 286aa409b2 Revert "libstore/build: automatic clean up of unsuccessfully built scratch outputs"
This reverts commit 42e2bd045c
because this is the root cause of the critical correctness bug.

Change-Id: Ia2360e24650a923034de72ffc193ecb73470cc48
2025-06-29 21:26:36 +02:00
Raito Bezarius 13e46d3d00 Revert "libstore: fix scratch output cleanup"
This reverts commit a0a00948df
because this was insufficient to fix the critical correctness bugs.

Change-Id: I6c7b560ebeebacbbbcc1cbf26e6ef50c38b84f7f
2025-06-29 21:26:11 +02:00
Raito Bezarius a60c1de715 Revert "libstore: don't delete already valid outputs after build"
This reverts commit e356d54d7a
because this was insufficient to fix the critical correctness bugs.

Change-Id: I91c3e368ffd13ade6a3cebbbacdb42655796ea56
2025-06-29 21:26:01 +02:00
Raito Bezarius 33122e79df libstore: weaken the top-level fallback temp dir to 0755 for macOS
Under macOS, the first level of directory has actually mode 0755 instead
of 0700 as macOS often do not possess the right primitives to chroot
inside of these directories, leading to
https://github.com/NixOS/nix/pull/11031.

Thanks to Emily for the heads-up on this type of matter.

Change-Id: I9d4e53717f61c9d573ff176f820610612804fbc3
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-28 17:37:43 +02:00
Raito Bezarius fd35e86fc5 libstore: fallback on creating a safe space in the default tempdir
If `settings.buildDir` cannot be written to, because we are in a chroot
store, unprivileged or anything.

We can and should always gracefully fallback to a *secure* location
inside of /tmp, i.e. `/tmp/<a directory under 0700>/<our temporary
directory for build under 0700>/...`.

This does not reintroduce CVE-2025-52991 because we are creating a
directory in-between compared to creating only ONE level of directory.

Fixes #876.

Change-Id: Ie521202923f763225e1901ab1b9b6c6132aaf548
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-27 17:39:59 +00:00
Raito Bezariusandeldritch horrors 9a59106c17 libstore: NIX_REMOTE=auto tries the daemon socket *then* direct access
In the past, it tried direct access if it *could* [1] perform direct
access.

This solves a bunch of errors people had when they tried the cgroup
feature and their scripts did not pass NIX_REMOTE=daemon manually
(nixos-rebuild-ng, home-manager activation from a root systemd unit,
etc.)

To avoid looping infinitely while receiving daemon connections, we
forcibly change the store URI when forking for a subdaemon to do direct
access automatically, this doesn't break forward usecases where you
point a daemon to another socket because we only change NIX_REMOTE="",
NIX_REMOTE=daemon, NIX_REMOTE=auto to a local and direct access.

All these usecases would end up infinitely looping no matter what
settings are set, because we are also responsible for creating the
daemon socket.

[1]: this happened all the time if you were `root`.

Related: https://github.com/NixOS/nixpkgs/pull/415701
Change-Id: I783fc795a9c2ee25b3d9f44f453f8f94b063371f
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-25 14:59:58 +00:00
Raito Bezarius 42e2bd045c libstore/build: automatic clean up of unsuccessfully built scratch outputs
When a build fails, its scratch output paths are not cleaned up.

Until recently, this was deemed not a problem but as part of the effort
to harden the Nix builds and protect these paths against being part of a
staged attack (race conditions, etc.), we automatically cleanup after
failed builds.

Fixes CVE-2025-52992.

Change-Id: I58481b1cc83826298b9d80d37fecf81f117ccb09
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-24 10:49:49 +00:00
Raito Bezarius 11c5e3bbcc libutil: ensure that _deletePath does NOT use absolute paths with dirfds
When calling `_deletePath` with a parent file descriptor, `openat` is
made effective by using relative paths to the directory file descriptor.

To avoid the problem, the signature is changed to resist misuse with an
assert in the prologue of the function.

Fixes CVE-2025-46415.

Change-Id: I6b3fc766bad2afe54dc27d47d1df3873e188de96
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-24 10:49:49 +00:00
Raito Bezarius dceb9438d2 libstore: ensure that passAsFile is created in the original temp dir
This ensures that `passAsFile` data is created inside the expected
temporary build directory by `openat()` from the parent directory file
descriptor.

Fixes CVE-2025-52993.

Change-Id: Ie5273446c4a19403088d0389ae8e3f473af8879a
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-24 10:46:02 +00:00
Raito Bezariusandeldritch horrors 2d836357dc libutil: writeFile variant for file descriptors
`writeFile` lose its `sync` boolean flag to make things simpler.

A new `writeFileAndSync` function is created and all call sites are
converted to it.

Change-Id: Ib871a5283a9c047db1e4fe48a241506e4aab9192
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-24 00:28:09 +02:00
Raito Bezarius 10509774ed libstore: chown to builder variant for file descriptors
We use it immediately for the build temporary directory.

Change-Id: I180193c63a2b98721f5fb8e542c4e39c099bb947
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-23 16:57:23 +02:00
Raito Bezarius bcf1f27fec libstore: open build directory as a dirfd as well
We now keep around a proper AutoCloseFD around the temporary directory
which we plan to use for openat operations and avoiding the build
directory being swapped out while we are doing something else.

Change-Id: I18d387b0f123ebf2d20c6405cd47ebadc5505f2a
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-23 16:57:23 +02:00
Raito Bezarius c7976e63a3 libutil: guess or invent a path from file descriptors
This is useful for certain error recovery paths (no pun intended) that
does not thread through the original path name.

Change-Id: I2d800740cb4f9912e64c923120d3f977c58ccb7e
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-23 16:57:23 +02:00
Raito BezariusandMaximilian Bosch cd129186ea libstore/s3: fulfill with PutObjectOutcome instead of HeadObjectOutcome
This was probably a typo introduced in
7453e2979f.

Unfortunately, AWS SDK is so well made that this typo became an assert
error in production.

AWS Outcome constructors contains
```
            // Move error from other type of outcome
            template<typename RT, typename ET,
enable_if_t<!std::is_convertible<RT, R>::value &&

std::is_convertible<ET, E>::value, int> = 0>
```

which means that when:

* RT → R is not possible (e.g. PutObjectOutcome → HeadObjectOutcome)
* ET → E is possible (e.g. S3Error → S3Error)

Then, we will instantiate the error-moving outcome constructor which
asserts `!o.success`… Though, the original outcome indeed succeeded.

Change-Id: I3809514ae0648e8c02b0f93fa64d91115a091cd9
Co-authored-by: Maximilian Bosch <maximilian@mbosch.me>
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-22 19:45:53 +00:00
Raito Bezarius 87d99da6ca libutil/cgroup: ensure that cleanup takes place even under interruptions
When Ctrl-C is sent to the workload, even across remote builds, the
whole process possess a global flag `_isInterrupted` which is checked in
certain filesystem operations, cancelling them, e.g. writeFile will
write nothing under interruption unwinding.

In addition, if any operation throws an exception before we `rmdir` the
cgroup, we may leave it hanging while we remove the state record.
Therefore, we put the final cleanup in a block.

In practice, reading statistics could lead to failures.

Control groups cleanups are critical though and should always be
performed.

Change-Id: I48fa87317b6a9f6663559bc8fa5f8a897f37011e
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-17 22:20:37 +02:00
1e34c37477 flake/inputs: 24.11-small -> 25.05-small
We upgrade to 25.05 release, which contains the curl commit
https://github.com/curl/curl/commit/5fbd78eb2dc4afbd8884e8eed27147fc3d4318f6
done in
https://github.com/NixOS/nixpkgs/pull/396200#issuecomment-2795944006.

This fixes HTTP transfers generating arbitrary errors and possibly
failing unusually.

Users who are already depending on 25.05-small or a recent unstable
already had the fix.

Special mention to the Linux kernel who gave me the opportunity to get
on a 24 hours bisection side quest to fix the local release engineering
test.

Special thanks to everyone who had to endure me ranting.

Change-Id: I866caf65d5ea103f1fa5eccd57df8031c9eacda0
Co-authored-by: eldritch horrors <pennae@lix.systems>
Co-authored-by: helle <helle@h3l.li>
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-13 18:15:47 +00:00
Raito Bezarius bea24c8d27 libutil/cgroup: destroy state record at destroy time
If state records are not destroyed at destroy time, this might confuse a
new build that thinks there's a remnant of a cgroup when actually it was
destroyed.

This fixes a bunch of inoffensive and noisy warnings about cgroups being
deleted by someone else.

Reported-by: Ramses <@rvdp:infosec.exchange>
Change-Id: Ib3d33f4ecd6143f33e032c5107b288b4ecabaee1
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-11 15:53:04 +02:00
Raito Bezarius 7bd82718e2 libstore/platform/linux: destroy cgroup before we release user locks
User locks are taken to avoid another build grabbing the same UID.

Under build user contention, it is possible to recycle the same UID from
another build which did not run the Goal destructor yet.

Prior to this change, cgroups were destroyed at Goal destruction time,
but user locks were released at `buildDone()` time.

Therefore, it was possible to have 2 builds fights for the same cgroup
and mess with it, resulting in confusion.

To avoid this, we override `cleanupHookFinally` in charge to release the
user locks and we destroy the cgroup before releasing the locks.

Statistics are kept in the `cgroup` object a bit longer and can be
obtained at `killSandbox(true)` time.

`AutoDestroyCgroup::kill` now ignore if the cgroup path has already been
destroyed, as kill is idempotent.

Reported-by: Ramses <@rvdp:infosec.exchange>
Reported-by: Frederico Schonborn <@fredericoschonborn:matrix.org>
Change-Id: Idfbf9aaf010c5f718f2c1c38548383d912d8ee95
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-11 13:47:38 +02:00
Raito Bezarius 797c6d4cd4 libutil/file-system: make AutoDelete not copyable and movable
Such a RAII structure should NEVER be copyable or movable, otherwise:

```
AutoDelete x;

x = AutoDelete(p, false);
```

will trigger the immediate deletion of `p`!

This fixes an annoying bug where the state record for cgroups was
deleted immediately as soon as it was created.

Change-Id: I2bfbc0815706700a0a75b79d1059cc552119b2c9
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 23:29:10 +02:00
Raito Bezarius 9f9fced2dd libstore/build/worker: clean up cgroup error messages typos
It's `delegated` and not `delgated`, also it's `DelegateSubgroup` and
not `DelegateSubtree` which I clearly hallucinated because of subtree
vs. sub(c)group.

Change-Id: Icfaa6116fa83416c431820978ef35aa8aa943feb
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 23:29:10 +02:00
a527bb251a libstore/build: cgroup delegation to sandbox
We offer full cgroup delegation to our sandbox now, required for running
containers inside the sandbox.

To run systemd-nspawn or containers managers inside the sandbox, there
is a need for one extra ingredient now: control over your own cgroup
subtree inside the sandbox.

If, in addition, you need multiple UIDs, for e.g. rootless usecases, you
need to run with the `uid-range` system feature.

Therefore, when the daemon or Nix runs under the right condition, e.g.
systemd-style delegation of the cgroup subtree while placing the
nix-daemon in a supervisor sub-cgroup, we create a new sub-cgroup for
each build based on the build UID and delegate that sub-cgroup to the
builder's process.

Additionally, `uid-range` always request the `cgroups` feature now, as
`uid-range` builds would probably always benefit from having cgroups
delegated, but the converse is not true.

Inspired from https://github.com/NixOS/nix/pull/11412 with a different
design that does not use function-local statics to derive the root
cgroup.

Co-authored-by: Linus Heckemann <git@sphalerite.org>
Co-authored-by: Parker Hoyes <contact@parkerhoyes.com>
Change-Id: Ic8947c5adaf4b5bbd153386e05fad65a935274fa
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 20:50:00 +02:00
Raito Bezarius 1783d5b348 libstore/build: drop cgroups experimental feature
We drop it to re-introduce it via the concept of build context which
will control in which cgroup a certain build should be spawned.

Change-Id: I4b4705d768129a6d7c0f061dc2163ba116088b18
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 16:00:51 +00:00
Raito Bezarius 21dbd7745d libstore/binary-cache-store: skip NAR listings if it's not possible to serialize it
Some source trees might not be representable inside of the NAR listing
format v1 as file paths (on Linux) are not guaranteed to be valid UTF-8.

When something like this happens on a large-scale build farm, a
mysterious "queued" but impossible to process job appears, this is
because we cannot write the NAR listing and serialization always fails.

Why did this work before? nlohmann was introduced _after_ such paths
were ingested, see: 09f00dd4d0.

What happened for such previously mis-serialized NAR listings?

```
curl -v 'https://cache.nixos.org/nz8p9hn00r6z7s57581c1hiv39pa1ia6.ls' |
brotli -d | jq .
```

This fixes the build of `sub-batch`
(https://github.com/kl/sub-batch/tree/master/tests/rename_invalid_utf8)
on ForkOS infrastructure.

Many thanks to Puck for the assistance on holding `rr` right on this one
and finding the history of these changes.

Change-Id: I2c2fbac70818e02810f9fd236c3a248187bf5fe7
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 15:22:22 +00:00
Raito Bezarius 1e71df37b7 doc/manual/rl-next: mention symbol value reuse
Forgotten in the symbol value reuse chain.

Change-Id: I7050f56cffcddce5fae4f74ebb35a9fe108a5dcf
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 16:25:29 +02:00
Raito Bezarius ebc8f56b52 libexpr/primops: pass the underlying Value of symbols if possible
Instead of allocating a new Value and copy the symbol string
representation inside of it, we can pass along the underlying Value,
which avoids (garbage collected) allocations.

This results in:

* a ~8 % reduction for `gc.totalBytes` over
  `nixos.ec2.closures.x86_64-linux` for NixOS 24.11. (920MiB → 842MiB)
* a slight reduction in CPU time due to less allocations being performed
  at all

Change-Id: I097f586dbc98f889fbc62d0a5f80c9d76ddedfd2
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 13:58:08 +02:00
Raito Bezarius 5e98a2159b libexpr/symbol-table: introduce InternedSymbol
The backing storage for symbols becomes a class storing a Value and a
string.

The Value is itself a string which contents points to the owned string.

Recovering a `SymbolStr` is still possible.

Change-Id: I171151abc3c0a513f2150c4b54edd61dea256cce
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 13:58:08 +02:00
Raito Bezarius 364e94fe23 libexpr/value: do not depend on Symbol
The symbol table will contain types that encloses a Value, thus, it
needs to depend upon the Value header, whereas the Value header depends
on `Symbol` for typedefs.

We move the typedefs in the place where they are used.

Change-Id: Ic533e5aad927b9bc4a9d1723430e90e86a4b5466
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 13:58:08 +02:00
Raito Bezarius 95ea358f98 libexpr/print: drop redundant constructor in emplace_back call
Change-Id: I79210edfede0a1d17f38b5834515f56d44c97466
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 13:58:08 +02:00
Raito BezariusandTom Hubrecht 5db71cfb3b libutil: add should emplace inside a ChunkedVector
This simplifies many call-sites where construction can take place
automatically.

Change-Id: I87f697d55375676345b388024eb8df900bf808de
Co-authored-by: Tom Hubrecht <github@mail.hubrecht.ovh>
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 13:58:06 +02:00
Raito Bezarius b70bbbe680 misc/pre-commit: add automatic clang-format of changed lines
Lix has a style guide:
https://wiki.lix.systems/books/lix-contributors/page/code but
contributors like me have been unable to enforce it, which is sad.

To avoid further violations of that style guide, we enable a pre-commit
hook for clang formatting of the changed lines.

Change-Id: I217452efa3ac8bd66b4d3a08a6fe9a241207790b
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-07 15:27:11 +02:00
Raito Bezarius c19a0fe288 devshell: add git-clang-format
This is useful to reformat only changed hunks of a file via
`clang-format`.

Change-Id: I9aa8526d75fd2301113ee57f3a2e595f3b03504f
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-07 15:27:11 +02:00
Raito Bezarius b792279780 tests/functional/lang: update error path locations
The coerce integer feature was not rebased before merge and we do not
have a merge queue, hence, after merge, the HEAD was in a broken state.

We take a commitment to invest into a merge queue now and do a fixup
here.

Change-Id: Ied9410690b542359859ab5f597f22ebceb857305
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-05-28 22:29:04 +02:00
Raito BezariusandLix Systems Gerrit 316aa591ac Merge "libexpr: coerce integers under the XP feature coerce-integers" into main 2025-05-28 19:39:59 +00:00
Raito Bezarius 1e40171ea4 libexpr: coerce integers under the XP feature coerce-integers
This introduces a new (demanded?) feature for coercing integers in
interpolation arguments under the experimental feature
`coerce-integers`.

This feature is being introduced behind an *experimental feature flag*
due to the cautious approach we're taking. The codebase has a track
record of revealing unexpected behaviors, often in subtle ways, so we
want to give this sufficient time and exposure before making it stable.

To remove the experimental flag, we want to see **at least two releases
or six months of real-world usage -- whichever is longer** -- that
demonstrate strong confidence the feature doesn't introduce regressions
or unintended side effects. If that level of confidence is reached,
we'll proceed to stabilize it.

Change-Id: I825904719eeba8f0e2a93cd6b93cfe6cebd7d827
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-05-27 11:42:53 +02:00
Raito BezariusandLix Systems Gerrit 4f433a6186 Merge changes I9f893374,Ief7a4756 into main
* changes:
  libexpr: rename `forceString` to `isInterpolation`
  libexpr: refactor string coercion modes
2025-05-20 20:45:21 +00:00
Raito Bezarius 3e4bffcc24 libexpr: rename forceString to isInterpolation
`ExprConcatStrings` tracks whether the expression is an interpolation or
not via an obscure boolean called `forceString`.

Instead, we rename it to `isInterpolation`.

This is a breaking change for the JSON AST representation.

Change-Id: I9f89337449b56f6e99a961e21169761f554c9896
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-05-20 19:14:49 +02:00
Raito BezariusandPierre Bourdon dab871f129 libexpr: refactor string coercion modes
Inspired by cl/3191 and
https://git.lix.systems/delroth/lix/commit/ae0247cbb4fc739ab013dc87d02e5f3191cf25ab.

`coerceToString` takes now an enumeration that lives in `value.hh`, this
enumeration is meant to represent increasing subsets of behaviors, e.g.
any level above Strict should do what the previous levels do and extra
behavior until `ToString`, which transforms many Nix values into an
arbitrary string representation, e.g. `null` to `""`.

Change-Id: Ief7a4756e8c0660e197623efebeaf07710746ec7
Signed-off-by: Raito Bezarius <raito@lix.systems>
Co-authored-by: Pierre Bourdon <delroth@gmail.com>
2025-05-20 19:14:49 +02:00
Raito BezariusandLix Systems Gerrit cfaeed469b Merge "libstore/ssh: remove echo started check" into main 2025-05-18 19:51:20 +00:00
Raito Bezarius 0dd8bf6c1c libstore/ssh: remove echo started check
The pre-flight `echo started` check over SSH was originally added in
577ebeaefb. As it is usual with these old
commits, understanding why is there a need for something is difficult.

The closest thing would be
> Fix a race starting the SSH master. We now wait synchronously for
> the SSH master to finish starting. This prevents the SSH clients
> from starting their own connections.

But, we removed SSH connection sharing, so this does not apply anymore.

Nonetheless, we believed this check was meant as a way to catch obvious
misconfigurations or SSH failures early, before handing off to
`nix-store`. However, this approach was not fruitful: it assumes the
remote has a `bash`-compatible shell, `echo` behaves in a standard way,
and no `ForceCommand` interferes—all of which are unreliable assumptions
in practice.

While the intent was to provide slightly better diagnostics (e.g. in
case of SSH hanging or returning an interactive shell), in practice it
does not meaningfully catch or improve real failure cases. The
underlying protocol or engine can and should handle those errors more
robustly anyway.

In contrast, this check *does* break several legitimate workflows,
including:

* remote builders using `ForceCommand` wrappers (e.g.
`nix-remote-build`-style setups), see
<https://discourse.nixos.org/t/wrapper-to-restrict-builder-access-through-ssh-worth-upstreaming/25834/15>,

* SSHing into minimal environments lacking `bash` (e.g. initrd,
busybox-based systems),

* configurations that don’t default to POSIX-like shells, e.g., nushell
enthusiasts.

As such, we’re removing this code. Protocol mismatch errors and SSH
failures can be rethought and handled more structurally elsewhere in the
engine.

Change-Id: I187f6881375d42ef83987a13a350c97964bbdb30
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-05-18 18:45:26 +02:00
Raito BezariusandLix Systems Gerrit 901940df00 Merge "libstore/profiles: do not dereference current generation if it doesn't exist" into main 2025-05-12 22:47:48 +00:00
Raito BezariusandQyriad d19a9e3039 libstore/profiles: do not dereference current generation if it doesn't exist
If the profile inode is invalid, e.g. invalid symlink, the current
generation cannot be discovered.

Nonetheless, this should not be a reason for an assert failure, instead
of crashing, just raise an error.

Fixes fj#801.

Change-Id: I63937672173bc3bf37196de98307800adc5757e1
Signed-off-by: Raito Bezarius <raito@lix.systems>
Co-authored-by: Qyriad <qyriad@qyriad.me>
2025-05-11 01:07:18 +02:00
Raito Bezariusandeldritch horrors 3f3060ebed misc/capnproto: patch to avoid monotonic time checks
From time to time, our KVM's CI machine clock goes in a weird direction
during NixOS tests, this is known on certain hardware, e.g. macOS, where
the check is disabled. Cap'n'Proto removed the check everywhere now.

Fixes fj#754.

We can remove the patch once it hits a stable version of Cap'n'Proto.

Change-Id: I4c9be5061c3b244f601486a8ac4521dff44ceb92
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-05-09 14:17:35 +02:00
Raito BezariusandLix Systems Gerrit 9cc0cda16f Merge "libstore/ssh: fix the SSH connectivity check with non-POSIXy shells" into main 2025-04-26 11:06:49 +00:00
Raito Bezarius 5ca168f6f2 chore: drop experimental feature recursive-nix
This was discussed as part of https://git.lix.systems/lix-project/lix/issues/767
with a one month long comment period.

This commit removes the recursive-nix feature from Lix. It has seen
limited usage and is difficult to stabilize, especially on non-Linux
systems where its behavior is underspecified.

Maintaining this feature complicates core work on the store, as we must
account for the potential presence of the daemon in the sandbox, adding
unnecessary complexity. Additionally, its inclusion in the
platform-independent local store creates risks for non-Linux platforms.

For more details on this removal, refer to the release note entry or the
issue entry.

Change-Id: I9137202f563c0a317f9c5da79cd9fd07d801427a
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-04-25 17:09:01 +02:00
Raito BezariusandLix Systems Gerrit a133633ecc Merge "lixexpr: Declutter error traces" into main 2025-04-24 19:07:06 +00:00
Raito BezariusandLix Systems Gerrit c36c83c4c6 Merge "dtrace: fix on macOS (except in the package)" into main 2025-04-24 16:33:48 +00:00
Raito Bezarius a6adaab2bc libmain: undo accidental lix command stabilization
When external subcommand support landed, the `lix` binary gave
accidental access to nix3 CLI as well.

This was not the intended effect of this feature, we want to keep `lix`
as a namespace for a new redesign of the CLI (that has not started yet),
while giving that namespace for external subcommands.

To this end, we sever the nix3 CLI connection for the `lix` binary.

Fixes fj#508.

Change-Id: I13a0748bcbf87343c8e0e89439bd4af3a0ac3118
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-03-28 21:08:52 +01:00
Raito BezariusandGerrit Code Review b1a8c0f2e0 Merge "libfetchers: ensure that lastModified is a uint64_t" into main 2025-03-13 16:44:10 +00:00
Raito BezariusandGerrit Code Review 271f0e2507 Merge "libexpr: fix checkSourcePath purity regression" into main 2025-03-03 02:00:35 +00:00
Raito BezariusandGerrit Code Review d037b9e102 Merge "libexpr: rectify filtering logic for filter builtins under chroot stores" into main 2025-02-27 01:37:13 +00:00
Raito BezariusandGerrit Code Review 446af4c6fe Merge "flake: Add support for x86_64-freebsd." into main 2025-02-24 23:12:12 +00:00
Raito Bezarius 55cbabd316 feat(libstore/build): always create a new cgroup namespace
There's no reason to make it dependent upon `Xp::Cgroups`, this
increases the purity of the builders' views.

Change-Id: I3d934428177e6bb78cd700d1cce1ded8a88e9da0
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-02-25 00:00:48 +01:00
Raito BezariusandGerrit Code Review 43b0902390 Merge "feat: add support for external lix- prefixed commands in the CLI" into main 2025-02-24 10:51:27 +00:00
Raito Bezarius 5f50741ce9 feat: add support for external lix- prefixed commands in the CLI
This feature allows calling external binaries starting with `lix-` as
subcommands directly within the `lix` CLI, similar to how Git handles
external commands.

For example, a binary named `lix-example` can now be invoked using `lix
example`.

This change opens up the possibility for broader community-driven
extensions of the Lix CLI, offering flexibility that Flakes has somewhat
restricted by centralizing certain features around its own model of
dependency management.

By enabling users to introduce custom subcommands, we encourage
experimentation and diverse workflows, addressing the needs of those who
seek more modular, feature-rich approaches to package management and
system configuration beyond what Flakes currently offers [1] [2] [3].

This adds `lix` as a new binary to be able to use this feature.

[1]: https://github.com/nmattia/niv
[2]: https://github.com/andir/npins
[3]: https://github.com/nikstur/lon

Change-Id: Ic6344424a6a46fc9fb30432f00e21c3509659f8a
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-02-23 15:24:28 +01:00
Raito BezariusandGerrit Code Review c64a9db6bb Merge "doc/authors: add lilyinstarlight" into main 2025-02-23 09:47:26 +00:00
Raito Bezarius 6a41dae49a fix(gc): log sudden "path in use" exceptions and recover during GC gracefully
Original-Author: picnoir <picnoir@alternativebit.fr>

Inspired from
https://github.com/NixOS/nix/pull/11922/commits/ced8d311a593fcf9c3823e4e118474ac132d8e60
and adapted for Lix needs.

TL;DR: The topological sort should ensure that it is possible to delete
the path iterated upon. Nonetheless, in some cases,
`invalidatePathChecked` can still throw `PathInUse`, the exception
bubbles up and cancel the garbage collection procedure, leaving the rest
of the paths untouched. This change ensure that the error is logged for
further investigation but doesn't prevent the GC to continue when it
can.

After code review, we decided to make it a `printInfo` to inform the
user about sudden "in use" dependencies during garbage collection and
let them re-run garbage collection if they care about this.

References: https://github.com/NixOS/nix/issues/11923
References: https://git.lix.systems/lix-project/lix/issues/621
Change-Id: I5606c9afd16b5faa747b713fde2dc24016990ba3
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-01-20 19:48:06 +01:00
Raito BezariusandGerrit Code Review 763a61bb7c Merge "Revert "Reject weak hash algorithms as SRIs, and warn in any other cases"" into main 2024-11-02 19:50:52 +00:00
Raito BezariusandGerrit Code Review 6e2349d2e1 Revert "Reject weak hash algorithms as SRIs, and warn in any other cases"
This reverts commit 02c35ea9df.

Reason for revert: this code path is also used for `Input::getRev()`, i.e. flakes VCS revision validation, which, in the case of Git, are using SHA1.
As a result, this cause too much noise due to SHA1 revisions in Flakes.

Change-Id: I8064c1ebc26e4e83b627f0803a7a9ba56cfe1f37
2024-11-01 11:59:59 +00:00
Raito BezariusandGerrit Code Review 9f682204b5 Merge changes I85b6075a,Iee41b055 into main
* changes:
  libstore: ban unpacking case hacked filenames from NARs
  testsuite: add a NAR generator with some evil NARs
2024-10-27 18:12:18 +00:00
Raito BezariusandGerrit Code Review f9e7df01f3 Merge "daemon: stop eating SIGINTs" into main 2024-10-27 18:11:20 +00:00
Raito BezariusandGerrit Code Review f7edee7c14 Merge changes I8e11ddbe,Idb8d9a00 into main
* changes:
  nix-shell: stop using dynamic format strings!!
  tests: move nix-shell related tests to subdir
2024-10-27 18:10:17 +00:00
Raito Bezarius 8e05cc1e6c Revert "libstore: remove worker removeGoal"
Revert submission 1946

Reason for revert: regression in building (found via bisection)

Reported by users:
> error: path '/nix/store/04ca5xwvasz6s3jg0k7njz6rzi0d225w-jq-1.7.1-dev' does not exist in the store

Reverted changes: /q/submissionid:1946

Change-Id: I6f1a4b2f7d7ef5ca430e477fc32bca62fd97036b
2024-10-01 11:07:57 +00:00
Raito Bezariusandalois31 67f62bcdb4 doc/release-notes: add date for major release
Change-Id: I93aab93c069bb3989c3f8d17e0862899e6f76865
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-07-17 22:12:41 +00:00
Raito Bezarius b8cb7abcf0 chore: rebrand Nix to Lix when it makes sense
Here's my guide so far:

$ rg '((?!(recursive).*) Nix
(?!(daemon|store|expression|Rocks!|Packages|language|derivation|archive|account|user|sandbox|flake).*))'
-g '!doc/' --pcre2

All items from this query have been tackled. For the documentation side:
that's for https://git.lix.systems/lix-project/lix/issues/162.

Additionally, all remaining references to github.com/NixOS/nix which
were not relevant were also replaced.

Fixes: https://git.lix.systems/lix-project/lix/issues/148.
Fixes: https://git.lix.systems/lix-project/lix/issues/162.
Change-Id: Ib3451fae5cb8ab8cd9ac9e4e4551284ee6794545
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-06-01 20:31:24 +02:00
Raito BezariusandGerrit Code Review 8e1a883186 Merge "chore: remove incorrect maintainers/*.md documentation" into main 2024-05-20 12:35:20 +00:00
Raito Bezarius 93dbb698b3 chore: remove incorrect maintainers/*.md documentation
Fate has something different in store for the release process,
backporting process and the general maintainer documentation.

See https://git.lix.systems/lix-project/lix/issues/260.

Change-Id: I626686ff4059aee22a3ab1664b52581b2dbf6ed7
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-19 16:58:52 +02:00
Raito BezariusandJade Lovelace c52eba4424 feat: add credits field to release note generator
Now, we can credit folks for their work.

The credit generator is very basic, we probably want a database of
profiles and link to their preferred page or something.

Change-Id: Ida81905750371e5e125d0ce7e554d0526265cf8e
Co-Authored-By: Jade Lovelace <lix@jade.fyi>
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-14 19:26:09 -07:00
Raito BezariusandGerrit Code Review 4ebbd4362f Merge "feat(libstore): print the first line of stdout of SSH in case of failure" into main 2024-05-10 19:33:36 +00:00
Raito Bezarius 8404a1f66d feat(libstore): print the first line of stdout of SSH in case of failure
In case of failure to connect as can be seen in
https://buildbot.lix.systems/#/builders/39/builds/1386/steps/1/logs/stdio

It is difficult to understand what happened, if we enabled the talkative
verbose level, we could learn about the first line SSH sent us.

In practice, this is not workable, we can just make it warn all the
time.

Change-Id: Iaaf56894060a58f2dfc78254bb60b1c43482f9bb
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-10 20:22:47 +02:00
Raito Bezariusandjade 36d69864f3 chore: re-work the contribution guide
As per our bootstrap governance discussions, here's a very simple
proposal which links as much as possible to our wiki.

Change-Id: I88b1c43f933ff7e529151b1e933fad40283383c4
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-05-07 01:28:35 +00:00
Raito BezariusandGerrit Code Review 55350bd68d Merge "feat: unprivileged read-only open of SQLite DB" into main 2024-03-29 00:49:17 +00:00
Raito Bezarius 80b66b5065 libstore/filetransfer: use Lix UA and unnix error message
Once this commit lands, we are even more visible in analytics FWIW.

Change-Id: Id7e0c162315d0f191edbea9cb5fb82ce363704b9
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-26 16:06:27 +00:00
Raito Bezariusandjade ad8a4b380e libmain: version printer uses Lix instead of Nix
Change-Id: I014ff24b900c0b9a48b7a63c8bb8b86cde3ebe54
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-25 08:04:31 +00:00
Raito Bezarius 8044540c42 feat: unprivileged read-only open of SQLite DB
If the state SQLite database is configured to use a write-ahead-log, it
creates WAL files in the state directory.

When the state SQLite database is closed by the `nix-daemon` after
builds, those files are removed.

When an unprivileged user would like to open _in read only_ that
database, they cannot do so because they would need to create those WAL
files and they do not have the permission to do so.

For this, SQLite offers a "persistent WAL" feature [1] to leave the WAL
files around, even after closing the database.

This CL enable the persistent WAL mode.

Fixes: https://github.com/NixOS/nix/issues/10300
[1]: https://www.sqlite.org/wal.html

Change-Id: Id8ae534d7d2290457af28782e5215222ae051fe5
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-23 15:07:48 +01:00