Under macOS, the first level of directory has actually mode 0755 instead
of 0700 as macOS often do not possess the right primitives to chroot
inside of these directories, leading to
https://github.com/NixOS/nix/pull/11031.
Thanks to Emily for the heads-up on this type of matter.
Change-Id: I9d4e53717f61c9d573ff176f820610612804fbc3
Signed-off-by: Raito Bezarius <raito@lix.systems>
If `settings.buildDir` cannot be written to, because we are in a chroot
store, unprivileged or anything.
We can and should always gracefully fallback to a *secure* location
inside of /tmp, i.e. `/tmp/<a directory under 0700>/<our temporary
directory for build under 0700>/...`.
This does not reintroduce CVE-2025-52991 because we are creating a
directory in-between compared to creating only ONE level of directory.
Fixes#876.
Change-Id: Ie521202923f763225e1901ab1b9b6c6132aaf548
Signed-off-by: Raito Bezarius <raito@lix.systems>
eagerly consider outputs as not needing deletion during output
registration rather than only doing so after registration. not
waiting for registration to succeed may keep store paths alive
in the file system if registration fails for some reason; that
seem preferrable to the possibility of having another instance
of this bug. since we only leave *good* outputs around there's
not much to worry about except maybe bit of wasted disk space.
fixes#883
Change-Id: I8c22c92e39b9e203f1061278f86cde19dc4474a4
the daemon must use real store paths, not virtual store paths. using
virtual paths may inadvertently delete paths in the system nix store
when a build was run on a redirected store as root, which isn't good
Change-Id: Id048b236bda0e0ab1f3be6ccba0ddc1de2a3e941
Running (parallel?) nix in nix can lead to multiple instances trying
to create the state directories and failing on the createSymlink step,
because the link already exists.
`replaceSymlink` is already idempotent, so let's use that.
See also:
- https://github.com/NixOS/nix/pull/13368
- https://github.com/NixOS/nix/issues/2706
Change-Id: I7fadd0ce3c1ffcebc9d281c00e5b49c12af3d50b
In the past, it tried direct access if it *could* [1] perform direct
access.
This solves a bunch of errors people had when they tried the cgroup
feature and their scripts did not pass NIX_REMOTE=daemon manually
(nixos-rebuild-ng, home-manager activation from a root systemd unit,
etc.)
To avoid looping infinitely while receiving daemon connections, we
forcibly change the store URI when forking for a subdaemon to do direct
access automatically, this doesn't break forward usecases where you
point a daemon to another socket because we only change NIX_REMOTE="",
NIX_REMOTE=daemon, NIX_REMOTE=auto to a local and direct access.
All these usecases would end up infinitely looping no matter what
settings are set, because we are also responsible for creating the
daemon socket.
[1]: this happened all the time if you were `root`.
Related: https://github.com/NixOS/nixpkgs/pull/415701
Change-Id: I783fc795a9c2ee25b3d9f44f453f8f94b063371f
Signed-off-by: Raito Bezarius <raito@lix.systems>
killing a cgroup via `cgroup.kill` is not synchronous, we need to give
the processes in the group some time to wake up and exit. due to a few
historical accidents in the codebase we cannot do this asycnhronously,
e.g. with a kj promise without creating yet more problems. we will, at
some point in the future, have to move cgroup management into the main
daemon rather than doing it with RAII wrappers within every subdaemon.
Change-Id: I03bf9060144b5737729f2b05c25771c674fd154c
The reason this gets hit is because of the debugger in flakes. Otherwise
you never have a repl in pure mode anyway.
We evaluate the repl-overlay file in impure mode but this doesn't do
what one would initially expect.
Fixes: https://git.lix.systems/lix-project/lix/issues/777
Change-Id: I19b8ed2f5e9ce500b633b13301b42df69ab7deb3
idk how this mistake happened but it was really confusing to figure out
which one of these was right, so let's get rid of the impostor.
Change-Id: If3b6fb543e5976b1edad68fb143bfa994d1d6381
When a build fails, its scratch output paths are not cleaned up.
Until recently, this was deemed not a problem but as part of the effort
to harden the Nix builds and protect these paths against being part of a
staged attack (race conditions, etc.), we automatically cleanup after
failed builds.
Fixes CVE-2025-52992.
Change-Id: I58481b1cc83826298b9d80d37fecf81f117ccb09
Signed-off-by: Raito Bezarius <raito@lix.systems>
if a build directory is accessible to other users it is possible to
smuggle data in and out of build directories. usually this ins only
a build purity problem, but in combination with other issues it can
be used to break out of a build sandbox. to prevent this we default
to using a subdirectory of nixStateDir (which is more restrictive).
Fixes CVE-2025-52991.
Change-Id: Iacfc9b50534de158618c815f9fb99d7dae1be4d0
This allows using a userspace program, pasta, to handle comms between
the build sandbox, and the outside world; allowing for full isolation
including the network namespace, closing the "fixed-output derivation
talks to the host over an abstract domain socket" hole for good.
Fixes CVE-2025-46416.
Co-Authored-By: Puck Meerburg <puck@puckipedia.com>
Change-Id: Ifd499b7dbb3784600a6e842fede65fc031ff9f15
When calling `_deletePath` with a parent file descriptor, `openat` is
made effective by using relative paths to the directory file descriptor.
To avoid the problem, the signature is changed to resist misuse with an
assert in the prologue of the function.
Fixes CVE-2025-46415.
Change-Id: I6b3fc766bad2afe54dc27d47d1df3873e188de96
Signed-off-by: Raito Bezarius <raito@lix.systems>
This ensures that `passAsFile` data is created inside the expected
temporary build directory by `openat()` from the parent directory file
descriptor.
Fixes CVE-2025-52993.
Change-Id: Ie5273446c4a19403088d0389ae8e3f473af8879a
Signed-off-by: Raito Bezarius <raito@lix.systems>
`writeFile` lose its `sync` boolean flag to make things simpler.
A new `writeFileAndSync` function is created and all call sites are
converted to it.
Change-Id: Ib871a5283a9c047db1e4fe48a241506e4aab9192
Signed-off-by: Raito Bezarius <raito@lix.systems>
We use it immediately for the build temporary directory.
Change-Id: I180193c63a2b98721f5fb8e542c4e39c099bb947
Signed-off-by: Raito Bezarius <raito@lix.systems>
We now keep around a proper AutoCloseFD around the temporary directory
which we plan to use for openat operations and avoiding the build
directory being swapped out while we are doing something else.
Change-Id: I18d387b0f123ebf2d20c6405cd47ebadc5505f2a
Signed-off-by: Raito Bezarius <raito@lix.systems>
This is useful for certain error recovery paths (no pun intended) that
does not thread through the original path name.
Change-Id: I2d800740cb4f9912e64c923120d3f977c58ccb7e
Signed-off-by: Raito Bezarius <raito@lix.systems>
This was probably a typo introduced in
7453e2979f.
Unfortunately, AWS SDK is so well made that this typo became an assert
error in production.
AWS Outcome constructors contains
```
// Move error from other type of outcome
template<typename RT, typename ET,
enable_if_t<!std::is_convertible<RT, R>::value &&
std::is_convertible<ET, E>::value, int> = 0>
```
which means that when:
* RT → R is not possible (e.g. PutObjectOutcome → HeadObjectOutcome)
* ET → E is possible (e.g. S3Error → S3Error)
Then, we will instantiate the error-moving outcome constructor which
asserts `!o.success`… Though, the original outcome indeed succeeded.
Change-Id: I3809514ae0648e8c02b0f93fa64d91115a091cd9
Co-authored-by: Maximilian Bosch <maximilian@mbosch.me>
Signed-off-by: Raito Bezarius <raito@lix.systems>
We clarify that the *remote* daemon is too old. Otherwise it can be a bit confusing since you might have a local daemon as well, and it's not clear if the error is coming from the local or remote end
Change-Id: I17344c6f59bd7e0e62960c0025184d72ec3f012b
Step two for #496.
The idea is to allow `nix-build --arg config.allowUnfree true` do the
right thing in the future. However, that's a breaking change since
people might be relying on the ability to set `"config.allowUnfree"` as
attribute-name when auto-calling a Nix-expression.
As a first step, a warning got introduced in 2.92, the next step is now
to reject this usage in 2.94 and await feedback if any so that we can do
the change in a future Lix release.
Change-Id: I6e38fafe26e234204f5bba2a3a4c1da10f80e5f2
This introduces three new things:
* `handleException` which prints out exception details and its stack
trace.
* `handleExceptionWithAsyncTrace` which does the same, but also prints
the async trace if any.
* `LIX_BLOCK_ON` which is awaits a promise and adds an exception trace
if an exception got thrown, similar to `LIX_TRY_AWAIT`. However, this
is not supposed to be used in async functions, but on callsites of
`aio.blockOn()` which is especially useful for Hydra[1].
For `LIX_BLOCK_ON` I had to introduce another function because there's
apparently no way to implement all of it in a macro: on macros with
compound statements the return value must be a trivial expression at the
bottom, i.e. no `try`/`catch`. Now, returning the value from the
`try`-block requires the variable to be defined up-front, but for that
we'd need to know the type-name. Hence the construction with a
template-function being invoked by a macro that injects the current
source-location.
[1] https://git.lix.systems/lix-project/hydra/pulls/52
Change-Id: I56cc92c94f7e8f0be5d4dc5a7d8cb21a92e776ef
Added an additional check that all files present within a folder must be
used/referenced. Otherwise an InvalidLangTest will be created.
This ensures that there weren't any mishaps while migrating tests
resulting in files being ignored and hence some tests not being run.
Fixes: #852
Change-Id: Ie096c5670bc20325ba72c7d6ce33c06667c66ab1
Redesigns the test.toml to use a list instead of a directory
additionally it is now possible to do toml and matrix tests on singular
files as well as on a subset of files.
Fixes: #851
Change-Id: If8635109c6274f406ad68fe35315b9125f45f67d
Currently when a lang test fails, (or any snapshot assertion for that
matter) the error message is rather bulky.
This is due to both sides being printed fully, using escaped newlines
(i.e. everything is one line)
This is awful to read and check what the actual difference is. Also
there is no indication that one can update the golden files using the
cli flag.
This commit changes the error message when comparing snapshots against
something
a list of lines is shown, where the output differed. An additional note
about how to update the files automatically was added too
Change-Id: Ibedcf48018c27f924b807fbd42362fb608d27441
we no longer use thread pools for querying missing derivations. this
binds queryMissing to a single thread for now, but query performance
is still greatly improved. we may want to optimize the store code in
the near future too though since queryMissing is now fully cpu bound
Change-Id: I08a9c8cc199963ef5981572ca4a32d90dbdec028
we intentionally omit writers for the new types we add for serialization
purposes since we do not plan to asyncify the legacy ssh server side. if
we ever change our mind we can extract these types into a header and add
writers as needed. due to the inevitable network overhead of the old ssh
wires we don't bother to optimize serialization too much and instead opt
to make the code more readable; the performance difference does not show
up in practice since network latency dominates the few nanoseconds spent
on extra promise allocations and awaits by a couple orders of magnitude.
Change-Id: Id3ee9a01f8bfa63fa23082fa07de5c673fd70883
protocol version 0x204 dates back to nix 2.0 in 2017. that's old enough
to not worry and drop the gratuitous assertion crash we see it instead.
Change-Id: I8cf23373d4daabccab61f1cbb670947479f0d2bc
When Ctrl-C is sent to the workload, even across remote builds, the
whole process possess a global flag `_isInterrupted` which is checked in
certain filesystem operations, cancelling them, e.g. writeFile will
write nothing under interruption unwinding.
In addition, if any operation throws an exception before we `rmdir` the
cgroup, we may leave it hanging while we remove the state record.
Therefore, we put the final cleanup in a block.
In practice, reading statistics could lead to failures.
Control groups cleanups are critical though and should always be
performed.
Change-Id: I48fa87317b6a9f6663559bc8fa5f8a897f37011e
Signed-off-by: Raito Bezarius <raito@lix.systems>
this is a large step towards making RemoteStore a proper capnp rpc
interface, and it lets us get rid of the RemoteStore error handler
thread pool. this does mean we make six or more extra syscalls per
operation to set and clear socket non-blocking flags, but they are
pretty cheap compared to cross-thread wakeups and scheduling. once
we have real capnp rpc for store wires we can drop them again too.
Change-Id: I67dfebc8644a407cd4a8221ffcad02a938ac5abe
in the future we will want to instantiate either a sink, a source, both,
or streams, depending on how the fd is used. to do this we need to share
read buffers among sync and async readers. removing the FdSource we kept
in the connection also helps prove that we always use this buffer for io
Change-Id: Ib678e128ed6c4a07d6ce5ec1d3cde9eb3f5fc4ca
we don't need to double-buffer commands. only the subframe protocol
needs a buffered backing, and connection setup is special *anyway*.
Change-Id: I596f2bf8e297c3c5dc2befae674deafcf559d9a9
this may as well be called AsyncSocketStream since that will be what we
use it for, but hopefully it will not exist for long enough to need any
other socket functions to actually justify such highly specific naming.
Change-Id: Icf2fe88cf345405218e4b1bd440267e7f132f5c7
we also extend AsyncInputStream with a drainInto variant to give async
output streams rough feature parity with sync sinks. we still will not
add serialization support to streams though, that's far too expensive.
Change-Id: I60d5ab43610c45a40ea8740470a5eafe68064aea
otherwise stores containing async objects will cause crashes during
shutdown. currently there are no such stores, but that will change.
Change-Id: I05d46ba6831c641774edfe6aa99aa7d0de457429
store objects may hold on to network connections. if those connections
are async they're bound to the lifetime of the aio runtime, which ends
long before the static object destructors we need for nix-store today.
Change-Id: I4aa5466681a82f7e5008cc0b952fcba01d5b39d7
do not rely on Source/Sink `good()` or delayed guessing about whether
an exception was thrown by the daemon or not. mark connections as bad
for all local errors happening while communication is ongoing instead,
and leave it valid only when an exception was provided by the remote.
we may drop connections a bit too eagerly now, but all cases in which
that happens were vulnerable to protocol desynchronization. there are
still a few windows for this to happen left, but those are unfixable.
Change-Id: Iefaa66c552092c436b9de77aa3f8e09f847a966e
once we make our socket fds non-blocking we won't be able to easily use
plain FdSink for serialization. performance impact of using a temporary
buffer should be low since we don't send very many messages and even in
the simple local daemon case networking overhead is already quite high.
Change-Id: I550d73142570b7d2e7b0feb1bcc57d61e9b45178
we will need this during RemoteStore wire asyncification to be able to
use the old synchronous serializers. alternatively we could define all
serializers on the async types as well, but that'd be slow and far too
much unnecessarily duplicated code (that will be deleted soon anyway).
Change-Id: I6e4f334025844b808a697ddcd8f80ddcd8c3fc9c
it was never safe. both discarded the buffer of the source object,
possibly leading to silent data corruption. FdSource discarded the
fancy EOF error string as well, possibly causing bad error reports
Change-Id: Ib5c07986471b5af03d707230cd487259201952e9
Raito Bezarius