The Node struct should become an implementation detail when moving this
into libstore. A map from a node to its direct references is more
intuitive here.
Change-Id: I9fddce6b398b8bb97834e5586bee72b244885fdd
* Better name for refs
* Use std::optional<T> for distance
Suggested-by: eldritch horrors <pennae@lix.systems>
Change-Id: Ie35c3f2a7ea1a90ce3a9807025d0af9ea73e2403
Instead of logging directly, we now write into a `Strings` set that is
referenced by the caller.
While at it, added a test-case to ensure that self-reference invocations
and --all behave properly.
Change-Id: Ib183ab8e8e90436300e1c870fb3ae8f18730abbf
That way we get a line of output per test completed,
which makes it more obvious it's actually doing things.
Change-Id: Ifbbe8bdf64e7178d3c59349cf071eb5a9d0fcd32
staging-next banned !structuredAttrs && separateDebugInfo && disallowedRequisites
due to weird output interactions. Enable structuredAttrs so we can build again.
Also, fix type confusion that makes stdenv explode (https://github.com/NixOS/nixpkgs/issues/422989).
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: Ic0c773394ee79e10d427f27750d59892d6d1f1d1
this partially reverts commit 0cc021ee15,
which for some reason is completely broken on darwin: there seems to be
no way to receive process-directed signals on a non-main thread. trying
to do it anyway will fail silently. since we only ever used kj for this
to get signal handling timeouts on darwin (which lacks sigtimedwait) to
print a nice message about retrying ^C again we can work around this by
moving the message printing into a fresh, unrelated, non-signal thread.
Change-Id: I5939c6ec62a7e1dc1b3f16067f77277533949fa0
We missed xokdvium being author on cl/3300. This is something we
absolutely want to avoid.
We credit xokdvium in the RL and add a note on this problem.
Thanks to xokdvium for reaching out in private to us so we can repair
this mistake.
Change-Id: I094d0f95b6647104621d6b228e69a4529a300304
Signed-off-by: Raito Bezarius <raito@lix.systems>
many a cleanup path has been broken by interruptions being thrown every
time checkInterrupt is called. we should only throw *once* though; more
than one Interrupted exception for the same event is not only confusing
but also breaks all cleanup paths at the first checkInterrupt call site
(e.g. #900, the cgroup cleanup saga, temp dirs not being removed, etc).
Change-Id: Ibfabf7f6af6ac2b78ad93582c254bbc48fcb3073
we must be crash-safe *anyway*, and being unable to interrupt lix if it
gets stuck somewhere that never calls checkInterrupt is really annoying
Change-Id: I7c40271c3da7e69d8735e22b7b7c4751b5306ab6
macos doesn't have sigtimedwait and we need signal wait timeouts in
order to print a "please hit ^C again" message with a bit of delay.
Change-Id: If574fb1a9de0b19975b34fc63662b089eaedc9d2
another checkInterrupt can be a makeInterruptible wrapper now. this is
also necessary to add a second daemon socket for the new rpc protocol.
Change-Id: I55055f975335a75708f1f73edb75f7bfe77a5938
it was only needed because we forked subdaemons and couldn't reuse the
main aio root. we now fork+exec, so the main aio root is always valid.
Change-Id: Ia19e20d52d65fe72721292be091f182a8a77a7cb
all uses are DoSignalSave::Save now, and introducing new DontSave uses
should be avoided as much as possible. process management is already a
mess, simplifying it somewhat will make our life easier in the future.
Change-Id: I77eecabe45bee9de18fba0dfc948403d3ce46dfe
this resolves problems with aio roots becoming invalid after fork (which
so far forced us to run the daemon loop in an aio-rootless thread), does
not require restarting the signal handler thread in the subdaemon (since
we no longer lose it), and is a step towards solving #18 (with transient
daemons doing the store manipulation started transparently when needed).
Change-Id: Iad0149cbc807e31964407c9a83d12314702c8122
posix_spawn unsets CLOEXEC for fds that are dup'd onto their existing fd
number. this is very useful when inheriting fd numbers exceeding stderr.
Change-Id: I6f14585d424ded6741fdd087f0c4d33a05936bcc
the `from`/`to` naming only made sense for unidirectional output fds,
for others (and for the dup2 api in general) it was backwards. rename
them to `dup`/`from` to make this look more like the assignment it is
Change-Id: Iee50d06f9cfcea765ace6cfbe85b192829207e5f
writing to non-blocking fds happens during remote builds due to the way
file descriptions are shared between processes. we can either poll when
writing to non-blocking fds are reset fd flags. polling is just easier.
unfortunately there is no reasonable way to test this that isn't flaky.
fixes#896
Change-Id: I1d8666df57da97199247f0770c547d0180f6ce07
This results in anything that uses nixpkgs getting stopped in the
debugger inside of nixpkgs internals, which are usually irrelevant.
Let's default to the more useful option.
Fixes: https://git.lix.systems/lix-project/lix/issues/666
Change-Id: If4b94a3d488bfb2f634ee5a2bc195e7a4b5434a5
cancelling the promise returned by makeInterruptible could free the
fulfiller before the interrupt callback handle, and no order of the
attachments made a difference. we must resort to putting fulfillers
into shared_ptrs so we can capture them in interrupt callbacks now.
(alternatively we could add another kind of interrupt callback, but
the complexity of doing that outweighs the cost of one shared_ptr.)
fixes#895
Change-Id: I008b160482fd4d81a29d7e9e452dcda858b090b9
download progress reports send a STDERR_RESULT frame. many concurrent
downloads send many STDERR_RESULT frames. each of these frames has us
run the report loop once. since many frames can happen in very little
time we may receive many frames in a single read from the socket, and
that in turn means we don't have to fcntl that socket on every round.
we must still ensure that the socket is in the correct state for each
part of the loop, and this does mean we may run two unnecessary fcntl
sequences per processStderr call. that's a small price to pay though.
Change-Id: I7af607d8c759b76aff0f6016435955e2f9456923
these are used often enough that deduplicating them is worth it. we do
lose some error fidelity, but valid fds will never cause an error here
Change-Id: I2b91b4848f546a894a2a6c2d36c32a892fb73c9f
it's only called by verifyStore, and verifyStore is only called by the
daemon and `nix-store --verify`. both pass the promise to `blockOn()`.
Change-Id: I829c0d189fa913cd8566ddd1a578c50e60fb2ddb
all of them block on a promise very soon after starting. only
queryValidPaths needs to make sure not to swallow Interrupted
exceptions to exit quickly instead of trying all paths first.
Change-Id: I4f99f5d75d7057bad109dc0131aa58e84275e362
checkInterrupt is cheap, waiting for a promise isn't. checking for
interruptions before any top-level promise is awaited lets us drop
a bunch of checkInterrupt calls elsewhere, such as in thread pools
Change-Id: Id543edf9411e53b2a5bbec77d3084a8f65aaea46
This reverts commit a0a00948df
because this was insufficient to fix the critical correctness bugs.
Change-Id: I6c7b560ebeebacbbbcc1cbf26e6ef50c38b84f7f
This reverts commit e356d54d7a
because this was insufficient to fix the critical correctness bugs.
Change-Id: I91c3e368ffd13ade6a3cebbbacdb42655796ea56
We’re already allowing `/tmp` anyway, so this should be harmless,
and it fixes a regression in the default configuration caused by
moving the build directories out of `temp-dir`. (For instance, that
broke the Lix `guessOrInventPath.sockets` test.)
Note that removing `/tmp` breaks quite a few builds, so although it may
be a good idea in general it would require work on the Nixpkgs side.
Fixes: 749afbbe99
Change-Id: I6a6a69645f429bc50d4cb24283feda3d3091f534
Using `AllowDaemon::Disallow` here broke `ssh-ng://` remote builds in
multi‐user setups where the remote builder user does not have write
access to the store, now that the automatic store selection logic
has changed. Switch to the default behaviour for this path to fix that.
This causes `ssh-ng://` builds to use the daemon by default on the
remote end, even as `root`. I think this is desirable, as the previous
change already made `ssh://` behave this way, and the pitfalls of
local stores apply to remote builds too. For instance, there were
persistent `ulimit` issues on the NixOS Hydra macOS builders that were
resolved by forcing use of the daemon, and I believe the Linux builders
also go through the daemon these days due to using non‐`root` SSH
users. I believe that the `root` vs. non‐`root` difference is just
as confusing for remote builds as it is for local ones.
`ssh-ng://root@builder?remote-store=local` can be used to revert back
to the previous default if necessary.
Closes: #884
Fixes: 9a59106c17
Change-Id: I6a6a696410f46cd3f2f5a94073ea924ad45dc99c
This allows other functions to parameterize over it themselves. An
enum class is used to avoid API misuse.
Change-Id: I6a6a6964d2b5ad47ae5ea9eb11af9b6373ce2141
Under macOS, the first level of directory has actually mode 0755 instead
of 0700 as macOS often do not possess the right primitives to chroot
inside of these directories, leading to
https://github.com/NixOS/nix/pull/11031.
Thanks to Emily for the heads-up on this type of matter.
Change-Id: I9d4e53717f61c9d573ff176f820610612804fbc3
Signed-off-by: Raito Bezarius <raito@lix.systems>
If `settings.buildDir` cannot be written to, because we are in a chroot
store, unprivileged or anything.
We can and should always gracefully fallback to a *secure* location
inside of /tmp, i.e. `/tmp/<a directory under 0700>/<our temporary
directory for build under 0700>/...`.
This does not reintroduce CVE-2025-52991 because we are creating a
directory in-between compared to creating only ONE level of directory.
Fixes#876.
Change-Id: Ie521202923f763225e1901ab1b9b6c6132aaf548
Signed-off-by: Raito Bezarius <raito@lix.systems>
eagerly consider outputs as not needing deletion during output
registration rather than only doing so after registration. not
waiting for registration to succeed may keep store paths alive
in the file system if registration fails for some reason; that
seem preferrable to the possibility of having another instance
of this bug. since we only leave *good* outputs around there's
not much to worry about except maybe bit of wasted disk space.
fixes#883
Change-Id: I8c22c92e39b9e203f1061278f86cde19dc4474a4
the daemon must use real store paths, not virtual store paths. using
virtual paths may inadvertently delete paths in the system nix store
when a build was run on a redirected store as root, which isn't good
Change-Id: Id048b236bda0e0ab1f3be6ccba0ddc1de2a3e941
Running (parallel?) nix in nix can lead to multiple instances trying
to create the state directories and failing on the createSymlink step,
because the link already exists.
`replaceSymlink` is already idempotent, so let's use that.
See also:
- https://github.com/NixOS/nix/pull/13368
- https://github.com/NixOS/nix/issues/2706
Change-Id: I7fadd0ce3c1ffcebc9d281c00e5b49c12af3d50b
In the past, it tried direct access if it *could* [1] perform direct
access.
This solves a bunch of errors people had when they tried the cgroup
feature and their scripts did not pass NIX_REMOTE=daemon manually
(nixos-rebuild-ng, home-manager activation from a root systemd unit,
etc.)
To avoid looping infinitely while receiving daemon connections, we
forcibly change the store URI when forking for a subdaemon to do direct
access automatically, this doesn't break forward usecases where you
point a daemon to another socket because we only change NIX_REMOTE="",
NIX_REMOTE=daemon, NIX_REMOTE=auto to a local and direct access.
All these usecases would end up infinitely looping no matter what
settings are set, because we are also responsible for creating the
daemon socket.
[1]: this happened all the time if you were `root`.
Related: https://github.com/NixOS/nixpkgs/pull/415701
Change-Id: I783fc795a9c2ee25b3d9f44f453f8f94b063371f
Signed-off-by: Raito Bezarius <raito@lix.systems>
killing a cgroup via `cgroup.kill` is not synchronous, we need to give
the processes in the group some time to wake up and exit. due to a few
historical accidents in the codebase we cannot do this asycnhronously,
e.g. with a kj promise without creating yet more problems. we will, at
some point in the future, have to move cgroup management into the main
daemon rather than doing it with RAII wrappers within every subdaemon.
Change-Id: I03bf9060144b5737729f2b05c25771c674fd154c
The reason this gets hit is because of the debugger in flakes. Otherwise
you never have a repl in pure mode anyway.
We evaluate the repl-overlay file in impure mode but this doesn't do
what one would initially expect.
Fixes: https://git.lix.systems/lix-project/lix/issues/777
Change-Id: I19b8ed2f5e9ce500b633b13301b42df69ab7deb3
idk how this mistake happened but it was really confusing to figure out
which one of these was right, so let's get rid of the impostor.
Change-Id: If3b6fb543e5976b1edad68fb143bfa994d1d6381
When a build fails, its scratch output paths are not cleaned up.
Until recently, this was deemed not a problem but as part of the effort
to harden the Nix builds and protect these paths against being part of a
staged attack (race conditions, etc.), we automatically cleanup after
failed builds.
Fixes CVE-2025-52992.
Change-Id: I58481b1cc83826298b9d80d37fecf81f117ccb09
Signed-off-by: Raito Bezarius <raito@lix.systems>
if a build directory is accessible to other users it is possible to
smuggle data in and out of build directories. usually this ins only
a build purity problem, but in combination with other issues it can
be used to break out of a build sandbox. to prevent this we default
to using a subdirectory of nixStateDir (which is more restrictive).
Fixes CVE-2025-52991.
Change-Id: Iacfc9b50534de158618c815f9fb99d7dae1be4d0
Maximilian Bosch