we do not need explict redirection management now, and dieWithParent
doesn't need an override either. we'd much prefer to kill ssh if the
process in charge exits; even ssh multiplexers are not fazed by this
Change-Id: I81e28b7605df73c887878ea4716228c7ad0f5c6f
We always use the default temporary directory, because
`createUniqueDir` has an interface nice enough to use directly for
the few bespoke uses.
Change-Id: I6a6a696450b7c0a0bd76655632fb14d7c5e38199
This makes the paths more nondeterministic, but more reliably unique,
and lets us remove the retry loop.
Note that this adds random entropy to the build directory visible
inside derivations on Darwin and unsandboxed Linux. It was already
non‐deterministic in the presence of concurrent builds and similar,
but now we can reliably expect it to be different every time. On the
whole I think that’s a good thing, as it is impossible to ensure
a single consistent build directory and derivation outputs should
not depend on it.
Package reproducibility isn’t great on Darwin to begin with,
though, and the reproducibility bugs this will turn up in packages
will be more urgent to fix than when the build directory was mostly
consistent. A quick survey of my local store shows that many C, C++,
and Rust binaries contain build directory references, likely due to
use of `__FILE__` and its equivalents; non‐binary offenders include:
* Install logs included in the Rust and Cargo bootstrap compilers
* Example errors in the Rust documentation referencing build paths
* Configuration information installed with CPython itself
* Python 2 metadata from resholve’s closure
* Cython metadata
* Generated headers in Facebook libraries referencing source paths
* Generated CMake files in Facebook libraries referencing source paths
I haven’t built that much in this store since the last GC, so this is
probably only a small sample of the problems across the tree. These are
all instances of <https://reproducible-builds.org/docs/build-path/>,
though, and should probably just be treated as general reproducibility
bugs outside of contexts like the Linux sandbox where we can normalize
them away entirely.
I have implemented away build directory paths for C/C++, applied some
additional fixes for non‐`__FILE__`‐related issues in binaries
from ATF and LLVM, and fixed the derivation bug causing the CPython
3 issue, and will work on upstreaming these changes. Rust is working
on the problem upstream, with some temporary workarounds we can
potentially apply in Nixpkgs for now. The rest will require some
distributed effort.
Change-Id: I6a6a69645b4915c56c0fdef904aa81684e4136c6
Fixes: https://git.lix.systems/lix-project/lix/issues/973
Information about which commands were executed is really valuable to
debug Lix and is much more user relevant than the vast majority of the
e.g. build loop junk printed at debug level. Currently we have a *whole
lot* of call sites where we call execv* which should probably be cleaned
up, but that's future work.
I chose to print argv0 rather than the executable path if these differ,
since the code is shorter and since the command could be a fully
resolved symlink or so where argv0 is the actual command name being run.
However, it's not exactly *hard* to write std::ranges::drop_view(args,
1).
Change-Id: I73c3abb20b229d5e2d64277aa29cbbeed7764bab
We always use the default temporary directory, because
`createUniqueDir` has an interface nice enough to use directly for
the few bespoke uses.
Change-Id: I6a6a6964c15c31bb3e131fbe1db1837987a6d6dc
This makes the paths more nondeterministic, but more reliably unique,
and lets us remove the retry loop.
Note that this adds random entropy to the build directory visible
inside derivations on Darwin and unsandboxed Linux. It was already
non‐deterministic in the presence of concurrent builds and similar,
but now we can reliably expect it to be different every time. On the
whole I think that’s a good thing, as it is impossible to ensure
a single consistent build directory and derivation outputs should
not depend on it.
Package reproducibility isn’t great on Darwin to begin with,
though, and the reproducibility bugs this will turn up in packages
will be more urgent to fix than when the build directory was mostly
consistent. A quick survey of my local store shows that many C, C++,
and Rust binaries contain build directory references, likely due to
use of `__FILE__` and its equivalents; non‐binary offenders include:
* Install logs included in the Rust and Cargo bootstrap compilers
* Example errors in the Rust documentation referencing build paths
* Configuration information installed with CPython itself
* Python 2 metadata from resholve’s closure
* Cython metadata
* Generated headers in Facebook libraries referencing source paths
* Generated CMake files in Facebook libraries referencing source paths
I haven’t built that much in this store since the last GC, so this is
probably only a small sample of the problems across the tree. These are
all instances of <https://reproducible-builds.org/docs/build-path/>,
though, and should probably just be treated as general reproducibility
bugs outside of contexts like the Linux sandbox where we can normalize
them away entirely.
I have implemented away build directory paths for C/C++, applied some
additional fixes for non‐`__FILE__`‐related issues in binaries
from ATF and LLVM, and fixed the derivation bug causing the CPython
3 issue, and will work on upstreaming these changes. Rust is working
on the problem upstream, with some temporary workarounds we can
potentially apply in Nixpkgs for now. The rest will require some
distributed effort.
Change-Id: I6a6a69648f74d85c6fca86cc52f38fd957e4f9ad
remote store async io will need to set O_NONBLOCK on the connection fds,
and right now the number of fds can vary between connection types: local
connections have one one fd for the sink/source pair since they use unix
sockets, but ssh connections have two because ssh uses pipes. this makes
it rather hard to manage flags correctly, and even harder to wait for io
readiness on both directions using kj. using sockets for ssh fixes this.
Change-Id: I0f563ece7627cd3fbd0f5ce21c25140469729e5a
The pre-flight `echo started` check over SSH was originally added in
577ebeaefb. As it is usual with these old
commits, understanding why is there a need for something is difficult.
The closest thing would be
> Fix a race starting the SSH master. We now wait synchronously for
> the SSH master to finish starting. This prevents the SSH clients
> from starting their own connections.
But, we removed SSH connection sharing, so this does not apply anymore.
Nonetheless, we believed this check was meant as a way to catch obvious
misconfigurations or SSH failures early, before handing off to
`nix-store`. However, this approach was not fruitful: it assumes the
remote has a `bash`-compatible shell, `echo` behaves in a standard way,
and no `ForceCommand` interferes—all of which are unreliable assumptions
in practice.
While the intent was to provide slightly better diagnostics (e.g. in
case of SSH hanging or returning an interactive shell), in practice it
does not meaningfully catch or improve real failure cases. The
underlying protocol or engine can and should handle those errors more
robustly anyway.
In contrast, this check *does* break several legitimate workflows,
including:
* remote builders using `ForceCommand` wrappers (e.g.
`nix-remote-build`-style setups), see
<https://discourse.nixos.org/t/wrapper-to-restrict-builder-access-through-ssh-worth-upstreaming/25834/15>,
* SSHing into minimal environments lacking `bash` (e.g. initrd,
busybox-based systems),
* configurations that don’t default to POSIX-like shells, e.g., nushell
enthusiasts.
As such, we’re removing this code. Protocol mismatch errors and SSH
failures can be rethought and handled more structurally elsewhere in the
engine.
Change-Id: I187f6881375d42ef83987a13a350c97964bbdb30
Signed-off-by: Raito Bezarius <raito@lix.systems>
SSH runs the command provided to it in the user's login shell.
That's a feature and is impossible to bypass (trust me, we tried).
The previous implementation of `echo started; nix-store --stdio`
broke with shells that treat `;` differently, e.g. nushell,
which eats the output of everything but the last command in a chain.
In the more general case, this means that a sufficiently weird shell
can do _anything_ it wants to the command we pass via SSH, so
we're forced to rely on as little functionality as possible.
The subset we're hereby settling on is just "running `bash` runs bash".
We then run bash, in non-interactive mode, which gives us a somewhat
consistent environment do to things in.
This whole thing is extremely cursed, but fixing it _correctly_
requires pretty much entirely rewriting how remote stores work,
and I'm not doing that right now.
Fixes#805.
Change-Id: Icac846e8cd821cbca91860ddaa0f657b4317dbf8
Co-authored-by: eldritch horrors <pennae@lix.systems>
we no longer explicitly configure these ssh options, so we now
have *even less* reason to stick to old and awful terminology.
Change-Id: I8f38f5486e6918f69c61ddc2d6882b6bb50554c6
our implementation is mostly unused, completely untested, and simply
breaks when users configure connection sharing independently. we can
safely delete this "feature" and inherit user configuration instead.
also have the remote build test check that connection sharing works.
fixes#304, fixes#644
Change-Id: Iea44cc0f8e51a1d231ad186a95c7e310bbfeb303
This also adds documentation for it in distributed-builds.md as that's
possibly the most common use-case for remote ssh stores
Support for `std::optional<uint16_t>` in Setting is also added since
setting a port is optional.
The line `#include "lix/libutil/strings.hh"` fixes that templates instanciations
in lix/libutil/config-impl.hh were using string utils without including
the header (why are they even there btw)
Change-Id: Id806c117c48cdf158d9d1cb1e639b0df31d9bf11
clangd broke because it can't look through symlinks. compile_commands
manipulation does not fix it, clangd configuration does not fix it, a
vfs overlay does not fix it, and while a combination of those can fix
it with a bind mount in place that's just too cursed to even consider
clangd bug: https://github.com/llvm/llvm-project/issues/116877
Change-Id: I8e3e8489548eb3a7aa65ac9d12a5ec8abf814aec
eldritch horrors