Historically, Nix would support copying certificate authorities inside
the sandbox so you could use them.
In addition to that, the primitives consisting of leaking environment
variables via `impureEnvVars` and `extra-sandbox-paths` to render paths
external to the sandbox visible to the builder would also constitute a
mechanism to expose special inodes which should have no influence on the
output result, e.g. interception CAs.
Unfortunately, in nixpkgs, `lib.fetchers.proxyImpureEnvVars` set
`NIX_SSL_CERT_FILE` as an impure environment variable.
A confused user may set `ssl-cert-file` via `NIX_SSL_CERT_FILE` outside the
builder believing that this will set magically the right
`NIX_SSL_CERT_FILE` inside the sandbox, but this is not true.
The combination of impure environment variables and setting `caFile`
creates a weird interaction where `NIX_SSL_CERT_FILE` points to an
"outside the builder's world" inode *AND* `ssl-cert-file` creates
this very same certificate file in /etc/ssl/certs/ca-certificates.crt
without rewriting the environment variable.
This footgun is closed by making these two features mutually
incompatible with a warning and forcibly rewriting the SSL family of
environment variables even if it was set via impure environment
variables.
Users who truly meant to use `impureEnvVars` can obtain the right
behavior by setting `ssl-cert-file` to an empty string and will have to use
`extra-sandbox-paths`.
Users who meant to use `ssl-cert-file` will have everything work
automatically with a warning hinting at nixpkgs *fixing its own bug*,
i.e. passing `NIX_SSL_CERT_FILE` as an impure environment variable and
expecting the Nix interpreter to magically reconcile the diverging
values or expecting the user to actually do the work to render the path
visible available via `extra-sandbox-paths`.
Fixes#885.
Change-Id: I32f8b5ce20fe9b6a911768114c92f95fc886cc07
Signed-off-by: Raito Bezarius <raito@lix.systems>
Sometimes, `bindPath` will detect the source is a symlink and we are not
using the new mount API which support symlinks (kernel ≥ 5.12 IIRC?).
In those instances, we copy the inode to the target.
But some callers may want to follow the symlink in such circumstances,
we add a new default argument to the previous value and let caller
decide for themselves.
Change-Id: I8505b613fc614ce539eb89258fbbb7eaecebe23b
Signed-off-by: Raito Bezarius <raito@lix.systems>
`pathContentsGood` is used to assess the validity of a path as part of
derivation goals *in repair mode*.
When repair is used with a diverted store, i.e. a store where
fsPath(toRealPath(s)) != fsPath(s) for s a store path, this result in
utterly broken behavior because it will attempt to assess the goodness
of the *logical* store locations, most of the time: /nix/store/...
So, if you are repairing your system using a live NixOS ISO. Your ISO
contains a `/nix/store` (assumed to be good) and you repair your system
which is rooted at /mnt and contains its own /nix/store, that is, a Nix
store at /mnt/nix/store.
Performing the following operation `nix-store --verify --repair --store
/mnt` will assess the contents goodness of the ISO's Nix store.
To avoid this, we assess the path existence of the *physical path*, aka
the result of `store.toRealPath` applied to a *logical* store path
string representation and we verify the hash of the *physical path*.
The error messages are not taken care of in this CL as those are purely
cosmetic and helps the user understand what is going on.
Fixes#892.
Change-Id: Ib9e0153cb5683edcf37f1963ebf065ceba5e5dfb
Signed-off-by: Raito Bezarius <raito@lix.systems>
Instead of showing logical Nix store paths, we show the actual physical
location so that the user can stat by copy-pasting these paths.
The bad thing about this change is that certain Nix porcelain only
accept their logical counterparts.
Change-Id: Id0eb45d1bf08a23508dfc2bb694c88155654f585
Signed-off-by: Raito Bezarius <raito@lix.systems>
When a user runs a repair or check sequence, they might be confused of
seeing references to a logical /nix/store path rather than the actual
physical store location.
Change-Id: I042031a6159cdd1054e7e200a220bf6c321d5fb8
Signed-off-by: Raito Bezarius <raito@lix.systems>
jade: I rewrote this PR to be consistent between nix-instantiate and nix
eval. It turns out that nix eval *doesn't* copy to store with `--json`,
whereas nix-instantiate does. Wat.
Closes: https://github.com/lix-project/lix/pull/17
Co-authored-by: tyberiusprime <tyberiusprime@noreply.git.lix.systems>
Change-Id: Id22deec1cee0fed3bd5689567869b70bab26bae5
Originally, libboost_context and dependent dynamic libraries have been copied
into the nix package to remove boost from the closure and consequently reduce
the closure size. Since commit ef0de7c79f we do
not depend on Boost coroutines any more, so these libraries are not needed at
all any more and (somewhat ironically) only increase the package size. Remove
them.
Change-Id: I6a6a6964dc3e0b29dfad8b2b232b428ba3cc653c
Meson conveniently does lets you pass feature objects to `required :`
arguments[1], which is handy
[1]: https://mesonbuild.com/Build-options.html#features
Change-Id: I54194b235a9b3dc207f3f78e0a8c50f957e1fd1f
The issue requiring these environment variables to be set for Meson to find
Boost [1] has been fixed [2] for quite some time now. Drop them since they are
unnecessary in all supported nixpkgs releases (in fact they have already been
removed in the lix package there).
[1] https://github.com/NixOS/nixpkgs/issues/86131
[2] https://github.com/NixOS/nixpkgs/pull/315998
Change-Id: I6a6a69640a30e917cd7a983b9d846d023b393dcd
In certain scenarios, a path may exist but is a broken symlink. For
instance, this happens frequently if you are rescuing an existing NixOS
system via `nixos-enter` or a manual `chroot` invocation because the
rescued system `/etc` may have broken links or the system prior to the
root pivot may interfere.
Nonetheless, these dangling symlinks are not always fatal for the builds
so we can just warn and skip their setup.
The warnings will provide a better diagnostics for system administrators
also.
Fixes#893.
Change-Id: Ifa12be3a43f23c973d7b466e8b73bd776abf3e7b
Signed-off-by: Raito Bezarius <raito@lix.systems>
We were catching ForeignExceptions believing it came from the TRY_AWAIT
handler, but this was misguided.
`j.dump()` is evaluated in synchronous context, outside of the `try {
... } catch (...)` block from `TRY_AWAIT`.
Therefore, we need to use `JSON::Exception` directly.
The previous test case did not catch it because:
(1) https://git.lix.systems/lix-project/lix/issues/865 hid the fact that
`--arg` was wrong.
(2) we did not grep for the warning because… we were not even copying
the strange store path to the binary cache.
(3) checking for the NAR happened after the NAR directory was emptied
for test reasons and this was not even caught neither.
Anyway, the test case was completely busted and has now been tested
without this commit and after this commit and we can confirm that prior
to this commit, the test will fail with an exception trace.
Co-authored-by: Maximilian Bosch <maximilian@mbosch.me>
Change-Id: I8df5befd06c4a449072b987f82a67bc4437e7e49
Signed-off-by: Raito Bezarius <raito@lix.systems>
PushActivity does not work with async code since we have no such thing
as promise-local storage. it will be confusing at best, and completely
wrong at worst, with the current thread-local linking state. if we can
find a way to get promise-local storage we may want to bring this back
though, explicit context passing is rather error-prone. luckily we are
not using parent links for anything important, just to keep the multi-
line activity display from filling up with stuff we're already showing
Change-Id: Ie373d713080a3db811b2d5abd681f78137735e45
checking that the remote build actually succeeded only implied-trusted
remotes or CA derivations makes *absolutely* no sense. we should check
that builds have succeeded before trying to copy them from the remote.
Change-Id: Ib2cf216c580f4c577dd9fef8849acc033ae082b9
Including the `.cc` is possible but is suspicious.
Change-Id: Ie18fef1e30e517edff4ab96f4a9c339e2b3145b5
Signed-off-by: Raito Bezarius <raito@lix.systems>
postInstall needs to know the name of the (versioned) .so file since it
needs to be renamed for Darwin platforms. Unfortunately, the .so version
is not properly overrideable, so we need to use string replacement.
Change-Id: Idf9671f84fac955a52d82a20ec0f381d05fdc762
lowdown 1.4.0 changed the lowdown_opts to include a new and separate
lowdown_opts_term which allows for configuring values specific to
-Tterm (which we're using). This version should have been called 2.0.0
according to semver, hence 2.0.0 was released later without any actual
breaking changes to sort of migitate the problem.
We need to support lowdown >= 1.3 && < 1.4 since the ship has sailed for
updating lowdown in NixOS 25.05 as well as lowdown >= 1.4 or we'll be
stuck in Nixpkgs forever. Support for < 1.4 can be dropped as soon as
NixOS 25.05 is EOL, assuming this change lands before NixOS 25.11
branch-off.
We detect the changed API based on the lowdown version from pkg-config
and define LOWDOWN_SEPARATE_TERM_OPTS based on that. The ifdef is named
according to the specific API change that impacts us, so that it's
hopefully a little simpler to maintain going forward. In the new API,
all newly configurable settings use what would have been the (implicit)
default before. Changing some of these values, especially hpadding,
could be interesting in future changes.
Compared to cl/3081, this change makes sure to initialize all new fields
of lowdown_opts_term explicitly.
It seems that, while making -Tterm more configurable, lowdown's word
wrapping behavior changed slightly which broke basic_repl.test. I've
chosen to work around this by using builtins.add as an example which has
a very short documentation string, so wrapping doesn't matter.
Change-Id: Id73be4c0e43d7eb4f56e10a261b4254402698ff8
capnp does not handle fd passing correctly in all circumstances. we hit
such cirumstances when passing large closures path lists to build-hook.
since capnp seems to ignore fds passed in non-final segments of any rpc
message we just ensure that the capability including the log fd will be
small enough to not be fragmented on the receiving side of the channel.
cf https://github.com/capnproto/capnproto/issues/2359
Change-Id: Id22309264936b3a57bcc68a0753c3bfb3c9a43d2
Some users may have arbitrary needs to connect to their store URIs, e.g.
mTLS authentication, Kerberos authentication, custom renewal using any
RPC mechanism of their preference and so on.
To avoid encoding all these patterns in Lix itself, we push the
configuration to the plugin boundaries and offer a hook for end users to
inherit from `HttpBinaryCacheStore` and provide new store schemes like
`https+mtls://my.very.secure.cache?tls-certificate=...&tls-key=...` or
`https+krb5://my.kerberos.enabled.cache`.
Co-authored-by: George Shammas <george@shamm.as>
Co-authored-by: eldritch horrors <pennae@lix.systems>
Signed-off-by: Raito Bezarius <raito@lix.systems>
Change-Id: I79f322b1a74632500fc79d53f5c920f9e43fd0c4
Usually, EOFs are represented by returning 0 in the `read` APIs, at
least, this is what read(2) dictate.
As clever creature, we may sum zeroes sometimes (advanced form:
`buf->added(got)`) and forego handling the EOF condition.
To avoid the bug that lurked in remote-store.cc and caused busy looping
if the remote end disconnects suddenly, we return
`Result<Option<size_t>>` forcing the caller to perform a specific
processing for the EOF situation.
The conversion did not raise any other offending code path.
Change-Id: I185fdcb77aa82d87ab0802d66ac37c1363657a73
Signed-off-by: Raito Bezarius <raito@lix.systems>
Also adds an assert that store path hash part length is what we expect
because it's alarmingly easy to forget to truncate a hash before
throwing it into there. It's kind of messy code, someone could improve
it more later.
Change-Id: I5296ea3d5b854323d092f0256defb598dd5b87e8
remote builds failures used to be signaled via exit status 1 of the
build hook, which in turn only happened because the build errors we
got from remote stores was thrown and bubbled up to main which then
logged the error and exited with code 1. with rpc we cannot do this
any more. barring a rewrite of the worker infra to allow for errors
being reported with something other than process exit codes this is
the best can do. ideally we would wrap remote builds in a new goal.
(and then remove all exit code shenanigans from DerivationGoal too)
fixes#928
Change-Id: Idc3ede3cbaca34c8c8e40247da52794f2a5013b9
drop our reimplementation of splice for non-linux in favor of using kj
pumpTo. this avoids select() for its O(maxfd) behavior, and if kj ever
uses something more efficient than read/write loops we'll benefit too.
Change-Id: Id01ba84bf8831455af2d9755bf1a3039d215bb47
libarchive *should* not break with 0710 on the tmpdir root on darwin,
just like it doesn't break on linux, but for some reason it does. the
restriction to 0710 can be weakened to 0750 with causing any trouble.
fixes#921
Change-Id: Ia9fc2f8eb9695fc19cefae9857368d5a4e58c8b9
previously we only had one build hook in waiting at most because build
hook rpc was synchronous. now that it no longer is we attempt to start
one hook per derivation, which depending on scheduling can be a *very*
large number. restrict the waiting hook count to 4 to some concurrency
without collecting a large number of hooks that may never do anything.
Change-Id: Ic0b1125cec4acd69e8a0d4639c232e71b825e01d
although we only chown if the build was requested by a local daemon
user. daemonless invocations will not chown as they do not have to.
remote builds *can* chown to the remote builder user, but that does
not seem to happen (for some reason keep-failed is not propagated).
Change-Id: Ic0ead406b38b4ca0556fec42d84888efa25123bf
this makes the actual build directories used by builders invisible and
inaccessible to other processes on the system, avoiding another vector
for outside processes to interfere with builds or pass credentials the
build sandbox should not have access to into the build sandbox anyway.
fixes#919
Change-Id: Ifaa4d8e3940cfde1406e925f75c1375d2e86d81a
this touches both libutil and libstore because with no rpc users it
doesn't make that much sense to separate the two. note that all our
strings are represented as Data (ie, blobs) because capnp Text must
be nul-terminated. while it's technically possible to use Text with
strings containing non-terminating NULs it is a bit of a hassle and
could lead to rpc users erroneously stopping at the first NUL byte.
Change-Id: I4c75e03b79a226ffa8d7cd985e3ac632a0cd7c1c
we need this to generate dependency information, and it'll be the entry
point for custom codegen once we need it. a wrapper also makes it a lot
easier to generate a whole namespace's worth of rpc definitions at once
Change-Id: Iba7a1c92a8a40bede9ed71aa3ab455477ff5e568
if the hook accepts the build request we can handle the entire request
in tryBuildHook. there is no need to punt a partially handled build to
the caller (we only did this to minimize churn during asyncification).
Change-Id: Iec3e35a8103da4fc5fbef394cc28a134ee62a198
mapping the result of an await operation before unpacking it lets us
inject rpc type conversion functions without duplicating all that is
needed for proper exception wrapping and async error traces support.
Change-Id: Ibcba1cc6d2b275757e3475881ef20f95dd4d684f
Goals:
- Distribute reviews to people who can do the reviews
- Not prevent anything from getting done
- Allow giving away more commit access
Anti-goals:
- Silo people into particular areas
- Discourage contributing to any area
This was drafted by glancing at git logs. It is not likely to be very
accurate; the goal here is that we figure out a way to distribute
reviews to the right people.
Change-Id: I8be44bf7fdeca23da8099124eec7bc3a30e34627
`Outcome<void, T>` and `Result<std::optional<T>>` can be interpreted as
being the same thing, but the latter is easier to use: not only do they
allow TRY_AWAIT usage for their promises, we also don't have the error/
exception confusion of outcomes (where the T above is the "error" type)
Change-Id: I92c9241481cecc97e2992445b3dced53c82a2524
while this does require spawning a thread for every contended lock now
we don't expect performance to be impacted. only build-remote used the
synchronous method, and it only used it to serialize uploads to remote
builders. these uploads are expensive enough to dwarf the thread cost.
Change-Id: Iad0aa0cd738bc96fd06a90d655803dadffa09c47
Jade Lovelace