`xucred` already contain `cr_ngroups` and `cr_groups`, we just have to
use it properly.
Change-Id: Ic29b4c551f9d93e86e9ed2bef20889d27d5438f4
Signed-off-by: Raito Bezarius <raito@lix.systems>
Two things happens here:
- we consider supplementary groups that are known for authentication,
fixes#968.
- we check supplementary groups if they are our build users group and
throw an error if they are (new behavior).
Nonetheless, we did not remove the ability for `matchUser` to recurse
and check for groups that the user may have but the connection might not
have communicated as part of their groups, i.e. if a process reduces its
list of groups via a call to setgroups, it will still be authorized.
This will come in another commit.
The authorization NixOS test has been extended with a store ping test
running in systemd with DynamicUser=true *AND* a supplementary group in
allowed-users.
Co-authored-by: Tom Hubrecht <github@mail.hubrecht.ovh>
Change-Id: I25b2b8304d66a04651cea523b5585a5d15ceebe8
Signed-off-by: Raito Bezarius <raito@lix.systems>
On Linux, SO_PEERGROUPS is an option introduced in 2017, that is, in
Linux 4.13.
Thankfully, Lix targets 5.10+ kernels minimum.
It was chosen to allocate 128 gids by default and ramp up (2x) as needed
rather than allocate a full 65k of integers as it seems wasteful. I bet
the time to the 16 allocations should incur at most an additional
millisecond on a modern system, don't quote me on that though.
This is preparation to enable ability for the daemon to vet based on
supplementary groups.
Related to #968.
Suggested-by: alois31
Suggested-by: eldritch horrors
Change-Id: I26d698327db5d174bf70ca25b0afede132bd9169
Signed-off-by: Raito Bezarius <raito@lix.systems>
Rather than a bunch of booleans, use optional<T> as a way to convey that
the identifier is known or not.
This is preparatory work for SO_PEERGROUP usage.
Change-Id: I0e3ce69e4103f9df5a2161c695bfc650123c1549
Signed-off-by: Raito Bezarius <raito@lix.systems>
xattrs are revealing some unfortunate brittleness in real world
derivations that are getting -EINVAL errors while executing their test
code.
The reason for this is that Linux checks [1] UID delegations for xattrs
writes.
In the Lix sandbox, except if you enabled a uid-range feature, you have
exactly 3 UIDs: root, nixbld and nobody.
If your test code makes use of UIDs which have not been delegated, you
will receive an EINVAL on your operation. Test code is not resilient
with respect to the number of available UIDs in their namespace.
To avoid further issues for end users who are running into spurious
derivation build failures, we forbid xattrs again for now.
For more information about the plans, please consult or chime in [2].
Fixes#1105. Reopens#838. Fixes#1103.
[1]: https://elixir.bootlin.com/linux/v6.18.6/source/fs/posix_acl.c#L257
[2]: https://zulip.lix.systems/#narrow/channel/9-Store/topic/disablement.20of.20xattrs/with/5295
Change-Id: I864066b34cd8319d2271fac1b179cb4f950d836e
Signed-off-by: Raito Bezarius <raito@lix.systems>
Adds an example plugin implementing an mTLS-enabled binary cache store
(https+mtls:// scheme) using client certificates for authentication.
Darwin fix: don't link liblix* into plugins (host resolves symbols at
runtime via dynamic_lookup). Explicitly link curl so it binds to
Nix-store libcurl, not /usr/lib/libcurl. This prevents the plugin's
curl_easy_setopt calls from operating on the wrong libcurl instance.
Test portability: BSD sed -i wrapper, OpenSSL -sha256 for cert signing,
redirect test server output to log file.
Change-Id: I652b987d3ac45e31df50ff4ba1f523294438c2b6
xattrs scrubbing was gated behind Linux platforms, but Lix can safely
assume that POSIX APIs are available.
macOS implements these APIs modulo some creative ideas.
Fixes#1008.
Change-Id: I1566df4923bf24092d3fd32ccf6a96e4256e87b2
Signed-off-by: Raito Bezarius <raito@lix.systems>
Fixes#838.
xattrs were historically disabled in 2017, since then, NFS v4.2 gained
support for xattrs and almost all filesystems support xattrs. In
addition, moving the build directory to a xattrs-supporting filesystem
is always an option. Programs may exploit xattrs (including ACLs) during
their build process and testing phase, to better serve these programs,
we will remove this limitation.
Lix will use POSIX ACLs for its UNIX domain sockets in the future and be
one of these programs which will run tests making use of xattrs, while
not writing any xattrs in the derivation outputs themselves.
xattrs are still scrubbed from derivation outputs so it is not possible
to obtain an advantage by writing a security-related xattr to a well
chosen file.
Tests were added to test these scenarios on Linux. Darwin is carefully
excluded because of #1008 and #1090, solving that is welcome. In the
meantime, they are marked xfail.
Change-Id: Ia3255eeb8442e83db4f10dcb5a51cbc368a2550d
Signed-off-by: Raito Bezarius <raito@lix.systems>
I don't know how this was missed or no one really complained about it,
but perl-bindings was using CppNix if nix was not overlaid to our own
Lix.
If you overlay Lix manually, the resulting Lix's perl-bindings cannot
really refer to Lix itself, therefore, it's CppNix.
Change-Id: Ie5ce95564cb7fb5444acbea6124619c5121e1576
Signed-off-by: Raito Bezarius <raito@lix.systems>
When --check is ran with no output, this is seen as having *SOME*
invalid outputs, actually, ALL of them are invalid here.
Instead of reporting a confusing error, let's consider ALL of outputs
being invalid to be no output at all available and advise the user to
rerun the build without --check.
If there's *some* (>0) invalid outputs, let's print them exactly to give
a chance to the user to remediate this.
Fixes#485.
Change-Id: I00955ef9ea4f129e2c98d68c73b1e981f90278a0
Signed-off-by: Raito Bezarius <raito@lix.systems>
When `:load`-ing a file that doesn't result in a successful outcome, the
"loaded" file is removed again from the list of loaded files.
This avoids having it be used as part of `:reload` which will probably
not result in a positive outcome.
Instead, the user is expected to retry `:load` before issuing a
`:reload`.
Fixes#50.
Change-Id: I85d025cf39f79e8e02117628bffa20c81917dc3f
Signed-off-by: Raito Bezarius <raito@lix.systems>
If `:reload` is used with no file loaded, the environment is completely
cleared.
That's a surprising behavior and one of the root cause behind #50.
Environment clear is only performed if there's files to reload.
Change-Id: Icb014f490a83aaa4d758dbf37822db16d5bb65c1
Signed-off-by: Raito Bezarius <raito@lix.systems>
Only successful loads counts towards the list of loaded flake
references.
Change-Id: I04128f90e57ef8352183d72dc6def7a09d52173a
Signed-off-by: Raito Bezarius <raito@lix.systems>
`build_env` is assumed to be a certain trivial environment, e.g. a
singleton search path.
An environment in general is either empty, contain one or more search
paths separated by a colon.
It seems that the intent was to set the `build_shell` first as a
prepended path in the managed environment then extend via the
`build_env` parts.
This fixes a usecase when the `BUILD_TEST_ENV` is non-trivial.
Change-Id: If5b8ab976d867a57ce0b8d29255f64695e30a8b2
Signed-off-by: Raito Bezarius <raito@lix.systems>
Certain evaluation requires to pass impurity arguments, add `flags`
to add more arguments to the instantiation.
Change-Id: I538ca2ff12c07cc66e845b42f81baa87f11b8f0f
Signed-off-by: Raito Bezarius <raito@lix.systems>
Is there a situation where `home` can be None? This hinders the ability
to use it quickly without type gating in the test code itself.
Change-Id: Idd739ddad67e10d6f7a6aad283c0e3b7cd08106a
Signed-off-by: Raito Bezarius <raito@lix.systems>
Otherwise, this result in debug traces with a \n cutting the list of
controllers.
Change-Id: I8aba6379d335c4edab9d9ddb3fb411813d8dc426
Signed-off-by: Raito Bezarius <raito@lix.systems>
The prior use of the UID is uninteresting in general for consumers who
wants to learn things about _what_ the build is, not what is the UID of
the build user running it.
Achieving this with a place where a mapping between cgroup names and
build requests are maintained is non trivial because there's no "global"
daemon that keeps track of this information and such a daemon would
require a persistent location to track this.
Rather than solving that, we will just encode the derivation hash inside
the cgroup name. Userspace can simply pull this apart and query Nix to
obtain the original corresponding derivation for more information.
This provides userspace with a way to map cgroup names to pnames for
example. Going further, if you have a map between attribute paths and
derivation hashes, you can power a database of cgroup metrics per
attribute paths by combining all of that.
In addition to this rename, we adopt a systemd-ish convention for
template units (`@`).
As a result, userspace tooling that will scrape cgroup metrics will
discover a consistent path as long as they filter out the UID part,
especially in context of Id6c458aad30eaa08c3609ac8280a7dde8e8f3cf9
change.
This problem is tracked under #1073.
Change-Id: I238d0568a3e4b1ff3057781c0639528d666b4d37
Signed-off-by: Raito Bezarius <raito@lix.systems>
It's 2026 and the command handler was entirely manual!
We introduce a basic command handler framework for the REPL that handles
all the previous (implicit) features reasonably well.
This also builds up the infrastructure for nicer features.
Most important feature is that now we can let external plugins adds
commands to the REPL.
There's slight REPL changes:
- errors for debugger commands are more explicit.
- short-hand have a long-hand version.
- help shows command sorted by std::map natural key sorting (based on aliases)
while keeping the special syntaxes for the REPL language at the top.
- sections were introduced for Flakes commands.
- :st was merged into one command with an optional argument marker.
Change-Id: Ibd24f994491bf5d45d4b218fd84c0944ae5050fd
Signed-off-by: Raito Bezarius <raito@lix.systems>
This logic is used in the various build-related REPL commands and is
factored out to make it easier to write the handlers for each
build-related REPL commands.
Change-Id: Iaa18df489db75495b12924e9a76b3fff1975eb64
Signed-off-by: Raito Bezarius <raito@lix.systems>
The completions for debug commands is purposefully left out because it
is going to go away in the next changes.
Change-Id: Ibc78b7587de3f15857bb09a3a0732334693ad550
Signed-off-by: Raito Bezarius <raito@lix.systems>
As far as I can tell, there's no harm to let know a nix-shell invocation
about NIX_LOG_FD being stderr.
Fixes#336.
Change-Id: Ifdb3591813251d4bc481158d28dc18e1489de72c
Signed-off-by: Raito Bezarius <raito@lix.systems>
As far as I can tell, there's no harm to let stdenv know that they can
write to file descriptor 2 which is stderr inside of a nix3-develop
shell.
Contributes towards #336.
Change-Id: I51dcbcbc19a1698a0d1255f9d943d9ebd15d115d
Signed-off-by: Raito Bezarius <raito@lix.systems>
We offer the three usual options: verify, repair then delete and direct
people to report us corruption so we can fix if this is induced by us.
Fixes#447.
Change-Id: I0df61769d732d227333c206f312857c0593f7bce
Signed-off-by: Raito Bezarius <raito@lix.systems>
If the derivation does not start with D, do not return a simple
"expected string 'D'" error but a full error message.
This contributes towards #447.
Change-Id: Iee05f3918e4cc43e79f244ab2fd64a52cf2bb6d2
Signed-off-by: Raito Bezarius <raito@lix.systems>
Prior to I6a6a6964d2b5ad47ae5ea9eb11af9b6373ce2141 — `sudo nix
upgrade-nix` would perform direct store access.
This ensured a certain number of desireable properties for upgrading the
Lix binary itself.
We re-introduce direct store access for upgrading Lix binaries.
Fixes#1060.
Change-Id: I523c4d3023ed5fe9eff8fde9a266c56a0de47d8c
Signed-off-by: Raito Bezarius <raito@lix.systems>
Alternative to cl/4661 discussed in #1044.
It can be assumed that `$tmpdir/build-top` can be created safely without
any risk, this way, we don't need to reuse the random directory creation
primitive.
Fixes#1044.
Change-Id: Iec52477f3047fc40959b183c607312d5a40fc8c9
Signed-off-by: Raito Bezarius <raito@lix.systems>
Nix daemon runs in a service context, usually with systemd.
If the Nix daemon unit adds `[Service] CacheDirectory=nix-daemon`, it should
successfully use `/var/cache/nix-daemon` automatically.
Instead, it uses `/root/.cache` right now, which is really bad.
By default, we add `[Service] CacheDirectory=nix` now which means that
caches are moved into `/var/cache/nix`.
Fixes#634.
Change-Id: I854b1045bfdce8f60110aea70bda1bf6657dfd7b
Signed-off-by: Raito Bezarius <raito@lix.systems>
This adds the enablement code to support HTTP/3 if the user requests it.
We leave it disabled because h3 is not onpar with h2 performance.
Change-Id: I1fd3d4c97b972dcf36bccacc6c9a8290e22b31e0
Signed-off-by: Raito Bezarius <raito@lix.systems>
We start this section with shortcomings of unsandboxed builds.
Fixes#1018.
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: Ieb17e4340beab0c1197951813ae602de453a3fd9
Signed-off-by: Raito Bezarius <raito@lix.systems>
This helps users to debug whether their current profile symlinks are
correctly set.
Expected outputs look like this:
```
❯ sudo ./outputs/out/bin/nix doctor
[snip]
[PASS] All profiles are gcroots.
[PASS] Client protocol matches store protocol.
[INFO] You are trusted by store uri: local
[FAIL] Error: current generation cannot be discovered for profile:
'/nix/var/nix/profiles/default'
```
```
❯ ./outputs/out/bin/nix doctor
[snip]
[PASS] All profiles are gcroots.
[PASS] Client protocol matches store protocol.
[INFO] You are trusted by store uri: daemon
[PASS] You have 28 generations for profile
'/nix/var/nix/profiles/per-user/raito/profile'
The current generation number is '290'
```
Change-Id: I50c69cbeac3291d668f4c2332803411579adc944
Signed-off-by: Raito Bezarius <raito@lix.systems>
Co-authored-by: Qyriad <qyriad@qyriad.me>
Since we have async, we are unleashed and therefore we started to cause
people to run into system limits like number of open files.
We introduce an async semaphore which uses max(25 % of max open files,
max number of cores) which should lead to 256 maximum concurrent copies.
We believe that the copy operation is the only one that can cause issues
given that the daemon already runs with obscene fd limits.
Fixes#1022.
Change-Id: Iec433d10d5c5003962ea749cf8e32bafb314f0d9
Signed-off-by: Raito Bezarius <raito@lix.systems>
In case of empty messages, it is good to print the raw error code.
Additionally, we print request IDs which can help users to reconcile
what happened with the service provider.
Change-Id: I4d83c011c1b7a5514e3d1b21123df38308279044
Signed-off-by: Raito Bezarius <raito@lix.systems>
Completion status can be updated in two ways:
- progress callback
- transfer status update callback (esp. important in multipart
transfers)
It's guaranteed that one of the two will provide progress for our
promise.
This fixes#945.
Change-Id: Iac9e92df34f82fbc1facae7a411ccabf1688da52
Signed-off-by: Raito Bezarius <raito@lix.systems>
This fixes problems with S3 implementations that have mandatory payload
signing, e.g. Garage v2, AWS S3 itself, etc.
The problem manifested itself when the AWS SDK threw some error codes 99
(NETWORK_CONNECTION) with no messages and retried until it finally came
to a moment where the state machinery decided to send a
`Transfer-Encoding` header in a `Content-Encoding`/`Content-Length`-set
request with signed headers (even though payload signing is disabled),
causing the server to reject the transfer and crash the copy.
I did not debug super far what went wrong in AWS SDK, but I can confirm
this change makes transfers possible to finish with Garage v2.
Change-Id: Icc9e6a9f2afb0d760cf2d1e27816decd385a1d85
Signed-off-by: Raito Bezarius <raito@lix.systems>
libarchive's xz offers single threaded xz compression which is very slow
and provides ~10-20Mbps compression speed in addition to maxing a core.
In exchange, it achieves optimal compression ratios among all our
compression methods.
Nonetheless, xz prevent the saturation of 1Gbps+ connections and slow
down significantly decompression for end users. As these connections and
faster hardware is becoming prevalent for cache servers and clients, we
offer to default to zstd.
Lix is a "compress once, decompress many times" application. To avoid
incurring a high penalty to end users very sensitive to compress ratio
(very slow Internet connections), we dampen the consequences of
switching to zstd by increasing the default zstd level to 12.
On one example, xz will compress a 4.4GB file to 632MB, zstd on 12 will
compress it to 775MB, that is a ~18 % increase over the optimal xz
compression. zstd took 18 seconds to produce this file.
Increasing to level 14 leads to a 773MB file while taking 37s.
Increasing to level 16 leads to 735MB file while taking 66s.
Finally, xz took 77s, so a 50 % reduction in time taken to compress in
exchange of an increase of 18 % of the compressed size.
This change will reduce issues encountered in #945 but is probably not
the root cause.
References:
- https://discourse.nixos.org/t/switch-cache-nixos-org-to-zstd-to-fix-slow-nixos-updates-nix-downloads/23961
Change-Id: I7beda2bf2c1fed146dcb797b8f85dc290c486ab2
Signed-off-by: Raito Bezarius <raito@lix.systems>
As we saw actual users running into them throughout the debugging of
#920 and #1014.
It's best to document and point end users to this so they can take
measures for themselves.
Change-Id: I7d08407f4354055bf65fc6dd7d1624c5a9304402
Signed-off-by: Raito Bezarius <raito@lix.systems>
Fixes#1014.
Address family detection is performed by Pasta, we influence Pasta into
enabling dual stack by listing both an IPv4 & IPv6.
Change-Id: I4121e0eae7d5185c287957106adc36edf3f82a40
Signed-off-by: Raito Bezarius <raito@lix.systems>
Instead of letting inner shell perform the cleanup, we will fork/execvp
and finish the work ourselves. Once an interruption arrives, we wait
again and return the exit status code of the inner shell.
This idea was suggested by eldritch horrors.
Fixes#1020.
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: Id38af7ac33874aa1c4b7ca9c6123e3cbd8c3cd8c
Signed-off-by: Raito Bezarius <raito@lix.systems>
This was forgotten which should have been part of 64b9247f.
Change-Id: I3cf83d8699f2c1963ce530bbea6a511075202a79
Signed-off-by: Raito Bezarius <raito@lix.systems>
Legacy commands "main function" will now return an exit code.
Fun fact: `main_nix_build` was the only one not returning integers
already.
Change-Id: Ia43a16c3c3fb9a670e8889aefc4ee9b6528a7df4
Signed-off-by: Raito Bezarius <raito@lix.systems>
This also prevent `legacy` to return nothing as it's void.
Change-Id: Ic797544a59b04b41ad9e1c46af4cad3a1cb6fdc8
Signed-off-by: Raito Bezarius <raito@lix.systems>
Local store make use of SQLite as a cache for output maps, they can
diverge with actual store reality.
Causing a crash at realization time.
We can just throw an error instead.
Change-Id: I943a4bddfd2461a34933e9ec079a6638ab58e5b1
Signed-off-by: Raito Bezarius <raito@lix.systems>
Fixes fj#940.
When running `nix-shell`, the `$NIX_BUILD_TOP` environment variable is
set to `$TMPDIR` or `/tmp`.
nixpkgs stdenv uses $NIX_BUILD_TOP to create `$NIX_BUILD_TOP/env-vars`
which contains all the environment variables set by stdenv. This is used
for debugging purposes in combination with `--keep-failed` to reload the
bash environment of a derivation.
`$TMPDIR` is often unset, therefore, `/tmp/env-vars` was constantly
being created. On a multi-user system or, when you run Lix as root, you
might create a `/tmp/env-vars` with different permission bits.
As a result, `nix-shell` can cease to function because that file will
fail creation for an unprivileged user for example.
fj#940 rightfully remark that the code is not consistent between
nix3-develop and nix-shell and it should be reworked.
Change-Id: Iddf15945385d8bd497b2800b37fee5e1f97689b7
Signed-off-by: Raito Bezarius <raito@lix.systems>
UnsafeValueList held pointers from Value which were not necessarily GC
allocated, causing mayhem when evaluating something with genericClosure
(texlive environments).
We get rid of storing pointers and we let comparison take places on
const references.
We keep `gc_allocator<Value>` so that GC can scan things it allocates
inside the `res` list.
Co-authored-by: eldritch horrors <pennae@lix.systems>
Reported-by: qbit
Change-Id: I4fed3a3d9e18a2ef2d751f32d81801540e196f92
Signed-off-by: Raito Bezarius <raito@lix.systems>
nix develop should ignore output checks in general.
This was done only for the old way of specifying output checks, the
structured attrs way requires rewriting the JSON and removing the output
checks pieces.
We take a brutal approach of removing as many as possible including
non-recommended ways of doing it.
Fixes#997.
Change-Id: Iaf83029016c71b5171e56e15d4eadc1a60a8be98
Signed-off-by: Raito Bezarius <raito@lix.systems>
In preparations for a new representation of bindings that will make it
impossible to write an efficient `Bindings::find`.
Change-Id: I4e5a25b8d37d01b5728f7fe43978ceda2ab1b9b6
Signed-off-by: Raito Bezarius <raito@lix.systems>
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
The way zipAttrsWith works is to replace the attribute set value by a
call to a function (the argument of zipAttrsWith) over the list of
attributes sharing a common key.
Instead of that, we will insert into the resulting attribute set the
various lazy calls and return that.
Change-Id: I2aae054eb99b1d1f8b0e7c658cc8d3488e5cdb01
Signed-off-by: Raito Bezarius <raito@lix.systems>
C++ has the "spaceship" operator which can be auto-implemented and
generates efficiently a strong ordering.
Change-Id: Idfd1fd68039b395e54401cbe913454e0cbd80fb3
Signed-off-by: Raito Bezarius <raito@lix.systems>
Prior to this change, references or pointers could be mutated. In
practice, we do not require this capability in the codebase except in
zipAttrsWith.
This cleans up all easy sites in preparation to have a smarter
representation of attribute sets albeit one that requires constant
references.
Change-Id: I2be20cce040a9228bde9e5f7b42c0499fba9550b
Signed-off-by: Raito Bezarius <raito@lix.systems>
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
Currently, DerivationGoal prints a pretty generic message.
For many valid reasons, children may have better knowledge of the detail
of what has happened and would like to extend the error message.
What we did is to printError at convenient places but this is
counterproductive because the build error can bury the notes.
This is still not perfect because there's no fine-grained structured
information that children can use to act upon the generic messaging, but
this is already an improvement for LocalDerivationGoal and keep failed
which will occur in the next change.
Change-Id: I5835cbbb30c4f2aa64abefb83999018d30ca4a0c
Signed-off-by: Raito Bezarius <raito@lix.systems>
This is a collection of Lix plugins that showcase how to write one for
various usecases.
The first is a mTLS store plugin that enable mTLS cache URIs
(`https+mtls://`).
We enable meson build system support for this plugin but we are not
going to distribute it in the official packaging of Lix, we will
repackage each relevant plugin downstream in Nixpkgs.
These plugins have *NO* guarantee support, they are provided as useful
references and are possibly production-ready if your usecase is simple
enough.
Reference: https://github.com/NixOS/nix/pull/13030 (this change has
resemblances but our APIs are different, the tests harness is mostly
from CppNix).
Change-Id: Ib354271981b35dff6c134b12c4748c3eaf743fcb
Co-authored-by: Jörg Thalheim <joerg@thalheim.io>
Co-authored-by: László Vaskó <1771332+vlaci@users.noreply.github.com>
Signed-off-by: Raito Bezarius <raito@lix.systems>
This is useful to test extended features in Lix.
Change-Id: Idb2416a080329116677809b883950e6c33028a44
Signed-off-by: Raito Bezarius <raito@lix.systems>
Historically, Nix would support copying certificate authorities inside
the sandbox so you could use them.
In addition to that, the primitives consisting of leaking environment
variables via `impureEnvVars` and `extra-sandbox-paths` to render paths
external to the sandbox visible to the builder would also constitute a
mechanism to expose special inodes which should have no influence on the
output result, e.g. interception CAs.
Unfortunately, in nixpkgs, `lib.fetchers.proxyImpureEnvVars` set
`NIX_SSL_CERT_FILE` as an impure environment variable.
A confused user may set `ssl-cert-file` via `NIX_SSL_CERT_FILE` outside the
builder believing that this will set magically the right
`NIX_SSL_CERT_FILE` inside the sandbox, but this is not true.
The combination of impure environment variables and setting `caFile`
creates a weird interaction where `NIX_SSL_CERT_FILE` points to an
"outside the builder's world" inode *AND* `ssl-cert-file` creates
this very same certificate file in /etc/ssl/certs/ca-certificates.crt
without rewriting the environment variable.
This footgun is closed by making these two features mutually
incompatible with a warning and forcibly rewriting the SSL family of
environment variables even if it was set via impure environment
variables.
Users who truly meant to use `impureEnvVars` can obtain the right
behavior by setting `ssl-cert-file` to an empty string and will have to use
`extra-sandbox-paths`.
Users who meant to use `ssl-cert-file` will have everything work
automatically with a warning hinting at nixpkgs *fixing its own bug*,
i.e. passing `NIX_SSL_CERT_FILE` as an impure environment variable and
expecting the Nix interpreter to magically reconcile the diverging
values or expecting the user to actually do the work to render the path
visible available via `extra-sandbox-paths`.
Fixes#885.
Change-Id: I32f8b5ce20fe9b6a911768114c92f95fc886cc07
Signed-off-by: Raito Bezarius <raito@lix.systems>
Sometimes, `bindPath` will detect the source is a symlink and we are not
using the new mount API which support symlinks (kernel ≥ 5.12 IIRC?).
In those instances, we copy the inode to the target.
But some callers may want to follow the symlink in such circumstances,
we add a new default argument to the previous value and let caller
decide for themselves.
Change-Id: I8505b613fc614ce539eb89258fbbb7eaecebe23b
Signed-off-by: Raito Bezarius <raito@lix.systems>
`pathContentsGood` is used to assess the validity of a path as part of
derivation goals *in repair mode*.
When repair is used with a diverted store, i.e. a store where
fsPath(toRealPath(s)) != fsPath(s) for s a store path, this result in
utterly broken behavior because it will attempt to assess the goodness
of the *logical* store locations, most of the time: /nix/store/...
So, if you are repairing your system using a live NixOS ISO. Your ISO
contains a `/nix/store` (assumed to be good) and you repair your system
which is rooted at /mnt and contains its own /nix/store, that is, a Nix
store at /mnt/nix/store.
Performing the following operation `nix-store --verify --repair --store
/mnt` will assess the contents goodness of the ISO's Nix store.
To avoid this, we assess the path existence of the *physical path*, aka
the result of `store.toRealPath` applied to a *logical* store path
string representation and we verify the hash of the *physical path*.
The error messages are not taken care of in this CL as those are purely
cosmetic and helps the user understand what is going on.
Fixes#892.
Change-Id: Ib9e0153cb5683edcf37f1963ebf065ceba5e5dfb
Signed-off-by: Raito Bezarius <raito@lix.systems>
Instead of showing logical Nix store paths, we show the actual physical
location so that the user can stat by copy-pasting these paths.
The bad thing about this change is that certain Nix porcelain only
accept their logical counterparts.
Change-Id: Id0eb45d1bf08a23508dfc2bb694c88155654f585
Signed-off-by: Raito Bezarius <raito@lix.systems>
When a user runs a repair or check sequence, they might be confused of
seeing references to a logical /nix/store path rather than the actual
physical store location.
Change-Id: I042031a6159cdd1054e7e200a220bf6c321d5fb8
Signed-off-by: Raito Bezarius <raito@lix.systems>
In certain scenarios, a path may exist but is a broken symlink. For
instance, this happens frequently if you are rescuing an existing NixOS
system via `nixos-enter` or a manual `chroot` invocation because the
rescued system `/etc` may have broken links or the system prior to the
root pivot may interfere.
Nonetheless, these dangling symlinks are not always fatal for the builds
so we can just warn and skip their setup.
The warnings will provide a better diagnostics for system administrators
also.
Fixes#893.
Change-Id: Ifa12be3a43f23c973d7b466e8b73bd776abf3e7b
Signed-off-by: Raito Bezarius <raito@lix.systems>
We were catching ForeignExceptions believing it came from the TRY_AWAIT
handler, but this was misguided.
`j.dump()` is evaluated in synchronous context, outside of the `try {
... } catch (...)` block from `TRY_AWAIT`.
Therefore, we need to use `JSON::Exception` directly.
The previous test case did not catch it because:
(1) https://git.lix.systems/lix-project/lix/issues/865 hid the fact that
`--arg` was wrong.
(2) we did not grep for the warning because… we were not even copying
the strange store path to the binary cache.
(3) checking for the NAR happened after the NAR directory was emptied
for test reasons and this was not even caught neither.
Anyway, the test case was completely busted and has now been tested
without this commit and after this commit and we can confirm that prior
to this commit, the test will fail with an exception trace.
Co-authored-by: Maximilian Bosch <maximilian@mbosch.me>
Change-Id: I8df5befd06c4a449072b987f82a67bc4437e7e49
Signed-off-by: Raito Bezarius <raito@lix.systems>
Including the `.cc` is possible but is suspicious.
Change-Id: Ie18fef1e30e517edff4ab96f4a9c339e2b3145b5
Signed-off-by: Raito Bezarius <raito@lix.systems>
Some users may have arbitrary needs to connect to their store URIs, e.g.
mTLS authentication, Kerberos authentication, custom renewal using any
RPC mechanism of their preference and so on.
To avoid encoding all these patterns in Lix itself, we push the
configuration to the plugin boundaries and offer a hook for end users to
inherit from `HttpBinaryCacheStore` and provide new store schemes like
`https+mtls://my.very.secure.cache?tls-certificate=...&tls-key=...` or
`https+krb5://my.kerberos.enabled.cache`.
Co-authored-by: George Shammas <george@shamm.as>
Co-authored-by: eldritch horrors <pennae@lix.systems>
Signed-off-by: Raito Bezarius <raito@lix.systems>
Change-Id: I79f322b1a74632500fc79d53f5c920f9e43fd0c4
Usually, EOFs are represented by returning 0 in the `read` APIs, at
least, this is what read(2) dictate.
As clever creature, we may sum zeroes sometimes (advanced form:
`buf->added(got)`) and forego handling the EOF condition.
To avoid the bug that lurked in remote-store.cc and caused busy looping
if the remote end disconnects suddenly, we return
`Result<Option<size_t>>` forcing the caller to perform a specific
processing for the EOF situation.
The conversion did not raise any other offending code path.
Change-Id: I185fdcb77aa82d87ab0802d66ac37c1363657a73
Signed-off-by: Raito Bezarius <raito@lix.systems>
We missed xokdvium being author on cl/3300. This is something we
absolutely want to avoid.
We credit xokdvium in the RL and add a note on this problem.
Thanks to xokdvium for reaching out in private to us so we can repair
this mistake.
Change-Id: I094d0f95b6647104621d6b228e69a4529a300304
Signed-off-by: Raito Bezarius <raito@lix.systems>
This reverts commit a0a00948df
because this was insufficient to fix the critical correctness bugs.
Change-Id: I6c7b560ebeebacbbbcc1cbf26e6ef50c38b84f7f
This reverts commit e356d54d7a
because this was insufficient to fix the critical correctness bugs.
Change-Id: I91c3e368ffd13ade6a3cebbbacdb42655796ea56
Under macOS, the first level of directory has actually mode 0755 instead
of 0700 as macOS often do not possess the right primitives to chroot
inside of these directories, leading to
https://github.com/NixOS/nix/pull/11031.
Thanks to Emily for the heads-up on this type of matter.
Change-Id: I9d4e53717f61c9d573ff176f820610612804fbc3
Signed-off-by: Raito Bezarius <raito@lix.systems>
If `settings.buildDir` cannot be written to, because we are in a chroot
store, unprivileged or anything.
We can and should always gracefully fallback to a *secure* location
inside of /tmp, i.e. `/tmp/<a directory under 0700>/<our temporary
directory for build under 0700>/...`.
This does not reintroduce CVE-2025-52991 because we are creating a
directory in-between compared to creating only ONE level of directory.
Fixes#876.
Change-Id: Ie521202923f763225e1901ab1b9b6c6132aaf548
Signed-off-by: Raito Bezarius <raito@lix.systems>
In the past, it tried direct access if it *could* [1] perform direct
access.
This solves a bunch of errors people had when they tried the cgroup
feature and their scripts did not pass NIX_REMOTE=daemon manually
(nixos-rebuild-ng, home-manager activation from a root systemd unit,
etc.)
To avoid looping infinitely while receiving daemon connections, we
forcibly change the store URI when forking for a subdaemon to do direct
access automatically, this doesn't break forward usecases where you
point a daemon to another socket because we only change NIX_REMOTE="",
NIX_REMOTE=daemon, NIX_REMOTE=auto to a local and direct access.
All these usecases would end up infinitely looping no matter what
settings are set, because we are also responsible for creating the
daemon socket.
[1]: this happened all the time if you were `root`.
Related: https://github.com/NixOS/nixpkgs/pull/415701
Change-Id: I783fc795a9c2ee25b3d9f44f453f8f94b063371f
Signed-off-by: Raito Bezarius <raito@lix.systems>
When a build fails, its scratch output paths are not cleaned up.
Until recently, this was deemed not a problem but as part of the effort
to harden the Nix builds and protect these paths against being part of a
staged attack (race conditions, etc.), we automatically cleanup after
failed builds.
Fixes CVE-2025-52992.
Change-Id: I58481b1cc83826298b9d80d37fecf81f117ccb09
Signed-off-by: Raito Bezarius <raito@lix.systems>
When calling `_deletePath` with a parent file descriptor, `openat` is
made effective by using relative paths to the directory file descriptor.
To avoid the problem, the signature is changed to resist misuse with an
assert in the prologue of the function.
Fixes CVE-2025-46415.
Change-Id: I6b3fc766bad2afe54dc27d47d1df3873e188de96
Signed-off-by: Raito Bezarius <raito@lix.systems>
This ensures that `passAsFile` data is created inside the expected
temporary build directory by `openat()` from the parent directory file
descriptor.
Fixes CVE-2025-52993.
Change-Id: Ie5273446c4a19403088d0389ae8e3f473af8879a
Signed-off-by: Raito Bezarius <raito@lix.systems>
`writeFile` lose its `sync` boolean flag to make things simpler.
A new `writeFileAndSync` function is created and all call sites are
converted to it.
Change-Id: Ib871a5283a9c047db1e4fe48a241506e4aab9192
Signed-off-by: Raito Bezarius <raito@lix.systems>
We use it immediately for the build temporary directory.
Change-Id: I180193c63a2b98721f5fb8e542c4e39c099bb947
Signed-off-by: Raito Bezarius <raito@lix.systems>
We now keep around a proper AutoCloseFD around the temporary directory
which we plan to use for openat operations and avoiding the build
directory being swapped out while we are doing something else.
Change-Id: I18d387b0f123ebf2d20c6405cd47ebadc5505f2a
Signed-off-by: Raito Bezarius <raito@lix.systems>
Raito Bezarius