We offer full cgroup delegation to our sandbox now, required for running
containers inside the sandbox.
To run systemd-nspawn or containers managers inside the sandbox, there
is a need for one extra ingredient now: control over your own cgroup
subtree inside the sandbox.
If, in addition, you need multiple UIDs, for e.g. rootless usecases, you
need to run with the `uid-range` system feature.
Therefore, when the daemon or Nix runs under the right condition, e.g.
systemd-style delegation of the cgroup subtree while placing the
nix-daemon in a supervisor sub-cgroup, we create a new sub-cgroup for
each build based on the build UID and delegate that sub-cgroup to the
builder's process.
Additionally, `uid-range` always request the `cgroups` feature now, as
`uid-range` builds would probably always benefit from having cgroups
delegated, but the converse is not true.
Inspired from https://github.com/NixOS/nix/pull/11412 with a different
design that does not use function-local statics to derive the root
cgroup.
Co-authored-by: Linus Heckemann <git@sphalerite.org>
Co-authored-by: Parker Hoyes <contact@parkerhoyes.com>
Change-Id: Ic8947c5adaf4b5bbd153386e05fad65a935274fa
Signed-off-by: Raito Bezarius <raito@lix.systems>
the old protocols are largely untested, mostly unused, and have design
problems that make the RPC transition a lot harder, if not impossible.
in theory we could ship a transparent protocol-converting proxy that'd
isolate the daemon itself from old protocol versions, but that's a lot
of code to maintain for presumably little gain or even no gain at all.
Change-Id: I4c3f3bb34d39044f6aeb07c10caaf13b8340a220
Previously, paths not being deleted by gcDeleteSpecific would result in
(a) hardlinks not being cleaned up, and
(b) statistics not being reported correctly.
By throwing the error later, we fix both of these problems.
Change-Id: I8019f3e10d9f22e81ea87bb26b77f04ebc888a19
Remote binary caches support `write-nar-listing` options where they create a `HASH.ls` file for quick indexing without having to download the nar.
This commit makes experimental `nix store ls` attempt to read these files instead of downloading the full nar.
The difference is very obvious with large packages like stellarium:
nix store ls --store "https://cache.nixos.org" /nix/store/ijpvwgs9zamqaax5dy2cd0kxgz7lr7an-stellarium-25.1 -R
Change-Id: I6a37e0788b3a91c319331a8de69c51daf3efa955
The experimental `nix eval` command already supports a `--raw` flag.
This commit implements the same flag for the stable nix-instantiate command.
Until now instructions and scripts that didn't want to rely on experimental
features had to use workarounds such as:
nix-instantiate --eval <something> | tr -d \"
(which also undesirably also removes double quotation marks within the string), or
nix-instantiate --eval <something> | jq -j
(which undesirably depends on another package).
Co-authored-by: Raito Bezarius <raito@lix.systems>
Co-authored-by: Silvan Mosberger <silvan.mosberger@tweag.io>
Change-Id: Iced9a80ee7edd60af2385c5193485f1774175339
This introduces a new (demanded?) feature for coercing integers in
interpolation arguments under the experimental feature
`coerce-integers`.
This feature is being introduced behind an *experimental feature flag*
due to the cautious approach we're taking. The codebase has a track
record of revealing unexpected behaviors, often in subtle ways, so we
want to give this sufficient time and exposure before making it stable.
To remove the experimental flag, we want to see **at least two releases
or six months of real-world usage -- whichever is longer** -- that
demonstrate strong confidence the feature doesn't introduce regressions
or unintended side effects. If that level of confidence is reached,
we'll proceed to stabilize it.
Change-Id: I825904719eeba8f0e2a93cd6b93cfe6cebd7d827
Signed-off-by: Raito Bezarius <raito@lix.systems>
The pre-flight `echo started` check over SSH was originally added in
577ebeaefb. As it is usual with these old
commits, understanding why is there a need for something is difficult.
The closest thing would be
> Fix a race starting the SSH master. We now wait synchronously for
> the SSH master to finish starting. This prevents the SSH clients
> from starting their own connections.
But, we removed SSH connection sharing, so this does not apply anymore.
Nonetheless, we believed this check was meant as a way to catch obvious
misconfigurations or SSH failures early, before handing off to
`nix-store`. However, this approach was not fruitful: it assumes the
remote has a `bash`-compatible shell, `echo` behaves in a standard way,
and no `ForceCommand` interferes—all of which are unreliable assumptions
in practice.
While the intent was to provide slightly better diagnostics (e.g. in
case of SSH hanging or returning an interactive shell), in practice it
does not meaningfully catch or improve real failure cases. The
underlying protocol or engine can and should handle those errors more
robustly anyway.
In contrast, this check *does* break several legitimate workflows,
including:
* remote builders using `ForceCommand` wrappers (e.g.
`nix-remote-build`-style setups), see
<https://discourse.nixos.org/t/wrapper-to-restrict-builder-access-through-ssh-worth-upstreaming/25834/15>,
* SSHing into minimal environments lacking `bash` (e.g. initrd,
busybox-based systems),
* configurations that don’t default to POSIX-like shells, e.g., nushell
enthusiasts.
As such, we’re removing this code. Protocol mismatch errors and SSH
failures can be rethought and handled more structurally elsewhere in the
engine.
Change-Id: I187f6881375d42ef83987a13a350c97964bbdb30
Signed-off-by: Raito Bezarius <raito@lix.systems>
This teaches `filterANSIEscapes()` how to find the end of an OSC
sequence. It also keeps OSC 8 (hyperlinks) when not instructed to filter
out all escapes, just as it keeps colors.
This also relaxes the parsing of CSI escapes to find the end of the
sequence for invalid sequences, and handles better escapes that don't
start CSI or OSC.
This fixes the repl output for `:doc builtins.fetchGit`.
Fixes: https://git.lix.systems/lix-project/lix/issues/160
Change-Id: Id0000000f2a6956c042c883a4545edf347fa1799
with impure derivations gone we move on to dynamic derivations. this too
is not done in a single commit because dynamic derivations are invasive,
modifying semantics of all references to derivation output paths and all
derivation dependency calculations. removing dynamic derivations cleanly
is made significantly harder by the multiple did-you-mean-sum types, aka
"wrappers for std::variant", holding all derivation outpath information.
Change-Id: Ice7a7700c7b54c6a6061d4beb322b4175923d27a
The expected and the obtained path are now printed as part of the
error message, making comparing them easier when they're both at hand.
The extra rethrow for the hash-mismatch exception in the bmCheck case
has been removed, allowing the path to be registered as in the
non-check case. This makes having both paths at hand a lot more likely!
The determinism check logic was incorrect for content-addressed paths,
since it only ever tried to compare the path produced, even if this
was not the path expected (in the case of fixed-output derivations) or
the path previously produced (in the case of non-fixed CA
derivations). This made little sense, because that would always be the
same path if it exists! The determinism check is therefore now
bypassed for CA paths. Having a correct determinism check for
non-fixed CA derivations and running the diff hook for fixed-output
derivations would be nice, but feels out of scope and bypassing the
inapplicable logic isn't a regression from the previous behaviour.
Change-Id: I5fc14fb477c8c7d2f5bdedad5591af916f72b128
we don't remove the entire feature in one go to make review easier.
impure derivations are rather unintrusive on their own, at least if
we compare them to dynamic or ca derivations in general, so we will
be done with this soon. as it stands impure derivations cannot work
without ca derivations, and those we *really* want to leave behind.
Change-Id: I4f01d8d758b2c85dcd6c3078304b5ee1b52f65b0
ca derivations are what we're really after, but dynamic derivations
must also go because they depend on ca derivations. we can't easily
implement dynamic derivations any other way, so we remove them too.
impure derivations build on the content-addressed infrastructure in
ways we cannot easily detangle, so they too must go for time being.
see #815
Change-Id: If61371736dfd89cc71a1b2ae5a005757c3cb9484
This allows debugging the errors by pinpointing the offenders, instead
of trying to blindly guess what could have possibly gone wrong.
The wording has been improved to better explain the failure.
Closes#530
Co-authored-by: Ben Millwood <thebenmachine+git@gmail.com>
Change-Id: I84ba5d2d81e5d1867f53bd3bc80e615cab9fe274
When I read the note, I wasn't sure what was meant by connection
sharing. This adds a reference to SSH documentation for clarity.
Change-Id: I1d6e5c53fb6a6fac8380210967f56d86f0333e55
This was an absolute nightmare to diagnose. It turns out there's a
kernel bug: poll with events = POLLHUP will receive an event for NOT
POLLHUP internally in the kernel, delete their event subscription, and
then not receive events for any HUP later. lol! lmao!!
We choose to use plain old EVFILT_READ because the watched fd can be
either a socket or a pipe and it's preferable to eat some spurious
wakeups than have separate paths for those. The alternative is using
EVFILT_SOCK, a private API that's existed for years and which netty
uses for its sockets, but that doesn't work on pipes.
Fixes: https://git.lix.systems/lix-project/lix/issues/729
Change-Id: If72b5d7a39f00320a9acccdbe81121cdb1a04c45
this is much more useful than a frame pointing to the set but claiming
it as a function being called. if the function is actually at fault we
will now point to its attribute, although the position may be slightly
wrong if __toString was defined from set updates or builtin functions.
Change-Id: Ib3eb237a276d94426d9c6fc0e26eea72382d34a2
This was discussed as part of https://git.lix.systems/lix-project/lix/issues/767
with a one month long comment period.
This commit removes the recursive-nix feature from Lix. It has seen
limited usage and is difficult to stabilize, especially on non-Linux
systems where its behavior is underspecified.
Maintaining this feature complicates core work on the store, as we must
account for the potential presence of the daemon in the sandbox, adding
unnecessary complexity. Additionally, its inclusion in the
platform-independent local store creates risks for non-Linux platforms.
For more details on this removal, refer to the release note entry or the
issue entry.
Change-Id: I9137202f563c0a317f9c5da79cd9fd07d801427a
Signed-off-by: Raito Bezarius <raito@lix.systems>
We now properly shell out to the parser instead of hacking stuff
together with a regex. Stuff we get for free by doing this:
- Optional trailing semicolon
- Declaring nested attribute sets
- String identifiers, and future proofing for eventual grammar
improvements to identifiers
- Dynamic attributes
Change-Id: Ibf1ad815e5e27caf162df05ea5ba5b1b4955d9c9
our implementation is mostly unused, completely untested, and simply
breaks when users configure connection sharing independently. we can
safely delete this "feature" and inherit user configuration instead.
also have the remote build test check that connection sharing works.
fixes#304, fixes#644
Change-Id: Iea44cc0f8e51a1d231ad186a95c7e310bbfeb303
This reverts commit bba678e5c5.
Reason for revert: didn't fix the bug and created new ones (fj#794)
Change-Id: I0450205d3041b6c876737151a4051081c1366f1d
or more accurately, wrap them in a nix::Error subclass so we can display
them properly without crashing, and add some error context if available.
fixes#642fixes#753fixes#759fixes#769
Change-Id: I1aad0c0501fea83f9de3a1335eaa6adc20721616
allow opting in to serialization as integers via a trait type instead,
and add string-list serializers for the feature flag set enumerations.
fixes#738
Change-Id: I2746eb5ef1f15c01b4e681f9ba1615b6c6e64f44
USDT probes are statically defined trace points that have nearly zero
disabled-probe effect, i.e. we can put them in hot paths.
The use case for these is both similar and dissimilar to Rust tracing:
We still need better logging and a better structured rust-tracing
looking thing, but probes allow for quite easy programmable interactive
tracing in production, which we also care a lot about.
This CL comes with a perfunctory trace point in
libstore/file-transfer.cc for reading data out of the curl buffer. This
was mostly thrown in there so that I could see what the buffer sizes of
this were, and maybe be able to instrument the perf of the curl usages
in Lix in the future.
Fixes: https://git.lix.systems/lix-project/lix/issues/727
Change-Id: I0f5d9912d76bf3d6923bf53ebfd9b8d6c6e70aea
This is *just* in output messages that look relatively like the FOD
failure message. It's not great that we inconsistently use the old
format in random places.
Change-Id: I4b9ce73a98a3a50c924cfe1d7cded42f3756df6b
Well that is a mess. I don't like breaking derivation compatibility,
but I like non-deterministic evaluation much less.
We can break compat if we own up to it, which this does.
Fixes: https://git.lix.systems/lix-project/lix/issues/750
Change-Id: Ic9e2407393f1d42c2be604f80b4aa11bc872bc23
It's possible to put a path into the store in pure mode by pretending
it's locked like this:
$ echo 'lalala' > testfile
$ nix eval --expr '(builtins.fetchTree { path = "/home/ma27/testfile"; rev = "0000000000000000000000000000000000000000"; type = "path"; })'
{ lastModified = 1723656303; lastModifiedDate = "20240814172503"; narHash = "sha256-hOMY06A0ohaaCLwnhpZIMoAqi/8kG2vk30NRiqi0dfc="; outPath = "/nix/store/lhfz259iipmv9ky995rml8018jvriynh-source"; rev = "0000000000000000000000000000000000000000"; shortRev = "0000000"; }
$ cat /nix/store/lhfz259iipmv9ky995rml8018jvriynh-source
lalala
There was a fix in CppNix[1], but Puck noted that it's breaking
backwards-compatibility because
> [...] a github fetch with a fully specified rev is no longer considered "locked"
> this is for "purity" reasons, but it breaks any existing flake.nix files
I tried a way smaller correctness fix here:
* Each scheme can denote whether a `rev` is enough to consider itself
locked.
* If a `rev` is given and the scheme is OK with just a `rev` to be
locked, the input is marked as locked.
For `path` this is not the case anymore, i.e. it requires a NAR hash to
be locked down.
[1] https://github.com/nixos/nix/commit/071dd2b3a4e6c0b2106f1b6f14ec26e153d97446
Change-Id: Ibbbf4733c82bcfa1c24dfe099a896d8aaecd81cc
Documenting fetchTree is an exercise in frustration because of the sheer
amount of stringly typed everything in it. I do not know which fields
exist without reading the entirety of libfetchers. However, we can write
something slightly perfunctory but at least perhaps somewhat helpful
documentation-wise.
Fixes: https://git.lix.systems/lix-project/lix/issues/609
Change-Id: I991391b53fcd69172dbc7efb9d384e62bc847b91
