Lix blocks SIGPIPE, and std::cout ignores EPIPE (actually it probably stashes
an error code in some flag that no one ever reads, with the same end result).
Consequently, nix-eval-jobs would waste resources by continuing to evaluate
even when the reader interested in the results has long gone away. Instead, use
writeToStdout, which throws an error on EPIPE, leading to the process
terminating as desired.
Reported-by: Winter <winter@winter.cafe>
Change-Id: I962c09bab582a8ed27dd41c01b1519876a6a6964
Right now, the worker will not receive interrupts, as it has no signal handler
thread due to the way it forks, but this will change soon. Prepare to handle
worker interruption properly, by letting the worker exit successfully in this
case and adjusting the coordinator to handle this exit gracefully.
Change-Id: I9f79670b0d2004e7e2d8bf36cf67108e6a6a6964
Commit 5dc847b47b introduced usage of
std::chrono::milliseconds, which prints with the unit, leading to its
duplication. Remove this duplication again. Specifically, the explicitly
printed one is kept, due to better typography.
Change-Id: I724ea6be6f2a1349fe799602c3429f096a6a6964
URL literals were deprecated in 278fddc317,
effectively stabilizing the experimental feature. Enabling it has no effect any
more. Remove it.
Change-Id: I8fbca03b1a2be6a8cddfce644043ac0a6a6a6964
If BUILD_TEST_SHELL is not set or empty, None or the empty string respectively
should not make it into the PATH. Ensure this property.
Change-Id: I4ce9b0c06c407b465308b63b9cb64e7d6a6a6964
When the maximum memory size is chosen too large (relative to the free memory
usage), so that the evaluation workers don't fit into the free memory, they
will start being swapped out. Further increase in memory usage will not be
reflected in the RSS, so they will not exit, and instead fill up all swap too
until the system runs out of memory.
The BDW-GC keeps track of its heap size, including any parts of it that may be
swapped out. For this reason it does not suffer from a similar problem. While
not all memory usage by the evaluation worker is accounted for by the GC, a
simple affine model for the overhead works reasonably well in practice. The
specific parameter values were determined using nixpkgs evaluation, and appear
to work quite well also for somewhat different workloads (like the `hydraJobs`
of Lix, which include a bunch of NixOS system configurations). So this is what
gets used.
When Lix is configured without the BDW-GC, of course this approach cannot be
used. In this case the old strategy is retained.
Change-Id: I6cb5f0a9b4ceda9dd14be165dda108cd6a6a6964
The functional2 test suite was broken in the development shell for two mostly
separate reasons leading to "no such file or directory" errors:
* The `BUILD_TEST_SHELL` (already containing the correct path) would always be
set from the Meson option `build_test_shell`, even to the empty string if
that option is not set. Skip the overwrite in this case to make the
environment variable work again.
* Sandboxed builds would fail to find their builder `/bin/sh`, since in the
development shell busybox is not found, so no sandbox shell gets configured.
Use the shell from `BUILD_TEST_SHELL` instead. (How this does not break the
old functional tests remains a mystery.)
Change-Id: I6a6a696424e8caaef3f9b68e3738bfd58ea0b056
In 7b37d5ea6a, aside from subdaemons getting
properly executed, they also lost the ability to outlive their parent, due to
now getting set the parent death signal like most other processes spawned by
Lix. This has annoying consequences like all concurrent builds being forcefully
terminated on system updates requiring a nix-daemon restart. As the behaviour
change was not documented and the systemd service file retained
`KillMode=process`, it seems to have been accidental. Restore the old behaviour
of letting the subdaemons outlive their parent.
Change-Id: I6a6a69645312a90dbce55495c2fef3825dd3c097
The stdio stream identifiers (stdin, stdout, stderr) are allowed to be macros.
In musl libc they are, for example doing `#define stdout (stdout)`, breaking
compilation with an error when one of the clashing variables is attempted to be
initialized the "wrong" way:
../lix/libutil/processes.cc:272:7: error: expected class member or base class name
272 | , stdout(stdout ? std::make_unique<AsyncFdIoStream>(std::move(stdout)) : nullptr)
| ^
/nix/store/ziw42d7rvgnf3vkbfc8kry07kipwf1xm-musl-static-x86_64-unknown-linux-musl-1.2.5-dev/include/stdio.h:67:16: note: expanded from macro 'stdout'
67 | #define stdout (stdout)
| ^
Other places only cause warnings on musl:
../lix/libutil/processes.cc:254:17: warning: parentheses were disambiguated as redundant parentheses around declaration of variable named 'stdout' [-Wvexing-parse]
254 | std::string stdout;
| ^~~~~~
/nix/store/ziw42d7rvgnf3vkbfc8kry07kipwf1xm-musl-static-x86_64-unknown-linux-musl-1.2.5-dev/include/stdio.h:67:16: note: expanded from macro 'stdout'
67 | #define stdout (stdout)
| ^~~~~~~~
../lix/libutil/processes.cc:254:17: note: add a variable name to declare a 'std::string' (aka 'basic_string<char>') initialized with 'stdout'
254 | std::string stdout;
| ^
| varname
/nix/store/ziw42d7rvgnf3vkbfc8kry07kipwf1xm-musl-static-x86_64-unknown-linux-musl-1.2.5-dev/include/stdio.h:67:16: note: expanded from macro 'stdout'
67 | #define stdout (stdout)
| ^
../lix/libutil/processes.cc:254:5: note: add enclosing parentheses to perform a function-style cast
254 | std::string stdout;
| ^
| ( )
../lix/libutil/processes.cc:254:17: note: remove parentheses to silence this warning
254 | std::string stdout;
| ^
/nix/store/ziw42d7rvgnf3vkbfc8kry07kipwf1xm-musl-static-x86_64-unknown-linux-musl-1.2.5-dev/include/stdio.h:67:16: note: expanded from macro 'stdout'
67 | #define stdout (stdout)
| ^
However they are still wrong, since the macro could be more complicated. Fix
them as well.
Change-Id: I6a6a6964a50ef7dec8f05f0bd8fc8f13f3036d51
It was supposed to be removed in 480fdf146d, as
it is not needed any more with the prelinked library. Due to a mistake in
rebase conflict resolution it reappeared by accident. Actually remove it now.
Change-Id: I6a6a6964d175fdb0ba0ad9ac55d4d22d7b27ad3f
Without https://github.com/NixOS/nixpkgs/pull/434761 evaluation of the
`nixpkgsLibTests` will fail in CI with recent enough Lix, due to reliance on
the TOML integer saturation bug.
Reported-by: Sergei Zimmerman <sergei@zimmerman.foo>
Change-Id: I6a6a6964838009d2c525f67035f84072fdfad988
Previously two cryptography libraries were linked into Lix: OpenSSL used for
hashing and (in usual configurations) indirectly via curl for TLS, and Sodium
used only for handling the Ed25519 path info signatures. The latter is
functionally redundant since OpenSSL supports the same use case as well.
Reimplement the Ed25519 handling using OpenSSL and drop Sodium.
Fixes: https://git.lix.systems/lix-project/lix/issues/969
Change-Id: I6a6a696456b9d3ad7fdc2bf9b0759836a6247a38
It was only used for impure derivations, which were finally removed in commit
be07629820. Delete the unused function.
Change-Id: I6a6a696481711f68a8c3ea7eac7978fcf5884cce
Commit 5dc847b47b introduced it as a non-inline
function with definition in the header, which can result in linker errors like
the following:
/build/source/build/lix/libutil/backoff.hh:36: multiple definition of `nix::backoffTimeouts(unsigned int, std::chrono::duration<long, std::ratio<1l, 1000l> >, std::chrono::duration<long, std::ratio<1l, 1000l> >, std::chrono::duration<long, std::ratio<1l, 1000l> >)'; tests/unit/liblixutil-tests.p/libutil_backoff.cc.o:/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-gtest-static-x86_64-unknown-linux-musl-1.17.0-dev/include/gtest/gtest-printers.h:1223: first defined here
This error was observed during trying to bump `lixPackageSets.git` in nixpkgs.
I am not sure why it can't be observed in the in-tree `nixStatic` package but
the definition is wrong in any case.
Change-Id: I6a6a6964e218a03ca2a2e8eddbb72d44e06e904e
Notably, this adds the Requires.private required for proper static linkage of
Lix libraries. Some minor missing or duplicated dependencies are also fixed
along the way (although some optional dependencies are omitted due to their
size).
Fixes: https://git.lix.systems/lix-project/lix/issues/789
Change-Id: I6a6a696413d538124d9ac75c68f100cc3089284f
A bunch of dependencies were superfluous, while others were missing (in
particular on internal sub-libraries) and things just happened to work because
they leaked in or were explicitly required in other places. Make efforts to fix
them all.
Change-Id: I6a6a69643e35ac4b0b66a72f4e42a2ba5ed52488
Some of the Lix libraries always need to be linked in full due to their
reliance on static initializers. This was achieved internally using link_whole,
but they are still easy to abuse by external users who manually need to
remember passing linker flags such as `--whole-archive` (GNU) or `-force_load`
(Apple), and the obvious way to shove it in pkg-config breaks Meson due to
potentially including a library's flags multiple times, and then deduplicating
only the file names leaving a stale `-force_load` around causing trouble.
Instead we now "prelink" the static libraries, by merging them into one object
file. Since the static linker will always link entire object files, this will
have the same effect as whole-archive linking (except the library won't be
included if it's completely unused, which should not cause trouble since it's
unused after all, and dynamic libraries behave the same way). Unfortunately
Meson's native prelink functionality cannot be used due to missing (non-Apple)
Clang support [1], so write our own one. While not particularly portable, it
should work with Clang which is the only officially supported compiler, as well
as GCC.
[1] https://github.com/mesonbuild/meson/pull/14846
Change-Id: I6a6a6964a82241ce3b0b11fe8397fd451b8027f2
Meson only requires the `modules` option for Boost libraries that should be
linked against [1]. However, we use only header-only portions of the Boost
container library; in fact we only do not run into the disallowedReferences
because the linker drops the unused library. Remove the misleading option.
[1] https://mesonbuild.com/Dependencies.html#boost
Change-Id: I6a6a69648b806bf6bcf784391263f5ee9cd63a0b
Originally, libboost_context and dependent dynamic libraries have been copied
into the nix package to remove boost from the closure and consequently reduce
the closure size. Since commit ef0de7c79f we do
not depend on Boost coroutines any more, so these libraries are not needed at
all any more and (somewhat ironically) only increase the package size. Remove
them.
Change-Id: I6a6a6964dc3e0b29dfad8b2b232b428ba3cc653c
The issue requiring these environment variables to be set for Meson to find
Boost [1] has been fixed [2] for quite some time now. Drop them since they are
unnecessary in all supported nixpkgs releases (in fact they have already been
removed in the lix package there).
[1] https://github.com/NixOS/nixpkgs/issues/86131
[2] https://github.com/NixOS/nixpkgs/pull/315998
Change-Id: I6a6a69640a30e917cd7a983b9d846d023b393dcd
Commits 205c59367c and
325e7e1824 introduced real glibc store paths from
current nixpkgs unstable into the source. Since nixpkgs `fetchFromGitea` (and
similar fixed-output derivations) depends on a C library, on x86_64-linux they
will fail with the forbidden reference error:
error: the fixed-output derivation '/nix/store/wnmnj3jzc82y89sfmyicr04kilg8zs2k-source.drv' must not reference store paths but 1 such references were found:
/nix/store/q4wq65gl3r8fy746v9bbwgx4gzn0r2kl-glibc-2.40-66
Falsify the store path to prevent this failure.
Change-Id: I949033567bcad070f9a0a19cefdb33a79222e421
The first auto-GC request would not be registered as a waiter due to a logic
error. As a result, if that request was synchronous (as happens during
evaluation) it would be stuck forever waiting on a promise that will never be
fulfilled.
Register also the first request properly so that it is notified and unblocked
again when the GC has finished. Also add a test verifying that auto-GC
triggering during evaluation will not get stuck.
Fixes: https://git.lix.systems/lix-project/lix/issues/844
Change-Id: I157afdc737415261e48d6d01d46c586a2927a1ad
The nix-shell and nix-env subprocess spawned by :sh and :i respectively should
use the same settings as the nix repl they were launched from. Previously, this
was accomplished by sending them the entire configuration explicitly, causing
warnings like the following:
warning: Ignoring setting 'auto-allocate-uids' because experimental feature 'auto-allocate-uids' is not enabled
In addition, if connected to the daemon, all other settings would be forwarded
to the daemon, further causing large quantities of warnings like the following
if the user is not trusted:
warning: Ignoring the client-specified setting 'allow-symlinked-store', because it is a restricted setting and you are not a trusted user
Similarly to ece99fee23 (for the build hook) and
523965697d (for the post-build-hook), we will now
only send overridden settings to the subprocess. The resulting configuration is
the same, and all warnings are avoided because the client was already smart
enough to not send unchanged defaults to the daemon.
Fixes: https://git.lix.systems/lix-project/lix/issues/811
Change-Id: Icbe923cbe166aa3f462916f2b164285882564abd
If tearing occurs while the progress bar is rendering, flickering can occur.
This is particularly visible in multiline mode. Use the synchronized updates
specification [1] to make updates atomic on supported terminal emulators. On
unsupported terminals, the escape sequences should be ignored, leaving the
behaviour effectively unchanged (but there's nothing we could do better in this
situation).
[1] https://gitlab.com/gnachman/iterm2/-/wikis/synchronized-updates-spec
Change-Id: Ic91ca40422c73ac9f9e088c7cb78c1b8c28adc4e
The multiline progress bar would not clear itself before writing to stdout,
leading to interference with the redraw; the most visible symptom is after
building in the repl, where the lines indicating the build outputs would be
wiped instead of the progress bar header. Separate the steps of erasing and
redrawing the progress bar, so that arbitrary output can happen in between in
a non-awkward way. In addition to fixing the bug, the code is simplified.
Change-Id: I142cf38f5ba5cd672cd6016e996b2f7bf726e9cd
Starting with commit 0dbfa7b26e access would also
be allowed to ancestors of allowed paths. This is (ironically) a significant
purity regression, since several users of the purity checks will themselves
assume that arbitrary descent is allowed. For example, `builtins.readDir` and
`builtins.path` could now refer to the filesystem root, breaking purity
entirely in the latter case by allowing to read arbitrary files. Restore the
previous behaviour of only allowing access to explicitly allowed paths.
Change-Id: Ie64180733ab735da9873255e1ccbf95ba7c9161c
If a revision is specified in a way that looks like a commit hash, Lix expects
that it actually is a commit hash. Unlike Git, Mercurial will fall back to
bookmarks, tags and branches with the specified name when a commit with the
specified hash does not exist. Previously, an assertion failure would be thrown
due to the resulting commit hash mismatch. Tell Mercurial to only take commit
hashes into account, whose non-existence is then handled gracefully.
Change-Id: I98bf020187575f3cf8176831da85872d066c4d95
When using the Mercurial fetcher on a local repository without explicitly
specifying a branch or revision, previously always the tip of the default
branch would be fetched. This is likely unexpected by the user, and
inconsistent with the Git fetcher as well as the dirty case. To reduce
surprises and restore consistency, fetch the currently checked out revision
instead.
Change-Id: Id6d58f958b710f7a9661dc66ad4ddcb8d06a0cdd
The Mercurial fetcher previously used the revision number supplied by Mercurial
as its revision count. This is inappropriate, as the revision number is
incremented each time a change is added to the local repository, so also
changes only reachable from other heads will be counted. For the same reason,
it will also depend on whether such changes have been fetched before from the
same repository, introducing an impurity. Instead the revisions have to be
counted explicitly, similar to the Git fetcher.
Change-Id: Idb1e66ae2fb8b05101116c815f453efd6e95b09e
The progress bar is not supposed to show very short activities (<10ms) in order
to prevent excessive flickering. This behaviour was inadvertedly disabled in
commit da4e46dd1f due to not actually skipping
the short activity after calculating that it should have been. Fix this
oversight.
Fixes: https://git.lix.systems/lix-project/lix/issues/561
Change-Id: Ibee256a7018cdc431662cd8ea9f6c14ef1706972
* changes:
libexpr: generate builtins from data
treewide: generate global settings from data
libutil: generate experimental and deprecated features from data
Most builtins are now generated from data too, with two exceptions:
* Undocumented builtins, since supporting them would add complexity to the
generator, the harms of the current implementation mostly don't apply, and
the proper fix is to document them.
* `derivation` is somewhat magic (it is a function, but defined in the code as
a constant), so the current treatment of having it separately documented is
kept (for now, at least).
Since it is slightly easier to do and probably a good idea anyway, the builtin
function registrations generated this way are now processed directly in code
and don't go through global variables any more.
Unfortunately, a slight breaking change is introduced because the order of the
builtins' names in the symbol table changes. Hopefully, this will turn out to
not matter in practice.
Change-Id: I7b4379a93ae380b6524e41a916a21c5c6f70555e
Currently, a bunch of documentation is generated by embedding parts of it in
the nix executable, getting it out again by running it, and then postprocessing
the output. This is bad, since it creates a pointless dependency of the
documentation on the executable, and also makes documentation generation
impossible when cross-compiling.
Instead, both the code and the documentation should be generated from data, see
https://git.lix.systems/lix-project/lix/issues/292 . Here we start applying
this approach to the experimental and deprecated features, which are done in
one go since the technical implementation is very similar.
Of course, the actual benefits are not realised yet, since the offending
pattern is used in several more places. These will be fixed later.
Change-Id: I4c802052cc7e865c61119a34b8f1063c4decc9cb
Commit 4dbbd721eb intended to mark all settings
as overridden when they are. Unfortunately, due to an oversight, this marking
was accidentally performed in the implementation details of non-appendable
options. Move it to the common codepath so that it works for appendable options
too.
Fixes: https://git.lix.systems/lix-project/lix/issues/573
Change-Id: Idc3402bac48b19d832acd9b553e16e5791470c26
* changes:
treewide: make more settings conditionally available
libstore/build: only send overridden settings to the build hook
treewide: consistently mark overridden settings as such
Some settings only make sense on particular platforms, or only when a certain
experimental feature is enabled. Several of those were already conditionally
available. Do the same for a bunch more instead of silently ignoring them.
Exceptionally, the use-case-hack setting is not made conditional because it is
included in the test suite.
Change-Id: I29e66ad8ee6178a7c0eff9efb55c3410fae32514
The build hook is still running locally, so it will run with the same default
settings. Hence, just as with the daemon, it is enough to send it only the
overridden settings. This will prevent warnings like
warning: Ignoring setting 'auto-allocate-uids' because experimental feature 'auto-allocate-uids' is not enabled
when the user didn't actually set those settings.
This is inspired by and an alternative to [0].
[0] https://github.com/NixOS/nix/pull/10049
Change-Id: I77ea62cd017614b16b55979dd30e75f09f860d21
Only overridden settings are sent to the daemon, and we're going to do the same
for the build hook to. It needs to be ensured that overridden settings are in
fact consistently marked as such, so that they actually get sent.
Change-Id: I7cd58d925702f86cf2c35ad121eb191ceb62a355
In 0e6b3435a1, installation of the HTML manual
was accidentally dropped: setting install_dir on a custom_target only sets the
directory where something is going to be installed if it is installed at all,
but does not itself trigger installation. The latter has to be explicitly
requested, which is just what we do here to get the manual back.
Change-Id: Iff8b791de7e7cb4c8d747c2a9b1154b5fcc32fe0
The legitimate output of `nix path-info` may visually interfere with the
progress bar, by appending to stale progress output before the latter has been
erased. Conveniently, all expensive operations (evaluation or building) have
already been performed before, so we can simply wipe the progress bar at this
point to fix the issue.
Fixes: https://git.lix.systems/lix-project/lix/issues/343
Change-Id: Id9a807a5c882295b3e6fbf841f9c15dc96f67f6e
* Move the extended attribute deletion after the hardlink sanity check. We
shouldn't be removing extended attributes on random files.
* Make the entity owner-writable before attempting to remove extended
attributes, since this operation usually requires write access on the file,
and we shouldn't fail xattr deletion on a file that has been made unwritable
by the builder or a previous canonicalisation pass.
Fixes: https://git.lix.systems/lix-project/lix/issues/507
Change-Id: I7e6ccb71649185764cd5210f4a4794ee174afea6
When generating shell completions, no logging output should be visible because
it would destroy the shell prompt. Originally this was attempted to be done by
simply disabling the progress bar (ca946860ce),
since the situation is particularly bad there (the screen clearing required for
the rendering ends up erasing the shell prompt). Due to overlooking the
implementation of this hack, it was accidentally undone during a later change
(0dd1d8ca1c).
Since even with the hack correctly in place, it is still possible to mess up
the prompt by logging output (for example warnings for disabled experimental
features, or messages generated by `builtins.trace`), simply send it to the bit
bucket where it belongs. This was already done for bash and zsh
(9d840758a8), and it seems that fish was simply
missed at that time. The last trace of the no-longer-working and obsolete hack
is deleted too.
Fixes: https://git.lix.systems/lix-project/lix/issues/513
Change-Id: I59f1ebf90903034e2059298fa8d76bf970bc3315
When the multi-line log format is enabled, the progress bar usually occupies
multiple lines on the screen. When stopping the progress bar, only the last
line was wiped, leaving all others visible on the screen. Erase all lines
belonging to the progress bar to prevent these leftovers.
Asking the user for input is theoretically affected by a similar issue, but
this is not observed in practice since the only place where the user is asked
(whether configuration options coming from flakes should be accepted) does not
actually have multiple lines on the progress bar. However, there is no real
reason to not fix this either, so let's do it anyway.
Change-Id: Iaa5a701874fca32e6f06d85912835d86b8fa7a16
The AcceptFlakeConfig type used was missing its JSON serialisation definition,
so it was incorrectly serialised as an integer, ending up that way for example
in the nix.conf manual page. Declare a proper serialisation.
Change-Id: If8ec210f9d4dd42fe480c4e97d0a4920eb66a01e
The JSON serialisation should be declared in the header so that all translation
units can see it when needed, even though it seems that it has not been used
anywhere else so far. Unfortunately, this means we cannot use the
NLOHMANN_JSON_SERIALIZE_ENUM convenience macro, since it uses a slightly
different signature, but the code is not too bad either.
Change-Id: I6e2851b250e0b53114d2fecb8011ff1ea9379d0f
There have been multiple setting types for paths that are supposed to be
canonicalised, depending on whether zero or one, one, or any number of paths is
to be specified. Naturally, they behaved in slightly different ways in the
code. Simplify things by unifying them and removing special behaviour (mainly
the "multiple paths type can coerce to boolean" thing).
Change-Id: I7c1ce95e9c8e1829a866fb37d679e167811e9705
Commit 0dd1d8ca1c included an accidental revert
of 1461e6cdda (actually slightly worse), leading
to the progress bar not being stopped properly when a legacy command was
invoked with `--log-format bar` (or similar options that show a progress bar).
Move the progress bar stopping code to its proper place again to fix this
regression.
Change-Id: I676333da096d5990b717a387924bb988c9b73fab
The growth of the seccomp filter in 127ee1a101
made its compilation time significant (roughly 10 milliseconds have been
measured on one machine). For this reason, it is now precompiled and cached in
the parent process so that this overhead is not hit for every single build. It
is still not optimal when going through the daemon, because compilation still
happens once per client, but it's better than before and doing it only once for
the entire daemon requires excessive crimes with the current architecture.
Fixes: https://git.lix.systems/lix-project/lix/issues/461
Change-Id: I2277eaaf6bab9bd74bbbfd9861e52392a54b61a3
This is a preparation for precompiling the filter, which is done separately.
The behaviour should be unchanged for now.
Change-Id: I899aa7242962615949208597aca88913feba1cb8
The seccomp setup code was a huge chunk of conditionally compiled
platform-specific code. For this reason, it is appropriate to move it to the
platform-specific implementation file. Ideally its setup could be moved a bit
to make it happen at the same place as the Darwin restrictions, but that change
is going to be less mechanical.
Change-Id: I496aa3c4fabf34656aba1e32b0089044ab5b99f8
When MANPATH is unset or contains an empty component, a reasonable default is
used. Previously (after 3dced96741), when MANPATH
was unset, the shell hook would only place a location containing the Lix manual
pages there, and system-wide manual pages would become unavailable in the
development shell, which is undesired. Fix the issue by including an empty
component in this case.
Change-Id: Ib3c67a831d709fe2a87520e15917eebb59397bd1
Expose an option for disabling the BDW-GC build dependency entirely. Fix the
place where one of its headers was included (unnecessarily) without proper
guarding. Finally, use this machinery to exclude BDW-GC from the ASAN builds
entirely (its usage has already been disabled due to compatibility issues
anyway), to ensure this configuration is not regressed again.
Change-Id: I2ebe8094abf67e7d1e99eed971de3e99d071c10b
In the SSH code, the logger was conditionally paused, but unconditionally
resumed. This was fine as long as resuming the logger was idempotent. Starting
with 0dd1d8ca1c, it isn't any more, and the
behaviour of the code in question was missed. Consequently, an assertion
failure is triggered for example when performing builds against an "SSH" store
on localhost. Fix the issue by only resuming the logger when it has actually
been paused.
Fixes: https://git.lix.systems/lix-project/lix/issues/458
Change-Id: Ib1e4d047744a129f15730b7216f9c9368c2f4211
Unfortunately, io_uring is totally opaque to seccomp, and while currently there
are no dangerous operations implemented, there is no guarantee that it remains
this way. This means that io_uring should be blocked entirely to ensure that
the sandbox is future-proof. This has not been observed to cause issues in
practice.
Change-Id: I45d3895f95abe1bc103a63969f444c334dbbf50d
Previously, system call filtering (to prevent builders from storing files with
setuid/setgid permission bits or extended attributes) was performed using a
blocklist. While this looks simple at first, it actually carries significant
security and maintainability risks: after all, the kernel may add new syscalls
to achieve the same functionality one is trying to block, and it can even be
hard to actually add the syscall to the blocklist when building against a C
library that doesn't know about it yet. For a recent demonstration of this
happening in practice to Nix, see the introduction of fchmodat2 [0] [1].
The allowlist approach does not share the same drawback. While it does require
a rather large list of harmless syscalls to be maintained in the codebase,
failing to update this list (and roll out the update to all users) in time has
rather benign effects; at worst, very recent programs that already rely on new
syscalls will fail with an error the same way they would on a slightly older
kernel that doesn't support them yet. Most importantly, no unintended new ways
of performing dangerous operations will be silently allowed.
Another possible drawback is reduced system call performance due to the larger
filter created by the allowlist requiring more computation [2]. However, this
issue has not convincingly been demonstrated yet in practice, for example in
systemd or various browsers. To the contrary, it has been measured that the the
actual filter constructed here has approximately the same overhead as a very
simple filter blocking only one system call.
This commit tries to keep the behavior as close to unchanged as possible. The
system call list is in line with libseccomp 2.5.5 and glibc 2.39, which are the
latest versions at the point of writing. Since libseccomp 2.5.5 is already a
requirement and the distributions shipping this together with older versions of
glibc are mostly not a thing any more, this should not lead to more build
failures any more.
[0] https://github.com/NixOS/nixpkgs/issues/300635
[1] https://github.com/NixOS/nix/issues/10424
[2] https://github.com/flatpak/flatpak/pull/4462#issuecomment-1061690607
Change-Id: I541be3ea9b249bcceddfed6a5a13ac10b11e16ad
In f047e4357b, I missed the behavior that if
building without a dedicated build user (i.e. in single-user setups), seccomp
setup failures are silently ignored. This was introduced without explanation 7
years ago (ff6becafa8). Hopefully the only
use-case nowadays is causing spurious test suite successes when messing up the
seccomp filter during development. Let's try removing it.
Change-Id: Ibe51416d9c7a6dd635c2282990224861adf1ceab
Commit 0109368c3f missed to include a required
header, which is not noticed when the precompiled header is enabled because
it's included in that. Also include it in the file so that the build without
precompiled header works too.
Change-Id: Id7a7979684b64f937f7f8191612952d73c113015
Due to a leftover from a previous version where the buffer was allocated on the
stack, the change introduced in commit 4ec87742a1
accidentally passes the size of a pointer as the size of the buffer to the
decompressor. Since the former is much smaller (usually 8 bytes instead of 64
kilobytes), this is safe, but leads to considerable overhead; most notably, due
to excessive progress reports, which happen for each chunk. Pass the proper
buffer size instead.
Change-Id: If4bf472d33e21587acb5235a2d99e3cb10914633
* changes:
doc/release-notes: add for pretty printing improvements
libexpr/print: do not show elided nested items when there are none
libexpr/print: never show empty attrsets or derivations as «repeated»
libexpr/print: pretty-print idempotently
When the configured maximum depth has been reached, attribute sets and lists
are printed with ellipsis to indicate the elision of nested items. Previously,
this happened even in case the structure being printed is empty, so that such
items do not in fact exist. This is confusing, so stop doing it.
Change-Id: I0016970dad3e42625e085dc896e6f476b21226c9
The repeated value detection logic exists so that the occurrence of large
common substructures does not fill up the screen or the computer's memory.
However, empty attribute sets and derivations (when their detection is enabled)
are always cheap to print, and in practice I have observed them to make up a
significant majority of the cases where I was annoyed by the repeated value
detection kicking in. Furthermore, `nix-instantiate --eval` already disables
this logic for empty attribute sets, and empty lists are already exempted
everywhere. For these reasons, always print empty attribute sets and
derivations as what they are.
Change-Id: I5dac8e7739f9d726b76fd0521ec46f38af94463f
When pretty-printing is enabled, previously an unforced thunk would trigger
indentation, even when it subsequently does not evaluate to a nested structure.
The resulting output looked inconsistent, and furthermore pretty-printing was
not idempotent (since pretty-printing the same value again, which is now fully
evaluated, will not trigger indentation).
When strict evaluation is enabled, force the item before inspecting its type,
so that it is properly known whether it contains a nested structure.
Furthermore, there is no need to cause indentation for unforced thunks, since
the very next operation will be printing them as `«thunk»`.
This is mostly a port of https://github.com/NixOS/nix/pull/11100 , but we only
force the item when it's going to be forced anyway due to strict
pretty-printing, and a new test was written since the REPL testing framework in
Lix is different.
Co-Authored-By: Robert Hensing <robert@roberthensing.nl>
Change-Id: Ib7560fe531d09e05ca6b2037a523fe21a26d9d58
The insertion marker comment broke the list into two parts, the first
containing only the link to the upcoming release notes and the second the
past releases. This confused the generator, leading to the first part being
discarded. Indent the marker comment so that it's syntactically part of the
preceding item, and in particular doesn't split the list any more.
Change-Id: I357c51bb03e4e0d79a76d30158615fd9eda95ea8
Due to a mistake in the grammar, a dollar character implicitly escapes a second
dollar character that immediately follows, so that it cannot start an
interpolation. Unfortunately, this behaviour has since come to be relied upon,
so it cannot be fixed. Furthermore, the documentation on regular strings did
not mention this behaviour at all, while in the case of indented strings it was
rather implicit.
Mention it explicitly in both cases, and describe how an interpolation can
follow a dollar character (namely, by escaping that). Since we have to touch
that section anyway, state that any character (other than n, r, and t; but
notably including `$` even if not succeeded by `{`) can be escaped using a
backslash in regular strings.
Change-Id: I7e5d68a9a4130eec98ce8218b485168f4b31a677
Activities can set display attributes in their log output using the "Select
Graphics Rendition" functionality. To prevent interfering with subsequent text
displayed, these should be reset after writing the log line. The multiline
progress bar neglected to do this, resulting for example in a colorised
"building …" header in the next line. Reset the attributes properly, like the
standard progress bar already does.
Change-Id: I1dc69f4a1d747a76b83e8721a72d9bb0e5554488
Previously, the progress bar had two subtly different states in which the bar
would not actually render, both with their own shortcomings: inactive (which
was irreversible) and paused (reversible, but swallowing logs). Furthermore,
there was no way of resetting the statistics, so a very bad solution was
implemented (243c0f18da) that would create a new
logger for each line of the repl, leaking the previous one and discarding the
value of printBuildLogs. Finally, if stderr was not attached to a TTY, the
update thread was started even though the logger was not active, violating the
invariant required by the destructor (which is not observed because the logger
is leaked).
In this commit, the two aforementioned states are unified into a single one,
which can be exited again, correctly upholds the invariant that the update
thread is only running while the progress bar is active, and does not swallow
logs. The latter change in behavior is not expected to be a problems in the
rare cases where the paused state was used before, since other loggers (like
the simple one) don't exhibit it anyway. The startProgressBar/stopProgressBar
API is removed due to being a footgun, and a new method for properly resetting
the progress is added.
Co-Authored-By: Qyriad <qyriad@qyriad.me>
Change-Id: I2b7c3eb17d439cd0c16f7b896cfb61239ac7ff3a
The `allow-flake-configuration` option allows the user to control whether to
accept configuration options supplied by flakes. Unfortunately, setting this
to false really meant "ask each time" (with an option to remember the choice
for each specific option encountered). Let no mean no, and introduce (and
default to) a separate value for the "ask each time" behaviour.
Co-Authored-By: Jade Lovelace <lix@jade.fyi>
Change-Id: I7ccd67a95bfc92cffc1ebdc972d243f5191cc1b4
Unfetched submodules are included as empty directories in archives, so they end
up as such in the store when fetched in clean mode. Make sure the same happens
in dirty mode too. Fortunately, they are already correctly represented in the
ls-files output, so we just need to make sure to include the empty directory in
our filter.
Fixes: https://github.com/NixOS/nix/issues/6247
Change-Id: I60d06ff360cfa305d081b920838c893c06da801c
Alois Wohlschlager