In the first pass I erred on the cautious side, only migrating safe
bets, to here's some trivial migrations that I missed
Change-Id: I934011919837b0aa491113afdcad603cf6b9cbbb
Closes#987
The patch adds a flag `--no-instantiate` which only performs evaluation
without instantiating any derivations. Hence, GC root creation is also
skipped. To achieve that, Lix is also put in read-only mode and all
operations that require reading a derivation (e.g. constituents or
listing input derivations) are disabled fallback values are set.
This is a port of an upstream PR[1]. Given the divergence of the
codebases (different restructurings on both ends, no more CA derivations)
I decided to redo large portions from scratch instead of
cherry-picking the patches. Hence, the authorship.
Additionally the clean up of casts down to a local store are removed or
guarded behind an if, as done in the upstream PR.
[1] https://github.com/nix-community/nix-eval-jobs/pull/379
Co-authored-by: Jörg Thalheim <joerg@thalheim.io>
Change-Id: Ib84f44e7799bc5577fd2ee98912458f16ebeab81
95448347 made lix require libatomic if the platform is able to link a
simple program using atomics, but it should actually be the other way
around. We need to require libatomic if it fails.
Change-Id: I6a6a6964ca6ee90a59314ddf1865753e83713772
using sleep(1) as a synchronization mechanism does not work. use fifos instead.
fixes#690 and ci constantly falling over in the same exact fucking source line
Change-Id: I51725f8e439b6753f3212d2897dbb0620ad77a37
It's pretty bad that Bindings effectively wasted whole 8
bytes (4 for capacity and + 4 for alignment padding) to
store something it doesn't actually need. BindingsBuilder
allows the capacity to be checked at construction time,
after which the Bindings does not get mutated aside from
the ugly case of builtins, which doesn't get built all
at once.
For `nix search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello`
this shaves off around 53MB allocations out of 2GB used
for attrsets in total:
< "bytes": 2001170768,
---
> "bytes": 1947398072,
< "Bindings": 16,
---
> "Bindings": 8,
Nix PR: https://github.com/NixOS/nix/pull/13919
Change-Id: I939c5ac545f5abbca048370dcf4936346339d75c
Some platforms like 32-Bit PowerPC need linking against libatomic.
Try to compile and link a very simple snippet of code which uses atomics
and make libatomic required if it fails.
Because we're using `dependency('atomic')`, the required meson versions
gets bumped to 1.7.0. See https://mesonbuild.com/Dependencies.html#atomic-stdatomic
Change-Id: I6a6a696471e1d352fb161c537ba9023b97c2d31e
libarchive is not async and cannot be used async without involving green
threads, which have already proven to be very problematic. unpacking tar
archives is rare enough that spawning a new thread for each shouldn't be
too much overhead, and the additional data copy probably also won't hurt
performance too much. we may even benefit from being able to extract not
just one archive per event loop but as many archives as we can keep fed.
Change-Id: Iece82bd566ada0a2a49de54c4e69caf6d93f6720
using a sink for this has long been a bit weird anyway. originally it
was necessary due to api limitations, but it hasn't been for a while.
Change-Id: I3dfa157944618349bfd6f398ee1667fc31519d86
Without https://github.com/NixOS/nixpkgs/pull/434761 evaluation of the
`nixpkgsLibTests` will fail in CI with recent enough Lix, due to reliance on
the TOML integer saturation bug.
Reported-by: Sergei Zimmerman <sergei@zimmerman.foo>
Change-Id: I6a6a6964838009d2c525f67035f84072fdfad988
Fixes: https://git.lix.systems/lix-project/lix/issues/973
Information about which commands were executed is really valuable to
debug Lix and is much more user relevant than the vast majority of the
e.g. build loop junk printed at debug level. Currently we have a *whole
lot* of call sites where we call execv* which should probably be cleaned
up, but that's future work.
I chose to print argv0 rather than the executable path if these differ,
since the code is shorter and since the command could be a fully
resolved symlink or so where argv0 is the actual command name being run.
However, it's not exactly *hard* to write std::ranges::drop_view(args,
1).
Change-Id: I73c3abb20b229d5e2d64277aa29cbbeed7764bab
printTaggedWarning already colorized its messages. we can do the same
for most other log messages.
Change-Id: Idcd31bbf4f8d0d703395b0d2b7b9bc33264d969f
luckily none of these a format strings vulnerabilities because
boost::format is smart enough to throw an exception when given
fewer format string arguments than are requested by specifiers
Change-Id: I5fa78f0d1396263271f6e1dbcee9c0b2e9e18c34
always use log macros, which also have the benefit of respecting the
verbosity setting without needing virtual function calls to read it.
Change-Id: I1c605562a53e54140724d5225e040abcf49ac996
we add two variants: one that just prints a message at the warning
level (mirroring the other printer macros), and one that also adds
the colored "warning: " prefix the function added. since there are
no overriders of this function in tree it looks safe to remove it.
Change-Id: I7008fd0f31d59fbc9259472e29359c8df19ff87d
mostly useful for nix-eval-jobs which currently has to call the logger
functions directly because its main code *isn't* in the nix namespace.
Change-Id: Ia8440d86a293d9006ffef2562b1859e9aaa79a62
Previously two cryptography libraries were linked into Lix: OpenSSL used for
hashing and (in usual configurations) indirectly via curl for TLS, and Sodium
used only for handling the Ed25519 path info signatures. The latter is
functionally redundant since OpenSSL supports the same use case as well.
Reimplement the Ed25519 handling using OpenSSL and drop Sodium.
Fixes: https://git.lix.systems/lix-project/lix/issues/969
Change-Id: I6a6a696456b9d3ad7fdc2bf9b0759836a6247a38
Currently, DerivationGoal prints a pretty generic message.
For many valid reasons, children may have better knowledge of the detail
of what has happened and would like to extend the error message.
What we did is to printError at convenient places but this is
counterproductive because the build error can bury the notes.
This is still not perfect because there's no fine-grained structured
information that children can use to act upon the generic messaging, but
this is already an improvement for LocalDerivationGoal and keep failed
which will occur in the next change.
Change-Id: I5835cbbb30c4f2aa64abefb83999018d30ca4a0c
Signed-off-by: Raito Bezarius <raito@lix.systems>
It was only used for impure derivations, which were finally removed in commit
be07629820. Delete the unused function.
Change-Id: I6a6a696481711f68a8c3ea7eac7978fcf5884cce
Closes#551
This adds a special accessor that falls back to checking if a store-path
exists within a chroot if it's not a valid path. That way,
`genGraphString` can find out which files have which references before
the outputs are registered.
Change-Id: I03c9d508fa3c72e5c262194461a25d71f3f4de15
That way it's possible to inherit from LocalStoreAccessor to implement
special behavior such as an accessor that falls back to the chroot
directory if it can't find a store-path (which is what we'll do in the
next commit).
Change-Id: If689eb3f410e81e629f1d13cc2b48594fecb1001
That way we do now have linear complexity to determine output
references per output within each step of the topological sort.
Instead, this is done before and the topo-sort only filters the output
map for other derivation outputs.
Following up on this, we can re-use `outputGraph` to generate a tree
with references to display which files cause an output reference cycle
if needed.
Change-Id: Ibdd46e7b2e895bfeeebc173046d1297b41998181
If none is given, we fall back to whatever accessor we get from the
store.
To display which paths actually contain the references leading to
e.g. a cycle or triggering a disallowedRequisites error, we'd
potentially have to look into the chroot from the previously finished
build. This behavior should not be part of the local accessor by
default, but part of a "special" accessor. This change allows using such
an accessor for `genGraphString()`.
Now that we inject the accessor from the outside, we have to mock it
anyways in the tests. Hence, this also adds a testcase for the
precise=True case.
Change-Id: I58465fb944776c2b0262ba054d1f296ed2ae3406
The variant has on the left-hand side the topologically sorted vector
and the right-hand side is a pair showing the path and its parent that
represent a cycle in the graph making the sort impossible.
The goal is to implement #551 which needs to throw an error if the
topo-sort fails. However, the error-message is supposed to contain a
graph of store-paths and the API to generate this is inherently async.
Now, catching the exception and re-throwing another one is impossible
since `co_await` is forbidden in `catch`-blocks and adding another
topoSort variant that allows an async `makeError` also seems odd. Hence,
I decided to alter the data-structure in use a bit for this use-case.
One out of two uses of the function are affected after all.
Change-Id: I70a987f470437df8beb3b1cc203ff88701d0aa1b
it's broken, can write arbitrary file paths when run as root, and only
supports strings and recursive sets of strings. this was only used for
manpage generation in a build system that has not woken up since 1976.
fixes#974fixes#227
Change-Id: I4f18599685a3077c15ddc02c759558f986c8c6e4
Commit 5dc847b47b introduced it as a non-inline
function with definition in the header, which can result in linker errors like
the following:
/build/source/build/lix/libutil/backoff.hh:36: multiple definition of `nix::backoffTimeouts(unsigned int, std::chrono::duration<long, std::ratio<1l, 1000l> >, std::chrono::duration<long, std::ratio<1l, 1000l> >, std::chrono::duration<long, std::ratio<1l, 1000l> >)'; tests/unit/liblixutil-tests.p/libutil_backoff.cc.o:/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-gtest-static-x86_64-unknown-linux-musl-1.17.0-dev/include/gtest/gtest-printers.h:1223: first defined here
This error was observed during trying to bump `lixPackageSets.git` in nixpkgs.
I am not sure why it can't be observed in the in-tree `nixStatic` package but
the definition is wrong in any case.
Change-Id: I6a6a6964e218a03ca2a2e8eddbb72d44e06e904e
Closes#932
`connect-timeout` gets replaced by an exponential backoff for the
download timeout where the initial value is controlled by the setting
`initial-connect-timeout`.
Per iteration, the upper limit of the timeout is increased set to
timeout := min(max_connect_timeout, initial_connect_timeout * 2^i)
I decided to move the entire timeout / tracking of attempts into its own
class to not make the filetransfer implementation more complex. Also,
that allows us to write unit-tests for it.
Setting `--download-attempts` to `0` is forbidden now and an exception
will be thrown. For `--offline` we set it to `1`, the behavior is
equivalent to what it was before: whether the max tries were exceeded is
only checked after the first download exception got thrown, i.e. there's
still one attempt being made.
The end-result - with timeouts being caused by a wrongly set proxy -
looks like this:
$ env HTTPS_PROXY=1.1.1.1 nix store ping --store https://example.com
warning: error: unable to download 'https://example.com/nix-cache-info': Connection timed out after 5006 milliseconds (curl error code=28); retrying in 422ms ms (attempt 1/5)
warning: error: unable to download 'https://example.com/nix-cache-info': Connection timed out after 10010 milliseconds (curl error code=28); retrying in 1003ms ms (attempt 2/5)
warning: error: unable to download 'https://example.com/nix-cache-info': Connection timed out after 20020 milliseconds (curl error code=28); retrying in 2018ms ms (attempt 3/5)
warning: error: unable to download 'https://example.com/nix-cache-info': Connection timed out after 40007 milliseconds (curl error code=28); retrying in 4087ms ms (attempt 4/5)
error: unable to download 'https://example.com/nix-cache-info': Connection timed out after 80074 milliseconds (curl error code=28)
Change-Id: I9e8d08d78275bcf60080d663febc9e075243d36b
Currenlty, when a test group is invalid already, we also throw unsued
file errors.
This leads to clutter as more often than not, the unused files are
caused by an invalid configuration, making the debug stack bigger
without reason.
With this commit the behavior is changed to only error about unused
files, when no other configuration issues were found
Change-Id: I92a819753f13b8ed5a07dae53ecaee5d84b5ce64
Currenlty one is required to always write the bulky `mark.parametrize`
with indirect and things
This commit adds a custom decorator for usage of files, which hides the
parametrization complexity from the user.
Change-Id: I526e016d12006669dc302dfc5af619735399c503
piegames