This is mostly to deal with fatal errors in NixOS tests, but it's useful
for all cases; assertion failures landing in syslog would be nice too,
but that's not as easy to do.
CC: https://git.lix.systems/lix-project/lix/issues/693
Change-Id: I6b5bd9800db3f5affe33f2946ea218e4a78f0c07
This also adds documentation for it in distributed-builds.md as that's
possibly the most common use-case for remote ssh stores
Support for `std::optional<uint16_t>` in Setting is also added since
setting a port is optional.
The line `#include "lix/libutil/strings.hh"` fixes that templates instanciations
in lix/libutil/config-impl.hh were using string utils without including
the header (why are they even there btw)
Change-Id: Id806c117c48cdf158d9d1cb1e639b0df31d9bf11
also mark the sync version as NeverAsync. a blocking wait on sqlite
locks in a coroutine may never finish if it's a different coroutine
on the same executor that is holding the lock, not another process.
this propagates to the sqlite core interface, but no further. we'll
assume that caches do not block on a database for very long, and we
can't reasonably propagate never-async-ness out of stores unless we
touch everything we'd touch for the async transition, again, twice.
store code already assumes that it can block for however long it'll
feel like that moment. we keep thread pools around for this reason.
Change-Id: I62f77e1ac333cbe2e4e646dbcb1571463f2cf3fc
this does not change the API exposed by the library, so it'll be safe
for users to substitute stock capnp. we do want it for our own builds
though, at least until we can switch to capnp 2 or fully go for rust.
Change-Id: Ia8ac6af0165f6f61ee02345179dc0a93a0889a06
4138fc7622 mistakenly removed an early
exit from non-main worker threads. this led to exceptions not ending
`process()` calls in a timely manner and instead draining the entire
work queue first, which for e.g. Interrupted errors would cause many
duplicated reports per worker thread instead of only one per thread.
it also did not properly rethrow a work item exception in all cases,
e.g. when all work had completed by the time `process()` was called.
due to a mistake in the thread starting check it was possible that a
system would require n² work items to start n worker threads if some
work items process quickly enough while other items block for a bit.
Change-Id: I7d59483580cda1c19980f9296074216a0fbf2c4e
* changes:
fix: pre-build-hook did not run when useChroot == false
libstore/build: say what was failing when pre/post build hooks fail
daemon: complain much louder about unknown std::exception instances falling out
The lix daemon wants to avoid orphan processes outliving a lix build.
In order to do that it kills all processes under the build user's UID
after and before a build.
When using sandboxes under Linux, this is unecessary, as builds are run
inside a PID namespace, which guarantees that processes cannot outlive
the "init" process.
Partially fixes https://git.lix.systems/lix-project/lix/issues/667
Change-Id: Idb2cdaad30169b75d730e8a18b360330516faf8b
Upgrading a read transaction to a write transaction will immediately
fail if another connection/process started a write transaction in the
meantime. For transactions that we expect to perform a write after a
read, starting with `BEGIN IMMEDIATE` gives us a write transaction
immediately instead of beginning with a read transaction and upgrading
it.
Change-Id: I82b8a3843382940a4fa4ffe45f30b365b690ff86
It turns out there is actually some history here.
Eight years ago, in 7f5b750b40, the
pre-build-hook condition got changed to check that useChroot is true
(which we will put down as "eelco making a mistake" because it makes no
sense) and that this->drv was a Derivation (as opposed to a
BasicDerivation as would be the case if the derivation was not present
on disk). The intent of this change was that pre-build-hook would not
run when the derivation doesn't exist, so that some hydras would not
explode.
However, this broke later when both cases became Derivation such that
AFAICT it will always run it if useChroot is true, which is absolutely
not the intended behaviour, but it is a reasonable interpretation as
well, just inconvenient for certain pre-build-hook usages.
So, I think the safest and most compatible behaviour is to run the
pre-build-hook but not guarantee that the derivation actually exists on
disk, since it already didn't run in many cases. Maybe the more correct
way is to actually write out the derivation if we are running a
pre-build-hook, but post-build-hook has exactly the same situation and
it's expected, so idk.
Fixes: https://git.lix.systems/lix-project/lix/issues/674
Change-Id: I75b828ae2a07ab373f44083507eb243459bd081d
Specifically, for errors where the provided and expected hashes differ.
Also, I've updated `fetchTarball`'s docs to explain the allowed values
for the `sha256` attribute.
Change-Id: Ic9aa777dc0a3c1ccdb4c98048745067f529bd986
Using time_t meant a warning could be printed arbitrarily soon depending
on when the second rolled over relative to when the SQLite transaction
was attempted.
Change-Id: I93401aea65f90e37cd450a293c4dd4d64fba7628
the two instances that used synchronous locking from promises weren't in
danger of deadlocks: both only serialized access to the sqlite database.
Change-Id: I2041a7a31f39cffde2080d0bb2402e2ead8d953f
while these two are currently safe using synchronous mutexes we can't
easily guarantee this forever. since we have async mutexes too now we
may as well use them; the overhead they incur is small enough that we
can just ignore it here. this can be revisited later, when necessary.
Change-Id: Ia4cb2361d5638aa48e3b982a05e75ae9c4f254cd
we can't use Sync with stl mutex types since yielding from one coroutine
to another while holding a lock is a deadlock hazard. if we don't expose
blocking lock operations from Sync though and emulate blocking with more
promises we can reuse most of the Sync code instead of rewriting it all.
hiding them completely would make it impossible to migrate current users
to promise-based code though, so we use our new NeverAsync lint instead.
Change-Id: I7eebf18e29473963040e7448da27c52012f1f134
Robert made these improvements on the PR that cherry-picked
e55cd3beea /
https://gerrit.lix.systems/c/lix/+/2057
from Lix into CppNix:
https://github.com/NixOS/nix/pull/11921
Push log source description out of libutil and report build hook @nix warning correctly
(cherry picked from commit 03d4bfd852dce9a050f984e887c887a43581796c)
test: Move unusual-logging to run only in logging test case
(cherry picked from commit 1421420e862434321c46511a3152016e443dd479)
Remove redundant warning: prefix from structured build log warning
(cherry picked from commit f3c722cab24f7a0de8c3573d25e91749f4f16234)
Change-Id: I7da99046f2a41b3c58e62351119bc89bcc25a703
gc tryLocks first, then deletes. we can use the same order since there's
no sequence of operations that would let us lock our new directory *and*
have the gc delete it (except outright bugs). the worst possible outcome
is that we create a few directories that'll be deleted by a later gc run
Change-Id: I6d3524f4990f8deee10804e6fbb0d0a26f3d57fe
we now create a temporary file, lock it, move it into place, and return.
if any of these steps (except creation) fails the GC must have found and
deleted our new file before we got the chance to use it. we don't need a
lock around file creation anymore, we especially don't need a mutex that
covers an flock call, and we don't need the GC to clobber what it finds.
(clobbering *is* kept for compatibility with older nix implementations.)
if we ever leave trash around it'll be cleaned up by some future gc run.
Change-Id: I078ae55fb7a9dfcdae28d6917d138842188ed528
sadly we need both a synchronous and an asynchronous promise for this
since destructors cannot be async. we also cannot use forked promises
since multiple threads may be waiting for an auto-gc to complete, but
forked promises are bound to an event loop (and thus a single thread)
Change-Id: I7b4fdbd229c4a1e01bbf80858e93347024f2e333
constructors can't be async, and root creation must be made async.
creating roots outside of goal classes and passing a witness type
is a lot easier than changing the constructor structure of today.
Change-Id: Ic84a92f3db2e1a9a9164047456f227048c6871bd
Jade Lovelace