The variant has on the left-hand side the topologically sorted vector
and the right-hand side is a pair showing the path and its parent that
represent a cycle in the graph making the sort impossible.
The goal is to implement #551 which needs to throw an error if the
topo-sort fails. However, the error-message is supposed to contain a
graph of store-paths and the API to generate this is inherently async.
Now, catching the exception and re-throwing another one is impossible
since `co_await` is forbidden in `catch`-blocks and adding another
topoSort variant that allows an async `makeError` also seems odd. Hence,
I decided to alter the data-structure in use a bit for this use-case.
One out of two uses of the function are affected after all.
Change-Id: I70a987f470437df8beb3b1cc203ff88701d0aa1b
Closes#932
`connect-timeout` gets replaced by an exponential backoff for the
download timeout where the initial value is controlled by the setting
`initial-connect-timeout`.
Per iteration, the upper limit of the timeout is increased set to
timeout := min(max_connect_timeout, initial_connect_timeout * 2^i)
I decided to move the entire timeout / tracking of attempts into its own
class to not make the filetransfer implementation more complex. Also,
that allows us to write unit-tests for it.
Setting `--download-attempts` to `0` is forbidden now and an exception
will be thrown. For `--offline` we set it to `1`, the behavior is
equivalent to what it was before: whether the max tries were exceeded is
only checked after the first download exception got thrown, i.e. there's
still one attempt being made.
The end-result - with timeouts being caused by a wrongly set proxy -
looks like this:
$ env HTTPS_PROXY=1.1.1.1 nix store ping --store https://example.com
warning: error: unable to download 'https://example.com/nix-cache-info': Connection timed out after 5006 milliseconds (curl error code=28); retrying in 422ms ms (attempt 1/5)
warning: error: unable to download 'https://example.com/nix-cache-info': Connection timed out after 10010 milliseconds (curl error code=28); retrying in 1003ms ms (attempt 2/5)
warning: error: unable to download 'https://example.com/nix-cache-info': Connection timed out after 20020 milliseconds (curl error code=28); retrying in 2018ms ms (attempt 3/5)
warning: error: unable to download 'https://example.com/nix-cache-info': Connection timed out after 40007 milliseconds (curl error code=28); retrying in 4087ms ms (attempt 4/5)
error: unable to download 'https://example.com/nix-cache-info': Connection timed out after 80074 milliseconds (curl error code=28)
Change-Id: I9e8d08d78275bcf60080d663febc9e075243d36b
Concept: what if you could, in your fancy terminal, in the year of our
lord 2025, just click on the attrs you're looking at to go to where
they're defined. Currently we only expose this info as
builtins.unsafeGetAttrPos, which is inconvenient as it's not
discoverable to users.
By putting it in this more visible yet invisible spot, it's more likely
to be more useful to more people.
In the current state, this is not the most useful ever due to stuff like
https://github.com/neovim/neovim/discussions/35097. However, it can be
expanded by perhaps adding something like the url format setting ripgrep
has.
Change-Id: I3947f97d5c2056d59099af468d7b855486438227
this is an equivalent of the regular kj read interface which also takes
a min/max pair. we do not need this very often though, so we'll keep it
as a separate method for now. if we do find we use it more than read we
can still rename read to readSingle and readRange to read. we will see.
Change-Id: Ib04ca146911adae7081cf4b2df097217ea5fe9f8
Usually, EOFs are represented by returning 0 in the `read` APIs, at
least, this is what read(2) dictate.
As clever creature, we may sum zeroes sometimes (advanced form:
`buf->added(got)`) and forego handling the EOF condition.
To avoid the bug that lurked in remote-store.cc and caused busy looping
if the remote end disconnects suddenly, we return
`Result<Option<size_t>>` forcing the caller to perform a specific
processing for the EOF situation.
The conversion did not raise any other offending code path.
Change-Id: I185fdcb77aa82d87ab0802d66ac37c1363657a73
Signed-off-by: Raito Bezarius <raito@lix.systems>
This is useful for certain error recovery paths (no pun intended) that
does not thread through the original path name.
Change-Id: I2d800740cb4f9912e64c923120d3f977c58ccb7e
Signed-off-by: Raito Bezarius <raito@lix.systems>
processStderr of RemoteStore wants to be a promise and it must be used
from connection setup, so the pool factory callback must be a promise.
Change-Id: I9ac742b6048ae6dba0bfa5dcb58971386229690b
the rpc transition will require sync and async objects to share a single
io buffer (since defining serializers on async is an immense pain in the
tail, slow, and ultimately not necessary). a generic buffer class allows
us to reuse existing serializers more readily (reuse them at all, even).
Change-Id: I5ebba8449f26f2bb76016818928183c7e0123be0
the old protocols are largely untested, mostly unused, and have design
problems that make the RPC transition a lot harder, if not impossible.
in theory we could ship a transparent protocol-converting proxy that'd
isolate the daemon itself from old protocol versions, but that's a lot
of code to maintain for presumably little gain or even no gain at all.
Change-Id: I4c3f3bb34d39044f6aeb07c10caaf13b8340a220
This moves the original test suite for `filterANSIEscapes` into the same
file as the newer tests. There is some overlap between the old and new
tests but that doesn't hurt anything so I kept them as-is.
Change-Id: Id00000009919024a5f206ec9a7bc0022541ff612
This teaches `filterANSIEscapes()` how to find the end of an OSC
sequence. It also keeps OSC 8 (hyperlinks) when not instructed to filter
out all escapes, just as it keeps colors.
This also relaxes the parsing of CSI escapes to find the end of the
sequence for invalid sequences, and handles better escapes that don't
start CSI or OSC.
This fixes the repl output for `:doc builtins.fetchGit`.
Fixes: https://git.lix.systems/lix-project/lix/issues/160
Change-Id: Id0000000f2a6956c042c883a4545edf347fa1799
This was an absolute nightmare to diagnose. It turns out there's a
kernel bug: poll with events = POLLHUP will receive an event for NOT
POLLHUP internally in the kernel, delete their event subscription, and
then not receive events for any HUP later. lol! lmao!!
We choose to use plain old EVFILT_READ because the watched fd can be
either a socket or a pipe and it's preferable to eat some spurious
wakeups than have separate paths for those. The alternative is using
EVFILT_SOCK, a private API that's existed for years and which netty
uses for its sockets, but that doesn't work on pipes.
Fixes: https://git.lix.systems/lix-project/lix/issues/729
Change-Id: If72b5d7a39f00320a9acccdbe81121cdb1a04c45
otherwise lix may crash when e.g. nix search receives invalid regex.
we now also give better error messages for regex errors during eval.
fixes#803
Change-Id: Icc7c578ff488ba520efac5d898572ccf4486e9a8
This could previously crash lix:
Before:
$ nix eval -E '{type="derivation"; drvPath="";}'
nix: lix/libutil/file-system.cc:45: Path nix::canonPath(PathView, bool): Assertion `path != ""' failed.
Aborted (core dumped)
After:
$ nix eval -E '{type="derivation"; drvPath="";}'
error:
… while evaluating the drvPath of a derivation
at «string»:1:21:
1| {type="derivation"; drvPath="";}
| ^
error: path '' is not in the Nix store
Fixes#536
Change-Id: I406dc9e58047be8f263cf2e4bc3ed5da75a46602
There is a pull request [1] addressing these upstream--it doesn't appear
likely to be merged anytime soon though... this is a no-op til we enable
-Wdeprecated-declarations, but helps in the direction of #744
[1]: https://github.com/emil-e/rapidcheck/pull/325
Change-Id: I27e2c7d81df152de8674696f2a56d5f21c414ce3
we generally do not want to catch or throw these. catching them to print
and discard is fine, tests are largely exempt, and cases in which we can
be certain where the exception came from are also fine to *catch*. we'll
try to never *throw* (or rethrow) these if possible though because doing
so will make it impossible to construct async traces for the exceptions.
Change-Id: I3b71c32ecd16afc2246c946472f5629a1fa31f2c
Well, I was trying to figure out
https://git.lix.systems/lix-project/lix/issues/729 in which this feature
is clearly just broken on macOS, but frustratingly, it seems that it
*does* work, except for the daemon. um........ sure.
Change-Id: Iaa962f045c16fdfa82854151c90a03e5cf0eea47
Yikes!! I wonder if we have any other ones of these that just .. didn't
get added to a meson file?
Change-Id: I19480ab03cdbecf608e523d5b6c3980233f4f445
or more accurately, wrap them in a nix::Error subclass so we can display
them properly without crashing, and add some error context if available.
fixes#642fixes#753fixes#759fixes#769
Change-Id: I1aad0c0501fea83f9de3a1335eaa6adc20721616
allow opting in to serialization as integers via a trait type instead,
and add string-list serializers for the feature flag set enumerations.
fixes#738
Change-Id: I2746eb5ef1f15c01b4e681f9ba1615b6c6e64f44
we want to own this specialization fully so we can change the default
serializer behavior without also forcing downstream users of our code
to use the same behavior. it'll also let us do things we cannot do in
regular nlohmann::json, such as selectively enabling serialization of
enums as integral types, or using `to_json`/`from_json` overloads for
not-default-constructible types instead of serializer specializations
Change-Id: I91a1db362e37d654090f1824b1cd3ce783d32134
this will become a proper specialization of `nlohmann::basic_json` soon.
specialing basic_json will let us get rid of our `adl_serializer` hacks,
and it'll open the door to better enum serializing behavior without also
forcing all those who use lix as a library to set certain defines (which
may not even be possible depending on how those users use json already).
Change-Id: I5228d2b9df581a189552c993363207cfbd20f445
this doesn't do much, just wrap a few nlohmann headers in headers of our
own (and delete includes we don't need because they're transitively seen
by other includes). doing this now will make the next change much nicer.
Change-Id: I166933102ea86bb5322ebbf9ba9411f96032a53b
if a copyNAR generator was not drained to completion it would not read
the full nar data from its source. this could happen if the copier was
passed to parseAndDump wrapped as a source because copyNAR would yield
nar metadata *before* it had read it, and GeneratorSource will drain a
generator fully *only* if the source is allowed to throw EndOfFile. in
the parseAndDump case this never happened because parseAndDump expects
to be given an unterminated stream, and thus the combination left some
nar metadata in the input Source, breaking the remote store protocols.
fixes#732
Change-Id: Ia59a53375992bfcdb7bc6b37764ca779622bc8f7
well, oops. on slow io (as can happen with ssh remote builders) we could
have extended a nar read buffer past what was actually read, injecting a
span of zeroes into the read buffer where we requested some data but got
a partial result instead. also add some tests that would've caught this.
Change-Id: I67aa06b4715aeec6a5bdacaafa9b79849e664e2f
connection pools of remote stores can currently block. this is not a
problem when each request runs on a dedicated thread, but with async
code this is no longer true. if a remote store has exhausted all its
available connections on one executor and another job starts *on the
same executor* we'll deadlock if that job makes another remote store
request. unlike with sqlite previously it's not reasonable, not even
necessary, to make the pools unbounded: since we use pools only with
remote stores we can asyncify all of them at once, and since they're
leaves of all call stacks we do not have much code to change either.
Change-Id: I8c457e27893e22c2cfc35933307a5283c986805a
sadly this is a visitor-only interface; async generators are not yet a
thing and preliminary benchmarks say that overhead would be too large.
Change-Id: I0460d18eba94441cf3101d46bfffdb69cdda81d1
we'll use this in NarAccessor to provide actually safe indexing of
archives. NarAccessor currently is not fully correct: it relies on
the parser not buffering anything to produce correct file offsets,
but only the nar implementation itself can reasonably expect that.
Change-Id: I64f300b86d8844b876a3ace723546ea7d7b4628b
this is where they should've been from the start, but during the first
rewrite it made little sense to move them. we have bigger plans today,
so we'll finally clean that up too. note the `Map` transform type that
is needed to make the current macros work. it shall be only temporary.
Change-Id: I928d197dbfe27b68cf8634d149c3259e86fbf123
4138fc7622 mistakenly removed an early
exit from non-main worker threads. this led to exceptions not ending
`process()` calls in a timely manner and instead draining the entire
work queue first, which for e.g. Interrupted errors would cause many
duplicated reports per worker thread instead of only one per thread.
it also did not properly rethrow a work item exception in all cases,
e.g. when all work had completed by the time `process()` was called.
due to a mistake in the thread starting check it was possible that a
system would require n² work items to start n worker threads if some
work items process quickly enough while other items block for a bit.
Change-Id: I7d59483580cda1c19980f9296074216a0fbf2c4e
references remain forbidden because std::optional does not want to
contain them, and specializing generators to use pointers where we
can't use optionals is simply too much work for a feature we don't
even need. reference wrappers and bindings still work well enough.
Change-Id: I2e6ca74719584ce16e2357c452fdd5c5a9e23d5a
Maximilian Bosch