we can't create these any more except by reading an old json-formatted
derivation that used them. since we cannot do anything with a deferred
derivation even when read we will remove json support for them as well
Change-Id: I4f9ea0b7c6469f57977784037f7710f939e40a2c
now that we have no deferred hashes (since floating ca derivations were
the only way to create them) we can safely remove this enumeration too.
Change-Id: Ic72ed90500fcee7aa5b3b5a302477fa515acf1be
only FODs can be content-addressed now, and those are always fixed.
FODs are also never sandboxed, so we do not need that field either.
Change-Id: I1be62b3ec85e08ec003cc8769723328d19777728
this mostly takes the form of removes feature checks and the associated
"ca derivations enabled" branches, but for the realisation info command
turns into a stub. we keep it around for compatibility, but from now on
it will always throws "ca derivations not implemented" errors when run.
Change-Id: I0abea5f76262013415330adcca2b498c6dca555b
this goal is only involved for output paths that aren't known at initial
build time, which in turn can only happen if they are ca paths. since we
can no longer create ca derivations during eval *or* read them from disk
we can now assume that we will never run this goal. there are still some
vestiges like output known-ness we can't remove yet, so those must stay.
Change-Id: I989e5ad4600c628bcbe8e17e1b082ce8d73a3bd9
we remove not only support for *building* a ca derivation, but also
support for *resolving* ca derivations as part of a build. we never
have to resolve derivations from here on, so this code is now dead.
Change-Id: I0346442d5fa00eb927177545ae61315f588477cc
we no longer have any experimental features depending on ca derivations,
so we can start removing them. since ca derivations are very invasive we
will need a while to remove all of the explicitly experimental code, and
even then we will not have removed *all* code related to ca derivations.
especially in the derivation goals there is a lot of code that is not as
easy to disentangle from experimental features as some would have hoped.
Change-Id: Ia456aadc6164613ded343f571318494d9310a549
only dynamic derivations could produce a non-opaque drvPath. since
dynamic derivations are no longer supported we can have drvPath be
opaque at all times, simplifying downstream code significantly and
making quite a few methods unnecessary. discardOutputPath was only
called on drvPath members anyway and thus reduces to a copy, other
operations at the very least are no longer recursive. some vestige
of dynamic derivations remains in DerivedPathMap though (for now).
Change-Id: Ifb4ad53a3c67800be5a62540068c8279d4ae0046
string context doesn't need any tests because it's never persisted or
shown to the user. getting rid of recursive string context means that
the context string parsers can be a lot simpler from here on forward.
Change-Id: I58443679ad76c0f28ea5f4eb8bfb3874f270e764
as with impure derivations it is still possible to garbage-collect
existing xp-dyn-drv derivations. we once again don't introduce any
new kinds of errors, we only change the dynamic type of exceptions
from MissingExperimentalFeature to UnimplementedError (although we
do throw FormatError when reading xp-dyn-drv derivations now, that
seems to make a little more sense than "feature not implemented").
Change-Id: Ic26e5b6c9c9e2533093e27f6cf901dc9db57c83e
only dynamic derivation produce text-hashed derivation outputs. toFile
produces text-hashed store paths, so we cannot remove text hashing now
without breaking stores, but we can disallow it in derivation outputs.
Change-Id: I95ff9882a59153a7d5fd509f5c9fd85925f30d02
with impure derivations gone we move on to dynamic derivations. this too
is not done in a single commit because dynamic derivations are invasive,
modifying semantics of all references to derivation output paths and all
derivation dependency calculations. removing dynamic derivations cleanly
is made significantly harder by the multiple did-you-mean-sum types, aka
"wrappers for std::variant", holding all derivation outpath information.
Change-Id: Ice7a7700c7b54c6a6061d4beb322b4175923d27a
writing them is technically still supported because what makes a
derivation impure is entirely specified by magically named data,
but without derivationStrict being able to pass these through to
libstore there is no way (besides reading existing files) to get
any new impure derivations into an existing store. it will still
be possible to garbage-collect existing impure derivations since
the gc process does not need to read them as derivations, and we
are not introducing any new kinds of unsupported-feature errors.
Change-Id: I648f53129ce67ee2b48d0591219759812dd557da
we don't remove the entire feature in one go to make review easier.
impure derivations are rather unintrusive on their own, at least if
we compare them to dynamic or ca derivations in general, so we will
be done with this soon. as it stands impure derivations cannot work
without ca derivations, and those we *really* want to leave behind.
Change-Id: I4f01d8d758b2c85dcd6c3078304b5ee1b52f65b0
ca derivations are what we're really after, but dynamic derivations
must also go because they depend on ca derivations. we can't easily
implement dynamic derivations any other way, so we remove them too.
impure derivations build on the content-addressed infrastructure in
ways we cannot easily detangle, so they too must go for time being.
see #815
Change-Id: If61371736dfd89cc71a1b2ae5a005757c3cb9484
it's an eval-time only setting, the daemon doesn't use it anywhere. this
is a hack, but until we have a much better settings system we are stuck.
fixes#680
Change-Id: I532088b0279f13da0a0a65c2bd2e5f9d1dfb39da
this can be extremely distracting when running in a repl that has access
to all of nixpkgs from its static env. not only do we print all bindings
in the stack trace, we also print them *again* every time we enter *any*
but the outermost frames. duplicating this environment information isn't
all that useful and mostly just flushes useful information out of sight.
fixes#816
Change-Id: I027e6a565e82663275e21333bb1d00d498cd4127
this way we don't have to even check whether we need a debug frame when
the debugger isn't enabled. not doing this gives us an eval performance
improvement of roughly 7% on nixos system eval and 2% for `nix search`.
Change-Id: I1cdad3de61f865ea54d6e09d63a281688e828768
we'll want to wrap some exprs for debug purposes, and dynamic casts
cannot look through such wrappers. dedicated casting functions can.
Change-Id: I1fba0ec52d281a1b8de85a62e4948bfae536bcfc
add a position parameter to the autocaller instead, and pass it much
more accurate position information where we have it easily available
Change-Id: If2f1c3006ca3f2b413492842905d079a8b752542
errors for individual attrs should report the position of the attribute,
not the position of the set they were taken from (which we'll assume was
the intent here). this is already the case for most attributes, only the
the extra-special __structuredAttrs and __ignoreNulls weren't annotated.
this includes not calling atPos(v) on errors builders since the position
of v is always invalid. error messages are unchanged by this part of the
patch (and the caller adds a trace pointing to the derivation name too).
Change-Id: Ia3540f9ce1f2505275e49dd4e4336b2a59d5b336
replInitFilesFunction is always valid, but pointing to it is misleading
because the source is inaccessible. every replInit is better identified
by its path if it isn't a lambda; determinePos will return noPos anyway
Change-Id: I0b6fcd2f95f121a802c87cb1085d3a0b1c585aeb
it's not useful. nix-instantiate will always point to the wrong location
because "at the toplevel" is not a position we can point to. all nesting
conversions will point to strange places: the *original* definition of a
lambda value (or partial applications thereof), or nowhere at all (since
only lambda values can have their positions determined in the formatter)
Change-Id: I56bd9e1cf3cae11b4198f148a141ca758e4fe8ac
this is much more useful than a frame pointing to the set but claiming
it as a function being called. if the function is actually at fault we
will now point to its attribute, although the position may be slightly
wrong if __toString was defined from set updates or builtin functions.
Change-Id: Ib3eb237a276d94426d9c6fc0e26eea72382d34a2
by the time enumerateOutputs is called we've already established that
vFlake will evaluate to an attrset (as a locked flake it can't be any
other kind of value). the outputs attribute is likewise already a set
because the nix code that builds it says so. adding positions here is
merely an unnecessary debugging aid for developers working on flakes.
Change-Id: I2a10ddf1400f7650c6e303433f9fee8813f0188d
only attrsets, lambdas, and app nodes can have positions determined, and
none of these three types are incomplete. calling determinePos before we
force a value is thus only a convoluted copy operation for its argument.
Change-Id: I7e647aacca1fc8250f7bb11bb0071a4f7186047f
otherwise lix may crash when e.g. nix search receives invalid regex.
we now also give better error messages for regex errors during eval.
fixes#803
Change-Id: Icc7c578ff488ba520efac5d898572ccf4486e9a8
we no longer explicitly configure these ssh options, so we now
have *even less* reason to stick to old and awful terminology.
Change-Id: I8f38f5486e6918f69c61ddc2d6882b6bb50554c6
our implementation is mostly unused, completely untested, and simply
breaks when users configure connection sharing independently. we can
safely delete this "feature" and inherit user configuration instead.
also have the remote build test check that connection sharing works.
fixes#304, fixes#644
Change-Id: Iea44cc0f8e51a1d231ad186a95c7e310bbfeb303
This reverts commit bba678e5c5.
Reason for revert: didn't fix the bug and created new ones (fj#794)
Change-Id: I0450205d3041b6c876737151a4051081c1366f1d
448c7d50e1 was a bit over-eager and didn't
make the `ref(enable_shared_from_this &)` constructor explicit. this has
confused hydra maintainers, and is just generally bad practice since any
reference is allowed to implicitly convert, even if those references are
not associated with an active control block. we can't avoid this problem
entirely, but we can make what happens more explicit by ... well, making
the involved constructor explicit. enable_shared_from_this is statically
unsafe in principle and we really have to get rid of this nonsense soon.
Change-Id: I8b48ef4353e6301b61af3569083f42bc7379b0a4
this gives us a semblance of a stack trace if async tasks fail, like:
Async task trace (probably incomplete):
#0: virtual kj::Promise<Result<Goal::WorkResult>> nix::LocalDerivationGoal::tryLocalBuild() (lix/libstore/build/local-derivation-goal.cc:257:33)
building of '//0ppb5aj4p3vp0icnkq4yibajw1czvdja-bash52-009.drv^out' from .drv file
#1: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::outputsSubstitutionTried() (lix/libstore/build/derivation-goal.cc:381:47)
building of '//0ppb5aj4p3vp0icnkq4yibajw1czvdja-bash52-009.drv^out' from .drv file
#2: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::loadDerivation() (lix/libstore/build/derivation-goal.cc:213:41)
building of '//0ppb5aj4p3vp0icnkq4yibajw1czvdja-bash52-009.drv^out' from .drv file
#3: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::gaveUpOnSubstitution() (lix/libstore/build/derivation-goal.cc:453:62)
building of '//f8rkb1vam9y9vi9scsm1v213b01rm72h-bash-5.2p37.drv^out' from .drv file
#4: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::outputsSubstitutionTried() (lix/libstore/build/derivation-goal.cc:381:47)
building of '//f8rkb1vam9y9vi9scsm1v213b01rm72h-bash-5.2p37.drv^out' from .drv file
#5: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::loadDerivation() (lix/libstore/build/derivation-goal.cc:213:41)
building of '//f8rkb1vam9y9vi9scsm1v213b01rm72h-bash-5.2p37.drv^out' from .drv file
#6: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::gaveUpOnSubstitution() (lix/libstore/build/derivation-goal.cc:453:62)
building of '//qa9jvc4j6fphzwr63k5c829b2hr04q6z-foo.drv^*' from .drv file
#7: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::outputsSubstitutionTried() (lix/libstore/build/derivation-goal.cc:381:47)
building of '//qa9jvc4j6fphzwr63k5c829b2hr04q6z-foo.drv^*' from .drv file
#8: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::loadDerivation() (lix/libstore/build/derivation-goal.cc:213:41)
building of '//qa9jvc4j6fphzwr63k5c829b2hr04q6z-foo.drv^*' from .drv file
#9: virtual kj::Promise<Result<void>> nix::Store::buildPaths(const std::vector<DerivedPath> &, BuildMode, std::shared_ptr<Store>) (lix/libstore/build/entry-points.cc:20:11)
it's not much, but it's a lot better than nothing. sadly we're forced
to disable another clang-tidy warning now, but since that one's about
assert side effects being compiled out in release mode and we've long
since decided to just not do that we are not affected by this at all.
also see #724 for a related but different approach to origin tracing.
Change-Id: I8161f82ca39d0b271316ea4df80513acc1b06a03
we generally do not want to catch or throw these. catching them to print
and discard is fine, tests are largely exempt, and cases in which we can
be certain where the exception came from are also fine to *catch*. we'll
try to never *throw* (or rethrow) these if possible though because doing
so will make it impossible to construct async traces for the exceptions.
Change-Id: I3b71c32ecd16afc2246c946472f5629a1fa31f2c
this wraps a non-lix exception so we'll be able to add async traces. to
not interfere with existing infrastructure we also add a BaseException,
deriving from BaseError would have foreign exceptions match on the very
broad catches we have in libmain even though we want these logged only.
Change-Id: I5545788c299e9bbdb4d730458914bfd816870794
we'll also assert instead of throwing an exception because that just
seems more useful. this should never *ever* happen, and it is on the
callers of such conversions to ensure that the conversion *is* safe.
Change-Id: Ib0696af4f037046f2d45bf5b1b255393ea9b5f05
don't throw if a downcast fails, have that particular casting method
return an optional instead and check the result at the call site. we
also rename dynamic_pointer_cast to try_cast_shared for consistency.
Change-Id: I65ee0fb498b1b2b8b418935fce4358a654f984e5
we don't need to unsafely cast a shared_from_this pointer to create a
ref (the pointer must be non-null to not invoke nasal demons anyway).
Change-Id: I133fe5f07b2cff8ec2c925f0528f4c5288827261
don't store them as shared_ptrs and cast them back to ref when needed.
we have sufficiently few instances of this pattern that we can spare a
few bytes to make the code less implicitly unsafe (and a bit clearer).
Change-Id: Ied573ab2e116ae228730390ff4c6b0469c59ff6a
this affects both :edit in the repl and the `nix edit` command. given a
huge line number they'd previously crash with a stack trace because the
conversion to int would throw out_of_range instead of invalid_argument.
Change-Id: I4ea1a6e65a83373d54aae72839a9b017df076006
or more accurately, wrap them in a nix::Error subclass so we can display
them properly without crashing, and add some error context if available.
fixes#642fixes#753fixes#759fixes#769
Change-Id: I1aad0c0501fea83f9de3a1335eaa6adc20721616
technically it doesn't *have* to be NeverAsync, but not marking it as
such unconditionally requires templating DebugState over asyncness of
its callback (which then requires templating EvalState, which, *NO*.)
Change-Id: I4980d45b541c2e40328beac139b18c6c1ba0957c
throwing debuggable errors is effectively never-async, and realising a
context requires acquiring a context first, i.e. evaluating something.
since realiseContext is only used by primops the effect is very minor.
Change-Id: I73b3b277188700e5cdf6f30599fb6674ec1ab753
this only affects undefined names outside of with scopes. sending those
errors to the debugger when enabled seems rather less useful; unlike in
the case of with scopes there's nothing meaningful to inspect. avoiding
the debugger also lets us avoid marking the entire parser `NeverAsync`.
cf #761
Change-Id: I3599b826ff5b101acf9a3ba70dcdf9e8e02067f0
allow opting in to serialization as integers via a trait type instead,
and add string-list serializers for the feature flag set enumerations.
fixes#738
Change-Id: I2746eb5ef1f15c01b4e681f9ba1615b6c6e64f44
we want to own this specialization fully so we can change the default
serializer behavior without also forcing downstream users of our code
to use the same behavior. it'll also let us do things we cannot do in
regular nlohmann::json, such as selectively enabling serialization of
enums as integral types, or using `to_json`/`from_json` overloads for
not-default-constructible types instead of serializer specializations
Change-Id: I91a1db362e37d654090f1824b1cd3ce783d32134
this will become a proper specialization of `nlohmann::basic_json` soon.
specialing basic_json will let us get rid of our `adl_serializer` hacks,
and it'll open the door to better enum serializing behavior without also
forcing all those who use lix as a library to set certain defines (which
may not even be possible depending on how those users use json already).
Change-Id: I5228d2b9df581a189552c993363207cfbd20f445
this doesn't do much, just wrap a few nlohmann headers in headers of our
own (and delete includes we don't need because they're transitively seen
by other includes). doing this now will make the next change much nicer.
Change-Id: I166933102ea86bb5322ebbf9ba9411f96032a53b
if a copyNAR generator was not drained to completion it would not read
the full nar data from its source. this could happen if the copier was
passed to parseAndDump wrapped as a source because copyNAR would yield
nar metadata *before* it had read it, and GeneratorSource will drain a
generator fully *only* if the source is allowed to throw EndOfFile. in
the parseAndDump case this never happened because parseAndDump expects
to be given an unterminated stream, and thus the combination left some
nar metadata in the input Source, breaking the remote store protocols.
fixes#732
Change-Id: Ia59a53375992bfcdb7bc6b37764ca779622bc8f7
this would've caught the missing aio await. also disable some rather
annoying deprecation and missing initializer warnings in dev builds.
Change-Id: Ibbb9dfc67eada0e7843081b669b7c8726a02ec29
spawning threads is not free. reusing threads is also not free, but a
lot cheaper than spawning now ones. the difference can add up quite a
bit on nixos system evaluation; we've seen 20k threads cycled through
on even modestly sized configurations, and at 20µs per thread just to
launch and tear down (excluding any work done) that's potentially not
insignificant. in our test config this was a 3% eval time difference.
Change-Id: I40f901813e700fb99f5def82da54ca969711961b
if two derivations have different outputs of the same other derivation
as inputs there's a race window in which wanted output tracking broke:
- start building depender-a
- schedule dependency^a for depender-a
- substitute dependency^a
- dependency runs path validity checks, yields
- start building depender-b
- depender-b adds b to the wanted set of dependency
- dependency resumes from validity checks with allValid
- depender-b is unblocked, but dependency^b is not in the store
Change-Id: Id4dba517bbbdbae96bf430d2c68f63eccd3b99dd
well, oops. on slow io (as can happen with ssh remote builders) we could
have extended a nar read buffer past what was actually read, injecting a
span of zeroes into the read buffer where we requested some data but got
a partial result instead. also add some tests that would've caught this.
Change-Id: I67aa06b4715aeec6a5bdacaafa9b79849e664e2f
eldritch horrors