string data shares a buffer with the binary string length field. size
calculations for string read buffers always include the length field;
sufficiently large length fields can cause these calculations to wrap.
a malicious nar could use this for OOB writes in the daemon (as root).
since we use strings only as tags for archive members and for symlinks
with their OS-dependent length limits we can simply limit string size.
1 MiB should be sufficient for all symlinks, and tags are always tiny.
Change-Id: I89fb05f73c1dbeda45d91244aba4cd526a3d83e1
libarchive *should* not break with 0710 on the tmpdir root on darwin,
just like it doesn't break on linux, but for some reason it does. the
restriction to 0710 can be weakened to 0750 with causing any trouble.
fixes#921
Change-Id: Ia9fc2f8eb9695fc19cefae9857368d5a4e58c8b9
although we only chown if the build was requested by a local daemon
user. daemonless invocations will not chown as they do not have to.
remote builds *can* chown to the remote builder user, but that does
not seem to happen (for some reason keep-failed is not propagated).
Change-Id: Ic0ead406b38b4ca0556fec42d84888efa25123bf
(cherry picked from commit ae3b8e58c3)
this makes the actual build directories used by builders invisible and
inaccessible to other processes on the system, avoiding another vector
for outside processes to interfere with builds or pass credentials the
build sandbox should not have access to into the build sandbox anyway.
fixes#919
Change-Id: Ifaa4d8e3940cfde1406e925f75c1375d2e86d81a
(cherry picked from commit 9d5a5c4dc0)
if a build directory is accessible to other users it is possible to
smuggle data in and out of build directories. usually this ins only
a build purity problem, but in combination with other issues it can
be used to break out of a build sandbox. to prevent this we default
to using a subdirectory of nixStateDir (which is more restrictive).
Fixes CVE-2025-52991.
Change-Id: Iacfc9b50534de158618c815f9fb99d7dae1be4d0
This allows using a userspace program, pasta, to handle comms between
the build sandbox, and the outside world; allowing for full isolation
including the network namespace, closing the "fixed-output derivation
talks to the host over an abstract domain socket" hole for good.
Fixes CVE-2025-46416.
Co-Authored-By: Puck Meerburg <puck@puckipedia.com>
Change-Id: Ifd499b7dbb3784600a6e842fede65fc031ff9f15
it's an eval-time only setting, the daemon doesn't use it anywhere. this
is a hack, but until we have a much better settings system we are stuck.
fixes#680
Change-Id: I532088b0279f13da0a0a65c2bd2e5f9d1dfb39da
(cherry picked from commit 5917db84aa)
ca derivations are what we're really after, but dynamic derivations
must also go because they depend on ca derivations. we can't easily
implement dynamic derivations any other way, so we remove them too.
impure derivations build on the content-addressed infrastructure in
ways we cannot easily detangle, so they too must go for time being.
see #815
Change-Id: If61371736dfd89cc71a1b2ae5a005757c3cb9484
(cherry picked from commit d8e2f53d07)
this can be extremely distracting when running in a repl that has access
to all of nixpkgs from its static env. not only do we print all bindings
in the stack trace, we also print them *again* every time we enter *any*
but the outermost frames. duplicating this environment information isn't
all that useful and mostly just flushes useful information out of sight.
fixes#816
Change-Id: I027e6a565e82663275e21333bb1d00d498cd4127
this way we don't have to even check whether we need a debug frame when
the debugger isn't enabled. not doing this gives us an eval performance
improvement of roughly 7% on nixos system eval and 2% for `nix search`.
Change-Id: I1cdad3de61f865ea54d6e09d63a281688e828768
we'll want to wrap some exprs for debug purposes, and dynamic casts
cannot look through such wrappers. dedicated casting functions can.
Change-Id: I1fba0ec52d281a1b8de85a62e4948bfae536bcfc
add a position parameter to the autocaller instead, and pass it much
more accurate position information where we have it easily available
Change-Id: If2f1c3006ca3f2b413492842905d079a8b752542
errors for individual attrs should report the position of the attribute,
not the position of the set they were taken from (which we'll assume was
the intent here). this is already the case for most attributes, only the
the extra-special __structuredAttrs and __ignoreNulls weren't annotated.
this includes not calling atPos(v) on errors builders since the position
of v is always invalid. error messages are unchanged by this part of the
patch (and the caller adds a trace pointing to the derivation name too).
Change-Id: Ia3540f9ce1f2505275e49dd4e4336b2a59d5b336
replInitFilesFunction is always valid, but pointing to it is misleading
because the source is inaccessible. every replInit is better identified
by its path if it isn't a lambda; determinePos will return noPos anyway
Change-Id: I0b6fcd2f95f121a802c87cb1085d3a0b1c585aeb
it's not useful. nix-instantiate will always point to the wrong location
because "at the toplevel" is not a position we can point to. all nesting
conversions will point to strange places: the *original* definition of a
lambda value (or partial applications thereof), or nowhere at all (since
only lambda values can have their positions determined in the formatter)
Change-Id: I56bd9e1cf3cae11b4198f148a141ca758e4fe8ac
this is much more useful than a frame pointing to the set but claiming
it as a function being called. if the function is actually at fault we
will now point to its attribute, although the position may be slightly
wrong if __toString was defined from set updates or builtin functions.
Change-Id: Ib3eb237a276d94426d9c6fc0e26eea72382d34a2
by the time enumerateOutputs is called we've already established that
vFlake will evaluate to an attrset (as a locked flake it can't be any
other kind of value). the outputs attribute is likewise already a set
because the nix code that builds it says so. adding positions here is
merely an unnecessary debugging aid for developers working on flakes.
Change-Id: I2a10ddf1400f7650c6e303433f9fee8813f0188d
only attrsets, lambdas, and app nodes can have positions determined, and
none of these three types are incomplete. calling determinePos before we
force a value is thus only a convoluted copy operation for its argument.
Change-Id: I7e647aacca1fc8250f7bb11bb0071a4f7186047f
otherwise lix may crash when e.g. nix search receives invalid regex.
we now also give better error messages for regex errors during eval.
fixes#803
Change-Id: Icc7c578ff488ba520efac5d898572ccf4486e9a8
we no longer explicitly configure these ssh options, so we now
have *even less* reason to stick to old and awful terminology.
Change-Id: I8f38f5486e6918f69c61ddc2d6882b6bb50554c6
our implementation is mostly unused, completely untested, and simply
breaks when users configure connection sharing independently. we can
safely delete this "feature" and inherit user configuration instead.
also have the remote build test check that connection sharing works.
fixes#304, fixes#644
Change-Id: Iea44cc0f8e51a1d231ad186a95c7e310bbfeb303
This reverts commit bba678e5c5.
Reason for revert: didn't fix the bug and created new ones (fj#794)
Change-Id: I0450205d3041b6c876737151a4051081c1366f1d
448c7d50e1 was a bit over-eager and didn't
make the `ref(enable_shared_from_this &)` constructor explicit. this has
confused hydra maintainers, and is just generally bad practice since any
reference is allowed to implicitly convert, even if those references are
not associated with an active control block. we can't avoid this problem
entirely, but we can make what happens more explicit by ... well, making
the involved constructor explicit. enable_shared_from_this is statically
unsafe in principle and we really have to get rid of this nonsense soon.
Change-Id: I8b48ef4353e6301b61af3569083f42bc7379b0a4
this gives us a semblance of a stack trace if async tasks fail, like:
Async task trace (probably incomplete):
#0: virtual kj::Promise<Result<Goal::WorkResult>> nix::LocalDerivationGoal::tryLocalBuild() (lix/libstore/build/local-derivation-goal.cc:257:33)
building of '//0ppb5aj4p3vp0icnkq4yibajw1czvdja-bash52-009.drv^out' from .drv file
#1: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::outputsSubstitutionTried() (lix/libstore/build/derivation-goal.cc:381:47)
building of '//0ppb5aj4p3vp0icnkq4yibajw1czvdja-bash52-009.drv^out' from .drv file
#2: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::loadDerivation() (lix/libstore/build/derivation-goal.cc:213:41)
building of '//0ppb5aj4p3vp0icnkq4yibajw1czvdja-bash52-009.drv^out' from .drv file
#3: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::gaveUpOnSubstitution() (lix/libstore/build/derivation-goal.cc:453:62)
building of '//f8rkb1vam9y9vi9scsm1v213b01rm72h-bash-5.2p37.drv^out' from .drv file
#4: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::outputsSubstitutionTried() (lix/libstore/build/derivation-goal.cc:381:47)
building of '//f8rkb1vam9y9vi9scsm1v213b01rm72h-bash-5.2p37.drv^out' from .drv file
#5: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::loadDerivation() (lix/libstore/build/derivation-goal.cc:213:41)
building of '//f8rkb1vam9y9vi9scsm1v213b01rm72h-bash-5.2p37.drv^out' from .drv file
#6: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::gaveUpOnSubstitution() (lix/libstore/build/derivation-goal.cc:453:62)
building of '//qa9jvc4j6fphzwr63k5c829b2hr04q6z-foo.drv^*' from .drv file
#7: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::outputsSubstitutionTried() (lix/libstore/build/derivation-goal.cc:381:47)
building of '//qa9jvc4j6fphzwr63k5c829b2hr04q6z-foo.drv^*' from .drv file
#8: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::loadDerivation() (lix/libstore/build/derivation-goal.cc:213:41)
building of '//qa9jvc4j6fphzwr63k5c829b2hr04q6z-foo.drv^*' from .drv file
#9: virtual kj::Promise<Result<void>> nix::Store::buildPaths(const std::vector<DerivedPath> &, BuildMode, std::shared_ptr<Store>) (lix/libstore/build/entry-points.cc:20:11)
it's not much, but it's a lot better than nothing. sadly we're forced
to disable another clang-tidy warning now, but since that one's about
assert side effects being compiled out in release mode and we've long
since decided to just not do that we are not affected by this at all.
also see #724 for a related but different approach to origin tracing.
Change-Id: I8161f82ca39d0b271316ea4df80513acc1b06a03
we generally do not want to catch or throw these. catching them to print
and discard is fine, tests are largely exempt, and cases in which we can
be certain where the exception came from are also fine to *catch*. we'll
try to never *throw* (or rethrow) these if possible though because doing
so will make it impossible to construct async traces for the exceptions.
Change-Id: I3b71c32ecd16afc2246c946472f5629a1fa31f2c
this wraps a non-lix exception so we'll be able to add async traces. to
not interfere with existing infrastructure we also add a BaseException,
deriving from BaseError would have foreign exceptions match on the very
broad catches we have in libmain even though we want these logged only.
Change-Id: I5545788c299e9bbdb4d730458914bfd816870794
we'll also assert instead of throwing an exception because that just
seems more useful. this should never *ever* happen, and it is on the
callers of such conversions to ensure that the conversion *is* safe.
Change-Id: Ib0696af4f037046f2d45bf5b1b255393ea9b5f05
don't throw if a downcast fails, have that particular casting method
return an optional instead and check the result at the call site. we
also rename dynamic_pointer_cast to try_cast_shared for consistency.
Change-Id: I65ee0fb498b1b2b8b418935fce4358a654f984e5
we don't need to unsafely cast a shared_from_this pointer to create a
ref (the pointer must be non-null to not invoke nasal demons anyway).
Change-Id: I133fe5f07b2cff8ec2c925f0528f4c5288827261
don't store them as shared_ptrs and cast them back to ref when needed.
we have sufficiently few instances of this pattern that we can spare a
few bytes to make the code less implicitly unsafe (and a bit clearer).
Change-Id: Ied573ab2e116ae228730390ff4c6b0469c59ff6a
this affects both :edit in the repl and the `nix edit` command. given a
huge line number they'd previously crash with a stack trace because the
conversion to int would throw out_of_range instead of invalid_argument.
Change-Id: I4ea1a6e65a83373d54aae72839a9b017df076006
or more accurately, wrap them in a nix::Error subclass so we can display
them properly without crashing, and add some error context if available.
fixes#642fixes#753fixes#759fixes#769
Change-Id: I1aad0c0501fea83f9de3a1335eaa6adc20721616
technically it doesn't *have* to be NeverAsync, but not marking it as
such unconditionally requires templating DebugState over asyncness of
its callback (which then requires templating EvalState, which, *NO*.)
Change-Id: I4980d45b541c2e40328beac139b18c6c1ba0957c
throwing debuggable errors is effectively never-async, and realising a
context requires acquiring a context first, i.e. evaluating something.
since realiseContext is only used by primops the effect is very minor.
Change-Id: I73b3b277188700e5cdf6f30599fb6674ec1ab753
this only affects undefined names outside of with scopes. sending those
errors to the debugger when enabled seems rather less useful; unlike in
the case of with scopes there's nothing meaningful to inspect. avoiding
the debugger also lets us avoid marking the entire parser `NeverAsync`.
cf #761
Change-Id: I3599b826ff5b101acf9a3ba70dcdf9e8e02067f0
allow opting in to serialization as integers via a trait type instead,
and add string-list serializers for the feature flag set enumerations.
fixes#738
Change-Id: I2746eb5ef1f15c01b4e681f9ba1615b6c6e64f44
we want to own this specialization fully so we can change the default
serializer behavior without also forcing downstream users of our code
to use the same behavior. it'll also let us do things we cannot do in
regular nlohmann::json, such as selectively enabling serialization of
enums as integral types, or using `to_json`/`from_json` overloads for
not-default-constructible types instead of serializer specializations
Change-Id: I91a1db362e37d654090f1824b1cd3ce783d32134
this will become a proper specialization of `nlohmann::basic_json` soon.
specialing basic_json will let us get rid of our `adl_serializer` hacks,
and it'll open the door to better enum serializing behavior without also
forcing all those who use lix as a library to set certain defines (which
may not even be possible depending on how those users use json already).
Change-Id: I5228d2b9df581a189552c993363207cfbd20f445
this doesn't do much, just wrap a few nlohmann headers in headers of our
own (and delete includes we don't need because they're transitively seen
by other includes). doing this now will make the next change much nicer.
Change-Id: I166933102ea86bb5322ebbf9ba9411f96032a53b
if a copyNAR generator was not drained to completion it would not read
the full nar data from its source. this could happen if the copier was
passed to parseAndDump wrapped as a source because copyNAR would yield
nar metadata *before* it had read it, and GeneratorSource will drain a
generator fully *only* if the source is allowed to throw EndOfFile. in
the parseAndDump case this never happened because parseAndDump expects
to be given an unterminated stream, and thus the combination left some
nar metadata in the input Source, breaking the remote store protocols.
fixes#732
Change-Id: Ia59a53375992bfcdb7bc6b37764ca779622bc8f7
this would've caught the missing aio await. also disable some rather
annoying deprecation and missing initializer warnings in dev builds.
Change-Id: Ibbb9dfc67eada0e7843081b669b7c8726a02ec29
spawning threads is not free. reusing threads is also not free, but a
lot cheaper than spawning now ones. the difference can add up quite a
bit on nixos system evaluation; we've seen 20k threads cycled through
on even modestly sized configurations, and at 20µs per thread just to
launch and tear down (excluding any work done) that's potentially not
insignificant. in our test config this was a 3% eval time difference.
Change-Id: I40f901813e700fb99f5def82da54ca969711961b
if two derivations have different outputs of the same other derivation
as inputs there's a race window in which wanted output tracking broke:
- start building depender-a
- schedule dependency^a for depender-a
- substitute dependency^a
- dependency runs path validity checks, yields
- start building depender-b
- depender-b adds b to the wanted set of dependency
- dependency resumes from validity checks with allValid
- depender-b is unblocked, but dependency^b is not in the store
Change-Id: Id4dba517bbbdbae96bf430d2c68f63eccd3b99dd
well, oops. on slow io (as can happen with ssh remote builders) we could
have extended a nar read buffer past what was actually read, injecting a
span of zeroes into the read buffer where we requested some data but got
a partial result instead. also add some tests that would've caught this.
Change-Id: I67aa06b4715aeec6a5bdacaafa9b79849e664e2f
there's no real usefulness to this output in build logs. in interactive
development builds -v can be passed to meson to restore the old output.
Change-Id: Iac4c0573fd1cac5fc9044b950ef52ee50448e6fc
this reverts commit 749a323597. the pool
does hurt performance and concurreny, so let's revert now that we can.
Change-Id: I0b154cf04d14fa2cb4d2028f9cc865e463cd2265
connection pools of remote stores can currently block. this is not a
problem when each request runs on a dedicated thread, but with async
code this is no longer true. if a remote store has exhausted all its
available connections on one executor and another job starts *on the
same executor* we'll deadlock if that job makes another remote store
request. unlike with sqlite previously it's not reasonable, not even
necessary, to make the pools unbounded: since we use pools only with
remote stores we can asyncify all of them at once, and since they're
leaves of all call stacks we do not have much code to change either.
Change-Id: I8c457e27893e22c2cfc35933307a5283c986805a
