Commit Graph
17770 Commits
Author SHA1 Message Date
jadeandLix Systems Gerrit 9c2dba4ee2 Merge "Revert "libstore: don't use curl decompression support"" into main 2025-04-09 16:01:37 +00:00
eldritch horrorsandLix Systems Gerrit 350bf7e42d Revert "libstore: don't use curl decompression support"
This reverts commit bba678e5c5.

Reason for revert: didn't fix the bug and created new ones (fj#794)

Change-Id: I0450205d3041b6c876737151a4051081c1366f1d
2025-04-09 11:28:50 +00:00
Commentator2.0 fb0ef6ca6b Made parser warnings use Error info
no other warn calls were found, which have position information

Change-Id: If9773b86a756e5b2e4096d3f6b3a5114bfed5bee
2025-04-08 10:21:53 +02:00
eldritch horrors 2ef4b69760 libutil: disallow implicit ref creations
448c7d50e1 was a bit over-eager and didn't
make the `ref(enable_shared_from_this &)` constructor explicit. this has
confused hydra maintainers, and is just generally bad practice since any
reference is allowed to implicitly convert, even if those references are
not associated with an active control block. we can't avoid this problem
entirely, but we can make what happens more explicit by ... well, making
the involved constructor explicit. enable_shared_from_this is statically
unsafe in principle and we really have to get rid of this nonsense soon.

Change-Id: I8b48ef4353e6301b61af3569083f42bc7379b0a4
2025-04-06 20:29:25 +02:00
eldritch horrors b4c6bfd72a libutil: trace unwinding through async tasks
this gives us a semblance of a stack trace if async tasks fail, like:

    Async task trace (probably incomplete):
    #0: virtual kj::Promise<Result<Goal::WorkResult>> nix::LocalDerivationGoal::tryLocalBuild() (lix/libstore/build/local-derivation-goal.cc:257:33)
    	building of '//0ppb5aj4p3vp0icnkq4yibajw1czvdja-bash52-009.drv^out' from .drv file
    #1: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::outputsSubstitutionTried() (lix/libstore/build/derivation-goal.cc:381:47)
    	building of '//0ppb5aj4p3vp0icnkq4yibajw1czvdja-bash52-009.drv^out' from .drv file
    #2: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::loadDerivation() (lix/libstore/build/derivation-goal.cc:213:41)
    	building of '//0ppb5aj4p3vp0icnkq4yibajw1czvdja-bash52-009.drv^out' from .drv file
    #3: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::gaveUpOnSubstitution() (lix/libstore/build/derivation-goal.cc:453:62)
    	building of '//f8rkb1vam9y9vi9scsm1v213b01rm72h-bash-5.2p37.drv^out' from .drv file
    #4: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::outputsSubstitutionTried() (lix/libstore/build/derivation-goal.cc:381:47)
    	building of '//f8rkb1vam9y9vi9scsm1v213b01rm72h-bash-5.2p37.drv^out' from .drv file
    #5: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::loadDerivation() (lix/libstore/build/derivation-goal.cc:213:41)
    	building of '//f8rkb1vam9y9vi9scsm1v213b01rm72h-bash-5.2p37.drv^out' from .drv file
    #6: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::gaveUpOnSubstitution() (lix/libstore/build/derivation-goal.cc:453:62)
    	building of '//qa9jvc4j6fphzwr63k5c829b2hr04q6z-foo.drv^*' from .drv file
    #7: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::outputsSubstitutionTried() (lix/libstore/build/derivation-goal.cc:381:47)
    	building of '//qa9jvc4j6fphzwr63k5c829b2hr04q6z-foo.drv^*' from .drv file
    #8: kj::Promise<Result<Goal::WorkResult>> nix::DerivationGoal::loadDerivation() (lix/libstore/build/derivation-goal.cc:213:41)
    	building of '//qa9jvc4j6fphzwr63k5c829b2hr04q6z-foo.drv^*' from .drv file
    #9: virtual kj::Promise<Result<void>> nix::Store::buildPaths(const std::vector<DerivedPath> &, BuildMode, std::shared_ptr<Store>) (lix/libstore/build/entry-points.cc:20:11)

it's not much, but it's a lot better than nothing. sadly we're forced
to disable another clang-tidy warning now, but since that one's about
assert side effects being compiled out in release mode and we've long
since decided to just not do that we are not affected by this at all.

also see #724 for a related but different approach to origin tracing.

Change-Id: I8161f82ca39d0b271316ea4df80513acc1b06a03
2025-04-06 12:14:51 +02:00
eldritch horrors 8d8bb60796 treewide: lint against non-lix exceptions
we generally do not want to catch or throw these. catching them to print
and discard is fine, tests are largely exempt, and cases in which we can
be certain where the exception came from are also fine to *catch*. we'll
try to never *throw* (or rethrow) these if possible though because doing
so will make it impossible to construct async traces for the exceptions.

Change-Id: I3b71c32ecd16afc2246c946472f5629a1fa31f2c
2025-04-06 12:14:38 +02:00
eldritch horrors 63d550938b treewide: derive all lix exceptions from BaseError
even the non-errors. we should probably insert a BaseException here.

Change-Id: I1b1af8ba0bf49251fe9d1a24c6559db3ef4a59d2
2025-04-06 11:52:32 +02:00
eldritch horrors 8506a2aa00 libutil: add ForeignException
this wraps a non-lix exception so we'll be able to add async traces. to
not interfere with existing infrastructure we also add a BaseException,
deriving from BaseError would have foreign exceptions match on the very
broad catches we have in libmain even though we want these logged only.

Change-Id: I5545788c299e9bbdb4d730458914bfd816870794
2025-04-06 11:52:30 +02:00
FireFlyandLix Systems Gerrit f24eeae6e6 Merge "libstore: move away from deprecated curl calls" into main 2025-04-06 09:14:08 +00:00
FireFly 14252cb4a8 libstore: move away from deprecated curl calls
CURLINFO_PROTOCOL [1] is deprecated in favour of CURLINFO_SCHEME, which
unfortunately requires a case-insensitive string check to tell the URL
scheme used.  While we're here, also cache the status code returned from
`getHTTPStatus`.

CURLOPT_PROGRESSFUNCTION [2] is deprecated in favour of
CURLOPT_XFERINFOFUNCTION, which simply uses more appropriate types for
the parameters of the callback.

Another step toward fg#744

[1]: https://curl.se/libcurl/c/CURLINFO_PROTOCOL.html
[2]: https://curl.se/libcurl/c/CURLOPT_PROGRESSFUNCTION.html

Change-Id: Icd1feacad45ab5df71d95f4ef370df41ee1393ea
2025-04-06 00:13:20 +02:00
eldritch horrors 949a561540 libutil: explicitly mark ref(shared_ptr) as unsafe
we'll also assert instead of throwing an exception because that just
seems more useful. this should never *ever* happen, and it is on the
callers of such conversions to ensure that the conversion *is* safe.

Change-Id: Ib0696af4f037046f2d45bf5b1b255393ea9b5f05
2025-04-05 21:57:45 +02:00
eldritch horrors dd8dbb56fc libutil: make ref casts safer
don't throw if a downcast fails, have that particular casting method
return an optional instead and check the result at the call site. we
also rename dynamic_pointer_cast to try_cast_shared for consistency.

Change-Id: I65ee0fb498b1b2b8b418935fce4358a654f984e5
2025-04-05 21:57:45 +02:00
eldritch horrors 448c7d50e1 libutil: add shared_from_this support to ref
we don't need to unsafely cast a shared_from_this pointer to create a
ref (the pointer must be non-null to not invoke nasal demons anyway).

Change-Id: I133fe5f07b2cff8ec2c925f0528f4c5288827261
2025-04-05 21:57:45 +02:00
eldritch horrors aa358eac91 libutil: remove unused ref constructor
this one is dangerous anyway.

Change-Id: Id4fb777fff516cdb6a6e8e0b2fd7611418ae2e49
2025-04-05 21:57:45 +02:00
eldritch horrors c4b8b92904 treewide: store optional refs as exactly this
don't store them as shared_ptrs and cast them back to ref when needed.
we have sufficiently few instances of this pattern that we can spare a
few bytes to make the code less implicitly unsafe (and a bit clearer).

Change-Id: Ied573ab2e116ae228730390ff4c6b0469c59ff6a
2025-04-05 21:57:45 +02:00
eldritch horrors 14db530ac0 libexpr: restrict EvalErrorBuilder to building EvalErrors
Change-Id: Iadd0fee4c12c5d0b6ee74dccbf8785c829d20d95
2025-04-05 21:57:45 +02:00
eldritch horrors 15a85a936b libexpr: fix :edit crashing on huge line numbers
this affects both :edit in the repl and the `nix edit` command. given a
huge line number they'd previously crash with a stack trace because the
conversion to int would throw out_of_range instead of invalid_argument.

Change-Id: I4ea1a6e65a83373d54aae72839a9b017df076006
2025-04-05 21:57:45 +02:00
jadeandLix Systems Gerrit 7bf3d0fa2a Merge changes Iaa962f04,I19480ab0 into main
* changes:
  MonitorFdHup: introduce a test
  tests: fix the fmt.cc test
2025-04-05 17:12:48 +00:00
FireFly c292fbc46d tests: actually test printing of paths in unit test
This seems to be a copypasta mistake that slipped through when writing the
initial tests.  Changes the ValuePrintingTests::tPath to actually test a path
value instead of a string.

Change-Id: I20fe72635a0b1a6a0d72f674a4619da0de9f3d44
2025-04-05 17:11:17 +02:00
Jade Lovelace 7aec313597 MonitorFdHup: introduce a test
Well, I was trying to figure out
https://git.lix.systems/lix-project/lix/issues/729 in which this feature
is clearly just broken on macOS, but frustratingly, it seems that it
*does* work, except for the daemon. um........ sure.

Change-Id: Iaa962f045c16fdfa82854151c90a03e5cf0eea47
2025-04-04 16:59:23 -07:00
FireFly b7474fc4a9 libutil: use openssl EVP for hashing, address deprecated use
Replace the old-style per-hash-family deprecated functions with using
the newer EVP functions for computing hashes.  While we're here, also
make sure to raise errors if any of the openssl function calls fail.

A small step in the direction of fg#744.

Change-Id: I8c2100d57bc50c8f49c01a131c96c7163ba608ba
2025-04-04 19:06:33 +02:00
Jade Lovelace 6d8e4337e2 tests: fix the fmt.cc test
Yikes!! I wonder if we have any other ones of these that just .. didn't
get added to a meson file?

Change-Id: I19480ab03cdbecf608e523d5b6c3980233f4f445
2025-04-03 18:40:06 -07:00
jadeandLix Systems Gerrit 5a7e9e1746 Merge "MonitorFdHup: shorten loop delay" into main 2025-04-01 16:10:03 +00:00
Maximilian BoschandLix Systems Gerrit d169c092fc Merge "fetchers: don't consider a path locked if a rev is specified" into main 2025-03-29 18:51:45 +00:00
Raito Bezarius a6adaab2bc libmain: undo accidental lix command stabilization
When external subcommand support landed, the `lix` binary gave
accidental access to nix3 CLI as well.

This was not the intended effect of this feature, we want to keep `lix`
as a namespace for a new redesign of the CLI (that has not started yet),
while giving that namespace for external subcommands.

To this end, we sever the nix3 CLI connection for the `lix` binary.

Fixes fj#508.

Change-Id: I13a0748bcbf87343c8e0e89439bd4af3a0ac3118
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-03-28 21:08:52 +01:00
Linus Heckemann b52db5e7c1 gc: fix paths-not-deleted error message
The newline was previously omitted when fewer than summaryThreshold paths were present.

Change-Id: Ieb9292262f9b5a666784a4273de534941bc31998
2025-03-28 09:46:22 +01:00
eldritch horrors b918f1c307 tidy: add a lint to forbid nlohmann::json::parse
Change-Id: I165c7330d7ea7f17522ade8a0666d0d7d86b9d38
2025-03-27 08:56:14 +00:00
eldritch horrors 0d47773d76 treewide: handle JSON parse errors
or more accurately, wrap them in a nix::Error subclass so we can display
them properly without crashing, and add some error context if available.

fixes #642
fixes #753
fixes #759
fixes #769

Change-Id: I1aad0c0501fea83f9de3a1335eaa6adc20721616
2025-03-27 08:56:14 +00:00
jadeandLix Systems Gerrit 80348f7411 Merge "docs: document unsafeGetAttrPos" into main 2025-03-27 00:51:29 +00:00
Sizhe Zhao ad09164d0c docs: document unsafeGetAttrPos
Closes https://git.lix.systems/lix-project/lix/issues/770.

See also https://github.com/NixOS/nix/pull/12760.

Change-Id: Ibe7a9a41a84f749e3e9bedce8da28039fa6e6d2f
2025-03-27 00:58:32 +08:00
jadeandLix Systems Gerrit f270d3ebe0 Merge changes Ib81a5db1,Ib6d68594,Id91e1fd4 into main
* changes:
  docs: document the cursed file syntax for new CLI
  docs: document the cursed file syntax for old cli
  doc preprocessor: support indent directives
2025-03-26 16:05:26 +00:00
Stanislav AlekseevandLix Systems Gerrit 21dac7513c Merge "Revert "fix: allow access to ca-certs in FODs on darwin"" into main 2025-03-26 13:00:59 +00:00
Stanislav AlekseevandLix Systems Gerrit 5a4da16094 Revert "fix: allow access to ca-certs in FODs on darwin"
This reverts commit 692de16113.

Reason for revert: it's wrong at it doesn't actually work

Change-Id: I495eed65bb7547ceffe98f979633c080e83ac154
2025-03-26 10:27:58 +00:00
QyriadandJade Lovelace f5c13758bc docs: document the cursed file syntax for new CLI
This documents that all installables in the nix3 commands, in their
--file/-f form accept an extended syntax for the file argument,
which is the same as it is for nix-build and friends, some of which were
well known in their nix-build forms (e.g. `nix-build '<nixpkgs>' -A hello)
but are not well known in their nix3 forms (people rarely know that you
can `nix build -f '<nixpkgs>' firefox`).

Like the old CLI syntax, as documented in [1], file arguments also
accept https:// tarball URLs, `flake:` prefixed flakerefs, and the
cursed `channel:` prefixed hardened URLs.

[1]: Ib6d68594a16132805ba5d97526e16f7b3633117e

Change-Id: Ib81a5db1f60d5916f0f792d82054f3ac65717121
2025-03-25 17:06:53 -07:00
QyriadandJade Lovelace 11225f435d docs: document the cursed file syntax for old cli
This documents the fact that nix-build, nix-env, nix-instantiate, and
nix-shell accept an extended syntax for their file arguments, including
some well-known (but not well documented) aspects, like being able to
specify `<nixpkgs>`, but also https:// tarball URLs, `flake:` prefixed
flakerefs, and the cursed `channel:` prefixed hardcoded URLs

Same thing for the new CLI incoming :)

Change-Id: Ib6d68594a16132805ba5d97526e16f7b3633117e
2025-03-25 17:06:53 -07:00
Jade Lovelace 989498407b doc preprocessor: support indent directives
Janky Python makes the world go around. In repentance I removed some
jank from it.

Change-Id: Id91e1fd40cb73049cb64deab6f81a7193132fa82
2025-03-25 17:06:53 -07:00
eldritch horrors fc58e7b217 libexpr: mark eval error handling NeverAsync
technically it doesn't *have* to be NeverAsync, but not marking it as
such unconditionally requires templating DebugState over asyncness of
its callback (which then requires templating EvalState, which, *NO*.)

Change-Id: I4980d45b541c2e40328beac139b18c6c1ba0957c
2025-03-25 21:26:47 +01:00
eldritch horrors 8a90195cdb libexpr: don't debugThrow from async code
running a debugger is never-async since it holds an EvalState reference.

fixes #761

Change-Id: Ie1b4df6f81bb5614f11ea183eb9d1058fde73121
2025-03-25 21:23:05 +01:00
eldritch horrors d2f4b4042d libexpr: move realiseContext back to EvalState
throwing debuggable errors is effectively never-async, and realising a
context requires acquiring a context first, i.e. evaluating something.
since realiseContext is only used by primops the effect is very minor.

Change-Id: I73b3b277188700e5cdf6f30599fb6674ec1ab753
2025-03-25 20:14:45 +01:00
eldritch horrors 0ff8c2b06f libexpr: make undefined var errors non-debuggable in the parser
this only affects undefined names outside of with scopes. sending those
errors to the debugger when enabled seems rather less useful; unlike in
the case of with scopes there's nothing meaningful to inspect. avoiding
the debugger also lets us avoid marking the entire parser `NeverAsync`.

cf #761

Change-Id: I3599b826ff5b101acf9a3ba70dcdf9e8e02067f0
2025-03-25 20:14:45 +01:00
eldritch horrors 6bf8c8a4f2 nix-eval-jobs: fix a clang-tidy lint
we don't run clang-tidy on nej in ci, but it's annoying locally.

Change-Id: I6f0ae3220425ac9e9f20adaf492873f8b27a630f
2025-03-25 20:14:45 +01:00
eldritch horrors c3929c78f3 libutil: disallow enum serialization by default
allow opting in to serialization as integers via a trait type instead,
and add string-list serializers for the feature flag set enumerations.

fixes #738

Change-Id: I2746eb5ef1f15c01b4e681f9ba1615b6c6e64f44
2025-03-25 10:44:52 +01:00
eldritch horrors e5b9c86e59 libutil: remove *-json.hh headers
we can inline them into their parents now.

fixes #598

Change-Id: I7bc9371885cc0adde11da1e2c67bf0a4cf833a98
2025-03-25 10:44:50 +01:00
jadeandLix Systems Gerrit 4181796ed2 Merge changes I0f5d9912,I442f16c7 into main
* changes:
  observability: add the first USDT probe
  tree-wide: put in more keep-sorted
2025-03-24 16:37:28 +00:00
Stanislav AlekseevandLix Systems Gerrit f0fd789f68 Merge "fix: allow access to ca-certs in FODs on darwin" into main 2025-03-24 07:32:12 +00:00
WeetHetandJade Lovelace 692de16113 fix: allow access to ca-certs in FODs on darwin
Change-Id: Id955be88eb5d419d57262d5983841a1e6accee9f
2025-03-23 19:13:44 -07:00
Jade Lovelace 12bf0b6be6 MonitorFdHup: shorten loop delay
Apparently this is sometimes a problem for tests containing race
conditions, since it caused the daemon processes to stick around for a
second. Doesn't make writing such tests any less racey and foolish, but
we can stop doing the silly thing regardless.

CC: https://github.com/NixOS/nix/pull/12714#discussion_r2009265904
Change-Id: Iad6e55cf78c4a4517082194fa00a30d921224457
2025-03-23 19:00:23 -07:00
Jade Lovelace cad275307e observability: add the first USDT probe
USDT probes are statically defined trace points that have nearly zero
disabled-probe effect, i.e. we can put them in hot paths.

The use case for these is both similar and dissimilar to Rust tracing:
We still need better logging and a better structured rust-tracing
looking thing, but probes allow for quite easy programmable interactive
tracing in production, which we also care a lot about.

This CL comes with a perfunctory trace point in
libstore/file-transfer.cc for reading data out of the curl buffer. This
was mostly thrown in there so that I could see what the buffer sizes of
this were, and maybe be able to instrument the perf of the curl usages
in Lix in the future.

Fixes: https://git.lix.systems/lix-project/lix/issues/727
Change-Id: I0f5d9912d76bf3d6923bf53ebfd9b8d6c6e70aea
2025-03-23 18:37:47 -07:00
Jade Lovelace b037258836 tree-wide: put in more keep-sorted
Change-Id: I442f16c7116f603de74004fbeccf23ae32b38e69
2025-03-23 18:22:46 -07:00
eldritch horrors 56df5ba164 libutil: remove json-utils.hh
we can merge it into json.hh instead.

Change-Id: Ic40c25fa759bf52bb69eae5d7c0597f260fd94a6
2025-03-23 22:29:58 +00:00