Commit Graph
195 Commits
Author SHA1 Message Date
Tom Hubrecht e29263b638 libutil/hash: Add default values for Hash::to_string
The sane default is to print in SRI format, and include the hash type

Change-Id: I7efcb5dcfbe56c82dbf1928e2bb35cdd3ff089c6
2026-01-10 18:26:15 +00:00
Tom Hubrecht b482ebbbc8 treewide: Rename Base to HashFormat
Co-authored-by: Yueh-Shun Li <shamrocklee@posteo.net>

`base` is ambiguous, since it's not about the digital bases, but about
the format of hashes. Base16, Base32 and Base64 are all character maps
for binary encoding.

Documentation of the format is also added.

Cherry-pick of:
https://github.com/NixOS/nix/pull/7708/commits/838c70f62116328ce01cb41a01886e4f1b9a727f
https://github.com/NixOS/nix/pull/7708/commits/5043e6cf4ea537dfe599470797c5b310ab0e94b9

Change-Id: Ief5425f3c2056a4cca75838091e4dfa5cca88872
2026-01-06 11:22:06 +00:00
Maximilian Bosch 74f5d66b39 perl: unset NDEBUG after loading Perl headers
Otherwise, loading Perl bindings fails early with[1]

    undefined symbol: Perl_pad_sv at /nix/store/h2jsb5i4yfblr2f3ac2c7zpmlmj7zjym-perl-5.40.0/lib/perl5/5.40.0/XSLoader.pm line 94

Apparently, it's expected behavior by Perl that this symbol only exists
with `DEBUGGING` being set, hence it's used by the headers. However,
`pkgs.perl` from nixpkgs is apparently not built with `-DDEBUGGING`
causing this error.

Now, `NDEBUG` is manually unset after loading the Perl
headers rather than setting `DEBUGGING` causing the error mentioned
above.

I confirmed that this not only fixes the problem described above, but
running the Hydra tests with

    diff --git a/perl/lib/Nix/Store.xs b/perl/lib/Nix/Store.xs
    index dfdd64d28..14788266c 100644
    --- a/perl/lib/Nix/Store.xs
    +++ b/perl/lib/Nix/Store.xs
    @@ -27,6 +27,7 @@ using namespace nix;

     static AsyncIoRoot & aio()
     {
    +assert(false);
	 static thread_local AsyncIoRoot root;
	 return root;
     }

still results in assertion errors.

Finally, added a small install-check that importing `Nix::Store` works
fine.

[1] https://git.lix.systems/lix-project/hydra/issues/69

Change-Id: I58521777eb0f94b766a9813aa4bbd06f9052bd35
2025-12-05 18:36:35 +01:00
Tom Hubrecht 0c70cc6259 libutil: Reintroduce compile assertion that was mistakenly dropped
https://git.lix.systems/lix-project/lix/commit/74513483bc5572d988a059b8e964662d66f1667f
removed this check which is actually load-bearing for Lix (c.f.
https://git.lix.systems/lix-project/lix/commit/96a36833084f66b79ba778e719862b8219557773
).

Fixes #677

Change-Id: I3cc9a16c64c5189f788c510609afcf8bad997741
2025-11-26 01:56:00 +00:00
Alois Wohlschlager 451a14980b libstore: use OpenSSL for Ed25519 signatures
Previously two cryptography libraries were linked into Lix: OpenSSL used for
hashing and (in usual configurations) indirectly via curl for TLS, and Sodium
used only for handling the Ed25519 path info signatures. The latter is
functionally redundant since OpenSSL supports the same use case as well.
Reimplement the Ed25519 handling using OpenSSL and drop Sodium.

Fixes: https://git.lix.systems/lix-project/lix/issues/969

Change-Id: I6a6a696456b9d3ad7fdc2bf9b0759836a6247a38
2025-08-25 17:11:45 +00:00
eldritch horrors fa116c96f7 perl: ensure that stores are destroyed after aio roots
otherwise stores containing async objects will cause crashes during
shutdown. currently there are no such stores, but that will change.

Change-Id: I05d46ba6831c641774edfe6aa99aa7d0de457429
2025-06-17 14:34:05 +02:00
eldritch horrors 976f6de81e libstore: remove realisation query support
only a daemon wire operation and the perl bindings could initiate these
queries at this point. the daemon ops can throw an error instead (as if
the daemon were older) and realistically should never be queries if the
client hasn't evaluated a ca derivation on a given store, and perl code
is best off dying early. nothing known except hydra uses these bdingins
anyway, and we control our hydra so we don't need backward compat code.

Change-Id: Ia7df27aba59a4a4a692ae014f407415f3bea63f2
2025-05-20 17:43:46 +00:00
eldritch horrors dc47f9aa72 libstore: don't return optionals from Derivation::path
output paths are always known now that CA and deferred outputs are gone.

Change-Id: I359d13ffb5141f1e07a5fc55425831af3332c22e
2025-05-20 17:43:46 +00:00
eldritch horrors 84c1df46ea libstore: remove DerivedPathMap
single-level maps suffice now that dynamic derivations are gone.

Change-Id: If29998b104b31255292ab0c789622d7d27040f69
2025-05-12 13:37:54 +02:00
eldritch horrors 949a561540 libutil: explicitly mark ref(shared_ptr) as unsafe
we'll also assert instead of throwing an exception because that just
seems more useful. this should never *ever* happen, and it is on the
callers of such conversions to ensure that the conversion *is* safe.

Change-Id: Ib0696af4f037046f2d45bf5b1b255393ea9b5f05
2025-04-05 21:57:45 +02:00
eldritch horrors a186bc5021 treewide: add json forwarder headers
this doesn't do much, just wrap a few nlohmann headers in headers of our
own (and delete includes we don't need because they're transitively seen
by other includes). doing this now will make the next change much nicer.

Change-Id: I166933102ea86bb5322ebbf9ba9411f96032a53b
2025-03-23 20:42:39 +00:00
eldritch horrors 81d5528646 libstore: asyncify Store::queryPathInfo{,Uncached}
Change-Id: I182c396ec0bfdfc8febd6752774e668b3d97dcc1
2025-03-05 18:49:45 +01:00
eldritch horrors 90caba8489 libstore: asyncify Store::isValidPath
Change-Id: I2b98781c04944282fd1219cbec0804fd1ceb6765
2025-03-05 18:49:45 +01:00
eldritch horrors 03ab0fcb53 libstore: asyncify Store::queryRealisation{,Uncached}
Change-Id: I4c4d75c773493fcd63c131e30a2e4d6ec5d230ac
2025-03-05 18:49:45 +01:00
eldritch horrors 3fe2be1bcf libstore: asyncify Store::queryPathFromHashPart
Change-Id: I7ba33a0a27542350f4b89ce1cfe0afbd39134bce
2025-03-05 18:49:45 +01:00
eldritch horrors 446e22323e libstore: pass around prepared dumps, not paths and filters
this fully decouples the possibly-never-async bits of dumping from the
generation of dump bitstream. having the two separate will allow us to
change store import methods to use async streams, not our sync sources

Change-Id: I9dbd5e30ad3ee380c244b4a3760c11e37db3895f
2025-03-03 03:04:49 +01:00
eldritch horrors f12cb77442 libstore: split Store::addToStore
same as before, only for a different method.

Change-Id: Id7238b26a25c21fe09df7147b16a39cc5371d8c0
2025-03-03 02:02:49 +00:00
eldritch horrors 719f58b930 libstore: asyncify Store::topoSortPaths
Change-Id: Iaad42a6f021434a5f160c88ca7c8afe6ff280806
2025-02-24 15:09:02 +00:00
eldritch horrors c6f5a6a196 perl: fix computeFSClosure
we must await the promise. also add a Werror argument to not break this again.

Change-Id: Ia02fee1720c0280853bc299d3d20301370d93e56
2025-02-24 15:09:02 +00:00
eldritch horrors 8fc0e0d24b libstore: asyncify Store::exportPath{,s}
Change-Id: I92915a3cb27bda73e7b9f3b32dc1feea3f47abda
2025-02-22 23:24:26 +00:00
eldritch horrors 81275c9ad7 libstore: asyncify Store::addTempRoot
Change-Id: Ie3ef2a637d54b22f0bf9c8aba14d2f7546c6cd9a
2025-02-14 15:28:50 +00:00
eldritch horrors af43fdec70 libstore: asyncify Store::addToStore
Change-Id: I0da60f3dc74525395bbf75fe6db56fce572cca7c
2025-02-10 12:54:19 +00:00
eldritch horrors e85242d6da libstore: asyncify Store::importPaths
Change-Id: Ic330b9213209fd7d3cccf90af7f3d6b41aede9b5
2025-02-08 17:26:19 +00:00
eldritch horrors ecfda8abe2 libstore: asyncify openStore
Change-Id: Ia152bfd2014851590328e732f434dff6695304bd
2025-02-08 12:45:15 +00:00
eldritch horrors 68947e7a65 libstore: asyncify Store::derivationFromPath
Change-Id: Ic6b54642da08f258c113d5076b34b7c13096f540
2025-02-04 14:03:54 +00:00
Jade LovelaceandJade Lovelace f5fbf9bce0 perl: simplify packaging slightly
Change-Id: Ib9049642b4afd7aa529501672cfa50d983f85e4f
2025-01-29 10:10:11 -08:00
eldritch horrors 0ad79775b6 libstore: asyncify Store worker entry points
Change-Id: Iafa0c4093064095d196c738cc7e9ba1d8e906b8c
2025-01-25 13:01:21 +00:00
Jade Lovelace bf3ebde25f pkg-config: remove legacy include paths!
This is a breaking change for non-migrated external clients. External
users always need to use <lix/libcmd/foo.hh> type include paths now.

This is as was always planned with the include rearrangement.

Change-Id: I269be91ff9f9cc94d5d3043cf3e0bdf8db1d8edb
2025-01-16 00:12:57 -08:00
Jade Lovelace 103cd44a04 Include rearrangement: rest
Change-Id: If9b5502ff348d358d7007b885e26e98a0d228f39
2024-11-18 12:31:19 -08:00
Jade Lovelace 370ac940dd refactor: make HashType and Base enum classes for type safety
Change-Id: I9fbd55a9d50464a56fe11cb42a06a206914150d8
2024-08-08 14:53:17 -07:00
Tom Hubrecht a39ba22ff7 util.hh: Delete remaining file and clean up headers
Change-Id: Ic1f68e6af658e94ef7922841dd3ad4c69551ef56
2024-05-29 12:38:51 +02:00
Jade LovelaceandQyriad c97e17144e packaging: rename nixexpr -> lixexpr and so on
This breaks downstreams linking to us on purpose to make sure that if
someone is linking to Lix they're doing it on purpose and crucially not
mixing up Nix and Lix versions in compatibility code.

We still need to fix the internal includes to follow the same schema so
we can drop the single-level include system entirely. However, this
requires a little more effort.

This adds pkg-config for libfetchers and config.h.

Migration path:
expr.hh      -> lix/libexpr/expr.hh
nix/config.h -> lix/config.h

To apply this migration automatically, remove all `<nix/>` from
includes, so: `#include <nix/expr.hh>` -> `#include <expr.hh>`. Then,
the correct paths will be resolved from the tangled mess, and the
clang-tidy automated fix will work.

Then run the following for out of tree projects:

```
lix_root=$HOME/lix
(cd $lix_root/clang-tidy && nix develop -c 'meson setup build && ninja -C build')
run-clang-tidy -checks='-*,lix-fixincludes' -load=$lix_root/clang-tidy/build/liblix-clang-tidy.so -p build/ -fix src
```

Related: https://git.lix.systems/lix-project/nix-eval-jobs/pulls/5
Fixes: https://git.lix.systems/lix-project/lix/issues/279
Change-Id: I7498e903afa6850a731ef8ce77a70da6b2b46966
2024-05-23 16:45:23 -06:00
Qyriad 4f5e532b8b meson: implement perl bindings
Change-Id: Ie1bfb0aa784e6136a82d518a652d0ae60c4b047a
2024-04-02 11:22:42 -06:00
John EricsonandRobert Hensing 7ad66cb3ef Allow dynamic derivation deps in inputDrvs
We use the same nested map representation we used for goals, again in
order to save space. We might someday want to combine with `inputDrvs`,
by doing `V = bool` instead of `V = std::set<OutputName>`, but we are
not doing that yet for sake of a smaller diff.

The ATerm format for Derivations also needs to be extended, in addition
to the in-memory format. To accomodate this, we added a new basic
versioning scheme, so old versions of Nix will get nice errors. (And
going forward, if the ATerm format changes again the errors will be even
better.)

`parsedStrings`, an internal function used as part of parsing
derivations in A-Term format, used to consume the final `]` but expect
the initial `[` to already be consumed. This made for what looked like
unbalanced brackets at callsites, which was confusing. Now it consumes
both which is hopefully less confusing.

As part of testing, we also created a unit test for the A-Term format for
regular non-experimental derivations too.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>

Apply suggestions from code review

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-09-07 10:39:37 -04:00
John EricsonandCale Gibbard 903700c5e1 Simplify ContentAddress
Whereas `ContentAddressWithReferences` is a sum type complex because different
varieties support different notions of reference, and
`ContentAddressMethod` is a nested enum to support that,
`ContentAddress` can be a simple pair of a method and hash.

`ContentAddress` does not need to be a sum type on the outside because
the choice of method doesn't effect what type of hashes we can use.

Co-Authored-By: Cale Gibbard <cgibbard@gmail.com>
2023-07-07 07:30:01 -04:00
Robert HensingandGitHub e641de085b Merge pull request #3746 from obsidiansystems/path-info
Introduce `StoreReferences` and `ContentAddressWithReferences`
2023-04-17 15:49:48 +02:00
Robert Hensing 1c0b680ef9 libstore: Remove lockCPU dead code
Left over from 9747ea84b, https://github.com/NixOS/nix/pull/5821
2023-04-07 17:50:40 +02:00
Robert Hensing a692c43729 Move loadConfFile() to initLibStore
Part of an effort to make it easier to initialize the right things,
by moving code into the appropriate libraries.

Using libstore without loading the config file is risky, as sqlite
may then be misconfigured. See https://github.com/cachix/cachix/issues/475
2023-04-07 16:24:18 +02:00
John Ericson d381248ec0 No inheritance for TextInfo and FixedOutputInfo 2023-02-28 12:14:11 -05:00
John Ericson 85bb865d20 Revert "Remove some designated initializers"
This reverts commit ee9eb83a84.
2023-02-28 11:57:20 -05:00
John Ericson d12f57c2c0 Merge remote-tracking branch 'upstream/master' into path-info 2023-02-28 11:34:34 -05:00
Maximilian Bosch 51013da921 perl: run initLibStore() on openStore()
Since #7478 it's mandatory that `initLibStore()` is called for store
operations. However that's not the case when running `openStore()` in
Perl using the perl-bindings. That breaks e.g. `hydra-eval-jobset` when
built against Nix 2.13 which uses small portions of the store API.
2023-02-02 15:56:00 +01:00
John Ericson ee9eb83a84 Remove some designated initializers
With the switch to C++20, the rules became more strict, and we can no
longer initialize base classes. Make them comments instead.

(BTW
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2287r1.html
this offers some new syntax for this use-case. Hopefully this will be
adopted and we can eventually use it.)
2023-02-01 11:25:56 -05:00
John Ericson b3d91239ae Make ValidPathInfo have plain StorePathSet references like before
This change can wait for another PR.
2023-01-14 16:42:03 -05:00
John Ericson 91617f80ec Fix perl bindings 2023-01-06 16:00:10 -05:00
John Ericson 6a168254ce Use named field initialization for references 2023-01-06 12:24:20 -05:00
John Ericson 8ba089597f Merge remote-tracking branch 'upstream/master' into path-info 2022-03-10 15:48:14 +00:00
Eelco Dolstra df552ff53e Remove std::string alias (for real this time)
Also use std::string_view in a few more places.
2022-02-25 16:13:02 +01:00
Ben Radford 861404a87b Add missing Nix::Store import to fix nix-serve StoreDir. 2021-11-23 12:56:19 +00:00
John Ericson f4f3203aa7 Merge remote-tracking branch 'upstream/master' into path-info 2021-09-30 22:41:53 +00:00