Commit Graph
17923 Commits
Author SHA1 Message Date
eldritch horrors 6785f5c720 libstore: rename query{,Static}PartialDerivationOutputMap
neither are actually partial now, and the the non-Static variant has a
non-Partial wrapper which merely returns the Partial result unchanged.

Change-Id: I5fa86682883c2305cc12c711ccff58537b7a278d
2025-05-20 17:43:46 +00:00
eldritch horrors 6b5f82e78b libstore: deoptionalize queryPartialDerivationOutputMap
derivation outpaths are now statically known at all times. the one snag
here is that the wires encode even statically known paths as optionals,
forcing us to check for this any time we receive an output map. remotes
answering with nullopt paths for derivations we still support now would
be a protocol error on its own though, so we do not diagnose it deeply.

Change-Id: Ib7080b2a0c45c3506233e87c8ef6842576f61050
2025-05-20 17:43:46 +00:00
eldritch horrors ca7f6ff96b libstore: remove unused realisation methods
Change-Id: I7e7371cdbe477f25e9410272ad635eddadf32101
2025-05-20 17:43:46 +00:00
eldritch horrors 68ab8797b5 libstore: remove unused realisation disk caching
we don't need to touch the schema of the cache here. keeping the table
around doesn't hurt (and avoids cppnix breakage) thanks to foreign key
constraints and the ca bits of the schema being independent enough for
us to just ignore them (and not having to do any maintenance on them).

Change-Id: Ib5d8eb1cd838826d88eb65bbf8f245703a2482da
2025-05-20 17:43:46 +00:00
eldritch horrors 976f6de81e libstore: remove realisation query support
only a daemon wire operation and the perl bindings could initiate these
queries at this point. the daemon ops can throw an error instead (as if
the daemon were older) and realistically should never be queries if the
client hasn't evaluated a ca derivation on a given store, and perl code
is best off dying early. nothing known except hydra uses these bdingins
anyway, and we control our hydra so we don't need backward compat code.

Change-Id: Ia7df27aba59a4a4a692ae014f407415f3bea63f2
2025-05-20 17:43:46 +00:00
eldritch horrors aa69d39c0f libstore: drop feature-gated realisation queries
these will never run without the ability to enable the feature.

Change-Id: I917024e8c3c5b1f422c9e5a509130998bee4e511
2025-05-20 17:43:46 +00:00
eldritch horrors dc47f9aa72 libstore: don't return optionals from Derivation::path
output paths are always known now that CA and deferred outputs are gone.

Change-Id: I359d13ffb5141f1e07a5fc55425831af3332c22e
2025-05-20 17:43:46 +00:00
eldritch horrors 1cbb6ba21c libstore: remove Store::registerDrvOutput
it's only used by the RegisterDrvOutput daemon wire operation now, and
that one we can safely stub out to throw an error when called instead.

Change-Id: If29716976392c9c7a2a05b151dfe80b2c8d9c07d
2025-05-20 17:43:46 +00:00
eldritch horrors f25dc923ca libstore: remove ca support from common store api
this removes the ca-derivations system feature and, perhaps most
importantly, realisation closure copy support. the latter is not
needed any more and its existence blocks some more code removal.

Change-Id: I2931b03637e25d35252ae6bd5f34f0c0168d80e9
2025-05-20 17:43:46 +00:00
eldritch horrors 484319fd2d libstore: remove unused Derivation::tryResolve
Change-Id: Ia36b066badf60b3727ec6c6a04057c3c97461e2c
2025-05-20 17:43:46 +00:00
eldritch horrors 01dcbf3359 libstore: remove Derivation::hasKnownOutputPaths
it's always true now that floating and deferred outputs are gone.

Change-Id: Ie694b9af4d2c247c0fb4fdebadd55a0a487b9828
2025-05-20 17:43:46 +00:00
eldritch horrors e543ac686f libstore: remove DerivationOutput::Deferred
we can't create these any more except by reading an old json-formatted
derivation that used them. since we cannot do anything with a deferred
derivation even when read we will remove json support for them as well

Change-Id: I4f9ea0b7c6469f57977784037f7710f939e40a2c
2025-05-20 17:43:46 +00:00
eldritch horrors d03be35c44 libstore: remove DrvHash::Kind
now that we have no deferred hashes (since floating ca derivations were
the only way to create them) we can safely remove this enumeration too.

Change-Id: Ic72ed90500fcee7aa5b3b5a302477fa515acf1be
2025-05-20 17:43:46 +00:00
eldritch horrors e3717b728c libstore: trivialize DerivationType::ContentAddressed
only FODs can be content-addressed now, and those are always fixed.
FODs are also never sandboxed, so we do not need that field either.

Change-Id: I1be62b3ec85e08ec003cc8769723328d19777728
2025-05-20 17:43:46 +00:00
eldritch horrors bfd10db217 libstore: remove DerivationOutput::CAFloating
nothing can create floating ca outputs any more.

Change-Id: Ic69f4a22066e1f5c0837f44e8d4fa2d93ca20ff6
2025-05-20 17:43:46 +00:00
eldritch horrors a7866d56b8 cli: remove ca support from commands
this mostly takes the form of removes feature checks and the associated
"ca derivations enabled" branches, but for the realisation info command
turns into a stub. we keep it around for compatibility, but from now on
it will always throws "ca derivations not implemented" errors when run.

Change-Id: I0abea5f76262013415330adcca2b498c6dca555b
2025-05-20 17:43:46 +00:00
eldritch horrors 6567707dc1 libstore: remove DrvOutputSubstitutionGoal
this goal is only involved for output paths that aren't known at initial
build time, which in turn can only happen if they are ca paths. since we
can no longer create ca derivations during eval *or* read them from disk
we can now assume that we will never run this goal. there are still some
vestiges like output known-ness we can't remove yet, so those must stay.

Change-Id: I989e5ad4600c628bcbe8e17e1b082ce8d73a3bd9
2025-05-20 11:28:12 +00:00
eldritch horrors b60735791e libstore: remove unused drvOutputReferences
ca derivation build was the last remaining user.

Change-Id: Ib0e6d954e7802a53d1328c4c18f5f12dddeb838a
2025-05-20 13:27:04 +02:00
eldritch horrors d4e88b98e9 libstore: remove ca derivation build support
we remove not only support for *building* a ca derivation, but also
support for *resolving* ca derivations as part of a build. we never
have to resolve derivations from here on, so this code is now dead.

Change-Id: I0346442d5fa00eb927177545ae61315f588477cc
2025-05-20 13:27:04 +02:00
eldritch horrors 2cc420c1e5 libstore: remove ca derivation read support
we can now no longer read ca store derivations from disk.

Change-Id: I233edea597b550dd7e8c78a555b7f12760a993dd
2025-05-20 13:27:04 +02:00
eldritch horrors 8d5bc9ed48 libexpr: remove ca derivation eval support
Change-Id: I8c06825d0fa7544b8bc9e3bda948e84a5f21ee16
2025-05-20 13:27:04 +02:00
eldritch horrors dad28eca75 cli: disallow ca derivations
we no longer have any experimental features depending on ca derivations,
so we can start removing them. since ca derivations are very invasive we
will need a while to remove all of the explicitly experimental code, and
even then we will not have removed *all* code related to ca derivations.
especially in the derivation goals there is a lot of code that is not as
easy to disentangle from experimental features as some would have hoped.

Change-Id: Ia456aadc6164613ded343f571318494d9310a549
2025-05-20 13:27:04 +02:00
piegames 0fbbb1e49b libexpr: Switch StaticEnv to LinearMap
Change-Id: If98bfafce9fa5235fe962274c03c619fe965dd60
2025-05-19 16:21:10 +02:00
piegamesandLix Systems Gerrit eb18a90afb Merge "libutil: Introduce LinearMap" into main 2025-05-19 14:20:34 +00:00
piegames bd8ec106fa libutil: Introduce LinearMap
Change-Id: I68ce4c1dc17b0742690e49f62206c65f5a1a4a30
2025-05-19 15:35:52 +02:00
Lily BallardandLix Systems Gerrit 8fa0363b91 Merge "libutil: read window size from stdout if stderr fails" into main 2025-05-18 21:09:16 +00:00
Raito BezariusandLix Systems Gerrit cfaeed469b Merge "libstore/ssh: remove echo started check" into main 2025-05-18 19:51:20 +00:00
Raito Bezarius 0dd8bf6c1c libstore/ssh: remove echo started check
The pre-flight `echo started` check over SSH was originally added in
577ebeaefb. As it is usual with these old
commits, understanding why is there a need for something is difficult.

The closest thing would be
> Fix a race starting the SSH master. We now wait synchronously for
> the SSH master to finish starting. This prevents the SSH clients
> from starting their own connections.

But, we removed SSH connection sharing, so this does not apply anymore.

Nonetheless, we believed this check was meant as a way to catch obvious
misconfigurations or SSH failures early, before handing off to
`nix-store`. However, this approach was not fruitful: it assumes the
remote has a `bash`-compatible shell, `echo` behaves in a standard way,
and no `ForceCommand` interferes—all of which are unreliable assumptions
in practice.

While the intent was to provide slightly better diagnostics (e.g. in
case of SSH hanging or returning an interactive shell), in practice it
does not meaningfully catch or improve real failure cases. The
underlying protocol or engine can and should handle those errors more
robustly anyway.

In contrast, this check *does* break several legitimate workflows,
including:

* remote builders using `ForceCommand` wrappers (e.g.
`nix-remote-build`-style setups), see
<https://discourse.nixos.org/t/wrapper-to-restrict-builder-access-through-ssh-worth-upstreaming/25834/15>,

* SSHing into minimal environments lacking `bash` (e.g. initrd,
busybox-based systems),

* configurations that don’t default to POSIX-like shells, e.g., nushell
enthusiasts.

As such, we’re removing this code. Protocol mismatch errors and SSH
failures can be rethought and handled more structurally elsewhere in the
engine.

Change-Id: I187f6881375d42ef83987a13a350c97964bbdb30
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-05-18 18:45:26 +02:00
Lily Ballard 8630cedbca libutil: read window size from stdout if stderr fails
This fixes the output of `nix help 2>/dev/null` so it stops wrapping at
60 columns.

Change-Id: Id0000000842b5befd73afbd6aff2825bdeae46af
2025-05-17 12:19:06 -07:00
Linus HeckemannandLix Systems Gerrit a7634f87aa Merge "libstore/local-derivation-goal: better debuggability for FOD mismatches" into main 2025-05-16 15:18:11 +00:00
eldritch horrors 8029cec3c0 libstore: remove some unused code
these must've been left around by accident if past cleanups.

Change-Id: I3458ae45e362e7912a6bbe1f31956c41094144d3
2025-05-16 13:16:27 +02:00
Rebecca Turner 3f355b8fd1 .editorconfig: json: init with 4 space indent
Relevant for `version.json`. Noticed while running the 2.93.0 release.

Change-Id: I9f740a3cd412c1d27a76c3feb4e05d89319de3b4
2025-05-15 15:11:57 -07:00
Rebecca Turner 61920dd663 releng: Update README.md documentation
These are (polished versions of) some notes I wrote down while preparing
the 2.93.0 release with Jade.

Fixes: https://git.lix.systems/lix-project/lix/issues/441

Change-Id: Ib7f0b83ce2984a86d3a0c354e707fc6ee569a7ea
2025-05-15 10:47:10 -07:00
Lily BallardandLix Systems Gerrit b7ce00fc55 Merge "libutil: move filterANSIEscapes tests" into main 2025-05-15 01:42:34 +00:00
rebecca “wiggles” turnerandLix Systems Gerrit fed92d56ee Merge changes Id732c31a,I022583e9,I2f08bd2f,I85090f4c into main
* changes:
  aws s3 cp: Note issue with `--checksum-algorithm=SHA256`
  releng/docker_assemble.py: fix empty `auths` error
  releng: remove unused variables
  releng: ignore `prev-git-branch.txt`
2025-05-14 19:15:53 +00:00
Lily BallardandLix Systems Gerrit 5ed5efe6fc Merge "libcmd: use correct stream for ANSI testing for markdown" into main 2025-05-14 02:15:18 +00:00
Lily Ballard bc5d7ad458 libutil: move filterANSIEscapes tests
This moves the original test suite for `filterANSIEscapes` into the same
file as the newer tests. There is some overlap between the old and new
tests but that doesn't hurt anything so I kept them as-is.

Change-Id: Id00000009919024a5f206ec9a7bc0022541ff612
2025-05-13 19:04:52 -07:00
Lily BallardandLix Systems Gerrit 6237c50161 Merge "libutil: handle OSC escapes in filterANSIEscapes()" into main 2025-05-14 02:02:08 +00:00
Lily Ballard 207b5d81bf libutil: handle OSC escapes in filterANSIEscapes()
This teaches `filterANSIEscapes()` how to find the end of an OSC
sequence. It also keeps OSC 8 (hyperlinks) when not instructed to filter
out all escapes, just as it keeps colors.

This also relaxes the parsing of CSI escapes to find the end of the
sequence for invalid sequences, and handles better escapes that don't
start CSI or OSC.

This fixes the repl output for `:doc builtins.fetchGit`.

Fixes: https://git.lix.systems/lix-project/lix/issues/160
Change-Id: Id0000000f2a6956c042c883a4545edf347fa1799
2025-05-13 18:59:08 -07:00
Rebecca Turner a3b3b06a21 aws s3 cp: Note issue with --checksum-algorithm=SHA256
There may or may not be a bug in `garage` here. Previously we added
`--checksum-algorithm=SHA256` here to fix it, but when @rbt was
running the release for 2.93.0, she found it actually made the S3
uploads fail.

If this command is failing, here are some links to investigate.

See: https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/963
See: https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/1017
See: https://github.com/boto/boto3/issues/4392
See: https://docs.aws.amazon.com/cli/v1/userguide/cli-configure-envvars.html#envvars-list-AWS_REQUEST_CHECKSUM_CALCULATION
Change-Id: Id732c31aa715191e78c7f0246e1b74cc202e675c
2025-05-13 12:45:21 -07:00
Rebecca Turner 4611d1f331 releng/docker_assemble.py: fix empty auths error
I had a `~/.docker/config.json` which was missing an `auths` key, which
caused an error. The release automation succesfully ignored the error,
but it was noisy. Using `json_obj.get('auths', {})` instead of
`json_obj['auths']` fixes this `KeyError`.

Change-Id: I022583e9e668bf8ad7bdc1fa5a3305aee2f18d85
2025-05-13 12:18:32 -07:00
Rebecca Turner 2e25580d8d releng: remove unused variables
`lib` and `config` were unused here. In the future maybe we should
integrate `deadnix` or something similar for linting.

Change-Id: I2f08bd2f87f74b90a5f76ea7db7e6d4db1663450
2025-05-13 12:18:31 -07:00
Rebecca Turner 53673b7331 releng: ignore prev-git-branch.txt
This file is created by `python -m releng tag`.

Change-Id: I85090f4c6d9c2ba9a991b47df6a7382b80f7ff52
2025-05-13 12:18:31 -07:00
Raito BezariusandLix Systems Gerrit 901940df00 Merge "libstore/profiles: do not dereference current generation if it doesn't exist" into main 2025-05-12 22:47:48 +00:00
eldritch horrors 3684c2f8a0 libutil: remove dynamic derivations feature
Change-Id: Id48c775197e89bfda711c5cb03752980785c3d26
2025-05-12 13:37:54 +02:00
eldritch horrors ccdd916226 libstore: de-ref {Derived,Built}Path::Built drvPath
they're no longer recursive, so this is perfectly fine.

Change-Id: If565a557f2c2074e2a96a7f89c51ff1c51146b36
2025-05-12 13:37:54 +02:00
eldritch horrors 84c1df46ea libstore: remove DerivedPathMap
single-level maps suffice now that dynamic derivations are gone.

Change-Id: If29998b104b31255292ab0c789622d7d27040f69
2025-05-12 13:37:54 +02:00
eldritch horrors 68dfcfc6a4 treewide: don't resolveDerivedPath opaque paths
resolution of opaque paths is just an expensive `->path`.

Change-Id: I0c8d8b908f358d0bd26d1dcba84b2de6cbdc7c29
2025-05-12 13:37:54 +02:00
eldritch horrors f5e2e78266 libexpr: remove more obsolete DerivedPath methods
split from the prior commit for easier review.

Change-Id: Iabf1bc759cb56a04211d4cb07769f53b0f302b39
2025-05-12 13:37:54 +02:00
eldritch horrors 5f723e96e6 libstore: flatten {,Single}{Built,Derived}Path
only dynamic derivations could produce a non-opaque drvPath. since
dynamic derivations are no longer supported we can have drvPath be
opaque at all times, simplifying downstream code significantly and
making quite a few methods unnecessary. discardOutputPath was only
called on drvPath members anyway and thus reduces to a copy, other
operations at the very least are no longer recursive. some vestige
of dynamic derivations remains in DerivedPathMap though (for now).

Change-Id: Ifb4ad53a3c67800be5a62540068c8279d4ae0046
2025-05-12 13:37:54 +02:00