Commit Graph
17688 Commits
Author SHA1 Message Date
Mario RodasandLix Systems Gerrit dc69d64bff Merge "Fix typos in documentation" into main 2025-03-18 17:32:53 +00:00
jadeandLix Systems Gerrit 6e6ff58272 Merge "feat!: remove experimental repl-flakes" into main 2025-03-18 16:02:55 +00:00
Tom Hubrecht a5e2376350 documentation: Explain how a float will be transformed into a string
`prim_toString` uses `state.coerceToString`, which in turns defers to
`std::to_string` for floats, which 'Converts a floating point value to a
string as if by std::sprintf(buf, "%f", value)'.
Finally, the `%f` specifier 'Converts floating-point number to the
decimal notation in the style [-]ddd.ddd. Precision specifies the exact
number of digits to appear after the decimal point character. The
default precision is 6.'

Closes #747

Change-Id: I42339651005d20f272459cf9f80b274f2076b1e3
2025-03-18 13:42:25 +01:00
eldritch horrors 3ed99ad26e n-e-j: use full lix warning config
this would've caught the missing aio await. also disable some rather
annoying deprecation and missing initializer warnings in dev builds.

Change-Id: Ibbb9dfc67eada0e7843081b669b7c8726a02ec29
2025-03-18 09:13:11 +00:00
eldritch horrors 274933c729 n-e-j: add missing aio block on Store::queryMissing
Change-Id: I7c2350d2bf0d729d26f67a2b2181b2ebc7efb4f8
2025-03-18 09:13:11 +00:00
Mario Rodas 992866bc91 Fix typos in documentation
Change-Id: I93c3801290d03244ea8db96679d002589ae90b74
2025-03-18 04:20:00 +00:00
jadeandLix Systems Gerrit 5243a6f8b4 Merge "fix uint64_t-to-size_t narrowing for 32bit systems" into main 2025-03-18 01:42:59 +00:00
jadeandLix Systems Gerrit ea4cf48879 Merge "post-build-hook: don't emit config settings that aren't overridden" into main 2025-03-17 23:31:32 +00:00
jadeandLix Systems Gerrit 15c6679daf Merge "release-notes: note the flake.lock fix" into main 2025-03-17 23:02:44 +00:00
KFearsandJade Lovelace 2c21882767 feat!: remove experimental repl-flakes
This is a backport of CppNix PR 10299

Fixes #557

Change-Id: I848e8c5af4bde09afba053a5911fbbd54004fbb8
2025-03-17 16:01:53 -07:00
jadeandLix Systems Gerrit 41e10862cb Merge "fix: bad error message for nix store delete --ignore-liveness" into main 2025-03-17 22:33:51 +00:00
Jade Lovelace 523965697d post-build-hook: don't emit config settings that aren't overridden
This is a redesign from 9b1f3cbc13 where
this was introduced.

I deleted the AbstractConfig::toKeyValue since it was conspicuously and
obviously broken for two years since 450e5ec618.
I asked myself if anyone was using it, given that it only emitted
settings that were aliases (broken!), and found that nobody used it.

The motivation for this change is the same for only emitting overridden
settings to the protocol: the nix inside there may not be able to parse
our defaults, as is the case of CppNix since the consensual
accept-flake-config was added to Lix.

Fixes: https://git.lix.systems/lix-project/lix/issues/739
Change-Id: Ib9874a52137f1f22220c25bcfa2425a4802509c7
2025-03-17 15:25:52 -07:00
eldritch horrors 6aff96d9cd fix uint64_t-to-size_t narrowing for 32bit systems
fixes #733

Change-Id: Ib0f4d90e32c81f0bd1b7ccacb576b867bfa59690
2025-03-17 23:11:58 +01:00
jadeandLix Systems Gerrit 8c5a1d54d9 Merge "logging: always print post build hook logs" into main 2025-03-17 21:30:32 +00:00
Jade Lovelace d857424488 release-notes: note the flake.lock fix
Change-Id: I1c5f7a5cba12da1bfb2896ffc04ec89862cb346d
2025-03-17 12:07:29 -07:00
rebecca “wiggles” turnerandLix Systems Gerrit 175091efb5 Merge "run.hh: Declare chrootHelper" into main 2025-03-17 19:07:01 +00:00
jadeandLix Systems Gerrit 3cfc8be580 Merge changes I6a83bdb3,If4ee20c3 into main
* changes:
  README: link to the manual
  plugins: support nix_plugin_entry, do some minor reworks
2025-03-17 18:13:31 +00:00
Jade Lovelace c597589d70 logging: always print post build hook logs
Missing these is really annoying, causes massive problems for debugging
(if you don't to enable build logs), and there's basically no reason we
can't just always show them.

In the future we might want to emit these to syslog/systemd/etc, but for
now let's just always print them to the console.

CC: https://git.lix.systems/lix-project/lix/issues/675

Change-Id: Ib6c47f3e34e553505298ed49c8bdb43f3db317a4
2025-03-17 11:12:25 -07:00
rebecca “wiggles” turnerandLix Systems Gerrit 366f55303a Merge "justfile: enable just build install test" into main 2025-03-17 17:48:55 +00:00
Rebecca Turner aa135863fb run.hh: Declare chrootHelper
Change-Id: I94c61e87a7649286c81f1716d60d5a85be876da2
2025-03-17 10:47:26 -07:00
Jade Lovelace d325bec052 README: link to the manual
Well that's silly. We definitely should have done that before.

Change-Id: I6a83bdb3847bb291ac57d30ea28ee33ad7121dbf
2025-03-16 16:54:11 -07:00
Jade Lovelace adf45deeca plugins: support nix_plugin_entry, do some minor reworks
Plugins are already allowed to fail to load, let's be even more tolerant
of them failing to load by ignoring inaccessible ones with a warning.

This change also significantly reworks the docs for plugins.

This is a tiny extremely partial backport of the C API at
https://github.com/NixOS/nix/pull/8699.

Fixes: https://git.lix.systems/lix-project/lix/issues/740
CC: https://git.lix.systems/lix-project/lix/issues/359

Change-Id: If4ee20c3daaf26c8184a415eef3e20ca5b5e7aef
2025-03-16 16:43:54 -07:00
Rebecca Turner 2a336813ad pre-commit: fix stage name warnings
Fixes these warnings:

[WARNING] hook id `check-executables-have-shebangs` uses deprecated
stage names (commit) which will be removed in a future version.
run: `pre-commit migrate-config` to automatically fix this.

Change-Id: Iab7d35999244df0eb4c32541b1f9c6a2dc2f3e28
2025-03-16 22:57:11 +00:00
rebecca “wiggles” turnerandLix Systems Gerrit ccab8cdd30 Merge changes I3fc0f159,I06068cbd,I76405e94,I255e2d94 into main
* changes:
  registerNix* -> registerLegacyNix*
  LegacyCommands -> LegacyCommandRegistry
  Commands -> CommandMap
  RegisterCommand::Commands -> CommandMap
2025-03-16 21:10:17 +00:00
Maximilian BoschandLix Systems Gerrit a3796499ba Merge "libstore: set connect-timeout to 5" into main 2025-03-16 18:09:30 +00:00
giliceandLix Systems Gerrit 6b6b901d16 Merge "bench: rewrite in python, add instruction count profiling" into main 2025-03-16 17:06:41 +00:00
Rebecca Turner e7463af241 justfile: enable just build install test
Previously, only the `install` target in the `justfile` took extra
`*OPTIONS`, which meant you could run `just build install test` (or
`just clean setup build install test` for a clean build). This is much
more convenient than `just build && just install && just test`.

However, sometimes you *do* need extra options for some of those other
targets, so over time they have gained extra arguments. But these
prevent you from chaining the targets together:

    $ just clean setup build install test
    rm -rf build
    meson setup build --prefix="$PWD/outputs/out" $mesonFlags build install test
    usage: meson [-h]
                 {setup,configure,dist,install,introspect,init,test,wrap,subprojects,rewrite,compile,devenv,env2mfile,reprotest,format,fmt,help}
                 ...
    meson: error: unrecognized arguments: install test
    error: Recipe `setup` failed on line 13 with exit code 2

As a compromise, I've renamed the targets with extra arguments to
include a `-custom` suffix, and added aliases for the old targets to
call the `-custom`-suffixed target with no extra arguments.

This makes it possible to run `just build install test` again, but keeps
the ability to run `just build-custom EXTRA_MESON_BUILD_ARGS`.

BONUS:
- Added `test-unit` and `test-integration`, because I always forget the
  arguments to run a particular test suite and the names of those test
  suites.
- Added doc comments to `lint` and `lint-fix`.

Change-Id: I61ec66f5e4d38c12bbae4fa226d7b4cea94579d1
2025-03-15 21:40:56 -07:00
Rebecca Turner d86b857505 registerNix* -> registerLegacyNix*
Once I add `registerNixDaemon` and similar for the new-style commands,
they'll conflict with the old `register*` functions, so let's add
`legacy` to their names.

Change-Id: I3fc0f15985abe43bc1f257be3ebd9021b66055a3
2025-03-15 21:28:15 -07:00
Rebecca Turner 53bf387022 LegacyCommands -> LegacyCommandRegistry
This name made sense at the time, but now I'm looking at making the
corresponding change to `CommandRegistry`, and I don't think `Commands`
is a good name for _that_ type, but I don't want the types to be
mismatched, so here we are.

Change-Id: I06068cbde00f49ff3d720c505a567a152b00b61c
2025-03-15 21:28:15 -07:00
Rebecca Turner 2ecbbc6e33 Commands -> CommandMap
This name, for a different type, is also vague.

Change-Id: I76405e940f3f1761bf0d66ba985d8c1dfc0b2d59
2025-03-15 21:27:22 -07:00
Rebecca Turner f1543bdf62 RegisterCommand::Commands -> CommandMap
This name is vague.

Change-Id: I255e2d94a72193ca1a10f2abf4a3011c75ec9ecf
2025-03-15 21:26:04 -07:00
Rebecca Turner 29732f19a2 pre-commit: add keep-sorted hook
This sorts lines alphabetically between `keep-sorted start` and
`keep-sorted end` markers.

See: https://github.com/google/keep-sorted

Closes #730

Change-Id: Ib36f3da81fcf5e2f588ffb998860456405607eea
2025-03-15 18:03:56 -07:00
Maximilian Bosch 7359c39076 libstore: set connect-timeout to 5
Right now, Lix waits for quite a while five times[1] if a substituter is
down. If the substituter is not reachable for that long, it's probably
down or so slow that it doesn't make sense to download from it. Also,
most people would abort earlier assuming that Lix just behaves weird in
that case.

Reducing the timeout to 5s to make the behavior a little better.

If people live in areas with poor connection where a higher timeout
would be appropriate, I'd argue that they should increase the timeout
rather than requiring everbody else to set it to a non-zero value.

Additionally, communicate how often we're re-trying.

[1] Because of `download-attempts` from `nix.conf(5)`

Change-Id: I8500dce0c8230247dd492e43cc7af4d3b58c4710
2025-03-15 16:17:25 +01:00
Maximilian BoschandGerrit Code Review 801567adf0 Merge "nix-eval-jobs: reenable tests, rework waitpid handling" into main 2025-03-15 13:56:23 +00:00
Maximilian Bosch 4c5efd4548 nix-eval-jobs: reenable tests, rework waitpid handling
Closes #703

The problem boils down to:

* A testcase with a segfaulting worker due to a stack overflow from the
  evaluation.

* When the controller encounters that the pipes are down, the worker's
  state is checked and a potential infrec is reported if it stopped with
  SIGSEGV.

* However, in a few cases, the worker didn't exit yet, but only had closed
  pipes (which is how the error is detected). In that case, a different
  error message is printed breaking the test.

This bug handling only affects error cases, so this race doesn't happen
on normal shutdown. To mitigate the problem a bit, the controller will
wait for a second and re-check the state of the worker process then.

While this should make the error reporting for end-users a little more
reliable, it's still no guarantee for a non-racy test. Hence, the test
asserts against both cases now and it passes if either the infrec error
or the "worker still running, but pipes are closed" error is returned.

Change-Id: Ifdc7a05bc86b6aecd5d03118d3e3ffc9affe1c5e
2025-03-14 19:50:07 +01:00
gilice 765cd3823a bench: rewrite in python, add instruction count profiling
This is to some extent a simple transliteration of bench.sh, but also
adds some additional features:
- the ability to select which benchmarks to run, instead of having to
  run all at once (via --cases option)
- help text (--help), and argument parsing
- ability to profile instruction counts (via --mode icount).
Instruction count is a stable metric, that correlates to the execution
time, but is less affected by the environment (like other heavy programs
running, or the system hardware).
Other projects (most notably rustc) use instruction count as a primary target metric.

Change-Id: I791cac499e64983efb8bcd4ea1faef13ded09ee6
2025-03-14 19:24:21 +01:00
jadeandGerrit Code Review 85d614538f Merge "releng: workaround awscli breaking garage by changing checksum algo" into main 2025-03-14 18:22:23 +00:00
pamplemousseandGerrit Code Review af15a446ea Merge "Allow diff-closures to output JSON" into main 2025-03-14 13:11:11 +00:00
eldritch horrors cbf7929277 libstore: use thread pools for remote stderr handling
spawning threads is not free. reusing threads is also not free, but a
lot cheaper than spawning now ones. the difference can add up quite a
bit on nixos system evaluation; we've seen 20k threads cycled through
on even modestly sized configurations, and at 20µs per thread just to
launch and tear down (excluding any work done) that's potentially not
insignificant. in our test config this was a 3% eval time difference.

Change-Id: I40f901813e700fb99f5def82da54ca969711961b
2025-03-14 09:39:09 +00:00
Jade Lovelace 0f4c5b3b8a releng: workaround awscli breaking garage by changing checksum algo
We use https://garagehq.deuxfleurs.fr for https://docs.lix.systems,
https://releases.lix.systems, https://cache.lix.systems. It's generally
great, but AWS doesn't, erm, care, about other implementations and broke
their client library.

We already ran into
https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/824, which was
mitigated by a garage upgrade to create a new error. These bugs were
what got us:
- https://github.com/boto/boto3/issues/4392
- https://github.com/aws/aws-cli/issues/9214

Error:
upload failed: release/manual/.nojekyll to s3://docs/manual/lix/nightly/.nojekyll An error occurred (InvalidRequest) when calling the PutObject operation: Bad request: invalid checksum algorithm

The missing checksum algorithm is CRC32NVME, with a bug filed here:
https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/963

Change-Id: Ib78a89034bf0f2a6773fc505a347b2aadb775e93
2025-03-13 15:08:43 -07:00
ToastandGerrit Code Review a4514e44f8 Merge "Repl: error out if :log is called without a derivation" into main 2025-03-13 19:15:47 +00:00
jadeandGerrit Code Review 20edd45ae8 Merge "releng: Document release validation checks" into main 2025-03-13 17:53:29 +00:00
Toast 84fbc9d644 Repl: error out if :log is called without a derivation
The repl crashes with an assertion error at lix/libutil/file-system.cc:45
if you run ':log' without any derivation, so I added a check to error out
gracefully

Change-Id: I7af70040f884e905f0514ea9883b2380fe148f15
2025-03-13 18:24:09 +01:00
Raito BezariusandGerrit Code Review b1a8c0f2e0 Merge "libfetchers: ensure that lastModified is a uint64_t" into main 2025-03-13 16:44:10 +00:00
Julian Stecklina 0e59e5b308 libfetchers: ensure that lastModified is a uint64_t
When lastModified comes via inputFromAttrs it ends up as string in the
Attrs map. This results in:

error: input attribute 'lastModified' is not an integer

Fix by handling it like revCount, which already does the right thing.
If added a test and confirmed that it catches the issue.

Also kudos to alexander.sieg@cyberus-technology.de for helping with
debugging this!

Change-Id: I8378fcaea986d798cb8458d4e6e15c2a92c2520a
2025-03-13 09:34:59 +01:00
Rebecca TurnerandJade Lovelace 761001f58d releng: Document release validation checks
After the release is created, we do some post-release validation for the
Lix packaging in Nixpkgs. This change documents those checks.

NB Most of these should probably be *pre*-release validation checks!

Closes #627.

Change-Id: I773e703582ccc8349987abd1e860a189a437cf3b
2025-03-12 13:52:13 -07:00
K900 40601ea3ab flake: don't override nix-prefetch-git when we don't need to
https://github.com/NixOS/nixpkgs/pull/384099 has landed.
Change-Id: Ic4034dcd3028f7f8e476bf436dd9e1cf174060c5
2025-03-12 22:21:20 +03:00
Pamplemousse 04cc3db4df Allow diff-closures to output JSON
Change-Id: Ia17ea93d98b38e3415c35257daf07c7978f50ef4
2025-03-12 08:03:49 +01:00
ZitroneandGerrit Code Review 85a140accb Merge "libexpr: show all missing and unexpected arguments instead of just one" into main 2025-03-11 21:54:32 +00:00
Zitrone f6c4034e2f libexpr: show all missing and unexpected arguments instead of just one
also only give suggestions for unused arguments

Change-Id: Iae8e72defbbe2571a803dfd7216745b39848ccb2
2025-03-11 20:21:24 +01:00