Commit Graph
2299 Commits
Author SHA1 Message Date
Raito Bezarius 5ca168f6f2 chore: drop experimental feature recursive-nix
This was discussed as part of https://git.lix.systems/lix-project/lix/issues/767
with a one month long comment period.

This commit removes the recursive-nix feature from Lix. It has seen
limited usage and is difficult to stabilize, especially on non-Linux
systems where its behavior is underspecified.

Maintaining this feature complicates core work on the store, as we must
account for the potential presence of the daemon in the sandbox, adding
unnecessary complexity. Additionally, its inclusion in the
platform-independent local store creates risks for non-Linux platforms.

For more details on this removal, refer to the release note entry or the
issue entry.

Change-Id: I9137202f563c0a317f9c5da79cd9fd07d801427a
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-04-25 17:09:01 +02:00
Qyriad 7dbdd5bd0c build: set XDG_CONFIG_HOME to /dummy too
This prevents the user's nix.conf from being mistakenly used during docs
generation.

Fixes #802

Change-Id: I23336b49a8aa2ddf92e803e573321383ea2e5e81
2025-04-22 16:45:44 +00:00
piegames abb8ad29c0 repl: Always use parser, allow trailing ; in assignments
We now properly shell out to the parser instead of hacking stuff
together with a regex. Stuff we get for free by doing this:

- Optional trailing semicolon
- Declaring nested attribute sets
- String identifiers, and future proofing for eventual grammar
improvements to identifiers
- Dynamic attributes

Change-Id: Ibf1ad815e5e27caf162df05ea5ba5b1b4955d9c9
2025-04-21 20:19:25 +02:00
eldritch horrors f92235e1d2 libstore: stop using ssh connection sharing
our implementation is mostly unused, completely untested, and simply
breaks when users configure connection sharing independently. we can
safely delete this "feature" and inherit user configuration instead.

also have the remote build test check that connection sharing works.

fixes #304, fixes #644

Change-Id: Iea44cc0f8e51a1d231ad186a95c7e310bbfeb303
2025-04-21 13:43:44 +00:00
jadeandLix Systems Gerrit 9c2dba4ee2 Merge "Revert "libstore: don't use curl decompression support"" into main 2025-04-09 16:01:37 +00:00
eldritch horrorsandLix Systems Gerrit 350bf7e42d Revert "libstore: don't use curl decompression support"
This reverts commit bba678e5c5.

Reason for revert: didn't fix the bug and created new ones (fj#794)

Change-Id: I0450205d3041b6c876737151a4051081c1366f1d
2025-04-09 11:28:50 +00:00
Maximilian BoschandLix Systems Gerrit d169c092fc Merge "fetchers: don't consider a path locked if a rev is specified" into main 2025-03-29 18:51:45 +00:00
eldritch horrors 0d47773d76 treewide: handle JSON parse errors
or more accurately, wrap them in a nix::Error subclass so we can display
them properly without crashing, and add some error context if available.

fixes #642
fixes #753
fixes #759
fixes #769

Change-Id: I1aad0c0501fea83f9de3a1335eaa6adc20721616
2025-03-27 08:56:14 +00:00
jadeandLix Systems Gerrit f270d3ebe0 Merge changes Ib81a5db1,Ib6d68594,Id91e1fd4 into main
* changes:
  docs: document the cursed file syntax for new CLI
  docs: document the cursed file syntax for old cli
  doc preprocessor: support indent directives
2025-03-26 16:05:26 +00:00
Stanislav AlekseevandLix Systems Gerrit 21dac7513c Merge "Revert "fix: allow access to ca-certs in FODs on darwin"" into main 2025-03-26 13:00:59 +00:00
Stanislav AlekseevandLix Systems Gerrit 5a4da16094 Revert "fix: allow access to ca-certs in FODs on darwin"
This reverts commit 692de16113.

Reason for revert: it's wrong at it doesn't actually work

Change-Id: I495eed65bb7547ceffe98f979633c080e83ac154
2025-03-26 10:27:58 +00:00
QyriadandJade Lovelace 11225f435d docs: document the cursed file syntax for old cli
This documents the fact that nix-build, nix-env, nix-instantiate, and
nix-shell accept an extended syntax for their file arguments, including
some well-known (but not well documented) aspects, like being able to
specify `<nixpkgs>`, but also https:// tarball URLs, `flake:` prefixed
flakerefs, and the cursed `channel:` prefixed hardcoded URLs

Same thing for the new CLI incoming :)

Change-Id: Ib6d68594a16132805ba5d97526e16f7b3633117e
2025-03-25 17:06:53 -07:00
Jade Lovelace 989498407b doc preprocessor: support indent directives
Janky Python makes the world go around. In repentance I removed some
jank from it.

Change-Id: Id91e1fd40cb73049cb64deab6f81a7193132fa82
2025-03-25 17:06:53 -07:00
eldritch horrors c3929c78f3 libutil: disallow enum serialization by default
allow opting in to serialization as integers via a trait type instead,
and add string-list serializers for the feature flag set enumerations.

fixes #738

Change-Id: I2746eb5ef1f15c01b4e681f9ba1615b6c6e64f44
2025-03-25 10:44:52 +01:00
jadeandLix Systems Gerrit 4181796ed2 Merge changes I0f5d9912,I442f16c7 into main
* changes:
  observability: add the first USDT probe
  tree-wide: put in more keep-sorted
2025-03-24 16:37:28 +00:00
Stanislav AlekseevandLix Systems Gerrit f0fd789f68 Merge "fix: allow access to ca-certs in FODs on darwin" into main 2025-03-24 07:32:12 +00:00
WeetHetandJade Lovelace 692de16113 fix: allow access to ca-certs in FODs on darwin
Change-Id: Id955be88eb5d419d57262d5983841a1e6accee9f
2025-03-23 19:13:44 -07:00
Jade Lovelace cad275307e observability: add the first USDT probe
USDT probes are statically defined trace points that have nearly zero
disabled-probe effect, i.e. we can put them in hot paths.

The use case for these is both similar and dissimilar to Rust tracing:
We still need better logging and a better structured rust-tracing
looking thing, but probes allow for quite easy programmable interactive
tracing in production, which we also care a lot about.

This CL comes with a perfunctory trace point in
libstore/file-transfer.cc for reading data out of the curl buffer. This
was mostly thrown in there so that I could see what the buffer sizes of
this were, and maybe be able to instrument the perf of the curl usages
in Lix in the future.

Fixes: https://git.lix.systems/lix-project/lix/issues/727
Change-Id: I0f5d9912d76bf3d6923bf53ebfd9b8d6c6e70aea
2025-03-23 18:37:47 -07:00
jadeandLix Systems Gerrit 53b41a7f9b Merge "UX: Show users SRI hashes everywhere instead of base32" into main 2025-03-23 17:46:11 +00:00
jadeandLix Systems Gerrit 3f45d5e020 Merge "fix!: file type flake inputs are always recursive hashed" into main 2025-03-23 17:45:41 +00:00
Jade Lovelace f695aa2bd2 UX: Show users SRI hashes everywhere instead of base32
This is *just* in output messages that look relatively like the FOD
failure message. It's not great that we inconsistently use the old
format in random places.

Change-Id: I4b9ce73a98a3a50c924cfe1d7cded42f3756df6b
2025-03-21 13:30:41 -07:00
Jade Lovelace b22bee91f5 fix!: file type flake inputs are always recursive hashed
Well that is a mess. I don't like breaking derivation compatibility,
but I like non-deterministic evaluation much less.

We can break compat if we own up to it, which this does.

Fixes: https://git.lix.systems/lix-project/lix/issues/750

Change-Id: Ic9e2407393f1d42c2be604f80b4aa11bc872bc23
2025-03-21 13:19:15 -07:00
5339ffb234 fix: split macos sandbox profile to avoid size errors
Fixes: https://git.lix.systems/lix-project/lix/issues/752
Fixes: https://git.lix.systems/lix-project/lix/issues/718

Co-authored-by: Poliorcetics <poliorcetics@users.noreply.github.com>
Change-Id: Icfa95ffd40f5a60f67e14148924bc77b42232e19
2025-03-21 18:52:43 +00:00
Maximilian Bosch 62fb98ae55 fetchers: don't consider a path locked if a rev is specified
It's possible to put a path into the store in pure mode by pretending
it's locked like this:

    $ echo 'lalala' > testfile
    $ nix eval --expr '(builtins.fetchTree { path = "/home/ma27/testfile"; rev = "0000000000000000000000000000000000000000"; type = "path"; })'
    { lastModified = 1723656303; lastModifiedDate = "20240814172503"; narHash = "sha256-hOMY06A0ohaaCLwnhpZIMoAqi/8kG2vk30NRiqi0dfc="; outPath = "/nix/store/lhfz259iipmv9ky995rml8018jvriynh-source"; rev = "0000000000000000000000000000000000000000"; shortRev = "0000000"; }
    $ cat /nix/store/lhfz259iipmv9ky995rml8018jvriynh-source
    lalala

There was a fix in CppNix[1], but Puck noted that it's breaking
backwards-compatibility because

> [...] a github fetch with a fully specified rev is no longer considered "locked"
> this is for "purity" reasons, but it breaks any existing flake.nix files

I tried a way smaller correctness fix here:

* Each scheme can denote whether a `rev` is enough to consider itself
  locked.
* If a `rev` is given and the scheme is OK with just a `rev` to be
  locked, the input is marked as locked.

For `path` this is not the case anymore, i.e. it requires a NAR hash to
be locked down.

[1] https://github.com/nixos/nix/commit/071dd2b3a4e6c0b2106f1b6f14ec26e153d97446

Change-Id: Ibbbf4733c82bcfa1c24dfe099a896d8aaecd81cc
2025-03-20 13:17:34 +00:00
jadeandLix Systems Gerrit 430bcc482c Merge "release-notes: add some for gc gremlins" into main 2025-03-18 20:02:25 +00:00
jadeandLix Systems Gerrit be4ab653eb Merge "fetchTree: make invisible when not available, sorta document" into main 2025-03-18 19:32:12 +00:00
Jade LovelaceandJade Lovelace fcea7379d3 fetchTree: make invisible when not available, sorta document
Documenting fetchTree is an exercise in frustration because of the sheer
amount of stringly typed everything in it. I do not know which fields
exist without reading the entirety of libfetchers. However, we can write
something slightly perfunctory but at least perhaps somewhat helpful
documentation-wise.

Fixes: https://git.lix.systems/lix-project/lix/issues/609
Change-Id: I991391b53fcd69172dbc7efb9d384e62bc847b91
2025-03-18 11:34:28 -07:00
Jade Lovelace 24e413c147 release-notes: add some for gc gremlins
Originally was in cl/2687 but that CL got canned.

Change-Id: Ib504e263d72669f2418ef02d624bcba26b169622
2025-03-18 10:50:23 -07:00
Mario RodasandLix Systems Gerrit 42541e030c Merge "Fix broken links in documentation" into main 2025-03-18 17:33:31 +00:00
Mario RodasandLix Systems Gerrit dc69d64bff Merge "Fix typos in documentation" into main 2025-03-18 17:32:53 +00:00
jadeandLix Systems Gerrit 6e6ff58272 Merge "feat!: remove experimental repl-flakes" into main 2025-03-18 16:02:55 +00:00
Mario Rodas 1da97ab24e Fix broken links in documentation
Change-Id: I582341a001d9541db82794eb64ddfc32c6582b83
2025-03-18 04:20:00 +00:00
Mario Rodas 992866bc91 Fix typos in documentation
Change-Id: I93c3801290d03244ea8db96679d002589ae90b74
2025-03-18 04:20:00 +00:00
jadeandLix Systems Gerrit ea4cf48879 Merge "post-build-hook: don't emit config settings that aren't overridden" into main 2025-03-17 23:31:32 +00:00
jadeandLix Systems Gerrit 15c6679daf Merge "release-notes: note the flake.lock fix" into main 2025-03-17 23:02:44 +00:00
KFearsandJade Lovelace 2c21882767 feat!: remove experimental repl-flakes
This is a backport of CppNix PR 10299

Fixes #557

Change-Id: I848e8c5af4bde09afba053a5911fbbd54004fbb8
2025-03-17 16:01:53 -07:00
Jade Lovelace 523965697d post-build-hook: don't emit config settings that aren't overridden
This is a redesign from 9b1f3cbc13 where
this was introduced.

I deleted the AbstractConfig::toKeyValue since it was conspicuously and
obviously broken for two years since 450e5ec618.
I asked myself if anyone was using it, given that it only emitted
settings that were aliases (broken!), and found that nobody used it.

The motivation for this change is the same for only emitting overridden
settings to the protocol: the nix inside there may not be able to parse
our defaults, as is the case of CppNix since the consensual
accept-flake-config was added to Lix.

Fixes: https://git.lix.systems/lix-project/lix/issues/739
Change-Id: Ib9874a52137f1f22220c25bcfa2425a4802509c7
2025-03-17 15:25:52 -07:00
jadeandLix Systems Gerrit 8c5a1d54d9 Merge "logging: always print post build hook logs" into main 2025-03-17 21:30:32 +00:00
Jade Lovelace d857424488 release-notes: note the flake.lock fix
Change-Id: I1c5f7a5cba12da1bfb2896ffc04ec89862cb346d
2025-03-17 12:07:29 -07:00
jadeandLix Systems Gerrit 3cfc8be580 Merge changes I6a83bdb3,If4ee20c3 into main
* changes:
  README: link to the manual
  plugins: support nix_plugin_entry, do some minor reworks
2025-03-17 18:13:31 +00:00
Jade Lovelace c597589d70 logging: always print post build hook logs
Missing these is really annoying, causes massive problems for debugging
(if you don't to enable build logs), and there's basically no reason we
can't just always show them.

In the future we might want to emit these to syslog/systemd/etc, but for
now let's just always print them to the console.

CC: https://git.lix.systems/lix-project/lix/issues/675

Change-Id: Ib6c47f3e34e553505298ed49c8bdb43f3db317a4
2025-03-17 11:12:25 -07:00
rebecca “wiggles” turnerandLix Systems Gerrit 366f55303a Merge "justfile: enable just build install test" into main 2025-03-17 17:48:55 +00:00
Jade Lovelace adf45deeca plugins: support nix_plugin_entry, do some minor reworks
Plugins are already allowed to fail to load, let's be even more tolerant
of them failing to load by ignoring inaccessible ones with a warning.

This change also significantly reworks the docs for plugins.

This is a tiny extremely partial backport of the C API at
https://github.com/NixOS/nix/pull/8699.

Fixes: https://git.lix.systems/lix-project/lix/issues/740
CC: https://git.lix.systems/lix-project/lix/issues/359

Change-Id: If4ee20c3daaf26c8184a415eef3e20ca5b5e7aef
2025-03-16 16:43:54 -07:00
Maximilian BoschandLix Systems Gerrit a3796499ba Merge "libstore: set connect-timeout to 5" into main 2025-03-16 18:09:30 +00:00
Rebecca Turner e7463af241 justfile: enable just build install test
Previously, only the `install` target in the `justfile` took extra
`*OPTIONS`, which meant you could run `just build install test` (or
`just clean setup build install test` for a clean build). This is much
more convenient than `just build && just install && just test`.

However, sometimes you *do* need extra options for some of those other
targets, so over time they have gained extra arguments. But these
prevent you from chaining the targets together:

    $ just clean setup build install test
    rm -rf build
    meson setup build --prefix="$PWD/outputs/out" $mesonFlags build install test
    usage: meson [-h]
                 {setup,configure,dist,install,introspect,init,test,wrap,subprojects,rewrite,compile,devenv,env2mfile,reprotest,format,fmt,help}
                 ...
    meson: error: unrecognized arguments: install test
    error: Recipe `setup` failed on line 13 with exit code 2

As a compromise, I've renamed the targets with extra arguments to
include a `-custom` suffix, and added aliases for the old targets to
call the `-custom`-suffixed target with no extra arguments.

This makes it possible to run `just build install test` again, but keeps
the ability to run `just build-custom EXTRA_MESON_BUILD_ARGS`.

BONUS:
- Added `test-unit` and `test-integration`, because I always forget the
  arguments to run a particular test suite and the names of those test
  suites.
- Added doc comments to `lint` and `lint-fix`.

Change-Id: I61ec66f5e4d38c12bbae4fa226d7b4cea94579d1
2025-03-15 21:40:56 -07:00
Maximilian Bosch 7359c39076 libstore: set connect-timeout to 5
Right now, Lix waits for quite a while five times[1] if a substituter is
down. If the substituter is not reachable for that long, it's probably
down or so slow that it doesn't make sense to download from it. Also,
most people would abort earlier assuming that Lix just behaves weird in
that case.

Reducing the timeout to 5s to make the behavior a little better.

If people live in areas with poor connection where a higher timeout
would be appropriate, I'd argue that they should increase the timeout
rather than requiring everbody else to set it to a non-zero value.

Additionally, communicate how often we're re-trying.

[1] Because of `download-attempts` from `nix.conf(5)`

Change-Id: I8500dce0c8230247dd492e43cc7af4d3b58c4710
2025-03-15 16:17:25 +01:00
pamplemousseandGerrit Code Review af15a446ea Merge "Allow diff-closures to output JSON" into main 2025-03-14 13:11:11 +00:00
Julian Stecklina 0e59e5b308 libfetchers: ensure that lastModified is a uint64_t
When lastModified comes via inputFromAttrs it ends up as string in the
Attrs map. This results in:

error: input attribute 'lastModified' is not an integer

Fix by handling it like revCount, which already does the right thing.
If added a test and confirmed that it catches the issue.

Also kudos to alexander.sieg@cyberus-technology.de for helping with
debugging this!

Change-Id: I8378fcaea986d798cb8458d4e6e15c2a92c2520a
2025-03-13 09:34:59 +01:00
Pamplemousse 04cc3db4df Allow diff-closures to output JSON
Change-Id: Ia17ea93d98b38e3415c35257daf07c7978f50ef4
2025-03-12 08:03:49 +01:00
ZitroneandGerrit Code Review 85a140accb Merge "libexpr: show all missing and unexpected arguments instead of just one" into main 2025-03-11 21:54:32 +00:00