Compare commits

..
Author SHA1 Message Date
Raito Bezarius e3ac0d9c19 release: merge release 2.94.2 back to mainline
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh

Change-Id: I01c4b3caeaf2c180d69b406b0dbd663ed1e4392d
2026-05-04 19:02:23 +02:00
Raito Bezarius c8a447db88 release: 2.94.2 "Açaí na tigela"
Release produced with releng/create_release.xsh

Change-Id: I7cfa601006515be2710bc01111c6adc34ee5c048
2026-05-04 19:02:22 +02:00
Raito Bezarius ec912eda76 release: release notes for 2.94.2
Release created with releng/create_release.xsh

Change-Id: I117f10a79a674d262446555206e4976bb330f491
2026-05-04 19:02:20 +02:00
eldritch horrorsandRaito Bezarius 667b40f1ff libutil: fix nar parser buffer overflow
string data shares a buffer with the binary string length field. size
calculations for string read buffers always include the length field;
sufficiently large length fields can cause these calculations to wrap.
a malicious nar could use this for OOB writes in the daemon (as root).

since we use strings only as tags for archive members and for symlinks
with their OS-dependent length limits we can simply limit string size.
1 MiB should be sufficient for all symlinks, and tags are always tiny.

Change-Id: I89fb05f73c1dbeda45d91244aba4cd526a3d83e1
2026-05-04 19:01:44 +02:00
Raito Bezarius 64599c010c releng/keys: update the way to receive the ephemeral key
I don't understand how `ssh -l lix-releng` is supposed to work if it
doesn't say which host to target.

Change-Id: I791f3f3f49ecd5884c9e86b5d3b617fc139e031f
Signed-off-by: Raito Bezarius <raito@lix.systems>
2026-05-04 19:01:44 +02:00
Raito Bezarius 1953c4ae4b releng/environment: update staging parameters
These parameters are now created on https://s3.afnix.fr.

Change-Id: I96b6fd913429ee46d04c412cb141edd288665ced
Signed-off-by: Raito Bezarius <raito@lix.systems>
2026-05-04 19:01:44 +02:00
YurekaandRaito Bezarius 7e56afa0b1 releng: Adapt for AFNix S3
Change-Id: I29dbd62dcc70595ba3f2ac2a466a5c26a28aea99
(cherry picked from commit 0c63036c7d)
2026-05-04 19:01:44 +02:00
Florian KlinkandRaito Bezarius 5d6134064e libcstore: Fix null deref in writeDebugInfo for non-directory NARs
When index-debug-info is enabled and the store path being copied is a
regular file (not a directory), std::get_if<nar_index::Directory>
returns nullptr since the NAR root is a File variant. The loop then
immediately dereferences buildIdDir->contents on the null pointer,
causing a segfault.

Add a null check at the top of the loop to break early when the NAR
root is not a directory.

Change-Id: I3a6e792b84cc12c837ecaddf4fee889e1bcb6397
(cherry picked from commit 6c7ccc2588)
2026-05-04 16:33:04 +00:00
sterni 1d36c544be libcmd: add support for lowdown >= 3.0.0
lowdown 3.0.0 merged some flags into one to save on bits and did not add
any aliases for backward compatibility.

As with the changes for lowdown >= 1.4, we define a preprocessor flag to
gate the changes on and add a job to CI to ensure that lowdown < 3.0
keeps working (which is used by NixOS 25.11).

The channel version this Lix branch uses doesn't have bmake with support
for all tested platforms, so we can't properly build lowdown 3.0 (which
depends on it) for tests in CI. Given this is a relatively simple
backport tested on other Lix branches, it's probably fine.

Change-Id: I20a3e2fdaa05906f032ff66911c42867557fdd11
(cherry picked from commit e839708839aa132c8694abfdf83409a619f72c5a)
2026-04-10 16:04:52 +02:00
Jade Lovelace 7b210490aa version.json: 2.94.2, if we do backport more stuff
Change-Id: Ide1ce2a59a80a524d62c77681c2ea45e859a77e2
2026-03-13 12:17:56 -07:00
Jade Lovelace fba0fa5767 release: merge release 2.94.1 back to mainline
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh

Change-Id: I2c10392b0b80bff519501c8018ed51aa6a151ca1
2026-03-13 09:39:07 -07:00
Jade Lovelace 533429d89a release: 2.94.1 "Açaí na tigela"
Release produced with releng/create_release.xsh

Change-Id: Ie38032fe8dfebed83f9d24762ce48f73b23149b5
2026-03-13 09:39:06 -07:00
Jade Lovelace c135090468 release: release notes for 2.94.1
Release created with releng/create_release.xsh

Change-Id: I970e8f5422e3705917b04c09d28498fd7b849d27
2026-03-13 09:38:52 -07:00
Jade Lovelace 72f7965679 2.94.1: version
Change-Id: I92eeeebfd6b966ce77833785482db989962f8c9e
2026-03-13 09:38:44 -07:00
Raito Bezarius b7be40c785 libstore/build: fix starvation during substitution
When the destructor of PathSubstitutionGoal is run, this happens in a
sync context and can cause starvation of all ongoing IO w.r.t. to other
substitutions, including our own substitution.

While there's only a decompressor thread per stream, the other side of
the IO runs on the event loop.

In order to fix this, it is sufficient to remove the thread indirection
and inline the async code.

Fixes #1126. Great thanks to horrors' patience.

Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I3eb37bc37d156f0f5528364e568fdaa2ced58011
Signed-off-by: Raito Bezarius <raito@lix.systems>
(cherry picked from commit 505d0669dc)
2026-02-11 15:39:06 +00:00
Qyriad 6350f51458 libutil: include LIX_MAJOR, LIX_MINOR, and LIX_PATCH macros
Backport of I7d8a4648890fce7ff15695876c9b9d3a6a6a6964 to 2.94 branch.

Change-Id: Ia6b8d89007974c8cf873fa1f536ce8416a6a6964
2026-01-15 11:48:26 +00:00
eldritch horrorsandRaito Bezarius c6f3f3a0d3 libexpr: fix app chain extension
during the value rewrite we accidentally broke extension of incomplete
primop application. this only shows up when binding on incomplete call
to a primop to a name, binding an incomplete call to *that* to another
name, and then finally calling the second binding with enough args for
a complete primop application. since this only shows up when calling a
primop with three or more args it took a while to surface. we have few
builtins that match this: foldl', replaceStrings, and substring. these
are not used incompletely in this manner very often, so this lingered.

fixes #1102

Change-Id: I218dffc14ae876efc86a86c7eb6c895e2405201c
2026-01-14 22:09:56 +00:00
Maximilian Bosch de4cfec46a libstore: fix reporting output cycles on drvs with references to other drvs
Closes #1064

The culprit here is that `genGraphString` is only invoked with the
store-paths associated with the outputs of the derivation, so when
filling `dependents`, the `graph_data.find(p)` call would return the end
of the iterator when doing this for references to other store-paths.

As a result, the code wrote information behind the graph data-structure
causing a corruption. For me, this resulted in a SIGSEGV most of the
time and in a few cases in an uncaught `map::at`-exception as reported
by Niklas.

This patch changes two aspects of the original implementation:

* When filling `dependents` in the node-set, use `map.at()` instead of
  `map.find()->second`. The latter doesn't make any sense and was the
  cause of corrupting memory. The `at` would've made it far easier to
  spot this in the first place.

* Filter out store-paths that don't belong to a different output of the
  derivation when creating `outputGraph`. This variable is used on two
  places, `genGraphString` and for topological sorting.

  The latter already filters out store-paths from a different drv, so
  this is happening now when creating the variable in the first place
  such that `genGraphString` never ends up with corrupt data in the
  first place. This is the actual bugfix.

Implemented a regression-test for this case to be sure.

Change-Id: Ie02144d89c32b0a776cb1ece0601d0229315ebc3
(cherry picked from commit 0a5f474a25)
2025-12-08 07:20:51 +01:00
Raito Bezarius 0873bed39d nix/upgrade-nix: disallow daemon connections for the store
Prior to I6a6a6964d2b5ad47ae5ea9eb11af9b6373ce2141 — `sudo nix
upgrade-nix` would perform direct store access.

This ensured a certain number of desireable properties for upgrading the
Lix binary itself.

We re-introduce direct store access for upgrading Lix binaries.

Fixes #1060.

Change-Id: I523c4d3023ed5fe9eff8fde9a266c56a0de47d8c
Signed-off-by: Raito Bezarius <raito@lix.systems>
(cherry picked from commit d2ca1810b1)
2025-12-06 22:20:18 +00:00
Zoe ZuserandRaito Bezarius 5dcb90548f meson: fix libstore.pc
typo of aws-cpp-sdk-transfer as aws-cpp-std-transfer prevents linking
against lix

Change-Id: Id847eab2601698696030d31fcd51288aa5f3d274
(cherry picked from commit 06f987fb0c)
2025-12-05 16:58:28 +00:00
529 changed files with 2940 additions and 4112 deletions
+3 -3
View File
@@ -24,8 +24,8 @@ def map_contents_recursively(transformer):
def process_command:
.[0] as $context |
.[1] as $body |
# XXX FUTURE: drop sections once mdBook is at 0.5.0 or above in nixpkgs
$body | (.items? // .sections) |= map(map_contents_recursively(if $context.renderer == "html" then transform_anchors_html else transform_anchors_strip end))
;
$body + {
sections: $body.sections | map(map_contents_recursively(if $context.renderer == "html" then transform_anchors_html else transform_anchors_strip end)),
};
process_command
+5 -6
View File
@@ -22,21 +22,20 @@ fold.level = 30
# not want to disable the links preprocessor entirely though because that requires
# disabling *all* built-in preprocessors and selectively reenabling those we want.
[preprocessor.substitute]
command = "python3 substitute.py"
command = "python3 doc/manual/substitute.py"
before = ["anchors", "links"]
[preprocessor.anchors]
renderers = ["html"]
command = "jq --from-file anchors.jq"
command = "jq --from-file doc/manual/anchors.jq"
[output.markdown]
# XXX FUTURE: may be reenabled once mdBook 0.5.0 or above and matching mdbook-linkchecker are in nixpkgs
#[output.linkcheck]
[output.linkcheck]
# no Internet during the build (in the sandbox)
#follow-web-links = false
follow-web-links = false
# mdbook-linkcheck does not understand [foo]{#bar} style links, resulting in
# excessive "Potential incomplete link" warnings. No other kind of warning was
# produced at the time of writing.
#warning-policy = "ignore"
warning-policy = "ignore"
+6 -15
View File
@@ -61,11 +61,6 @@ cole-h:
display_name: Cole Helbling
github: cole-h
commentator2.0:
display_name: Commentator2.0 (Rutile)
forgejo: commentatorforall
github: CommentatorForAll
delan:
display_name: delan
forgejo: delan
@@ -78,6 +73,9 @@ detroyejr:
display_name: Jonathan De Troye
github: detroyejr
edef:
github: edef1c
edolstra:
display_name: Eelco Dolstra
github: edolstra
@@ -130,11 +128,6 @@ jade:
just1602:
forgejo: just1602
k900:
display_name: K900
forgejo: K900
github: K900
kasimeka:
display_name: ورد
forgejo: janw4ld
@@ -186,11 +179,6 @@ midnightveil:
forgejo: midnightveil
github: midnightveil
milibopp:
display_name: Emilia Bopp
forgejo: milibopp
github: milibopp
nan-git:
display_name: NaN-git
github: NaN-git
@@ -244,6 +232,9 @@ roberth:
display_name: Robert Hensing
github: roberth
sandydoo:
github: sandydoo
seppel3210:
github: Seppel3210
+2 -4
View File
@@ -42,9 +42,8 @@ manual = custom_target(
'-c',
'''
@0@ @INPUT0@ @CURRENT_SOURCE_DIR@ > @DEPFILE@
cd @3@
@1@ build . -d @2@ | { grep -Fv "because fragment resolution isn't implemented" || :; }
cd @SOURCE_ROOT@
@1@ build doc/manual -d @2@ | { grep -Fv "because fragment resolution isn't implemented" || :; }
rm -rf @2@/manual
mv @2@/html @2@/manual
find @2@/manual -iname meson.build -delete
@@ -52,7 +51,6 @@ manual = custom_target(
python.full_path(),
mdbook.full_path(),
meson.current_build_dir(),
meson.current_source_dir()
),
],
input : [
@@ -83,7 +81,7 @@ manual = custom_target(
depfile : 'manual.d',
env : {
'RUST_LOG': 'info',
'MANUAL_SUBSTITUTE_SEARCH': meson.current_build_dir() / 'src',
'MDBOOK_SUBSTITUTE_SEARCH': meson.current_build_dir() / 'src',
},
)
manual_md = manual[1]
-15
View File
@@ -1,15 +0,0 @@
---
synopsis: "Add `builtins.warn` for emitting warnings from Nix code"
cls: [2248]
category: "Features"
credits: [milibopp, Qyriad]
---
Lix now has a builtin function for emitting warnings.
Like `builtins.trace`, it takes two arguments: the message to emit, and the expression to return.
_Unlike_ `builtins.trace`, `builtins.warn` requires the first argument — the message — to be a string.
In the future we may extend `builtins.warn` to accept a more structured API.
To go along with this, we also have two new config settings:
- [`debugger-on-warn`](@docroot@/command-ref/conf-file.md#conf-debugger-on-warn), which, when used with `--debugger`, makes `builtins.warn` also function like [`builtins.break`](@docroot@/language/builtins.md#builtins-break).
- [`abort-on-warn`](@docroot@/command-ref/conf-file.md#conf-abort-on-warn), which aborts evaluation entirely after the warning is emitted.
@@ -1,13 +0,0 @@
---
synopsis: "Deprecate shadowing internal files through the Nix search path"
issues: [998]
cls: [4632]
category: "Breaking Changes"
credits: [thubrecht]
---
As Lix uses the path `<nix/fetchurl.nix>` for bootstrapping purposes, the ability to shadow it by adding `nix=/some/path` (or `/other/path` that contains a `nix` directory) to the search path is not desirable.
To alleviate potential issues, Lix now emits a warning when the Nix search path contains potential shadows for internal files, which will be changed to an error in a future release.
The warning can be disabled by enabling the deprecated feature `nix-path-shadow`.
@@ -1,10 +0,0 @@
---
synopsis: "Warn instead of erroring when the final destination of a transfer changes in-flight"
cls: [4641]
issues: [fj#1004]
category: "Miscellany"
credits: [thubrecht]
---
Lix will now emit a warning during downloads where the final destination changes suddently mid-transfer instead of throwing an error.
This transfer behavior has been known to happen very rarely while fetching from some CDNs.
@@ -1,10 +0,0 @@
---
synopsis: 'functional lang migration'
issues: [lix#856]
cls: [3213, 3214, 3215, 3224, 4092, 4093, 4094, 4095, 4096, 4097, 4098, 4099, 4100, 4101, 4102, 4103, 4104, 4105, 4106, 4107, 4108, 4109, 4110, 4111, 4112, 4113, 4114, 4115, 4116, 4117, 4122, 4123, 4269, 4270, 4271, 4272, 4273, 4274, 4347, 4348, 4349, 4350, 4351, 4352, 4569, 4570, 4571, 4572, 4573, 4574, 4596, 4597, 4598, 4599, 4600, 4601, 4602, 4603, 4604, 4605]
category: Development
credits: [piegames, commentator2.0]
---
We have done it! The functional/lang framework has now been fully migrated to functional2/lang.
This means: no more `just clean` and `just install` mess and whatever because one removed a test.
The lang test suite is also getting a face lift, with an improved folder structure and restructuring of many tests.
-10
View File
@@ -1,10 +0,0 @@
---
synopsis: "Allow remote builders to be configured using TOML"
cls: [4533]
category: "Features"
credits: [commentator2.0, Qyriad]
---
Lix now supports configuring remote builders using a TOML file instead of the old, very cursed and incomprehensible format.
This comes with not only a human-understandable file, but also with better messages and error reports on misconfiguration.
A more detailed Documentation can be found on the [distributed-builds](@docroot@/advanced-topics/distributed-builds.md) Wiki-page
-8
View File
@@ -1,8 +0,0 @@
---
synopsis: "Default to showing build logs in the new-style (nix3) CLI"
cls: [4674]
category: "Miscellany"
credits: [k900]
---
Lix will now show logs by default, in addition to the progress bar, when invoked through the new-style "nix3" CLI (`nix build`, etc)
@@ -1,31 +0,0 @@
---
synopsis: "Move /root/.cache/nix to /var/cache/nix by default"
cls: [4671]
issues: [fj#634]
category: "Breaking Changes"
credits: [raito]
---
By default, Lix attempts to locate a cache directory for its operations (such
as the narinfo cache) by checking the value of `$XDG_CACHE_DIR`.
However, since the Nix daemon is a system service, using `$XDG_CACHE_DIR` is
not typical in this context.
To address this, systemd provides a better solution. Specifically, when
`CacheDirectory=` is set in the `[Service]` section of a systemd unit, it
automatically sets the `$CACHE_DIRECTORY` environment variable and systemd will
manage that cache directory for us.
Now, our systemd unit includes `CacheDirectory=nix`, which sets the
`$CACHE_DIRECTORY` and takes precedence over `$XDG_CACHE_DIR`.
If the daemon is run under user units, systemd will automatically set
`$XDG_CACHE_DIR`.
If neither of these variables is set, Lix falls back to its default behavior.
By default, Lix will try to find a cache directory for its various operations
(e.g. narinfo cache) by looking into `$XDG_CACHE_DIR`.
In summary, what was stored in `/root/.cache/nix` is now moved to
`/var/cache/nix/nix`.
-9
View File
@@ -1,9 +0,0 @@
---
synopsis: "Add an indication of nix-shell nesting depth"
cls: [4657]
issues: [fj#826]
category: "Improvements"
credits: [thubrecht]
---
When in a nix shell (either via a `nix-shell` or a `nix develop` invocation), a variable `NIX_SHELL_LEVEL` is exported to indicate the nesting depth of nix shells.
-20
View File
@@ -1,20 +0,0 @@
---
synopsis: Derivations can now be printed in detail in `nix repl`
cls: [3842]
category: Improvements
credits: [Lunaphied]
---
Traditionally derivations printed in the REPL would only print a formatted object
representing the path of the derivation file it refers to. This makes inspecting
the enhanced derivation attribute sets encountered from `mkDerivation` or similar
wrappers more difficult. Even the `:p`/`:print` command would not elaborate attribute sets
tagged as a derivation.
With this change you can now use `:p`/`:print` to directly inspect a derivation
by providing one as the top-level object. Derivation attribute sets will only be
printed two levels deep and internal derivation attrsets will remain in unexpanded
path form as before. `drvAttrs` will also be elided as these attributes are already
present in the top-level attribute set of the derivation. These heuristics provide
a balance between readability and functionality. When the `:p`/`:print` is omitted,
a bare derivation is printed in the path format as before.
@@ -1,37 +0,0 @@
---
synopsis: Remove `fetch-closure` experimental feature
issues: [fj#1010]
cls: [4595]
category: "Breaking Changes"
credits: [just1602]
---
The `fetch-closure` experimental feature has been removed.
Outside of allowing the user to import closure from binary cache,
`fetchClosure` also allow you to do the following:
* rewrite non-CA path to CA
* reject non-CA paths at fetching time
* reject CA paths at fetching time
Some people are using those mechanism to prevent users from having to build any
package and force going via the declared cache or as a way to use ancient/old
software without paying the evaluation cost of a second nixpkgs.
Both use cases are somewhat of an antipattern in Nix semantics. If the user
cannot fetch a program directly via the substituter mechanism and fall back to
local build, this is a feature AND a misconfiguration. If the user cannot build
certain derivations because they are too expensive, the build directives should
pass `-j0` or similar.
As for the second usecase, there's a different way to do it that also allows to
have a way to reproduce the paths that are hardcoded in that file, perform
`import (fetchurl "https://my-cache/${hashparts storepath}.drv")` rather, i.e.
an IFD to a possibly well known name. The backend can generate them on the fly
or once, and possess stable names.
Finally, as for the non-CA → CA features, Lix removed ca-derivations.
fetchClosure offers ca-derivations-like features which suffers from similar
shortcomings albeit lessened. It only follows that we should rather deprecate
and remove these capabilities.
-15
View File
@@ -1,15 +0,0 @@
---
synopsis: "Lix daemons are now fully socket-activated on systemd setups"
cls: []
issues: [1030]
category: "Miscellany"
credits: [horrors]
---
When launched by systemd, Lix no longer uses a persistent daemon process and uses systemd socket
activation instead. This is necessary to support the `cgroups` and `auto-allocate-uids` features
and may improve observability of daemon behavior with common systemd-based monitoring solutions.
The old behavior with a single persistent daemon is still available, but disabled by default. It
is not possible to enable both a persistent daemon and socket activation, starting one stops the
other automatically. Existing installations should not require any changes when they're updated.
@@ -41,17 +41,106 @@ contains Nix.
> If you are building via the Lix daemon (default on Linux and macOS), it is the Lix daemon user account (that is, `root`) that should have SSH access to a user (not necessarily `root`) on the remote machine.
>
> Furthermore, `root` needs to have the public host keys for the remote system in its `.ssh/known_hosts`.
> To add them to `known_hosts` for root, do `ssh-keyscan HOST | sudo tee -a ~root/.ssh/known_hosts`.
> To add them to `known_hosts` for root, do `ssh-keyscan USER@HOST | sudo tee -a ~root/.ssh/known_hosts`.
>
> If you cant or dont want to configure `root` to be able to access the remote machine, you can use a private Nix store instead by passing e.g. `--store ~/my-nix` when running a Nix command from the local machine.
## Configuration
The list of remote machines can be specified on the command line or in
the Lix configuration file. The former is convenient for testing.
Additionally, there are two supported formats to configure remote builders:
The legacy, "space"-separated format and starting with Lix 2.95.0, a TOML.
the Lix configuration file. The former is convenient for testing. For
example, the following command allows you to build a derivation for
`x86_64-darwin` on a Linux machine:
```console
$ uname
Linux
$ nix build --impure \
--expr '(with import <nixpkgs> { system = "x86_64-darwin"; }; runCommand "foo" {} "uname > $out")' \
--builders 'ssh://mac x86_64-darwin'
[1/0/1 built, 0.0 MiB DL] building foo on ssh://mac
$ cat ./result
Darwin
```
It is possible to specify multiple builders separated by a semicolon or
a newline, e.g.
```console
--builders 'ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd'
```
Each machine specification consists of the following elements, separated
by spaces. Only the first element is required. To leave a field at its
default, set it to `-`.
1. The URI of the remote store in the format
`ssh://[username@]hostname[?port=<port>]`, e.g. `ssh://nix@mac` or `ssh://mac`.
If the ssh server is not listening on port 22 (e.g. port 1337 in this case)
the URI would be `ssh://nix@mac?port=1337`
For backward compatibility, `ssh://` may be omitted. The hostname
may be an alias defined in your `~/.ssh/config`.
2. A comma-separated list of Nix platform type identifiers, such as
`x86_64-darwin`. It is possible for a machine to support multiple
platform types, e.g., `i686-linux,x86_64-linux`. If omitted, this
defaults to the local platform type.
3. The SSH identity file to be used to log in to the remote machine. If
omitted, SSH will use its regular identities.
4. The maximum number of builds that Lix will execute in parallel on
the machine. Typically this should be equal to the number of CPU
cores. For instance, the machine `itchy` in the example will execute
up to 8 builds in parallel.
5. The “speed factor”, indicating the relative speed of the machine. If
there are multiple machines of the right type, Lix will prefer the
fastest, taking load into account.
6. A comma-separated list of *supported features*. If a derivation has
the `requiredSystemFeatures` attribute, then Lix will only perform
the derivation on a machine that has the specified features. For
instance, the attribute
```nix
requiredSystemFeatures = [ "kvm" ];
```
will cause the build to be performed on a machine that has the `kvm`
feature.
7. A comma-separated list of *mandatory features*. A machine will only
be used to build a derivation if all of the machines mandatory
features appear in the derivations `requiredSystemFeatures`
attribute.
8. The (base64-encoded) public host key of the remote machine. If omitted, SSH
will use its regular known-hosts file. Specifically, the field is calculated
via `base64 -w0 /etc/ssh/ssh_host_ed25519_key.pub`.
For example, the machine specification
nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm
nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2
nix@poochie.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 1 2 kvm benchmark
specifies several machines that can perform `i686-linux` builds.
However, `poochie` will only do builds that have the attribute
```nix
requiredSystemFeatures = [ "benchmark" ];
```
or
```nix
requiredSystemFeatures = [ "benchmark" "kvm" ];
```
`itchy` cannot do builds that require `kvm`, but `scratchy` does support
such builds. For regular builds, `itchy` will be preferred over
`scratchy` because it has a higher speed factor.
Remote builders can also be configured in `nix.conf`, e.g.
@@ -70,180 +159,3 @@ option `builders-use-substitutes` in your local `nix.conf`.
To build only on remote builders and disable building on the local
machine, you can use the option `--max-jobs 0`.
---
Each machine specification consists of the following attributes.
How those are combined within the configuration file differs for the formats, and will be explained further down.
1. `uri` (**required**)
The URI of the remote store in the format
`ssh[-ng]://[username@]hostname[?port=<port>]`, e.g. `ssh://nix@mac` or `ssh://mac`.
If the ssh server is not listening on port 22 (e.g. port 1337 in this case)
the URI would be `ssh[-ng]://nix@mac?port=1337`. The hostname
may be an alias defined in your `~/.ssh/config`.
2. `system-types` (**optional**)
A list of Nix platform type identifiers, such as
`x86_64-darwin`. It is possible for a machine to support multiple
platform types, e.g., `i686-linux` and `x86_64-linux`.
Defaults to the local platform type
3. `ssh-key` (**optional**)
The SSH identity file to be used to log in to the remote machine.
Defaults to SSHs regular identities.
4. `jobs` (**optional**)
The maximum number of builds that Lix will execute in parallel on
the machine. Typically, this should be equal to the number of CPU
cores divided by the cores within the target machines configuration, i.e. `jobs * cores ~= cpu cores`
Defaults to 1; must be a positive integer.
5. `speed-factor`
The “speed factor”, indicating the relative speed of the machine. If
there are multiple machines of the right type, Lix will prefer the
fastest, taking load into account.
Defaults to 1; must be a positive float.
6. `supported-features` (**optional**)
A list of *supported features*. If a derivation has
the `requiredSystemFeatures` attribute, then Lix will only schedule
the derivation on a machine that has the specified features. For
example, the attribute
```nix
requiredSystemFeatures = [ "kvm" ];
```
will cause the build to be performed on a machine that has the `kvm`
feature.
Defaults to an empty list.
7. `mandatory-features` (**optional**)
A list of *mandatory features*. A machine will only
be used to build a derivation if all the machines mandatory
features appear in the derivations `requiredSystemFeatures`
attribute.
Defaults to an empty list.
8. `ssh-public-host-key` (**optional**)
The public host key of the remote machine.
Defaults to basic ssh behavior (checking contests of the known-hosts file)
### Using a TOML configuration
Each machine is configured as an attribute within the map called `machines`.
The attributes name is the machines name.
Attributes can be in any order.
For example:
```toml
version = 1
[machines.andesite]
uri = "ssh://lix@andesite.lix.systems" # toml also allows for comments
system-types = ["i686-linux"]
jobs = 8
speed-factor = 1.0
supported-features = ["kvm"]
ssh-key = "/home/deepslate/.ssh/id_ed25519"
[machines.diorite]
uri = "ssh://lix@diorite.lix.systems"
system-types = ["i686-linux"]
jobs = 8
speed-factor = 2.0
ssh-key = "/home/deepslate/.ssh/id_ed25519"
[machines.granite]
uri = "ssh://lix@granite.lix.systems"
system-types = ["i686-linux"]
jobs = 1
speed-factor = 2.0
supported-features = ["kvm", "benchmark"]
ssh-key = "/home/deepslate/.ssh/id_ed25519"
[machines.legacy]
uri = "ssh://nix@nix-15-11.nixos.org"
enable = false
```
> **Note**
>
> If the version tag is omitted (e.g. in the CLI), it defaults to the latest version.
> It is strongly recommended to always provide a version tag for configuration within files to avoid breakage.
For testing purposes, one can also define a builder ad hoc on the CLI as follows:
`--builders 'machines.andesite = {uri = "ssh://lix@andesite.lix.systems", jobs = 8}'`
#### Special handling of fields
- `enable` (**optional**)
If set to false, the declared machine will not be loaded.
This allows one to statically disable machines.
Defaults to true
### Using the legacy format
> **Warning**
>
> This format is frozen and new features / configuration options will not be backported to this format.
It is possible to specify multiple builders separated by a semicolon or
a newline, e.g.
```console
--builders 'ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd'
```
Every machine specification consists of the elements listed in the section above, seperated by any amount of spaces or tabs.
The Attributes need to be provided **in order** and without names.
To leave a field at its default, set it to `-`.
Lists are colon seperated, without additional spaces.
```
lix@andesite.lix.systems i686-linux /home/deepslate/.ssh/id_ed25519 8 1 kvm
lix@diorite.lix.systems i686-linux /home/deepslate/.ssh/id_ed25519 8 2
lix@granite.lix.systems i686-linux /home/deepslate/.ssh/id_ed25519 1 2 kvm benchmark
```
#### Special handling of fields
- `uri`: Due to backward compatibility, the `ssh://` may be omitted for the store-uri.
- `ssh-public-host-key`: The key must be provided encoded in base64. Specifically calculated via `base64 -w0 /etc/ssh/ssh_host_ed25519_key.pub`
### Format detection
At first, the given configuration is being parsed syntactically as a toml.
If parsing fails and the given configuration contains a `"` the error is presented to the user, as those characters are necessary for TOML, but disallowed for the legacy format.
Otherwise, parsing is retried using the legacy format.
If non-syntactic errors are detected within the toml, the exception will always be shown to the user directly.
## Builder selection
The configuration(s) above specify several machines that can perform `i686-linux` builds.
However, `granite` will only do builds that have the attribute
```nix
requiredSystemFeatures = [ "benchmark" ];
```
or
```nix
requiredSystemFeatures = [ "benchmark" "kvm" ];
```
`diorite` cannot do builds that require `kvm`, but `andesite` does support
such builds. For regular builds, `diorite` will be preferred over
`andesite` because it has a higher speed factor.
-15
View File
@@ -141,21 +141,6 @@ To inspect the canonical source of truth on what the state of the buildsystem co
$ meson introspect
```
#### LLD
The development shell on Linux uses LLD by default for faster link times.
This is set using `mesonFlags`, so to override it, you can simplify re-specify the linker to Meson:
```bash
$ just setup-custom -Dc_link_args=-fuse-ld=ld -Dcpp_link_args=-fuse-ld=ld
```
While using LLD, you may find it helpful to use ThinLTO for even further improvements to link times for incremental builds:
```bash
$ just setup-custom -Db_lto=true -Db_lto_mode=thin -Db_thinlto_cache=true
```
## Sending changes to Gerrit for review {#sending-to-gerrit}
We use Gerrit for all our code review in Lix.
+1
View File
@@ -401,6 +401,7 @@ I grepped `lix/` for `get[eE]nv\("` to find the mentions in Lix code.
**Expected value**: the path to an executable shell
- `PRINT_PATH` - Undocumented. Used by `nix-prefetch-url` as an alternative form of `--print-path`. Why???
- `_NIX_IN_TEST` - If present with any value, makes `fetchClosure` accept file URLs in addition to HTTP ones. Why is this not `_NIX_FORCE_HTTP`??
Not used anywhere else.
- `NIX_ALLOW_EVAL` - Used by eval-cache tests to block evaluation if set to `0`.
@@ -5,7 +5,7 @@
FIXME(Lix): This section does not document the most common modern practices in terms of avoiding channels, pinning, declarative software installation (see flakey-profile or home-manager or NixOS), or using flakes, etc.
It is, however, likely correct at a technical level.
For more information on modern practices, see the [resources](https://wiki.lix.systems/books/lix-users/page/nix-resources) page on the Lix site.
For more information on modern practices, see the [resources](https://lix.systems/resources) page on the Lix site.
</div>
+34
View File
@@ -1,4 +1,38 @@
# Lix 2.94 "Açaí na tigela" (2025-11-17)
# Lix 2.94.2 (2026-05-04)
## Fixes
- Fix unsigned overflow leading to out-of-band write in the NAR parser [cl/5553](https://gerrit.lix.systems/c/lix/+/5553)
The NAR parser contained an unsigned integer overflow that could be used by an
attacker to write arbitrary data to an unknown memory location and possibly
achieve code execution. A successful attack on the system-wide Lix daemon
could lead to privilege escalation to root. Any process that involves NAR
serialization could trigger this issue, including (but not limited to)
- local user interaction, whether the users are trusted or untrusted
- malicious substituters sending malformed NARs
- remote builders sending malformed build results
- remote daemons sending malformed inputs when requesting remote builds
Successful attacks using this bug require ASLR weakening of some sort, whether
by architecture constraints (e.g. on 32 bit systems, where little randomization
is possible) or system configuration (e.g. low ASLR entropy when loading
libraries), and millions of attempts. Local attacks can be mounted in less than
an hour. Remote builds typically require a fresh SSH connection for each build
and are thus less susceptible. Only one attempt can be made by substituters for
every build using substituters, they are thus not a likely vector for attacks.
At the time of writing, MITRE has not assigned this a CVE yet.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae), [Raito Bezarius](https://git.lix.systems/raito), [edef](https://github.com/edef1c), and [sandydoo](https://github.com/sandydoo) for this.
# Lix 2.94.1 (2026-03-13)
# Lix 2.94.0 (2025-11-17)
+2 -7
View File
@@ -70,15 +70,10 @@ def do_include(content: str, relative_md_path: Path, source_root: Path, search_p
def recursive_replace(data, book_root, search_path):
match data:
# XXX FUTURE: drop sections once mdBook is at 0.5.0 or above in nixpkgs
case {'sections': sections}:
return data | dict(
sections = [recursive_replace(section, book_root, search_path) for section in sections],
)
case {'items': items}:
return data | dict(
items = [recursive_replace(item, book_root, search_path) for item in items],
)
case {'Chapter': chapter}:
path_to_chapter = Path(chapter['path'])
chapter_content = chapter['content']
@@ -124,10 +119,10 @@ def main():
context, book = json.load(sys.stdin)
# book_root is the directory where book contents leave (ie, src/)
book_root = Path(context['root']) / context['config']['book'].get('src', 'src')
book_root = Path(context['root']) / context['config']['book']['src']
# includes pointing into @generated@ will look here
search_path = Path(os.environ['MANUAL_SUBSTITUTE_SEARCH'])
search_path = Path(os.environ['MDBOOK_SUBSTITUTE_SEARCH'])
# Find @var@ in all parts of our recursive book structure.
replaced_content = recursive_replace(book, book_root, search_path)
+6 -17
View File
@@ -222,13 +222,9 @@
busybox-sandbox-shell = final.busybox-sandbox-shell or final.default-busybox-sandbox-shell;
};
lix-clang-tidy = final.callPackage ./subprojects/lix-clang-tidy {
# FIXME: To be removed when switching to nixos-25.11-small
llvmPackages = final.llvmPackages_20;
};
lix-clang-tidy = final.callPackage ./subprojects/lix-clang-tidy { };
nix-eval-jobs = final.callPackage ./subprojects/nix-eval-jobs {
stdenv = currentStdenv;
srcDir = ./subprojects/nix-eval-jobs;
};
@@ -262,9 +258,7 @@
sha512 = "2a4d0rqh8gkw4ca3gkzddp0hjpmmw74cbks8k0inhh0vizmgbn188zdv6m1kgmr019b99g7insli8js3ci1ji7y4n5nk704bswf3z3i";
};
nativeBuildInputs = prevAttrs.nativeBuildInputs ++ [ final.buildPackages.bmake ];
postInstall = lib.replaceStrings [ "lowdown.so.1" ] [ "lowdown.so.2" ] (
prevAttrs.postInstall or ""
);
postInstall = lib.replaceStrings [ "lowdown.so.1" ] [ "lowdown.so.2" ] prevAttrs.postInstall;
});
capnproto = prev.capnproto.overrideAttrs (old: {
@@ -433,20 +427,17 @@
'';
# clang-tidy run against the Lix codebase using the Lix clang-tidy plugin
clang-tidy = forAllSystems (
system:
clang-tidy =
let
pkgs = nixpkgsFor.${system}.native;
nixpkgs = nixpkgsFor.x86_64-linux.native;
inherit (nixpkgs) pkgs;
in
pkgs.callPackage ./package.nix {
# Required since we don't support gcc stdenv
stdenv = pkgs.clangStdenv;
# FIXME: To be removed when switching to nixos-25.11-small
llvmPackages = pkgs.llvmPackages_20;
versionSuffix = "";
lintInsteadOfBuild = true;
}
);
};
# Make sure that nix-env still produces the exact same result
# on a particular version of Nixpkgs.
@@ -591,8 +582,6 @@
inherit stdenv versionSuffix;
busybox-sandbox-shell = pkgs.busybox-sandbox-shell or pkgs.default-busybox-sandbox;
internalApiDocs = false;
# Use LLD in the dev shell by default for faster link times.
useLld = stdenv.hostPlatform.isLinux;
};
pre-commit = self.hydraJobs.pre-commit.${pkgs.system} or { };
in
-9
View File
@@ -25,7 +25,6 @@
#include "lix/libutil/shlex.hh"
#include "nix-build.hh"
#include "lix/libstore/temporary-dir.hh"
#include "lix/libutil/strings.hh"
extern char * * environ __attribute__((weak)); // Man what even is this
@@ -426,14 +425,6 @@ static int main_nix_build(AsyncIoRoot & aio, std::string programName, Strings ar
env["__ETC_PROFILE_SOURCED"] = "1";
}
// Set NIX_SHELL_LEVEL
env["NIX_SHELL_LEVEL"] = std::to_string(
getEnvNonEmpty("NIX_SHELL_LEVEL")
.and_then([](std::string lvl) { return string2Int<size_t>(lvl); })
.value_or(0)
+ 1
);
// Don't use defaultTempDir() here! We want to preserve the user's TMPDIR for the shell
env["NIX_BUILD_TOP"] = env["TMPDIR"] = env["TEMPDIR"] = env["TMP"] = env["TEMP"] =
getEnvNonEmpty("TMPDIR").value_or(buildTopTmpDir);
+19 -8
View File
@@ -64,7 +64,7 @@ static int main_nix_collect_garbage(AsyncIoRoot & aio, std::string programName,
{
bool removeOld = false;
GCOptions options = {.action = GCOptions::gcDeleteDead};
GCOptions options;
LegacyArgs(aio, programName, [&](Strings::iterator & arg, const Strings::iterator & end) {
if (*arg == "--help")
@@ -75,13 +75,12 @@ static int main_nix_collect_garbage(AsyncIoRoot & aio, std::string programName,
else if (*arg == "--delete-older-than") {
removeOld = true;
deleteOlderThan = getArg(*arg, arg, end);
} else if (*arg == "--dry-run") {
options.action = GCOptions::gcReturnDead;
} else if (*arg == "--max-freed") {
options.maxFreed = std::max(getIntArg<int64_t>(*arg, arg, end, true), (int64_t) 0);
} else {
return false;
}
else if (*arg == "--dry-run") dryRun = true;
else if (*arg == "--max-freed")
options.maxFreed = std::max(getIntArg<int64_t>(*arg, arg, end, true), (int64_t) 0);
else
return false;
return true;
}).parseCmdline(argv);
@@ -93,12 +92,24 @@ static int main_nix_collect_garbage(AsyncIoRoot & aio, std::string programName,
}
// Run the actual garbage collector.
if (!dryRun) {
options.action = GCOptions::gcDeleteDead;
} else {
options.action = GCOptions::gcReturnDead;
}
auto store = aio.blockOn(openStore());
auto & gcStore = require<GcStore>(*store);
GCResults results;
PrintFreed freed(options.action, results);
PrintFreed freed(true, results);
aio.blockOn(gcStore.collectGarbage(options, results));
if (dryRun) {
// Only print results for dry run; when !dryRun, paths will be printed as they're deleted.
for (auto & i : results.paths) {
printInfo("%s", Uncolored(i));
}
}
return 0;
}
}
+6 -2
View File
@@ -711,8 +711,12 @@ static void opGC(std::shared_ptr<Store> store, AsyncIoRoot & aio, Strings opFlag
}
else {
PrintFreed freed(options.action, results);
PrintFreed freed(options.action == GCOptions::gcDeleteDead, results);
aio.blockOn(gcStore.collectGarbage(options, results));
if (options.action != GCOptions::gcDeleteDead)
for (auto & i : results.paths)
cout << i << std::endl;
}
}
@@ -745,7 +749,7 @@ opDelete(std::shared_ptr<Store> store, AsyncIoRoot & aio, Strings opFlags, Strin
auto & gcStore = require<GcStore>(*store);
GCResults results;
PrintFreed freed(options.action, results);
PrintFreed freed(true, results);
aio.blockOn(gcStore.collectGarbage(options, results));
}
+3 -9
View File
@@ -75,15 +75,6 @@ CopyCommand::CopyCommand()
});
}
void CopyCommand::run()
{
if (requireStore && srcUri.empty() && dstUri.empty()) {
throw UsageError("you must pass '--from' and/or '--to'");
}
StoreCommand::run();
}
ref<Store> CopyCommand::createStore(AsyncIoRoot & in)
{
return srcUri.empty() ? StoreCommand::createStore(in) : in.blockOn(openStore(srcUri));
@@ -91,6 +82,9 @@ ref<Store> CopyCommand::createStore(AsyncIoRoot & in)
ref<Store> CopyCommand::getDstStore()
{
if (srcUri.empty() && dstUri.empty())
throw UsageError("you must pass '--from' and/or '--to'");
return aio().blockOn(dstUri.empty() ? openStore() : openStore(dstUri));
}
+2 -2
View File
@@ -15,6 +15,8 @@ namespace nix {
extern std::string programPath;
extern char * * savedArgv;
class EvalState;
struct Pos;
class Store;
@@ -54,11 +56,9 @@ private:
struct CopyCommand : virtual StoreCommand
{
std::string srcUri, dstUri;
bool requireStore = true;
CopyCommand();
void run() override;
ref<Store> createStore(AsyncIoRoot & in) override;
ref<Store> getDstStore();
+3 -16
View File
@@ -9,27 +9,14 @@
namespace nix {
typedef std::function<int(AsyncIoRoot &, std::string, std::list<std::string>)> MainFunction;
struct LegacyCommandRegistry
{
typedef std::function<int(AsyncIoRoot &, std::string, std::list<std::string>)> MainFunction;
typedef std::function<
int(AsyncIoRoot &, std::string, std::list<std::string>, std::span<char *>)>
RawMainFunction;
using LegacyCommandMap = std::map<std::string, RawMainFunction>;
using LegacyCommandMap = std::map<std::string, MainFunction>;
static LegacyCommandMap * commands;
static void add(const std::string & name, MainFunction fun)
{
addWithRaw(
name,
[fun](AsyncIoRoot & aio, std::string name, std::list<std::string> args, std::span<char *>) {
return fun(aio, name, args);
}
);
}
static void addWithRaw(const std::string & name, RawMainFunction fun)
{
if (!commands) commands = new LegacyCommandMap;
(*commands)[name] = fun;
+4
View File
@@ -74,7 +74,11 @@ std::string renderMarkdownToTerminal(std::string_view markdown, StandardOutputSt
.vmargin = 0,
#endif /* LOWDOWN_SEPARATE_TERM_OPTS */
.feat = LOWDOWN_COMMONMARK | LOWDOWN_FENCED | LOWDOWN_DEFLIST | LOWDOWN_TABLES,
#ifdef LOWDOWN_CONSOLIDATED_OFLAGS
.oflags = LOWDOWN_NOLINK,
#else
.oflags = LOWDOWN_TERM_NOLINK,
#endif /* LOWDOWN_CONSOLIDATED_OFLAGS */
};
if (!shouldANSI(fileno)) {
opts.oflags |= LOWDOWN_TERM_NOANSI;
+3 -6
View File
@@ -209,13 +209,12 @@ struct NixRepl
void printValue(std::ostream & str,
Value & v,
unsigned int maxDepth = std::numeric_limits<unsigned int>::max(),
unsigned int derivationPathDepth = 0)
unsigned int maxDepth = std::numeric_limits<unsigned int>::max())
{
::nix::printValue(state, str, v, PrintOptions {
.ansiColors = true,
.force = true,
.derivationPathDepth = derivationPathDepth,
.derivationPaths = true,
.maxDepth = maxDepth,
.prettyIndent = 2,
.errors = ErrorPrintBehavior::ThrowTopLevel,
@@ -794,10 +793,8 @@ ProcessLineResult NixRepl::processLine(std::string line)
evalString(arg, v);
if (v.type() == nString) {
std::cout << v.str();
} else if (v.type() == nAttrs && state.isDerivation(v)) {
printValue(std::cout, v, 2, 1);
} else {
printValue(std::cout, v, std::numeric_limits<unsigned int>::max(), 0);
printValue(std::cout, v);
}
std::cout << std::endl;
}
-2
View File
@@ -97,8 +97,6 @@ public:
res.reserve(size_);
for (Size n = 0; n < size_; n++)
res.emplace_back(&attrs[n]);
// NOTE: this sort uses a total order, so the iteration over pointers is not an issue
// NOLINTNEXTLINE(bugprone-nondeterministic-pointer-iteration-order)
std::sort(res.begin(), res.end(), [&](const Attr * a, const Attr * b) {
std::string_view sa = symbols[a->name], sb = symbols[b->name];
return sa < sb;
-96
View File
@@ -1,96 +0,0 @@
---
name: addErrorContext
args: [message, expr]
---
This adds a `message` to be shown in the stacktrace in the event of
a failure during the evaluation of `expr`.
For example, if a file `err.nix` contains the following:
```nix
let
countDown =
n:
if n == 0 then
throw "kaboom"
else
builtins.addErrorContext "while counting down; n = ${toString n}" ("x" + countDown (n - 1));
in
countDown 2
```
Then, evaluating the file will give the following stack trace:
```console
$ nix-instantiate --show-trace err.nix
error:
… from call site
at /home/plop/git.lix.systems/lix-project/lix/err.nix:9:1:
8| in
9| countDown 2
| ^
10|
… while calling 'countDown'
at /home/plop/git.lix.systems/lix-project/lix/err.nix:3:5:
2| countDown =
3| n:
| ^
4| if n == 0 then
… while calling the 'addErrorContext' builtin
at /home/plop/git.lix.systems/lix-project/lix/err.nix:7:7:
6| else
7| builtins.addErrorContext "while counting down; n = ${toString n}" ("x" + countDown (n - 1));
| ^
8| in
… while counting down; n = 2
… from call site
at /home/plop/git.lix.systems/lix-project/lix/err.nix:7:80:
6| else
7| builtins.addErrorContext "while counting down; n = ${toString n}" ("x" + countDown (n - 1));
| ^
8| in
… while calling 'countDown'
at /home/plop/git.lix.systems/lix-project/lix/err.nix:3:5:
2| countDown =
3| n:
| ^
4| if n == 0 then
… while calling the 'addErrorContext' builtin
at /home/plop/git.lix.systems/lix-project/lix/err.nix:7:7:
6| else
7| builtins.addErrorContext "while counting down; n = ${toString n}" ("x" + countDown (n - 1));
| ^
8| in
… while counting down; n = 1
… from call site
at /home/plop/git.lix.systems/lix-project/lix/err.nix:7:80:
6| else
7| builtins.addErrorContext "while counting down; n = ${toString n}" ("x" + countDown (n - 1));
| ^
8| in
… while calling 'countDown'
at /home/plop/git.lix.systems/lix-project/lix/err.nix:3:5:
2| countDown =
3| n:
| ^
4| if n == 0 then
… caused by explicit throw
at /home/plop/git.lix.systems/lix-project/lix/err.nix:5:7:
4| if n == 0 then
5| throw "kaboom"
| ^
6| else
error: kaboom
```
-6
View File
@@ -1,6 +0,0 @@
---
name: appendContext
args: [s, ctx]
---
Appends the attribute set `ctx` as a context to the string `s`, see [`getContext`](#builtins-getContext) for details on the format of the context.
-22
View File
@@ -1,22 +0,0 @@
---
name: derivationStrict
args: [args]
renameInGlobalScope: false
---
Constructs a [store derivation](../glossary.md#gloss-store-derivation) from the attribute set `args`
(c.f. [derivation](#builtins-derivation). Unlike `derivation` the produced store derivation is placed
in the store *immediately* when this builtin is called, while `derivation` may defer placing store
derivations in the store until it is proven that they are used.
It then returns a new attrset with *only* the following attributes:
- `drvPath` containing the path of the store derivation;
- For each output of the derivation (`out`, `dev`, etc): an attribute named after that output containing the output path
> **Note**
>
> In contrast to [`builtins.derivation`](#builtins-derivation), this computes
> the derivation set in a fully *strict* manner, i.e. the values of the attributes
> directly computed, whereas using `builtins.derivation` will produce an attrset
> whose values will be evaluated when they are used at a later point.
+72
View File
@@ -0,0 +1,72 @@
---
name: fetchClosure
args: [args]
experimentalFeature: fetch-closure
---
Fetch a store path [closure](@docroot@/glossary.md#gloss-closure) from a binary cache, and return the store path as a string with context.
This function can be invoked in three ways, that we will discuss in order of preference.
**Fetch a content-addressed store path**
Example:
```nix
builtins.fetchClosure {
fromStore = "https://cache.nixos.org";
fromPath = /nix/store/ldbhlwhh39wha58rm61bkiiwm6j7211j-git-2.33.1;
}
```
This is the simplest invocation, and it does not require the user of the expression to configure [`trusted-public-keys`](@docroot@/command-ref/conf-file.md#conf-trusted-public-keys) to ensure their authenticity.
If your store path is [input addressed](@docroot@/glossary.md#gloss-input-addressed-store-object) instead of content addressed, consider the other two invocations.
**Fetch any store path and rewrite it to a fully content-addressed store path**
Example:
```nix
builtins.fetchClosure {
fromStore = "https://cache.nixos.org";
fromPath = /nix/store/r2jd6ygnmirm2g803mksqqjm4y39yi6i-git-2.33.1;
toPath = /nix/store/ldbhlwhh39wha58rm61bkiiwm6j7211j-git-2.33.1;
}
```
This example fetches `/nix/store/r2jd...` from the specified binary cache,
and rewrites it into the content-addressed store path
`/nix/store/ldbh...`.
Like the previous example, no extra configuration or privileges are required.
To find out the correct value for `toPath` given a `fromPath`,
use [`nix store make-content-addressed`](@docroot@/command-ref/new-cli/nix3-store-make-content-addressed.md):
```console
# nix store make-content-addressed --from https://cache.nixos.org /nix/store/r2jd6ygnmirm2g803mksqqjm4y39yi6i-git-2.33.1
rewrote '/nix/store/r2jd6ygnmirm2g803mksqqjm4y39yi6i-git-2.33.1' to '/nix/store/ldbhlwhh39wha58rm61bkiiwm6j7211j-git-2.33.1'
```
Alternatively, set `toPath = ""` and find the correct `toPath` in the error message.
**Fetch an input-addressed store path as is**
Example:
```nix
builtins.fetchClosure {
fromStore = "https://cache.nixos.org";
fromPath = /nix/store/r2jd6ygnmirm2g803mksqqjm4y39yi6i-git-2.33.1;
inputAddressed = true;
}
```
It is possible to fetch an [input-addressed store path](@docroot@/glossary.md#gloss-input-addressed-store-object) and return it as is.
However, this is the least preferred way of invoking `fetchClosure`, because it requires that the input-addressed paths are trusted by the Lix configuration.
**`builtins.storePath`**
`fetchClosure` is similar to [`builtins.storePath`](#builtins-storePath) in that it allows you to use a previously built store path in a Nix expression.
However, `fetchClosure` is more reproducible because it specifies a binary cache from which the path can be fetched.
Also, using content-addressed store paths does not require users to configure [`trusted-public-keys`](@docroot@/command-ref/conf-file.md#conf-trusted-public-keys) to ensure their authenticity.
-48
View File
@@ -1,48 +0,0 @@
---
name: fetchMercurial
args: [args]
renameInGlobalScope: false
---
Fetch a Mercurial repository. *args* can be a URL, in which case the default
branch of the repo at that URL is fetched. Otherwise, it can be an
attribute with the following attributes (all except `url` optional):
- `url`
The URL of the repo.
- `name` (default: `"source"`)
The name of the directory the repo should be exported to in the store.
- `rev`
The revision to fetch.
This is typically a commit hash.
> **Note**
>
> Currently, `rev` can either contain a revision or a branch/tag name.
The return value is an attrset containing the following keys:
- `outPath` (`string`)
Resulting store path of the fetch process.
- `branch` (`string`)
The branch of the fetch repository.
- `rev` (`string`)
The revision that was fetched.
- `revCount` (`int`)
The number of revsets for this branch.
- `shortRev` (`string`)
The first *12* characters of `rev`.
-59
View File
@@ -1,59 +0,0 @@
---
name: scopedImport
implementation: "[](EvalState & state, Value ** args, Value & v) { import(state, *args[1], args[0], v); }"
args: [scope, path]
renameInGlobalScope: false
---
> **Warning**
>
> This builtin's use is heavily discouraged, it has many drawbacks and may be removed
> in a future version of Lix.
Functions like [`import`](#builtins-import) with the exception that
it takes a `scope`, which is a set of attributes to be added to the
lexical scope of the expression.
This essentially allows overriding the ambient builtin variables.
For example, if `foo.nix` is a file containing the following content:
```nix
x
```
then the following expression
```nix
scopedImport { x = 1; } ./foo.nix
```
will evaluate to `1`.
Another application is overriding builtin functions or constants, e.g. to
trace all calls to `map`, one can do:
```nix
let
overrides = {
map = f: xs: builtins.trace "map called!" (map f xs);
# Ensure that our override gets propagated by calls to
# import/scopedImport.
import = fn: scopedImport overrides fn;
scopedImport = attrs: fn: scopedImport (overrides // attrs) fn;
# Also update builtins.
builtins = builtins // overrides;
};
in scopedImport overrides ./bla.nix
```
Similarly, it can be used to extend the set of builtin functions.
> **Warning**
>
> One of the downsides of `scopedImport` is that it bypasses the evaluation cache.
> This means that importing a file multiple times will lead to multiple expensive
> parsings and evaluations.
+2
View File
@@ -13,3 +13,5 @@ in a new path (e.g. `/nix/store/ld01dnzc…-source-source`).
Not available in [pure evaluation mode](@docroot@/command-ref/conf-file.md#conf-pure-eval).
Lix may change this, tracking issue: <https://git.lix.systems/lix-project/lix/issues/402>
See also [`builtins.fetchClosure`](#builtins-fetchClosure).
@@ -1,33 +0,0 @@
---
name: unsafeDiscardStringContext
args: [s]
---
Returns a copy of the string `s` with all string context associated with `s` removed.
Since string context is used for dependency tracking the returned string will also have
*no dependencies* on store objects, even when the original string `s` had such dependencies.
This function is mainly useful when discarding dependencies is explicitly required, e.g.
to produce a string listing all inputs of a derivation without propagating these inputs
as dependencies into all *users* of the listing. For example, the derivation `dep` in the
following example will pull `hello` into its closure despite never using it while `nodep`
will not:
```nix
dep = runCommand "dep" {
inherit hello;
} "echo hello is at $hello >$out";
nodep = runCommand "nodep" {
hello = builtins.unsafeDiscardStringContext hello;
} "echo hello is at $hello >$out";
```
This behavior also makes this function unsafe: if `s` contains the path of a
store object that is not present in the store then any use of `s` in a
derivation tree will attempt to realize that path in the store, but no use
of `unsafeDiscardStringContext s` will. This can lead to derivation outputs that
refer to paths that were never created.
Lix cannot determine whether such reference are safe or not and must pass
this obligation to the user.
-7
View File
@@ -1,7 +0,0 @@
---
name: warn
args: [msg, e2]
---
Evaluate string *msg* and print it on standard error. Then return *e2*.
This function is useful for warning about unexpected conditions without aborting evaluation.
If the [`debugger-on-warn`](@docroot@/command-ref/conf-file.md#conf-debugger-on-trace) option is set to `true` and the `--debugger` flag is given, the interactive debugger will be started when `warn` is called (like [`break`](@docroot@/language/builtins.md#builtins-break)).
+13 -3
View File
@@ -20,11 +20,21 @@ inline Value::Value(app_t, EvalMemory & mem, Value & lhs, Value & rhs)
}
inline Value::Value(app_t, EvalMemory & mem, Value & lhs, std::span<Value> args)
: Value(app_t{}, mem, lhs, args, {})
{
auto app = static_cast<Value::App *>(mem.allocBytes(sizeof(Value::App) + args.size_bytes()));
}
inline Value::Value(
app_t, EvalMemory & mem, const Value & lhs, std::span<Value> baseArgs, std::span<Value> moreArgs
)
{
auto app = static_cast<Value::App *>(
mem.allocBytes(sizeof(Value::App) + baseArgs.size_bytes() + moreArgs.size_bytes())
);
app->_left = lhs;
app->_n = args.size();
std::copy(args.begin(), args.end(), app->_args);
app->_n = baseArgs.size() + moreArgs.size();
std::copy(baseArgs.begin(), baseArgs.end(), app->_args);
std::copy(moreArgs.begin(), moreArgs.end(), app->_args + baseArgs.size());
raw = tag(tApp, app);
}
+8 -67
View File
@@ -4,14 +4,12 @@
#include "lix/libutil/archive.hh"
#include "lix/libutil/ansicolor.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/deprecated-features.hh"
#include "lix/libutil/error.hh"
#include "lix/libutil/english.hh"
#include "lix/libutil/fmt.hh"
#include "lix/libexpr/primops.hh"
#include "lix/libexpr/print-options.hh"
#include "lix/libmain/shared.hh"
#include "lix/libutil/logging.hh"
#include "lix/libutil/suggestions.hh"
#include "lix/libutil/types.hh"
#include "lix/libstore/store-api.hh"
@@ -349,61 +347,6 @@ EvalPaths::EvalPaths(
allowPath(path);
}
}
#if LIX_MAJOR >= 3 || (LIX_MAJOR == 2 && LIX_MINOR >= 96)
#warning \
"The feature nix-path-shadow was deprecated in 2.95 with a warning, which needs to be turned into an error in 2.96"
#endif
if (!featureSettings.isEnabled(DeprecatedFeature::NixPathShadow)) {
for (auto & [prefix, path] : searchPath_.elements) {
// Match on the 'nix' prefix
if (prefix.s == "nix") {
logWarning(
{.msg = HintFmt(
"The prefix '%s' is reserved for internal use by Lix in the Nix search "
"path, its usage is deprecated and will be forbidden in the future.\n"
"Use %s to silence this warning.\n"
"This is due to adding '%s=%s' in the Nix search path, either through the "
"environment variable '%s' or by passing the flag %s to the nix "
"invocation.",
"nix",
"--extra-deprecated-features nix-path-shadow",
prefix.s,
path.s,
"NIX_PATH",
"-I"
)}
);
} else
// Match prefixless paths that contain a `nix` directory
if (auto res =
prefix.suffixIfPotentialMatch("nix").and_then([&](std::string_view s) {
return aio.blockOn(resolveSearchPathPath(path))
.and_then([&](std::string r) {
Path res = s.length() ? concatStrings(r, "/", s): r;
return pathExists(res) ? std::optional(res) : std::nullopt;
});
}))
{
logWarning(
{.msg = HintFmt(
"Shadowing '%s' by configuring the nix-path is deprecated and "
"will be forbidden in the future.\n"
"Use %s to silence this warning.\n"
"This is due to adding '%s' to the nix-path without a prefix, "
"either by passing the flag '-I %s' to the nix invocation or by "
"adding this path to the environment variable '%s'.",
"<nix/...>",
"--extra-deprecated-features nix-path-shadow",
path.s,
path.s,
"NIX_PATH"
)}
);
}
}
}
}
Evaluator::Evaluator(
@@ -1696,7 +1639,14 @@ void EvalState::callFunction(Value & fun, std::span<Value> args, Value & vRes, c
Value vCur(fun);
auto makeAppChain = [&]() { vRes = {NewValueAs::app, ctx.mem, vCur, args}; };
auto makeAppChain = [&]() {
if (vCur.isApp()) {
auto & app = vCur.app();
vRes = {NewValueAs::app, ctx.mem, app.left(), app.args(), args};
} else {
vRes = {NewValueAs::app, ctx.mem, vCur, args};
}
};
const Attr * functor;
@@ -2981,15 +2931,6 @@ Expr & Evaluator::parseStdin()
);
}
std::optional<DebugTrace const *> Evaluator::nextDebugTrace() const
{
if (!debug) {
return std::nullopt;
}
return debug->traces().next();
}
kj::Promise<Result<EvalPaths::PathResult<SourcePath, ThrownError>>>
EvalPaths::findFile(const std::string_view path)
-3
View File
@@ -568,9 +568,6 @@ public:
*/
void evalLazily(Expr & e, Value & v);
/** If debugging is enabled, returns the next trace. Otherwise, std::nullopt. */
std::optional<DebugTrace const *> nextDebugTrace() const;
private:
Expr * parse(
char * text,
+1 -3
View File
@@ -9,17 +9,15 @@ class EvalState;
struct Value;
void prim_addDrvOutputDependencies(EvalState & state, Value * * args, Value & v);
void prim_fetchClosure(EvalState & state, Value * * args, Value & v);
void prim_fetchTree(EvalState & state, Value * * args, Value & v);
void prim_fetchGit(EvalState & state, Value * * args, Value & v);
void prim_fetchMercurial(EvalState & state, Value ** args, Value & v);
void prim_fetchTarball(EvalState & state, Value * * args, Value & v);
void prim_fetchurl(EvalState & state, Value * * args, Value & v);
void prim_fromTOML(EvalState & state, Value * * args, Value & v);
void prim_appendContext(EvalState & state, Value ** args, Value & v);
void prim_getContext(EvalState & state, Value * * args, Value & v);
void prim_hasContext(EvalState & state, Value * * args, Value & v);
void prim_unsafeDiscardOutputDependency(EvalState & state, Value * * args, Value & v);
void prim_unsafeDiscardStringContext(EvalState & state, Value ** args, Value & v);
namespace flake {
+2 -9
View File
@@ -15,12 +15,10 @@ subdir('flake')
libexpr_setting_definitions = files(
# keep-sorted start
'settings/abort-on-warn.md',
'settings/allow-import-from-derivation.md',
'settings/allow-unsafe-native-code-during-evaluation.md',
'settings/allowed-uris.md',
'settings/debugger-on-trace.md',
'settings/debugger-on-warn.md',
'settings/eval-cache.md',
'settings/eval-system.md',
'settings/ignore-try.md',
@@ -53,10 +51,8 @@ builtin_definitions = files(
'builtins/abort.md',
'builtins/add.md',
'builtins/addDrvOutputDependencies.md',
'builtins/addErrorContext.md',
'builtins/all.md',
'builtins/any.md',
'builtins/appendContext.md',
'builtins/attrNames.md',
'builtins/attrValues.md',
'builtins/baseNameOf.md',
@@ -71,13 +67,12 @@ builtin_definitions = files(
'builtins/concatMap.md',
'builtins/concatStringsSep.md',
'builtins/deepSeq.md',
'builtins/derivationStrict.md',
'builtins/dirOf.md',
'builtins/div.md',
'builtins/elem.md',
'builtins/elemAt.md',
'builtins/fetchClosure.md',
'builtins/fetchGit.md',
'builtins/fetchMercurial.md',
'builtins/fetchTarball.md',
'builtins/fetchTree.md',
'builtins/fetchurl.md',
@@ -131,7 +126,6 @@ builtin_definitions = files(
'builtins/readFileType.md',
'builtins/removeAttrs.md',
'builtins/replaceStrings.md',
'builtins/scopedImport.md',
'builtins/seq.md',
'builtins/sort.md',
'builtins/split.md',
@@ -152,9 +146,7 @@ builtin_definitions = files(
'builtins/tryEval.md',
'builtins/typeOf.md',
'builtins/unsafeDiscardOutputDependency.md',
'builtins/unsafeDiscardStringContext.md',
'builtins/unsafeGetAttrPos.md',
'builtins/warn.md',
'builtins/zipAttrsWith.md',
# keep-sorted end
)
@@ -227,6 +219,7 @@ libexpr_sources = files(
'parser/parser.cc',
'primops.cc',
'primops/context.cc',
'primops/fetchClosure.cc',
'primops/fetchMercurial.cc',
'primops/fetchTree.cc',
'primops/fromTOML.cc',
+11 -39
View File
@@ -35,24 +35,6 @@ AttrName::AttrName(PosIdx pos, std::unique_ptr<Expr> e) : pos(pos), expr(std::mo
{
}
static JSON stringToJSON(std::string_view s)
{
try {
JSON value = s;
// dump it now to catch invalid utf8 strings early. this code path is not
// hot, so the extra memory allocation and encoding is not worth avoiding
(void) value.dump();
return value;
} catch (nlohmann::json::type_error & e) { // NOLINT(lix-foreign-exceptions)
if (e.id == 316) {
// invalid utf8 in string! serialize as byte array instead
return s | std::ranges::to<std::vector<unsigned char>>();
} else {
throw; // NOLINT(lix-foreign-exceptions)
}
}
}
JSON Expr::toJSON(const SymbolTable & symbols) const
{
abort();
@@ -75,7 +57,7 @@ JSON ExprLiteral::toJSON(const SymbolTable & symbols) const
break;
case nString:
valueType = "String";
value = stringToJSON(v.str());
value = v.str();
break;
case nPath:
valueType = "Path";
@@ -95,7 +77,10 @@ JSON ExprLiteral::toJSON(const SymbolTable & symbols) const
JSON ExprVar::toJSON(const SymbolTable & symbols) const
{
return {{"_type", "ExprVar"}, {"value", stringToJSON(symbols[name])}};
return {
{"_type", "ExprVar"},
{"value", symbols[name]}
};
}
JSON ExprInheritFrom::toJSON(SymbolTable const & symbols) const
@@ -131,8 +116,6 @@ void ExprAttrs::addBindingsToJSON(JSON & out, const SymbolTable & symbols) const
typedef const decltype(attrs)::value_type * Attr;
std::vector<Attr> sorted;
for (auto & i : attrs) sorted.push_back(&i);
// NOTE: this sort uses a total order, so the iteration over pointers is not an issue
// NOLINTNEXTLINE(bugprone-nondeterministic-pointer-iteration-order)
std::sort(sorted.begin(), sorted.end(), [&](Attr a, Attr b) {
std::string_view sa = symbols[a->first], sb = symbols[b->first];
return sa < sb;
@@ -141,19 +124,11 @@ void ExprAttrs::addBindingsToJSON(JSON & out, const SymbolTable & symbols) const
for (auto & i : sorted) {
switch (i->second.kind) {
case AttrDef::Kind::Plain:
case AttrDef::Kind::Inherited: {
const auto key = i->second.kind == AttrDef::Kind::Plain ? "attrs" : "inherit";
auto name = stringToJSON(symbols[i->first]);
if (name.is_string()) {
out[key][name] = i->second.e->toJSON(symbols);
} else {
out[fmt("binary_%s", key)][key].push_back({
{"name", name},
{"value", i->second.e->toJSON(symbols)},
});
}
out["attrs"][std::string(symbols[i->first])] = i->second.e->toJSON(symbols);
break;
case AttrDef::Kind::Inherited:
out["inherit"][std::string(symbols[i->first])] = i->second.e->toJSON(symbols);
break;
}
case AttrDef::Kind::InheritedFrom: {
auto & select = i->second.e->cast<ExprSelect>();
auto & from = select.e->cast<ExprInheritFrom>();
@@ -173,7 +148,7 @@ void ExprAttrs::addBindingsToJSON(JSON & out, const SymbolTable & symbols) const
for (const auto & [from, syms] : inheritsFrom) {
JSON attrs = JSON::array();
for (auto sym : syms)
attrs.push_back(stringToJSON(symbols[sym]));
attrs.push_back(symbols[sym]);
out["inheritFrom"].push_back({
{"from", inheritFromExprs[from]->toJSON(symbols)},
{"attrs", attrs}
@@ -211,13 +186,11 @@ JSON ExprList::toJSON(const SymbolTable & symbols) const
void SimplePattern::addBindingsToJSON(JSON & out, const SymbolTable & symbols) const
{
// name must be alphanumeric
out["arg"] = symbols[name];
}
void AttrsPattern::addBindingsToJSON(JSON & out, const SymbolTable & symbols) const
{
// name must be alphanumeric
if (name)
out["arg"] = symbols[name];
@@ -225,7 +198,6 @@ void AttrsPattern::addBindingsToJSON(JSON & out, const SymbolTable & symbols) co
// same expression being printed in two different ways depending on its
// context. always use lexicographic ordering to avoid this.
for (const Formal & i : lexicographicOrder(symbols)) {
// names must be alphanumeric
if (i.def)
out["formals"][std::string(symbols[i.name])] = i.def->toJSON(symbols);
else
@@ -339,7 +311,7 @@ JSON printAttrPathToJson(const SymbolTable & symbols, const AttrPath & attrPath)
JSON out = JSON::array();
for (auto & i : attrPath) {
if (i.symbol)
out.push_back(stringToJSON(symbols[i.symbol]));
out.push_back(symbols[i.symbol]);
else
out.push_back(i.expr->toJSON(symbols));
}
-5
View File
@@ -32,11 +32,6 @@ struct AttrName
std::unique_ptr<Expr> expr;
AttrName(PosIdx pos, Symbol s);
AttrName(PosIdx pos, std::unique_ptr<Expr> e);
inline bool isDynamic()
{
return !symbol;
}
};
typedef std::vector<AttrName> AttrPath;
+6 -16
View File
@@ -203,16 +203,6 @@ struct sep : sor<
using seps = star<t::sep>;
// Delimiters.
namespace d {
// Terminators.
namespace t {
struct inherit : one<';'> { };
struct binding : one<';'> { };
struct attrs : one<'}'> { };
}
}
// marker for semantic rules. not handling one of these in an action that cares about
// semantics is probably an error.
@@ -402,7 +392,7 @@ struct _attr {
struct expr : semantic, seq<
TAO_PEGTL_STRING("${"), seps,
must<grammar::v1::expr>, seps,
must<d::t::attrs>
must<one<'}'>>
> {};
};
struct attr : _attr, sor<
@@ -421,7 +411,7 @@ struct inherit : _inherit, seq<
t::kw_inherit, seps,
opt<one<'('>, seps, _inherit::from, seps, must<one<')'>>, seps>,
opt<_inherit::attrs, seps>,
must<d::t::inherit>
must<one<';'>>
> {};
struct _binding {
@@ -440,7 +430,7 @@ struct bindings : opt<
list<
sor<
inherit,
seq<binding, seps, must<d::t::binding>>
seq<binding, seps, must<one<';'>>>
>,
seps
>
@@ -449,9 +439,9 @@ struct bindings : opt<
struct repl_binding : binding {};
struct repl_bindings : seq<
list<repl_binding, d::t::binding, t::sep>,
list<repl_binding, one<';'>, t::sep>,
/* Optional semicolon at the end for convenience */
opt<seps, d::t::binding>
opt<seps, one<';'>>
> {};
struct op {
@@ -506,7 +496,7 @@ struct _expr {
Init...,
OpenMod<one<'{'>>, seps,
bindings, seps,
must<d::t::attrs>
must<one<'}'>>
> {};
struct select;
-3
View File
@@ -56,9 +56,6 @@ error_message_for(grammar::v1::expr::select) = "expecting selection expression";
error_message_for(grammar::v1::t::kw_then) = "expecting 'then'";
error_message_for(grammar::v1::t::kw_else) = "expecting 'else'";
error_message_for(grammar::v1::t::kw_in) = "expecting 'in'";
error_message_for(grammar::v1::d::t::binding) = "expecting ';' to end binding";
error_message_for(grammar::v1::d::t::inherit) = "expecting ';' to end 'inherit' bindings";
error_message_for(grammar::v1::d::t::attrs) = "expecting '}' to end attribute set";
struct SyntaxErrors
{
+66 -98
View File
@@ -6,6 +6,7 @@
#include "lix/libutil/logging.hh"
namespace nix::parser {
struct IndStringLine {
// String containing only the leading whitespace of the line. May be empty.
std::string_view indentation;
@@ -39,7 +40,6 @@ struct State
void badLineEndingFound(const PosIdx pos, bool warnOnly);
void nulFound(const PosIdx pos);
void addAttr(ExprAttrs * attrs, AttrPath && attrPath, std::unique_ptr<Expr> e, const PosIdx pos);
void mergeAttrs(AttrPath & attrPath, ExprSet * source, ExprSet * target);
void validateLambdaAttrs(AttrsPattern & pattern, PosIdx pos = noPos);
std::unique_ptr<Expr> stripIndentation(const PosIdx pos, std::vector<IndStringLine> && line);
@@ -132,113 +132,81 @@ inline void State::addAttr(ExprAttrs * attrs, AttrPath && attrPath, std::unique_
AttrPath::iterator i;
// All attrpaths have at least one attr
assert(!attrPath.empty());
// Walk the attrpath up to the parent of the attribute we want to insert, moving `attrs` along
// and creating new empty intermediate attrsets as necessary.
// Checking attrPath validity.
// ===========================
for (i = attrPath.begin(); i + 1 < attrPath.end(); i++) {
AttrName & attr = *i;
if (attr.isDynamic()) {
// Simply insert an empty attrset (but dynamic)
if (i->symbol) {
ExprAttrs::AttrDefs::iterator j = attrs->attrs.find(i->symbol);
if (j != attrs->attrs.end()) {
if (j->second.kind != ExprAttrs::AttrDef::Kind::Inherited) {
ExprSet * attrs2 = dynamic_cast<ExprSet *>(j->second.e.get());
if (!attrs2) {
attrPath.erase(i + 1, attrPath.end());
dupAttr(attrPath, pos, j->second.pos);
}
attrs = attrs2;
} else {
attrPath.erase(i + 1, attrPath.end());
dupAttr(attrPath, pos, j->second.pos);
}
} else {
auto next = attrs->attrs.emplace(std::piecewise_construct,
std::tuple(i->symbol),
std::tuple(std::make_unique<ExprSet>(), pos));
attrs = static_cast<ExprSet *>(next.first->second.e.get());
}
} else {
auto & next = attrs->dynamicAttrs.emplace_back(std::move(i->expr), std::make_unique<ExprSet>(), pos);
attrs = static_cast<ExprSet *>(next.valueExpr.get());
} else if (ExprAttrs::AttrDefs::iterator j = attrs->attrs.find(i->symbol);
j != attrs->attrs.end())
{
// Try to walk down the next attribute, throw duplicate error if not possible
auto & [foundName, foundDef] = *j;
if (foundDef.kind == ExprAttrs::AttrDef::Kind::Inherited) {
attrPath.erase(i + 1, attrPath.end());
return dupAttr(attrPath, pos, foundDef.pos);
}
ExprSet * foundAttrs = dynamic_cast<ExprSet *>(foundDef.e.get());
if (!foundAttrs) {
attrPath.erase(i + 1, attrPath.end());
return dupAttr(attrPath, pos, foundDef.pos);
}
attrs = foundAttrs;
} else {
// Simply insert an empty attrset
auto next = attrs->attrs.emplace(
std::piecewise_construct,
std::tuple(attr.symbol),
std::tuple(std::make_unique<ExprSet>(), pos)
);
attrs = static_cast<ExprSet *>(next.first->second.e.get());
}
}
// Expr insertion.
// ==========================
AttrName & attr = *i;
if (i->symbol) {
ExprAttrs::AttrDefs::iterator j = attrs->attrs.find(i->symbol);
if (j != attrs->attrs.end()) {
// This attr path is already defined. However, if both
// e and the expr pointed by the attr path are two attribute sets,
// we want to merge them.
// Otherwise, throw an error.
auto * ae = dynamic_cast<ExprSet *>(e.get());
auto * jAttrs = dynamic_cast<ExprSet *>(j->second.e.get());
if (jAttrs && ae) {
if (ae->inheritFromExprs && !jAttrs->inheritFromExprs)
jAttrs->inheritFromExprs = std::make_unique<std::list<std::unique_ptr<Expr>>>();
for (auto & ad : ae->attrs) {
auto j2 = jAttrs->attrs.find(ad.first);
if (j2 != jAttrs->attrs.end()) // Attr already defined in iAttrs, error.
return dupAttr(ad.first, j2->second.pos, ad.second.pos);
if (ad.second.kind == ExprAttrs::AttrDef::Kind::InheritedFrom) {
auto & sel = dynamic_cast<ExprSelect &>(*ad.second.e);
auto & from = dynamic_cast<ExprInheritFrom &>(*sel.e);
from.displ += jAttrs->inheritFromExprs->size();
}
jAttrs->attrs.emplace(ad.first, std::move(ad.second));
}
std::ranges::move(ae->dynamicAttrs, std::back_inserter(jAttrs->dynamicAttrs));
if (ae->inheritFromExprs)
std::ranges::move(*ae->inheritFromExprs, std::back_inserter(*jAttrs->inheritFromExprs));
} else {
dupAttr(attrPath, pos, j->second.pos);
}
} else {
// Before inserting new attrs, check for __override and throw an error
// (the error will initially be a warning to ease migration)
if (!featureSettings.isEnabled(Dep::RecSetOverrides) && i->symbol == s.overrides) {
if (auto set = dynamic_cast<ExprSet *>(attrs); set && set->recursive)
overridesFound(pos);
}
if (attr.isDynamic()) {
attrs->dynamicAttrs.emplace_back(std::move(attr.expr), std::move(e), pos);
} else if (ExprAttrs::AttrDefs::iterator j = attrs->attrs.find(attr.symbol);
j != attrs->attrs.end())
{
// This attr path is already defined. However, if both
// e and the expr pointed by the attr path are two attribute sets,
// we want to merge them.
// Otherwise, throw an error.
auto & [foundName, foundDef] = *j;
auto * insertAttrs = dynamic_cast<ExprSet *>(e.get());
auto * foundAttrs = dynamic_cast<ExprSet *>(foundDef.e.get());
if (!foundAttrs || !insertAttrs) {
return dupAttr(attrPath, pos, foundDef.pos);
// This attr path is not defined. Let's create it.
e->setName(i->symbol);
attrs->attrs.emplace(std::piecewise_construct,
std::tuple(i->symbol),
std::tuple(std::move(e), pos));
}
mergeAttrs(attrPath, insertAttrs, foundAttrs);
} else {
// This attr path is not defined. Let's create it.
// Before inserting new attrs, check for __override and throw an error
// (the error will initially be a warning to ease migration)
if (!featureSettings.isEnabled(Dep::RecSetOverrides) && attr.symbol == s.overrides) {
if (auto set = dynamic_cast<ExprSet *>(attrs); set && set->recursive) {
overridesFound(pos);
}
}
e->setName(attr.symbol);
attrs->attrs.emplace(
std::piecewise_construct, std::tuple(attr.symbol), std::tuple(std::move(e), pos)
);
}
}
/* mutably merge source into target. attrPath is only for error messages */
inline void State::mergeAttrs(AttrPath & attrPath, ExprSet * source, ExprSet * target)
{
if (source->inheritFromExprs && !target->inheritFromExprs) {
target->inheritFromExprs = std::make_unique<std::list<std::unique_ptr<Expr>>>();
}
for (auto & [insertKey, insertDef] : source->attrs) {
if (auto collision = target->attrs.find(insertKey); collision != target->attrs.end()) {
// Attr already defined in target, recurse merge if possible otherwise error.
auto * collisionInsert = dynamic_cast<ExprSet *>(insertDef.e.get());
auto * collisionTarget = dynamic_cast<ExprSet *>(collision->second.e.get());
if (!collisionInsert || !collisionTarget) {
attrPath.push_back(AttrName(insertDef.pos, insertKey));
return dupAttr(attrPath, insertDef.pos, collision->second.pos);
}
// Push insertKey to the attrPath for error propagation (pop afterwards), then recurse
// merge
attrPath.push_back(AttrName(insertDef.pos, insertKey));
mergeAttrs(attrPath, collisionInsert, collisionTarget);
attrPath.pop_back();
}
if (insertDef.kind == ExprAttrs::AttrDef::Kind::InheritedFrom) {
auto & sel = dynamic_cast<ExprSelect &>(*insertDef.e);
auto & from = dynamic_cast<ExprInheritFrom &>(*sel.e);
from.displ += target->inheritFromExprs->size();
}
target->attrs.emplace(insertKey, std::move(insertDef));
}
std::ranges::move(source->dynamicAttrs, std::back_inserter(target->dynamicAttrs));
if (source->inheritFromExprs) {
std::ranges::move(*source->inheritFromExprs, std::back_inserter(*target->inheritFromExprs));
attrs->dynamicAttrs.emplace_back(std::move(i->expr), std::move(e), pos);
}
}
+25 -41
View File
@@ -251,6 +251,13 @@ static void import(EvalState & state, Value & vPath, Value * vScope, Value & v)
}
}
static RegisterPrimOp primop_scopedImport(PrimOp{
{.name = "scopedImport",
.arity = 2,
.fun = [](EvalState & state, Value ** args, Value & v) { import(state, *args[1], args[0], v); }
}
});
static void prim_import(EvalState & state, Value * * args, Value & v)
{
import(state, *args[0], nullptr, v);
@@ -263,11 +270,6 @@ extern "C" typedef void (*ValueInitializer)(EvalState & state, Value & v);
/* Load a ValueInitializer from a DSO and return whatever it initializes */
void prim_importNative(EvalState & state, Value * * args, Value & v)
{
# if LIX_MAJOR > 2 || (LIX_MAJOR == 2 && LIX_MINOR >= 97)
#pragma message ("Folks, we need to rip this out since we've reached 2.97 See fj#796 for more details")
# endif
printTaggedWarning("builtins.importNative is deprecated and will be removed in Lix 2.97, please migrate away from it. You can browse issue #795 for more details.");
auto path = realisePath(state, *args[0]);
std::string sym(state.forceStringNoCtx(*args[1], noPos, "while evaluating the second argument passed to builtins.importNative"));
@@ -634,7 +636,7 @@ static void prim_genericClosure(EvalState & state, Value * * args, Value & v)
static void prim_break(EvalState & state, Value * * args, Value & v)
{
if (auto const trace = state.ctx.nextDebugTrace()) {
if (auto trace = state.ctx.debug ? state.ctx.debug->traces().next() : std::nullopt) {
auto error = EvalError(ErrorInfo {
.level = lvlInfo,
.msg = HintFmt("breakpoint reached"),
@@ -678,6 +680,12 @@ static void prim_addErrorContext(EvalState & state, Value * * args, Value & v)
}
}
static RegisterPrimOp primop_addErrorContext(PrimOp{{
.name = "__addErrorContext",
.arity = 2,
.fun = prim_addErrorContext,
}});
static void prim_ceil(EvalState & state, Value * * args, Value & v)
{
auto value = state.forceFloat(*args[0], noPos,
@@ -763,46 +771,16 @@ static void prim_trace(EvalState & state, Value * * args, Value & v)
printError("trace: %1%", Uncolored(args[0]->str()));
else
printError("trace: %1%", Uncolored(ValuePrinter(state, *args[0])));
if (evalSettings.debuggerOnTrace) {
if (auto const trace = state.ctx.nextDebugTrace()) {
state.ctx.debug->onEvalError(nullptr, (*trace)->env, (*trace)->expr);
}
if (auto last = evalSettings.builtinsTraceDebugger && state.ctx.debug
? state.ctx.debug->traces().next()
: std::nullopt)
{
state.ctx.debug->onEvalError(nullptr, (*last)->env, (*last)->expr);
}
state.forceValue(*args[1], noPos);
v = *args[1];
}
static void prim_warn(EvalState & state, Value ** args, Value & v)
{
// We only accept a string argument for now. The use case for pretty printing a value is covered
// by `trace`. By rejecting non-strings we allow future versions to add more features without
// breaking existing code.
auto const msg =
state.forceString(*args[0], noPos, "while evaluating message for builtins.warn");
printTaggedWarning("%s", Uncolored(msg));
if (evalSettings.abortOnWarn) {
state.ctx.errors.make<Abort>("evaluation aborted (abort-on-warn)")
.debugThrow();
}
if (evalSettings.debuggerOnWarn) {
if (auto const trace = state.ctx.nextDebugTrace()) {
auto const error = EvalError(ErrorInfo{
.level = lvlWarn,
.msg = HintFmt("builtins.warn reached"),
});
state.ctx.debug->onEvalError(&error, (*trace)->env, (*trace)->expr);
}
}
state.forceValue(*args[1], noPos);
v = *args[1];
}
/* Takes two arguments and evaluates to the second one. Used as the
* builtins.traceVerbose implementation when --trace-verbose is not enabled
@@ -1248,6 +1226,12 @@ drvName, Bindings * attrs, Value & v)
v.mkAttrs(result);
}
static RegisterPrimOp primop_derivationStrict(PrimOp{{
.name = "derivationStrict",
.arity = 1,
.fun = prim_derivationStrict,
}});
/* Return a placeholder string for the specified output that will be
substituted by the corresponding output path at build time. For
example, 'placeholder "out"' returns the string
+16 -2
View File
@@ -7,13 +7,20 @@
namespace nix {
void prim_unsafeDiscardStringContext(EvalState & state, Value ** args, Value & v)
static void prim_unsafeDiscardStringContext(EvalState & state, Value * * args, Value & v)
{
NixStringContext context;
auto s = state.coerceToString(noPos, *args[0], context, "while evaluating the argument passed to builtins.unsafeDiscardStringContext");
v.mkString(*s);
}
static RegisterPrimOp primop_unsafeDiscardStringContext({
.name = "__unsafeDiscardStringContext",
.arity = 1,
.fun = prim_unsafeDiscardStringContext
});
void prim_hasContext(EvalState & state, Value * * args, Value & v)
{
NixStringContext context;
@@ -158,7 +165,7 @@ void prim_getContext(EvalState & state, Value * * args, Value & v)
See the commentary above unsafeGetContext for details of the
context representation.
*/
void prim_appendContext(EvalState & state, Value ** args, Value & v)
static void prim_appendContext(EvalState & state, Value * * args, Value & v)
{
NixStringContext context;
auto orig = state.forceString(*args[0], context, noPos, "while evaluating the first argument passed to builtins.appendContext");
@@ -234,4 +241,11 @@ void prim_appendContext(EvalState & state, Value ** args, Value & v)
v.mkString(orig, context);
}
static RegisterPrimOp primop_appendContext({
.name = "__appendContext",
.arity = 2,
.fun = prim_appendContext
});
}
+201
View File
@@ -0,0 +1,201 @@
#include "lix/libexpr/eval.hh"
#include "lix/libexpr/extra-primops.hh"
#include "lix/libstore/store-api.hh"
#include "lix/libstore/make-content-addressed.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/url.hh"
namespace nix {
/**
* Handler for the content addressed case.
*
* @param state Evaluator state and store to write to.
* @param fromStore Store containing the path to rewrite.
* @param fromPath Source path to be rewritten.
* @param toPathMaybe Path to write the rewritten path to. If empty, the error shows the actual path.
* @param v Return `Value`
*/
static void runFetchClosureWithRewrite(EvalState & state, Store & fromStore, const StorePath & fromPath, const std::optional<StorePath> & toPathMaybe, Value &v) {
// establish toPath or throw
if (!toPathMaybe || !state.aio.blockOn(state.ctx.store->isValidPath(*toPathMaybe))) {
auto rewrittenPath =
state.aio.blockOn(makeContentAddressed(fromStore, *state.ctx.store, fromPath));
if (toPathMaybe && *toPathMaybe != rewrittenPath)
throw Error({
.msg = HintFmt("rewriting '%s' to content-addressed form yielded '%s', while '%s' was expected",
state.ctx.store->printStorePath(fromPath),
state.ctx.store->printStorePath(rewrittenPath),
state.ctx.store->printStorePath(*toPathMaybe)),
});
if (!toPathMaybe)
throw Error({
.msg = HintFmt(
"rewriting '%s' to content-addressed form yielded '%s'\n"
"Use this value for the 'toPath' attribute passed to 'fetchClosure'",
state.ctx.store->printStorePath(fromPath),
state.ctx.store->printStorePath(rewrittenPath)),
});
}
auto toPath = *toPathMaybe;
// check and return
auto resultInfo = state.aio.blockOn(state.ctx.store->queryPathInfo(toPath));
if (!resultInfo->isContentAddressed(*state.ctx.store)) {
// We don't perform the rewriting when outPath already exists, as an optimisation.
// However, we can quickly detect a mistake if the toPath is input addressed.
throw Error({
.msg = HintFmt(
"The 'toPath' value '%s' is input-addressed, so it can't possibly be the result of rewriting to a content-addressed path.\n\n"
"Set 'toPath' to an empty string to make Lix report the correct content-addressed path.",
state.ctx.store->printStorePath(toPath)),
});
}
state.ctx.paths.mkStorePathString(toPath, v);
}
/**
* Fetch the closure and make sure it's content addressed.
*/
static void runFetchClosureWithContentAddressedPath(EvalState & state, Store & fromStore, const StorePath & fromPath, Value & v) {
if (!state.aio.blockOn(state.ctx.store->isValidPath(fromPath)))
state.aio.blockOn(copyClosure(fromStore, *state.ctx.store, RealisedPath::Set { fromPath }));
auto info = state.aio.blockOn(state.ctx.store->queryPathInfo(fromPath));
if (!info->isContentAddressed(*state.ctx.store)) {
throw Error({
.msg = HintFmt(
"The 'fromPath' value '%s' is input-addressed, but 'inputAddressed' is set to 'false' (default).\n\n"
"If you do intend to fetch an input-addressed store path, add\n\n"
" inputAddressed = true;\n\n"
"to the 'fetchClosure' arguments.\n\n"
"Note that to ensure authenticity input-addressed store paths, users must configure a trusted binary cache public key on their systems. This is not needed for content-addressed paths.",
state.ctx.store->printStorePath(fromPath)),
});
}
state.ctx.paths.mkStorePathString(fromPath, v);
}
/**
* Fetch the closure and make sure it's input addressed.
*/
static void runFetchClosureWithInputAddressedPath(EvalState & state, Store & fromStore, const StorePath & fromPath, Value & v) {
if (!state.aio.blockOn(state.ctx.store->isValidPath(fromPath)))
state.aio.blockOn(copyClosure(fromStore, *state.ctx.store, RealisedPath::Set { fromPath }));
auto info = state.aio.blockOn(state.ctx.store->queryPathInfo(fromPath));
if (info->isContentAddressed(*state.ctx.store)) {
throw Error({
.msg = HintFmt(
"The store object referred to by 'fromPath' at '%s' is not input-addressed, but 'inputAddressed' is set to 'true'.\n\n"
"Remove the 'inputAddressed' attribute (it defaults to 'false') to expect 'fromPath' to be content-addressed",
state.ctx.store->printStorePath(fromPath)),
});
}
state.ctx.paths.mkStorePathString(fromPath, v);
}
typedef std::optional<StorePath> StorePathOrGap;
void prim_fetchClosure(EvalState & state, Value * * args, Value & v)
{
state.forceAttrs(*args[0], noPos, "while evaluating the argument passed to builtins.fetchClosure");
std::optional<std::string> fromStoreUrl;
std::optional<StorePath> fromPath;
std::optional<StorePathOrGap> toPath;
std::optional<bool> inputAddressedMaybe;
for (auto & attr : *args[0]->attrs()) {
const auto & attrName = state.ctx.symbols[attr.name];
auto attrHint = [&]() -> std::string {
return "while evaluating the '" + attrName + "' attribute passed to builtins.fetchClosure";
};
if (attrName == "fromPath") {
NixStringContext context;
fromPath = state.coerceToStorePath(attr.pos, attr.value, context, attrHint());
}
else if (attrName == "toPath") {
state.forceValue(attr.value, attr.pos);
bool isEmptyString = attr.value.type() == nString && attr.value.str().empty();
if (isEmptyString) {
toPath = StorePathOrGap {};
}
else {
NixStringContext context;
toPath = state.coerceToStorePath(attr.pos, attr.value, context, attrHint());
}
}
else if (attrName == "fromStore")
fromStoreUrl = state.forceStringNoCtx(attr.value, attr.pos, attrHint());
else if (attrName == "inputAddressed")
inputAddressedMaybe = state.forceBool(attr.value, attr.pos, attrHint());
else
throw Error({
.msg = HintFmt("attribute '%s' isn't supported in call to 'fetchClosure'", attrName),
});
}
if (!fromPath)
throw Error({
.msg = HintFmt("attribute '%s' is missing in call to 'fetchClosure'", "fromPath"),
});
bool inputAddressed = inputAddressedMaybe.value_or(false);
if (inputAddressed) {
if (toPath)
throw Error({
.msg = HintFmt("attribute '%s' is set to true, but '%s' is also set. Please remove one of them",
"inputAddressed",
"toPath"),
});
}
if (!fromStoreUrl)
throw Error({
.msg = HintFmt("attribute '%s' is missing in call to 'fetchClosure'", "fromStore"),
});
auto parsedURL = parseURL(*fromStoreUrl);
if (parsedURL.scheme != "http" &&
parsedURL.scheme != "https" &&
!(getEnv("_NIX_IN_TEST").has_value() && parsedURL.scheme == "file"))
throw Error({
.msg = HintFmt("'fetchClosure' only supports http:// and https:// stores"),
});
if (!parsedURL.query.empty())
throw Error({
.msg = HintFmt("'fetchClosure' does not support URL query parameters (in '%s')", *fromStoreUrl),
});
auto fromStore = state.aio.blockOn(openStore(parsedURL.to_string()));
if (toPath)
runFetchClosureWithRewrite(state, *fromStore, *fromPath, *toPath, v);
else if (inputAddressed)
runFetchClosureWithInputAddressedPath(state, *fromStore, *fromPath, v);
else
runFetchClosureWithContentAddressedPath(state, *fromStore, *fromPath, v);
}
}
+8 -1
View File
@@ -5,7 +5,7 @@
namespace nix {
void prim_fetchMercurial(EvalState & state, Value ** args, Value & v)
static void prim_fetchMercurial(EvalState & state, Value * * args, Value & v)
{
std::string url;
std::optional<Hash> rev;
@@ -102,4 +102,11 @@ void prim_fetchMercurial(EvalState & state, Value ** args, Value & v)
state.ctx.paths.allowPath(tree.storePath);
}
static RegisterPrimOp r_fetchMercurial({
.name = "fetchMercurial",
.arity = 1,
.fun = prim_fetchMercurial
});
}
+3 -5
View File
@@ -48,12 +48,10 @@ struct PrintOptions
bool force = false;
/**
* Determines at what depth derivations start getting printed as paths.
* When `force` is true and our print depth is greater than or equal to
* this value, print derivations as `«derivation /nix/store/...»`
* instead of as attribute sets.
* If true and `force` is set, print derivations as
* `«derivation /nix/store/...»` instead of as attribute sets.
*/
size_t derivationPathDepth = std::numeric_limits<size_t>::max();
bool derivationPaths = false;
/**
* If true, track which values have been printed and skip them on
+3 -19
View File
@@ -281,9 +281,7 @@ private:
void printAttrs(Value & v, size_t depth)
{
bool shouldSimplifyDerivations = options.force && depth >= options.derivationPathDepth;
bool isDerivation = state.isDerivation(v);
if (shouldSimplifyDerivations && isDerivation) {
if (options.force && options.derivationPaths && state.isDerivation(v)) {
printDerivation(v);
} else if (seen && !v.attrs()->empty() && !seen->insert(v.attrs()).second) {
printRepeated();
@@ -306,7 +304,7 @@ private:
for (auto & i : sorted) {
printSpace(prettyPrint);
if (attrsPrinted >= options.maxAttrs ) {
if (attrsPrinted >= options.maxAttrs) {
printElided(sorted.size() - printedHere, "attribute", "attributes");
break;
}
@@ -326,21 +324,6 @@ private:
}
output << " = ";
// Elide repeated drvAttrs attribute.
if (!shouldSimplifyDerivations && isDerivation && i.first == "drvAttrs") {
state.forceValue(i.second->value, noPos);
if (i.second->value.type() == ValueType::nAttrs) {
printElided(i.second->value.attrs()->size(), "attribute", "attributes");
} else {
print(i.second->value, depth + 1);
}
output << ";";
attrsPrinted++;
printedHere++;
continue;
}
print(i.second->value, depth + 1);
output << ";";
attrsPrinted++;
@@ -599,6 +582,7 @@ public:
seen.reset();
}
ValuesSeen seen;
print(v, 0);
}
};
-10
View File
@@ -1,10 +0,0 @@
---
name: abort-on-warn
internalName: abortOnWarn
type: bool
default: false
---
If set to true, [`builtins.warn`](@docroot@/language/builtins.md#builtins-warn) will throw an error when logging a warning.
This will give you a stack trace that leads to the location of the warning.
This is useful for finding information about warnings in third-party Nix code when you can not start the interactive debugger, such as when Nix is called from a non-interactive script. See [`debugger-on-warn`](#conf-debugger-on-warn).
Currently, a stack trace can only be produced when the debugger is enabled, or when evaluation is aborted.
+1 -1
View File
@@ -1,6 +1,6 @@
---
name: debugger-on-trace
internalName: debuggerOnTrace
internalName: builtinsTraceDebugger
type: bool
default: false
---
-9
View File
@@ -1,9 +0,0 @@
---
name: debugger-on-warn
internalName: debuggerOnWarn
type: bool
default: false
---
If set to true and the `--debugger` flag is given, [`builtins.warn`](@docroot@/language/builtins.md#builtins-warn) will enter the debugger like [`builtins.break`](@docroot@/language/builtins.md#builtins-break).
This is useful for debugging warnings in third-party Nix code.
Use [`debugger-on-trace`](#conf-debugger-on-trace) to also enter the debugger on legacy warnings that are logged with [`builtins.trace`](@docroot@/language/builtins.md#builtins-trace).
+4
View File
@@ -544,6 +544,10 @@ public:
/// lazy and/or partial application of a function.
Value(app_t, EvalMemory & mem, Value & lhs, std::span<Value> args);
/// Constructs a nix language value of type "lambda", which represents a
/// lazy and/or partial application of a function.
Value(app_t, EvalMemory & mem, const Value & lhs, std::span<Value> baseArgs, std::span<Value> moreArgs);
/// Constructs a nix language value of type "external", which is only used
/// by plugins. Do any existing plugins even use this mechanism?
Value(external_t, ExternalValueBase & external)
+7 -29
View File
@@ -4,7 +4,6 @@
#include "lix/libstore/store-api.hh"
#include "lix/libstore/gc-store.hh"
#include "lix/libutil/c-calls.hh"
#include "lix/libutil/logging.hh"
#include "lix/libutil/result.hh"
#include "lix/libutil/signals.hh"
#include "lix/libmain/loggers.hh"
@@ -33,6 +32,8 @@
namespace nix {
char * * savedArgv;
static bool gcWarning = true;
void printGCWarning()
@@ -93,8 +94,6 @@ try {
std::vector<const StorePath *> willSubstituteSorted = {};
std::for_each(willSubstitute.begin(), willSubstitute.end(),
[&](const StorePath &p) { willSubstituteSorted.push_back(&p); });
// NOTE: this sort uses a total order, so the iteration over pointers is not an issue
// NOLINTNEXTLINE(bugprone-nondeterministic-pointer-iteration-order)
std::sort(willSubstituteSorted.begin(), willSubstituteSorted.end(),
[](const StorePath *lhs, const StorePath *rhs) {
if (lhs->name() == rhs->name())
@@ -404,34 +403,13 @@ RunPager::~RunPager()
}
}
PrintFreed::~PrintFreed()
{
// When in dry-run mode, print the paths on stdout
if (action == GCOptions::gcReturnLive || action == GCOptions::gcReturnDead) {
for (auto & i : results.paths) {
logger->cout("%s", i);
};
}
switch (action) {
case GCOptions::gcReturnLive: {
notice("%1% store paths would be kept\n", results.paths.size());
break;
}
case GCOptions::gcReturnDead: {
notice("%1% store paths would be deleted\n", results.paths.size());
break;
}
case GCOptions::gcDeleteDead:
case GCOptions::gcDeleteSpecific:
case GCOptions::gcTryDeleteSpecific: {
notice(
"%1% store paths deleted, %2% freed\n",
if (show)
std::cout << fmt("%d store paths deleted, %s freed\n",
results.paths.size(),
showBytes(results.bytesFreed)
);
break;
}
}
showBytes(results.bytesFreed));
}
}
+3 -8
View File
@@ -1,7 +1,6 @@
#pragma once
///@file
#include "lix/libstore/gc-store.hh"
#include "lix/libutil/args.hh"
#include "lix/libutil/args/root.hh"
#include "lix/libmain/common-args.hh"
@@ -100,14 +99,10 @@ struct GCResults;
struct PrintFreed
{
GCOptions::GCAction action;
bool show;
const GCResults & results;
PrintFreed(GCOptions::GCAction action, const GCResults & results)
: action(action)
, results(results)
{
}
PrintFreed(bool show, const GCResults & results)
: show(show), results(results) { }
~PrintFreed();
};
+1
View File
@@ -225,6 +225,7 @@ try {
auto * buildIdDir = std::get_if<nar_index::Directory>(&narIndex);
for (auto subdir : { "lib", "debug", ".build-id" }) {
if (!buildIdDir) break;
// get returns nullptr subdir does not exist, and std::get_if propagates it.
buildIdDir = std::get_if<nar_index::Directory>(get(buildIdDir->contents, subdir));
}
+8 -1
View File
@@ -139,7 +139,14 @@ Goal::WorkResult DerivationGoal::timedOut(Error && ex)
kj::Promise<Result<Goal::WorkResult>> DerivationGoal::workImpl() noexcept
{
KJ_DEFER({ actLock.reset(); });
// always clear the slot token, no matter what happens. not doing this
// can cause builds to get stuck on exceptions (or other early exits).
// ideally we'd use scoped slot tokens instead of keeping them in some
// goal member variable, but we cannot do this yet for legacy reasons.
KJ_DEFER({
actLock.reset();
slotToken = {};
});
BOOST_OUTCOME_CO_TRY(auto result, co_await (useDerivation ? getDerivation() : haveDerivation()));
result.storePath = drvPath;
+3 -1
View File
@@ -246,7 +246,7 @@ struct DerivationGoal : public Goal
WorkResult timedOut(Error && ex);
kj::Promise<Result<WorkResult>> workImpl() noexcept override;
kj::Promise<Result<WorkResult>> workImpl() noexcept override final;
/**
* Add wanted outputs to an already existing derivation goal.
@@ -307,6 +307,8 @@ struct DerivationGoal : public Goal
virtual void cleanupPostOutputsRegisteredModeNonCheck();
protected:
AsyncSemaphore::Token slotToken;
kj::TimePoint lastChildActivity = kj::minValue;
kj::Promise<Result<WorkResult>> wrapChildHandler(kj::Promise<Result<WorkResult>> handler
-6
View File
@@ -22,12 +22,6 @@ kj::Promise<void> Goal::waitForAWhile()
kj::Promise<Result<Goal::WorkResult>> Goal::work() noexcept
try {
// always clear the slot token, no matter what happens. not doing this
// can cause builds to get stuck on exceptions (or other early exist).
// ideally we'd use scoped slot tokens instead of keeping them in some
// goal member variable, but we cannot do this yet for legacy reasons.
KJ_DEFER({ slotToken = {}; });
BOOST_OUTCOME_CO_TRY(auto result, co_await workImpl());
trace("done");
-3
View File
@@ -82,9 +82,6 @@ struct Goal
*/
std::string name;
protected:
AsyncSemaphore::Token slotToken;
public:
struct [[nodiscard]] WorkResult {
ExitCode exitCode;
+10 -16
View File
@@ -227,14 +227,9 @@ retry:
}
auto & localStore = getLocalStore();
bool sandboxFallbackAllowed = true;
if (localStore.config().storeDir != localStore.config().realStoreDir.get()) {
#if __linux__
if (!useChroot) {
printTaggedWarning("auto-enabling the sandbox due to using a diverted store");
}
useChroot = true;
sandboxFallbackAllowed = false;
#else
throw Error("building using a diverted store is not supported on this platform");
#endif
@@ -268,9 +263,7 @@ retry:
if (!mountAndPidNamespacesSupported()) {
if (!settings.sandboxFallback)
throw Error("this system does not support the kernel namespaces that are required for sandboxing; use '--no-sandbox' to disable sandboxing. Pass --debug for diagnostics on what is broken.");
if (!sandboxFallbackAllowed)
throw Error("Sandboxing is enabled due to using a diverted store, but this system does not support the kernel namespaces that are required. Pass --debug for diagnostics on what is broken.");
printTaggedWarning("auto-disabling sandboxing because the prerequisite namespaces are not available");
debug("auto-disabling sandboxing because the prerequisite namespaces are not available");
useChroot = false;
}
@@ -1832,15 +1825,18 @@ try {
);
}
outputGraph[scratchOutputs.at(name)] = StorePathSet{};
std::visit(
overloaded{/* Since we'll use the already installed versions of these, we
can treat them as leaves and ignore any references they
have. */
[&](const AlreadyRegistered &) {
outputGraph[scratchOutputs.at(name)] = StorePathSet{};
},
[&](const AlreadyRegistered &) {},
[&](const PerhapsNeedToRegister & refs) {
outputGraph[scratchOutputs.at(name)] = refs.refs;
for (auto & ref : refs.refs) {
if (inverseOutputMap.find(ref) != inverseOutputMap.end()) {
outputGraph[scratchOutputs.at(name)].insert(ref);
}
}
}
},
*orifu
@@ -1851,10 +1847,8 @@ try {
topoSort(outputsToSort, {[&](const std::string & name) {
StringSet dependencies;
for (auto & path : outputGraph.at(scratchOutputs.at(name))) {
auto outputName = inverseOutputMap.find(path);
if (outputName != inverseOutputMap.end()) {
dependencies.insert(outputName->second);
}
auto outputName = inverseOutputMap.at(path);
dependencies.insert(outputName);
}
return dependencies;
}});
+39 -78
View File
@@ -27,13 +27,6 @@ PathSubstitutionGoal::PathSubstitutionGoal(
maintainExpectedSubstitutions = worker.expectedSubstitutions.addTemporarily(1);
}
PathSubstitutionGoal::~PathSubstitutionGoal()
{
cleanup();
}
Goal::WorkResult PathSubstitutionGoal::done(
ExitCode result,
BuildResult::Status status,
@@ -76,8 +69,6 @@ kj::Promise<Result<Goal::WorkResult>> PathSubstitutionGoal::tryNext() noexcept
try {
trace("trying next substituter");
cleanup();
if (subs.size() == 0) {
/* None left. Terminate this goal and let someone else deal
with it. */
@@ -205,61 +196,36 @@ kj::Promise<Result<Goal::WorkResult>> PathSubstitutionGoal::tryToRun() noexcept
try {
trace("trying to run");
if (!slotToken.valid()) {
slotToken = co_await worker.substitutions.acquire();
}
maintainRunningSubstitutions = worker.runningSubstitutions.addTemporarily(1);
auto pipe = kj::newPromiseAndCrossThreadFulfiller<void>();
outPipe = kj::mv(pipe.fulfiller);
thr = std::async(std::launch::async, [this]() {
AsyncIoRoot aio;
/* Wake up the worker loop when we're done. */
Finally updateStats([this]() { outPipe->fulfill(); });
auto & fetchPath = subPath ? *subPath : storePath;
try {
ReceiveInterrupts receiveInterrupts;
auto act = logger->startActivity(
actSubstitute, Logger::Fields{worker.store.printStorePath(storePath), sub->getUri()}
);
aio.blockOn(copyStorePath(
*sub,
worker.store,
fetchPath,
repair,
sub->config().isTrusted ? NoCheckSigs : CheckSigs,
&act
));
} catch (const EndOfFile &) {
throw EndOfFile(
"NAR for '%s' fetched from '%s' is incomplete",
sub->printStorePath(fetchPath),
sub->getUri()
);
}
});
co_await pipe.promise;
co_return co_await finished();
} catch (...) {
co_return result::current_exception();
}
kj::Promise<Result<Goal::WorkResult>> PathSubstitutionGoal::finished() noexcept
try {
trace("substitute finished");
auto & fetchPath = subPath ? *subPath : storePath;
do {
try {
slotToken = {};
thr.get();
break;
try {
AsyncSemaphore::Token slotToken = co_await worker.substitutions.acquire();
auto act = logger->startActivity(
actSubstitute,
Logger::Fields{worker.store.printStorePath(storePath), sub->getUri()}
);
maintainRunningSubstitutions = worker.runningSubstitutions.addTemporarily(1);
TRY_AWAIT(copyStorePath(
*sub,
worker.store,
fetchPath,
repair,
sub->config().isTrusted ? NoCheckSigs : CheckSigs,
&act
));
break;
} catch (const EndOfFile &) {
throw EndOfFile(
"NAR for '%s' fetched from '%s' is incomplete",
sub->printStorePath(fetchPath),
sub->getUri()
);
}
} catch (std::exception & e) { // NOLINT(lix-foreign-exceptions)
printError("%1%", Uncolored(e.what()));
@@ -271,10 +237,20 @@ try {
substituterFailed = true;
}
}
/* Try the next substitute. */
/* Try the next substitute */
co_return co_await tryNext();
} while (false);
co_return co_await finished();
} catch (...) {
co_return result::current_exception();
}
kj::Promise<Result<Goal::WorkResult>> PathSubstitutionGoal::finished() noexcept
try {
trace("substitute finished");
worker.markContentsGood(storePath);
printMsg(lvlChatty, "substitution of path '%s' succeeded", worker.store.printStorePath(storePath));
@@ -294,19 +270,4 @@ try {
} catch (...) {
co_return result::current_exception();
}
void PathSubstitutionGoal::cleanup()
{
try {
if (thr.valid()) {
// FIXME: signal worker thread to quit.
thr.get();
}
} catch (...) {
ignoreExceptionInDestructor();
}
}
}
-9
View File
@@ -48,11 +48,6 @@ struct PathSubstitutionGoal : public Goal
*/
kj::Own<kj::CrossThreadPromiseFulfiller<void>> outPipe;
/**
* The substituter thread.
*/
std::future<void> thr;
/**
* Whether to try to repair a valid path.
*/
@@ -85,7 +80,6 @@ public:
RepairFlag repair = NoRepair,
std::optional<ContentAddress> ca = std::nullopt
);
~PathSubstitutionGoal();
kj::Promise<Result<WorkResult>> workImpl() noexcept override;
@@ -97,9 +91,6 @@ public:
kj::Promise<Result<WorkResult>> tryToRun() noexcept;
kj::Promise<Result<WorkResult>> finished() noexcept;
/* Called by destructor, can't be overridden */
void cleanup() override final;
JobCategory jobCategory() const override {
return JobCategory::Substitution;
};
+15 -30
View File
@@ -1093,6 +1093,15 @@ struct curlFileTransfer : public FileTransfer
co_return result::current_exception();
}
void throwChangedTarget(std::string_view what, std::string_view from, std::string_view to)
{
if (!from.empty() && from != to) {
throw FileTransferError(
Misc, {}, "uri %s changed %s from %s to %s during transfer", uri, what, from, to
);
}
}
kj::Promise<Result<void>> prepareRetry(
const std::string & context,
const std::chrono::milliseconds & waitTime,
@@ -1104,7 +1113,7 @@ struct curlFileTransfer : public FileTransfer
"%s; retrying from offset %d in %d ms (attempt %d/%d)",
Uncolored(context),
totalReceived,
waitTime.count(),
waitTime,
Uncolored(attempt),
Uncolored(tries)
);
@@ -1112,7 +1121,7 @@ struct curlFileTransfer : public FileTransfer
printTaggedWarning(
"%s; retrying in %d ms (attempt %d/%d)",
Uncolored(context),
waitTime.count(),
waitTime,
Uncolored(attempt),
Uncolored(tries)
);
@@ -1126,41 +1135,17 @@ struct curlFileTransfer : public FileTransfer
kj::Promise<Result<void>> restartTransfer(const std::chrono::milliseconds & timeout)
try {
auto onChange =
[&](std::string_view what, std::string_view from, std::string_view to, bool throw_
) -> void {
if (!from.empty() && from != to) {
FileTransferError e = FileTransferError(
Misc,
{},
"uri %s changed %s from %s to %s during transfer",
uri,
what,
from,
to
);
if (throw_) {
throw e;
}
logWarning(e.info());
}
};
// use the effective URI of the previous transfer for retries. this avoids
// some silent corruption if a redirect changes between starting and retry
const auto & uri = metadata.effectiveUri.empty() ? this->uri : metadata.effectiveUri;
auto newMeta = TRY_AWAIT(startTransfer(uri, timeout, totalReceived));
onChange("final destination", metadata.effectiveUri, newMeta.effectiveUri, false);
onChange("ETag", metadata.etag, newMeta.etag, true);
onChange(
throwChangedTarget("final destination", metadata.effectiveUri, newMeta.effectiveUri);
throwChangedTarget("ETag", metadata.etag, newMeta.etag);
throwChangedTarget(
"immutable url",
metadata.immutableUrl.value_or(""),
newMeta.immutableUrl.value_or(""),
true
newMeta.immutableUrl.value_or("")
);
co_return result::success();
} catch (...) {
-1
View File
@@ -18,7 +18,6 @@
#include <kj/async.h>
#include <queue>
#include <regex>
#include <thread>
#include <errno.h>
#include <fcntl.h>
+43 -327
View File
@@ -1,17 +1,48 @@
#include "lix/libstore/machines.hh"
#include "lix/libstore/globals.hh"
#include "lix/libstore/store-api.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/strings.hh"
#include <numeric>
#include <algorithm>
#include <toml.hpp>
namespace nix {
Machine::Machine(decltype(storeUri) storeUri,
decltype(systemTypes) systemTypes,
decltype(sshKey) sshKey,
decltype(maxJobs) maxJobs,
decltype(speedFactor) speedFactor,
decltype(supportedFeatures) supportedFeatures,
decltype(mandatoryFeatures) mandatoryFeatures,
decltype(sshPublicHostKey) sshPublicHostKey) :
storeUri(
// Backwards compatibility: if the URI is schemeless, is not a path,
// and is not one of the special store connection words, prepend
// ssh://.
storeUri.find("://") != std::string::npos
|| storeUri.find("/") != std::string::npos
|| storeUri == "auto"
|| storeUri == "daemon"
|| storeUri == "local"
|| storeUri.starts_with("auto?")
|| storeUri.starts_with("daemon?")
|| storeUri.starts_with("local?")
|| storeUri.starts_with("?")
? storeUri
: "ssh://" + storeUri),
systemTypes(systemTypes),
sshKey(sshKey),
maxJobs(maxJobs),
speedFactor(speedFactor == 0.0f ? 1.0f : speedFactor),
supportedFeatures(supportedFeatures),
mandatoryFeatures(mandatoryFeatures),
sshPublicHostKey(sshPublicHostKey)
{
if (speedFactor < 0.0)
throw UsageError("speed factor must be >= 0");
}
bool Machine::systemSupported(const std::string & system) const
{
return system == "builtin" || (systemTypes.count(system) > 0);
@@ -40,7 +71,6 @@ try {
StoreConfig::Params storeParams;
if (storeUri.starts_with("ssh://")) {
// Remote builds become flakey, when having more than one ssh connection
storeParams["max-connections"] = "1";
}
@@ -70,8 +100,6 @@ try {
co_return result::current_exception();
}
namespace machines_legacy_parsing {
static std::vector<std::string> expandBuilderLines(const std::string & builders)
{
std::vector<std::string> result;
@@ -138,43 +166,15 @@ static Machine parseBuilderLine(const std::string & line)
if (!isSet(0))
throw FormatError("bad machine specification: store URL was not found at the first column of a row: '%s'", line);
auto storeUri = tokens[0];
// Backwards compatibility: if the URI is schemeless, is not a path,
// and is not one of the special store connection words, prepend
// ssh://.
storeUri = storeUri.find("://") != std::string::npos || storeUri.find("/") != std::string::npos
|| storeUri == "auto" || storeUri == "daemon" || storeUri == "local"
|| storeUri.starts_with("auto?") || storeUri.starts_with("daemon?")
|| storeUri.starts_with("local?") || storeUri.starts_with("?")
? storeUri
: "ssh://" + storeUri;
auto systemTypes = isSet(1) ? tokenizeString<std::set<std::string>>(tokens[1], ",")
: std::set<std::string>{settings.thisSystem};
auto sshKey = isSet(2) ? tokens[2] : "";
auto maxJobs = isSet(3) ? parseUnsignedIntField(3) : 1U;
auto speedFactor = isSet(4) ? parseFloatField(4) : 1.0f;
auto supportedFeatures =
isSet(5) ? tokenizeString<std::set<std::string>>(tokens[5], ",") : std::set<std::string>{};
auto mandatoryFeatures =
isSet(6) ? tokenizeString<std::set<std::string>>(tokens[6], ",") : std::set<std::string>{};
auto sshPublicHostKey = isSet(7) ? ensureBase64(7) : "";
speedFactor = speedFactor == 0.0f ? 1.0f : speedFactor;
if (speedFactor < 0.0) {
throw UsageError("speed factor must be >= 0");
}
return {
storeUri,
systemTypes,
sshKey,
maxJobs,
speedFactor,
supportedFeatures,
mandatoryFeatures,
sshPublicHostKey
tokens[0],
isSet(1) ? tokenizeString<std::set<std::string>>(tokens[1], ",") : std::set<std::string>{settings.thisSystem},
isSet(2) ? tokens[2] : "",
isSet(3) ? parseUnsignedIntField(3) : 1U,
isSet(4) ? parseFloatField(4) : 1.0f,
isSet(5) ? tokenizeString<std::set<std::string>>(tokens[5], ",") : std::set<std::string>{},
isSet(6) ? tokenizeString<std::set<std::string>>(tokens[6], ",") : std::set<std::string>{},
isSet(7) ? ensureBase64(7) : ""
};
}
@@ -192,287 +192,3 @@ Machines getMachines()
}
}
namespace machines_toml_parsing {
static constexpr int MIN_VERSION = 1;
static constexpr int LATEST_VERSION = 1;
// Toml format:
// [[machines.andesite]]
// uri = "..."
//
// [[machines.diorite]]
// ...
template<typename T>
static toml::result<T, std::string> parse(const toml::value & data, const std::string & key)
{
try {
return toml::success(toml::get<T>(data.at(key)));
} catch (toml::type_error & e) { // NOLINT(lix-foreign-exceptions)
return toml::failure<std::string>({e.what()});
} catch (std::out_of_range & _) { // NOLINT(lix-foreign-exceptions)
const auto ei =
toml::make_error_info(fmt("%s must be present", key), data, "but was not set");
return toml::failure(toml::format_error(ei));
}
}
template<typename T>
static toml::result<T, std::string>
parse(const toml::value & data, const std::string & key, T defaultValue)
{
if (!data.contains(key)) {
return toml::success(defaultValue);
}
try {
return toml::success(toml::get<T>(data.at(key)));
} catch (toml::type_error & e) { // NOLINT(lix-foreign-exceptions)
// invalid value
return toml::failure<std::string>({e.what()});
}
}
static const std::set<std::string> EXPECTED_KEYS = {
"uri",
"system-types",
"ssh-key",
"jobs",
"speed-factor",
"supported-features",
"mandatory-features",
"ssh-public-host-key",
"enable",
};
static toml::result<float, std::string> getSpeedFactor(const toml::value & data)
{
if (data.contains("speed-factor")) {
auto sf = data.at("speed-factor");
if (sf.is_integer()) {
return toml::success(static_cast<float>(sf.as_integer()));
}
if (sf.is_floating()) {
return toml::success(static_cast<float>(sf.as_floating()));
}
return toml::failure(toml::format_error(toml::make_error_info(
"bad_cast to floating for `speed-factor`", sf, "Was neither an integer nor a float"
)));
}
return toml::success(1.0f);
}
static toml::result<Machine, std::vector<std::string>> parseMachine(const toml::value & data)
{
std::vector<std::string> errs;
if (!data.is_table()) {
errs.push_back(toml::format_error(toml::make_error_info(
"Each machine must be a table", data, "This should be a table. Did you mean `.uri = `?"
)));
return toml::failure(errs);
}
// parsing
auto storeUri = parse<std::string>(data, "uri");
auto systemTypes = parse<std::vector<std::string>>(
data, "system-types", std::vector<std::string>{settings.thisSystem}
);
auto sshKey = parse<std::string>(data, "ssh-key", "");
auto maxJobs = parse<int>(data, "jobs", 1U);
auto speedFactor = getSpeedFactor(data);
auto supportedFeatures =
parse<std::vector<std::string>>(data, "supported-features", std::vector<std::string>{});
auto mandatoryFeatures =
parse<std::vector<std::string>>(data, "mandatory-features", std::vector<std::string>{});
auto sshPublicHostKey = parse<std::string>(data, "ssh-public-host-key", "");
// parsing validation
if (storeUri.is_err()) {
errs.push_back(storeUri.as_err());
}
if (systemTypes.is_err()) {
errs.push_back(systemTypes.as_err());
}
if (sshKey.is_err()) {
errs.push_back(sshKey.as_err());
}
if (maxJobs.is_err()) {
errs.push_back(maxJobs.as_err());
}
if (speedFactor.is_err()) {
errs.push_back(speedFactor.as_err());
}
if (supportedFeatures.is_err()) {
errs.push_back(supportedFeatures.as_err());
}
if (mandatoryFeatures.is_err()) {
errs.push_back(mandatoryFeatures.as_err());
}
if (sshPublicHostKey.is_err()) {
errs.push_back(sshPublicHostKey.as_err());
}
// value validation
if (maxJobs.is_ok() && maxJobs.as_ok() < 0) {
auto ei =
toml::make_error_info("jobs must be >= 0", data.at("jobs"), "but got negative value");
errs.push_back(toml::format_error(ei));
}
if (speedFactor.is_ok() && speedFactor.as_ok() < 0.0) {
auto ei = toml::make_error_info(
"speed factor must be >= 0", data.at("speed-factor"), "but got negative value"
);
errs.push_back(toml::format_error(ei));
}
for (const auto & [key, _] : data.as_table()) {
if (!EXPECTED_KEYS.contains(key)) {
errs.push_back(toml::format_error(toml::make_error_info(
fmt("unexpected key `%s`", key), data.at(key), "should not be present"
)));
}
}
if (!errs.empty()) {
return toml::failure(errs);
}
return toml::success<Machine>({
storeUri.unwrap(),
std::set(systemTypes.unwrap().begin(), systemTypes.unwrap().end()),
sshKey.unwrap(),
static_cast<unsigned>(maxJobs.unwrap()),
speedFactor.unwrap(),
std::set(supportedFeatures.unwrap().begin(), supportedFeatures.unwrap().end()),
std::set(mandatoryFeatures.unwrap().begin(), mandatoryFeatures.unwrap().end()),
base64Encode(sshPublicHostKey.unwrap()),
});
}
static toml::result<Machines, std::vector<std::string>> parseToml(const toml::value & data)
{
auto const array_name = "machines";
std::vector<std::string> parserErrors;
Machines machines;
// Empty config
if (data.size() == 0) {
return toml::success<Machines>({});
}
if (!data.is_table()) {
parserErrors.push_back(
"Top level must be a table. This should never throw as this is required by the toml "
"SPEC"
);
return toml::failure(parserErrors);
}
if (auto config_version = parse<int>(data, "version", LATEST_VERSION); config_version.is_err())
{
parserErrors.push_back(config_version.as_err());
} else if (config_version.as_ok() < MIN_VERSION || config_version.as_ok() > LATEST_VERSION) {
parserErrors.push_back(
fmt("Unable to parse Machines of version %d, only versions between %d and %d are "
"supported.",
config_version.as_ok(),
MIN_VERSION,
LATEST_VERSION)
);
}
if (!parserErrors.empty()) {
return toml::failure(parserErrors);
}
auto & tbl = data.as_table();
std::string unexpected_keys;
for (auto it = tbl.begin(); it != tbl.end(); ++it) {
if (it->first == array_name || it->first == "version") {
// expected keys
continue;
}
unexpected_keys += ", " + it->first;
}
if (unexpected_keys.size()) {
parserErrors.push_back(fmt("unexpected keys found: %s", unexpected_keys.erase(0, 2)));
}
if (!data.at(array_name).is_table()) {
parserErrors.push_back(
fmt("Expected key `%s` to be a table of name -> machine configurations", array_name)
);
return toml::failure(parserErrors);
}
for (const auto & [name, machine] : data.at(array_name).as_table()) {
auto const res = parseMachine(machine);
if (res.is_err()) {
auto err = res.as_err();
parserErrors.push_back(fmt("for machine %s:", name));
parserErrors.insert(parserErrors.end(), err.begin(), err.end());
continue;
}
auto enable = parse<bool>(machine, "enable", true);
if (enable.is_ok()) {
if (enable.unwrap()) {
// Check if it hasn't been statically disabled
// But still throw parsing errors if it was
machines.push_back(res.unwrap());
}
} else {
parserErrors.push_back(enable.as_err());
}
}
if (!parserErrors.empty()) {
return toml::failure(parserErrors);
}
return toml::success(machines);
}
static std::optional<Machines> getMachines()
{
toml::value data;
auto buildersStr = settings.builders.get();
try {
if (buildersStr.size() > 0 && buildersStr.at(0) == '@') {
data = toml::parse(buildersStr.substr(1));
} else {
data = toml::parse_str(settings.builders);
}
} catch (toml::syntax_error const & e) { // NOLINT(lix-foreign-exceptions)
if (toLower(buildersStr).contains("toml") || buildersStr.contains("\"")) {
// Yes, we are sure this is a TOML and no this shitty legacy format
// so we can safely throw the syntax error here
throw UsageError(fmt("invalid Machines TOML syntax: \n%s", e.what()));
}
return {};
} catch (toml::file_io_error const & _) { // NOLINT(lix-foreign-exceptions)
// sadly we have to do this otherwise we break the old format,
// which requires **silently ignoring** invalid files
return {};
}
auto const fromToml = parseToml(data);
if (fromToml.is_ok()) {
return fromToml.unwrap();
}
auto const & errs = fromToml.as_err();
std::string msg = "invalid Machines TOML:\n";
msg += concatStringsSep("\n", errs);
throw UsageError(msg);
}
}
Machines getMachines()
{
auto const toml_result = machines_toml_parsing::getMachines();
if (toml_result.has_value()) {
return toml_result.value();
}
debug("Trying again with legacy format");
return machines_legacy_parsing::getMachines();
}
}
+9
View File
@@ -41,6 +41,15 @@ struct Machine {
*/
bool mandatoryMet(const std::set<std::string> & features) const;
Machine(decltype(storeUri) storeUri,
decltype(systemTypes) systemTypes,
decltype(sshKey) sshKey,
decltype(maxJobs) maxJobs,
decltype(speedFactor) speedFactor,
decltype(supportedFeatures) supportedFeatures,
decltype(mandatoryFeatures) mandatoryFeatures,
decltype(sshPublicHostKey) sshPublicHostKey);
kj::Promise<Result<std::pair<ref<Store>, Pipe>>> openStore() const;
};
+1 -1
View File
@@ -265,7 +265,7 @@ static std::map<StorePath, Node> mkGraph(
for (auto & node : graph_data) {
for (auto & ref : node.second.dependencies) {
graph_data.find(ref)->second.dependents.insert(node.first);
graph_data.at(ref).dependents.insert(node.first);
}
}
+1 -1
View File
@@ -858,7 +858,7 @@ void LinuxLocalDerivationGoal::prepareSandbox()
if (buildUser && (buildUser->getUIDCount() != 1 || settings.useCgroups)) {
context.cgroup.emplace(
settings.nixStateDir + "/cgroups",
fmt("nix-build-drvHash-%s", drvPath.hashPart()),
fmt("nix-build-uid-%d", buildUser->getUID()),
buildUser->getUID(),
buildUser->getGID()
);
+1 -4
View File
@@ -1537,10 +1537,7 @@ try {
try {
(*lk)->push_back(TRY_AWAIT(openStore(uri)));
} catch (Error & e) {
logWarning(
{.msg = HintFmt("Failed to setup the substituter at URI '%s': %s", uri, e.msg())
}
);
logWarning(e.info());
}
}
+1 -1
View File
@@ -384,7 +384,7 @@ struct Parser
buffer.clear(); \
std::move(str); \
})
#define READ_STRING() READ_STRING_LIMITED(std::numeric_limits<size_t>::max())
#define READ_STRING() READ_STRING_LIMITED(1048576)
#define READ_PADDING(size) \
do { \
if ((size) % 8) { \
@@ -1,6 +0,0 @@
---
name: nix-path-shadow
internalName: NixPathShadow
---
Allows shadowing `<nix/fetchurl.nix>` by configuration of the [*nix path*](@docroot@/language/builtin-constants.html#builtins-nixPath) to a value containing `nix=/some/path`.
@@ -0,0 +1,5 @@
---
name: fetch-closure
internalName: FetchClosure
---
Enable the use of the [`fetchClosure`](@docroot@/language/builtins.md#builtins-fetchClosure) built-in function in the Nix language.
+1 -1
View File
@@ -158,6 +158,7 @@ experimental_feature_definitions = files(
'experimental-features/cgroups.md',
'experimental-features/coerce-integers.md',
'experimental-features/daemon-trust-override.md',
'experimental-features/fetch-closure.md',
'experimental-features/flake-self-attrs.md',
'experimental-features/flakes.md',
'experimental-features/lix-custom-sub-commands.md',
@@ -172,7 +173,6 @@ deprecated_feature_definitions = files(
# keep-sorted start
'deprecated-features/ancient-let.md',
'deprecated-features/cr-line-endings.md',
'deprecated-features/nix-path-shadow.md',
'deprecated-features/nul-bytes.md',
'deprecated-features/rec-set-overrides.md',
'deprecated-features/shadow-internal-symbols.md',
-8
View File
@@ -11,14 +11,6 @@
#include <span>
#include <stdint.h> // IWYU pragma: keep (this is used literally everywhere)
// NOTE: This check is load-bearing, c.f.
// https://git.lix.systems/lix-project/lix/commit/96a36833084f66b79ba778e719862b8219557773
// It is put here so that every consumer of our headers are prohibited from
// compiling with NDEBUG as some asserts are used in inline code
#ifdef NDEBUG
#error "Lix may not be built with assertions disabled (i.e. with -DNDEBUG)."
#endif
namespace nix {
typedef std::list<std::string> Strings;
+2 -18
View File
@@ -73,24 +73,8 @@ Path getHome()
Path getCacheDir()
{
// We follow systemd semantics here:
// https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html#RuntimeDirectory=
static auto cacheDir = [] {
auto userCacheDir = getEnv("XDG_CACHE_HOME");
auto serviceCacheDir = getEnv("CACHE_DIRECTORY");
if (serviceCacheDir) {
return *serviceCacheDir;
}
if (userCacheDir) {
return *userCacheDir;
}
return getHome() + "/.cache";
}();
return cacheDir;
auto cacheDir = getEnv("XDG_CACHE_HOME");
return cacheDir ? *cacheDir : getHome() + "/.cache";
}
+41 -48
View File
@@ -55,6 +55,7 @@
#endif
static constexpr int SUBDAEMON_CONNECTION_FD = 0;
static constexpr int SUBDAEMON_SETTINGS_FD = 3;
namespace nix {
@@ -345,6 +346,9 @@ try {
peer.pidKnown ? fmt("pid %1%", peer.pid) : "unknown peer"
);
Pipe settings;
settings.create();
// Fork a child to handle the connection. make sure it's called with
// argv0 `nix-daemon` so we don't try to run `nix --for` when called
// from more modern scripts that assume nix-command being available.
@@ -359,7 +363,11 @@ try {
fmt("%1%", int(verbosity)),
},
.dieWithParent = false,
.redirections = {{.dup = SUBDAEMON_CONNECTION_FD, .from = remote.get()}}
.redirections =
{
{.dup = SUBDAEMON_CONNECTION_FD, .from = remote.get()},
{.dup = SUBDAEMON_SETTINGS_FD, .from = settings.readSide.get()},
}
};
if (forceTrustClientOpt) {
options.args.push_back(
@@ -367,6 +375,15 @@ try {
);
}
runProgram2(options).release();
FdSink sink(settings.writeSide.get());
std::map<std::string, Config::SettingInfo> overriddenSettings;
globalConfig.getSettings(overriddenSettings, true);
for (auto & setting : overriddenSettings) {
sink << 1 << setting.first << setting.second.value;
}
sink << 0;
sink.flush();
} catch (Error & error) {
auto ei = error.info();
// FIXME: add to trace?
@@ -380,37 +397,12 @@ try {
co_return result::current_exception();
}
static void
daemonInstance(AsyncIoRoot & aio, std::optional<TrustedFlag> forceTrustClientOpt, char * peerPidArg)
static void daemonInstance(AsyncIoRoot & aio, std::optional<TrustedFlag> forceTrustClientOpt)
{
// Handle socket-based activation by systemd.
const auto [launchedByManager, connectionFd] = []() -> std::pair<bool, int> {
auto listenFds = getEnv("LISTEN_FDS");
if (listenFds) {
if (getEnv("LISTEN_PID") != std::to_string(getpid()) || listenFds != "1") {
throw Error("unexpected systemd environment variables");
}
closeOnExec(SD_LISTEN_FDS_START);
return {true, SD_LISTEN_FDS_START};
} else {
return {false, SUBDAEMON_CONNECTION_FD};
}
}();
PeerInfo peer = getPeerInfo(connectionFd);
PeerInfo peer = getPeerInfo(SUBDAEMON_CONNECTION_FD);
TrustedFlag trusted;
std::string user;
// replace peerPidArg contents with the peer pid if possible. the forking daemon does
// this as a debugging aid and it is easy enough to do it here also, so we just do it
if (peerPidArg && peer.pidKnown) {
auto pidForArgv = std::to_string(peer.pid);
if (pidForArgv.size() <= strlen(peerPidArg)) {
memset(peerPidArg, ' ', strlen(peerPidArg));
strcpy(peerPidArg, pidForArgv.c_str());
}
}
if (forceTrustClientOpt) {
trusted = *forceTrustClientOpt;
} else {
@@ -425,8 +417,23 @@ daemonInstance(AsyncIoRoot & aio, std::optional<TrustedFlag> forceTrustClientOpt
forceTrustClientOpt ? " by override" : ""
);
{
FdSource source(SUBDAEMON_SETTINGS_FD);
/* Read the parent's settings. */
while (readNum<unsigned>(source)) {
auto name = readString(source);
auto value = readString(source);
settings.set(name, value);
}
if (close(SUBDAEMON_SETTINGS_FD) < 0) {
throw SysError("preparing subdaemon connection");
}
}
// Background the daemon.
if (!launchedByManager && setsid() == -1) {
if (setsid() == -1) {
throw SysError("creating a new session");
}
@@ -439,8 +446,8 @@ daemonInstance(AsyncIoRoot & aio, std::optional<TrustedFlag> forceTrustClientOpt
}
// Handle the connection.
FdSource from(connectionFd);
FdSink to(connectionFd);
FdSource from(SUBDAEMON_CONNECTION_FD);
FdSink to(SUBDAEMON_CONNECTION_FD);
processConnection(aio, store, from, to, trusted);
}
@@ -518,14 +525,12 @@ runDaemon(AsyncIoRoot & aio, bool stdio, std::optional<TrustedFlag> forceTrustCl
}
}
static int
main_nix_daemon(AsyncIoRoot & aio, std::string programName, Strings argv, std::span<char *> rawArgv)
static int main_nix_daemon(AsyncIoRoot & aio, std::string programName, Strings argv)
{
{
auto stdio = false;
std::optional<TrustedFlag> isTrustedOpt = std::nullopt;
bool isInstance = false;
char * peerPidArg = nullptr;
Verbosity subdaemonLogLevel = lvlInfo;
LegacyArgs(aio, programName, [&](Strings::iterator & arg, const Strings::iterator & end) {
@@ -549,18 +554,6 @@ main_nix_daemon(AsyncIoRoot & aio, std::string programName, Strings argv, std::s
} else if (*arg == "--for") {
isInstance = true;
getArg(*arg, arg, end);
} else if (*arg == "--for-socket-activation") {
isInstance = true;
// HACK: too many copies and rewrites happen by the time we get here to
// be able to calculate a rawArgv offset. instead we will search for an
// exact match and blindly assume that it's the one we want to rewrite.
for (auto [i, rawArg] : enumerate(rawArgv)) {
if (rawArg == *arg) {
peerPidArg = rawArg + strlen("--for-");
peerPidArg[-1] = ' ';
break;
}
}
} else if (*arg == "--log-level") {
if (auto level = string2Int<int>(getArg(*arg, arg, end)); level) {
subdaemonLogLevel = static_cast<Verbosity>(std::min<int>(lvlVomit, *level));
@@ -575,7 +568,7 @@ main_nix_daemon(AsyncIoRoot & aio, std::string programName, Strings argv, std::s
if (isInstance) {
verbosity = Verbosity(std::min<uint64_t>(subdaemonLogLevel, lvlVomit));
daemonInstance(aio, isTrustedOpt, peerPidArg);
daemonInstance(aio, isTrustedOpt);
} else {
runDaemon(aio, stdio, isTrustedOpt);
}
@@ -585,7 +578,7 @@ main_nix_daemon(AsyncIoRoot & aio, std::string programName, Strings argv, std::s
}
void registerLegacyNixDaemon() {
LegacyCommandRegistry::addWithRaw("nix-daemon", main_nix_daemon);
LegacyCommandRegistry::add("nix-daemon", main_nix_daemon);
}
struct CmdDaemon : StoreCommand
-8
View File
@@ -257,10 +257,6 @@ try {
output.second = DerivationOutput::InputAddressed { .path = StorePath::dummy };
drv.env[output.first] = "";
}
// outputs default to [ "out" ]
drv.env.try_emplace("outputs", "out");
auto hashesModulo = TRY_AWAIT(hashDerivationModulo(*evalStore, drv, true));
for (auto & output : drv.outputs) {
@@ -356,10 +352,6 @@ struct Common : InstallableCommand, MixProfile
buildEnvironment.toBash(out, ignoreVars);
// Add NIX_SHELL_LEVEL
out << "printf -v NIX_SHELL_LEVEL \"%d\" \"${NIX_SHELL_LEVEL:-0}\" || NIX_SHELL_LEVEL=0\n";
out << "export NIX_SHELL_LEVEL=$(( NIX_SHELL_LEVEL + 1 ))\n";
for (auto & var : savedVars)
out << fmt("%s=\"$%s${nix_saved_%s:+:$nix_saved_%s}\"\n", var, var, var, var);
+1 -1
View File
@@ -110,7 +110,7 @@ struct CmdEval : MixJSON, InstallableCommand, MixReadOnlyOption
v,
PrintOptions{
.force = true,
.derivationPathDepth = 0,
.derivationPaths = true,
.errors = ErrorPrintBehavior::ThrowTopLevel,
}
)
+8 -9
View File
@@ -455,6 +455,8 @@ void registerNixHelp()
int mainWrapped(AsyncIoRoot & aio, int argc, char ** argv)
{
savedArgv = argv;
/* The chroot helper needs to be run before any threads have been
started. */
if (argc > 0 && argv[0] == chrootHelperName) {
@@ -491,19 +493,14 @@ int mainWrapped(AsyncIoRoot & aio, int argc, char ** argv)
registerLegacyCommands();
auto legacy = (*LegacyCommandRegistry::commands)[programName];
if (legacy) {
return legacy(
aio,
std::string(baseNameOf(argv[0])),
Strings(argv + 1, argv + argc),
{argv + 1, argv + argc}
);
return legacy(aio, std::string(baseNameOf(argv[0])), Strings(argv + 1, argv + argc));
}
}
evalSettings.pureEval.setDefault(true);
setLogFormat(LogFormat::barWithLogs);
setLogFormat(LogFormat::bar);
settings.verboseBuild = false;
// FIXME: stop messing about with log verbosity depending on if it is interactive use
if (isatty(STDERR_FILENO)) {
verbosity = lvlNotice;
@@ -524,7 +521,8 @@ int mainWrapped(AsyncIoRoot & aio, int argc, char ** argv)
if (argc == 2 && std::string(argv[1]) == "__dump-language") {
experimentalFeatureSettings.experimentalFeatures.override(ExperimentalFeatures{}
| Xp::Flakes);
| Xp::Flakes
| Xp::FetchClosure);
evalSettings.pureEval.override(false);
Evaluator state(aio, {}, aio.blockOn(openStore("dummy://")));
auto res = JSON::object();
@@ -648,6 +646,7 @@ int mainWrapped(AsyncIoRoot & aio, int argc, char ** argv)
return 0;
}
}
int main(int argc, char * * argv)
+1 -2
View File
@@ -12,7 +12,6 @@ struct CmdMakeContentAddressed : virtual CopyCommand, virtual StorePathsCommand,
CmdMakeContentAddressed()
{
realiseMode = Realise::Outputs;
requireStore = false;
}
std::string description() override
@@ -29,7 +28,7 @@ struct CmdMakeContentAddressed : virtual CopyCommand, virtual StorePathsCommand,
void run(ref<Store> srcStore, StorePaths && storePaths) override
{
auto dstStore = getDstStore();
auto dstStore = aio().blockOn(dstUri.empty() ? openStore() : openStore(dstUri));
auto remappings = aio().blockOn(makeContentAddressed(*srcStore, *dstStore,
StorePathSet(storePaths.begin(), storePaths.end())));
+1 -13
View File
@@ -150,17 +150,6 @@ struct CmdShell : InstallablesCommand, MixEnvironment
auto unixPathString = concatStringsSep(":", unixPath);
(void) sys::setenv("PATH", unixPathString, 1);
(void) sys::setenv("IN_NIX_SHELL", ignoreEnvironment ? "pure" : "impure", 1);
// Set NIX_SHELL_LEVEL
(void) sys::setenv(
"NIX_SHELL_LEVEL",
std::to_string(
getEnvNonEmpty("NIX_SHELL_LEVEL")
.and_then([](std::string lvl) { return string2Int<size_t>(lvl); })
.value_or(0)
+ 1
),
1
);
Strings args;
for (auto & arg : command) args.push_back(arg);
@@ -241,7 +230,6 @@ void registerNixRun()
void chrootHelper(int argc, char * * argv)
{
#if __linux__
int p = 1;
std::string storeDir = argv[p++];
std::string realStoreDir = argv[p++];
@@ -251,6 +239,7 @@ void chrootHelper(int argc, char * * argv)
while (p < argc)
args.push_back(argv[p++]);
#if __linux__
uid_t uid = getuid();
uid_t gid = getgid();
@@ -321,7 +310,6 @@ void chrootHelper(int argc, char * * argv)
throw SysError("unable to exec '%s'", cmd);
#else
std::string storeDir = argv[1];
throw Error("mounting the Nix store on '%s' is not supported on this platform", storeDir);
#endif
}
+1 -1
View File
@@ -58,7 +58,7 @@ struct CmdStoreDelete : StorePathsCommand
}
GCResults results;
PrintFreed freed(options.action, results);
PrintFreed freed(true, results);
aio().blockOn(gcStore.collectGarbage(options, results));
}
};
+1 -1
View File
@@ -40,7 +40,7 @@ struct CmdStoreGC : StoreCommand, MixDryRun
options.action = dryRun ? GCOptions::gcReturnDead : GCOptions::gcDeleteDead;
GCResults results;
PrintFreed freed(options.action, results);
PrintFreed freed(options.action == GCOptions::gcDeleteDead, results);
aio().blockOn(gcStore.collectGarbage(options, results));
}
};
+13
View File
@@ -49,6 +49,19 @@ struct CmdUpgradeNix : MixDryRun, EvalCommand
});
}
// NOTE(Raito): we override the store creation
// to prevent any store daemon connection.
//
// An upgrade, by nature, requires a direct store access
// to avoid having the daemon die in the middle of changing the binary.
//
// If more commands needs that, we can move it into a mixin. This was deliberately not done
// here.
virtual ref<Store> createStore(AsyncIoRoot & aio) override
{
return aio.blockOn(openStore(settings.storeUri.get(), {}, AllowDaemon::Disallow));
}
/**
* This command is stable before the others
*/
+5 -3
View File
@@ -68,10 +68,12 @@ class AuthorInfoDB:
def format_link(ident: str, gh_part: str, fj_part: str) -> str:
# FIXME: deprecate github as default
if ident.isdigit():
ident = f"lix#{ident}"
if ident.startswith("gh#"):
raise ValueError('Unprefixed issue numbers are disallowed until the Lix 2.95 cycle, when they will become Forgejo references instead of their previous state of CppNix GitHub issues.'
'\nIf you mean a Lix Forgejo reference, prefix with fj# or lix#'
'\nIf you mean a CppNix issue, prefix the number with gh# or nix#')
elif ident.startswith("gh#"):
num, link, base = int(ident[3:]), ident, f"{GH_REPO_BASE}/{gh_part}"
elif ident.startswith("nix#"):
num, link, base = int(ident[4:]), ident, f"{GH_REPO_BASE}/{gh_part}"
+5 -2
View File
@@ -350,6 +350,11 @@ if lowdown.version().version_compare('>= 1.4.0')
add_project_arguments('-DLOWDOWN_SEPARATE_TERM_OPTS', language: 'cpp')
endif
# TODO(sterni): drop the corresponding #ifdef after NixOS 25.11 is EOL which still distributes lowdown < 3.0.0
if lowdown.version().version_compare('>= 3.0.0')
add_project_arguments('-DLOWDOWN_CONSOLIDATED_OFLAGS', language: 'cpp')
endif
# HACK(Qyriad): rapidcheck's pkg-config doesn't include the libs lol
# Note: technically we 'check' for rapidcheck twice, for the internal-api-docs handling above,
# but Meson will cache the result of the first one, and the required : arguments are different.
@@ -586,8 +591,6 @@ add_project_arguments(
'-Wdeprecated-copy',
'-Wignored-qualifiers',
'-Werror=suggest-override',
# Undefined behavior.
'-Werror=embedded-directive',
# don't let nlohmann automatically cast enums to ints. we don't set this
# inside libutil headers to not force this behavior on users of libutil.
'-DJSON_DISABLE_ENUM_SERIALIZATION=1',
+16 -28
View File
@@ -1,42 +1,30 @@
#!/usr/bin/env python3
import subprocess
import argparse
def get_targets_of_rule(build_root: str, rule_name: str) -> list[str]:
return (
subprocess.check_output(["ninja", "-C", build_root, "-t", "targets", "rule", rule_name])
.decode()
.strip()
.splitlines()
)
return subprocess.check_output(['ninja', '-C', build_root, '-t', 'targets', 'rule', rule_name]).decode().strip().splitlines()
def ninja_build(build_root: str, targets: list[str]):
subprocess.check_call(["ninja", "-C", build_root, "--", *targets])
subprocess.check_call(['ninja', '-C', build_root, '--', *targets])
def main():
ap = argparse.ArgumentParser(description="Builds required targets for clang-tidy")
ap.add_argument("build_root", help="Ninja build root", type=str)
import argparse
ap = argparse.ArgumentParser(description='Builds required targets for clang-tidy')
ap.add_argument('build_root', help='Ninja build root', type=str)
args = ap.parse_args()
targets = (
[t for t in get_targets_of_rule(args.build_root, "CUSTOM_COMMAND") if t.endswith(".gen.hh")]
+ [
t
for t in get_targets_of_rule(args.build_root, "CUSTOM_COMMAND_DEP")
if t.endswith(".capnp.h")
]
+ [
t
for t in get_targets_of_rule(args.build_root, "CUSTOM_COMMAND")
if t.endswith(".gen.inc")
]
)
targets = [
t for t in get_targets_of_rule(args.build_root, 'CUSTOM_COMMAND')
if t.endswith('.gen.hh')
] + [
t for t in get_targets_of_rule(args.build_root, 'CUSTOM_COMMAND_DEP')
if t.endswith('.capnp.h')
] + [
t for t in get_targets_of_rule(args.build_root, 'CUSTOM_COMMAND')
if t.endswith('.gen.inc')
]
ninja_build(args.build_root, targets)
if __name__ == "__main__":
if __name__ == '__main__':
main()
+44 -46
View File
@@ -12,80 +12,78 @@ because the run-clang-tidy UX is so questionable.
# https://github.com/mesonbuild/meson/issues/1564
import multiprocessing
import subprocess
import os
import sys
from pathlib import Path
import argparse
def default_concurrency() -> int:
return min(multiprocessing.cpu_count(), int(os.environ.get("NIX_BUILD_CORES", "16")))
def default_concurrency():
return min(multiprocessing.cpu_count(),
int(os.environ.get("NIX_BUILD_CORES", "16")))
def go(
exe: str,
plugin_path: Path,
compile_commands_json_dir: Path,
jobs: int,
paths: list[Path],
werror: bool,
fix: bool,
):
def go(exe: str, plugin_path: Path, compile_commands_json_dir: Path, jobs: int,
paths: list[Path], werror: bool, fix: bool):
args = [
# XXX(Jade): This explicitly invokes it with python because of a nixpkgs bug
# XXX: This explicitly invokes it with python because of a nixpkgs bug
# where clang-unwrapped does not patch interpreters in run-clang-tidy.
# However, making clang-unwrapped depend on python is also silly, so idk.
sys.executable,
exe,
"-quiet",
"-load",
'-quiet',
'-load',
plugin_path,
"-p",
'-p',
compile_commands_json_dir,
"-j",
'-j',
str(jobs),
"-header-filter",
r"lix/[^/]+/.*\.hh",
'-header-filter',
r'lix/[^/]+/.*\.hh'
]
if werror:
args += ["-warnings-as-errors", "*"]
args += ['-warnings-as-errors', '*']
if fix:
args += ["-fix"]
args += ["--"]
args += ['-fix']
args += ['--']
args += paths
os.execvp(sys.executable, args)
def main():
ap = argparse.ArgumentParser(description="Runs run-clang-tidy for you")
import argparse
ap = argparse.ArgumentParser(description='Runs run-clang-tidy for you')
ap.add_argument('--jobs',
'-j',
type=int,
default=default_concurrency(),
help='Parallel linting jobs to run')
ap.add_argument('--plugin-path',
type=Path,
help='Path to the Lix clang-tidy plugin')
# FIXME: maybe we should integrate this so it just fixes the compdb for you and throws it in a tempdir?
ap.add_argument(
"--jobs", "-j", type=int, default=default_concurrency(), help="Parallel linting jobs to run"
)
ap.add_argument("--plugin-path", type=Path, help="Path to the Lix clang-tidy plugin")
# FIXME(Jade): maybe we should integrate this so it just fixes the compdb for you and throws it in a tempdir?
ap.add_argument(
"--compdb-path",
'--compdb-path',
type=Path,
help="Path to the directory containing the fixed-up compilation database from clean_compdb",
help=
'Path to the directory containing the fixed-up compilation database from clean_compdb'
)
ap.add_argument("--werror", action="store_true", help="Warnings get turned into errors")
ap.add_argument("--fix", action="store_true", help="Apply fixes for warnings")
ap.add_argument(
"--run-clang-tidy-path", default="run-clang-tidy", help="Path to run-clang-tidy"
)
ap.add_argument("paths", nargs="*", help="Source paths to check")
ap.add_argument('--werror',
action='store_true',
help='Warnings get turned into errors')
ap.add_argument('--fix',
action='store_true',
help='Apply fixes for warnings')
ap.add_argument('--run-clang-tidy-path',
default='run-clang-tidy',
help='Path to run-clang-tidy')
ap.add_argument('paths', nargs='*', help='Source paths to check')
args = ap.parse_args()
go(
args.run_clang_tidy_path,
args.plugin_path,
args.compdb_path,
args.jobs,
args.paths,
args.werror,
args.fix,
)
go(args.run_clang_tidy_path, args.plugin_path, args.compdb_path, args.jobs,
args.paths, args.werror, args.fix)
if __name__ == "__main__":
if __name__ == '__main__':
main()
+19 -12
View File
@@ -5,20 +5,18 @@
import json
import shlex
import argparse
def process_compdb(compdb: list[dict]) -> list[dict]:
def munch_command(args: list[str]) -> list[str]:
out = []
eat_next = False
for i, arg in enumerate(args):
if arg in ["-fpch-preprocess", "-fpch-instantiate-templates"]:
if arg in ['-fpch-preprocess', '-fpch-instantiate-templates']:
# -fpch-preprocess as used with gcc, -fpch-instantiate-templates as used by clang
continue
if arg == "-include-pch" or (
arg == "-include" and args[i + 1] == "precompiled-headers.hh"
):
elif arg == '-include-pch' or (arg == '-include' and args[i + 1] == 'precompiled-headers.hh'):
# -include-pch some-pch (clang), or -include some-pch (gcc)
eat_next = True
continue
@@ -29,25 +27,34 @@ def process_compdb(compdb: list[dict]) -> list[dict]:
def chomp(item: dict) -> dict:
item = item.copy()
item["command"] = shlex.join(munch_command(shlex.split(item["command"])))
item['command'] = shlex.join(munch_command(shlex.split(item['command'])))
return item
def cmdfilter(item: dict) -> bool:
file = item["file"]
return not file.endswith("precompiled-headers.hh") and not file.endswith(".rs")
file = item['file']
return (
not file.endswith('precompiled-headers.hh')
and not file.endswith('.rs')
)
return [chomp(x) for x in compdb if cmdfilter(x)]
def main():
ap = argparse.ArgumentParser(description="Delete pch arguments from compilation database")
ap.add_argument("input", type=argparse.FileType("r"), help="Input json file")
ap.add_argument("output", type=argparse.FileType("w"), help="Output json file")
import argparse
ap = argparse.ArgumentParser(
description='Delete pch arguments from compilation database')
ap.add_argument('input',
type=argparse.FileType('r'),
help='Input json file')
ap.add_argument('output',
type=argparse.FileType('w'),
help='Output json file')
args = ap.parse_args()
input_json = json.load(args.input)
json.dump(process_compdb(input_json), args.output, indent=2)
if __name__ == "__main__":
if __name__ == '__main__':
main()
-13
View File
@@ -1,13 +0,0 @@
diff --git i/include/rapidcheck/Gen.hpp w/include/rapidcheck/Gen.hpp
index 0c48004..2eb143f 100644
--- i/include/rapidcheck/Gen.hpp
+++ w/include/rapidcheck/Gen.hpp
@@ -2,6 +2,8 @@
#include <cassert>
+#include <exception>
+
#include "rapidcheck/detail/Any.h"
#include "rapidcheck/detail/ImplicitParam.h"
#include "rapidcheck/gen/detail/GenerationHandler.h"
+1 -1
View File
@@ -1,4 +1,4 @@
foreach config : [ 'nix-daemon.socket', 'nix-daemon.service', 'nix-daemon@.service' ]
foreach config : [ 'nix-daemon.socket', 'nix-daemon.service' ]
configure_file(
input : config + '.in',
output : config,
+3 -2
View File
@@ -1,7 +1,6 @@
[Unit]
Description=Nix Daemon
Documentation=man:nix-daemon https://docs.lix.systems/manual/lix/stable
Conflicts=nix-daemon.socket
RequiresMountsFor=@storedir@
RequiresMountsFor=@localstatedir@
RequiresMountsFor=@localstatedir@/nix/db
@@ -9,9 +8,11 @@ ConditionPathIsReadWrite=@localstatedir@/nix/daemon-socket
[Service]
ExecStart=@@bindir@/nix-daemon nix-daemon --daemon
CacheDirectory=nix
KillMode=process
LimitNOFILE=1048576
TasksMax=1048576
Delegate=yes
DelegateSubgroup=supervisor
[Install]
WantedBy=multi-user.target

Some files were not shown because too many files have changed in this diff Show More