Compare commits

..
Author SHA1 Message Date
eldritch horrors e0beb9b118 [WIP] remove boost coroutine uses
we no longer need these since sinkToSource and sourceToSink are gone.

Change-Id: Ibbf440e2cf71bf3e9f3b833af2d78a21fb1b3193
2024-06-30 21:21:58 +02:00
eldritch horrors f274bfddad [WIP] remove sinkToSource and related boehm patch
Change-Id: I1379841299713175d0225b82a67f50660f9eb5e2
2024-06-30 21:21:58 +02:00
eldritch horrors d08787e42e [WIP] remove remaining sinkToSource uses
Change-Id: Id1ee0d2ad4a3774f4bbb960d76f0f76ac4f3eff9
2024-06-30 21:21:58 +02:00
eldritch horrors 0a1520c602 [WIP] generatorize protocol serializers
this is cursed. deeply and profoundly cursed. under NO CIRCUMSTANCES
must protocol serializer helpers be applied to temporaries! doing so
will inevitably cause dangling references and cause the entire thing
to crash. we need to do this even so to get rid of boost coroutines,
and likewise to encapsulate the serializers we suffer today at least
a little bit to allow a gradual migration to an actual IPC protocol.

Change-Id: I2921ba451e04d86798752d140885d3c5cc08e146
2024-06-30 21:21:58 +02:00
eldritch horrors 5264d72846 [WIP] narFromPath as generator
Change-Id: Ifa783c2c65c06ddd1d0212016d5bfd07666ea91c
2024-06-30 21:21:58 +02:00
eldritch horrors 8fb3b89d0f [WIP] libstore: remove a sinkToSouce from old daemon protocol
this doesn't have a test because this code path is only reached by
clients that predate 2.4, and we really should not be caring about
those any more right now. even the test suite doesn't, and the few
tests that might care are disabled because they will not even work

Change-Id: Id9eb190065138fedb2c7d90c328ff9eb9d97385b
2024-06-30 21:21:58 +02:00
eldritch horrors 425f3e6964 [WIP] libstore: rewrite the nar parser as a contents generator
this is not completely necessary at this point because the parser right
now already returns a generator to pass through all input data it read,
but the nar parser *was* very lax and would accept nars that weren't in
canonical form (defined as the form dumpPath would return). nar hashing
depends on these things, and as such rewriting the parser now allows us
to reject non-canonical nars that extract to the same store contents as
their canonical counterpart, but have different nar hashes.

gcc 12 is broken and causes double frees when yielding certain types of
move-only values, so this also pins gcc to 13 for the default stdenv in
23.11-based builds.

Change-Id: Iccd319e3bd5912d8297014c84c495edc59019bb7
2024-06-30 21:21:58 +02:00
eldritch horrors e11945065b libstore: turn copyNAR into a generator
Change-Id: Id452f6a03faa1037ff13af0f63e32883966ff40d
2024-06-30 21:21:58 +02:00
eldritch horrors 2112a39d2e libstore: turn the NAR parser into a passthrough generator
this will let us turn copyNAR into a generator as well, which in turn is
necessary to turn the users of copyNAR into generators without resorting
to sinkToSource coroutines. currently this uses the SerializingTransform
in all cases, even for copyNAR where it is not necessary. should this be
a performance problem we can easily swap out the transform for one which
does not produce any bytes of its own, but that should not be necessary.

Change-Id: I7e685879318fcbb78d8b88abfddd7752360eb0ce
2024-06-30 21:21:58 +02:00
eldritch horrors 48e3637aa3 libutil: remove sourceToSink
Change-Id: I2867ba2b3a7b59af44c1ec51b08329fccba41ca1
2024-06-30 21:21:58 +02:00
eldritch horrors d658cf5a0e libutil: remove makeDecompressionSink
Change-Id: I4258227b5dbbb735a75b477d8a57007bfca305e9
2024-06-30 21:21:58 +02:00
eldritch horrors 93344a1379 libstore: make BinaryCacheStore::getFile return a source
this lets us remove the last true remaining uses of
makeDecompressionSink.

Change-Id: I146ca2bbe1a9ae9a367117a7b8a304b23a63e5e2
2024-06-30 21:21:58 +02:00
eldritch horrors 9f769794d9 libutil: rewrite RewritingSink as source
the rewriting sink was just broken. when given a rewrite set that
contained a key that is also a proper infix of another key it was
possible to produce an incorrectly rewritten result if the writer
used the wrong block size. fixing this duplicates rewriteStrings,
to avoid this we'll rewrite rewriteStrings to use RewritingSource
in a new mode that'll allow rewrites we had previously forbidden.

Change-Id: I57fa0a9a994e654e11d07172b8e31d15f0b7e8c0
2024-06-30 21:21:58 +02:00
eldritch horrors d1230c7218 libutil: remove RewritingSink match/size tracking
size tracking can be done with a LengthSink and a tee. match tracking
was defeated by never having done any match tracking, all users would
see the same (empty) set of matches at all times. match tracking with
bytes offsets alone would not be sufficient in the general case, only
because computeHashModulo uses a single rewrite would it have worked.

Change-Id: Idb214b5222e0ea24f450f5505712a342b63d7570
2024-06-30 21:21:45 +02:00
eldritch horrors 50e1364a80 libutil: turn HashModuloSink into a free function
Change-Id: I5878007502fa68c2816a0f4c61f7d0e60bdde702
2024-06-30 21:08:17 +02:00
eldritch horrors 3e29a98903 libutil: return sources from runProgram2
this much more closely mimics what is actually happening: we're reading
data from somewhere else, actively, rather than passively waiting. with
the data flow matching the underlying system interactions better we can
remove a few sinkToSource calls that merely exists to undo the mismatch
caused by not treating subprocess output as a data source to begin with

Change-Id: If4abfc2f8398fb5e88c9b91a8bdefd5504bb2d11
2024-06-30 21:05:06 +02:00
eldritch horrors 14d539a274 libutil: return a program handle from runProgram2
this will let us also return a source for the program output later,
which will in turn make sinkToSource unnecessary for program output
processing. this may also reopen a path for provigin program input,
but that still needs a proper async io framework to avoid problems.

Change-Id: Iaf93f47db99c38cfaf134bd60ed6a804d7ddf688
2024-06-30 20:58:02 +02:00
eldritch horrors a5ae26726b enable -Werror=unused-result
we do not have any of these warnings appearing at the moment, but it
seems like a good idea to enable [[nodiscard]] checking anyway. once
we start introducing more functions with must-use conditions we will
need such checking, and the rust stdlib has proven them very useful.

Change-Id: Ibb6b042ae1ec5f527f8dc2809a7816a4c1548ae2
2024-06-30 20:51:10 +02:00
eldritch horrors 5f826bc600 libutil: convert readFileSource to a generator
Change-Id: I5f92b15fd367d46eb047d74ab6e317b4f51a46d3
2024-06-30 19:29:07 +02:00
eldritch horrors f659e93c5b libstore: convert dumpPath to a generator
Change-Id: Ic4cf5562504aa29130304469936f958c0426e5ef
2024-06-30 19:29:07 +02:00
eldritch horrors 1540bdbcfc libutil: allow construction of sources from generators
Change-Id: I78ff8d0720f06bce731e26d5e1c53b1382bbd589
2024-06-30 19:29:07 +02:00
eldritch horrors 379a892afb libutil: begin porting serialization to generators
generators are a better basis for serializers than streaming into sinks
as we do currently for many reasons, such as being usable as sources if
one wishes to (without requiring an intermediate sink to serialize full
data sets into memory, or boost coroutines to turn sinks into sources),
composing more naturally (as one can just yield a sub-generator instead
of being forced to wrap entire substreams into clunky functions or even
more clunky custom types to implement operator<< on), allowing wrappers
to transform data with clear ownership semantics (removing the need for
explicit memory allocations and Source wrappers), and many other things

Change-Id: I361d89ff556354f6930d9204f55117565f2f7f20
2024-06-30 19:29:06 +02:00
eldritch horrors 5f18b52274 libutil: convert drainFD to a Bytes generator
the `*Source` name is a slight misnomer since we do also have a
Source type, but we can probably live with this for time being.

Change-Id: I54eb2e59a4009014e324797f16b80b962759c7d3
2024-06-30 17:38:11 +02:00
eldritch horrors d96601fe55 libutil: allow draining Generator<Bytes> into sinks
Change-Id: I442d03a5399096d4baca9a2618b4c4b64db36c4b
2024-06-30 17:33:28 +02:00
eldritch horrors 2521f52465 libutil: add Bytes type
not used anywhere yet, but we'll use this a lot soon for generators that
return file contents, wire protocol fragments, or indeed any byte stream

Change-Id: I01a46f9bf9d75aaf4a5d7662773b99f498862a28
2024-06-30 17:30:34 +02:00
eldritch horrors 11307e04d1 libutil: generator type with on-yield value mapping
this will be the basis of non-boost coroutines in lix. anything that is
a boost coroutine *should* be representable with a Generator coroutine,
and many things that are not currently boost coroutines but behave much
like one (such as, notably, serializers) should be as well. this allows
us to greatly simplify many things that look like iteration but aren't.

Change-Id: I2cebcefa0148b631fb30df4c8cfa92167a407e34
2024-06-30 17:23:41 +02:00
354 changed files with 4023 additions and 9389 deletions
-20
View File
@@ -16,23 +16,3 @@ Checks:
- -bugprone-unchecked-optional-access
# many warnings, seems like a questionable lint
- -bugprone-branch-clone
# extremely noisy before clang 19: https://github.com/llvm/llvm-project/issues/93959
- -bugprone-multi-level-implicit-pointer-conversion
# all thrown exceptions must derive from std::exception
- hicpp-exception-baseclass
# capturing async lambdas are dangerous
- cppcoreguidelines-avoid-capturing-lambda-coroutines
# crimes must be appropriately declared as crimes
- cppcoreguidelines-pro-type-cstyle-cast
- lix-*
# This can not yet be applied to Lix itself since we need to do source
# reorganization so that lix/ include paths work.
- -lix-fixincludes
# This lint is included as an example, but the lib function it replaces is
# already gone.
- -lix-hasprefixsuffix
CheckOptions:
bugprone-reserved-identifier.AllowedIdentifiers: '__asan_default_options'
bugprone-unused-return-value.AllowCastToVoid: true
+1 -1
View File
@@ -26,4 +26,4 @@ See our [Hacking guide](https://git.lix.systems/lix-project/lix/src/branch/main/
## License
Lix is released under [LGPL-2.1-or-later](./COPYING).
Lix is released under the [LGPL v2.1](./COPYING).
@@ -34,7 +34,6 @@ private:
OptionalFileEntryRef File,
StringRef SearchPath, StringRef RelativePath,
const Module *Imported,
bool ModuleImported,
SrcMgr::CharacteristicKind FileType) override;
};
@@ -47,7 +46,7 @@ void FixIncludesCallbacks::LexedFileChanged(FileID, LexedFileChangeReason,
void FixIncludesCallbacks::InclusionDirective(
SourceLocation, const Token &, StringRef FileName, bool IsAngled,
CharSourceRange FilenameRange, OptionalFileEntryRef File, StringRef,
StringRef, const Module *, bool, SrcMgr::CharacteristicKind) {
StringRef, const Module *, SrcMgr::CharacteristicKind) {
if (Ignore)
return;
@@ -2,7 +2,6 @@
#include <clang-tidy/ClangTidyModuleRegistry.h>
#include "FixIncludes.hh"
#include "HasPrefixSuffix.hh"
#include "CharPtrCast.hh"
namespace nix::clang_tidy {
using namespace clang;
@@ -13,7 +12,6 @@ class NixClangTidyChecks : public ClangTidyModule {
void addCheckFactories(ClangTidyCheckFactories &CheckFactories) override {
CheckFactories.registerCheck<HasPrefixSuffixCheck>("lix-hasprefixsuffix");
CheckFactories.registerCheck<FixIncludesCheck>("lix-fixincludes");
CheckFactories.registerCheck<CharPtrCastCheck>("lix-charptrcast");
}
};
+13
View File
@@ -0,0 +1,13 @@
project('lix-clang-tidy', ['cpp', 'c'],
version : '0.1',
default_options : ['warning_level=3', 'cpp_std=c++20'])
llvm = dependency('Clang', version: '>= 14', modules: ['libclang'])
sources = files(
'HasPrefixSuffix.cc',
'LixClangTidyChecks.cc',
'FixIncludes.cc',
)
shared_module('lix-clang-tidy', sources,
dependencies: llvm)
+1 -1
View File
@@ -20,7 +20,7 @@ OUTPUT_DIRECTORY = @docdir@
# for a project that appears at the top of each page and should give viewer a
# quick idea about the purpose of the project. Keep the description short.
PROJECT_BRIEF = "Lix: A modern, delicious implementation of the Nix package manager; unstable internal interfaces"
PROJECT_BRIEF = "Nix, the purely functional package manager; unstable internal interfaces"
# If the GENERATE_LATEX tag is set to YES, doxygen will generate LaTeX output.
# The default value is: YES.
-26
View File
@@ -44,11 +44,6 @@ cole-h:
display_name: Cole Helbling
github: cole-h
delan:
display_name: delan
forgejo: delan
github: delan
edolstra:
display_name: Eelco Dolstra
github: edolstra
@@ -57,11 +52,6 @@ ericson:
display_name: John Ericson
github: ericson2314
goldstein:
display_name: goldstein
forgejo: goldstein
github: GoldsteinE
horrors:
display_name: eldritch horrors
forgejo: pennae
@@ -70,17 +60,10 @@ horrors:
iFreilicht:
github: iFreilicht
isabelroses:
forgejo: isabelroses
github: isabelroses
jade:
forgejo: jade
github: lf-
kjeremy:
github: kjeremy
kloenk:
forgejo: kloenk
github: kloenk
@@ -103,20 +86,11 @@ midnightveil:
ncfavier:
github: ncfavier
piegames:
display_name: piegames
forgejo: piegames
github: piegamesde
puck:
display_name: puck
forgejo: puck
github: puckipedia
quantumjump:
display_name: Quantum Jump
github: QuantumBJump
r-vdp:
github: r-vdp
+1 -1
View File
@@ -345,7 +345,7 @@ const redirects = {
"linux": "uninstall.html#linux",
"macos": "uninstall.html#macos",
"uninstalling": "uninstall.html",
},
}
"contributing/hacking.html": {
"nix-with-flakes": "#building-nix-with-flakes",
"classic-nix": "#building-nix",
+12
View File
@@ -0,0 +1,12 @@
---
synopsis: "Add a `build-dir` setting to set the backing directory for builds"
cls: 1514
credits: [roberth, tomberek]
category: Improvements
---
`build-dir` can now be set in the Nix configuration to choose the backing directory for the build sandbox.
This can be useful on systems with `/tmp` on tmpfs, or simply to relocate large builds to another disk.
Also, `XDG_RUNTIME_DIR` is no longer considered when selecting the default temporary directory,
as it's not intended to be used for large amounts of data.
@@ -0,0 +1,14 @@
---
synopsis: Add log formats `multiline` and `multiline-with-logs`
cls: [1369]
credits: [kloenk]
category: Improvements
---
Added two new log formats (`multiline` and `multiline-with-logs`) that display
current activities below each other for better visibility.
These formats attempt to use the maximum available lines
(defaulting to 25 if unable to determine) and print up to that many lines.
The status bar is displayed as the first line, with each subsequent
activity on its own line.
+12
View File
@@ -0,0 +1,12 @@
---
synopsis: "`nix copy` is now several times faster at `querying info about /nix/store/...`"
cls: [1462]
issues: [fj#366]
credits: [jade]
category: Fixes
---
We fixed a locking bug that serialized `querying info about /nix/store/...`
onto just one thread such that it was eating `O(paths to copy * latency)` time
while setting up to copy paths to s3 and other stores. It is now `nproc` times
faster.
@@ -0,0 +1,21 @@
---
synopsis: "Lix no longer speaks the Nix remote-build worker protocol to clients or servers older than CppNix 2.3"
cls: [1207, 1208, 1206, 1205, 1204, 1203, 1479]
issues: [fj#325]
credits: [jade]
category: Breaking Changes
---
CppNix 2.3 was released in 2019, and is the new oldest supported version. We
will increase our support baseline in the future up to a final version of CppNix
2.18 (which may happen soon given that it is the only still-packaged and thus
still-tested >2.3 version), but this step already removes a significant amount
of dead, untested, code paths.
Lix speaks the same version of the protocol as CppNix 2.18 and that fact will
never change in the future; the Lix plans to replace the protocol for evolution
will entail a complete incompatible replacement that will be supported in
parallel with the old protocol. Lix will thus retain remote build compatibility
with CppNix as long as CppNix maintains protocol compatibility with 2.18, and
as long as Lix retains legacy protocol support (which will likely be a long
time given that we plan to convert it to a frozen-in-time shim).
@@ -0,0 +1,8 @@
---
synopsis: "`nix repl` now allows tab-completing the special repl :colon commands"
cls: 1367
credits: Qyriad
category: Improvements
---
The REPL (`nix repl`) supports pressing `<TAB>` to complete a partial expression, but now also supports completing the special :colon commands as well (`:b`, `:edit`, `:doc`, etc), if the line starts with a colon.
+10
View File
@@ -0,0 +1,10 @@
---
synopsis: "Lix now supports building with UndefinedBehaviorSanitizer"
cls: [1483]
credits: [jade]
category: Development
---
You can now build Lix with the configuration option `-Db_sanitize=undefined` and it will both work and pass tests. AddressSanitizer support is also coming soon.
For a list of undefined behaviour fixed by sanitizer usage, see [the gerrit topic "undefined-behaviour"](https://gerrit.lix.systems/q/topic:%22undefined-behaviour%22).
+2 -3
View File
@@ -196,9 +196,8 @@
- [C++ style guide](contributing/cxx.md)
- [Release Notes](release-notes/release-notes.md)
- [Upcoming release](release-notes/rl-next.md)
<!-- RELENG-AUTO-INSERTION-MARKER (see releng/release_notes.py) -->
- [Lix 2.91 (2024-08-12)](release-notes/rl-2.91.md)
- [Lix 2.90 (2024-07-10)](release-notes/rl-2.90.md)
<!-- RELENG-AUTO-INSERTION-MARKER (see releng/release_notes.py) -->
- [Lix 2.90 (FIXME date)](release-notes/rl-2.90.md)
- [Nix 2.18 (2023-09-20)](release-notes/rl-2.18.md)
- [Nix 2.17 (2023-07-24)](release-notes/rl-2.17.md)
- [Nix 2.16 (2023-05-31)](release-notes/rl-2.16.md)
-1
View File
@@ -175,7 +175,6 @@ These are specified in `crossSystems` in `flake.nix`; feel free to submit change
- `armv6l-linux`
- `armv7l-linux`
- `aarch64-linux`
- `riscv64-linux`
For example, to cross-compile Lix for `armv6l-linux` from another Linux, use the following:
+4 -1
View File
@@ -247,6 +247,7 @@ To ensure that characterization testing doesn't make it harder to intentionally
The integration tests are defined in the Nix flake under the `hydraJobs.tests` attribute.
These tests include everything that needs to interact with external services or run Lix in a non-trivial distributed setup.
Because these tests are expensive and require more than what the standard github-actions setup provides, they only run on the master branch (on <https://hydra.nixos.org/jobset/nix/master>).
You can run them manually with `nix build .#hydraJobs.tests.{testName}` or `nix-build -A hydraJobs.tests.{testName}`
@@ -427,12 +428,14 @@ I grepped `src/` for `get[eE]nv\("` to find the mentions in Lix code.
- `NIX_SHOW_STATS_PATH` - Writes those statistics into a file at the given path instead of stdout. Undocumented.
- `NIX_SHOW_SYMBOLS` - Dumps the symbol table into the show-stats json output.
- `TERM` - If `dumb` or unset, disables ANSI colour output.
- `FORCE_COLOR`, `CLICOLOR_FORCE` - Enables ANSI colour output if `NO_COLOR`/`NOCOLOR` not set.
- `NO_COLOR`, `NOCOLOR` - Disables ANSI colour output.
- `_NIX_DEVELOPER_SHOW_UNKNOWN_LOCATIONS` - Highlights unknown locations in errors.
- `NIX_PROFILE` - Selects which profile `nix-env` will operate on. Documented elsewhere.
- `NIX_SSHOPTS` - Options passed to `ssh(1)` when using a ssh remote store.
Incorrectly documented on `nix-copy-closure` which is *surely* not the only place they are used??
- `_NIX_TEST_NO_LSOF` - Used on non-Linux, non-macOS platforms to disable using `lsof` when finding gc roots.
Since https://git.lix.systems/lix-project/lix/issues/156 was fixed, this should probably just be removed as it was a bad workaround for a macOS issue.
- `_NIX_TEST_GC_SYNC_1` - Path to a pipe that is used to block the GC briefly to validate invariants from the test suite.
- `_NIX_TEST_GC_SYNC_2` - Path to a pipe that is used to block the GC briefly to validate invariants from the test suite.
- `_NIX_TEST_FREE_SPACE_FILE` - Path to a file containing a decimal number with the free space that the GC is to believe it has.
@@ -292,12 +292,6 @@ Derivations can declare some infrequently used optional attributes.
(associative) arrays. For example, the attribute `hardening.format = true`
ends up as the Bash associative array element `${hardening[format]}`.
> **Warning**
>
> If set to `true`, other advanced attributes such as [`allowedReferences`](#adv-attr-allowedReferences), [`allowedReferences`](#adv-attr-allowedReferences), [`allowedRequisites`](#adv-attr-allowedRequisites),
[`disallowedReferences`](#adv-attr-disallowedReferences) and [`disallowedRequisites`](#adv-attr-disallowedRequisites), maxSize, and maxClosureSize.
will have no effect.
- [`outputChecks`]{#adv-attr-outputChecks}\
When using [structured attributes](#adv-attr-structuredAttrs), the `outputChecks`
attribute allows defining checks per-output.
@@ -332,6 +326,7 @@ Derivations can declare some infrequently used optional attributes.
```
- [`unsafeDiscardReferences`]{#adv-attr-unsafeDiscardReferences}\
When using [structured attributes](#adv-attr-structuredAttrs), the
attribute `unsafeDiscardReferences` is an attribute set with a boolean value for each output name.
If set to `true`, it disables scanning the output for runtime dependencies.
+6 -92
View File
@@ -26,8 +26,6 @@
| Logical conjunction (`AND`) | *bool* `&&` *bool* | left | 12 |
| Logical disjunction (`OR`) | *bool* <code>\|\|</code> *bool* | left | 13 |
| [Logical implication] | *bool* `->` *bool* | none | 14 |
| \[Experimental\] [Function piping] | *expr* |> *func* | left | 15 |
| \[Experimental\] [Function piping] | *expr* <| *func* | right | 16 |
[string]: ./values.md#type-string
[path]: ./values.md#type-path
@@ -61,10 +59,8 @@ The result is a [Boolean] value.
## Arithmetic
Numbers will retain their type unless mixed with other numeric types:
Pure integer operations will always return integers, whereas any operation involving at least one floating point number returns a floating point number.
Integer overflow (of 64-bit signed integers) and division by zero are defined to throw an error.
Numbers are type-compatible:
Pure integer operations will always return integers, whereas any operation involving at least one floating point number return a floating point number.
See also [Comparison] and [Equality].
@@ -147,103 +143,21 @@ All comparison operators are implemented in terms of `<`, and the following equi
| *a* `>` *b* | *b* `<` *a* |
| *a* `>=` *b* | `! (` *a* `<` *b* `)` |
Note that the above behaviour violates IEEE 754 for floating point numbers with respect to NaN, for instance.
This may be fixed in a future major language revision.
[Comparison]: #comparison-operators
## Equality
The following equality comparison rules are followed in order:
- Comparisons are first, sometimes, performed by identity (pointer value), and whether or not this occurs varies depending on the context in which the comparison is performed; for example, through `builtins.elem`, comparison of lists, or other cases.
The exact instances in which this occurs, aside from direct list and attribute set comparisons as discussed below, are too dependent on implementation details to meaningfully document.
See [note on identity comparison](#identity-comparison) below.
- Comparisons between a combination of integers and floating point numbers are first converted to floating point then compared as floating point.
- Comparisons between values of differing types, besides the ones mentioned in the above rule, are unequal.
- Strings are compared as their string values, disregarding string contexts.
- Paths are compared as their absolute form (since they are stored as such).
- [Functions][function] are always considered unequal, including with themselves.
- The following are compared in the typical manner:
- Integers
- Floating point numbers have equality comparison per IEEE 754.
Note that this means that just like in most languages, floating point arithmetic results are not typically equality comparable, and should instead be compared by checking that the absolute difference is less than some error margin.
- Booleans
- Null
- [Attribute sets][attribute set] are compared following these rules in order:
- If both attribute sets have the same identity (via pointer equality), they are considered equal, regardless of whether the contents have reflexive equality (e.g. even if there are functions contained within).
See [note on identity comparison](#identity-comparison) below.
- If both attribute sets have `type = "derivation"` and have an attribute `outPath` that is equal, they are considered equal.
This means that two results of `builtins.derivation`, regardless of other things added to their attributes via `//` afterwards (or `passthru` in nixpkgs), will compare equal if they passed the same arguments to `builtins.derivation`.
- Otherwise, they are compared element-wise in an unspecified order.
Although this order *may* be deterministic in some cases, this is not guaranteed, and correct code must not rely on this ordering behaviour.
The order determines which elements are evaluated first and thus, if there are throwing values in the attribute set, which of those get evaluated, if any, before the comparison returns an unequal result.
- Lists are compared following these rules in order:
- If both lists have the same identity (via pointer equality), they are considered equal, regardless of whether the contents have reflexive equality (e.g. even if there are functions contained within).
See [note on identity comparison](#identity-comparison) below.
- Otherwise, they are compared element-wise in list order.
- [Attribute sets][attribute set] and [list]s are compared recursively, and therefore are fully evaluated.
- Comparison of [function]s always returns `false`.
- Numbers are type-compatible, see [arithmetic] operators.
- Floating point numbers only differ up to a limited precision.
[function]: ./constructs.md#functions
[Equality]: #equality
### Identity comparison
In the current revision of the Nix language, values are first compared by identity (pointer equality).
This means that values that are not reflexively equal (that is, they do not satisfy `a == a`), such as functions, are nonetheless sometimes compared as equal with themselves if they are placed in attribute sets or lists, or are compared through other indirect means.
Whether identity comparison applies to a given usage of the language aside from direct list and attribute set comparison is strongly dependent on implementation details to the point it is not feasible to document the exact instances.
This is rather unfortunate behaviour which is regrettably load-bearing on nixpkgs (such as with the `type` attribute of NixOS options) and cannot be changed for the time being.
It may be changed in a future major language revision.
Correct code must not rely on this behaviour.
For example:
```
nix-repl> let f = x: 1; s = { func = f; }; in [ (f == f) (s == s) ]
[ false true ]
```
## Logical implication
Equivalent to `!`*b1* `||` *b2*.
[Logical implication]: #logical-implication
## \[Experimental\] Function piping
*This language feature is still experimental and may change at any time. Enable `--extra-experimental-features pipe-operator` to use it.*
Pipes are a dedicated operator for function application, but with reverse order and a lower binding strength.
This allows you to chain function calls together in way that is more natural to read and requires less parentheses.
`a |> f b |> g` is equivalent to `g (f b a)`.
`g <| f b <| a` is equivalent to `g (f b a)`.
Example code snippet:
```nix
defaultPrefsFile = defaultPrefs
|> lib.mapAttrsToList (
key: value: ''
// ${value.reason}
pref("${key}", ${builtins.toJSON value.value});
''
)
|> lib.concatStringsSep "\n"
|> pkgs.writeText "nixos-default-prefs.js";
```
Note how `mapAttrsToList` is called with two arguments (the lambda and `defaultPrefs`),
but moving the last argument in front of the rest improves the reading flow.
This is common for functions with long first argument, including all `map`-like functions.
[Function piping]: #experimental-function-piping
+9 -15
View File
@@ -7,16 +7,13 @@
*Strings* can be written in three ways.
The most common way is to enclose the string between double quotes,
e.g., `"foo bar"`. Strings can span multiple lines. The backslash
(`\`) can be used to escape characters: newlines, carriage returns
and tabs may be written as `\n`, `\r` and `\t` respectively; any
other characters can be preceded by a backslash to remove any
special meaning they may have, like the special characters `"` and
`\` and the character sequence `${`.
e.g., `"foo bar"`. Strings can span multiple lines. The special
characters `"` and `\` and the character sequence `${` must be
escaped by prefixing them with a backslash (`\`). Newlines, carriage
returns and tabs can be written as `\n`, `\r` and `\t`,
respectively.
You can include the results of other expressions into a string by enclosing them in `${ }`, a feature known as [string interpolation].
Due to a parser issue that has since come to be relied upon, the character sequence `$${` is interpreted literally and does not introduce an interpolation.
To express a `$` character immediately followed by an interpolation, the former must be escaped.
[string interpolation]: ./string-interpolation.md
@@ -46,16 +43,16 @@
Note that the whitespace and newline following the opening `''` is
ignored if there is no non-whitespace text on the initial line.
Indented strings support [string interpolation].
Since `${` and `''` have special meaning in indented strings, you
need a way to quote them. `$` can be escaped by prefixing it with
`''` (that is, two single quotes), i.e., `''$`. `''` can be escaped
by prefixing it with `'`, i.e., `'''`. Linefeed, carriage-return and tab
by prefixing it with `'`, i.e., `'''`. `$` removes any special
meaning from the following `$`. Linefeed, carriage-return and tab
characters can be written as `''\n`, `''\r`, `''\t`, and `''\`
escapes any other character.
Indented strings support [string interpolation] using `${ }` the same way regular strings do.
`$${` is interpreted literally in indented strings as well, so the `$` character must be escaped if it is to be followed by an interpolation.
Indented strings are primarily useful in that they allow multi-line
string literals to follow the indentation of the enclosing Nix
expression, and that less escaping is typically necessary for
@@ -88,9 +85,6 @@
Numbers, which can be *integers* (like `123`) or *floating point*
(like `123.43` or `.27e13`).
Integers in the Nix language are 64-bit signed integers.
Integer overflow is defined to throw an error.
See [arithmetic] and [comparison] operators for semantics.
[arithmetic]: ./operators.md#arithmetic
+2 -2
View File
@@ -1,7 +1,7 @@
# Lix 2.90 "Vanilla Ice Cream" (2024-07-10)
# Lix 2.90 "Vanilla Ice Cream" (FIXME date)
# Lix 2.90.0 (2024-07-10)
# Lix 2.90.0 (FIXME date)
## Breaking Changes
- Deprecate the online flake registries and vendor the default registry [fj#183](https://git.lix.systems/lix-project/lix/issues/183) [fj#110](https://git.lix.systems/lix-project/lix/issues/110) [fj#116](https://git.lix.systems/lix-project/lix/issues/116) [#8953](https://github.com/NixOS/nix/issues/8953) [#9087](https://github.com/NixOS/nix/issues/9087) [cl/1127](https://gerrit.lix.systems/c/lix/+/1127)
-599
View File
@@ -1,599 +0,0 @@
# Lix 2.91 "Dragon's Breath" (2024-08-12)
# Lix 2.91.3 (2025-06-30)
## Fixes
- Revert CVE-2025-52992 failed mitigation [fj#883](https://git.lix.systems/lix-project/lix/issues/883) [fj#887](https://git.lix.systems/lix-project/lix/issues/887) [cl/3420](https://gerrit.lix.systems/c/lix/+/3420) [cl/3524](https://gerrit.lix.systems/c/lix/+/3524) [cl/3523](https://gerrit.lix.systems/c/lix/+/3523) [cl/3522](https://gerrit.lix.systems/c/lix/+/3522)
Following the initial mitigation of **CVE-2025-52992** in `cl/3420`, we
received reports of **unexpected deletion of in-use store paths**.
Upon investigation, we found that the patch did **not correctly cancel all
automatic deleters**, resulting in potentially critical path loss during normal
operation.
Given the severity and time-sensitive nature of the situation ([see incident
report](https://lix.systems/blog/2025-06-27-lix-critical-bug/)), we evaluated
possible options to repair the behavior safely. However, we concluded that a
rushed fix would either:
* **Overdelete**, i.e. breaking running systems, or,
* **Underdelete**, effectively **reopening CVE-2025-52992** while leaving
orphaned paths behind.
As **CVE-2025-52992 has no known exploit vector**, and correctness is critical
in the Lix project, we have **fully reverted the previous mitigations**.
Moving forward, the Lix team will rework this code path in a **long-term,
correctness-first fix** on the main branch. We will explore backporting it to
stable channels once its safety is assured.
We are deeply sorry for the stability incident and the Lix team remain
available for assisting you in recovering your systems.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Fallback to safe temp dir when build-dir is unwritable [fj#876](https://git.lix.systems/lix-project/lix/issues/876) [cl/3502](https://gerrit.lix.systems/c/lix/+/3502)
Non-daemon builds started failing with a permission error after introducing the `build-dir` option:
```
$ nix build --store ~/scratch nixpkgs#hello --rebuild
error: creating directory '/nix/var/nix/builds/nix-build-hello-2.12.2.drv-0': Permission denied
```
This happens because:
1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
2. The user lacks permissions for that path.
We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
**Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
# Lix 2.91.2 (2025-06-23)
## Breaking Changes
- Fixed output derivations can be run using `pasta` network isolation [fj#285](https://git.lix.systems/lix-project/lix/issues/285) [cl/3418](https://gerrit.lix.systems/c/lix/+/3418)
Fixed output derivations traditionally run in the host network namespace.
On Linux this allows such derivations to communicate with other sandboxes
or the host using the abstract Unix domains socket namespace; this hasn't
been unproblematic in the past and has been used in two distinct exploits
to break out of the sandbox. For this reason fixed output derivations can
now run in a network namespace (provided by [`pasta`]), restricted to TCP
and UDP communication with the rest of the world. When enabled this could
be a breaking change and we classify it as such, even though we don't yet
enable or require such isolation by default. We may enforce this in later
releases of Lix once we have sufficient confidence that breakage is rare.
[`pasta`]: https://passt.top/
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) and [puck](https://git.lix.systems/puck) for this.
## Fixes
- Always clean up scratch paths after derivations failed to build [cl/3420](https://gerrit.lix.systems/c/lix/+/3420)
Previously, scratch paths created during builds were not always cleaned up if
the derivation failed, potentially leaving behind unnecessary temporary files
or directories in the Nix store.
This fix ensures that such paths are consistently removed after a failed build,
improving Nix store hygiene, hardening Lix against mis-reuse of failed builds
scratch paths.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- `build-dir` no longer defaults to `temp-dir` [cl/3419](https://gerrit.lix.systems/c/lix/+/3419)
The directory in which temporary build directories are created no longer defaults
to the value of the `temp-dir` setting to avoid builders making their directories
world-accessible. This behavior has been used to escape the build sandbox and can
cause build impurities even when not used maliciously. We now default to `builds`
in `NIX_STATE_DIR` (which is `/nix/var/nix/builds` in the default configuration).
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
# Lix 2.91.1 (2024-10-18)
## Fixes
- `<nix/fetchurl.nix>` now uses TLS verification [#11585](https://github.com/NixOS/nix/pull/11585)
Previously `<nix/fetchurl.nix>` did not do TLS verification. This was because the Nix sandbox in the past did not have access to TLS certificates, and Nix checks the hash of the fetched file anyway. However, this can expose authentication data from `netrc` and URLs to man-in-the-middle attackers. In addition, Nix now in some cases (such as when using impure derivations) does *not* check the hash. Therefore we have now enabled TLS verification. This means that downloads by `<nix/fetchurl.nix>` will now fail if you're fetching from a HTTPS server that does not have a valid certificate.
`<nix/fetchurl.nix>` is also known as the builtin derivation builder `builtin:fetchurl`. It's not to be confused with the evaluation-time function `builtins.fetchurl`, which was not affected by this issue.
Many thanks to [Eelco Dolstra](https://github.com/edolstra) for this.
# Lix 2.91.0 (2024-08-12)
## Breaking Changes
- Block io_uring in the Linux sandbox [cl/1611](https://gerrit.lix.systems/c/lix/+/1611)
The io\_uring API has the unfortunate property that it is not possible to selectively decide which operations should be allowed.
This, together with the fact that new operations are routinely added, makes it a hazard to the proper function of the sandbox.
Therefore, any access to io\_uring has been made unavailable inside the sandbox.
As such, attempts to execute any system calls forming part of this API will fail with the error `ENOSYS`, as if io\_uring support had not been configured into the kernel.
Many thanks to [alois31](https://git.lix.systems/alois31) for this.
- The `build-hook` setting is now deprecated
Build hooks communicate with the daemon using a custom, internal, undocumented protocol that is entirely unversioned and cannot be changed.
Since we intend to change it anyway we must unfortunately deprecate the current build hook infrastructure.
We do not expect this to impact most users—we have not found any uses of `build-hook` in the wild—but if this does affect you, we'd like to hear from you!
- Lix no longer speaks the Nix remote-build worker protocol to clients or servers older than CppNix 2.3 [fj#325](https://git.lix.systems/lix-project/lix/issues/325) [cl/1207](https://gerrit.lix.systems/c/lix/+/1207) [cl/1208](https://gerrit.lix.systems/c/lix/+/1208) [cl/1206](https://gerrit.lix.systems/c/lix/+/1206) [cl/1205](https://gerrit.lix.systems/c/lix/+/1205) [cl/1204](https://gerrit.lix.systems/c/lix/+/1204) [cl/1203](https://gerrit.lix.systems/c/lix/+/1203) [cl/1479](https://gerrit.lix.systems/c/lix/+/1479)
CppNix 2.3 was released in 2019, and is the new oldest supported version. We
will increase our support baseline in the future up to a final version of CppNix
2.18 (which may happen soon given that it is the only still-packaged and thus
still-tested >2.3 version), but this step already removes a significant amount
of dead, untested, code paths.
Lix speaks the same version of the protocol as CppNix 2.18 and that fact will
never change in the future; the Lix plans to replace the protocol for evolution
will entail a complete incompatible replacement that will be supported in
parallel with the old protocol. Lix will thus retain remote build compatibility
with CppNix as long as CppNix maintains protocol compatibility with 2.18, and
as long as Lix retains legacy protocol support (which will likely be a long
time given that we plan to convert it to a frozen-in-time shim).
Many thanks to [jade](https://git.lix.systems/jade) for this.
## Features
- Pipe operator `|>` (experimental) [fj#438](https://git.lix.systems/lix-project/lix/issues/438) [cl/1654](https://gerrit.lix.systems/c/lix/+/1654)
Implementation of the pipe operator (`|>`) in the language as described in [RFC 148](https://github.com/NixOS/rfcs/pull/148).
The feature is still marked experimental, enable `--extra-experimental-features pipe-operator` to use it.
Many thanks to [piegames](https://git.lix.systems/piegames) and [eldritch horrors](https://git.lix.systems/pennae) for this.
## Improvements
- Trace which part of a `foo.bar.baz` expression errors [cl/1505](https://gerrit.lix.systems/c/lix/+/1505) [cl/1506](https://gerrit.lix.systems/c/lix/+/1506)
Previously, if an attribute path selection expression like `linux_4_9.meta.description` it wouldn't show you which one of those parts in the attribute path, or even that that line of code is what caused evaluation of the failing expression.
The previous error looks like this:
```
pkgs.linuxKernel.kernels.linux_4_9.meta.description
error:
… while evaluating the attribute 'linuxKernel.kernels.linux_4_9.meta.description'
at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:5:
277| } // lib.optionalAttrs config.allowAliases {
278| linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
| ^
279| linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";
… while calling the 'throw' builtin
at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:17:
277| } // lib.optionalAttrs config.allowAliases {
278| linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
| ^
279| linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";
error: linux 4.9 was removed because it will reach its end of life within 22.11
```
Now, the error will look like this:
```
pkgs.linuxKernel.kernels.linux_4_9.meta.description
error:
… while evaluating the attribute 'linuxKernel.kernels.linux_4_9.meta.description'
at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:5:
277| } // lib.optionalAttrs config.allowAliases {
278| linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
| ^
279| linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";
… while evaluating 'pkgs.linuxKernel.kernels.linux_4_9' to select 'meta' on it
at «string»:1:1:
1| pkgs.linuxKernel.kernels.linux_4_9.meta.description
| ^
… caused by explicit throw
at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:17:
277| } // lib.optionalAttrs config.allowAliases {
278| linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
| ^
279| linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";
error: linux 4.9 was removed because it will reach its end of life within 22.11
```
Not only does the line of code that referenced the failing attribute show up in the trace, it also tells you that it was specifically the `linux_4_9` part that failed.
This includes if the failing part is a top-level binding:
```
let
inherit (pkgs.linuxKernel.kernels) linux_4_9;
in linux_4_9.meta.description
error:
… while evaluating 'linux_4_9' to select 'meta.description' on it
at «string»:3:4:
2| inherit (pkgs.linuxKernel.kernels) linux_4_9;
3| in linux_4_9.meta.description
| ^
… while evaluating the attribute 'linux_4_9'
at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:5:
277| } // lib.optionalAttrs config.allowAliases {
278| linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
| ^
279| linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";
… caused by explicit throw
at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:17:
277| } // lib.optionalAttrs config.allowAliases {
278| linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
| ^
279| linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";
error: linux 4.9 was removed because it will reach its end of life within 22.11
```
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- Confusing 'invalid path' errors are now 'path does not exist' [cl/1161](https://gerrit.lix.systems/c/lix/+/1161) [cl/1160](https://gerrit.lix.systems/c/lix/+/1160) [cl/1159](https://gerrit.lix.systems/c/lix/+/1159)
Previously, if a path did not exist in a Nix store, it was referred to as the internal name "path is invalid".
This is, however, very confusing, and there were numerous such errors that were exactly the same, making it hard to debug.
These errors are now more specific and refer to the path not existing in the store.
Many thanks to [julia](https://git.lix.systems/midnightveil) for this.
- Add a `build-dir` setting to set the backing directory for builds [gh#10303](https://github.com/NixOS/nix/pull/10303) [gh#10312](https://github.com/NixOS/nix/pull/10312) [gh#10883](https://github.com/NixOS/nix/pull/10883) [cl/1514](https://gerrit.lix.systems/c/lix/+/1514)
`build-dir` can now be set in the Nix configuration to choose the backing directory for the build sandbox.
This can be useful on systems with `/tmp` on tmpfs, or simply to relocate large builds to another disk.
Also, `XDG_RUNTIME_DIR` is no longer considered when selecting the default temporary directory,
as it's not intended to be used for large amounts of data.
Many thanks to [Robert Hensing](https://github.com/roberth) and [Tom Bereknyei](https://github.com/tomberek) for this.
- Better usage of colour control environment variables [cl/1699](https://gerrit.lix.systems/c/lix/+/1699) [cl/1702](https://gerrit.lix.systems/c/lix/+/1702)
Lix now heeds `NO_COLOR`/`NOCOLOR` for more output types, such as that used in `nix search`, `nix flake metadata` and similar.
It also now supports `CLICOLOR_FORCE`/`FORCE_COLOR` to force colours regardless of whether there is a terminal on the other side.
It now follows rules compatible with those described on <https://bixense.com/clicolors/> with `CLICOLOR` defaulted to enabled.
That is to say, the following procedure is followed in order:
- NO_COLOR or NOCOLOR set
Always disable colour
- CLICOLOR_FORCE or FORCE_COLOR set
Enable colour
- The output is a tty; TERM != "dumb"
Enable colour
- Otherwise
Disable colour
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Distinguish between explicit throws and errors that happened while evaluating a throw [cl/1511](https://gerrit.lix.systems/c/lix/+/1511)
Previously, errors caused by an expression like `throw "invalid argument"` were treated like an error that happened simply while some builtin function was being called:
```
let
throwMsg = p: throw "${p} isn't the right package";
in throwMsg "linuz"
error:
… while calling the 'throw' builtin
at «string»:2:17:
1| let
2| throwMsg = p: throw "${p} isn't the right package";
| ^
3| in throwMsg "linuz"
error: linuz isn't the right package
```
But the error didn't just happen "while" calling the `throw` builtin — it's a throw error!
Now it looks like this:
```
let
throwMsg = p: throw "${p} isn't the right package";
in throwMsg "linuz"
error:
… caused by explicit throw
at «string»:2:17:
1| let
2| throwMsg = p: throw "${p} isn't the right package";
| ^
3| in throwMsg "linuz"
error: linuz isn't the right package
```
This also means that incorrect usage of `throw` or errors evaluating its arguments are easily distinguishable from explicit throws:
```
let
throwMsg = p: throw "${p} isn't the right package";
in throwMsg { attrs = "error when coerced in string interpolation"; }
error:
… while calling the 'throw' builtin
at «string»:2:17:
1| let
2| throwMsg = p: throw "${p} isn't the right package";
| ^
3| in throwMsg { attrs = "error when coerced in string interpolation"; }
… while evaluating a path segment
at «string»:2:24:
1| let
2| throwMsg = p: throw "${p} isn't the right package";
| ^
3| in throwMsg { attrs = "error when coerced in string interpolation"; }
error: cannot coerce a set to a string: { attrs = "error when coerced in string interpolation"; }
```
Here, instead of an actual thrown error, a type error happens first (trying to coerce an attribute set to a string), but that type error happened *while* calling `throw`.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- `nix flake metadata` prints modified date [cl/1700](https://gerrit.lix.systems/c/lix/+/1700)
Ever wonder "gee, when *did* I update nixpkgs"?
Wonder no more, because `nix flake metadata` now simply tells you the times every locked flake input was updated:
```
<...>
Description: The purely functional package manager
Path: /nix/store/c91yi8sxakc2ry7y4ac1smzwka4l5p78-source
Revision: c52cff582043838bbe29768e7da232483d52b61d-dirty
Last modified: 2024-07-31 22:15:54
Inputs:
├───flake-compat: github:edolstra/flake-compat/0f9255e01c2351cc7d116c072cb317785dd33b33
│ Last modified: 2023-10-04 06:37:54
├───nix2container: github:nlewo/nix2container/3853e5caf9ad24103b13aa6e0e8bcebb47649fe4
│ Last modified: 2024-07-10 13:15:56
├───nixpkgs: github:NixOS/nixpkgs/e21630230c77140bc6478a21cd71e8bb73706fce
│ Last modified: 2024-07-25 11:26:27
├───nixpkgs-regression: github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2
│ Last modified: 2022-01-24 11:20:45
└───pre-commit-hooks: github:cachix/git-hooks.nix/f451c19376071a90d8c58ab1a953c6e9840527fd
Last modified: 2024-07-15 04:21:09
```
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Hash mismatch diagnostics for fixed-output derivations include the URL [cl/1536](https://gerrit.lix.systems/c/lix/+/1536)
Now, when building fixed-output derivations, Lix will guess the URL that was used in the derivation using the `url` or `urls` properties in the derivation environment.
This is a layering violation but making these diagnostics tractable when there are multiple instances of the `AAAA` hash is too significant of an improvement to pass it up.
```
error: hash mismatch in fixed-output derivation '/nix/store/sjfw324j4533lwnpmr5z4icpb85r63ai-x1.drv':
likely URL: https://meow.puppy.forge/puppy.tar.gz
specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
got: sha256-a1Qvp3FOOkWpL9kFHgugU1ok5UtRPSu+NwCZKbbaEro=
```
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Add log formats `multiline` and `multiline-with-logs` [cl/1369](https://gerrit.lix.systems/c/lix/+/1369)
Added two new log formats (`multiline` and `multiline-with-logs`) that display
current activities below each other for better visibility.
These formats attempt to use the maximum available lines
(defaulting to 25 if unable to determine) and print up to that many lines.
The status bar is displayed as the first line, with each subsequent
activity on its own line.
Many thanks to [kloenk](https://git.lix.systems/kloenk) for this.
- Lix will now show the package descriptions in when running `nix flake show`. [cl/1540](https://gerrit.lix.systems/c/lix/+/1540)
When running `nix flake show`, Lix will now show the package descriptions, if they exist.
Before:
```shell
$ nix flake show
path:/home/isabel/dev/lix-show?lastModified=1721736108&narHash=sha256-Zo8HP1ur7Q2b39hKUEG8EAh/opgq8xJ2jvwQ/htwO4Q%3D
└───packages
└───x86_64-linux
├───aNoDescription: package 'simple'
├───bOneLineDescription: package 'simple'
├───cMultiLineDescription: package 'simple'
├───dLongDescription: package 'simple'
└───eEmptyDescription: package 'simple'
```
After:
```shell
$ nix flake show
path:/home/isabel/dev/lix-show?lastModified=1721736108&narHash=sha256-Zo8HP1ur7Q2b39hKUEG8EAh/opgq8xJ2jvwQ/htwO4Q%3D
└───packages
└───x86_64-linux
├───aNoDescription: package 'simple'
├───bOneLineDescription: package 'simple' - 'one line'
├───cMultiLineDescription: package 'simple' - 'line one'
├───dLongDescription: package 'simple' - 'abcdefghijklmnopqrstuvwxyz'
└───eEmptyDescription: package 'simple'
```
Many thanks to [kjeremy](https://github.com/kjeremy) and [isabelroses](https://git.lix.systems/isabelroses) for this.
- Eliminate some pretty-printing surprises [#11100](https://github.com/NixOS/nix/pull/11100) [cl/1616](https://gerrit.lix.systems/c/lix/+/1616) [cl/1617](https://gerrit.lix.systems/c/lix/+/1617) [cl/1618](https://gerrit.lix.systems/c/lix/+/1618)
Some inconsistent and surprising behaviours have been eliminated from the pretty-printing used by the REPL and `nix eval`:
* Lists and attribute sets that contain only a single item without nested structures are no longer sometimes inappropriately indented in the REPL, depending on internal state of the evaluator.
* Empty attribute sets and derivations are no longer shown as `«repeated»`, since they are always cheap to print.
This matches the existing behaviour of `nix-instantiate` on empty attribute sets.
Empty lists were never printed as `«repeated»` already.
* The REPL by default does not print nested attribute sets and lists, and indicates elided items with an ellipsis.
Previously, the ellipsis was printed even when the structure was empty, so that such items do not in fact exist.
Since this behaviour was confusing, it does not happen any more.
Before:
```
nix-repl> :p let x = 1 + 2; in [ [ x ] [ x ] ]
[
[
3
]
[ 3 ]
]
nix-repl> let inherit (import <nixpkgs> { }) hello; in [ hello hello ]
[
«derivation /nix/store/fqs92lzychkm6p37j7fnj4d65nq9fzla-hello-2.12.1.drv»
«repeated»
]
nix-repl> let x = {}; in [ x ]
[
{ ... }
]
```
After:
```
nix-repl> :p let x = 1 + 2; in [ [ x ] [ x ] ]
[
[ 3 ]
[ 3 ]
]
nix-repl> let inherit (import <nixpkgs> { }) hello; in [ hello hello ]
[
«derivation /nix/store/fqs92lzychkm6p37j7fnj4d65nq9fzla-hello-2.12.1.drv»
«derivation /nix/store/fqs92lzychkm6p37j7fnj4d65nq9fzla-hello-2.12.1.drv»
]
nix-repl> let x = {}; in [ x ]
[
{ }
]
```
Many thanks to [alois31](https://git.lix.systems/alois31) and [Robert Hensing](https://github.com/roberth) for this.
- `nix registry add` now requires a shorthand flakeref on the 'from' side [cl/1494](https://gerrit.lix.systems/c/lix/+/1494)
The 'from' argument must now be a shorthand flakeref like `nixpkgs` or `nixpkgs/nixos-20.03`, making it harder to accidentally swap the 'from' and 'to' arguments.
Registry entries that map from other flake URLs can still be specified in registry.json, the `nix.registry` option in NixOS, or the `--override-flake` option in the CLI, but they are not guaranteed to work correctly.
Many thanks to [delan](https://git.lix.systems/delan) for this.
- Allow automatic rejection of configuration options from flakes [cl/1541](https://gerrit.lix.systems/c/lix/+/1541)
Setting `accept-flake-config` to `false` now respects user choice by automatically rejecting configuration options set by flakes.
The old behaviour of asking each time is still available (and default) by setting it to the special value `ask`.
Many thanks to [alois31](https://git.lix.systems/alois31) for this.
- `nix repl` now allows tab-completing the special repl :colon commands [cl/1367](https://gerrit.lix.systems/c/lix/+/1367)
The REPL (`nix repl`) supports pressing `<TAB>` to complete a partial expression, but now also supports completing the special :colon commands as well (`:b`, `:edit`, `:doc`, etc), if the line starts with a colon.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- `:edit`ing a file in Nix store no longer reloads the repl [fj#341](https://git.lix.systems/lix-project/lix/issues/341) [cl/1620](https://gerrit.lix.systems/c/lix/+/1620)
Calling `:edit` from the repl now only reloads if the file being edited was outside of Nix store.
That means that all the local variables are now preserved across `:edit`s of store paths.
This is always safe because the store is read-only.
Many thanks to [goldstein](https://git.lix.systems/goldstein) for this.
- `:log` in repl now works on derivation paths [fj#51](https://git.lix.systems/lix-project/lix/issues/51) [cl/1716](https://gerrit.lix.systems/c/lix/+/1716)
`:log` can now accept store derivation paths in addition to derivation expressions.
Many thanks to [goldstein](https://git.lix.systems/goldstein) for this.
## Fixes
- Define integer overflow in the Nix language as an error [fj#423](https://git.lix.systems/lix-project/lix/issues/423) [cl/1594](https://gerrit.lix.systems/c/lix/+/1594) [cl/1595](https://gerrit.lix.systems/c/lix/+/1595) [cl/1597](https://gerrit.lix.systems/c/lix/+/1597) [cl/1609](https://gerrit.lix.systems/c/lix/+/1609)
Previously, integer overflow in the Nix language invoked C++ level signed overflow, which was undefined behaviour, but *probably* manifested as wrapping around on overflow.
Since prior to the public release of Lix, Lix had C++ signed overflow defined to crash the process and nobody noticed this having accidentally removed overflow from the Nix language for three months until it was caught by fiddling around.
Given the significant body of actual Nix code that has been evaluated by Lix in that time, it does not appear that nixpkgs or much of importance depends on integer overflow, so it is safe to turn into an error.
Some other overflows were fixed:
- `builtins.fromJSON` of values greater than the maximum representable value in a signed 64-bit integer will generate an error.
- `nixConfig` in flakes will no longer accept negative values for configuration options.
Integer overflow now looks like the following:
```
» nix eval --expr '9223372036854775807 + 1'
error: integer overflow in adding 9223372036854775807 + 1
```
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Fix nix-collect-garbage --dry-run [fj#432](https://git.lix.systems/lix-project/lix/issues/432) [cl/1566](https://gerrit.lix.systems/c/lix/+/1566)
`nix-collect-garbage --dry-run` did not previously give any output - it simply
exited without even checking to see what paths would be deleted.
```
$ nix-collect-garbage --dry-run
$
```
We updated the behaviour of the flag such that instead it prints out how many
paths it *would* delete, but doesn't actually delete them.
```
$ nix-collect-garbage --dry-run
finding garbage collector roots...
determining live/dead paths...
...
<nix store paths>
...
2670 store paths deleted, 0.00MiB freed
$
```
Many thanks to [Quantum Jump](https://github.com/QuantumBJump) for this.
- Fix unexpectedly-successful GC failures on macOS [fj#446](https://git.lix.systems/lix-project/lix/issues/446) [cl/1723](https://gerrit.lix.systems/c/lix/+/1723)
Has the following happened to you on macOS? This failure has been successfully eliminated, thanks to our successful deployment of advanced successful-failure detection technology (it's just `if (failed && errno == 0)`. Patent pending<sup>not really</sup>):
```
$ nix-store --gc --print-dead
finding garbage collector roots...
error: Listing pid 87261 file descriptors: Undefined error: 0
```
Many thanks to [jade](https://git.lix.systems/jade) for this.
- `nix copy` is now several times faster at `querying info about /nix/store/...` [fj#366](https://git.lix.systems/lix-project/lix/issues/366) [cl/1462](https://gerrit.lix.systems/c/lix/+/1462)
We fixed a locking bug that serialized `querying info about /nix/store/...`
onto just one thread such that it was eating `O(paths to copy * latency)` time
while setting up to copy paths to s3 and other stores. It is now `nproc` times
faster.
Many thanks to [jade](https://git.lix.systems/jade) for this.
## Development
- clang-tidy support [fj#147](https://git.lix.systems/lix-project/lix/issues/147) [cl/1697](https://gerrit.lix.systems/c/lix/+/1697)
`clang-tidy` can be used to lint Lix with a limited set of lints using `ninja -C build clang-tidy` and `ninja -C build clang-tidy-fix`.
In practice, this fixes the built-in meson rule that was used the same as above being broken ever since precompiled headers were introduced.
Many thanks to [jade](https://git.lix.systems/jade) for this.
- Lix now supports building with UndefinedBehaviorSanitizer [cl/1483](https://gerrit.lix.systems/c/lix/+/1483) [cl/1481](https://gerrit.lix.systems/c/lix/+/1481) [cl/1669](https://gerrit.lix.systems/c/lix/+/1669)
You can now build Lix with the configuration option `-Db_sanitize=undefined,address` and it will both work and pass tests with both AddressSanitizer and UndefinedBehaviorSanitizer enabled.
To use ASan specifically, you have to set `-Dgc=disabled`, which an error message will tell you to do if necessary anyhow.
Furthermore, tests passing with Clang ASan+UBSan is checked on every change in CI.
For a list of undefined behaviour found by tooling usage, see [the gerrit topic "undefined-behaviour"](https://gerrit.lix.systems/q/topic:%22undefined-behaviour%22).
Many thanks to [jade](https://git.lix.systems/jade) for this.
+39 -38
View File
@@ -62,37 +62,38 @@ let
++ autoLayered
++ extraPkgs;
users = {
users =
{
root = {
uid = 0;
shell = "${pkgs.bashInteractive}/bin/bash";
home = "/root";
gid = 0;
groups = [ "root" ];
description = "System administrator";
};
nobody = {
uid = 65534;
shell = "${pkgs.shadow}/bin/nologin";
home = "/var/empty";
gid = 65534;
groups = [ "nobody" ];
description = "Unprivileged account (don't use!)";
};
}
// lib.listToAttrs (
map (n: {
name = "nixbld${toString n}";
value = {
uid = 30000 + n;
gid = 30000;
groups = [ "nixbld" ];
description = "Nix build user ${toString n}";
root = {
uid = 0;
shell = "${pkgs.bashInteractive}/bin/bash";
home = "/root";
gid = 0;
groups = [ "root" ];
description = "System administrator";
};
}) (lib.lists.range 1 32)
);
nobody = {
uid = 65534;
shell = "${pkgs.shadow}/bin/nologin";
home = "/var/empty";
gid = 65534;
groups = [ "nobody" ];
description = "Unprivileged account (don't use!)";
};
}
// lib.listToAttrs (
map (n: {
name = "nixbld${toString n}";
value = {
uid = 30000 + n;
gid = 30000;
groups = [ "nixbld" ];
description = "Nix build user ${toString n}";
};
}) (lib.lists.range 1 32)
);
groups = {
root.gid = 0;
@@ -192,11 +193,13 @@ let
in
''
{
${lib.concatStringsSep "\n" (
builtins.map (output: ''
${output} = { outPath = "${lib.getOutput output drv}"; };
'') outputs
)}
${
lib.concatStringsSep "\n" (
builtins.map (output: ''
${output} = { outPath = "${lib.getOutput output drv}"; };
'') outputs
)
}
outputs = [ ${lib.concatStringsSep " " (builtins.map (x: "\"${x}\"") outputs)} ];
name = "${drv.name}";
outPath = "${drv}";
@@ -358,10 +361,8 @@ let
"org.opencontainers.image.source" = "https://git.lix.systems/lix-project/lix";
"org.opencontainers.image.vendor" = "Lix project";
"org.opencontainers.image.version" = pkgs.nix.version;
"org.opencontainers.image.description" =
"Minimal Lix container image, with some batteries included.";
}
// lib.optionalAttrs (lixRevision != null) { "org.opencontainers.image.revision" = lixRevision; };
"org.opencontainers.image.description" = "Minimal Lix container image, with some batteries included.";
} // lib.optionalAttrs (lixRevision != null) { "org.opencontainers.image.revision" = lixRevision; };
};
meta = {
Generated
+11 -70
View File
@@ -16,30 +16,14 @@
"type": "github"
}
},
"lowdown-src": {
"flake": false,
"locked": {
"lastModified": 1633514407,
"narHash": "sha256-Dw32tiMjdK9t3ETl5fzGrutQTzh2rufgZV4A/BbxuD4=",
"owner": "kristapsdz",
"repo": "lowdown",
"rev": "d2c2b44ff6c27b936ec27358a2653caaef8f73b8",
"type": "github"
},
"original": {
"owner": "kristapsdz",
"repo": "lowdown",
"type": "github"
}
},
"nix2container": {
"flake": false,
"locked": {
"lastModified": 1749158376,
"narHash": "sha256-uirStFNxauh0lxzBowcp28X+Sq7JgsBIDnbwbAfZwf8=",
"lastModified": 1712990762,
"narHash": "sha256-hO9W3w7NcnYeX8u8cleHiSpK2YJo7ecarFTUlbybl7k=",
"owner": "nlewo",
"repo": "nix2container",
"rev": "0f8974c58755dba441df03598eefd1e1cd50e341",
"rev": "20aad300c925639d5d6cbe30013c8357ce9f2a2e",
"type": "github"
},
"original": {
@@ -48,44 +32,18 @@
"type": "github"
}
},
"nix_2_18": {
"inputs": {
"flake-compat": [
"flake-compat"
],
"lowdown-src": "lowdown-src",
"nixpkgs": "nixpkgs",
"nixpkgs-regression": [
"nixpkgs-regression"
]
},
"locked": {
"lastModified": 1730375271,
"narHash": "sha256-RrOFlDGmRXcVRV2p2HqHGqvzGNyWoD0Dado/BNlJ1SI=",
"owner": "NixOS",
"repo": "nix",
"rev": "0f665ff6779454f2117dcc32e44380cda7f45523",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "2.18.9",
"repo": "nix",
"type": "github"
}
},
"nixpkgs": {
"locked": {
"lastModified": 1705033721,
"narHash": "sha256-K5eJHmL1/kev6WuqyqqbS1cdNnSidIZ3jeqJ7GbrYnQ=",
"lastModified": 1718111384,
"narHash": "sha256-7tSst0S5FOmcgvNtfy6cjZX5w8CabCVAfAeCkhY4OVg=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "a1982c92d8980a0114372973cbdfe0a307f1bdea",
"rev": "a508a44af0c1b1b57785c34d8b54783536273eeb",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-23.05-small",
"ref": "nixos-24.05-small",
"repo": "nixpkgs",
"type": "github"
}
@@ -106,30 +64,14 @@
"type": "github"
}
},
"nixpkgs_2": {
"locked": {
"lastModified": 1757198069,
"narHash": "sha256-m3VUcOD4rTs8J7S+3dOjWMrAjw6RcITC3XYQ98zhEFs=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "0747026fc57ecb9c28901c7f7a2b5dc40e8af43c",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-25.05-small",
"repo": "nixpkgs",
"type": "github"
}
},
"pre-commit-hooks": {
"flake": false,
"locked": {
"lastModified": 1733318908,
"narHash": "sha256-SVQVsbafSM1dJ4fpgyBqLZ+Lft+jcQuMtEL3lQWx2Sk=",
"lastModified": 1712055707,
"narHash": "sha256-4XLvuSIDZJGS17xEwSrNuJLL7UjDYKGJSbK1WWX2AK8=",
"owner": "cachix",
"repo": "git-hooks.nix",
"rev": "6f4e2a2112050951a314d2733a994fbab94864c6",
"rev": "e35aed5fda3cc79f88ed7f1795021e559582093a",
"type": "github"
},
"original": {
@@ -142,8 +84,7 @@
"inputs": {
"flake-compat": "flake-compat",
"nix2container": "nix2container",
"nix_2_18": "nix_2_18",
"nixpkgs": "nixpkgs_2",
"nixpkgs": "nixpkgs",
"nixpkgs-regression": "nixpkgs-regression",
"pre-commit-hooks": "pre-commit-hooks"
}
+21 -112
View File
@@ -1,20 +1,9 @@
{
description = "Lix: A modern, delicious implementation of the Nix package manager";
description = "The purely functional package manager";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.05-small";
nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.05-small";
nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
# Required because Nix 2.18 is not in Nixpkgs ≥ 25.05 anymore.
nix_2_18 = {
url = "github:NixOS/nix/2.18.9";
# NOTE(Raito): this is not possible because patches on libseccomp does not apply anymore on this Nix.
# Let's keep the latest known nixpkgs useable with Nix 2.18 for our tests.
# inputs.nixpkgs.follows = "nixpkgs";
inputs.nixpkgs-regression.follows = "nixpkgs-regression";
inputs.flake-compat.follows = "flake-compat";
};
pre-commit-hooks = {
url = "github:cachix/git-hooks.nix";
flake = false;
@@ -36,7 +25,6 @@
nixpkgs-regression,
pre-commit-hooks,
nix2container,
nix_2_18,
flake-compat,
}:
@@ -45,7 +33,7 @@
# This notice gets echoed as a dev shell hook, and can be turned off with
# `touch .nocontribmsg`
sgr = builtins.fromJSON ''"\u001b["'';
sgr = ''['';
freezePage = "https://wiki.lix.systems/books/lix-contributors/page/freezes-and-recommended-contributions";
codebaseOverview = "https://wiki.lix.systems/books/lix-contributors/page/codebase-overview";
contribNotice = builtins.toFile "lix-contrib-notice" ''
@@ -71,8 +59,7 @@
(Run `touch .nocontribmsg` to hide this message.)
'';
versionJson = builtins.fromJSON (builtins.readFile ./version.json);
officialRelease = versionJson.official_release;
officialRelease = false;
# Set to true to build the release notes for the next release.
buildUnreleasedNotes = true;
@@ -103,7 +90,6 @@
"armv6l-linux"
"armv7l-linux"
"riscv64-linux"
"aarch64-linux"
# FIXME: still broken in 24.05: fails to build rustc(??) due to missing -lstdc++ dep
# "x86_64-freebsd"
# FIXME: broken dev shell due to python
@@ -113,7 +99,8 @@
stdenvs = [
"gccStdenv"
"clangStdenv"
"stdenv"
# FIXME: gcc 12 (default in 23.11) has bugs that break the nar parser.
"gcc13Stdenv" # "stdenv"
"libcxxStdenv"
"ccacheStdenv"
];
@@ -153,10 +140,13 @@
system = crossSystem;
}
// lib.optionalAttrs (crossSystem == "x86_64-freebsd") { useLLVM = true; };
overlays = [ (overlayFor (p: p.${stdenv})) ];
overlays = [
(overlayFor (p: p.${stdenv}))
(final: prev: { nixfmt = final.callPackage ./nix-support/nixfmt.nix { }; })
];
};
stdenvs = forAllStdenvs (make-pkgs null);
native = stdenvs.stdenvPackages;
native = stdenvs.gcc13StdenvPackages;
in
{
inherit stdenvs native;
@@ -173,21 +163,10 @@
{
nixStable = prev.nix;
nixVersions = prev.nixVersions // {
nix_2_3 = prev.nixVersions.nix_2_3.overrideAttrs (old: {
meta = old.meta // {
knownVulnerabilities = [ ];
};
});
# Nix 2.18 has been removed from Nixpkgs ≥ 25.05, so we need to reintroduce it ourselves for our tests.
nix_2_18 = nix_2_18.outputs.packages.${currentStdenv.hostPlatform.system}.default;
};
# Forward from the previous stage as we dont want it to pick the lowdown override
nixUnstable = prev.nixUnstable;
check-headers = final.buildPackages.callPackage ./maintainers/check-headers.nix { };
check-syscalls = final.buildPackages.callPackage ./maintainers/check-syscalls.nix { };
default-busybox-sandbox-shell = final.busybox.override {
useMusl = true;
@@ -219,7 +198,7 @@
busybox-sandbox-shell = final.busybox-sandbox-shell or final.default-busybox-sandbox-shell;
};
lix-clang-tidy = final.callPackage ./subprojects/lix-clang-tidy { };
pegtl = final.nix.passthru.pegtl;
# Export the patched version of boehmgc that Lix uses into the overlay
# for consumers of this flake.
@@ -300,52 +279,6 @@
# System tests.
tests = import ./tests/nixos { inherit lib nixpkgs nixpkgsFor; } // {
# This is x86_64-linux only, just because we have significantly
# cheaper x86_64-linux compute in CI.
# It is clangStdenv because clang's sanitizers are nicer.
asanBuild = self.packages.x86_64-linux.nix-clangStdenv.override {
# Improve caching of non-code changes by not changing the
# derivation name every single time, since this will never be seen
# by users anyway.
versionSuffix = "";
sanitize = [
"address"
"undefined"
];
# it is very hard to make *every* CI build use this option such
# that we don't wind up building Lix twice, so we do it here where
# we are already doing so.
werror = true;
};
# Although this might be nicer to do with pre-commit, that would
# require adding 12MB of nodejs to the dev shell, whereas building it
# in CI with Nix avoids that at a cost of slower feedback on rarely
# touched files.
jsSyntaxCheck =
let
nixpkgs = nixpkgsFor.x86_64-linux.native;
inherit (nixpkgs) pkgs;
docSources = lib.fileset.toSource {
root = ./doc;
fileset = lib.fileset.fileFilter (f: f.hasExt "js") ./doc;
};
in
pkgs.runCommand "js-syntax-check" { } ''
find ${docSources} -type f -print -exec ${pkgs.nodejs-slim}/bin/node --check '{}' ';'
touch $out
'';
# clang-tidy run against the Lix codebase using the Lix clang-tidy plugin
clang-tidy =
let
nixpkgs = nixpkgsFor.x86_64-linux.native;
inherit (nixpkgs) pkgs;
in
pkgs.callPackage ./package.nix {
versionSuffix = "";
lintInsteadOfBuild = true;
};
# Make sure that nix-env still produces the exact same result
# on a particular version of Nixpkgs.
@@ -368,16 +301,13 @@
in
pkgs.symlinkJoin {
name = "nixpkgs-lib-tests";
paths = [
testWithNix
]
# NOTE: nixpkgs 25.05 is being ... *creative*, and requires this dance to override
# the evaluator used for the test. it will break again in the future, don't worry.
++ lib.optionals pkgs.stdenv.isLinux [
((pkgs.callPackage "${nixpkgs}/ci/eval" { inherit nix; }).attrpathsSuperset {
evalSystem = system;
})
];
paths =
[ testWithNix ]
# FIXME: This is disabled on darwin due to a nixpkgs bug https://github.com/NixOS/nixpkgs/issues/319147
# After that is fixed, it should be restored to use lib/tests/release.nix as before, rather than this reimplementation.
++ lib.optionals pkgs.stdenv.isLinux [
(import (nixpkgs + "/pkgs/test/release") { inherit pkgs lib nix; })
];
}
);
};
@@ -399,25 +329,6 @@
pkgs = nixpkgsFor.x86_64-linux.native;
};
releaseTests = lib.foldl lib.recursiveUpdate { } [
(lib.genAttrs (linux64BitSystems ++ darwinSystems) (system: {
nativeBuild = self.packages.${system}.nix;
}))
(lib.genAttrs (linux64BitSystems) (system: {
staticBuild = self.packages.${system}.nix-static;
}))
{
x86_64-linux = {
# TODO add more cross/static release targets?
crossBuild.aarch64-linux = self.packages.x86_64-linux.nix-aarch64-linux;
# TODO wire up a nixos installer test with that lix and
# run it, once nixpkgs can actually do that (again). :/
# # nix build .#nixosTests.installer.{btrfsSimple,luksroot,lvm,simple,switchToFlake}
};
}
];
# NOTE *do not* add fresh derivations to checks, always add them to
# hydraJobs first (so CI will pick them up) and only link them here
checks = forAvailableSystems (
@@ -444,8 +355,6 @@
rec {
inherit (nixpkgsFor.${system}.native) nix;
default = nix;
inherit (nixpkgsFor.${system}.native) lix-clang-tidy;
}
// (
lib.optionalAttrs (builtins.elem system linux64BitSystems) {
@@ -482,7 +391,7 @@
pkgs: stdenv:
let
nix = pkgs.callPackage ./package.nix {
inherit stdenv versionSuffix;
inherit stdenv officialRelease versionSuffix;
busybox-sandbox-shell = pkgs.busybox-sandbox-shell or pkgs.default-busybox-sandbox;
internalApiDocs = false;
};
@@ -512,7 +421,7 @@
makeShell pkgs pkgs.stdenv
))
// {
default = self.devShells.${system}.native-clangStdenvPackages;
default = self.devShells.${system}.native-gcc13StdenvPackages;
}
);
};
-10
View File
@@ -25,13 +25,3 @@ install *OPTIONS: (build OPTIONS)
# Run tests
test *OPTIONS:
meson test -C build --print-errorlogs {{ OPTIONS }}
alias clang-tidy := lint
lint:
ninja -C build clang-tidy
alias clang-tidy-fix := lint-fix
lint-fix:
ninja -C build clang-tidy-fix
+91 -25
View File
@@ -9,10 +9,13 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d468802bab17cbc0cc575e9b053f41e72aa36bfa6b7f55e3529ffa43161b97fa"
[[package]]
name = "countme"
version = "3.0.1"
name = "cbitset"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7704b5fdd17b18ae31c4c1da5a2e0305a2bf17b5249300a9ee9ed7b72114c636"
checksum = "29b6ad25ae296159fb0da12b970b2fe179b234584d7cd294c891e2bbb284466b"
dependencies = [
"num-traits",
]
[[package]]
name = "dissimilar"
@@ -30,26 +33,19 @@ dependencies = [
"once_cell",
]
[[package]]
name = "hashbrown"
version = "0.14.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1"
[[package]]
name = "lix-doc"
version = "0.0.1"
dependencies = [
"expect-test",
"rnix",
"rowan",
]
[[package]]
name = "memoffset"
version = "0.9.1"
name = "num-traits"
version = "0.2.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a"
checksum = "da0df0e5185db44f69b44f26786fe401b6c293d1907744beaa7fa62b2e5a517a"
dependencies = [
"autocfg",
]
@@ -61,25 +57,43 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3fdb12b2476b595f9358c5161aa467c2438859caa136dec86c26fdd2efe17b92"
[[package]]
name = "rnix"
version = "0.11.0"
name = "proc-macro2"
version = "1.0.79"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb35cedbeb70e0ccabef2a31bcff0aebd114f19566086300b8f42c725fc2cb5f"
checksum = "e835ff2298f5721608eb1a980ecaee1aef2c132bf95ecc026a11b7bf3c01c02e"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.35"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "291ec9ab5efd934aaf503a6466c5d5251535d108ee747472c3977cc5acc868ef"
dependencies = [
"proc-macro2",
]
[[package]]
name = "rnix"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0a9b645f0edba447dbfc6473dd22999f46a1d00ab39e777a2713a1cf34a1597b"
dependencies = [
"cbitset",
"rowan",
]
[[package]]
name = "rowan"
version = "0.15.15"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32a58fa8a7ccff2aec4f39cc45bf5f985cec7125ab271cf681c279fd00192b49"
checksum = "1ea7cadf87a9d8432e85cb4eb86bd2e765ace60c24ef86e79084dcae5d1c5a19"
dependencies = [
"countme",
"hashbrown",
"memoffset",
"rustc-hash",
"text-size",
"smol_str",
"text_unit",
"thin-dst",
]
[[package]]
@@ -89,7 +103,59 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2"
[[package]]
name = "text-size"
version = "1.1.1"
name = "serde"
version = "1.0.197"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f18aa187839b2bdb1ad2fa35ead8c4c2976b64e4363c386d45ac0f7ee85c9233"
checksum = "3fb1c873e1b9b056a4dc4c0c198b24c3ffa059243875552b2bd0933b1aee4ce2"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.197"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7eb0b34b42edc17f6b7cac84a52a1c5f0e1bb2227e997ca9011ea3dd34e8610b"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "smol_str"
version = "0.1.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fad6c857cbab2627dcf01ec85a623ca4e7dcb5691cbaa3d7fb7653671f0d09c9"
dependencies = [
"serde",
]
[[package]]
name = "syn"
version = "2.0.53"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7383cd0e49fff4b6b90ca5670bfd3e9d6a733b3f90c686605aa7eec8c4996032"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "text_unit"
version = "0.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "20431e104bfecc1a40872578dbc390e10290a0e9c35fffe3ce6f73c15a9dbfc2"
[[package]]
name = "thin-dst"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db3c46be180f1af9673ebb27bc1235396f61ef6965b3fe0dbb2e624deb604f0e"
[[package]]
name = "unicode-ident"
version = "1.0.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3354b9ac3fae1ff6755cb6db53683adb661634f67557942dea4facebec0fee4b"
+2 -4
View File
@@ -1,6 +1,6 @@
[package]
description = "Nix function documentation tool, stripped down into a library"
edition = "2021"
edition = "2018"
name = "lix-doc"
version = "0.0.1"
license = "BSD-2-Clause OR MIT"
@@ -12,9 +12,7 @@ repository = "https://github.com/lf-/nix-doc"
crate_type = ["staticlib"]
[dependencies]
rnix = "0.11.0"
# Necessary because rnix fails to export a critical trait (Rowan's AstNode).
rowan = "0.15.0"
rnix = "0.8.0"
[dev-dependencies]
expect-test = "1.1.0"
+130 -311
View File
@@ -1,5 +1,5 @@
// SPDX-FileCopyrightText: 2024 Jade Lovelace
// SPDX-FileCopyrightText: 2024 Lunaphied
//
// SPDX-License-Identifier: BSD-2-Clause OR MIT
//! library components of nix-doc
@@ -7,16 +7,13 @@ pub mod pprint;
use crate::pprint::pprint_args;
use rnix::ast::{self, Lambda};
use rnix::{NodeOrToken, SyntaxKind};
use rnix::SyntaxNode;
// Needed because rnix fucked up and didn't reexport this, oops.
use rowan::ast::AstNode;
use rnix::types::{Lambda, TypedNode};
use rnix::SyntaxKind::*;
use rnix::{NodeOrToken, SyntaxNode, TextUnit, WalkEvent};
use std::ffi::{CStr, CString};
use std::fs;
use std::iter;
use std::os::raw::c_char;
use std::panic;
@@ -26,104 +23,66 @@ use std::{fmt::Display, str};
pub type Result<T> = std::result::Result<T, Box<dyn std::error::Error>>;
const DOC_INDENT: usize = 3;
struct SearchResult {
/// Name of the function
identifier: String,
/// Dedented documentation comment
/// Dedented documentation comments
doc: String,
/// Parameter block for the function
param_block: String,
}
fn find_pos(file: &str, line: usize, col: usize) -> usize {
let mut lines = 1;
let mut line_start = 0;
let mut it = file.chars().enumerate().peekable();
while let Some((count, ch)) = it.next() {
if ch == '\n' || ch == '\r' {
lines += 1;
let addend = if ch == '\r' && it.peek().map(|x| x.1) == Some('\n') {
it.next();
1
} else {
0
};
line_start = count + addend;
}
let col_diff = ((count as i32) - (line_start as i32)).abs() as usize;
if lines == line && col_diff == col {
return count;
}
}
unreachable!();
}
impl SearchResult {
fn format<P: Display>(&self, filename: P, line: usize) -> String {
format!(
"**Synopsis:** `{}` = {}\n\n{}\n\n# {}",
self.identifier.as_str(),
self.param_block,
self.doc,
indented(&self.doc, DOC_INDENT),
format!("{}:{}", filename, line).as_str(),
)
}
}
/// Converts Nix compatible line endings (Nix accepts `\r`, `\n`, *and* `\r\n` as endings), to
/// standard `\n` endings for use within Rust land.
fn convert_endings(s: &str) -> String {
let mut out = String::with_capacity(s.len());
let mut it = s.chars().peekable();
while let Some(ch) = it.next() {
if ch == '\n' || ch == '\r' {
out.push('\n');
if ch == '\r' && it.peek().map(|&c| c == '\n').unwrap_or(false) {
// Consume `\n` in `\r\n`.
it.next();
}
} else {
out.push(ch);
}
}
out
}
/// Converts the position information from Lix itself into an character index into the file itself.
/// Expects an input string that's already had it's line endings normalized.
///
/// Note that this returns a *byte* offset, not a character offset.
fn find_pos(s: &str, line: usize, col: usize) -> usize {
// Nix line positions are 1-indexed.
let mut lines = 1;
for (byte_pos, ch) in s.char_indices() {
// If we find a newline, increase the line count.
if ch == '\n' {
lines += 1;
}
// We've arrived at the correct line.
if lines == line {
// Column position is 1-indexed, and it's a *byte* offset, because Nix doesn't actually
// support UTF-8. Rust does though, so we need to convert to a proper byte index to
// match rnix. Lix also doesn't consider the line endings part of the column offset so
// we implicitly add one to advance to the character *after* that.
return byte_pos + col;
}
}
// If things never match that should be literally impossible.
unreachable!();
}
/// Represents a forwarded token from rnix's AST over to lix-doc.
#[derive(Debug, Clone)]
enum DocToken {
Comment(String),
Whitespace(String),
}
/// Determine if a given token string contains more than two newlines, this is used to determine when
/// we hit blank lines between comments indicating a contextually unrelated comment.
fn has_empty_line(tok: &DocToken) -> bool {
// It's either solely whitespace with two newlines inside somewhere, or it's
// contained inside a comment token and we don't want to count that as empty.
if let DocToken::Whitespace(s) = tok {
s.chars().filter(|&c| c == '\n').take(2).count() == 2
} else {
false
}
/// Emits a string `s` indented by `indent` spaces
fn indented(s: &str, indent: usize) -> String {
let indent_s = iter::repeat(' ').take(indent).collect::<String>();
s.split('\n')
.map(|line| indent_s.clone() + line)
.collect::<Vec<_>>()
.join("\n")
}
/// Cleans up a single line, erasing prefix single line comments but preserving indentation
// NOTE: We have a bit of a conflict of interest problem here due to the inconsistent format of
// doc comments. Some doc comments will use a series of single line comments that may then contain `*`
// characters to represent a list. Some will be multiline comments that don't prefix individual lines
// with `*`, only using them for lists directly, and some will prefix lines with `*` as a leading
// character to mark the block. There's no way to disambiguate all three, but we do our best to
// make the common case pretty.
fn cleanup_single_line(s: &str) -> &str {
fn cleanup_single_line<'a>(s: &'a str) -> &'a str {
let mut cmt_new_start = 0;
let mut iter = s.char_indices().peekable();
while let Some((idx, ch)) = iter.next() {
@@ -131,9 +90,7 @@ fn cleanup_single_line(s: &str) -> &str {
let (_, next_ch) = iter.peek().unwrap_or(&(0, '\n'));
// if we find a character, save the byte position after it as our new string start
// This has special handling for `>` because some Nixpkgs documentation has `*>` right
// after the start of their doc comments, and we want to strip the `*` still.
if ch == '#' || (ch == '*' && (*next_ch == '>' || next_ch.is_whitespace())) {
if ch == '#' || (ch == '*' && next_ch.is_whitespace()) {
cmt_new_start = idx + 1;
break;
}
@@ -146,12 +103,15 @@ fn cleanup_single_line(s: &str) -> &str {
&s[cmt_new_start..]
}
/// Erases indents in comments based on the indentation of the first line.
/// Erases indents in comments. This is *almost* a normal dedent function, but it starts by looking
/// at the second line if it can.
fn dedent_comment(s: &str) -> String {
let mut whitespaces = 0;
let mut lines = s.lines();
let first = lines.next();
// scan for whitespace
for line in s.lines() {
for line in lines.chain(first) {
let line_whitespace = line.chars().take_while(|ch| ch.is_whitespace()).count();
if line_whitespace != line.len() {
@@ -161,6 +121,16 @@ fn dedent_comment(s: &str) -> String {
}
}
// maybe the first considered line we found was indented further, so let's look for more lines
// that might have a shorter indent. In the case of one line, do nothing.
for line in s.lines().skip(1) {
let line_whitespace = line.chars().take_while(|ch| ch.is_whitespace()).count();
if line_whitespace != line.len() {
whitespaces = line_whitespace.min(whitespaces);
}
}
// delete up to `whitespaces` whitespace characters from each line and reconstitute the string
let mut out = String::new();
for line in s.lines() {
@@ -173,163 +143,69 @@ fn dedent_comment(s: &str) -> String {
out
}
/// Takes a series of comment and whitespace strings and output a clean single block of text to use
/// as the output documentation comment block.
/// Deletes whitespace and leading comment characters
///
/// This function expects to be given the tokens in reverse order (proceeding upwards from the
/// first comment above the definitions), this allows us to properly enforce the below conditions.
/// The output from this function will be reordered and ready for display.
///
/// The two types of documentation comments we expect are:
///
/// - A single multiline comment not whitespace separated from the start.
/// - A series of back to back single line comments not separated by whitespace.
///
/// Any other combination will be filtered out.
///
/// Once an empty line is encountered, we know no more valid documentation comments remain and stop.
fn cleanup_comments<I: Iterator<Item = DocToken>>(tokens: &mut I) -> String {
// Keep track of when we've found a single line and multiline comment, we use this to
// only process a single multiline or back to back single lines.
let mut found_single_line = false;
// Comments that have survived our filtering phase and should be cleaned up.
let mut valid = vec![];
// Filter out comments that don't meet the characteristics of documentation comments.
for tok in tokens {
if has_empty_line(&tok) {
// Take tokens until we hit whitespace containing an empty line.
break;
}
// Only care about comments from this point on.
if let DocToken::Comment(comment) = tok {
// Now determine if it's a single line comment.
let is_single_line = comment.starts_with('#');
// We've found a single line comment if we've found one before or we just found one.
found_single_line |= is_single_line;
// What we do next is only special when we hit a multiline comment.
if !is_single_line {
// If we've hit a multiline comment as our first comment, take that one alone.
if !found_single_line {
// Otherwise we've hit a multiline comment immediately and this is our
// one and only doc comment to worry about.
valid.push(comment);
}
// Otherwise we've hit a multiline comment after single line comments, in either
// case this means we're done processing comments.
break;
}
// Otherwise this is a new single line comment to push to the stack.
valid.push(comment);
}
}
// Cleanup comments for user consumption.
/// Oversight we are choosing to ignore: if you put # characters at the beginning of lines in a
/// multiline comment, they will be deleted.
fn cleanup_comments<S: AsRef<str>, I: DoubleEndedIterator<Item = S>>(comment: &mut I) -> String {
dedent_comment(
&valid
.into_iter()
&comment
.rev()
.map(|small_comment| {
small_comment
// Trim off start of multiline comments.
.as_ref()
// space before multiline start
.trim_start()
// multiline starts
.trim_start_matches("/*")
// Trim off end of multiline comments.
// trailing so we can grab multiline end
.trim_end()
// multiline ends
.trim_end_matches("*/")
// Trim off any internal whitespace that's trapped inside comments themselves.
// extra space that was in the multiline
.trim()
// Split comments by newlines to extract lines of multiline comments.
.split('\n')
// Cleanup single line comments and a few more tweaks for multiline comments.
// erase single line comments and such
.map(cleanup_single_line)
.collect::<Vec<_>>()
// Reconstruct the multiline comment's whitespace.
.join("\n")
})
.collect::<Vec<_>>()
// We've found that when multiple back to back single line comments are used in Nixpkgs,
// they make more sense to represent as if someone inserted line breaks into the Markdown
// properly, so we join them with linebreaks that markdown will pass through.
.join("\n\n"),
.join("\n"),
)
}
/// Get the docs for a specific function.
// TODO: Improve error reporting?
/// Get the docs for a specific function
pub fn get_function_docs(filename: &str, line: usize, col: usize) -> Option<String> {
let content = fs::read(filename).ok()?;
let decoded = convert_endings(str::from_utf8(&content).ok()?);
let decoded = str::from_utf8(&content).ok()?;
let pos = find_pos(&decoded, line, col);
let rowan_pos = rnix::TextSize::from(pos as u32);
let rowan_pos = TextUnit::from_usize(pos);
let tree = rnix::parse(decoded);
// The minimum length of a lambda is 4 characters and thus the range we're looking for must be
// at least 4 characters long `_: 3` being an example of a minimal length lambda.
let rowan_range = rnix::TextRange::at(rowan_pos, 4.into());
// Parse the file using rnix.
let root = rnix::Root::parse(&decoded).ok().ok()?;
// Extract the inner expression that represents the Root node and extract the top level expression.
let expr = root.expr()?;
// There are two cases we have to be able to handle
// 1. A straightforward definition with an attrset binding to a lambda that's defined inline.
// 2. A lambda defined in a standalone file where the attrset binding imports that file directly.
// The latter case will not be able to find the binding so we must be able to handle not finding it.
// Find the deepest node or token that covers the position given by Lix.
let covering = expr.syntax().covering_element(rowan_range);
// Climb up until we find the lambda node that contains that token.
let mut lambda = None;
for ancestor in covering.ancestors() {
if ancestor.kind() == SyntaxKind::NODE_LAMBDA {
lambda = Some(ancestor);
break;
for node in tree.node().preorder() {
match node {
WalkEvent::Enter(n) => {
if n.text_range().start() >= rowan_pos && n.kind() == NODE_LAMBDA {
lambda = Lambda::cast(n);
break;
}
}
WalkEvent::Leave(_) => (),
}
}
// There is literally always a lambda or something has gone very very wrong.
let lambda =
ast::Lambda::cast(
lambda.expect("no lambda found; what.")
) .expect("not a rnix::ast::Lambda; what.");
// Search up, hopefully to find the binding so we can get the identifier name.
// TODO: Just provide this directly from the C++ code to make it possible to always have the correct identifier.
let mut binding = None;
for ancestor in lambda.syntax().ancestors() {
if ancestor.kind() == SyntaxKind::NODE_ATTRPATH_VALUE {
binding = Some(ancestor);
}
}
// Convert the binding to an identifier if it was found, otherwise use a placeholder.
let identifier;
identifier = match binding.clone() {
Some(binding) => ast::AttrpathValue::cast(binding)
.expect("not an rnix::ast::AttrpathValue; what")
.attrpath()
.expect("AttrpathValue has no attrpath; what.")
.to_string(),
_ => "<unknown binding>".to_string(),
};
// Find all the comments on the binding or the lambda if we have to fall back.
let comment_node = binding.as_ref().unwrap_or(lambda.syntax());
let comment = find_comment(comment_node).unwrap_or_else(String::new);
// And display them properly for the markdown function in Lix.
Some(visit_lambda(identifier, comment, &lambda).format(filename, line))
let lambda = lambda?;
let res = visit_lambda("func".to_string(), &lambda);
Some(res.format(filename, line))
}
fn visit_lambda(name: String, comment: String, lambda: &Lambda) -> SearchResult {
fn visit_lambda(name: String, lambda: &Lambda) -> SearchResult {
// grab the arguments
let param_block = pprint_args(lambda);
let param_block = pprint_args(&lambda);
// find the doc comment
let comment = find_comment(lambda.node().clone()).unwrap_or_else(|| "".to_string());
SearchResult {
identifier: name,
@@ -338,47 +214,39 @@ fn visit_lambda(name: String, comment: String, lambda: &Lambda) -> SearchResult
}
}
fn find_comment(node: &SyntaxNode) -> Option<String> {
let mut it = node
.siblings_with_tokens(rowan::Direction::Prev)
// Skip ourselves as we're always the first token returned.
.skip(1)
.peekable();
// Consume up to one whitespace token before the first comment. There might not always be
// whitespace such as the (rather unusual) case of `/* meow */x = a: 3`.
if matches!(it.peek(), Some(NodeOrToken::Token(token)) if token.kind() == SyntaxKind::TOKEN_WHITESPACE) {
it.next();
}
let comments = it.map_while(|element| match element {
NodeOrToken::Token(token) => {
match token.kind() {
// Map the tokens we're interested in to our internal token type.
SyntaxKind::TOKEN_COMMENT => Some(DocToken::Comment(token.text().to_owned())),
SyntaxKind::TOKEN_WHITESPACE => {
Some(DocToken::Whitespace(token.text().to_owned()))
}
// If we hit a different token type, we know we've gone past relevant comments
// and should stop.
_ => None,
}
fn find_comment(node: SyntaxNode) -> Option<String> {
let mut node = NodeOrToken::Node(node);
let mut comments = Vec::new();
loop {
loop {
if let Some(new) = node.prev_sibling_or_token() {
node = new;
break;
} else {
node = NodeOrToken::Node(node.parent()?);
}
// If we hit a node entry we've definitely gone past comments that would be related to
// this node and we should retreat.
_ => None,
});
}
// For the curious, `into_iter()` here consumes the binding producing an owned value allowing us to avoid
// making the original binding mutable, we don't reuse it later so this is a cute way to handle it, though
// there's probably a better way we just can't remember.
Some(cleanup_comments(&mut comments.into_iter())).filter(|c| !c.is_empty())
match node.kind() {
TOKEN_COMMENT => match &node {
NodeOrToken::Token(token) => comments.push(token.text().clone()),
NodeOrToken::Node(_) => unreachable!(),
},
// This stuff is found as part of `the-fn = f: ...`
// here: ^^^^^^^^
NODE_KEY | TOKEN_ASSIGN => (),
t if t.is_trivia() => (),
_ => break,
}
}
let doc = cleanup_comments(&mut comments.iter().map(|c| c.as_str()));
Some(doc).filter(|it| !it.is_empty())
}
/// Get the docs for a function in the given file path at the given file position and return it as
/// a C string pointer
#[no_mangle]
pub extern "C" fn lixdoc_get_function_docs(
pub extern "C" fn nd_get_function_docs(
filename: *const c_char,
line: usize,
col: usize,
@@ -401,9 +269,9 @@ pub extern "C" fn lixdoc_get_function_docs(
.unwrap_or(ptr::null())
}
/// Call this to free a string from `lixdoc_get_function_docs`.
/// Call this to free a string from nd_get_function_docs
#[no_mangle]
pub extern "C" fn lixdoc_free_string(s: *const c_char) {
pub extern "C" fn nd_free_string(s: *const c_char) {
unsafe {
// cast note: this cast is turning something that was cast to const
// back to mut
@@ -415,57 +283,35 @@ pub extern "C" fn lixdoc_free_string(s: *const c_char) {
mod tests {
use super::*;
#[test]
fn test_line_conversion() {
let fakefile = "abc\rdef\r\nghi";
assert_eq!(convert_endings(fakefile), "abc\ndef\nghi");
}
#[test]
fn test_bytepos() {
let fakefile = "abc\ndef\nghi";
assert_eq!(find_pos(fakefile, 2, 2), 5);
}
#[test]
fn test_bytepos_unusual() {
let fakefile = convert_endings("abc\rdef\r\nghi");
assert_eq!(find_pos(&fakefile, 2, 2), 5);
assert_eq!(find_pos(&fakefile, 3, 2), 9);
}
/// This test is to check that we correctly resolve byte positions even when inconsistent with
/// character positions.
#[test]
fn test_bytepos_cursed() {
let fakefile = "hello\nwórld";
// Try to find the position of the `r` after world, which will be wrong if we don't handle
// UTF-8 properly.
let pos = find_pos(&fakefile, 2, 4);
dbg!(&fakefile[pos..]);
assert_eq!(pos, 9)
let fakefile = "abc\rdef\r\nghi";
assert_eq!(find_pos(fakefile, 2, 2), 5);
assert_eq!(find_pos(fakefile, 3, 2), 10);
}
#[test]
fn test_comment_stripping() {
let ex1 = [DocToken::Comment(
"/* blah blah blah\n foooo baaar\n blah */".to_string(),
)];
let ex1 = ["/* blah blah blah\n foooo baaar\n blah */"];
assert_eq!(
cleanup_comments(&mut ex1.into_iter()),
"blah blah blah\n foooo baaar\n blah"
cleanup_comments(&mut ex1.iter()),
"blah blah blah\n foooo baaar\nblah"
);
let ex2 = ["# a1", "# a2", "# aa"]
.into_iter()
.map(|s| DocToken::Comment(s.to_string()));
assert_eq!(cleanup_comments(&mut ex2.into_iter()), "aa\n\n a2\n\na1");
let ex2 = ["# a1", "# a2", "# aa"];
assert_eq!(cleanup_comments(&mut ex2.iter()), "aa\n a2\na1");
}
#[test]
fn test_dedent() {
let ex1 = "a\n b\n c\n d";
assert_eq!(dedent_comment(ex1), ex1);
assert_eq!(dedent_comment(ex1), "a\nb\nc\n d");
let ex2 = "a\nb\nc";
assert_eq!(dedent_comment(ex2), ex2);
let ex3 = " a\n b\n\n c";
@@ -489,31 +335,4 @@ mod tests {
let ex1 = " **Foo**:";
assert_eq!(cleanup_single_line(ex1), ex1);
}
// TODO: Next CL
//#[test]
//fn comment_test_complex() {
// let testcase = r#"
// rec {
// /*
// Hello
// 23
// This is a comment.
// this is another comment.
// and this is a third comment.
// Way
// go
// */
// meow = { g }: {a, b ? 4, ...}: g: c: 5;
// # And another comment.
// cat = 34;
// # inner layer.
// "inner-layer" = outer: meow;
// }
// "#;
// // Need to find the location of the lambda, we do a quick hack.
// let location = dbg!(testcase.find("{ g }").unwrap() as u32);
//
// //get_function_docs(filename, line, col)
//}
}
+20 -43
View File
@@ -1,59 +1,36 @@
// SPDX-FileCopyrightText: 2024 Jade Lovelace
// SPDX-FileCopyrightText: 2024 Lunaphied
//
// SPDX-License-Identifier: BSD-2-Clause OR MIT
use rnix::ast::{Expr, Lambda};
use rowan::ast::AstNode;
use rnix::types::{Lambda, TypedNode};
use rnix::SyntaxKind::*;
/// Pretty-prints the arguments to a function
pub fn pprint_args(lambda: &Lambda) -> String {
// TODO: handle docs directly on NODE_IDENT args (uncommon case)
let mut lambda = lambda.clone();
let mut depth = 0;
let mut out = String::new();
loop {
let arg = lambda.param().unwrap();
for child in arg.syntax().children_with_tokens() {
//dbg!(child.kind());
match child {
rowan::NodeOrToken::Node(node) => {
out.push_str(&node.text().to_string());
if node.kind() == rnix::SyntaxKind::NODE_PAT_ENTRY {
out.push_str(&",\n");
}
}
rowan::NodeOrToken::Token(token) => {
use rnix::SyntaxKind::{
TOKEN_COMMENT, TOKEN_ELLIPSIS, TOKEN_L_BRACE, TOKEN_QUESTION, TOKEN_R_BRACE,
};
match token.kind() {
TOKEN_COMMENT | TOKEN_ELLIPSIS | TOKEN_QUESTION | TOKEN_L_BRACE
| TOKEN_R_BRACE => {
//dbg!(&token);
out.push_str(&token.text().to_string());
if token.kind() == TOKEN_COMMENT {
out.push('\n');
}
}
_ => {}
}
//out.push_str(&token.text().to_string());
let arg = lambda.arg().unwrap();
match arg.kind() {
NODE_IDENT => {
out += &format!("*{}*", &arg.to_string());
out.push_str(": ");
let body = lambda.body().unwrap();
if body.kind() == NODE_LAMBDA {
lambda = Lambda::cast(body).unwrap();
} else {
break;
}
}
}
out.push_str(": ");
let body = lambda.body().unwrap();
if let Expr::Lambda(inner) = body {
lambda = inner;
// If we recurse we want the next line of recursion to be indented and on a new line.
out.push('\n');
for _ in 0..=depth {
out.push('\t');
NODE_PATTERN => {
out += &format!("*{}*", &arg.to_string());
out.push_str(": ");
break;
}
t => {
unreachable!("unhandled arg type {:?}", t);
}
depth += 1;
} else {
// If we don't find an inner lambda we're done with argument handling.
break;
}
}
out.push_str("...");
+2 -2
View File
@@ -106,7 +106,7 @@ def do_category(author_info: AuthorInfoDB, entries: list[Tuple[pathlib.Path, Any
links = []
links += [format_issue(str(s)) for s in listify(entry.metadata.get('issues', []))]
links += [format_pr(str(s)) for s in listify(entry.metadata.get('prs', []))]
links += [format_cl(int(cl)) for cl in listify(entry.metadata.get('cls', []))]
links += [format_cl(cl) for cl in listify(entry.metadata.get('cls', []))]
if links != []:
header += " " + " ".join(links)
if header:
@@ -129,7 +129,7 @@ def run_on_dir(author_info: AuthorInfoDB, d):
entries = defaultdict(list)
for p in paths:
try:
e = frontmatter.load(p) # type: ignore
e = frontmatter.load(p)
if 'synopsis' not in e.metadata:
raise Exception('missing synopsis')
unknownKeys = set(e.metadata.keys()) - set(KNOWN_KEYS)
-16
View File
@@ -1,16 +0,0 @@
{
runCommandNoCC,
lib,
libseccomp,
writeShellScriptBin,
}:
let
syscalls-csv = runCommandNoCC "syscalls.csv" { } ''
echo ${lib.escapeShellArg libseccomp.src}
tar -xf ${lib.escapeShellArg libseccomp.src} --strip-components=2 ${libseccomp.name}/src/syscalls.csv
mv syscalls.csv "$out"
'';
in
writeShellScriptBin "check-syscalls" ''
${./check-syscalls.sh} ${syscalls-csv}
''
-7
View File
@@ -1,7 +0,0 @@
#!/usr/bin/env bash
set -e
diff -u <(awk < src/libstore/platform/linux.cc '/BEGIN extract-syscalls/ { extracting = 1; next }
match($0, /allowSyscall\(ctx, SCMP_SYS\(([^)]*)\)\);|\/\/ skip ([^ ]*)/, result) { print result[1] result[2] }
/END extract-syscalls/ { extracting = 0; next }') <(tail -n+2 "$1" | cut -d, -f 1)
+25 -77
View File
@@ -199,27 +199,23 @@ configdata = { }
# Dependencies
#
gc_opt = get_option('gc').disable_if(
'address' in get_option('b_sanitize'),
error_message: 'gc does far too many memory crimes for ASan'
)
boehm = dependency('bdw-gc', required : gc_opt, version : '>=8.2.6', include_type : 'system')
boehm = dependency('bdw-gc', required : get_option('gc'), version : '>=8.2.6')
configdata += {
'HAVE_BOEHMGC': boehm.found().to_int(),
}
boost = dependency('boost', required : true, modules : ['container'], include_type : 'system')
boost = dependency('boost', required : true, modules : ['container'])
# cpuid only makes sense on x86_64
cpuid_required = is_x64 ? get_option('cpuid') : false
cpuid = dependency('libcpuid', 'cpuid', required : cpuid_required, include_type : 'system')
cpuid = dependency('libcpuid', 'cpuid', required : cpuid_required)
configdata += {
'HAVE_LIBCPUID': cpuid.found().to_int(),
}
# seccomp only makes sense on Linux
seccomp_required = is_linux ? get_option('seccomp-sandboxing') : false
seccomp = dependency('libseccomp', 'seccomp', required : seccomp_required, version : '>=2.5.5', include_type : 'system')
seccomp = dependency('libseccomp', 'seccomp', required : seccomp_required, version : '>=2.5.5')
if is_linux and not seccomp.found()
warning('Sandbox security is reduced because libseccomp has not been found! Please provide libseccomp if it supports your CPU architecture.')
endif
@@ -227,29 +223,19 @@ configdata += {
'HAVE_SECCOMP': seccomp.found().to_int(),
}
# fcntl(F_GETPATH) returns the path of an fd on macOS and BSDs
configdata += {
'HAVE_F_GETPATH': cxx.has_header_symbol('fcntl.h', 'F_GETPATH').to_int(),
}
libarchive = dependency('libarchive', required : true, include_type : 'system')
libarchive = dependency('libarchive', required : true)
brotli = [
dependency('libbrotlicommon', required : true, include_type : 'system'),
dependency('libbrotlidec', required : true, include_type : 'system'),
dependency('libbrotlienc', required : true, include_type : 'system'),
dependency('libbrotlicommon', required : true),
dependency('libbrotlidec', required : true),
dependency('libbrotlienc', required : true),
]
openssl = dependency('libcrypto', 'openssl', required : true, include_type : 'system')
openssl = dependency('libcrypto', 'openssl', required : true)
# FIXME: confirm we actually support such old versions of aws-sdk-cpp
aws_sdk = dependency('aws-cpp-sdk-core', required : false, version : '>=1.8', include_type : 'system')
aws_sdk_transfer = dependency(
'aws-cpp-sdk-transfer',
required : aws_sdk.found(),
fallback : ['aws_sdk', 'aws_cpp_sdk_transfer_dep'],
include_type : 'system',
)
aws_sdk = dependency('aws-cpp-sdk-core', required : false, version : '>=1.8')
aws_sdk_transfer = dependency('aws-cpp-sdk-transfer', required : aws_sdk.found(), fallback : ['aws_sdk', 'aws_cpp_sdk_transfer_dep'])
if aws_sdk.found()
# The AWS pkg-config adds -std=c++11.
# https://github.com/aws/aws-sdk-cpp/issues/2673
@@ -269,12 +255,7 @@ if aws_sdk.found()
)
endif
aws_s3 = dependency(
'aws-cpp-sdk-s3',
required : aws_sdk.found(),
fallback : ['aws_sdk', 'aws_cpp_sdk_s3_dep'],
include_type : 'system',
)
aws_s3 = dependency('aws-cpp-sdk-s3', required : aws_sdk.found(), fallback : ['aws_sdk', 'aws_cpp_sdk_s3_dep'])
if aws_s3.found()
# The AWS pkg-config adds -std=c++11.
# https://github.com/aws/aws-sdk-cpp/issues/2673
@@ -291,35 +272,30 @@ configdata += {
'ENABLE_S3': aws_s3.found().to_int(),
}
sqlite = dependency('sqlite3', 'sqlite', version : '>=3.6.19', required : true, include_type : 'system')
sqlite = dependency('sqlite3', 'sqlite', version : '>=3.6.19', required : true)
sodium = dependency('libsodium', 'sodium', required : true, include_type : 'system')
sodium = dependency('libsodium', 'sodium', required : true)
curl = dependency('libcurl', 'curl', required : true, include_type : 'system')
curl = dependency('libcurl', 'curl', required : true)
editline = dependency('libeditline', 'editline', version : '>=1.14', required : true, include_type : 'system')
editline = dependency('libeditline', 'editline', version : '>=1.14', required : true)
lowdown = dependency('lowdown', version : '>=0.9.0', required : true, include_type : 'system')
# TODO(sterni): drop the corresponding #ifdef after NixOS 25.05 is EOL which still distributes lowdown < 1.4.0
if lowdown.version().version_compare('>= 1.4.0')
add_project_arguments('-DLOWDOWN_SEPARATE_TERM_OPTS', language: 'cpp')
endif
lowdown = dependency('lowdown', version : '>=0.9.0', required : true)
# HACK(Qyriad): rapidcheck's pkg-config doesn't include the libs lol
# Note: technically we 'check' for rapidcheck twice, for the internal-api-docs handling above,
# but Meson will cache the result of the first one, and the required : arguments are different.
rapidcheck_meson = dependency('rapidcheck', required : enable_tests, include_type : 'system')
rapidcheck_meson = dependency('rapidcheck', required : enable_tests)
rapidcheck = declare_dependency(dependencies : rapidcheck_meson, link_args : ['-lrapidcheck'])
gtest = [
dependency('gtest', required : enable_tests, include_type : 'system'),
dependency('gtest_main', required : enable_tests, include_type : 'system'),
dependency('gmock', required : enable_tests, include_type : 'system'),
dependency('gmock_main', required : enable_tests, include_type : 'system'),
dependency('gtest', required : enable_tests),
dependency('gtest_main', required : enable_tests),
dependency('gmock', required : enable_tests),
dependency('gmock_main', required : enable_tests),
]
toml11 = dependency('toml11', version : '>=3.7.0', required : true, method : 'cmake', include_type : 'system')
toml11 = dependency('toml11', version : '>=3.7.0', required : true, method : 'cmake')
pegtl = dependency(
'pegtl',
@@ -327,10 +303,9 @@ pegtl = dependency(
required : true,
method : 'cmake',
modules : [ 'taocpp::pegtl' ],
include_type : 'system',
)
nlohmann_json = dependency('nlohmann_json', required : true, include_type : 'system')
nlohmann_json = dependency('nlohmann_json', required : true)
# lix-doc is a Rust project provided via buildInputs and unfortunately doesn't have any way to be detected.
# Just declare it manually to resolve this.
@@ -339,10 +314,6 @@ nlohmann_json = dependency('nlohmann_json', required : true, include_type : 'sys
# *absolutely* are not going to make it work)
lix_doc = declare_dependency(link_args : [ '-llix_doc' ])
if is_freebsd
libprocstat = declare_dependency(link_args : [ '-lprocstat' ])
endif
#
# Build-time tools
#
@@ -381,13 +352,6 @@ endif
# FIXME(Qyriad): the autoconf system checks that busybox has the "standalone" feature, indicating
# that busybox sh won't run busybox applets as builtins (which would break our sandbox).
pasta_path = get_option('pasta-path')
# we can't check the pasta version because passt misuses stdio (it calls _exit()
# after printing the version, which will never print the version unless run from
# a terminal). pasta isn't mandatory yet due to high fetcher breakage potential.
# we *will* enable it in our own packaging, but distributions are not forced to.
pasta = find_program(pasta_path, required : false, native : false)
lsof = find_program('lsof', native : true)
# This is how Nix does generated headers...
@@ -434,11 +398,6 @@ check_funcs = [
'strsignal',
'sysconf',
]
if is_linux or is_freebsd
# musl does not have close_range as of 2024-08-10
# patch: https://www.openwall.com/lists/musl/2024/08/01/9
check_funcs += [ 'close_range' ]
endif
foreach funcspec : check_funcs
define_name = 'HAVE_' + funcspec.underscorify().to_upper()
define_value = cxx.has_function(funcspec).to_int()
@@ -480,14 +439,12 @@ add_project_arguments(
# It would be nice for our headers to be idempotent instead.
'-include', 'config.h',
'-Wno-deprecated-declarations',
'-Wno-missing-field-initializers',
'-Wimplicit-fallthrough',
'-Werror=switch',
'-Werror=switch-enum',
'-Werror=unused-result',
'-Wdeprecated-copy',
'-Wignored-qualifiers',
'-Werror=suggest-override',
# Enable assertions in libstdc++ by default. Harmless on libc++. Benchmarked
# at ~1% overhead in `nix search`.
#
@@ -520,14 +477,7 @@ if cxx.get_id() == 'clang' and get_option('b_sanitize') != ''
add_project_link_arguments('-shared-libsan', language : 'cpp')
endif
# Clang gets grumpy about missing libasan symbols if -shared-libasan is not
# passed when building shared libs, at least on Linux
if cxx.get_id() == 'clang' and 'address' in get_option('b_sanitize')
add_project_link_arguments('-shared-libasan', language : 'cpp')
endif
add_project_link_arguments('-pthread', language : 'cpp')
if cxx.get_linker_id() in ['ld.bfd', 'ld.gold']
add_project_link_arguments('-Wl,--no-copy-dt-needed-entries', language : 'cpp')
endif
@@ -542,7 +492,7 @@ endif
# maintainers/buildtime_report.sh BUILD-DIR to simply work in clang builds.
#
# They can also be manually viewed at https://ui.perfetto.dev
if get_option('profile-build').require(cxx.get_id() == 'clang').enabled()
if get_option('profile-build').require(meson.get_compiler('cpp').get_id() == 'clang').enabled()
add_project_arguments('-ftime-trace', language: 'cpp')
endif
@@ -582,5 +532,3 @@ if enable_tests
subdir('tests/unit')
subdir('tests/functional')
endif
subdir('meson/clang-tidy')
+2 -10
View File
@@ -1,7 +1,7 @@
# vim: filetype=meson
option('enable-build', type : 'boolean', value : true,
description : 'set to false to not actually build. Only really makes sense with -Dinternal-api-docs=true',
description : 'Set to false to not actually build. Only really makes sense with -Dinternal-api-docs=true',
)
option('gc', type : 'feature',
@@ -24,10 +24,6 @@ option('sandbox-shell', type : 'string', value : 'busybox',
description : 'path to a statically-linked shell to use as /bin/sh in sandboxes (usually busybox)',
)
option('pasta-path', type : 'string', value : 'pasta',
description : 'path to the location of pasta (provided by passt)',
)
option('enable-tests', type : 'boolean', value : true,
description : 'whether to enable tests or not (requires rapidcheck and gtest)',
)
@@ -41,7 +37,7 @@ option('tests-brief', type : 'boolean', value : false,
)
option('profile-build', type : 'feature', value: 'disabled',
description : 'whether to enable -ftime-trace in clang builds, allowing for diagnosing the cause of build time.'
description : 'whether to enable -ftime-trace in clang builds, allowing for speeding up the build.'
)
option('store-dir', type : 'string', value : '/nix/store',
@@ -72,7 +68,3 @@ option('profile-dir', type : 'string', value : 'etc/profile.d',
option('enable-pch-std', type : 'boolean', value : true,
description : 'whether to use precompiled headers for C++\'s standard library (breaks clangd if you\'re using GCC)',
)
option('lix-clang-tidy-checks-path', type : 'string', value : '',
description: 'path to lix-clang-tidy-checks library file, if providing it externally. Uses an internal one if this is not set',
)
@@ -1,21 +0,0 @@
#!/usr/bin/env python3
import subprocess
def get_targets_of_rule(build_root: str, rule_name: str) -> list[str]:
return subprocess.check_output(['ninja', '-C', build_root, '-t', 'targets', 'rule', rule_name]).decode().strip().splitlines()
def ninja_build(build_root: str, targets: list[str]):
subprocess.check_call(['ninja', '-C', build_root, '--', *targets])
def main():
import argparse
ap = argparse.ArgumentParser(description='Builds required targets for clang-tidy')
ap.add_argument('build_root', help='Ninja build root', type=str)
args = ap.parse_args()
targets = [t for t in get_targets_of_rule(args.build_root, 'CUSTOM_COMMAND') if t.endswith('gen.hh')]
ninja_build(args.build_root, targets)
if __name__ == '__main__':
main()
-53
View File
@@ -1,53 +0,0 @@
#!/usr/bin/env python3
# Deletes the PCH arguments from a compilation database, to workaround nixpkgs
# stdenv having a cc-wrapper that is impossible to use for anything except cc
# itself, for example, clang-tidy.
import json
import shlex
def process_compdb(compdb: list[dict]) -> list[dict]:
def munch_command(args: list[str]) -> list[str]:
out = []
eat_next = False
for i, arg in enumerate(args):
if arg == '-fpch-preprocess':
# as used with gcc
continue
elif arg == '-include-pch' or (arg == '-include' and args[i + 1] == 'precompiled-headers.hh'):
# -include-pch some-pch (clang), or -include some-pch (gcc)
eat_next = True
continue
if not eat_next:
out.append(arg)
eat_next = False
return out
def chomp(item: dict) -> dict:
item = item.copy()
item['command'] = shlex.join(munch_command(shlex.split(item['command'])))
return item
return [chomp(x) for x in compdb if not x['file'].endswith('precompiled-headers.hh')]
def main():
import argparse
ap = argparse.ArgumentParser(
description='Delete pch arguments from compilation database')
ap.add_argument('input',
type=argparse.FileType('r'),
help='Input json file')
ap.add_argument('output',
type=argparse.FileType('w'),
help='Output json file')
args = ap.parse_args()
input_json = json.load(args.input)
json.dump(process_compdb(input_json), args.output, indent=2)
if __name__ == '__main__':
main()
-98
View File
@@ -1,98 +0,0 @@
# The clang-tidy target for Lix
run_clang_tidy = find_program('run-clang-tidy', required : false)
# Although this looks like it wants to be pkg-config, pkg-config does not
# really work for *plugins*, which are executable-like .so files that also
# cannot be found via find_program. Fun!
if get_option('lix-clang-tidy-checks-path') != ''
lix_clang_tidy_so = get_option('lix-clang-tidy-checks-path')
lix_clang_tidy_so_found = true
else
lix_clang_tidy_subproj = subproject(
'lix-clang-tidy',
required : false,
default_options : {'build-by-default': false}
)
if lix_clang_tidy_subproj.found()
lix_clang_tidy_so = lix_clang_tidy_subproj.get_variable('lix_clang_tidy')
lix_clang_tidy_so_found = true
else
lix_clang_tidy_so_found = false
endif
endif
# Due to numerous problems, such as:
# - Meson does not expose pch targets, but *fine*, I can just ask Ninja for
# them with `ninja -t targets rule cpp_PCH` and build them manually:
# https://github.com/mesonbuild/meson/issues/13499
# - Nixpkgs stdenv buries the cc-wrapper under a giant pile of assumptions
# about the cc-wrapper actually being used on the cc of a stdenv, rather than
# independently for clang-tidy, and we need to use cc-wrapper to get the
# correct hardening flags so that clang-tidy can actually parse the PCH file
#
# I give up. I am going to delete the damn PCH args and then it will work.
meson.add_postconf_script(
python,
meson.current_source_dir() / 'clean_compdb.py',
meson.global_build_root() / 'compile_commands.json',
meson.current_build_dir() / 'compile_commands.json',
)
# Horrible hack to get around not being able to depend on another target's
# generated headers in any way in the meson DSL
# https://github.com/mesonbuild/meson/issues/12817 which was incorrectly
# closed, if you *actually* need to generate the files once.
# Also related: https://github.com/mesonbuild/meson/issues/3667
#
# Or we could ban meson generators because their design is broken.
build_all_generated_headers = custom_target(
command : [
python,
meson.current_source_dir() / 'build_required_targets.py',
meson.global_build_root(),
],
output : 'generated_headers.stamp',
build_by_default : false,
build_always_stale : true,
)
if lix_clang_tidy_so_found
run_clang_tidy_args = [
'-load',
lix_clang_tidy_so,
'-p',
# We have to workaround a run-clang-tidy bug too, so we must give the
# directory name rather than the actual compdb file.
# https://github.com/llvm/llvm-project/issues/101440
meson.current_build_dir(),
'-quiet',
]
run_target(
'clang-tidy',
command : [
# XXX: This explicitly invokes it with python because of a nixpkgs bug
# where clang-unwrapped does not patch interpreters in run-clang-tidy.
# However, making clang-unwrapped depend on python is also silly, so idk.
python,
run_clang_tidy,
run_clang_tidy_args,
'-warnings-as-errors',
'*',
],
depends : [
build_all_generated_headers,
],
)
run_target(
'clang-tidy-fix',
command : [
python,
run_clang_tidy,
run_clang_tidy_args,
'-fix',
],
depends : [
build_all_generated_headers,
],
)
endif
-6
View File
@@ -4,9 +4,3 @@ subdir('zsh')
subdir('systemd')
subdir('flake-registry')
runinpty = configure_file(
copy : true,
input : meson.current_source_dir() / 'runinpty.py',
output : 'runinpty.py',
)
-60
View File
@@ -1,60 +0,0 @@
{
lib,
stdenv,
buildPackages,
fetchurl,
getconf,
gitUpdater,
testers,
}:
stdenv.mkDerivation (finalAttrs: {
pname = "passt";
version = "2025_02_17.a1e48a0";
src = fetchurl {
url = "https://passt.top/passt/snapshot/passt-${finalAttrs.version}.tar.gz";
hash = "sha256-/FUXxeYv3Lb0DiXmbS2PUzfLL5ZwHJ42tiuH7YnlljE=";
};
postPatch = ''
substituteInPlace Makefile --replace-fail \
'PAGE_SIZE=$(shell getconf PAGE_SIZE)' \
"PAGE_SIZE=$(${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe getconf} PAGE_SIZE)"
'';
makeFlags = [
"prefix=${placeholder "out"}"
"VERSION=${finalAttrs.version}"
];
passthru = {
tests.version = testers.testVersion { package = finalAttrs.finalPackage; };
updateScript = gitUpdater { url = "https://passt.top/passt"; };
};
meta = with lib; {
homepage = "https://passt.top/passt/about/";
description = "Plug A Simple Socket Transport";
longDescription = ''
passt implements a translation layer between a Layer-2 network interface
and native Layer-4 sockets (TCP, UDP, ICMP/ICMPv6 echo) on a host.
It doesn't require any capabilities or privileges, and it can be used as
a simple replacement for Slirp.
pasta (same binary as passt, different command) offers equivalent
functionality, for network namespaces: traffic is forwarded using a tap
interface inside the namespace, without the need to create further
interfaces on the host, hence not requiring any capabilities or
privileges.
'';
license = [
licenses.bsd3 # and
licenses.gpl2Plus
];
platforms = platforms.linux;
maintainers = with maintainers; [ _8aed ];
mainProgram = "passt";
};
})
+23
View File
@@ -0,0 +1,23 @@
{
stdenv,
cmake,
ninja,
fetchFromGitHub,
}:
stdenv.mkDerivation {
pname = "pegtl";
version = "3.2.7";
src = fetchFromGitHub {
repo = "PEGTL";
owner = "taocpp";
rev = "refs/tags/3.2.7";
hash = "sha256-IV5YNGE4EWVrmg2Sia/rcU8jCuiBynQGJM6n3DCWTQU=";
};
nativeBuildInputs = [
cmake
ninja
];
}
+1 -1
View File
@@ -106,7 +106,7 @@ pre-commit-run {
};
treefmt = {
enable = true;
settings.formatters = [ pkgs.nixfmt-rfc-style ];
settings.formatters = [ pkgs.nixfmt ];
};
};
}
-77
View File
@@ -1,77 +0,0 @@
#!/usr/bin/env python3
# SPDX-FileCopyrightText: 2001, 2002, 2003, 2004, 2005, 2006 Python Software Foundation; All Rights Reserved
# SPDX-FileCopyrightText: 2024 Jade Lovelace
# SPDX-License-Identifier: LGPL-2.1-or-later
"""
This script exists to lose Lix a dependency on expect(1) for the ability to run
something in a pty.
Yes, it could be replaced by script(1) but macOS and Linux script(1) have
diverged sufficiently badly that even specifying a subcommand to run is not the
same.
"""
import pty
import sys
import os
from termios import ONLCR, ONLRET, ONOCR, OPOST, TCSAFLUSH, tcgetattr, tcsetattr
from tty import setraw
import termios
def setup_terminal():
# does not matter which fd we use because we are in a fresh pty
modi = tcgetattr(pty.STDOUT_FILENO)
[iflag, oflag, cflag, lflag, ispeed, ospeed, cc] = modi
# Turning \n into \r\n is not cool, Linux!
oflag &= ~ONLCR
# I don't know what "implementation dependent postprocessing means" but it
# sounds bad
oflag &= ~OPOST
# Assume that NL performs the role of CR; do not insert CRs at column 0
oflag |= ONLRET | ONOCR
modi = [iflag, oflag, cflag, lflag, ispeed, ospeed, cc]
tcsetattr(pty.STDOUT_FILENO, TCSAFLUSH, modi)
def spawn(argv: list[str]):
"""
As opposed to pty.spawn, this one more seriously controls the pty settings.
Necessary to turn off such fun functionality as onlcr (LF to CRLF).
This is essentially copy pasted from pty.spawn, since there is no way to
hook the child pre-execve
"""
pid, master_fd = pty.fork()
if pid == pty.CHILD:
setup_terminal()
os.execlp(argv[0], *argv)
try:
mode = tcgetattr(pty.STDIN_FILENO)
setraw(pty.STDIN_FILENO)
restore = True
except termios.error:
restore = False
try:
pty._copy(master_fd, pty._read, pty._read) # type: ignore
finally:
if restore:
tcsetattr(pty.STDIN_FILENO, TCSAFLUSH, mode) # type: ignore
os.close(master_fd)
return os.waitpid(pid, 0)[1]
def main():
if len(sys.argv) == 1:
print(f'Usage: {sys.argv[0]} [command args]', file=sys.stderr)
sys.exit(1)
sys.exit(os.waitstatus_to_exitcode(spawn(sys.argv[1:])))
if __name__ == '__main__':
main()
+1 -2
View File
@@ -1,2 +1 @@
d @localstatedir@/nix/daemon-socket 0755 root root - -
d @localstatedir@/nix/builds 0755 root root 7d -
d @localstatedir@/nix/daemon-socket 0755 root root - -
+1 -1
View File
@@ -1,6 +1,6 @@
[Unit]
Description=Nix Daemon
Documentation=man:nix-daemon https://docs.lix.systems/manual/lix/stable
Documentation=man:nix-daemon https://nixos.org/manual
RequiresMountsFor=@storedir@
RequiresMountsFor=@localstatedir@
RequiresMountsFor=@localstatedir@/nix/db
+22 -26
View File
@@ -5,33 +5,29 @@
system,
}:
let
rootPaths = [
nix
cacert
];
installerClosureInfo = buildPackages.closureInfo { inherit rootPaths; };
installerClosureInfo = buildPackages.closureInfo {
rootPaths = [
nix
cacert
];
};
meta.description = "Distribution-independent Lix bootstrap binaries for ${system}";
in
buildPackages.runCommand "lix-binary-tarball-${nix.version}"
{
inherit meta;
passthru.rootPaths = rootPaths;
}
''
cp ${installerClosureInfo}/registration $TMPDIR/reginfo
buildPackages.runCommand "lix-binary-tarball-${nix.version}" { inherit meta; } ''
cp ${installerClosureInfo}/registration $TMPDIR/reginfo
dir=lix-${nix.version}-${system}
fn=$out/$dir.tar.xz
mkdir -p $out/nix-support
echo "file binary-dist $fn" >> $out/nix-support/hydra-build-products
tar cvfJ $fn \
--owner=0 --group=0 --mode=u+rw,uga+r \
--mtime='1970-01-01' \
--absolute-names \
--hard-dereference \
--transform "s,$TMPDIR/reginfo,$dir/.reginfo," \
--transform "s,$NIX_STORE,$dir/store,S" \
$TMPDIR/reginfo \
$(cat ${installerClosureInfo}/store-paths)
''
dir=lix-${nix.version}-${system}
fn=$out/$dir.tar.xz
mkdir -p $out/nix-support
echo "file binary-dist $fn" >> $out/nix-support/hydra-build-products
tar cvfJ $fn \
--owner=0 --group=0 --mode=u+rw,uga+r \
--mtime='1970-01-01' \
--absolute-names \
--hard-dereference \
--transform "s,$TMPDIR/reginfo,$dir/.reginfo," \
--transform "s,$NIX_STORE,$dir/store,S" \
$TMPDIR/reginfo \
$(cat ${installerClosureInfo}/store-paths)
''
+65
View File
@@ -0,0 +1,65 @@
# Copy of `nixfmt-rfc-style` vendored from `nixpkgs` master:
# https://github.com/NixOS/nixpkgs/blob/ab6071eb54cc9b66dda436111d4f569e4e56cbf4/pkgs/by-name/ni/nixfmt-rfc-style/package.nix
{
haskell,
haskellPackages,
fetchFromGitHub,
}:
let
inherit (haskell.lib.compose) justStaticExecutables;
raw-pkg = haskellPackages.callPackage (
{
mkDerivation,
base,
cmdargs,
directory,
fetchzip,
filepath,
lib,
megaparsec,
mtl,
parser-combinators,
safe-exceptions,
scientific,
text,
transformers,
unix,
}:
mkDerivation {
pname = "nixfmt";
version = "0.6.0-unstable-2024-03-14";
src = fetchFromGitHub {
owner = "serokell";
repo = "nixfmt";
rev = "8d13b593fa8d8d6e5075f541f3231222a08e84df";
hash = "sha256-HtXvzmfN4wk45qiKZ7V+/5WBV7jnTHfd7iBwF4XGl64=";
};
isLibrary = true;
isExecutable = true;
libraryHaskellDepends = [
base
megaparsec
mtl
parser-combinators
scientific
text
transformers
];
executableHaskellDepends = [
base
cmdargs
directory
filepath
safe-exceptions
text
unix
];
jailbreak = true;
homepage = "https://github.com/serokell/nixfmt";
description = "An opinionated formatter for Nix";
license = lib.licenses.mpl20;
mainProgram = "nixfmt";
}
) { };
in
justStaticExecutables raw-pkg
+121 -159
View File
@@ -27,11 +27,7 @@
libcpuid,
libseccomp,
libsodium,
lix-clang-tidy ? null,
llvmPackages,
lsof,
# FIXME: remove default after dropping NixOS 24.05
lowdown-unsandboxed ? lowdown,
lowdown,
mdbook,
mdbook-linkcheck,
@@ -39,9 +35,7 @@
meson,
ninja,
openssl,
# FIXME: we need passt 2024_12_11.09478d5 or newer, i.e. nixos 25.05 or later
passt-lix ? __forDefaults.passt-lix,
pegtl,
pegtl ? __forDefaults.pegtl,
pkg-config,
python3,
rapidcheck,
@@ -58,29 +52,16 @@
pname ? "lix",
versionSuffix ? "",
officialRelease ? __forDefaults.versionJson.official_release,
officialRelease ? false,
# Set to true to build the release notes for the next release.
buildUnreleasedNotes ? true,
internalApiDocs ? false,
# Support garbage collection in the evaluator.
enableGC ? sanitize == null || !builtins.elem "address" sanitize,
# List of Meson sanitize options. Accepts values of b_sanitize, e.g.
# "address", "undefined", "thread".
# Enabling the "address" sanitizer will disable garbage collection in the evaluator.
sanitize ? null,
# Turn compiler warnings into errors.
werror ? false,
lintInsteadOfBuild ? false,
# Not a real argument, just the only way to approximate let-binding some
# stuff for argument defaults.
__forDefaults ? {
canRunInstalled = stdenv.buildPlatform.canExecute stdenv.hostPlatform;
versionJson = builtins.fromJSON (builtins.readFile ./version.json);
boehmgc-nix = boehmgc.override { enableLargeConfig = true; };
editline-lix = editline.overrideAttrs (prev: {
@@ -90,7 +71,7 @@
lix-doc = callPackage ./lix-doc/package.nix { };
build-release-notes = callPackage ./maintainers/build-release-notes.nix { };
passt-lix = callPackage ./misc/passt.nix { };
pegtl = callPackage ./misc/pegtl.nix { };
},
}:
let
@@ -98,39 +79,51 @@ let
inherit (lib) fileset;
inherit (stdenv) hostPlatform buildPlatform;
version = __forDefaults.versionJson.version + versionSuffix;
versionJson = builtins.fromJSON (builtins.readFile ./version.json);
version = versionJson.version + versionSuffix;
aws-sdk-cpp-nix =
if aws-sdk-cpp == null then
null
else
aws-sdk-cpp.override {
apis = [
"s3"
"transfer"
];
customMemoryManagement = false;
};
aws-sdk-cpp-nix = aws-sdk-cpp.override {
apis = [
"s3"
"transfer"
];
customMemoryManagement = false;
};
# Reimplementation of Nixpkgs' Meson cross file, with some additions to make
# it actually work.
mesonCrossFile = builtins.toFile "lix-cross-file.conf" ''
[properties]
# Meson is convinced that if !buildPlatform.canExecute hostPlatform then we cannot
# build anything at all, which is not at all correct. If we can't execute the host
# platform, we'll just disable tests and doc gen.
needs_exe_wrapper = false
mesonCrossFile =
let
cpuFamily =
platform:
with platform;
if isAarch32 then
"arm"
else if isx86_32 then
"x86"
else
platform.uname.processor;
in
builtins.toFile "lix-cross-file.conf" ''
[properties]
# Meson is convinced that if !buildPlatform.canExecute hostPlatform then we cannot
# build anything at all, which is not at all correct. If we can't execute the host
# platform, we'll just disable tests and doc gen.
needs_exe_wrapper = false
[binaries]
# Meson refuses to consider any CMake binary during cross compilation if it's
# not explicitly specified here, in the cross file.
# https://github.com/mesonbuild/meson/blob/0ed78cf6fa6d87c0738f67ae43525e661b50a8a2/mesonbuild/cmake/executor.py#L72
cmake = 'cmake'
'';
[binaries]
# Meson refuses to consider any CMake binary during cross compilation if it's
# not explicitly specified here, in the cross file.
# https://github.com/mesonbuild/meson/blob/0ed78cf6fa6d87c0738f67ae43525e661b50a8a2/mesonbuild/cmake/executor.py#L72
cmake = 'cmake'
'';
# The internal API docs need these for the build, but if we're not building
# Nix itself, then these don't need to be propagated.
maybePropagatedInputs = lib.optional enableGC boehmgc-nix ++ [ nlohmann_json ];
maybePropagatedInputs = [
boehmgc-nix
nlohmann_json
];
# .gitignore has already been processed, so any changes in it are irrelevant
# at this point. It is not represented verbatim for test purposes because
@@ -153,7 +146,6 @@ let
(fileset.fileFilter (f: lib.strings.hasPrefix "nix-profile" f.name) ./scripts)
];
in
assert (lintInsteadOfBuild -> lix-clang-tidy != null);
stdenv.mkDerivation (finalAttrs: {
inherit pname version;
@@ -166,120 +158,100 @@ stdenv.mkDerivation (finalAttrs: {
topLevelBuildFiles
functionalTestFiles
]
++ lib.optionals (!finalAttrs.dontBuild || internalApiDocs || lintInsteadOfBuild) [
++ lib.optionals (!finalAttrs.dontBuild || internalApiDocs) [
./doc
./misc
./src
./COPYING
]
++ lib.optionals lintInsteadOfBuild [ ./.clang-tidy ]
)
);
};
VERSION_SUFFIX = versionSuffix;
outputs = [
"out"
]
++ lib.optionals (!finalAttrs.dontBuild) [
"dev"
"doc"
];
outputs =
[ "out" ]
++ lib.optionals (!finalAttrs.dontBuild) [
"dev"
"doc"
];
dontBuild = lintInsteadOfBuild;
dontBuild = false;
mesonFlags =
let
sanitizeOpts = lib.optional (
sanitize != null
) "-Db_sanitize=${builtins.concatStringsSep "," sanitize}";
in
lib.optionals hostPlatform.isLinux [
# You'd think meson could just find this in PATH, but busybox is in buildInputs,
# which don't actually get added to PATH. And buildInputs is correct over
# nativeBuildInputs since this should be a busybox executable on the host.
"-Dsandbox-shell=${lib.getExe' busybox-sandbox-shell "busybox"}"
"-Dpasta-path=${lib.getExe' passt-lix "pasta"}"
]
++ lib.optional hostPlatform.isStatic "-Denable-embedded-sandbox-shell=true"
++ lib.optional (finalAttrs.dontBuild && !lintInsteadOfBuild) "-Denable-build=false"
++ lib.optional lintInsteadOfBuild "-Dlix-clang-tidy-checks-path=${lix-clang-tidy}/lib/liblix-clang-tidy.so"
++ lib.optional (finalAttrs.dontBuild) "-Denable-build=false"
++ [
# mesonConfigurePhase automatically passes -Dauto_features=enabled,
# so we must explicitly enable or disable features that we are not passing
# dependencies for.
(lib.mesonEnable "gc" enableGC)
(lib.mesonEnable "internal-api-docs" internalApiDocs)
(lib.mesonBool "enable-tests" (finalAttrs.finalPackage.doCheck || lintInsteadOfBuild))
(lib.mesonBool "enable-tests" finalAttrs.finalPackage.doCheck)
(lib.mesonBool "enable-docs" canRunInstalled)
(lib.mesonBool "werror" werror)
]
++ lib.optional (hostPlatform != buildPlatform) "--cross-file=${mesonCrossFile}"
++ sanitizeOpts;
++ lib.optional (hostPlatform != buildPlatform) "--cross-file=${mesonCrossFile}";
# We only include CMake so that Meson can locate toml11, which only ships CMake dependency metadata.
dontUseCmakeConfigure = true;
nativeBuildInputs = [
python3
meson
ninja
cmake
]
++ [
(lib.getBin lowdown-unsandboxed)
mdbook
mdbook-linkcheck
]
++ [
pkg-config
nativeBuildInputs =
[
python3
meson
ninja
cmake
]
++ [
(lib.getBin lowdown)
mdbook
mdbook-linkcheck
]
++ [
pkg-config
# Tests
git
mercurial
jq
lsof
]
++ lib.optional hostPlatform.isLinux util-linuxMinimal
++ lib.optional (!officialRelease && buildUnreleasedNotes) build-release-notes
++ lib.optional internalApiDocs doxygen
++ lib.optionals lintInsteadOfBuild [
# required for a wrapped clang-tidy
llvmPackages.clang-tools
# required for run-clang-tidy
llvmPackages.clang-unwrapped
];
# Tests
git
mercurial
jq
lsof
]
++ lib.optional hostPlatform.isLinux util-linuxMinimal
++ lib.optional (!officialRelease && buildUnreleasedNotes) build-release-notes
++ lib.optional internalApiDocs doxygen;
buildInputs = [
curl
bzip2
xz
brotli
editline-lix
openssl
sqlite
libarchive
boost
lowdown
libsodium
toml11
lix-doc
pegtl
]
++ lib.optionals hostPlatform.isLinux [
libseccomp
busybox-sandbox-shell
passt-lix
]
++ lib.optional internalApiDocs rapidcheck
++ lib.optional hostPlatform.isx86_64 libcpuid
# There have been issues building these dependencies
++ lib.optional (hostPlatform.canExecute buildPlatform) aws-sdk-cpp-nix
++ lib.optionals (finalAttrs.dontBuild) maybePropagatedInputs
# I am so sorry. This is because checkInputs are required to pass
# configure, but we don't actually want to *run* the checks here.
++ lib.optionals lintInsteadOfBuild finalAttrs.checkInputs;
buildInputs =
[
curl
bzip2
xz
brotli
editline-lix
openssl
sqlite
libarchive
boost
lowdown
libsodium
toml11
lix-doc
pegtl
]
++ lib.optionals hostPlatform.isLinux [
libseccomp
busybox-sandbox-shell
]
++ lib.optional internalApiDocs rapidcheck
++ lib.optional hostPlatform.isx86_64 libcpuid
# There have been issues building these dependencies
++ lib.optional (hostPlatform.canExecute buildPlatform) aws-sdk-cpp-nix
++ lib.optionals (finalAttrs.dontBuild) maybePropagatedInputs;
checkInputs = [
gtest
@@ -318,6 +290,13 @@ stdenv.mkDerivation (finalAttrs: {
install_name_tool -change ${boost}/lib/libboost_system.dylib $out/lib/libboost_system.dylib $out/lib/libboost_thread.dylib
''
+ ''
# Workaround https://github.com/NixOS/nixpkgs/issues/294890.
if [[ -n "''${doCheck:-}" ]]; then
appendToVar configureFlags "--enable-tests"
else
appendToVar configureFlags "--disable-tests"
fi
# Fix up /usr/bin/env shebangs relied on by the build
patchShebangs --build tests/ doc/manual/
'';
@@ -328,7 +307,7 @@ stdenv.mkDerivation (finalAttrs: {
enableParallelBuilding = true;
doCheck = canRunInstalled && !lintInsteadOfBuild;
doCheck = canRunInstalled;
mesonCheckFlags = [
"--suite=check"
@@ -340,19 +319,8 @@ stdenv.mkDerivation (finalAttrs: {
# Make sure the internal API docs are already built, because mesonInstallPhase
# won't let us build them there. They would normally be built in buildPhase,
# but the internal API docs are conventionally built with doBuild = false.
preInstall =
(lib.optionalString internalApiDocs ''
meson ''${mesonBuildFlags:-} compile "$installTargets"
'')
# evil, but like above, we do not want to run an actual build phase
+ lib.optionalString lintInsteadOfBuild ''
ninja clang-tidy
'';
installPhase = lib.optionalString lintInsteadOfBuild ''
runHook preInstall
touch $out
runHook postInstall
preInstall = lib.optional internalApiDocs ''
meson ''${mesonBuildFlags:-} compile "$installTargets"
'';
postInstall =
@@ -413,6 +381,8 @@ stdenv.mkDerivation (finalAttrs: {
pegtl
;
inherit officialRelease;
# The collection of dependency logic for this derivation is complicated enough that
# it's easier to parameterize the devShell off an already called package.nix.
mkDevShell =
@@ -420,17 +390,19 @@ stdenv.mkDerivation (finalAttrs: {
mkShell,
bashInteractive,
clang-tools,
clangbuildanalyzer,
doxygen,
glibcLocales,
just,
nixfmt-rfc-style,
llvmPackages,
nixfmt,
skopeo,
xonsh,
# Lix specific packages
pre-commit-checks,
contribNotice,
check-syscalls,
}:
let
glibcFix = lib.optionalAttrs (buildPlatform.isLinux && glibcLocales != null) {
@@ -444,8 +416,7 @@ stdenv.mkDerivation (finalAttrs: {
p.python-frontmatter
p.requests
p.xdg-base-dirs
p.packaging
p.xonsh
(p.toPythonModule xonsh.passthru.unwrapped)
]
);
pythonEnv = python3.withPackages pythonPackages;
@@ -476,18 +447,17 @@ stdenv.mkDerivation (finalAttrs: {
++ [ (lib.mesonBool "enable-pch-std" stdenv.cc.isClang) ];
packages =
lib.optional (stdenv.cc.isClang && hostPlatform == buildPlatform) llvmPackages.clang-tools
lib.optional (stdenv.cc.isClang && hostPlatform == buildPlatform) clang-tools
++ [
# Why are we providing a bashInteractive? Well, when you run
# `bash` from inside `nix develop`, say, because you are using it
# via direnv, you will by default get bash (unusable edition).
bashInteractive
check-syscalls
pythonEnv
# docker image tool
skopeo
just
nixfmt-rfc-style
nixfmt
# Included above when internalApiDocs is true, but we set that to
# false intentionally to save dev build time.
# To build them in a dev shell, you can set -Dinternal-api-docs=enabled when configuring.
@@ -495,14 +465,6 @@ stdenv.mkDerivation (finalAttrs: {
# Load-bearing order. Must come before clang-unwrapped below, but after clang_tools above.
stdenv.cc
]
++ [
pkgs.rust-analyzer
pkgs.cargo
pkgs.rustc
pkgs.rustfmt
pkgs.rustPlatform.rustLibSrc
pkgs.rustPlatform.rustcSrc
]
++ lib.optionals stdenv.cc.isClang [
# Required for clang-tidy checks.
llvmPackages.llvm
@@ -522,9 +484,9 @@ stdenv.mkDerivation (finalAttrs: {
return
fi
PATH=$prefix/bin''${PATH:+:''${PATH}}
PATH=$prefix/bin:$PATH
unset PYTHONPATH
export MANPATH=$out/share/man:''${MANPATH:-}
export MANPATH=$out/share/man:$MANPATH
# Make bash completion work.
XDG_DATA_DIRS+=:$out/share
+84
View File
@@ -0,0 +1,84 @@
AC_INIT(nix-perl, m4_esyscmd([bash -c "echo -n $(cat ../.version)$VERSION_SUFFIX"]))
AC_CONFIG_SRCDIR(MANIFEST)
AC_CONFIG_AUX_DIR(../config)
CFLAGS=
CXXFLAGS=
AC_PROG_CC
AC_PROG_CXX
AC_CANONICAL_HOST
# Use 64-bit file system calls so that we can support files > 2 GiB.
AC_SYS_LARGEFILE
AC_DEFUN([NEED_PROG],
[
AC_PATH_PROG($1, $2)
if test -z "$$1"; then
AC_MSG_ERROR([$2 is required])
fi
])
NEED_PROG(perl, perl)
NEED_PROG(curl, curl)
NEED_PROG(bzip2, bzip2)
NEED_PROG(xz, xz)
# Test that Perl has the open/fork feature (Perl 5.8.0 and beyond).
AC_MSG_CHECKING([whether Perl is recent enough])
if ! $perl -e 'open(FOO, "-|", "true"); while (<FOO>) { print; }; close FOO or die;'; then
AC_MSG_RESULT(no)
AC_MSG_ERROR([Your Perl version is too old. Lix requires Perl 5.8.0 or newer.])
fi
AC_MSG_RESULT(yes)
# Figure out where to install Perl modules.
AC_MSG_CHECKING([for the Perl installation prefix])
perlversion=$($perl -e 'use Config; print $Config{version};')
perlarchname=$($perl -e 'use Config; print $Config{archname};')
AC_SUBST(perllibdir, [${libdir}/perl5/site_perl/$perlversion/$perlarchname])
AC_MSG_RESULT($perllibdir)
# Look for libsodium.
PKG_CHECK_MODULES([SODIUM], [libsodium], [CXXFLAGS="$SODIUM_CFLAGS $CXXFLAGS"])
# Check for the required Perl dependencies (DBI and DBD::SQLite).
perlFlags="-I$perllibdir"
AC_ARG_WITH(dbi, AC_HELP_STRING([--with-dbi=PATH],
[prefix of the Perl DBI library]),
perlFlags="$perlFlags -I$withval")
AC_ARG_WITH(dbd-sqlite, AC_HELP_STRING([--with-dbd-sqlite=PATH],
[prefix of the Perl DBD::SQLite library]),
perlFlags="$perlFlags -I$withval")
AC_MSG_CHECKING([whether DBD::SQLite works])
if ! $perl $perlFlags -e 'use DBI; use DBD::SQLite;' 2>&5; then
AC_MSG_RESULT(no)
AC_MSG_FAILURE([The Perl modules DBI and/or DBD::SQLite are missing.])
fi
AC_MSG_RESULT(yes)
AC_SUBST(perlFlags)
PKG_CHECK_MODULES([NIX], [nix-store])
NEED_PROG([NIX], [nix])
# Expand all variables in config.status.
test "$prefix" = NONE && prefix=$ac_default_prefix
test "$exec_prefix" = NONE && exec_prefix='${prefix}'
for name in $ac_subst_vars; do
declare $name="$(eval echo "${!name}")"
declare $name="$(eval echo "${!name}")"
declare $name="$(eval echo "${!name}")"
done
rm -f Makefile.config
ln -sfn ../mk mk
AC_CONFIG_FILES([])
AC_OUTPUT
+13 -12
View File
@@ -35,18 +35,19 @@ perl.pkgs.toPerlModule (
ninja
];
buildInputs = [
nix
curl
bzip2
xz
perl
boost
perlPackages.DBI
perlPackages.DBDSQLite
]
++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium
++ lib.optional stdenv.isDarwin darwin.apple_sdk.frameworks.Security;
buildInputs =
[
nix
curl
bzip2
xz
perl
boost
perlPackages.DBI
perlPackages.DBDSQLite
]
++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium
++ lib.optional stdenv.isDarwin darwin.apple_sdk.frameworks.Security;
# Nixpkgs' Meson hook likes to set this to "plain".
mesonBuildType = "debugoptimized";
+6 -6
View File
@@ -77,7 +77,7 @@ SV * queryReferences(char * path)
SV * queryPathHash(char * path)
PPCODE:
try {
auto s = store()->queryPathInfo(store()->parseStorePath(path))->narHash.to_string(Base::Base32, true);
auto s = store()->queryPathInfo(store()->parseStorePath(path))->narHash.to_string(Base32, true);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
@@ -103,7 +103,7 @@ SV * queryPathInfo(char * path, int base32)
XPUSHs(&PL_sv_undef);
else
XPUSHs(sv_2mortal(newSVpv(store()->printStorePath(*info->deriver).c_str(), 0)));
auto s = info->narHash.to_string(base32 ? Base::Base32 : Base::Base16, true);
auto s = info->narHash.to_string(base32 ? Base32 : Base16, true);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
mXPUSHi(info->registrationTime);
mXPUSHi(info->narSize);
@@ -205,7 +205,7 @@ SV * hashPath(char * algo, int base32, char * path)
PPCODE:
try {
Hash h = hashPath(parseHashType(algo), path).first;
auto s = h.to_string(base32 ? Base::Base32 : Base::Base16, false);
auto s = h.to_string(base32 ? Base32 : Base16, false);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
@@ -216,7 +216,7 @@ SV * hashFile(char * algo, int base32, char * path)
PPCODE:
try {
Hash h = hashFile(parseHashType(algo), path);
auto s = h.to_string(base32 ? Base::Base32 : Base::Base16, false);
auto s = h.to_string(base32 ? Base32 : Base16, false);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
@@ -227,7 +227,7 @@ SV * hashString(char * algo, int base32, char * s)
PPCODE:
try {
Hash h = hashString(parseHashType(algo), s);
auto s = h.to_string(base32 ? Base::Base32 : Base::Base16, false);
auto s = h.to_string(base32 ? Base32 : Base16, false);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
@@ -238,7 +238,7 @@ SV * convertHash(char * algo, char * s, int toBase32)
PPCODE:
try {
auto h = Hash::parseAny(s, parseHashType(algo));
auto s = h.to_string(toBase32 ? Base::Base32 : Base::Base16, false);
auto s = h.to_string(toBase32 ? Base32 : Base16, false);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
+4 -1
View File
@@ -19,7 +19,10 @@ endif
cxx = meson.get_compiler('cpp')
perl = find_program('perl')
# Really version 5.8.0, but Perl's version string is of the form
# "This is perl 5, version 38, subversion 2", for 5.38.2, so as far
# as Meson is concerned, the version of Perl we need is 8 or greater.
perl = find_program('perl', version : '>=8')
# "compiler to convert Perl XS code into C code"
xsubpp = find_program('xsubpp')
+14 -24
View File
@@ -30,7 +30,7 @@ First, we prepare the release. `python -m releng prepare` is used for this.
Then we tag the release with `python -m releng tag`:
* Git HEAD is detached.
* `"official_release": true` is set in `version.json`, this is committed, and a
* `officialRelease = true` is set in `flake.nix`, this is committed, and a
release is tagged.
* The tag is merged back into the last branch (either `main` for new releases
or `release-MAJOR` for maintenance releases) with `git merge -s ours VERSION`
@@ -57,10 +57,12 @@ Next, we do the publication with `python -m releng upload`:
`nix upgrade-nix`.
* s3://releases/lix/lix-VERSION/ gets the following contents
* Binary tarballs
* Docs: `manual/`, primarily as an archive of old manuals
* Docs: `manual/` (FIXME: should we actually do this? what about putting it
on docs.lix.systems? I think doing both is correct, since the Web site
should not be an archive of random old manuals)
* Docs as tarball in addition to web.
* Source tarball
* Docker image
* Docker image (FIXME: upload to forgejo registry and github registry [in the future][upload-docker])
* s3://docs/manual/lix/MAJOR
* s3://docs/manual/lix/stable
@@ -78,7 +80,6 @@ Next, we do the publication with `python -m releng upload`:
FIXME: automate branch-off to `release-*` branch.
* **Manually** (FIXME?) switch back to the release branch, which now has the
correct revision.
* Deal with the external systems (see sections below).
* Post!!
* Merge release blog post to [lix-website].
* Toot about it! https://chaos.social/@lix_project
@@ -86,33 +87,22 @@ Next, we do the publication with `python -m releng upload`:
[lix-website]: https://git.lix.systems/lix-project/lix-website
[upload-docker]: https://git.lix.systems/lix-project/lix/issues/252
### Installer
The installer is cross-built to several systems from a Mac using `build-all.xsh` and `upload-to-lix.xsh` in the installer repo (FIXME: currently at least; maybe this should be moved here?).
The installer is cross-built to several systems from a Mac using
`build-all.xsh` and `upload-to-lix.xsh` in the installer repo (FIXME: currently
at least; maybe this should be moved here?) .
It installs a binary tarball (FIXME: [it should be taught to substitute from cache instead][installer-substitute]) from some URL; this is the `hydraJobs.binaryTarball`.
The default URLs differ by architecture and are [configured here][tarball-urls].
To automatically do the file changes for a new version, run `python3 set_version.py NEW_VERSION`, and submit the result for review.
It installs a binary tarball (FIXME: [it should be taught to substitute from
cache instead][installer-substitute])
from some URL; this is the `hydraJobs.binaryTarball`. The default URLs differ
by architecture and are [configured here][tarball-urls].
[installer-substitute]: https://git.lix.systems/lix-project/lix-installer/issues/13
[tarball-urls]: https://git.lix.systems/lix-project/lix-installer/src/commit/693592ed10d421a885bec0a9dd45e87ab87eb90a/src/settings.rs#L14-L28
### Web site
The website has various release-version dependent pieces.
You can update them with `python3 update_version.py NEW_VERSION`, which will regenerate the affected page sources.
These need the release to have been done first as they need hashes for tarballs and such.
### NixOS module
The NixOS module has underdeveloped releng in it.
Currently you have to do the whole branch-off dance manually to a `release-VERSION` branch and update the tarball URLs to point to the release versions manually.
FIXME: this should be unified with the `set_version.py` work in `lix-installer` and probably all the releng kept in here, or kept elsewhere.
Related: https://git.lix.systems/lix-project/lix/issues/439
## Infrastructure summary
* releases.lix.systems (`s3://releases`):
+2 -8
View File
@@ -1,13 +1,10 @@
import logging
import argparse
import sys
from . import create_release
from . import docker
from .environment import RelengEnvironment
from . import environment
import argparse
import sys
log = logging.getLogger(__name__)
def do_build(args):
if args.target == 'all':
@@ -24,9 +21,6 @@ def do_tag(args):
create_release.do_tag_merge(force_tag=args.force_tag,
no_check_git=args.no_check_git)
log.info('Merged the release commit into your last branch, and switched to a detached HEAD of the artifact to be released.')
log.info('After you are done with releasing, switch to your previous branch and push that branch for review.')
def do_upload(env: RelengEnvironment, args):
create_release.setup_creds(env)
+9 -36
View File
@@ -2,7 +2,6 @@ import json
import subprocess
import itertools
import textwrap
import logging
from pathlib import Path
import tempfile
import hashlib
@@ -12,12 +11,10 @@ from . import environment
from .environment import RelengEnvironment
from . import keys
from . import docker
from .version import VERSION, RELEASE_NAME, MAJOR, OFFICIAL_RELEASE
from .version import VERSION, RELEASE_NAME, MAJOR
from .gitutils import verify_are_on_tag, git_preconditions
from . import release_notes
log = logging.getLogger(__name__)
$RAISE_SUBPROC_ERROR = True
$XONSH_SHOW_TRACEBACK = True
@@ -42,25 +39,16 @@ def setup_creds(env: RelengEnvironment):
def official_release_commit_tag(force_tag=False):
print('[+] Setting officialRelease in version.json and tagging')
print('[+] Setting officialRelease in flake.nix and tagging')
prev_branch = $(git symbolic-ref --short HEAD).strip()
git switch --detach
# Must be done in two parts due to buffering (opening the file immediately
# would truncate it).
new_version_json = $(jq --indent 4 '.official_release = true' version.json)
with open('version.json', 'w') as fh:
fh.write(new_version_json)
git add version.json
sed -i 's/officialRelease = false/officialRelease = true/' flake.nix
git add flake.nix
message = f'release: {VERSION} "{RELEASE_NAME}"\n\nRelease produced with releng/create_release.xsh'
git commit -m @(message)
git tag @(['-f'] if force_tag else []) -a -m @(message) @(VERSION)
with open('releng/prev-git-branch.txt', 'w') as fh:
fh.write(prev_branch)
return prev_branch
@@ -241,24 +229,8 @@ def upload_artifacts(env: RelengEnvironment, noconfirm=False, no_check_git=False
print('[+] Upload manual')
upload_manual(env)
prev_branch = None
try:
with open('releng/prev-git-branch.txt', 'r') as fh:
prev_branch = fh.read().strip()
except FileNotFoundError:
log.warn('Cannot find previous git branch file, skipping pushing git objects')
if prev_branch:
print('[+] git push to the repo')
# We have to push the ref to gerrit for review at least such that the
# commit is known, before we can push it as a tag.
if env.git_repo_is_gerrit:
git push @(env.git_repo) f'{prev_branch}:refs/for/{prev_branch}'
else:
git push @(env.git_repo) f'{prev_branch}:{prev_branch}'
print('[+] git push tag')
git push @(['-f'] if force_push_tag else []) @(env.git_repo) f'{VERSION}:refs/tags/{VERSION}'
print('[+] git push tag')
git push @(['-f'] if force_push_tag else []) @(env.git_repo) f'{VERSION}:refs/tags/{VERSION}'
def do_tag_merge(force_tag=False, no_check_git=False):
@@ -278,14 +250,15 @@ def build_manual(eval_result):
def upload_manual(env: RelengEnvironment):
if OFFICIAL_RELEASE:
stable = json.loads($(nix eval --json '.#nix.officialRelease'))
if stable:
version = MAJOR
else:
version = 'nightly'
print('[+] aws s3 sync manual')
aws s3 sync @(MANUAL)/ @(env.docs_bucket)/manual/lix/@(version)/
if OFFICIAL_RELEASE:
if stable:
aws s3 sync @(MANUAL)/ @(env.docs_bucket)/manual/lix/stable/
+17 -3
View File
@@ -44,8 +44,23 @@ def upload_docker_images(target: DockerTarget, paths: list[Path]):
for path in paths:
digest_file = tmp / (path.name + '.digest')
tmp_image = tmp / 'tmp-image.tar.gz'
inspection = json.loads($(skopeo inspect docker-archive:@(path)))
# insecure-policy: we don't have any signature policy, we are just uploading an image
#
# Absurd: we copy it into an OCI image first so we can get the hash
# we need to upload it untagged, because skopeo has no "don't tag
# this" option.
# The reason for this is that forgejo's container registry throws
# away old versions of tags immediately, so we cannot use a temp
# tag, and it *does* reduce confusion to not upload tags that
# should not be used.
#
# Workaround for: https://github.com/containers/skopeo/issues/2354
log.info('skopeo copy to temp oci-archive %s', tmp_image)
skopeo --insecure-policy copy --format oci --all --digestfile @(digest_file) docker-archive:@(path) oci-archive:@(tmp_image)
inspection = json.loads($(skopeo inspect oci-archive:@(tmp_image)))
docker_arch = inspection['Architecture']
docker_os = inspection['Os']
@@ -53,9 +68,8 @@ def upload_docker_images(target: DockerTarget, paths: list[Path]):
log.info('Pushing image %s for %s to %s', path, docker_arch, target.registry_path)
# insecure-policy: we don't have any signature policy, we are just uploading an image
skopeo --insecure-policy copy --digestfile @(digest_file) --all docker-archive:@(path) f'docker://{target.registry_path}@@unknown-digest@@'
digest = digest_file.read_text().strip()
skopeo --insecure-policy copy --preserve-digests --all oci-archive:@(tmp_image) f'docker://{target.registry_path}@{digest}'
# skopeo doesn't give us the manifest size directly, so we just ask the registry
metadata = reg.image_info(target.registry_path, digest)
-3
View File
@@ -52,7 +52,6 @@ class RelengEnvironment:
releases_bucket: str
docs_bucket: str
git_repo: str
git_repo_is_gerrit: bool
docker_targets: list[DockerTarget]
@@ -80,7 +79,6 @@ STAGING = RelengEnvironment(
cache_store_overlay={'secret-key': 'staging.key'},
releases_bucket='s3://staging-releases',
git_repo='ssh://git@git.lix.systems/lix-project/lix-releng-staging',
git_repo_is_gerrit=False,
docker_targets=[
# latest will be auto tagged if appropriate
DockerTarget('git.lix.systems/lix-project/lix-releng-staging',
@@ -115,7 +113,6 @@ PROD = RelengEnvironment(
cache_store_overlay={'secret-key': 'prod.key'},
releases_bucket='s3://releases',
git_repo=guess_gerrit_remote(),
git_repo_is_gerrit=True,
docker_targets=[
# latest will be auto tagged if appropriate
DockerTarget('git.lix.systems/lix-project/lix',
+11 -26
View File
@@ -1,24 +1,11 @@
import subprocess
from packaging.version import Version
import json
from .version import VERSION
def remote_is_plausible(url: str) -> bool:
return ('git.lix.systems' in url and 'lix-project/lix' in url) or ('gerrit.lix.systems' in url and url.endswith('lix'))
def version_compare(v1: str, v2: str):
v1 = Version(v1)
v2 = Version(v2)
if v1 < v2:
return -1
elif v1 > v2:
return 1
elif v1 == v2:
return 0
else:
raise ValueError('these versions are beyond each others celestial plane')
return json.loads($(nix-instantiate --eval --json --argstr v1 @(v1) --argstr v2 @(v2) --expr '{v1, v2}: builtins.compareVersions v1 v2'))
def latest_tag_on_branch(branch: str) -> str:
@@ -26,18 +13,16 @@ def latest_tag_on_branch(branch: str) -> str:
def is_maintenance_branch(branch: str) -> bool:
"""
Returns whether the given branch is probably a maintenance branch.
try:
main_tag = latest_tag_on_branch('main')
current_tag = latest_tag_on_branch(branch)
This uses a heuristic: `main` should have a newer tag than a given
maintenance branch if there has been a major release since that maintenance
branch.
"""
assert remote_is_plausible($(git remote get-url origin).strip())
main_tag = latest_tag_on_branch('origin/main')
current_tag = latest_tag_on_branch(branch)
return version_compare(current_tag, main_tag) < 0
return version_compare(current_tag, main_tag) < 0
except subprocess.CalledProcessError:
# This is the case before Lix releases 2.90, since main *has* no
# release tag on it.
# FIXME: delete this case after 2.91
return False
def verify_are_on_tag():
-36
View File
@@ -1,36 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
shopt -s inherit_errexit failglob
nixpkgss=(
"$(nix eval --impure --raw --expr '(import ./flake.nix).inputs.nixpkgs.url')"
"github:NixOS/nixpkgs/nixos-unstable-small"
)
jobs=(
$(nix eval \
--json --apply '
let f = n: t:
if builtins.isAttrs t
then (if t.type or "" == "derivation"
then [ n ]
else builtins.concatMap (m: f "${n}.${m}" t.${m}) (builtins.attrNames t))
else [];
in f ".#.releaseTests"
' \
'.#.releaseTests' \
| jq -r '.[]'
)
)
for override in "${nixpkgss}"
do
(
set -x
nix build \
--log-format multiline \
--no-link \
--override-input nixpkgs "$override" \
"${jobs[@]}"
)
done
+1 -1
View File
@@ -16,7 +16,7 @@ def add_to_summary(date: str):
if VERSION_RL.exists():
return
MARKER = ' <!-- RELENG-AUTO-INSERTION-MARKER'
MARKER = '<!-- RELENG-AUTO-INSERTION-MARKER'
new_lines = []
for line in SUMMARY.read_text().splitlines():
-1
View File
@@ -4,4 +4,3 @@ version_json = json.load(open('version.json'))
VERSION = version_json['version']
MAJOR = '.'.join(VERSION.split('.')[:2])
RELEASE_NAME = version_json['release_name']
OFFICIAL_RELEASE = version_json['official_release']
-17
View File
@@ -1,17 +0,0 @@
/// @file This is very bothersome code that has to be included in every
/// executable to get the correct default ASan options. I am so sorry.
extern "C" [[gnu::retain]] const char *__asan_default_options()
{
// We leak a bunch of memory knowingly on purpose. It's not worthwhile to
// diagnose that memory being leaked for now.
//
// Instruction bytes are useful for finding the actual code that
// corresponds to an ASan report.
//
// TODO: setting log_path=asan.log or not: neither works, since you can't
// write to the fs in certain places in the testsuite, but you also cannot
// write arbitrarily to stderr in other places so the reports get eaten.
// pain 🥖
return "halt_on_error=1:abort_on_error=1:detect_leaks=0:print_summary=1:dump_instruction_bytes=1";
}
+3 -3
View File
@@ -42,7 +42,7 @@ static std::string makeLockFilename(const std::string & storeUri) {
// This avoids issues with the escaped URI being very long and causing
// path too long errors, while also avoiding any possibility of collision
// caused by simple truncation.
auto hash = hashString(HashType::SHA256, storeUri).to_string(Base::Base32, false);
auto hash = hashString(HashType::htSHA256, storeUri).to_string(Base::Base32, false);
return escapeUri(storeUri).substr(0, 48) + "-" + hash.substr(0, 16);
}
@@ -236,9 +236,9 @@ static int main_build_remote(int argc, char * * argv)
}
#if __APPLE__
futimes(bestSlotLock.get(), nullptr);
futimes(bestSlotLock.get(), NULL);
#else
futimens(bestSlotLock.get(), nullptr);
futimens(bestSlotLock.get(), NULL);
#endif
lock.reset();
+1 -1
View File
@@ -246,7 +246,7 @@ StorePath ProfileManifest::build(ref<Store> store)
StringSink sink;
sink << dumpPath(tempDir);
auto narHash = hashString(HashType::SHA256, sink.s);
auto narHash = hashString(htSHA256, sink.s);
ValidPathInfo info{
*store,
+1 -1
View File
@@ -37,7 +37,7 @@ struct ProfileElement
StorePathSet storePaths;
std::optional<ProfileElementSource> source;
bool active = true;
NixInt::Inner priority = DEFAULT_PRIORITY;
int priority = DEFAULT_PRIORITY;
std::string identifier() const;
+12
View File
@@ -1,11 +1,23 @@
#include "globals.hh"
#include "installable-attr-path.hh"
#include "outputs-spec.hh"
#include "command.hh"
#include "attr-path.hh"
#include "common-eval-args.hh"
#include "derivations.hh"
#include "eval-inline.hh"
#include "eval.hh"
#include "get-drvs.hh"
#include "store-api.hh"
#include "shared.hh"
#include "flake/flake.hh"
#include "eval-cache.hh"
#include "url.hh"
#include "registry.hh"
#include "build-result.hh"
#include <regex>
#include <queue>
#include <nlohmann/json.hpp>
+14
View File
@@ -1,11 +1,25 @@
#pragma once
///@file
#include "globals.hh"
#include "installable-value.hh"
#include "outputs-spec.hh"
#include "command.hh"
#include "attr-path.hh"
#include "common-eval-args.hh"
#include "derivations.hh"
#include "eval-inline.hh"
#include "eval.hh"
#include "get-drvs.hh"
#include "store-api.hh"
#include "shared.hh"
#include "eval-cache.hh"
#include "url.hh"
#include "registry.hh"
#include "build-result.hh"
#include <regex>
#include <queue>
#include <nlohmann/json.hpp>
+2 -2
View File
@@ -116,12 +116,12 @@ DerivedPathsWithInfo InstallableFlake::toDerivedPaths()
auto drvPath = attr->forceDerivation();
std::optional<NixInt::Inner> priority;
std::optional<NixInt> priority;
if (attr->maybeGetAttr(state->sOutputSpecified)) {
} else if (auto aMeta = attr->maybeGetAttr(state->sMeta)) {
if (auto aPriority = aMeta->maybeGetAttr("priority"))
priority = aPriority->getInt().value;
priority = aPriority->getInt();
}
return {{
+1 -1
View File
@@ -40,7 +40,7 @@ struct ExtraPathInfoValue : ExtraPathInfo
/**
* An optional priority for use with "build envs". See Package
*/
std::optional<NixInt::Inner> priority;
std::optional<NixInt> priority;
/**
* The attribute path associated with this value. The idea is
+2 -17
View File
@@ -11,28 +11,13 @@ namespace nix {
std::string renderMarkdownToTerminal(std::string_view markdown)
{
int windowWidth = getWindowSize().second;
size_t lowdown_cols = std::max(windowWidth - 5, 60);
struct lowdown_opts opts{
struct lowdown_opts opts {
.type = LOWDOWN_TERM,
#ifdef LOWDOWN_SEPARATE_TERM_OPTS
.term =
{
.cols = lowdown_cols,
.width = 0,
.hmargin = 0,
.hpadding = 4,
.vmargin = 0,
.centre = 0,
},
// maxdepth needs to be part of the ifdefs to match declaration order
.maxdepth = 20,
#else
.maxdepth = 20,
.cols = lowdown_cols,
.cols = (size_t) std::max(windowWidth - 5, 60),
.hmargin = 0,
.vmargin = 0,
#endif /* LOWDOWN_SEPARATE_TERM_OPTS */
.feat = LOWDOWN_COMMONMARK | LOWDOWN_FENCED | LOWDOWN_DEFLIST | LOWDOWN_TABLES,
.oflags = LOWDOWN_TERM_NOLINK,
};
+1 -1
View File
@@ -96,7 +96,7 @@ static int listPossibleCallback(char * s, char *** avp)
return p;
};
vp = check(static_cast<char **>(malloc(possible.size() * sizeof(char *))));
vp = check((char **) malloc(possible.size() * sizeof(char *)));
for (auto & p : possible)
vp[ac++] = check(strdup(p.c_str()));
+44 -61
View File
@@ -32,6 +32,7 @@
#include "local-fs-store.hh"
#include "signals.hh"
#include "print.hh"
#include "progress-bar.hh"
#include "gc-small-vector.hh"
#include "users.hh"
@@ -40,24 +41,24 @@
#include <gc/gc_cpp.h>
#endif
// XXX: These are for lix-doc features and will be removed in a future rewrite where this functionality is integrated more natively.
// XXX: These are for nix-doc features and will be removed in a future rewrite where this functionality is integrated more natively.
extern "C" {
char const *lixdoc_get_function_docs(char const *filename, size_t line, size_t col);
void lixdoc_free_string(char const *str);
char const *nd_get_function_docs(char const *filename, size_t line, size_t col);
void nd_free_string(char const *str);
}
namespace nix {
/** Wrapper around std::unique_ptr with a custom deleter for strings from nix-doc **/
using NdString = std::unique_ptr<const char, decltype(&lixdoc_free_string)>;
using NdString = std::unique_ptr<const char, decltype(&nd_free_string)>;
/**
* Fetch a string representing the doc comment using nix-doc and wrap it in an RAII wrapper.
*/
NdString lambdaDocsForPos(SourcePath const path, nix::Pos const &pos) {
std::string const file = path.to_string();
return NdString{lixdoc_get_function_docs(file.c_str(), pos.line, pos.column), &lixdoc_free_string};
return NdString{nd_get_function_docs(file.c_str(), pos.line, pos.column), &nd_free_string};
}
/**
@@ -251,7 +252,7 @@ void runNix(Path program, const Strings & args)
.program = settings.nixBinDir+ "/" + program,
.args = args,
.environment = subprocessEnv,
}).waitAndCheck();
}).wait();
return;
}
@@ -299,7 +300,7 @@ ReplExitStatus NixRepl::mainLoop()
/* Stop the progress bar because it interferes with the display of
the repl. */
logger->pause();
stopProgressBar();
std::string input;
@@ -536,7 +537,7 @@ ProcessLineResult NixRepl::processLine(std::string line)
<< " :t <expr> Describe result of evaluation\n"
<< " :u <expr> Build derivation, then start nix-shell\n"
<< " :doc <expr> Show documentation for the provided function (experimental lambda support)\n"
<< " :log <expr | .drv path> Show logs for a derivation\n"
<< " :log <expr> Show logs for a derivation\n"
<< " :te, :trace-enable [bool] Enable, disable or toggle showing traces for\n"
<< " errors\n"
<< " :?, :help Brings up this help menu\n"
@@ -650,14 +651,11 @@ ProcessLineResult NixRepl::processLine(std::string line)
// runProgram redirects stdout to a StringSink,
// using runProgram2 to allow editors to display their UI
runProgram2(RunOptions { .program = editor, .searchPath = true, .args = args }).waitAndCheck();
runProgram2(RunOptions { .program = editor, .searchPath = true, .args = args }).wait();
// Reload right after exiting the editor if path is not in store
// Store is immutable, so there could be no changes, so there's no need to reload
if (!state->store->isInStore(path.resolveSymlinks().path.abs())) {
state->resetFileCache();
reloadFiles();
}
// Reload right after exiting the editor
state->resetFileCache();
reloadFiles();
}
else if (command == ":t") {
@@ -676,49 +674,7 @@ ProcessLineResult NixRepl::processLine(std::string line)
runNix("nix-shell", {state->store->printStorePath(drvPath)});
}
else if (command == ":log") {
StorePath drvPath = ([&] {
auto maybeDrvPath = state->store->maybeParseStorePath(arg);
if (maybeDrvPath && maybeDrvPath->isDerivation()) {
return std::move(*maybeDrvPath);
} else {
Value v;
evalString(arg, v);
return getDerivationPath(v);
}
})();
Path drvPathRaw = state->store->printStorePath(drvPath);
settings.readOnlyMode = true;
Finally roModeReset([&]() {
settings.readOnlyMode = false;
});
auto subs = getDefaultSubstituters();
subs.push_front(state->store);
bool foundLog = false;
RunPager pager;
for (auto & sub : subs) {
auto * logSubP = dynamic_cast<LogStore *>(&*sub);
if (!logSubP) {
printInfo("Skipped '%s' which does not support retrieving build logs", sub->getUri());
continue;
}
auto & logSub = *logSubP;
auto log = logSub.getBuildLog(drvPath);
if (log) {
printInfo("got build log for '%s' from '%s'", drvPathRaw, logSub.getUri());
logger->writeToStdout(*log);
foundLog = true;
break;
}
}
if (!foundLog) throw Error("build log of '%s' is not available", drvPathRaw);
}
else if (command == ":b" || command == ":bl" || command == ":i" || command == ":sh") {
else if (command == ":b" || command == ":bl" || command == ":i" || command == ":sh" || command == ":log") {
Value v;
evalString(arg, v);
StorePath drvPath = getDerivationPath(v);
@@ -728,10 +684,9 @@ ProcessLineResult NixRepl::processLine(std::string line)
// TODO: this only shows a progress bar for explicitly initiated builds,
// not eval-time fetching or builds performed for IFD.
// But we can't just show it everywhere, since that would erase partial output from evaluation.
logger->resetProgress();
logger->resume();
startProgressBar();
Finally stopLogger([&]() {
logger->pause();
stopProgressBar();
});
state->store->buildPaths({
@@ -754,6 +709,34 @@ ProcessLineResult NixRepl::processLine(std::string line)
}
} else if (command == ":i") {
runNix("nix-env", {"-i", drvPathRaw});
} else if (command == ":log") {
settings.readOnlyMode = true;
Finally roModeReset([&]() {
settings.readOnlyMode = false;
});
auto subs = getDefaultSubstituters();
subs.push_front(state->store);
bool foundLog = false;
RunPager pager;
for (auto & sub : subs) {
auto * logSubP = dynamic_cast<LogStore *>(&*sub);
if (!logSubP) {
printInfo("Skipped '%s' which does not support retrieving build logs", sub->getUri());
continue;
}
auto & logSub = *logSubP;
auto log = logSub.getBuildLog(drvPath);
if (log) {
printInfo("got build log for '%s' from '%s'", drvPathRaw, logSub.getUri());
logger->writeToStdout(*log);
foundLog = true;
break;
}
}
if (!foundLog) throw Error("build log of '%s' is not available", drvPathRaw);
} else {
runNix("nix-shell", {drvPathRaw});
}
+1
View File
@@ -129,6 +129,7 @@ std::pair<SourcePath, uint32_t> findPackageFilename(EvalState & state, Value & v
try {
auto colon = fn.rfind(':');
if (colon == std::string::npos) fail();
std::string filename(fn, 0, colon);
auto lineno = std::stoi(std::string(fn, colon + 1, std::string::npos));
return {CanonPath(fn.substr(0, colon)), lineno};
} catch (std::invalid_argument & e) {
+2 -3
View File
@@ -1,6 +1,5 @@
#include "attr-set.hh"
#include "eval.hh"
#include "gc-alloc.hh"
#include "eval-inline.hh"
#include <algorithm>
@@ -20,7 +19,7 @@ Bindings * EvalState::allocBindings(size_t capacity)
throw Error("attribute set of size %d is too big", capacity);
nrAttrsets++;
nrAttrsInAttrsets += capacity;
return new (gcAllocBytes(sizeof(Bindings) + sizeof(Attr) * capacity)) Bindings((Bindings::size_t) capacity);
return new (allocBytes(sizeof(Bindings) + sizeof(Attr) * capacity)) Bindings((Bindings::size_t) capacity);
}
+3 -3
View File
@@ -50,7 +50,7 @@ struct AttrDb
Path cacheDir = getCacheDir() + "/nix/eval-cache-v5";
createDirs(cacheDir);
Path dbPath = cacheDir + "/" + fingerprint.to_string(Base::Base16, false) + ".sqlite";
Path dbPath = cacheDir + "/" + fingerprint.to_string(Base16, false) + ".sqlite";
state->db = SQLite(dbPath);
state->db.isCache();
@@ -306,7 +306,7 @@ struct AttrDb
case AttrType::Bool:
return {{rowId, queryAttribute.getInt(2) != 0}};
case AttrType::Int:
return {{rowId, int_t{NixInt{queryAttribute.getInt(2)}}}};
return {{rowId, int_t{queryAttribute.getInt(2)}}};
case AttrType::ListOfStrings:
return {{rowId, tokenizeString<std::vector<std::string>>(queryAttribute.getStr(2), "\t")}};
case AttrType::Missing:
@@ -449,7 +449,7 @@ Value & AttrCursor::forceValue()
else if (v.type() == nBool)
cachedValue = {root->db->setBool(getKey(), v.boolean), v.boolean};
else if (v.type() == nInt)
cachedValue = {root->db->setInt(getKey(), v.integer.value), int_t{v.integer}};
cachedValue = {root->db->setInt(getKey(), v.integer), int_t{v.integer}};
else if (v.type() == nAttrs)
; // FIXME: do something?
else
+19 -3
View File
@@ -4,10 +4,26 @@
#include "print.hh"
#include "eval.hh"
#include "eval-error.hh"
#include "gc-alloc.hh"
namespace nix {
/**
* Note: Various places expect the allocated memory to be zeroed.
*/
[[gnu::always_inline]]
inline void * allocBytes(size_t n)
{
void * p;
#if HAVE_BOEHMGC
p = GC_MALLOC(n);
#else
p = calloc(n, 1);
#endif
if (!p) throw std::bad_alloc();
return p;
}
[[gnu::always_inline]]
Value * EvalState::allocValue()
{
@@ -27,7 +43,7 @@ Value * EvalState::allocValue()
*valueAllocCache = GC_NEXT(p);
GC_NEXT(p) = nullptr;
#else
void * p = gcAllocBytes(sizeof(Value));
void * p = allocBytes(sizeof(Value));
#endif
nrValues++;
@@ -57,7 +73,7 @@ Env & EvalState::allocEnv(size_t size)
env = (Env *) p;
} else
#endif
env = (Env *) gcAllocBytes(sizeof(Env) + size * sizeof(Value *));
env = (Env *) allocBytes(sizeof(Env) + size * sizeof(Value *));
/* We assume that env->values has been cleared by the allocator; maybeThunk() and lookupVar fromWith expect this. */
+153 -90
View File
@@ -9,7 +9,6 @@
#include "store-api.hh"
#include "derivations.hh"
#include "downstream-placeholder.hh"
#include "gc-alloc.hh"
#include "globals.hh"
#include "eval-inline.hh"
#include "filetransfer.hh"
@@ -21,6 +20,7 @@
#include "flake/flakeref.hh"
#include <algorithm>
#include <chrono>
#include <iostream>
#include <sstream>
#include <cstring>
@@ -48,6 +48,48 @@ using json = nlohmann::json;
namespace nix {
static char * allocString(size_t size)
{
char * t;
#if HAVE_BOEHMGC
t = (char *) GC_MALLOC_ATOMIC(size);
#else
t = (char *) malloc(size);
#endif
if (!t) throw std::bad_alloc();
return t;
}
static char * dupString(const char * s)
{
char * t;
#if HAVE_BOEHMGC
t = GC_STRDUP(s);
#else
t = strdup(s);
#endif
if (!t) throw std::bad_alloc();
return t;
}
// When there's no need to write to the string, we can optimize away empty
// string allocations.
// This function handles makeImmutableString(std::string_view()) by returning
// the empty string.
static const char * makeImmutableString(std::string_view s)
{
const size_t size = s.size();
if (size == 0)
return "";
auto t = allocString(size + 1);
memcpy(t, s.data(), size);
t[size] = '\0';
return t;
}
RootValue allocRootValue(Value * v)
{
#if HAVE_BOEHMGC
@@ -70,6 +112,11 @@ std::string printValue(EvalState & state, Value & v)
return out.str();
}
void Value::print(EvalState & state, std::ostream & str, PrintOptions options)
{
printValue(state, str, *this, options);
}
const Value * getPrimOp(const Value &v) {
const Value * primOp = &v;
while (primOp->isPrimOpApp()) {
@@ -119,6 +166,32 @@ std::string showType(const Value & v)
#pragma GCC diagnostic pop
}
PosIdx Value::determinePos(const PosIdx pos) const
{
// Allow selecting a subset of enum values
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wswitch-enum"
switch (internalType) {
case tAttrs: return attrs->pos;
case tLambda: return lambda.fun->pos;
case tApp: return app.left->determinePos(pos);
default: return pos;
}
#pragma GCC diagnostic pop
}
bool Value::isTrivial() const
{
return
internalType != tApp
&& internalType != tPrimOpApp
&& (internalType != tThunk
|| (dynamic_cast<ExprAttrs *>(thunk.expr)
&& ((ExprAttrs *) thunk.expr)->dynamicAttrs.empty())
|| dynamic_cast<ExprLambda *>(thunk.expr)
|| dynamic_cast<ExprList *>(thunk.expr));
}
#if HAVE_BOEHMGC
/* Called when the Boehm GC runs out of memory. */
@@ -217,12 +290,6 @@ EvalState::EvalState(
, sRight(symbols.create("right"))
, sWrong(symbols.create("wrong"))
, sStructuredAttrs(symbols.create("__structuredAttrs"))
, sAllowedReferences(symbols.create("allowedReferences"))
, sAllowedRequisites(symbols.create("allowedRequisites"))
, sDisallowedReferences(symbols.create("disallowedReferences"))
, sDisallowedRequisites(symbols.create("disallowedRequisites"))
, sMaxSize(symbols.create("maxSize"))
, sMaxClosureSize(symbols.create("maxClosureSize"))
, sBuilder(symbols.create("builder"))
, sArgs(symbols.create("args"))
, sContentAddressed(symbols.create("__contentAddressed"))
@@ -467,6 +534,29 @@ std::ostream & operator<<(std::ostream & output, PrimOp & primOp)
return output;
}
PrimOp * Value::primOpAppPrimOp() const
{
Value * left = primOpApp.left;
while (left && !left->isPrimOp()) {
left = left->primOpApp.left;
}
if (!left)
return nullptr;
return left->primOp;
}
void Value::mkPrimOp(PrimOp * p)
{
p->check();
clearValue();
internalType = tPrimOp;
primOp = p;
}
Value * EvalState::addPrimOp(PrimOp && primOp)
{
/* Hack to make constants lazy: turn them into a application of
@@ -718,6 +808,43 @@ DebugTraceStacker::DebugTraceStacker(EvalState & evalState, DebugTrace t)
evalState.runDebugRepl(nullptr, trace.env, trace.expr);
}
void Value::mkString(std::string_view s)
{
mkString(makeImmutableString(s));
}
static void copyContextToValue(Value & v, const NixStringContext & context)
{
if (!context.empty()) {
size_t n = 0;
v.string.context = (const char * *)
allocBytes((context.size() + 1) * sizeof(char *));
for (auto & i : context)
v.string.context[n++] = dupString(i.to_string().c_str());
v.string.context[n] = 0;
}
}
void Value::mkString(std::string_view s, const NixStringContext & context)
{
mkString(s);
copyContextToValue(*this, context);
}
void Value::mkStringMove(const char * s, const NixStringContext & context)
{
mkString(s);
copyContextToValue(*this, context);
}
void Value::mkPath(const SourcePath & path)
{
mkPath(makeImmutableString(path.path.abs()));
}
inline Value * EvalState::lookupVar(Env * env, const ExprVar & var, bool noEval)
{
for (auto l = var.level; l; --l, env = env->up) ;
@@ -748,7 +875,7 @@ void EvalState::mkList(Value & v, size_t size)
{
v.mkList(size);
if (size > 2)
v.bigList.elems = gcAllocType<Value *>(size);
v.bigList.elems = (Value * *) allocBytes(size * sizeof(Value *));
nrListElems += size;
}
@@ -1237,20 +1364,7 @@ void ExprSelect::eval(EvalState & state, Env & env, Value & v)
Value * vCurrent = &vFirst;
// Position for the current attrset Value in this select chain.
PosIdx posCurrent;
try {
e->eval(state, env, vFirst);
} catch (Error & e) {
assert(this->e != nullptr);
state.addErrorTrace(
e,
getPos(),
"while evaluating '%s' to select '%s' on it",
ExprPrinter(state, *this->e),
showAttrPath(state.symbols, this->attrPath)
);
throw;
}
e->eval(state, env, vFirst);
try {
auto dts = state.debugRepl
@@ -1263,46 +1377,12 @@ void ExprSelect::eval(EvalState & state, Env & env, Value & v)
showAttrPath(state, env, attrPath))
: nullptr;
for (auto const & [partIdx, currentAttrName] : enumerate(attrPath)) {
for (auto const & currentAttrName : attrPath) {
state.nrLookups++;
Symbol const name = getName(currentAttrName, state, env);
// For formatting errors, which should be done only when needed.
auto partsSoFar = [&]() -> std::string {
std::stringstream ss;
// We start with the base thing this ExprSelect is selecting on.
assert(this->e != nullptr);
this->e->show(state.symbols, ss);
// Then grab each part of the attr path up to this one.
assert(partIdx < attrPath.size());
std::span<AttrName> const parts(
attrPath.begin(),
attrPath.begin() + partIdx
);
// And convert them to strings and join them.
for (auto const & part : parts) {
auto const partName = getName(part, state, env);
ss << "." << state.symbols[partName];
}
return ss.str();
};
try {
state.forceValue(*vCurrent, pos);
} catch (Error & e) {
state.addErrorTrace(
e,
getPos(),
"while evaluating '%s' to select '%s' on it",
partsSoFar(),
state.symbols[name]
);
throw;
}
state.forceValue(*vCurrent, pos);
if (vCurrent->type() != nAttrs) {
@@ -1318,10 +1398,7 @@ void ExprSelect::eval(EvalState & state, Env & env, Value & v)
"expected a set but found %s: %s",
showType(*vCurrent),
ValuePrinter(state, *vCurrent, errorPrintOptions)
).addTrace(
pos,
HintFmt("while selecting '%s' on '%s'", state.symbols[name], partsSoFar())
).debugThrow();
).withTrace(pos, "while selecting an attribute").debugThrow();
}
// Now that we know this is actually an attrset, try to find an attr
@@ -1612,15 +1689,6 @@ void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value &
try {
fn->fun(*this, vCur.determinePos(noPos), args, vCur);
} catch (ThrownError & e) {
// Distinguish between an error that simply happened while "throw"
// was being evaluated and an explicit thrown error.
if (fn->name == "throw") {
addErrorTrace(e, pos, "caused by explicit %s", "throw");
} else {
addErrorTrace(e, pos, "while calling the '%s' builtin", fn->name);
}
throw;
} catch (Error & e) {
addErrorTrace(e, pos, "while calling the '%1%' builtin", fn->name);
throw;
@@ -1787,7 +1855,7 @@ void EvalState::autoCallFunction(Bindings & args, Value & fun, Value & res)
Nix attempted to evaluate a function as a top level expression; in
this case it must have its arguments supplied either by default
values, or passed explicitly with '--arg' or '--argstr'. See
https://docs.lix.systems/manual/lix/stable/language/constructs.html#functions)", symbols[i.name])
https://nixos.org/manual/nix/stable/language/constructs.html#functions.)", symbols[i.name])
.atPos(i.pos).withFrame(*fun.lambda.env, *fun.lambda.fun).debugThrow();
}
}
@@ -1946,7 +2014,7 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
NixStringContext context;
std::vector<BackedStringView> s;
size_t sSize = 0;
NixInt n{0};
NixInt n = 0;
NixFloat nf = 0;
bool first = !forceString;
@@ -1962,7 +2030,7 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
Value. allocating a GC'd string directly and moving it into a
Value lets us avoid an allocation and copy. */
const auto c_str = [&] {
char * result = gcAllocString(sSize + 1);
char * result = allocString(sSize + 1);
char * tmp = result;
for (const auto & part : s) {
memcpy(tmp, part->data(), part->size());
@@ -1990,22 +2058,17 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
if (firstType == nInt) {
if (vTmp.type() == nInt) {
auto newN = n + vTmp.integer;
if (auto checked = newN.valueChecked(); checked.has_value()) {
n = NixInt(*checked);
} else {
state.error<EvalError>("integer overflow in adding %1% + %2%", n, vTmp.integer).atPos(i_pos).debugThrow();
}
n += vTmp.integer;
} else if (vTmp.type() == nFloat) {
// Upgrade the type from int to float;
firstType = nFloat;
nf = n.value;
nf = n;
nf += vTmp.fpoint;
} else
state.error<EvalError>("cannot add %1% to an integer", showType(vTmp)).atPos(i_pos).withFrame(env, *this).debugThrow();
} else if (firstType == nFloat) {
if (vTmp.type() == nInt) {
nf += vTmp.integer.value;
nf += vTmp.integer;
} else if (vTmp.type() == nFloat) {
nf += vTmp.fpoint;
} else
@@ -2130,7 +2193,7 @@ NixFloat EvalState::forceFloat(Value & v, const PosIdx pos, std::string_view err
try {
forceValue(v, pos);
if (v.type() == nInt)
return v.integer.value;
return v.integer;
else if (v.type() != nFloat)
error<TypeError>(
"expected a float but found %1%: %2%",
@@ -2317,7 +2380,7 @@ BackedStringView EvalState::coerceToString(
shell scripting convenience, just like `null'. */
if (v.type() == nBool && v.boolean) return "1";
if (v.type() == nBool && !v.boolean) return "";
if (v.type() == nInt) return std::to_string(v.integer.value);
if (v.type() == nInt) return std::to_string(v.integer);
if (v.type() == nFloat) return std::to_string(v.fpoint);
if (v.type() == nNull) return "";
@@ -2461,9 +2524,9 @@ bool EvalState::eqValues(Value & v1, Value & v2, const PosIdx pos, std::string_v
// Special case type-compatibility between float and int
if (v1.type() == nInt && v2.type() == nFloat)
return v1.integer.value == v2.fpoint;
return v1.integer == v2.fpoint;
if (v1.type() == nFloat && v2.type() == nInt)
return v1.fpoint == v2.integer.value;
return v1.fpoint == v2.integer;
// All other types are not compatible with each other.
if (v1.type() != v2.type()) return false;
@@ -2710,20 +2773,20 @@ Expr & EvalState::parseExprFromFile(const SourcePath & path, std::shared_ptr<Sta
}
Expr & EvalState::parseExprFromString(std::string s_, const SourcePath & basePath, std::shared_ptr<StaticEnv> & staticEnv, const ExperimentalFeatureSettings & xpSettings)
Expr & EvalState::parseExprFromString(std::string s_, const SourcePath & basePath, std::shared_ptr<StaticEnv> & staticEnv)
{
// NOTE this method (and parseStdin) must take care to *fully copy* their input
// into their respective Pos::Origin until the parser stops overwriting its input
// data.
auto s = make_ref<std::string>(s_);
s_.append("\0\0", 2);
return *parse(s_.data(), s_.size(), Pos::String{.source = s}, basePath, staticEnv, xpSettings);
return *parse(s_.data(), s_.size(), Pos::String{.source = s}, basePath, staticEnv);
}
Expr & EvalState::parseExprFromString(std::string s, const SourcePath & basePath, const ExperimentalFeatureSettings & xpSettings)
Expr & EvalState::parseExprFromString(std::string s, const SourcePath & basePath)
{
return parseExprFromString(std::move(s), basePath, staticBaseEnv, xpSettings);
return parseExprFromString(std::move(s), basePath, staticBaseEnv);
}
+29 -19
View File
@@ -3,19 +3,20 @@
#include "attr-set.hh"
#include "eval-error.hh"
#include "gc-alloc.hh"
#include "types.hh"
#include "value.hh"
#include "nixexpr.hh"
#include "symbol-table.hh"
#include "config.hh"
#include "experimental-features.hh"
#include "input-accessor.hh"
#include "search-path.hh"
#include "repl-exit-status.hh"
#include <map>
#include <optional>
#include <unordered_map>
#include <mutex>
#include <functional>
namespace nix {
@@ -37,10 +38,11 @@ namespace eval_cache {
class EvalCache;
}
/**
* Function that implements a primop.
*/
using PrimOpImpl = void(EvalState & state, PosIdx pos, Value ** args, Value & v);
typedef void (* PrimOpFun) (EvalState & state, const PosIdx pos, Value * * args, Value & v);
/**
* Info about a primitive operation, and its implementation
@@ -74,7 +76,7 @@ struct PrimOp
/**
* Implementation of the primop.
*/
std::function<PrimOpImpl> fun;
std::function<std::remove_pointer<PrimOpFun>::type> fun;
/**
* Optional experimental for this to be gated on.
@@ -113,7 +115,11 @@ struct Constant
bool impureOnly = false;
};
using ValMap = GcMap<std::string, Value *>;
#if HAVE_BOEHMGC
typedef std::map<std::string, Value *, std::less<std::string>, traceable_allocator<std::pair<const std::string, Value *> > > ValMap;
#else
typedef std::map<std::string, Value *> ValMap;
#endif
struct Env
{
@@ -161,10 +167,7 @@ public:
const Symbol sWith, sOutPath, sDrvPath, sType, sMeta, sName, sValue,
sSystem, sOverrides, sOutputs, sOutputName, sIgnoreNulls,
sFile, sLine, sColumn, sFunctor, sToString,
sRight, sWrong, sStructuredAttrs,
sAllowedReferences, sAllowedRequisites, sDisallowedReferences, sDisallowedRequisites,
sMaxSize, sMaxClosureSize,
sBuilder, sArgs,
sRight, sWrong, sStructuredAttrs, sBuilder, sArgs,
sContentAddressed, sImpure,
sOutputHash, sOutputHashAlgo, sOutputHashMode,
sRecurseForDerivations,
@@ -211,7 +214,7 @@ public:
/**
* Debugger
*/
std::function<ReplExitStatus(ref<EvalState> es, ValMap const & extraEnv)> debugRepl;
ReplExitStatus (* debugRepl)(ref<EvalState> es, const ValMap & extraEnv);
bool debugStop;
bool inDebugger = false;
int trylevel;
@@ -249,13 +252,21 @@ private:
/**
* A cache from path names to parse trees.
*/
using FileParseCache = GcMap<SourcePath, Expr *>;
#if HAVE_BOEHMGC
typedef std::map<SourcePath, Expr *, std::less<SourcePath>, traceable_allocator<std::pair<const SourcePath, Expr *>>> FileParseCache;
#else
typedef std::map<SourcePath, Expr *> FileParseCache;
#endif
FileParseCache fileParseCache;
/**
* A cache from path names to values.
*/
using FileEvalCache = GcMap<SourcePath, Value>;
#if HAVE_BOEHMGC
typedef std::map<SourcePath, Value, std::less<SourcePath>, traceable_allocator<std::pair<const SourcePath, Value>>> FileEvalCache;
#else
typedef std::map<SourcePath, Value> FileEvalCache;
#endif
FileEvalCache fileEvalCache;
SearchPath searchPath;
@@ -344,8 +355,8 @@ public:
/**
* Parse a Nix expression from the specified string.
*/
Expr & parseExprFromString(std::string s, const SourcePath & basePath, std::shared_ptr<StaticEnv> & staticEnv, const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
Expr & parseExprFromString(std::string s, const SourcePath & basePath, const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
Expr & parseExprFromString(std::string s, const SourcePath & basePath, std::shared_ptr<StaticEnv> & staticEnv);
Expr & parseExprFromString(std::string s, const SourcePath & basePath);
Expr & parseStdin();
@@ -568,8 +579,7 @@ private:
size_t length,
Pos::Origin origin,
const SourcePath & basePath,
std::shared_ptr<StaticEnv> & staticEnv,
const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
std::shared_ptr<StaticEnv> & staticEnv);
/**
* Current Nix call stack depth, used with `max-call-depth` setting to throw stack overflow hopefully before we run out of system stack.
@@ -727,15 +737,15 @@ private:
bool countCalls;
using PrimOpCalls = std::map<std::string, size_t>;
typedef std::map<std::string, size_t> PrimOpCalls;
PrimOpCalls primOpCalls;
using FunctionCalls = std::map<ExprLambda *, size_t>;
typedef std::map<ExprLambda *, size_t> FunctionCalls;
FunctionCalls functionCalls;
void incrFunctionCall(ExprLambda * fun);
using AttrSelects = std::map<PosIdx, size_t>;
typedef std::map<PosIdx, size_t> AttrSelects;
AttrSelects attrSelects;
friend struct ExprOpUpdate;
@@ -777,7 +787,7 @@ std::string showType(const Value & v);
*/
SourcePath resolveExprPath(SourcePath path);
static constexpr std::string_view corepkgsPrefix{"/__corepkgs__/"};
static const std::string corepkgsPrefix{"/__corepkgs__/"};
}
+19 -35
View File
@@ -51,46 +51,30 @@ void ConfigFile::apply()
else
assert(false);
bool trusted = whitelist.count(baseName);
if (!trusted) {
switch (nix::fetchSettings.acceptFlakeConfig.get()) {
case AcceptFlakeConfig::True: {
trusted = true;
break;
}
case AcceptFlakeConfig::Ask: {
auto trustedList = readTrustedList();
auto tlname = get(trustedList, name);
if (auto saved = tlname ? get(*tlname, valueS) : nullptr) {
trusted = *saved;
printInfo("Using saved setting for '%s = %s' from ~/.local/share/nix/trusted-settings.json.", name, valueS);
} else {
// FIXME: filter ANSI escapes, newlines, \r, etc.
if (std::tolower(logger->ask(fmt("Do you want to allow configuration setting '%s' to be set to '" ANSI_RED "%s" ANSI_NORMAL "' (y/N)? This may allow the flake to gain root, see the nix.conf manual page.", name, valueS)).value_or('n')) == 'y') {
trusted = true;
} else {
warn("you can set '%s' to '%b' to automatically reject configuration options supplied by flakes", "accept-flake-config", false);
}
if (std::tolower(logger->ask(fmt("do you want to permanently mark this value as %s (y/N)?", trusted ? "trusted": "untrusted" )).value_or('n')) == 'y') {
trustedList[name][valueS] = trusted;
writeTrustedList(trustedList);
}
if (!whitelist.count(baseName) && !nix::fetchSettings.acceptFlakeConfig) {
bool trusted = false;
auto trustedList = readTrustedList();
auto tlname = get(trustedList, name);
if (auto saved = tlname ? get(*tlname, valueS) : nullptr) {
trusted = *saved;
printInfo("Using saved setting for '%s = %s' from ~/.local/share/nix/trusted-settings.json.", name, valueS);
} else {
// FIXME: filter ANSI escapes, newlines, \r, etc.
if (std::tolower(logger->ask(fmt("do you want to allow configuration setting '%s' to be set to '" ANSI_RED "%s" ANSI_NORMAL "' (y/N)?", name, valueS)).value_or('n')) == 'y') {
trusted = true;
}
if (std::tolower(logger->ask(fmt("do you want to permanently mark this value as %s (y/N)?", trusted ? "trusted": "untrusted" )).value_or('n')) == 'y') {
trustedList[name][valueS] = trusted;
writeTrustedList(trustedList);
}
break;
}
case nix::AcceptFlakeConfig::False: {
trusted = false;
break;
};
if (!trusted) {
warn("ignoring untrusted flake configuration setting '%s'.\nPass '%s' to trust it", name, "--accept-flake-config");
continue;
}
}
if (trusted) {
debug("accepting trusted flake configuration setting '%s'", name);
globalConfig.set(name, valueS);
} else {
warn("ignoring untrusted flake configuration setting '%s', pass '%s' to trust it (may allow the flake to gain root, see the nix.conf manual page)", name, "--accept-flake-config");
}
globalConfig.set(name, valueS);
}
}
+6 -19
View File
@@ -137,16 +137,9 @@ static FlakeInput parseFlakeInput(EvalState & state,
case nBool:
attrs.emplace(state.symbols[attr.name], Explicit<bool> { attr.value->boolean });
break;
case nInt: {
auto intValue = attr.value->integer.value;
if (intValue < 0) {
state.error<EvalError>("negative value given for flake input attribute %1%: %2%", state.symbols[attr.name], intValue).debugThrow();
}
uint64_t asUnsigned = intValue;
attrs.emplace(state.symbols[attr.name], asUnsigned);
case nInt:
attrs.emplace(state.symbols[attr.name], (long unsigned int)attr.value->integer);
break;
}
default:
state.error<TypeError>("flake input attribute '%s' is %s while a string, Boolean, or integer is expected",
state.symbols[attr.name], showType(*attr.value)).debugThrow();
@@ -291,7 +284,7 @@ static Flake getFlake(
else if (setting.value->type() == nInt)
flake.config.settings.emplace(
state.symbols[setting.name],
state.forceInt(*setting.value, setting.pos, "").value);
state.forceInt(*setting.value, setting.pos, ""));
else if (setting.value->type() == nBool)
flake.config.settings.emplace(
state.symbols[setting.name],
@@ -880,14 +873,8 @@ static void prim_flakeRefToString(
for (const auto & attr : *args[0]->attrs) {
auto t = attr.value->type();
if (t == nInt) {
auto intValue = attr.value->integer.value;
if (intValue < 0) {
state.error<EvalError>("negative value given for flake ref attr %1%: %2%", state.symbols[attr.name], intValue).atPos(pos).debugThrow();
}
uint64_t asUnsigned = intValue;
attrs.emplace(state.symbols[attr.name], asUnsigned);
attrs.emplace(state.symbols[attr.name],
(uint64_t) attr.value->integer);
} else if (t == nBool) {
attrs.emplace(state.symbols[attr.name],
Explicit<bool> { attr.value->boolean });
@@ -937,7 +924,7 @@ Fingerprint LockedFlake::getFingerprint() const
// FIXME: as an optimization, if the flake contains a lock file
// and we haven't changed it, then it's sufficient to use
// flake.sourceInfo.storePath for the fingerprint.
return hashString(HashType::SHA256,
return hashString(htSHA256,
fmt("%s;%s;%d;%d;%s",
flake.sourceInfo->storePath.to_string(),
flake.lockedRef.subdir,
+6 -11
View File
@@ -85,8 +85,8 @@ std::pair<FlakeRef, std::string> parseFlakeRefWithFragment(
+ "(?:#(" + queryRegex + "))?",
std::regex::ECMAScript);
static std::regex flakeShorthandRegex(
flakeShorthandRegexS
static std::regex flakeRegex(
"((" + flakeIdRegexS + ")(?:/(?:" + refAndOrRevRegex + "))?)"
+ "(?:#(" + queryRegex + "))?",
std::regex::ECMAScript);
@@ -95,7 +95,7 @@ std::pair<FlakeRef, std::string> parseFlakeRefWithFragment(
/* Check if 'url' is a flake ID. This is an abbreviated syntax for
'flake:<flake-id>?ref=<ref>&rev=<rev>'. */
if (std::regex_match(url, match, flakeShorthandRegex)) {
if (std::regex_match(url, match, flakeRegex)) {
auto parsedURL = ParsedURL{
.url = url,
.base = "flake:" + match.str(1),
@@ -169,13 +169,14 @@ std::pair<FlakeRef, std::string> parseFlakeRefWithFragment(
if (subdir != "") {
if (parsedURL.query.count("dir"))
throw Error("flake URL '%s' has an inconsistent 'dir' parameter", url);
parsedURL.query.insert_or_assign("dir", subdir);
}
if (pathExists(flakeRoot + "/.git/shallow"))
parsedURL.query.insert_or_assign("shallow", "1");
return std::make_pair(
FlakeRef(Input::fromURL(parsedURL, isFlake), subdir),
FlakeRef(Input::fromURL(parsedURL, isFlake), getOr(parsedURL.query, "dir", "")),
fragment);
}
@@ -203,13 +204,7 @@ std::pair<FlakeRef, std::string> parseFlakeRefWithFragment(
std::string fragment;
std::swap(fragment, parsedURL.fragment);
// This has a special meaning for flakes and must not be passed to libfetchers.
// Of course this means that libfetchers cannot have fetchers
// expecting an argument `dir` 🫠
ParsedURL urlForFetchers(parsedURL);
urlForFetchers.query.erase("dir");
auto input = Input::fromURL(urlForFetchers, isFlake);
auto input = Input::fromURL(parsedURL, isFlake);
input.parent = baseDir;
return std::make_pair(
-23
View File
@@ -1,23 +0,0 @@
#include "gc-alloc.hh"
#include <cstring>
#include <string_view>
namespace nix
{
char const * gcCopyStringIfNeeded(std::string_view toCopyFrom)
{
if (toCopyFrom.empty()) {
return "";
}
size_t const size = toCopyFrom.size();
char * cstr = gcAllocString(size + 1);
memcpy(cstr, toCopyFrom.data(), size);
cstr[size] = '\0';
return cstr;
}
}
-150
View File
@@ -1,150 +0,0 @@
#pragma once
/// @file Aliases and wrapper functions that are transparently GC-enabled
/// if Lix is compiled with BoehmGC enabled.
#include <cstddef>
#include <cstring>
#include <list>
#include <map>
#include <new>
#include <string_view>
#include <vector>
#include "checked-arithmetic.hh"
#if HAVE_BOEHMGC
#include <functional> // std::less
#include <utility> // std::pair
#define GC_INCLUDE_NEW
#include <gc/gc.h>
#include <gc/gc_allocator.h>
#include <gc/gc_cpp.h>
/// calloc, transparently GC-enabled.
#define LIX_GC_CALLOC(size) GC_MALLOC(size)
/// strdup, transaprently GC-enabled.
#define LIX_GC_STRDUP(str) GC_STRDUP(str)
/// Atomic GC malloc() with GC enabled, or regular malloc() otherwise.
#define LIX_GC_MALLOC_ATOMIC(size) GC_MALLOC_ATOMIC(size)
namespace nix
{
/// Alias for std::map which uses BoehmGC's allocator conditional on this Lix
/// build having GC enabled.
template<typename KeyT, typename ValueT>
using GcMap = std::map<
KeyT,
ValueT,
std::less<KeyT>,
traceable_allocator<std::pair<KeyT const, ValueT>>
>;
/// Alias for std::vector which uses BoehmGC's allocator conditional on this Lix
/// build having GC enabled.
template<typename ItemT>
using GcVector = std::vector<ItemT, traceable_allocator<ItemT>>;
/// Alias for std::list which uses BoehmGC's allocator conditional on this Lix
/// build having GC enabled.
template<typename ItemT>
using GcList = std::list<ItemT, traceable_allocator<ItemT>>;
}
#else
#include <cstdlib>
/// calloc, transparently GC-enabled.
#define LIX_GC_CALLOC(size) calloc(size, 1)
/// strdup, transparently GC-enabled.
#define LIX_GC_STRDUP(str) strdup(str)
/// Atomic GC malloc() with GC enabled, or regular malloc() otherwise.
/// The returned memory must never contain pointers.
#define LIX_GC_MALLOC_ATOMIC(size) malloc(size)
namespace nix
{
/// Alias for std::map which uses BoehmGC's allocator conditional on this Lix
/// build having GC enabled.
template<typename KeyT, typename ValueT>
using GcMap = std::map<KeyT, ValueT>;
/// Alias for std::vector which uses BoehmGC's allocator conditional on this Lix
/// build having GC enabled.
template<typename ItemT>
using GcVector = std::vector<ItemT>;
/// Alias for std::list which uses BoehmGC's allocator conditional on this Lix
/// build having GC enabled.
template<typename ItemT>
using GcList = std::list<ItemT>;
}
#endif
namespace nix
{
[[gnu::always_inline]]
inline void * gcAllocBytes(size_t n)
{
// Note: various places expect the allocated memory to be zero.
// Hence: calloc().
void * ptr = LIX_GC_CALLOC(n);
if (ptr == nullptr) {
throw std::bad_alloc();
}
return ptr;
}
/// Typed, safe wrapper around calloc() (transparently GC-enabled). Allocates
/// enough for the requested count of the specified type. Also checks for
/// nullptr (and throws @ref std::bad_alloc), and casts the void pointer to
/// a pointer of the specified type, for type-convenient goodness.
template<typename T>
[[gnu::always_inline]]
inline T * gcAllocType(size_t howMany = 1)
{
// NOTE: size_t * size_t, which can definitely overflow.
// Unsigned integer overflow is definitely a bug, but isn't undefined
// behavior, so we can just check if we overflowed after the fact.
// However, people can and do request zero sized allocations, so we need
// to check that neither of our multiplicands were zero before complaining
// about it.
auto checkedSz = checked::Checked<size_t>(howMany) * sizeof(T);
size_t sz = checkedSz.valueWrapping();
if (checkedSz.overflowed()) {
// Congrats, you done did an overflow.
throw std::bad_alloc();
}
return static_cast<T *>(gcAllocBytes(sz));
}
/// GC-transparently allocates a buffer for a C-string of @ref size *bytes*,
/// meaning you should include the size needed by the NUL terminator in the
/// passed size. Memory allocated with this function must never contain other
/// pointers.
inline char * gcAllocString(size_t size)
{
char * cstr = static_cast<char *>(LIX_GC_MALLOC_ATOMIC(size));
if (cstr == nullptr) {
throw std::bad_alloc();
}
return cstr;
}
/// Returns a C-string copied from @ref toCopyFrom, or a single, static empty
/// string if @ref toCopyFrom is also empty.
char const * gcCopyStringIfNeeded(std::string_view toCopyFrom);
}
+99 -207
View File
@@ -1,7 +1,6 @@
#include "get-drvs.hh"
#include "eval-inline.hh"
#include "derivations.hh"
#include "eval.hh"
#include "store-api.hh"
#include "path-with-outputs.hh"
@@ -46,7 +45,7 @@ DrvInfo::DrvInfo(EvalState & state, ref<Store> store, const std::string & drvPat
}
std::string DrvInfo::queryName()
std::string DrvInfo::queryName() const
{
if (name == "" && attrs) {
auto i = attrs->find(state->sName);
@@ -57,7 +56,7 @@ std::string DrvInfo::queryName()
}
std::string DrvInfo::querySystem()
std::string DrvInfo::querySystem() const
{
if (system == "" && attrs) {
auto i = attrs->find(state->sSystem);
@@ -67,7 +66,7 @@ std::string DrvInfo::querySystem()
}
std::optional<StorePath> DrvInfo::queryDrvPath()
std::optional<StorePath> DrvInfo::queryDrvPath() const
{
if (!drvPath && attrs) {
Bindings::iterator i = attrs->find(state->sDrvPath);
@@ -81,7 +80,7 @@ std::optional<StorePath> DrvInfo::queryDrvPath()
}
StorePath DrvInfo::requireDrvPath()
StorePath DrvInfo::requireDrvPath() const
{
if (auto drvPath = queryDrvPath())
return *drvPath;
@@ -89,7 +88,7 @@ StorePath DrvInfo::requireDrvPath()
}
StorePath DrvInfo::queryOutPath()
StorePath DrvInfo::queryOutPath() const
{
if (!outPath && attrs) {
Bindings::iterator i = attrs->find(state->sOutPath);
@@ -102,138 +101,70 @@ StorePath DrvInfo::queryOutPath()
return *outPath;
}
void DrvInfo::fillOutputs(bool withPaths)
{
auto fillDefault = [&]() {
std::optional<StorePath> outPath = std::nullopt;
if (withPaths) {
outPath.emplace(this->queryOutPath());
}
this->outputs.emplace("out", outPath);
};
// lol. lmao even.
if (this->attrs == nullptr) {
fillDefault();
return;
}
Attr * outputs = this->attrs->get(this->state->sOutputs);
if (outputs == nullptr) {
fillDefault();
return;
}
// NOTE(Qyriad): I don't think there is any codepath that can cause this to error.
this->state->forceList(
*outputs->value,
outputs->pos,
"while evaluating the 'outputs' attribute of a derivation"
);
for (auto [idx, elem] : enumerate(outputs->value->listItems())) {
// NOTE(Qyriad): This error should be *extremely* rare in practice.
// It is impossible to construct with `stdenv.mkDerivation`,
// `builtins.derivation`, or even `derivationStrict`. As far as we can tell,
// it is only possible by overriding a derivation attrset already created by
// one of those with `//` to introduce the failing `outputs` entry.
auto errMsg = fmt("while evaluating output %d of a derivation", idx);
std::string_view outputName = state->forceStringNoCtx(
*elem,
outputs->pos,
errMsg
);
if (withPaths) {
// Find the attr with this output's name...
Attr * out = this->attrs->get(this->state->symbols.create(outputName));
if (out == nullptr) {
// FIXME: throw error?
continue;
}
// Meanwhile we couldn't figure out any circumstances
// that cause this to error.
state->forceAttrs(*out->value, outputs->pos, errMsg);
// ...and evaluate its `outPath` attribute.
Attr * outPath = out->value->attrs->get(this->state->sOutPath);
if (outPath == nullptr) {
continue;
// FIXME: throw error?
}
NixStringContext context;
// And idk what could possibly cause this one to error
// that wouldn't error before here.
auto storePath = state->coerceToStorePath(
outPath->pos,
*outPath->value,
context,
errMsg
);
this->outputs.emplace(outputName, storePath);
} else {
this->outputs.emplace(outputName, std::nullopt);
}
}
}
DrvInfo::Outputs DrvInfo::queryOutputs(bool withPaths, bool onlyOutputsToInstall)
{
// If we haven't already cached the outputs set, then do so now.
if (outputs.empty()) {
// FIXME: this behavior seems kind of busted, since whether or not this
// DrvInfo will have paths is forever determined by the *first* call to
// this function??
fillOutputs(withPaths);
/* Get the outputs list. */
Bindings::iterator i;
if (attrs && (i = attrs->find(state->sOutputs)) != attrs->end()) {
state->forceList(*i->value, i->pos, "while evaluating the 'outputs' attribute of a derivation");
/* For each output... */
for (auto elem : i->value->listItems()) {
std::string output(state->forceStringNoCtx(*elem, i->pos, "while evaluating the name of an output of a derivation"));
if (withPaths) {
/* Evaluate the corresponding set. */
Bindings::iterator out = attrs->find(state->symbols.create(output));
if (out == attrs->end()) continue; // FIXME: throw error?
state->forceAttrs(*out->value, i->pos, "while evaluating an output of a derivation");
/* And evaluate its outPath attribute. */
Bindings::iterator outPath = out->value->attrs->find(state->sOutPath);
if (outPath == out->value->attrs->end()) continue; // FIXME: throw error?
NixStringContext context;
outputs.emplace(output, state->coerceToStorePath(outPath->pos, *outPath->value, context, "while evaluating an output path of a derivation"));
} else
outputs.emplace(output, std::nullopt);
}
} else
outputs.emplace("out", withPaths ? std::optional{queryOutPath()} : std::nullopt);
}
// Things that operate on derivations like packages, like `nix-env` and `nix build`,
// allow derivations to specify which outputs should be used in those user-facing
// cases if the user didn't specify an output explicitly.
// If the caller just wanted all the outputs for this derivation, though,
// then we're done here.
if (!onlyOutputsToInstall || !attrs)
return outputs;
// Regardless of `meta.outputsToInstall`, though, you can select into a derivation
// output by its attribute, e.g. `pkgs.lix.dev`, which (lol?) sets the magic
// attribute `outputSpecified = true`, and changes the `outputName` attr to the
// explicitly selected-into output.
if (Attr * outSpecAttr = attrs->get(state->sOutputSpecified)) {
bool outputSpecified = this->state->forceBool(
*outSpecAttr->value,
outSpecAttr->pos,
"while evaluating the 'outputSpecified' attribute of a derivation"
);
if (outputSpecified) {
auto maybeOut = outputs.find(queryOutputName());
if (maybeOut == outputs.end()) {
throw Error("derivation does not have output '%s'", queryOutputName());
}
return Outputs{*maybeOut};
}
Bindings::iterator i;
if (attrs && (i = attrs->find(state->sOutputSpecified)) != attrs->end() && state->forceBool(*i->value, i->pos, "while evaluating the 'outputSpecified' attribute of a derivation")) {
Outputs result;
auto out = outputs.find(queryOutputName());
if (out == outputs.end())
throw Error("derivation does not have output '%s'", queryOutputName());
result.insert(*out);
return result;
}
/* Check for `meta.outputsToInstall` and return `outputs` reduced to that. */
const Value * outTI = queryMeta("outputsToInstall");
if (!outTI) return outputs;
auto errMsg = Error("this derivation has bad 'meta.outputsToInstall'");
/* ^ this shows during `nix-env -i` right under the bad derivation */
if (!outTI->isList()) throw errMsg;
Outputs result;
for (auto elem : outTI->listItems()) {
if (elem->type() != nString) throw errMsg;
auto out = outputs.find(elem->string.s);
if (out == outputs.end()) throw errMsg;
result.insert(*out);
else {
/* Check for `meta.outputsToInstall` and return `outputs` reduced to that. */
const Value * outTI = queryMeta("outputsToInstall");
if (!outTI) return outputs;
auto errMsg = Error("this derivation has bad 'meta.outputsToInstall'");
/* ^ this shows during `nix-env -i` right under the bad derivation */
if (!outTI->isList()) throw errMsg;
Outputs result;
for (auto elem : outTI->listItems()) {
if (elem->type() != nString) throw errMsg;
auto out = outputs.find(elem->string.s);
if (out == outputs.end()) throw errMsg;
result.insert(*out);
}
return result;
}
return result;
}
std::string DrvInfo::queryOutputName()
std::string DrvInfo::queryOutputName() const
{
if (outputName == "" && attrs) {
Bindings::iterator i = attrs->find(state->sOutputName);
@@ -310,8 +241,8 @@ NixInt DrvInfo::queryMetaInt(const std::string & name, NixInt def)
if (v->type() == nString) {
/* Backwards compatibility with before we had support for
integer meta fields. */
if (auto n = string2Int<NixInt::Inner>(v->string.s))
return NixInt{*n};
if (auto n = string2Int<NixInt>(v->string.s))
return *n;
}
return def;
}
@@ -419,95 +350,56 @@ static void getDerivations(EvalState & state, Value & vIn,
Value v;
state.autoCallFunction(autoArgs, vIn, v);
bool shouldRecurse = getDerivation(state, v, pathPrefix, drvs, ignoreAssertionFailures);
if (!shouldRecurse) {
// We're done here.
return;
/* Process the expression. */
if (!getDerivation(state, v, pathPrefix, drvs, ignoreAssertionFailures)) ;
else if (v.type() == nAttrs) {
/* Dont consider sets we've already seen, e.g. y in
`rec { x.d = derivation {...}; y = x; }`. */
if (!done.insert(v.attrs).second) return;
/* !!! undocumented hackery to support combining channels in
nix-env.cc. */
bool combineChannels = v.attrs->find(state.symbols.create("_combineChannels")) != v.attrs->end();
/* Consider the attributes in sorted order to get more
deterministic behaviour in nix-env operations (e.g. when
there are names clashes between derivations, the derivation
bound to the attribute with the "lower" name should take
precedence). */
for (auto & i : v.attrs->lexicographicOrder(state.symbols)) {
debug("evaluating attribute '%1%'", state.symbols[i->name]);
if (!std::regex_match(std::string(state.symbols[i->name]), attrRegex))
continue;
std::string pathPrefix2 = addToPath(pathPrefix, state.symbols[i->name]);
if (combineChannels)
getDerivations(state, *i->value, pathPrefix2, autoArgs, drvs, done, ignoreAssertionFailures);
else if (getDerivation(state, *i->value, pathPrefix2, drvs, ignoreAssertionFailures)) {
/* If the value of this attribute is itself a set,
should we recurse into it? => Only if it has a
`recurseForDerivations = true' attribute. */
if (i->value->type() == nAttrs) {
Bindings::iterator j = i->value->attrs->find(state.sRecurseForDerivations);
if (j != i->value->attrs->end() && state.forceBool(*j->value, j->pos, "while evaluating the attribute `recurseForDerivations`"))
getDerivations(state, *i->value, pathPrefix2, autoArgs, drvs, done, ignoreAssertionFailures);
}
}
}
}
if (v.type() == nList) {
else if (v.type() == nList) {
// NOTE we can't really deduplicate here because small lists don't have stable addresses
// and can cause spurious duplicate detections due to v being on the stack.
for (auto [n, elem] : enumerate(v.listItems())) {
std::string joinedAttrPath = addToPath(pathPrefix, fmt("%d", n));
bool shouldRecurse = getDerivation(state, *elem, joinedAttrPath, drvs, ignoreAssertionFailures);
if (shouldRecurse) {
getDerivations(
state,
*elem,
joinedAttrPath,
autoArgs,
drvs,
done,
ignoreAssertionFailures
);
}
}
return;
} else if (v.type() != nAttrs) {
state.error<TypeError>(
"expression was expected to be a derivation or collection of derivations, but instead was %s",
showType(v.type(), true)
).debugThrow();
}
/* Dont consider sets we've already seen, e.g. y in
`rec { x.d = derivation {...}; y = x; }`. */
auto const &[_, didInsert] = done.insert(v.attrs);
if (!didInsert) {
return;
}
// FIXME: what the fuck???
/* !!! undocumented hackery to support combining channels in
nix-env.cc. */
bool combineChannels = v.attrs->find(state.symbols.create("_combineChannels")) != v.attrs->end();
/* Consider the attributes in sorted order to get more
deterministic behaviour in nix-env operations (e.g. when
there are names clashes between derivations, the derivation
bound to the attribute with the "lower" name should take
precedence). */
for (auto & attr : v.attrs->lexicographicOrder(state.symbols)) {
debug("evaluating attribute '%1%'", state.symbols[attr->name]);
// FIXME: only consider attrs with identifier-like names?? Why???
if (!std::regex_match(std::string(state.symbols[attr->name]), attrRegex)) {
continue;
}
std::string joinedAttrPath = addToPath(pathPrefix, state.symbols[attr->name]);
if (combineChannels) {
getDerivations(state, *attr->value, joinedAttrPath, autoArgs, drvs, done, ignoreAssertionFailures);
} else if (getDerivation(state, *attr->value, joinedAttrPath, drvs, ignoreAssertionFailures)) {
/* If the value of this attribute is itself a set,
should we recurse into it? => Only if it has a
`recurseForDerivations = true' attribute. */
if (attr->value->type() == nAttrs) {
Attr * recurseForDrvs = attr->value->attrs->get(state.sRecurseForDerivations);
if (recurseForDrvs == nullptr) {
continue;
}
bool shouldRecurse = state.forceBool(
*recurseForDrvs->value,
attr->pos,
fmt("while evaluating the '%s' attribute", Magenta("recurseForDerivations"))
);
if (!shouldRecurse) {
continue;
}
getDerivations(
state,
*attr->value,
joinedAttrPath,
autoArgs,
drvs,
done,
ignoreAssertionFailures
);
}
std::string pathPrefix2 = addToPath(pathPrefix, fmt("%d", n));
if (getDerivation(state, *elem, pathPrefix2, drvs, ignoreAssertionFailures))
getDerivations(state, *elem, pathPrefix2, autoArgs, drvs, done, ignoreAssertionFailures);
}
}
else
state.error<TypeError>("expression does not evaluate to a derivation (or a set or list of those)").debugThrow();
}
+19 -14
View File
@@ -19,11 +19,11 @@ public:
private:
EvalState * state;
std::string name;
std::string system;
std::optional<std::optional<StorePath>> drvPath;
std::optional<StorePath> outPath;
std::string outputName;
mutable std::string name;
mutable std::string system;
mutable std::optional<std::optional<StorePath>> drvPath;
mutable std::optional<StorePath> outPath;
mutable std::string outputName;
Outputs outputs;
/**
@@ -37,23 +37,22 @@ private:
bool checkMeta(Value & v);
void fillOutputs(bool withPaths = true);
public:
/**
* path towards the derivation
*/
std::string attrPath;
DrvInfo(EvalState & state) : state(&state) { };
DrvInfo(EvalState & state, std::string attrPath, Bindings * attrs);
DrvInfo(EvalState & state, ref<Store> store, const std::string & drvPathWithOutputs);
std::string queryName();
std::string querySystem();
std::optional<StorePath> queryDrvPath();
StorePath requireDrvPath();
StorePath queryOutPath();
std::string queryOutputName();
std::string queryName() const;
std::string querySystem() const;
std::optional<StorePath> queryDrvPath() const;
StorePath requireDrvPath() const;
StorePath queryOutPath() const;
std::string queryOutputName() const;
/**
* Return the unordered map of output names to (optional) output paths.
* The "outputs to install" are determined by `meta.outputsToInstall`.
@@ -81,7 +80,13 @@ public:
bool hasFailed() { return failed; };
};
using DrvInfos = GcList<DrvInfo>;
#if HAVE_BOEHMGC
typedef std::list<DrvInfo, traceable_allocator<DrvInfo>> DrvInfos;
#else
typedef std::list<DrvInfo> DrvInfos;
#endif
/**
* If value `v` denotes a derivation, return a DrvInfo object
+13 -20
View File
@@ -2,7 +2,6 @@
#include "value.hh"
#include "eval.hh"
#include <limits>
#include <variant>
#include <nlohmann/json.hpp>
@@ -82,46 +81,42 @@ class JSONSax : nlohmann::json_sax<json> {
public:
JSONSax(EvalState & state, Value & v) : state(state), rs(new JSONState(&v)) {};
bool null() override
bool null()
{
rs->value(state).mkNull();
rs->add();
return true;
}
bool boolean(bool val) override
bool boolean(bool val)
{
rs->value(state).mkBool(val);
rs->add();
return true;
}
bool number_integer(number_integer_t val) override
bool number_integer(number_integer_t val)
{
rs->value(state).mkInt(val);
rs->add();
return true;
}
bool number_unsigned(number_unsigned_t val_) override
bool number_unsigned(number_unsigned_t val)
{
if (val_ > std::numeric_limits<NixInt::Inner>::max()) {
throw Error("unsigned json number %1% outside of Nix integer range", val_);
}
NixInt::Inner val = val_;
rs->value(state).mkInt(val);
rs->add();
return true;
}
bool number_float(number_float_t val, const string_t & s) override
bool number_float(number_float_t val, const string_t & s)
{
rs->value(state).mkFloat(val);
rs->add();
return true;
}
bool string(string_t & val) override
bool string(string_t & val)
{
rs->value(state).mkString(val);
rs->add();
@@ -129,7 +124,7 @@ public:
}
#if NLOHMANN_JSON_VERSION_MAJOR >= 3 && NLOHMANN_JSON_VERSION_MINOR >= 8
bool binary(binary_t&) override
bool binary(binary_t&)
{
// This function ought to be unreachable
assert(false);
@@ -137,37 +132,35 @@ public:
}
#endif
bool start_object(std::size_t len) override
bool start_object(std::size_t len)
{
rs = std::make_unique<JSONObjectState>(std::move(rs));
return true;
}
bool key(string_t & name) override
bool key(string_t & name)
{
dynamic_cast<JSONObjectState*>(rs.get())->key(name, state);
return true;
}
bool end_object() override {
bool end_object() {
rs = rs->resolve(state);
rs->add();
return true;
}
bool end_array() override {
bool end_array() {
return end_object();
}
bool start_array(size_t len) override {
bool start_array(size_t len) {
rs = std::make_unique<JSONListState>(std::move(rs),
len != std::numeric_limits<size_t>::max() ? len : 128);
return true;
}
bool
parse_error(std::size_t, const std::string &, const nlohmann::detail::exception & ex) override
{
bool parse_error(std::size_t, const std::string&, const nlohmann::detail::exception& ex) {
throw JSONParseError("%s", ex.what());
}
};
-3
View File
@@ -22,7 +22,6 @@ libexpr_sources = files(
'eval.cc',
'function-trace.cc',
'get-drvs.cc',
'gc-alloc.cc',
'json-to-value.cc',
'nixexpr.cc',
'parser/parser.cc',
@@ -31,7 +30,6 @@ libexpr_sources = files(
'print-ambiguous.cc',
'print.cc',
'search-path.cc',
'value.cc',
'value-to-json.cc',
'value-to-xml.cc',
'flake/config.cc',
@@ -60,7 +58,6 @@ libexpr_headers = files(
'function-trace.hh',
'gc-small-vector.hh',
'get-drvs.hh',
'gc-alloc.hh',
'json-to-value.hh',
'nixexpr.hh',
'parser/change_head.hh',
-7
View File
@@ -50,13 +50,6 @@ void ExprVar::show(const SymbolTable & symbols, std::ostream & str) const
str << symbols[name];
}
void ExprInheritFrom::show(SymbolTable const & symbols, std::ostream & str) const
{
str << "(/* expanded inherit (expr) */ ";
fromExpr->show(symbols, str);
str << ")";
}
void ExprSelect::show(const SymbolTable & symbols, std::ostream & str) const
{
str << "(";
+4 -9
View File
@@ -75,7 +75,6 @@ struct ExprInt : Expr
NixInt n;
Value v;
ExprInt(NixInt n) : n(n) { v.mkInt(n); };
ExprInt(NixInt::Inner n) : n(n) { v.mkInt(n); };
Value * maybeThunk(EvalState & state, Env & env) override;
COMMON_METHODS
};
@@ -145,18 +144,14 @@ struct ExprVar : Expr
*/
struct ExprInheritFrom : ExprVar
{
ref<Expr> fromExpr;
ExprInheritFrom(PosIdx pos, Displacement displ, ref<Expr> fromExpr)
: ExprVar(pos, {}), fromExpr(fromExpr)
ExprInheritFrom(PosIdx pos, Displacement displ): ExprVar(pos, {})
{
this->level = 0;
this->displ = displ;
this->fromWith = nullptr;
}
void show(SymbolTable const & symbols, std::ostream & str) const override;
void bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> & env) override;
void bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> & env);
};
struct ExprSelect : Expr
@@ -212,7 +207,7 @@ struct ExprAttrs : Expr
AttrDef() { };
template<typename T>
T chooseByKind(const T & plain, const T & inherited, const T & inheritedFrom) const
const T & chooseByKind(const T & plain, const T & inherited, const T & inheritedFrom) const
{
switch (kind) {
case Kind::Plain:
@@ -227,7 +222,7 @@ struct ExprAttrs : Expr
};
typedef std::map<Symbol, AttrDef> AttrDefs;
AttrDefs attrs;
std::unique_ptr<std::vector<ref<Expr>>> inheritFromExprs;
std::unique_ptr<std::vector<std::unique_ptr<Expr>>> inheritFromExprs;
struct DynamicAttrDef {
std::unique_ptr<Expr> nameExpr, valueExpr;
PosIdx pos;
+3 -13
View File
@@ -434,8 +434,6 @@ struct op {
struct and_ : _op<TAO_PEGTL_STRING("&&"), 12> {};
struct or_ : _op<TAO_PEGTL_STRING("||"), 13> {};
struct implies : _op<TAO_PEGTL_STRING("->"), 14, kind::rightAssoc> {};
struct pipe_right : _op<TAO_PEGTL_STRING("|>"), 15> {};
struct pipe_left : _op<TAO_PEGTL_STRING("<|"), 16, kind::rightAssoc> {};
};
struct _expr {
@@ -523,7 +521,6 @@ struct _expr {
app
> {};
/* Order matters here. The order is the parsing order, not the precedence order: '<=' must be parsed before '<'. */
struct _binary_operator : sor<
operator_<op::implies>,
operator_<op::update>,
@@ -532,8 +529,6 @@ struct _expr {
operator_<op::minus>,
operator_<op::mul>,
operator_<op::div>,
operator_<op::pipe_right>,
operator_<op::pipe_left>,
operator_<op::less_eq>,
operator_<op::greater_eq>,
operator_<op::less>,
@@ -624,13 +619,10 @@ struct nothing : p::nothing<Rule> {
static_assert(!std::is_base_of_v<semantic, Rule>);
};
template<typename Self, typename OpCtx, typename AttrPathT, typename ExprT>
struct operator_semantics
{
private:
operator_semantics() = default;
friend Self;
public:
struct operator_semantics {
struct has_attr : grammar::op::has_attr {
AttrPathT path;
};
@@ -657,8 +649,6 @@ public:
grammar::op::minus,
grammar::op::mul,
grammar::op::div,
grammar::op::pipe_right,
grammar::op::pipe_left,
has_attr
> op;
};
+11 -43
View File
@@ -12,6 +12,7 @@
#include "state.hh"
#include <charconv>
#include <clocale>
#include <memory>
// flip this define when doing parser development to enable some g checks.
@@ -42,6 +43,7 @@ error_message_for(p::one<'}'>) = "expecting '}'";
error_message_for(p::one<'"'>) = "expecting '\"'";
error_message_for(p::one<';'>) = "expecting ';'";
error_message_for(p::one<')'>) = "expecting ')'";
error_message_for(p::one<'='>) = "expecting '='";
error_message_for(p::one<']'>) = "expecting ']'";
error_message_for(p::one<':'>) = "expecting ':'";
error_message_for(p::string<'\'', '\''>) = "expecting \"''\"";
@@ -113,29 +115,6 @@ struct ExprState
return std::make_unique<ExprCall>(pos, std::make_unique<ExprVar>(fn), std::move(args));
}
std::unique_ptr<Expr> pipe(PosIdx pos, State & state, bool flip = false)
{
if (!state.xpSettings.isEnabled(Xp::PipeOperator))
throw ParseError({
.msg = HintFmt("Pipe operator is disabled"),
.pos = state.positions[pos]
});
// Reverse the order compared to normal function application: arg |> fn
std::unique_ptr<Expr> fn, arg;
if (flip) {
fn = popExprOnly();
arg = popExprOnly();
} else {
arg = popExprOnly();
fn = popExprOnly();
}
std::vector<std::unique_ptr<Expr>> args{1};
args[0] = std::move(arg);
return std::make_unique<ExprCall>(pos, std::move(fn), std::move(args));
}
std::unique_ptr<Expr> order(PosIdx pos, bool less, State & state)
{
return call(pos, state.s.lessThan, !less);
@@ -185,8 +164,6 @@ struct ExprState
[&] (Op::concat) { return applyBinary<ExprOpConcatLists>(pos); },
[&] (has_attr & a) { return applyUnary<ExprOpHasAttr>(std::move(a.path)); },
[&] (Op::unary_minus) { return negate(pos, state); },
[&] (Op::pipe_right) { return pipe(pos, state, true); },
[&] (Op::pipe_left) { return pipe(pos, state); },
})(op)
};
}
@@ -278,8 +255,7 @@ struct AttrState : SubexprState {
std::vector<AttrName> attrs;
template <typename T>
void pushAttr(T && attr, PosIdx) { attrs.emplace_back(std::forward<T>(attr)); }
void pushAttr(auto && attr, PosIdx) { attrs.emplace_back(std::move(attr)); }
};
template<> struct BuildAST<grammar::attr::simple> {
@@ -315,8 +291,7 @@ struct InheritState : SubexprState {
std::unique_ptr<Expr> from;
PosIdx fromPos;
template <typename T>
void pushAttr(T && attr, PosIdx pos) { attrs.emplace_back(std::forward<T>(attr), pos); }
void pushAttr(auto && attr, PosIdx pos) { attrs.emplace_back(std::move(attr), pos); }
};
template<> struct BuildAST<grammar::inherit::from> {
@@ -342,23 +317,18 @@ template<> struct BuildAST<grammar::inherit> : change_head<InheritState> {
});
}
}
if (s.from != nullptr) {
if (auto fromE = std::move(s.from)) {
if (!b.attrs.inheritFromExprs)
b.attrs.inheritFromExprs = std::make_unique<std::vector<ref<Expr>>>();
auto fromExpr = ref<Expr>(std::move(s.from));
b.attrs.inheritFromExprs->push_back(fromExpr);
b.attrs.inheritFromExprs = std::make_unique<std::vector<std::unique_ptr<Expr>>>();
b.attrs.inheritFromExprs->push_back(std::move(fromE));
for (auto & [i, iPos] : s.attrs) {
if (attrs.find(i.symbol) != attrs.end())
ps.dupAttr(i.symbol, iPos, attrs[i.symbol].pos);
auto inheritFrom = std::make_unique<ExprInheritFrom>(
s.fromPos,
b.attrs.inheritFromExprs->size() - 1,
fromExpr
);
auto from = std::make_unique<ExprInheritFrom>(s.fromPos, b.attrs.inheritFromExprs->size() - 1);
attrs.emplace(
i.symbol,
ExprAttrs::AttrDef(
std::make_unique<ExprSelect>(iPos, std::move(inheritFrom), i.symbol),
std::make_unique<ExprSelect>(iPos, std::move(from), i.symbol),
iPos,
ExprAttrs::AttrDef::Kind::InheritedFrom));
}
@@ -656,7 +626,7 @@ template<> struct BuildAST<grammar::expr::path> : p::maybe_nothing {};
template<> struct BuildAST<grammar::expr::uri> {
static void apply(const auto & in, ExprState & s, State & ps) {
bool noURLLiterals = ps.xpSettings.isEnabled(Xp::NoUrlLiterals);
static bool noURLLiterals = experimentalFeatureSettings.isEnabled(Xp::NoUrlLiterals);
if (noURLLiterals)
throw ParseError({
.msg = HintFmt("URL literals are disabled"),
@@ -857,8 +827,7 @@ Expr * EvalState::parse(
size_t length,
Pos::Origin origin,
const SourcePath & basePath,
std::shared_ptr<StaticEnv> & staticEnv,
const ExperimentalFeatureSettings & xpSettings)
std::shared_ptr<StaticEnv> & staticEnv)
{
parser::State s = {
symbols,
@@ -866,7 +835,6 @@ Expr * EvalState::parse(
basePath,
positions.addOrigin(origin, length),
exprSymbols,
xpSettings
};
parser::ExprState x;
+1 -2
View File
@@ -19,7 +19,6 @@ struct State
SourcePath basePath;
PosTable::Origin origin;
const Expr::AstSymbols & s;
const ExperimentalFeatureSettings & xpSettings;
void dupAttr(const AttrPath & attrPath, const PosIdx pos, const PosIdx prevPos);
void dupAttr(Symbol attr, const PosIdx pos, const PosIdx prevPos);
@@ -104,7 +103,7 @@ inline void State::addAttr(ExprAttrs * attrs, AttrPath && attrPath, std::unique_
auto * jAttrs = dynamic_cast<ExprAttrs *>(j->second.e.get());
if (jAttrs && ae) {
if (ae->inheritFromExprs && !jAttrs->inheritFromExprs)
jAttrs->inheritFromExprs = std::make_unique<std::vector<ref<Expr>>>();
jAttrs->inheritFromExprs = std::make_unique<std::vector<std::unique_ptr<Expr>>>();
for (auto & ad : ae->attrs) {
auto j2 = jAttrs->attrs.find(ad.first);
if (j2 != jAttrs->attrs.end()) // Attr already defined in iAttrs, error.
+44 -103
View File
@@ -346,7 +346,7 @@ void prim_importNative(EvalState & state, const PosIdx pos, Value * * args, Valu
state.error<EvalError>("could not open '%1%': %2%", path, dlerror()).debugThrow();
dlerror();
ValueInitializer func = reinterpret_cast<ValueInitializer>(dlsym(handle, sym.c_str()));
ValueInitializer func = (ValueInitializer) dlsym(handle, sym.c_str());
if(!func) {
char *message = dlerror();
if (message)
@@ -582,9 +582,9 @@ struct CompareValues
{
try {
if (v1->type() == nFloat && v2->type() == nInt)
return v1->fpoint < v2->integer.value;
return v1->fpoint < v2->integer;
if (v1->type() == nInt && v2->type() == nFloat)
return v1->integer.value < v2->fpoint;
return v1->integer < v2->fpoint;
if (v1->type() != v2->type())
state.error<EvalError>("cannot compare %s with %s", showType(*v1), showType(*v2)).debugThrow();
// Allow selecting a subset of enum values
@@ -622,13 +622,14 @@ struct CompareValues
}
};
/// NOTE: this type must NEVER be outside of GC-scanned memory.
#if HAVE_BOEHMGC
using UnsafeValueList = std::list<Value *, gc_allocator<Value *>>;
typedef std::list<Value *, gc_allocator<Value *>> ValueList;
#else
using UnsafeValueList = std::list<Value *>;
typedef std::list<Value *> ValueList;
#endif
static Bindings::iterator getAttr(
EvalState & state,
Symbol attrSym,
@@ -651,7 +652,7 @@ static void prim_genericClosure(EvalState & state, const PosIdx pos, Value * * a
state.forceList(*startSet->value, noPos, "while evaluating the 'startSet' attribute passed as argument to builtins.genericClosure");
UnsafeValueList workSet;
ValueList workSet;
for (auto elem : startSet->value->listItems())
workSet.push_back(elem);
@@ -667,7 +668,7 @@ static void prim_genericClosure(EvalState & state, const PosIdx pos, Value * * a
/* Construct the closure by applying the operator to elements of
`workSet', adding the result to `workSet', continuing until
no new elements are found. */
UnsafeValueList res;
ValueList res;
// `doneKeys' doesn't need to be a GC root, because its values are
// reachable from res.
auto cmp = CompareValues(state, noPos, "while comparing the `key` attributes of two genericClosure elements");
@@ -865,7 +866,7 @@ static void prim_tryEval(EvalState & state, const PosIdx pos, Value * * args, Va
/* increment state.trylevel, and decrement it when this function returns. */
MaintainCount trylevel(state.trylevel);
std::function<ReplExitStatus(ref<EvalState> es, ValMap const & extraEnv)> savedDebugRepl;
ReplExitStatus (* savedDebugRepl)(ref<EvalState> es, const ValMap & extraEnv) = nullptr;
if (state.debugRepl && evalSettings.ignoreExceptionsDuringTry)
{
/* to prevent starting the repl from exceptions withing a tryEval, null it. */
@@ -1213,20 +1214,6 @@ drvName, Bindings * attrs, Value & v)
handleOutputs(ss);
}
if (i->name == state.sAllowedReferences)
warn("In a derivation named '%s', 'structuredAttrs' disables the effect of the derivation attribute 'allowedReferences'; use 'outputChecks.<output>.allowedReferences' instead", drvName);
if (i->name == state.sAllowedRequisites)
warn("In a derivation named '%s', 'structuredAttrs' disables the effect of the derivation attribute 'allowedRequisites'; use 'outputChecks.<output>.allowedRequisites' instead", drvName);
if (i->name == state.sDisallowedReferences)
warn("In a derivation named '%s', 'structuredAttrs' disables the effect of the derivation attribute 'disallowedReferences'; use 'outputChecks.<output>.disallowedReferences' instead", drvName);
if (i->name == state.sDisallowedRequisites)
warn("In a derivation named '%s', 'structuredAttrs' disables the effect of the derivation attribute 'disallowedRequisites'; use 'outputChecks.<output>.disallowedRequisites' instead", drvName);
if (i->name == state.sMaxSize)
warn("In a derivation named '%s', 'structuredAttrs' disables the effect of the derivation attribute 'maxSize'; use 'outputChecks.<output>.maxSize' instead", drvName);
if (i->name == state.sMaxClosureSize)
warn("In a derivation named '%s', 'structuredAttrs' disables the effect of the derivation attribute 'maxClosureSize'; use 'outputChecks.<output>.maxClosureSize' instead", drvName);
} else {
auto s = state.coerceToString(pos, *i->value, context, context_below, true).toOwned();
drv.env.emplace(key, s);
@@ -1336,7 +1323,7 @@ drvName, Bindings * attrs, Value & v)
state.error<EvalError>("derivation cannot be both content-addressed and impure")
.atPos(v).debugThrow();
auto ht = parseHashTypeOpt(outputHashAlgo).value_or(HashType::SHA256);
auto ht = parseHashTypeOpt(outputHashAlgo).value_or(htSHA256);
auto method = ingestionMethod.value_or(FileIngestionMethod::Recursive);
for (auto & i : outputs) {
@@ -1764,7 +1751,7 @@ static void prim_hashFile(EvalState & state, const PosIdx pos, Value * * args, V
auto path = realisePath(state, pos, *args[1]);
v.mkString(hashString(*ht, path.readFile()).to_string(Base::Base16, false));
v.mkString(hashString(*ht, path.readFile()).to_string(Base16, false));
}
static RegisterPrimOp primop_hashFile({
@@ -2346,7 +2333,7 @@ static void prim_path(EvalState & state, const PosIdx pos, Value * * args, Value
else if (n == "recursive")
method = FileIngestionMethod { state.forceBool(*attr.value, attr.pos, "while evaluating the `recursive` attribute passed to builtins.path") };
else if (n == "sha256")
expectedHash = newHashAllowEmpty(state.forceStringNoCtx(*attr.value, attr.pos, "while evaluating the `sha256` attribute passed to builtins.path"), HashType::SHA256);
expectedHash = newHashAllowEmpty(state.forceStringNoCtx(*attr.value, attr.pos, "while evaluating the `sha256` attribute passed to builtins.path"), htSHA256);
else
state.error<EvalError>(
"unsupported argument '%1%' to 'addPath'",
@@ -2439,8 +2426,6 @@ static void prim_attrValues(EvalState & state, const PosIdx pos, Value * * args,
state.mkList(v, args[0]->attrs->size());
// FIXME: this is incredibly evil, *why*
// NOLINTBEGIN(cppcoreguidelines-pro-type-cstyle-cast)
unsigned int n = 0;
for (auto & i : *args[0]->attrs)
v.listElems()[n++] = (Value *) &i;
@@ -2454,7 +2439,6 @@ static void prim_attrValues(EvalState & state, const PosIdx pos, Value * * args,
for (unsigned int i = 0; i < n; ++i)
v.listElems()[i] = ((Attr *) v.listElems()[i])->value;
// NOLINTEND(cppcoreguidelines-pro-type-cstyle-cast)
}
static RegisterPrimOp primop_attrValues({
@@ -2530,13 +2514,13 @@ static struct LazyPosAcessors {
PrimOp primop_lineOfPos{
.arity = 1,
.fun = [] (EvalState & state, PosIdx pos, Value * * args, Value & v) {
v.mkInt(state.positions[PosIdx(args[0]->integer.value)].line);
v.mkInt(state.positions[PosIdx(args[0]->integer)].line);
}
};
PrimOp primop_columnOfPos{
.arity = 1,
.fun = [] (EvalState & state, PosIdx pos, Value * * args, Value & v) {
v.mkInt(state.positions[PosIdx(args[0]->integer.value)].column);
v.mkInt(state.positions[PosIdx(args[0]->integer)].column);
}
};
@@ -3006,8 +2990,7 @@ static void elemAt(EvalState & state, const PosIdx pos, Value & list, int n, Val
/* Return the n-1'th element of a list. */
static void prim_elemAt(EvalState & state, const PosIdx pos, Value * * args, Value & v)
{
NixInt::Inner elem = state.forceInt(*args[1], pos, "while evaluating the second argument passed to builtins.elemAt").value;
elemAt(state, pos, *args[0], elem, v);
elemAt(state, pos, *args[0], state.forceInt(*args[1], pos, "while evaluating the second argument passed to builtins.elemAt"), v);
}
static RegisterPrimOp primop_elemAt({
@@ -3291,19 +3274,17 @@ static RegisterPrimOp primop_all({
static void prim_genList(EvalState & state, const PosIdx pos, Value * * args, Value & v)
{
auto len_ = state.forceInt(*args[1], pos, "while evaluating the second argument passed to builtins.genList").value;
auto len = state.forceInt(*args[1], pos, "while evaluating the second argument passed to builtins.genList");
if (len_ < 0)
state.error<EvalError>("cannot create list of size %1%", len_).atPos(pos).debugThrow();
size_t len = len_;
if (len < 0)
state.error<EvalError>("cannot create list of size %1%", len).atPos(pos).debugThrow();
// More strict than striclty (!) necessary, but acceptable
// as evaluating map without accessing any values makes little sense.
state.forceFunction(*args[0], noPos, "while evaluating the first argument passed to builtins.genList");
state.mkList(v, len);
for (size_t n = 0; n < len; ++n) {
for (unsigned int n = 0; n < (unsigned int) len; ++n) {
auto arg = state.allocValue();
arg->mkInt(n);
(v.listElems()[n] = state.allocValue())->mkApp(args[0], arg);
@@ -3553,17 +3534,9 @@ static void prim_add(EvalState & state, const PosIdx pos, Value * * args, Value
if (args[0]->type() == nFloat || args[1]->type() == nFloat)
v.mkFloat(state.forceFloat(*args[0], pos, "while evaluating the first argument of the addition")
+ state.forceFloat(*args[1], pos, "while evaluating the second argument of the addition"));
else {
auto i1 = state.forceInt(*args[0], pos, "while evaluating the first argument of the addition");
auto i2 = state.forceInt(*args[1], pos, "while evaluating the second argument of the addition");
auto result_ = i1 + i2;
if (auto result = result_.valueChecked(); result.has_value()) {
v.mkInt(*result);
} else {
state.error<EvalError>("integer overflow in adding %1% + %2%", i1, i2).atPos(pos).debugThrow();
}
}
else
v.mkInt( state.forceInt(*args[0], pos, "while evaluating the first argument of the addition")
+ state.forceInt(*args[1], pos, "while evaluating the second argument of the addition"));
}
static RegisterPrimOp primop_add({
@@ -3582,18 +3555,9 @@ static void prim_sub(EvalState & state, const PosIdx pos, Value * * args, Value
if (args[0]->type() == nFloat || args[1]->type() == nFloat)
v.mkFloat(state.forceFloat(*args[0], pos, "while evaluating the first argument of the subtraction")
- state.forceFloat(*args[1], pos, "while evaluating the second argument of the subtraction"));
else {
auto i1 = state.forceInt(*args[0], pos, "while evaluating the first argument of the subtraction");
auto i2 = state.forceInt(*args[1], pos, "while evaluating the second argument of the subtraction");
auto result_ = i1 - i2;
if (auto result = result_.valueChecked(); result.has_value()) {
v.mkInt(*result);
} else {
state.error<EvalError>("integer overflow in subtracting %1% - %2%", i1, i2).atPos(pos).debugThrow();
}
}
else
v.mkInt( state.forceInt(*args[0], pos, "while evaluating the first argument of the subtraction")
- state.forceInt(*args[1], pos, "while evaluating the second argument of the subtraction"));
}
static RegisterPrimOp primop_sub({
@@ -3612,18 +3576,9 @@ static void prim_mul(EvalState & state, const PosIdx pos, Value * * args, Value
if (args[0]->type() == nFloat || args[1]->type() == nFloat)
v.mkFloat(state.forceFloat(*args[0], pos, "while evaluating the first of the multiplication")
* state.forceFloat(*args[1], pos, "while evaluating the second argument of the multiplication"));
else {
auto i1 = state.forceInt(*args[0], pos, "while evaluating the first argument of the multiplication");
auto i2 = state.forceInt(*args[1], pos, "while evaluating the second argument of the multiplication");
auto result_ = i1 * i2;
if (auto result = result_.valueChecked(); result.has_value()) {
v.mkInt(*result);
} else {
state.error<EvalError>("integer overflow in multiplying %1% * %2%", i1, i2).atPos(pos).debugThrow();
}
}
else
v.mkInt( state.forceInt(*args[0], pos, "while evaluating the first argument of the multiplication")
* state.forceInt(*args[1], pos, "while evaluating the second argument of the multiplication"));
}
static RegisterPrimOp primop_mul({
@@ -3650,12 +3605,10 @@ static void prim_div(EvalState & state, const PosIdx pos, Value * * args, Value
NixInt i1 = state.forceInt(*args[0], pos, "while evaluating the first operand of the division");
NixInt i2 = state.forceInt(*args[1], pos, "while evaluating the second operand of the division");
/* Avoid division overflow as it might raise SIGFPE. */
auto result_ = i1 / i2;
if (auto result = result_.valueChecked(); result.has_value()) {
v.mkInt(*result);
} else {
state.error<EvalError>("integer overflow in dividing %1% / %2%", i1, i2).atPos(pos).debugThrow();
}
if (i1 == std::numeric_limits<NixInt>::min() && i2 == -1)
state.error<EvalError>("overflow in integer division").atPos(pos).debugThrow();
v.mkInt(i1 / i2);
}
}
@@ -3670,9 +3623,8 @@ static RegisterPrimOp primop_div({
static void prim_bitAnd(EvalState & state, const PosIdx pos, Value * * args, Value & v)
{
auto i1 = state.forceInt(*args[0], pos, "while evaluating the first argument passed to builtins.bitAnd");
auto i2 = state.forceInt(*args[1], pos, "while evaluating the second argument passed to builtins.bitAnd");
v.mkInt(i1.value & i2.value);
v.mkInt(state.forceInt(*args[0], pos, "while evaluating the first argument passed to builtins.bitAnd")
& state.forceInt(*args[1], pos, "while evaluating the second argument passed to builtins.bitAnd"));
}
static RegisterPrimOp primop_bitAnd({
@@ -3686,10 +3638,8 @@ static RegisterPrimOp primop_bitAnd({
static void prim_bitOr(EvalState & state, const PosIdx pos, Value * * args, Value & v)
{
auto i1 = state.forceInt(*args[0], pos, "while evaluating the first argument passed to builtins.bitOr");
auto i2 = state.forceInt(*args[1], pos, "while evaluating the second argument passed to builtins.bitOr");
v.mkInt(i1.value | i2.value);
v.mkInt(state.forceInt(*args[0], pos, "while evaluating the first argument passed to builtins.bitOr")
| state.forceInt(*args[1], pos, "while evaluating the second argument passed to builtins.bitOr"));
}
static RegisterPrimOp primop_bitOr({
@@ -3703,10 +3653,8 @@ static RegisterPrimOp primop_bitOr({
static void prim_bitXor(EvalState & state, const PosIdx pos, Value * * args, Value & v)
{
auto i1 = state.forceInt(*args[0], pos, "while evaluating the first argument passed to builtins.bitXor");
auto i2 = state.forceInt(*args[1], pos, "while evaluating the second argument passed to builtins.bitXor");
v.mkInt(i1.value ^ i2.value);
v.mkInt(state.forceInt(*args[0], pos, "while evaluating the first argument passed to builtins.bitXor")
^ state.forceInt(*args[1], pos, "while evaluating the second argument passed to builtins.bitXor"));
}
static RegisterPrimOp primop_bitXor({
@@ -3786,19 +3734,13 @@ static RegisterPrimOp primop_toString({
non-negative. */
static void prim_substring(EvalState & state, const PosIdx pos, Value * * args, Value & v)
{
NixInt::Inner start = state.forceInt(*args[0], pos, "while evaluating the first argument (the start offset) passed to builtins.substring").value;
int start = state.forceInt(*args[0], pos, "while evaluating the first argument (the start offset) passed to builtins.substring");
if (start < 0)
state.error<EvalError>("negative start position in 'substring'").atPos(pos).debugThrow();
NixInt::Inner len = state.forceInt(*args[1], pos, "while evaluating the second argument (the substring length) passed to builtins.substring").value;
// Negative length may be idiomatically passed to builtins.substring to get
// the tail of the string.
if (len < 0) {
len = std::numeric_limits<NixInt::Inner>::max();
}
int len = state.forceInt(*args[1], pos, "while evaluating the second argument (the substring length) passed to builtins.substring");
// Special-case on empty substring to avoid O(n) strlen
// This allows for the use of empty substrings to efficently capture string context
@@ -3840,7 +3782,7 @@ static void prim_stringLength(EvalState & state, const PosIdx pos, Value * * arg
{
NixStringContext context;
auto s = state.coerceToString(pos, *args[0], context, "while evaluating the argument passed to builtins.stringLength");
v.mkInt(NixInt::Inner(s->size()));
v.mkInt(s->size());
}
static RegisterPrimOp primop_stringLength({
@@ -3864,7 +3806,7 @@ static void prim_hashString(EvalState & state, const PosIdx pos, Value * * args,
NixStringContext context; // discarded
auto s = state.forceString(*args[1], context, pos, "while evaluating the second argument passed to builtins.hashString");
v.mkString(hashString(*ht, s).to_string(Base::Base16, false));
v.mkString(hashString(*ht, s).to_string(Base16, false));
}
static RegisterPrimOp primop_hashString({
@@ -4220,8 +4162,7 @@ static void prim_compareVersions(EvalState & state, const PosIdx pos, Value * *
{
auto version1 = state.forceStringNoCtx(*args[0], pos, "while evaluating the first argument passed to builtins.compareVersions");
auto version2 = state.forceStringNoCtx(*args[1], pos, "while evaluating the second argument passed to builtins.compareVersions");
auto result = compareVersions(version1, version2);
v.mkInt(result < 0 ? -1 : result > 0 ? 1 : 0);
v.mkInt(compareVersions(version1, version2));
}
static RegisterPrimOp primop_compareVersions({
+2 -2
View File
@@ -31,7 +31,7 @@ static void prim_fetchMercurial(EvalState & state, const PosIdx pos, Value * * a
// be both a revision or a branch/tag name.
auto value = state.forceStringNoCtx(*attr.value, attr.pos, "while evaluating the `rev` attribute passed to builtins.fetchMercurial");
if (std::regex_match(value.begin(), value.end(), revRegex))
rev = Hash::parseAny(value, HashType::SHA1);
rev = Hash::parseAny(value, htSHA1);
else
ref = value;
}
@@ -73,7 +73,7 @@ static void prim_fetchMercurial(EvalState & state, const PosIdx pos, Value * * a
attrs2.alloc("branch").mkString(*input2.getRef());
// Backward compatibility: set 'rev' to
// 0000000000000000000000000000000000000000 for a dirty tree.
auto rev2 = input2.getRev().value_or(Hash(HashType::SHA1));
auto rev2 = input2.getRev().value_or(Hash(htSHA1));
attrs2.alloc("rev").mkString(rev2.gitRev());
attrs2.alloc("shortRev").mkString(rev2.gitRev().substr(0, 12));
if (auto revCount = input2.getRevCount())

Some files were not shown because too many files have changed in this diff Show More