Compare commits

..
Author SHA1 Message Date
eldritch horrors e0beb9b118 [WIP] remove boost coroutine uses
we no longer need these since sinkToSource and sourceToSink are gone.

Change-Id: Ibbf440e2cf71bf3e9f3b833af2d78a21fb1b3193
2024-06-30 21:21:58 +02:00
eldritch horrors f274bfddad [WIP] remove sinkToSource and related boehm patch
Change-Id: I1379841299713175d0225b82a67f50660f9eb5e2
2024-06-30 21:21:58 +02:00
eldritch horrors d08787e42e [WIP] remove remaining sinkToSource uses
Change-Id: Id1ee0d2ad4a3774f4bbb960d76f0f76ac4f3eff9
2024-06-30 21:21:58 +02:00
eldritch horrors 0a1520c602 [WIP] generatorize protocol serializers
this is cursed. deeply and profoundly cursed. under NO CIRCUMSTANCES
must protocol serializer helpers be applied to temporaries! doing so
will inevitably cause dangling references and cause the entire thing
to crash. we need to do this even so to get rid of boost coroutines,
and likewise to encapsulate the serializers we suffer today at least
a little bit to allow a gradual migration to an actual IPC protocol.

Change-Id: I2921ba451e04d86798752d140885d3c5cc08e146
2024-06-30 21:21:58 +02:00
eldritch horrors 5264d72846 [WIP] narFromPath as generator
Change-Id: Ifa783c2c65c06ddd1d0212016d5bfd07666ea91c
2024-06-30 21:21:58 +02:00
eldritch horrors 8fb3b89d0f [WIP] libstore: remove a sinkToSouce from old daemon protocol
this doesn't have a test because this code path is only reached by
clients that predate 2.4, and we really should not be caring about
those any more right now. even the test suite doesn't, and the few
tests that might care are disabled because they will not even work

Change-Id: Id9eb190065138fedb2c7d90c328ff9eb9d97385b
2024-06-30 21:21:58 +02:00
eldritch horrors 425f3e6964 [WIP] libstore: rewrite the nar parser as a contents generator
this is not completely necessary at this point because the parser right
now already returns a generator to pass through all input data it read,
but the nar parser *was* very lax and would accept nars that weren't in
canonical form (defined as the form dumpPath would return). nar hashing
depends on these things, and as such rewriting the parser now allows us
to reject non-canonical nars that extract to the same store contents as
their canonical counterpart, but have different nar hashes.

gcc 12 is broken and causes double frees when yielding certain types of
move-only values, so this also pins gcc to 13 for the default stdenv in
23.11-based builds.

Change-Id: Iccd319e3bd5912d8297014c84c495edc59019bb7
2024-06-30 21:21:58 +02:00
eldritch horrors e11945065b libstore: turn copyNAR into a generator
Change-Id: Id452f6a03faa1037ff13af0f63e32883966ff40d
2024-06-30 21:21:58 +02:00
eldritch horrors 2112a39d2e libstore: turn the NAR parser into a passthrough generator
this will let us turn copyNAR into a generator as well, which in turn is
necessary to turn the users of copyNAR into generators without resorting
to sinkToSource coroutines. currently this uses the SerializingTransform
in all cases, even for copyNAR where it is not necessary. should this be
a performance problem we can easily swap out the transform for one which
does not produce any bytes of its own, but that should not be necessary.

Change-Id: I7e685879318fcbb78d8b88abfddd7752360eb0ce
2024-06-30 21:21:58 +02:00
eldritch horrors 48e3637aa3 libutil: remove sourceToSink
Change-Id: I2867ba2b3a7b59af44c1ec51b08329fccba41ca1
2024-06-30 21:21:58 +02:00
eldritch horrors d658cf5a0e libutil: remove makeDecompressionSink
Change-Id: I4258227b5dbbb735a75b477d8a57007bfca305e9
2024-06-30 21:21:58 +02:00
eldritch horrors 93344a1379 libstore: make BinaryCacheStore::getFile return a source
this lets us remove the last true remaining uses of
makeDecompressionSink.

Change-Id: I146ca2bbe1a9ae9a367117a7b8a304b23a63e5e2
2024-06-30 21:21:58 +02:00
eldritch horrors 9f769794d9 libutil: rewrite RewritingSink as source
the rewriting sink was just broken. when given a rewrite set that
contained a key that is also a proper infix of another key it was
possible to produce an incorrectly rewritten result if the writer
used the wrong block size. fixing this duplicates rewriteStrings,
to avoid this we'll rewrite rewriteStrings to use RewritingSource
in a new mode that'll allow rewrites we had previously forbidden.

Change-Id: I57fa0a9a994e654e11d07172b8e31d15f0b7e8c0
2024-06-30 21:21:58 +02:00
eldritch horrors d1230c7218 libutil: remove RewritingSink match/size tracking
size tracking can be done with a LengthSink and a tee. match tracking
was defeated by never having done any match tracking, all users would
see the same (empty) set of matches at all times. match tracking with
bytes offsets alone would not be sufficient in the general case, only
because computeHashModulo uses a single rewrite would it have worked.

Change-Id: Idb214b5222e0ea24f450f5505712a342b63d7570
2024-06-30 21:21:45 +02:00
eldritch horrors 50e1364a80 libutil: turn HashModuloSink into a free function
Change-Id: I5878007502fa68c2816a0f4c61f7d0e60bdde702
2024-06-30 21:08:17 +02:00
eldritch horrors 3e29a98903 libutil: return sources from runProgram2
this much more closely mimics what is actually happening: we're reading
data from somewhere else, actively, rather than passively waiting. with
the data flow matching the underlying system interactions better we can
remove a few sinkToSource calls that merely exists to undo the mismatch
caused by not treating subprocess output as a data source to begin with

Change-Id: If4abfc2f8398fb5e88c9b91a8bdefd5504bb2d11
2024-06-30 21:05:06 +02:00
eldritch horrors 14d539a274 libutil: return a program handle from runProgram2
this will let us also return a source for the program output later,
which will in turn make sinkToSource unnecessary for program output
processing. this may also reopen a path for provigin program input,
but that still needs a proper async io framework to avoid problems.

Change-Id: Iaf93f47db99c38cfaf134bd60ed6a804d7ddf688
2024-06-30 20:58:02 +02:00
eldritch horrors a5ae26726b enable -Werror=unused-result
we do not have any of these warnings appearing at the moment, but it
seems like a good idea to enable [[nodiscard]] checking anyway. once
we start introducing more functions with must-use conditions we will
need such checking, and the rust stdlib has proven them very useful.

Change-Id: Ibb6b042ae1ec5f527f8dc2809a7816a4c1548ae2
2024-06-30 20:51:10 +02:00
eldritch horrors 5f826bc600 libutil: convert readFileSource to a generator
Change-Id: I5f92b15fd367d46eb047d74ab6e317b4f51a46d3
2024-06-30 19:29:07 +02:00
eldritch horrors f659e93c5b libstore: convert dumpPath to a generator
Change-Id: Ic4cf5562504aa29130304469936f958c0426e5ef
2024-06-30 19:29:07 +02:00
eldritch horrors 1540bdbcfc libutil: allow construction of sources from generators
Change-Id: I78ff8d0720f06bce731e26d5e1c53b1382bbd589
2024-06-30 19:29:07 +02:00
eldritch horrors 379a892afb libutil: begin porting serialization to generators
generators are a better basis for serializers than streaming into sinks
as we do currently for many reasons, such as being usable as sources if
one wishes to (without requiring an intermediate sink to serialize full
data sets into memory, or boost coroutines to turn sinks into sources),
composing more naturally (as one can just yield a sub-generator instead
of being forced to wrap entire substreams into clunky functions or even
more clunky custom types to implement operator<< on), allowing wrappers
to transform data with clear ownership semantics (removing the need for
explicit memory allocations and Source wrappers), and many other things

Change-Id: I361d89ff556354f6930d9204f55117565f2f7f20
2024-06-30 19:29:06 +02:00
eldritch horrors 5f18b52274 libutil: convert drainFD to a Bytes generator
the `*Source` name is a slight misnomer since we do also have a
Source type, but we can probably live with this for time being.

Change-Id: I54eb2e59a4009014e324797f16b80b962759c7d3
2024-06-30 17:38:11 +02:00
eldritch horrors d96601fe55 libutil: allow draining Generator<Bytes> into sinks
Change-Id: I442d03a5399096d4baca9a2618b4c4b64db36c4b
2024-06-30 17:33:28 +02:00
eldritch horrors 2521f52465 libutil: add Bytes type
not used anywhere yet, but we'll use this a lot soon for generators that
return file contents, wire protocol fragments, or indeed any byte stream

Change-Id: I01a46f9bf9d75aaf4a5d7662773b99f498862a28
2024-06-30 17:30:34 +02:00
eldritch horrors 11307e04d1 libutil: generator type with on-yield value mapping
this will be the basis of non-boost coroutines in lix. anything that is
a boost coroutine *should* be representable with a Generator coroutine,
and many things that are not currently boost coroutines but behave much
like one (such as, notably, serializers) should be as well. this allows
us to greatly simplify many things that look like iteration but aren't.

Change-Id: I2cebcefa0148b631fb30df4c8cfa92167a407e34
2024-06-30 17:23:41 +02:00
jadeandGerrit Code Review 5dc85e8b72 Merge "packaging: make pegtl use the __forDefaults mechanism" into main 2024-06-26 22:11:52 +00:00
jadeandGerrit Code Review 77c5364596 Merge "doc/hacking: fix up some outdated info about cross, hydra links" into main 2024-06-26 22:11:36 +00:00
eldritch horrors 3dd7d023f4 libmain: don't print empty lines
this most notably affects `nix eval`: if there is no progress bar to be
shown and no activities going on we should not print anything at all. a
progress bar with no activities would print a bunch of terminal escapes
*and a space*, which is not helpful in simple cases like nix eval -E 1.
notably this does *not* affect nix eval called on non-terminal outputs,
but it is slightly confusing nevertheless (and not difficult to avoid).

fixes https://git.lix.systems/lix-project/lix/issues/424

Change-Id: Iee793c79ba5a485d6606e0d292ed2eae6dfb7216
2024-06-26 17:44:04 +00:00
jadeandGerrit Code Review 9afb0fe41c Merge changes I476a2516,I8a274227 into main
* changes:
  doc/hacking: fix internal api docs section to say to enable it
  doc: Add more about the release note generator
2024-06-26 17:34:45 +00:00
alois31andGerrit Code Review 30da1b17d9 Merge "libmain/progress-bar: move implementation out of the header" into main 2024-06-26 16:05:44 +00:00
Jade Lovelace f7d54cb6b1 packaging: make pegtl use the __forDefaults mechanism
This avoids needing to pass it in when callPackage'ing Lix from external
code.

Change-Id: Ie07e84a151e38614064609a2f6dbff165e193be7
2024-06-26 00:44:46 -07:00
Jade Lovelace 85c1241201 doc/hacking: fix up some outdated info about cross, hydra links
We would like to build these with Hydra but we do not currently have a
Hydra to build them with conveniently.

Change-Id: I0832a33881138dd1caab3805df7ad097db347e62
2024-06-25 21:46:26 -07:00
Jade Lovelace 33d53c4983 doc/hacking: fix internal api docs section to say to enable it
I filed a bug to build these in releng in the future:
https://git.lix.systems/lix-project/lix/issues/422

Change-Id: I476a2516cc2be382d4b7c8529a02f9212a78fdb2
2024-06-25 21:26:18 -07:00
Jade Lovelace e537678f1e doc: Add more about the release note generator
Change-Id: I8a274227cb1b05d442d3f644603dd2844ecc9d05
2024-06-25 21:22:37 -07:00
jadeandGerrit Code Review 4ac2c496d4 Merge "change shebangs of all .sh scripts to bash" into main 2024-06-25 22:18:26 +00:00
jadeandGerrit Code Review aceef13682 Merge changes If0ddec6b,Iaa63ed18 into main
* changes:
  Add some release notes for things we did
  packaging: don't build internal api docs by default in dev shells
2024-06-25 22:16:04 +00:00
lunaphiedandGerrit Code Review f170870ae7 Merge ".envrc: remove MAKEFLAGS and use clang environment by default" into main 2024-06-25 20:42:46 +00:00
Lunaphied 97c86908a4 .envrc: remove MAKEFLAGS and use clang environment by default
MAKEFLAGS hasn't been relevant since we switched off the Make
buildsystem and using the clang environment by default gives you clangd
by default which most developers will want.

Change-Id: I9c11d0613577047e6c908f049c1ffaca5fb5ff67
2024-06-25 12:36:18 -06:00
eldritch horrors e6cd67591b libexpr: rewrite the parser with pegtl instead of flex/bison
this gives about 20% performance improvements on pure parsing. obviously
it will be less on full eval, but depending on how much parsing is to be
done (e.g. including hackage-packages.nix or not) it's more like 4%-10%.

this has been tested (with thousands of core hours of fuzzing) to ensure
that the ASTs produced by the new parser are exactly the same as the old
one would have produced. error messages will change (sometimes by a lot)
and are not yet perfect, but we would rather leave this as is for later.

test results for running only the parser (excluding the variable binding
code) in a tight loop with inputs and parameters as given are promising:

  - 40% faster on lix's package.nix at 10000 iterations
  - 1.3% faster on nixpkgs all-packages.nix at 1000 iterations
  - equivalent on all of nixpkgs concatenated at 100 iterations
    (excluding invalid files, each file surrounded with parens)

more realistic benchmarks are somewhere in between the extremes, parsing
once again getting the largest uplift. other realistic workloads improve
by a few percentage points as well, notably system builds are 4% faster.

Benchmarks summary (from ./bench/summarize.jq bench/bench-*.json)
old/bin/nix --extra-experimental-features 'nix-command flakes' eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  mean:     0.408s ± 0.025s
            user: 0.355s | system: 0.033s
  median:   0.389s
  range:    0.388s ... 0.442s
  relative: 1

new/bin/nix --extra-experimental-features 'nix-command flakes' eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  mean:     0.332s ± 0.024s
            user: 0.279s | system: 0.033s
  median:   0.314s
  range:    0.313s ... 0.361s
  relative: 0.814

---

old/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  mean:     6.133s ± 0.022s
            user: 5.395s | system: 0.437s
  median:   6.128s
  range:    6.099s ... 6.183s
  relative: 1

new/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  mean:     5.925s ± 0.025s
            user: 5.176s | system: 0.456s
  median:   5.934s
  range:    5.861s ... 5.943s
  relative: 0.966

---

GC_INITIAL_HEAP_SIZE=10g old/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  mean:     4.503s ± 0.027s
            user: 3.731s | system: 0.547s
  median:   4.499s
  range:    4.478s ... 4.541s
  relative: 1

GC_INITIAL_HEAP_SIZE=10g new/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  mean:     4.285s ± 0.031s
            user: 3.504s | system: 0.571s
  median:   4.281s
  range:    4.221s ... 4.328s
  relative: 0.951

---

old/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello
  mean:     16.475s ± 0.07s
            user: 14.088s | system: 1.572s
  median:   16.495s
  range:    16.351s ... 16.536s
  relative: 1

new/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello
  mean:     15.973s ± 0.013s
            user: 13.558s | system: 1.615s
  median:   15.973s
  range:    15.946s ... 15.99s
  relative: 0.97

---

Change-Id: Ie66ec2d045dec964632c6541e25f8f0797319ee2
2024-06-25 12:24:58 +00:00
jadeandGerrit Code Review c097ebe66b Merge "Revert "libfetchers: make attribute / URL query handling consistent"" into main 2024-06-25 10:19:52 +00:00
Jade Lovelace e19f27917c Add some release notes for things we did
Change-Id: If0ddec6b64a43c3d8f6cae39e0292863f3b49401
2024-06-24 16:26:12 -07:00
Jade Lovelace 1245340e44 packaging: don't build internal api docs by default in dev shells
These are totally available and you can just turn them on, but they have
very bad dependency tracking and thus bloat incremental change times,
which is not really ok.

Change-Id: Iaa63ed18a789e74fcb757248cd24c3b194afcc80
2024-06-24 15:57:38 -07:00
jadeandGerrit Code Review 3e151d4d77 Revert "libfetchers: make attribute / URL query handling consistent"
This reverts commit 35eec921af.

Reason for revert: Regressed nix-eval-jobs, and it appears to be this change is buggy/missing a case. It just needs another pass.

Code causing the problem in n-e-j, when invoked with `nix-eval-jobs --flake '.#hydraJobs'`:

```
n-e-j/tests/assets » ../../build/src/nix-eval-jobs --meta --workers 1 --flake .#hydraJobs
warning: unknown setting 'trusted-users'
warning: `--gc-roots-dir' not specified
error: unsupported Git input attribute 'dir'
error: worker error: error: unsupported Git input attribute 'dir'
```

```
  nix::Value *vRoot = [&]() {
        if (args.flake) {
            auto [flakeRef, fragment, outputSpec] =
                nix::parseFlakeRefWithFragmentAndExtendedOutputsSpec(
                    args.releaseExpr, nix::absPath("."));
            nix::InstallableFlake flake{
                {}, state, std::move(flakeRef), fragment, outputSpec,
                {}, {},    args.lockFlags};

            return flake.toValue(*state).first;
        } else {
            return releaseExprTopLevelValue(*state, autoArgs, args);
        }
    }();
```

Inspecting the program behaviour reveals that `dir` was in fact set in the URL going into the fetcher. This is in turn because unlike in the case changed in this commit, it was not erased before handing it to libfetchers, which is probably just a mistake.

```
(rr) up
3  0x00007ffff60262ae in nix::fetchers::Input::fromURL (url=..., requireTree=requireTree@entry=true) at src/libfetchers/fetchers.cc:39
warning: Source file is more recent than executable.
39              auto res = inputScheme->inputFromURL(url, requireTree);
(rr) p url
$1 = (const nix::ParsedURL &) @0x7fffdc874190: {url = "git+file:///home/jade/lix/nix-eval-jobs", 
  base = "git+file:///home/jade/lix/nix-eval-jobs", scheme = "git+file", authority = std::optional<std::string> = {[contained value] = ""}, 
  path = "/home/jade/lix/nix-eval-jobs", query = std::map with 1 element = {["dir"] = "tests/assets"}, fragment = ""}
(rr) up
4  0x00007ffff789d904 in nix::parseFlakeRefWithFragment (url=".#hydraJobs", baseDir=std::optional<std::string> = {...}, 
    allowMissing=allowMissing@entry=false, isFlake=isFlake@entry=true) at src/libexpr/flake/flakeref.cc:179
warning: Source file is more recent than executable.
179                                 FlakeRef(Input::fromURL(parsedURL, isFlake), getOr(parsedURL.query, "dir", "")),
(rr) p parsedURL
$2 = {url = "git+file:///home/jade/lix/nix-eval-jobs", base = "git+file:///home/jade/lix/nix-eval-jobs", scheme = "git+file", 
  authority = std::optional<std::string> = {[contained value] = ""}, path = "/home/jade/lix/nix-eval-jobs", query = std::map with 1 element = {
    ["dir"] = "tests/assets"}, fragment = ""}
(rr) list
174
175                             if (pathExists(flakeRoot + "/.git/shallow"))
176                                 parsedURL.query.insert_or_assign("shallow", "1");
177
178                             return std::make_pair(
179                                 FlakeRef(Input::fromURL(parsedURL, isFlake), getOr(parsedURL.query, "dir", "")),
180                                 fragment);
181                         }
```

Change-Id: Ib55a882eaeb3e59228857761dc1e3b2e366b0f5e
2024-06-24 22:49:17 +00:00
vigress8andJade Lovelace c7af89c797 change shebangs of all .sh scripts to bash
On operating systems where /bin/sh is not Bash, some scripts are invalid
because of bashisms, and building Lix fails with errors like this:
`render-manpage.sh: 3: set: Illegal option -o pipefail`
This modifies all scripts that use a `/bin/sh` shebang to `/usr/bin/env
bash`, including currently POSIX-compliant ones, to prevent any future
confusion.

Change-Id: Ia074cc6db42d40fc59a63726f6194ea0149ea5e0
2024-06-24 14:00:43 -07:00
Jade Lovelace d5637ee790 devShell: guard against running from another directory
I was working on nix-eval-jobs with a dev shell with some shenanigans to
run against a locally built Lix and it was getting really annoying when
`nix develop ../lix#` was messing up my other git repo's hooks.

This is a fix via blunt force, but it is at least obvious how it works.

Change-Id: Ia29eeb5be57ab6a2c88451c00ea18a51e4dfe65e
2024-06-24 13:41:38 -07:00
Jade Lovelace eb5de71adc justfile: accept extra options to just setup and pass them to meson
This lets you get the default options and still be able to add more.

Change-Id: Ife32c348b1498ff2ccdddf051a5bba520cfa36f0
2024-06-24 13:37:01 -07:00
Alois Wohlschlager 206a5dbb8f libmain/progress-bar: move implementation out of the header
Change-Id: Ib4b42ebea290ee575294df6b2f17a38a5d850b80
2024-06-24 20:39:50 +02:00
Robert HensingandK900 d86009bd76 Add build-dir setting, clean up default TMPDIR handling
This is a squash of upstream PRs #10303, #10312 and #10883.

fix: Treat empty TMPDIR as unset

Fixes an instance of

    nix: src/libutil/util.cc:139: nix::Path nix::canonPath(PathView, bool): Assertion `path != ""' failed.

... which I've been getting in one of my shells for some reason.
I have yet to find out why TMPDIR was empty, but it's no reason for
Nix to break.

(cherry picked from commit c3fb2aa1f9d1fa756dac38d3588c836c5a5395dc)

fix: Treat empty XDG_RUNTIME_DIR as unset

See preceding commit. Not observed in the wild, but is sensible
and consistent with TMPDIR behavior.

(cherry picked from commit b9e7f5aa2df3f0e223f5c44b8089cbf9b81be691)

local-derivation-goal.cc: Reuse defaultTempDir()

(cherry picked from commit fd31945742710984de22805ee8d97fbd83c3f8eb)

fix: remove usage of XDG_RUNTIME_DIR for TMP

(cherry picked from commit 1363f51bcb24ab9948b7b5093490a009947f7453)

tests/functional: Add count()

(cherry picked from commit 6221770c9de4d28137206bdcd1a67eea12e1e499)

Remove uncalled for message

(cherry picked from commit b1fe388d33530f0157dcf9f461348b61eda13228)

Add build-dir setting

(cherry picked from commit 8b16cced18925aa612049d08d5e78eccbf0530e4)
Change-Id: Ic7b75ff0b6a3b19e50a4ac8ff2d70f15c683c16a
2024-06-24 11:30:32 +03:00
Delan AzabaniandGerrit Code Review 4c3d93611f Merge "Fix build instructions in hacking.md and justfile" into main 2024-06-24 06:46:08 +00:00
Delan AzabaniandGerrit Code Review 37f276ba9d Merge "Fix compile error under gcc with -Denable-pch-std=false" into main 2024-06-24 06:45:40 +00:00
Delan Azabani 5d33e4cd59 Fix build instructions in hacking.md and justfile
The stdenv phases don’t actually do anything (at least not anymore),
and our justfile doesn’t behave the same as our docs.

This patch removes the stdenv phases from the docs, documents our
usage of just, and makes `just setup` heed `$mesonFlags`.

Fixes #413.
Fixes #414.

Change-Id: Ieb0b2a8ae420526238b5f9a73d7849ec6919995d
2024-06-24 12:55:00 +08:00
Delan Azabani 3baffbdcc5 Fix compile error under gcc with -Denable-pch-std=false
Following the latest hacking.md currently fails because of a missing
include in upstream editline. This patch fixes the build by adding
that missing include.

Fixes #410.

Change-Id: Iefd4cb687ed3da71ccda9fe9624f38e6ef4623e5
2024-06-24 12:47:04 +08:00
eldritch horrors d477b34d1d libutil: remove runProgram2 stdin functionality
this was only used in one place, and that place has been rewritten to
use a temporary file instead. keeping this around is not very helpful
at this time, and in any case we'd be better off rewriting subprocess
handling in rust where we not only have a much safer library for such
things but also async frameworks necessary for this easily available.

Change-Id: I6f8641b756857c84ae2602cdf41f74ee7a1fda02
2024-06-23 17:29:40 +00:00
eldritch horrors 2bbdaf0b19 libfetchers: write git commit message to tempfile
we want to remove runProgram's ability to provide stdin to a process
because the concurrency issues of handling both stdin and stdout are
much more pronounced once runProgram returns not is collected output
but a source. this is possible in the current c++ framework, however
it isn't necessary in almost all cases (as demonstrated by only this
single user existing) and in much better handled with a proper async
concurrency model that lets the caller handle both at the same time.

Change-Id: I29da1e1ad898d45e2e33a7320b246d5003e7712b
2024-06-23 17:29:40 +00:00
eldritch horrors 2fe9157808 flakes: add --commit-lock-file message test
we had no test to ensure that we generated a commit message at all?

Change-Id: Ic9aa8fde92b83e1ea6f61cd2a21867aa73d4e885
2024-06-23 17:29:40 +00:00
QyriadandGerrit Code Review 521e08cbde Merge changes I697f4f39,I9f25235d into main
* changes:
  doc-comment Fields for Activity and Result types
  mildly refactor the renderActivity if hell-chain
2024-06-23 16:55:36 +00:00
Maximilian BoschandGerrit Code Review 7c3b8229cc Merge "libstore: reduce loglevel of waiting for a machine to build" into main 2024-06-23 16:33:44 +00:00
Qyriad 701eb332bd doc-comment Fields for Activity and Result types
The ones we were able to figure out, at least.

Change-Id: I697f4f3942e35a7adf1b2a6cc28b3168d1de111c
2024-06-23 16:30:48 +00:00
Qyriad ce4dee0fa5 mildly refactor the renderActivity if hell-chain
This is primarily for readability, but iwrc chaining std::string's
operator+ is also pretty scuffed performance-wise, and this was doing a
lot of operator+ chainging.

Change-Id: I9f25235df153eb2bbb491f1ff7ebbeed9a8ec985
2024-06-23 16:30:38 +00:00
Maximilian BoschandGerrit Code Review 5f0062285c Merge "libfetchers: make attribute / URL query handling consistent" into main 2024-06-23 15:51:34 +00:00
eldritch horrors ce6cb14995 libutil: return Pid from startProcess, not pid_t
Change-Id: Icc8a15090c77f54ea7d9220aadedcd4a19922814
2024-06-23 11:52:49 +00:00
eldritch horrors 3d155fc509 libutil: give Pid proper resource semantics
copy-constructing or assigning from pid_t can easily lead to duplicate
Pid instances for the same process if a pid_t was used carelessly, and
Pid itself was copy-constructible. both could cause surprising results
such as killing processes twice (which could become very problemantic,
but luckily modern systems don't reuse PIDs all that quickly), or more
than one piece of the code believing it owns a process when neither do

Change-Id: Ifea7445f84200b34c1a1d0acc2cdffe0f01e20c6
2024-06-23 11:52:49 +00:00
eldritch horrors b43a2e84c4 libutil: make Pid -> pid_t operations explicit
Change-Id: I3137cc140590001fe7ba542844e735944a0a9255
2024-06-23 11:52:49 +00:00
eldritch horrors a9949f4760 libutil: add some serialize.hh serializer tests
Change-Id: I0116265a18bc44bba16c07bf419af70d5195f07d
2024-06-23 11:52:49 +00:00
eldritch horrors 39a1e248c9 libutil: remove sinkToSource eof callback
this is only used in one place, and only to set a nicer error message on
EndOfFile. the only caller that actually *catches* this exception should
provide an error message in that catch block rather than forcing support
for setting error message so deep into the stack. copyStorePath is never
called outside of PathSubstitutionGoal anyway, which catches everything.

Change-Id: Ifbae8706d781c388737706faf4c8a8b7917ca278
2024-06-23 11:52:49 +00:00
Artemis TosiniandGerrit Code Review f80d95e36d Merge "libstore: Start creating LocalDerivationGoal subclasses" into main 2024-06-23 08:11:51 +00:00
Artemis Tosiniandjade 12f5d27363 libstore: Start creating LocalDerivationGoal subclasses
LocalDerivationGoal includes a large number of low-level sandboxing
primitives for Darwin and Linux, intermingled with ifdefs.
Start creating platform-specific classes to make it easier to add new
platforms and review platform-specific code.

This change only creates support infrastructure and moves two function,
more functions will be moved in future changes.

Change-Id: I9fc29fa2a7345107d4fc96c46fa90b4eabf6bb89
2024-06-23 03:33:07 +00:00
Qyriad fb8553f63c mildly cleanup ExprSelect::eval
Better variable names, some comments, and a slight logic rearrange.

Change-Id: I9685ae252f83217aa85f06432234159c9ad19d1c
2024-06-22 18:52:57 -06:00
Qyriad e09cc60df9 doc-comment ExprSelect's fields
Change-Id: I63e79991a4bab93421266785e9258e0f5bb89b8f
2024-06-22 18:52:57 -06:00
Maximilian Bosch fc6a1451af libstore: reduce loglevel of waiting for a machine to build
This comes quite often when the available job slots on all remote
builders are exhausted and this is pretty spammy.

This isn't really an issue, but expected behavior.

A better way to display this is a nom-like approach where all scheduled
builds are shown in a tree and pending builds are being marked as such
IMHO.

Change-Id: I6bc14e6054f84e3eb0768127b490e263d8cdcf89
2024-06-22 17:38:25 +02:00
Maximilian Bosch 35eec921af libfetchers: make attribute / URL query handling consistent
The original idea was to fix lix#174, but for a user friendly solution,
I figured that we'd need more consistency:

* Invalid query params will cause an error, just like invalid
  attributes. This has the following two consequences:

  * The `?dir=`-param from flakes will be removed before the URL to be
    fetched is passed to libfetchers.

  * The tarball fetcher doesn't allow URLs with custom query params
    anymore. I think this was questionable anyways given that an
    arbitrary set of query params was silently removed from the URL you
    wanted to fetch. The correct way is to use an attribute-set
    with a key `url` that contains the tarball URL to fetch.

  * Same for the git & mercurial fetchers: in that case it doesn't even
    matter though: both fetchers added unused query params to the URL
    that's passed from the input scheme to the fetcher (`url2` in the code).
    It turns out that this was never used since the query parameters were
    erased again in `getActualUrl`.

* Validation happens for both attributes and URLs. Previously, a lot of
  fetchers validated e.g. refs/revs only when specified in a URL and
  the validity of attribute names only in `inputFromAttrs`.

  Now, all the validation is done in `inputFromAttrs` and `inputFromURL`
  constructs attributes that will be passed to `inputFromAttrs`.

* Accept all attributes as URL query parameters. That also includes
  lesser used ones such as `narHash`.

  And "output" attributes like `lastModified`: these could be declared
  already when declaring inputs as attribute rather than URL. Now the
  behavior is at least consistent.

  Personally, I think we should differentiate in the future between
  "fetched input" (basically the attr-set that ends up in the lock-file)
  and "unfetched input" earlier: both inputFrom{Attrs,URL} entrypoints
  are probably OK for unfetched inputs, but for locked/fetched inputs
  a custom entrypoint should be used. Then, the current entrypoints
  wouldn't have to allow these attributes anymore.

Change-Id: I1be1992249f7af8287cfc37891ab505ddaa2e8cd
2024-06-22 14:42:43 +02:00
Finn Behrens da4e46dd1f libmain: add progress bar with multiple status lines
Add the log-formats `multiline` and `multiline-with-logs` which offer
multiple current active building status lines.

Change-Id: Idd8afe62f8591b5d8b70e258c5cefa09be4cab03
2024-06-22 14:20:27 +02:00
QyriadandGerrit Code Review 21865ccce0 Merge "add a basic libmain test for the progress bar rendering" into main 2024-06-22 02:04:14 +00:00
jadeandGerrit Code Review 375f4c0337 Merge "libstore: remove operations that are never called by supported clients" into main 2024-06-21 20:46:18 +00:00
Jade Lovelace 50472aa5be libstore: remove operations that are never called by supported clients
I did a whole bunch of `git log -S` to find out exactly when all these
things were obsoleted and found the commit in which their usage was
removed, which I have added in either the error message or a comment.

I've also made *some* of the version checks into static asserts for when
we update the minimum supported protocol version.

In the end this is not a lot of code we are deleting, but it's code that
we will never have to support into the future when we build a protocol
bridge, which is why I did it. It is not in the support baseline.

Change-Id: Iea3c80795c75ea74f328cf7ede7cbedf8c41926b
2024-06-19 19:41:04 -07:00
1791 changed files with 154307 additions and 1 deletions
+50
View File
@@ -0,0 +1,50 @@
---
BasedOnStyle: LLVM
AccessModifierOffset: -4
AlignAfterOpenBracket: BlockIndent
AlignEscapedNewlines: Left
AlignOperands: DontAlign
AllowShortBlocksOnASingleLine: Always
AllowShortFunctionsOnASingleLine: Empty
AllowShortIfStatementsOnASingleLine: WithoutElse
AlwaysBreakBeforeMultilineStrings: true
AlwaysBreakTemplateDeclarations: Yes
BinPackArguments: false
BinPackParameters: false
BitFieldColonSpacing: None
BraceWrapping:
AfterCaseLabel: false
AfterClass: true
AfterControlStatement: MultiLine
AfterEnum: false
AfterFunction: true
AfterNamespace: false
AfterObjCDeclaration: false
AfterStruct: true
AfterUnion: true
AfterExternBlock: false
BeforeCatch: false
BeforeElse: false
BeforeLambdaBody: false
BeforeWhile: false
IndentBraces: false
SplitEmptyFunction: true
SplitEmptyRecord: false
SplitEmptyNamespace: true
BreakAfterAttributes: Always
BreakBeforeBinaryOperators: NonAssignment
BreakBeforeBraces: Custom
BreakConstructorInitializers: BeforeComma
ColumnLimit: 100
EmptyLineAfterAccessModifier: Leave
EmptyLineBeforeAccessModifier: Leave
FixNamespaceComments: false
IndentWidth: 4
InsertBraces: true
InsertTrailingCommas: Wrapped
LambdaBodyIndentation: Signature
PackConstructorInitializers: CurrentLine
PointerAlignment: Middle
SortIncludes: Never
SpaceAfterCStyleCast: true
SpaceAfterTemplateKeyword: false
+18
View File
@@ -0,0 +1,18 @@
UseColor: true
Checks:
- -*
- bugprone-*
# too many warnings
- -bugprone-assignment-in-if-condition
# too many warnings
- -bugprone-narrowing-conversions
# kind of nonsense
- -bugprone-easily-swappable-parameters
# too many warnings for now
- -bugprone-implicit-widening-of-multiplication-result
# Lix's exception handling is Questionable
- -bugprone-empty-catch
# many warnings
- -bugprone-unchecked-optional-access
# many warnings, seems like a questionable lint
- -bugprone-branch-clone
+18
View File
@@ -0,0 +1,18 @@
((c++-mode . (
(c-file-style . "k&r")
(c-basic-offset . 4)
(c-block-comment-prefix . " ")
(indent-tabs-mode . nil)
(tab-width . 4)
(show-trailing-whitespace . t)
(indicate-empty-lines . t)
(eval . (c-set-offset 'innamespace 0))
(eval . (c-set-offset 'defun-open 0))
(eval . (c-set-offset 'inline-open 0))
(eval . (c-set-offset 'arglist-intro '+))
(eval . (c-set-offset 'arglist-cont 0))
(eval . (c-set-offset 'arglist-cont-nonempty '+))
(eval . (c-set-offset 'substatement-open 0))
(eval . (c-set-offset 'access-label '-))
(eval . (c-set-offset 'inlambda 0))
)))
+31
View File
@@ -0,0 +1,31 @@
# EditorConfig configuration for nix
# http://EditorConfig.org
# Top-most EditorConfig file
root = true
# Unix-style newlines with a newline ending every file, utf-8 charset
[*]
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
charset = utf-8
# Match nix files, set indent to spaces with width of two
[*.nix]
indent_style = space
indent_size = 2
# Match c++/shell/perl, set indent to spaces with width of four
[*.{hpp,cc,hh,sh,pl}]
indent_style = space
indent_size = 4
# Match diffs, avoid to trim trailing whitespace
[*.{diff,patch}]
trim_trailing_whitespace = false
[*.md]
indent_style = space
indent_size = 2
max_line_length = 0
+5
View File
@@ -0,0 +1,5 @@
# shellcheck shell=bash
source_env_if_exists .envrc.local
# Use native-clangStdenvPackages to get clangd by default.
use flake ".#${LIX_SHELL_VARIANT:-native-clangStdenvPackages}" "${LIX_SHELL_EXTRA_ARGS[@]}"
export GTEST_BRIEF=1
+32
View File
@@ -0,0 +1,32 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: bug
assignees: ''
---
## Describe the bug
A clear and concise description of what the bug is.
If you have a problem with a specific package or NixOS,
you probably want to file an issue at https://github.com/NixOS/nixpkgs/issues.
## Steps To Reproduce
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
## Expected behavior
A clear and concise description of what you expected to happen.
## `nix --version` output
## Additional context
Add any other context about the problem here.
+24
View File
@@ -0,0 +1,24 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: feature
assignees: ''
---
## Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
## Describe the solution you'd like
A clear and concise description of what you want to happen.
## Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
## Additional context
Add any other context or screenshots about the feature request here.
+32
View File
@@ -0,0 +1,32 @@
---
name: Installer issue
about: Report problems with installation
title: ''
labels: installer
assignees: ''
---
## Platform
<!-- select the platform on which you tried to install Lix -->
- [ ] Linux: <!-- state your distribution, e.g. Arch Linux, Ubuntu, ... -->
- [ ] macOS
- [ ] WSL
## Additional information
<!-- state special circumstances on your system or additional steps you have taken prior to installation -->
## Output
<details><summary>Output</summary>
```log
<!-- paste console output here and remove this comment -->
```
</details>
@@ -0,0 +1,28 @@
---
name: Missing or incorrect documentation
about: Help us improve the reference manual
title: ''
labels: documentation
assignees: ''
---
## Problem
<!-- describe your problem -->
## Proposal
<!-- propose a solution -->
## Checklist
<!-- make sure this issue is not redundant or obsolete -->
- [ ] checked [latest Lix manual] \([source]\)
- [ ] checked [documentation issues] and [recent documentation changes] for possible duplicates
[latest Nix manual]: https://docs.lix.systems/manual/lix/nightly
[source]: https://git.lix.systems/lix-project/lix/src/main/doc/manual/src
[documentation issues]: https://git.lix.systems/lix-project/lix/issues?labels=151&state=all
[recent documentation changes]: https://gerrit.lix.systems/q/p:lix+path:%22%5Edoc/manual/.*%22
+11
View File
@@ -0,0 +1,11 @@
# Motivation
<!-- Briefly explain what the change is about and why it is desirable. -->
# Context
<!-- Provide context. Reference open issues if available. -->
<!-- Non-trivial change: Briefly outline the implementation strategy. -->
<!-- Invasive change: Discuss alternative designs or approaches you considered. -->
<!-- Large change: Provide instructions to reviewers how to read the diff. -->
+31
View File
@@ -0,0 +1,31 @@
outputs/
# GNU Global
GPATH
GRTAGS
GSYMS
GTAGS
# ccls
/.ccls-cache
result
result-*
.vscode/
.direnv/
# clangd and possibly more
.cache/
# Mac OS
.DS_Store
# ClangBuildAnalyzer output, see maintainers/buildtime_report.sh
buildtime.bin
.envrc.local
# We generate this with a Nix shell hook
/.pre-commit-config.yaml
/.nocontribmsg
/release
+1
View File
@@ -0,0 +1 @@
This is a file used by the dev shell shellHook in package.nix to check that this is actually a Lix repo before installing git hooks. Its contents have no meaning.
+53
View File
@@ -0,0 +1,53 @@
# Contributing to Lix
Welcome and thank you for considering contributing to Lix! We're currently in a soft release phase, and your support means a lot to us.
To ensure a smooth and effective contribution process, here is a summary of our guidelines:
## Getting help?
If you have any question regarding getting started or reporting bugs, feel free
to reach out to us.
On Matrix, we have a space at `#space:lix.systems`, composed of:
- [`#discuss:lix.systems`](https://matrix.to/#/#discuss:lix.systems) for discussions on Lix.
- [`#dev:lix.systems`](https://matrix.to/#/#dev:lix.systems) for the development channel on Lix.
## Report a bug
- Check if your bug has already been reported in the [issue tracker](https://git.lix.systems/lix-project/lix/issues).
- If you can't find the bug or feature, please open a new issue.
We maintain a copy of the upstream Nix bugs. Their organisation can be read about [here](https://wiki.lix.systems/books/lix-contributors/page/bug-tracker-organisation).
## Report a security vulnerability
For security vulnerabilities, reach out by email at `security at lix dot systems`.
## Making changes to Lix
Before diving into making changes, we want to engage with you and your ideas.
We have a few policies in effect; please take the time to familiarize yourself:
- [Style guide on code](https://wiki.lix.systems/books/lix-contributors/page/code)
- [Freeze policy and recommended contributions](https://wiki.lix.systems/books/lix-contributors/page/freezes-and-recommended-contributions)
To avoid duplication of effort, it may be a good idea to check out the list of
[pending pull requests](https://gerrit.lix.systems/q/status:open+-is:wip) (or "change lists", as Gerrit calls them). Once you have
an idea of what you might want to do, we recommend dropping a message on our
Matrix to ensure your contribution fits with our current schedule and plans
When you're ready and your changes are ready to go:
- Submit your code.
- Submitting a GitHub PR [on our mirror](https://github.com/lix-project/lix) is totally ok if that's easier for you and your change is relatively small (300 lines or so).
We may ask you to resubmit it as a Gerrit CL if it is necessary for the change you're making.
- Our primary code review system is [our Gerrit instance](https://gerrit.lix.systems), where you can open a change list (CL).
If you're new to Gerrit, check out [our wiki page about Gerrit](https://wiki.lix.systems/books/lix-contributors/page/gerrit).
- Make sure to link any related issues.
- If needed, indicate that the change is 'work in progress'.
You can obtain an account on our platforms by clicking "Sign In with GitHub" on the sign-in page.
+502
View File
@@ -0,0 +1,502 @@
GNU LESSER GENERAL PUBLIC LICENSE
Version 2.1, February 1999
Copyright (C) 1991, 1999 Free Software Foundation, Inc.
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
[This is the first released version of the Lesser GPL. It also counts
as the successor of the GNU Library Public License, version 2, hence
the version number 2.1.]
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
Licenses are intended to guarantee your freedom to share and change
free software--to make sure the software is free for all its users.
This license, the Lesser General Public License, applies to some
specially designated software packages--typically libraries--of the
Free Software Foundation and other authors who decide to use it. You
can use it too, but we suggest you first think carefully about whether
this license or the ordinary General Public License is the better
strategy to use in any particular case, based on the explanations below.
When we speak of free software, we are referring to freedom of use,
not price. Our General Public Licenses are designed to make sure that
you have the freedom to distribute copies of free software (and charge
for this service if you wish); that you receive source code or can get
it if you want it; that you can change the software and use pieces of
it in new free programs; and that you are informed that you can do
these things.
To protect your rights, we need to make restrictions that forbid
distributors to deny you these rights or to ask you to surrender these
rights. These restrictions translate to certain responsibilities for
you if you distribute copies of the library or if you modify it.
For example, if you distribute copies of the library, whether gratis
or for a fee, you must give the recipients all the rights that we gave
you. You must make sure that they, too, receive or can get the source
code. If you link other code with the library, you must provide
complete object files to the recipients, so that they can relink them
with the library after making changes to the library and recompiling
it. And you must show them these terms so they know their rights.
We protect your rights with a two-step method: (1) we copyright the
library, and (2) we offer you this license, which gives you legal
permission to copy, distribute and/or modify the library.
To protect each distributor, we want to make it very clear that
there is no warranty for the free library. Also, if the library is
modified by someone else and passed on, the recipients should know
that what they have is not the original version, so that the original
author's reputation will not be affected by problems that might be
introduced by others.
Finally, software patents pose a constant threat to the existence of
any free program. We wish to make sure that a company cannot
effectively restrict the users of a free program by obtaining a
restrictive license from a patent holder. Therefore, we insist that
any patent license obtained for a version of the library must be
consistent with the full freedom of use specified in this license.
Most GNU software, including some libraries, is covered by the
ordinary GNU General Public License. This license, the GNU Lesser
General Public License, applies to certain designated libraries, and
is quite different from the ordinary General Public License. We use
this license for certain libraries in order to permit linking those
libraries into non-free programs.
When a program is linked with a library, whether statically or using
a shared library, the combination of the two is legally speaking a
combined work, a derivative of the original library. The ordinary
General Public License therefore permits such linking only if the
entire combination fits its criteria of freedom. The Lesser General
Public License permits more lax criteria for linking other code with
the library.
We call this license the "Lesser" General Public License because it
does Less to protect the user's freedom than the ordinary General
Public License. It also provides other free software developers Less
of an advantage over competing non-free programs. These disadvantages
are the reason we use the ordinary General Public License for many
libraries. However, the Lesser license provides advantages in certain
special circumstances.
For example, on rare occasions, there may be a special need to
encourage the widest possible use of a certain library, so that it becomes
a de-facto standard. To achieve this, non-free programs must be
allowed to use the library. A more frequent case is that a free
library does the same job as widely used non-free libraries. In this
case, there is little to gain by limiting the free library to free
software only, so we use the Lesser General Public License.
In other cases, permission to use a particular library in non-free
programs enables a greater number of people to use a large body of
free software. For example, permission to use the GNU C Library in
non-free programs enables many more people to use the whole GNU
operating system, as well as its variant, the GNU/Linux operating
system.
Although the Lesser General Public License is Less protective of the
users' freedom, it does ensure that the user of a program that is
linked with the Library has the freedom and the wherewithal to run
that program using a modified version of the Library.
The precise terms and conditions for copying, distribution and
modification follow. Pay close attention to the difference between a
"work based on the library" and a "work that uses the library". The
former contains code derived from the library, whereas the latter must
be combined with the library in order to run.
GNU LESSER GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License Agreement applies to any software library or other
program which contains a notice placed by the copyright holder or
other authorized party saying it may be distributed under the terms of
this Lesser General Public License (also called "this License").
Each licensee is addressed as "you".
A "library" means a collection of software functions and/or data
prepared so as to be conveniently linked with application programs
(which use some of those functions and data) to form executables.
The "Library", below, refers to any such software library or work
which has been distributed under these terms. A "work based on the
Library" means either the Library or any derivative work under
copyright law: that is to say, a work containing the Library or a
portion of it, either verbatim or with modifications and/or translated
straightforwardly into another language. (Hereinafter, translation is
included without limitation in the term "modification".)
"Source code" for a work means the preferred form of the work for
making modifications to it. For a library, complete source code means
all the source code for all modules it contains, plus any associated
interface definition files, plus the scripts used to control compilation
and installation of the library.
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running a program using the Library is not restricted, and output from
such a program is covered only if its contents constitute a work based
on the Library (independent of the use of the Library in a tool for
writing it). Whether that is true depends on what the Library does
and what the program that uses the Library does.
1. You may copy and distribute verbatim copies of the Library's
complete source code as you receive it, in any medium, provided that
you conspicuously and appropriately publish on each copy an
appropriate copyright notice and disclaimer of warranty; keep intact
all the notices that refer to this License and to the absence of any
warranty; and distribute a copy of this License along with the
Library.
You may charge a fee for the physical act of transferring a copy,
and you may at your option offer warranty protection in exchange for a
fee.
2. You may modify your copy or copies of the Library or any portion
of it, thus forming a work based on the Library, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) The modified work must itself be a software library.
b) You must cause the files modified to carry prominent notices
stating that you changed the files and the date of any change.
c) You must cause the whole of the work to be licensed at no
charge to all third parties under the terms of this License.
d) If a facility in the modified Library refers to a function or a
table of data to be supplied by an application program that uses
the facility, other than as an argument passed when the facility
is invoked, then you must make a good faith effort to ensure that,
in the event an application does not supply such function or
table, the facility still operates, and performs whatever part of
its purpose remains meaningful.
(For example, a function in a library to compute square roots has
a purpose that is entirely well-defined independent of the
application. Therefore, Subsection 2d requires that any
application-supplied function or table used by this function must
be optional: if the application does not supply it, the square
root function must still compute square roots.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Library,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Library, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote
it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Library.
In addition, mere aggregation of another work not based on the Library
with the Library (or with a work based on the Library) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may opt to apply the terms of the ordinary GNU General Public
License instead of this License to a given copy of the Library. To do
this, you must alter all the notices that refer to this License, so
that they refer to the ordinary GNU General Public License, version 2,
instead of to this License. (If a newer version than version 2 of the
ordinary GNU General Public License has appeared, then you can specify
that version instead if you wish.) Do not make any other change in
these notices.
Once this change is made in a given copy, it is irreversible for
that copy, so the ordinary GNU General Public License applies to all
subsequent copies and derivative works made from that copy.
This option is useful when you wish to copy part of the code of
the Library into a program that is not a library.
4. You may copy and distribute the Library (or a portion or
derivative of it, under Section 2) in object code or executable form
under the terms of Sections 1 and 2 above provided that you accompany
it with the complete corresponding machine-readable source code, which
must be distributed under the terms of Sections 1 and 2 above on a
medium customarily used for software interchange.
If distribution of object code is made by offering access to copy
from a designated place, then offering equivalent access to copy the
source code from the same place satisfies the requirement to
distribute the source code, even though third parties are not
compelled to copy the source along with the object code.
5. A program that contains no derivative of any portion of the
Library, but is designed to work with the Library by being compiled or
linked with it, is called a "work that uses the Library". Such a
work, in isolation, is not a derivative work of the Library, and
therefore falls outside the scope of this License.
However, linking a "work that uses the Library" with the Library
creates an executable that is a derivative of the Library (because it
contains portions of the Library), rather than a "work that uses the
library". The executable is therefore covered by this License.
Section 6 states terms for distribution of such executables.
When a "work that uses the Library" uses material from a header file
that is part of the Library, the object code for the work may be a
derivative work of the Library even though the source code is not.
Whether this is true is especially significant if the work can be
linked without the Library, or if the work is itself a library. The
threshold for this to be true is not precisely defined by law.
If such an object file uses only numerical parameters, data
structure layouts and accessors, and small macros and small inline
functions (ten lines or less in length), then the use of the object
file is unrestricted, regardless of whether it is legally a derivative
work. (Executables containing this object code plus portions of the
Library will still fall under Section 6.)
Otherwise, if the work is a derivative of the Library, you may
distribute the object code for the work under the terms of Section 6.
Any executables containing that work also fall under Section 6,
whether or not they are linked directly with the Library itself.
6. As an exception to the Sections above, you may also combine or
link a "work that uses the Library" with the Library to produce a
work containing portions of the Library, and distribute that work
under terms of your choice, provided that the terms permit
modification of the work for the customer's own use and reverse
engineering for debugging such modifications.
You must give prominent notice with each copy of the work that the
Library is used in it and that the Library and its use are covered by
this License. You must supply a copy of this License. If the work
during execution displays copyright notices, you must include the
copyright notice for the Library among them, as well as a reference
directing the user to the copy of this License. Also, you must do one
of these things:
a) Accompany the work with the complete corresponding
machine-readable source code for the Library including whatever
changes were used in the work (which must be distributed under
Sections 1 and 2 above); and, if the work is an executable linked
with the Library, with the complete machine-readable "work that
uses the Library", as object code and/or source code, so that the
user can modify the Library and then relink to produce a modified
executable containing the modified Library. (It is understood
that the user who changes the contents of definitions files in the
Library will not necessarily be able to recompile the application
to use the modified definitions.)
b) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (1) uses at run time a
copy of the library already present on the user's computer system,
rather than copying library functions into the executable, and (2)
will operate properly with a modified version of the library, if
the user installs one, as long as the modified version is
interface-compatible with the version that the work was made with.
c) Accompany the work with a written offer, valid for at
least three years, to give the same user the materials
specified in Subsection 6a, above, for a charge no more
than the cost of performing this distribution.
d) If distribution of the work is made by offering access to copy
from a designated place, offer equivalent access to copy the above
specified materials from the same place.
e) Verify that the user has already received a copy of these
materials or that you have already sent this user a copy.
For an executable, the required form of the "work that uses the
Library" must include any data and utility programs needed for
reproducing the executable from it. However, as a special exception,
the materials to be distributed need not include anything that is
normally distributed (in either source or binary form) with the major
components (compiler, kernel, and so on) of the operating system on
which the executable runs, unless that component itself accompanies
the executable.
It may happen that this requirement contradicts the license
restrictions of other proprietary libraries that do not normally
accompany the operating system. Such a contradiction means you cannot
use both them and the Library together in an executable that you
distribute.
7. You may place library facilities that are a work based on the
Library side-by-side in a single library together with other library
facilities not covered by this License, and distribute such a combined
library, provided that the separate distribution of the work based on
the Library and of the other library facilities is otherwise
permitted, and provided that you do these two things:
a) Accompany the combined library with a copy of the same work
based on the Library, uncombined with any other library
facilities. This must be distributed under the terms of the
Sections above.
b) Give prominent notice with the combined library of the fact
that part of it is a work based on the Library, and explaining
where to find the accompanying uncombined form of the same work.
8. You may not copy, modify, sublicense, link with, or distribute
the Library except as expressly provided under this License. Any
attempt otherwise to copy, modify, sublicense, link with, or
distribute the Library is void, and will automatically terminate your
rights under this License. However, parties who have received copies,
or rights, from you under this License will not have their licenses
terminated so long as such parties remain in full compliance.
9. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Library or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Library (or any work based on the
Library), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Library or works based on it.
10. Each time you redistribute the Library (or any work based on the
Library), the recipient automatically receives a license from the
original licensor to copy, distribute, link with or modify the Library
subject to these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties with
this License.
11. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Library at all. For example, if a patent
license would not permit royalty-free redistribution of the Library by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Library.
If any portion of this section is held invalid or unenforceable under any
particular circumstance, the balance of the section is intended to apply,
and the section as a whole is intended to apply in other circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
12. If the distribution and/or use of the Library is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Library under this License may add
an explicit geographical distribution limitation excluding those countries,
so that distribution is permitted only in or among countries not thus
excluded. In such case, this License incorporates the limitation as if
written in the body of this License.
13. The Free Software Foundation may publish revised and/or new
versions of the Lesser General Public License from time to time.
Such new versions will be similar in spirit to the present version,
but may differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the Library
specifies a version number of this License which applies to it and
"any later version", you have the option of following the terms and
conditions either of that version or of any later version published by
the Free Software Foundation. If the Library does not specify a
license version number, you may choose any version ever published by
the Free Software Foundation.
14. If you wish to incorporate parts of the Library into other free
programs whose distribution conditions are incompatible with these,
write to the author to ask for permission. For software which is
copyrighted by the Free Software Foundation, write to the Free
Software Foundation; we sometimes make exceptions for this. Our
decision will be guided by the two goals of preserving the free status
of all derivatives of our free software and of promoting the sharing
and reuse of software generally.
NO WARRANTY
15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO
WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW.
EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR
OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY
KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE
LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME
THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN
WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY
AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU
FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR
CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE
LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING
RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Libraries
If you develop a new library, and you want it to be of the greatest
possible use to the public, we recommend making it free software that
everyone can redistribute and change. You can do so by permitting
redistribution under these terms (or, alternatively, under the terms of the
ordinary General Public License).
To apply these terms, attach the following notices to the library. It is
safest to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least the
"copyright" line and a pointer to where the full notice is found.
<one line to give the library's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Also add information on how to contact you by electronic and paper mail.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the library, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the
library `Frob' (a library for tweaking knobs) written by James Random Hacker.
<signature of Ty Coon>, 1 April 1990
Ty Coon, President of Vice
That's all there is to it!
+29
View File
@@ -0,0 +1,29 @@
# Lix
**Lix** is an implementation of **Nix**, a powerful package management system for Linux and other Unix systems that makes package management reliable and reproducible.
Read more about us at https://lix.systems.
## Installation
On Linux and macOS the easiest way to install Lix is to run the following shell command
(as a user other than root):
```console
$ curl -sSf -L https://install.lix.systems/lix | sh -s -- install
```
For systems that **already have a Nix implementation installed**, such as NixOS systems, read our [install page](https://lix.systems/install)
## Building And Developing
See our [Hacking guide](https://git.lix.systems/lix-project/lix/src/branch/main/doc/manual/src/contributing/hacking.md) in our manual for instruction on how to to set up a development environment and build Lix from source.
## Additional Resources
- [Our wiki](https://wiki.lix.systems)
- [Matrix - #space:lix.systems](https://matrix.to/#/#space:lix.systems)
## License
Lix is released under the [LGPL v2.1](./COPYING).
+3
View File
@@ -0,0 +1,3 @@
bench-*.json
bench-*.md
nixpkgs
+91
View File
@@ -0,0 +1,91 @@
# Benchmarking scripts for Lix
These are very much WIP, and have a few clumsy assumptions that we would
somewhat rather be fixed, but we have committed them to let others be able to
do benchmarking in the mean time.
## Benchmarking procedure
Build some Lixes you want to compare, by whichever means you wish.
Get a computer that is not busy and *strongly preferably* is bare-metal or at
least not a cloud VM (e.g. go make coffee when running benchmarks).
From the root of a Lix checkout, run `./bench/bench.sh resultlink-one
resultlink-two`, where `resultlink-one` and `resultlink-two` are the result
links from the builds you want to test (they can be any directory with bin/nix
in it, however).
To get the summary again, run `./bench/summarize.jq bench/bench-*.json`.
## Example results
(vim tip: `:r !bench/summarize.jq bench/bench-*.json` to dump it directly into
your editor)
```
result-asserts/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6
c6fe19ccb59cac54b5b3f25e160870 hello
mean: 15.993s ± 0.081s
user: 13.321s | system: 1.865s
median: 15.994s
range: 15.829s ... 16.096s
relative: 1
result/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19cc
b59cac54b5b3f25e160870 hello
mean: 15.897s ± 0.075s
user: 13.248s | system: 1.843s
median: 15.88s
range: 15.807s ... 16.047s
relative: 0.994
---
result/bin/nix --extra-experimental-features 'nix-command flakes' eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
mean: 0.4s ± 0.024s
user: 0.335s | system: 0.046s
median: 0.386s
range: 0.379s ... 0.43s
relative: 1
result-asserts/bin/nix --extra-experimental-features 'nix-command flakes' eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
mean: 0.404s ± 0.024s
user: 0.338s | system: 0.046s
median: 0.386s
range: 0.384s ... 0.436s
relative: 1.008
---
result-asserts/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
mean: 5.838s ± 0.023s
user: 5.083s | system: 0.464s
median: 5.845s
range: 5.799s ... 5.867s
relative: 1
result/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
mean: 5.788s ± 0.044s
user: 5.056s | system: 0.439s
median: 5.79s
range: 5.715s ... 5.876s
relative: 0.991
---
GC_INITIAL_HEAP_SIZE=10g result-asserts/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
mean: 4.147s ± 0.021s
user: 3.457s | system: 0.487s
median: 4.147s
range: 4.123s ... 4.195s
relative: 1
GC_INITIAL_HEAP_SIZE=10g result/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
mean: 4.149s ± 0.027s
user: 3.483s | system: 0.456s
median: 4.142s
range: 4.126s ... 4.215s
relative: 1
---
```
Executable
+62
View File
@@ -0,0 +1,62 @@
#!/usr/bin/env bash
set -euo pipefail
shopt -s inherit_errexit
scriptdir=$(cd "$(dirname -- "$0")" ; pwd -P)
cd "$scriptdir/.."
if [[ $# -lt 2 ]]; then
# FIXME(jade): it is a reasonable use case to want to run a benchmark run
# on just one build. However, since we are using hyperfine in comparison
# mode, we would have to combine the JSON ourselves to support that, which
# would probably be better done by writing a benchmarking script in
# not-bash.
echo "Fewer than two result dirs given, nothing to compare!" >&2
echo "Pass some directories (with names indicating which alternative they are) with bin/nix in them" >&2
echo "Usage: ./bench/bench.sh result-1 result-2 [result-3...]" >&2
exit 1
fi
_exit=""
trap "$_exit" EXIT
# XXX: yes this is very silly. flakes~!!
nix build --impure --expr '(builtins.getFlake "git+file:.").inputs.nixpkgs.outPath' -o bench/nixpkgs
export NIX_REMOTE="$(mktemp -d)"
_exit='rm -rfv "$NIX_REMOTE"; $_exit'
export NIX_PATH="nixpkgs=bench/nixpkgs:nixos-config=bench/configuration.nix"
builds=("$@")
flake_args="--extra-experimental-features 'nix-command flakes'"
hyperfineArgs=(
--parameter-list BUILD "$(IFS=,; echo "${builds[*]}")"
--warmup 2 --runs 10
)
declare -A cases
cases=(
[search]="{BUILD}/bin/nix $flake_args search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello"
[rebuild]="{BUILD}/bin/nix $flake_args eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'"
[rebuild-lh]="GC_INITIAL_HEAP_SIZE=10g {BUILD}/bin/nix eval $flake_args --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'"
[parse]="{BUILD}/bin/nix $flake_args eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix"
)
benches=(
rebuild
rebuild-lh
search
parse
)
for k in "${benches[@]}"; do
taskset -c 2,3 \
chrt -f 50 \
hyperfine "${hyperfineArgs[@]}" --export-json="bench/bench-${k}.json" --export-markdown="bench/bench-${k}.md" "${cases[$k]}"
done
echo "Benchmarks summary (from ./bench/summarize.jq bench/bench-*.json)"
bench/summarize.jq bench/*.json
+325
View File
@@ -0,0 +1,325 @@
{
config,
pkgs,
lib,
...
}:
{
boot = {
initrd = {
availableKernelModules = [
"xhci_pci"
"ahci"
];
kernelModules = [ "dm-snapshot" ];
luks.devices = {
croot = {
device = "/dev/sdb";
allowDiscards = true;
};
};
};
kernelModules = [ "kvm-intel" ];
kernelPackages = pkgs.linuxPackages_latest;
loader = {
systemd-boot.enable = true;
efi.canTouchEfiVariables = true;
};
};
hardware = {
enableRedistributableFirmware = true;
cpu.intel.updateMicrocode = true;
opengl.driSupport32Bit = true;
opengl.extraPackages = with pkgs; [
vaapiIntel
intel-media-driver
intel-compute-runtime
];
};
fileSystems = {
"/" = {
device = "/dev/sda2";
fsType = "xfs";
options = [ "noatime" ];
};
"/boot" = {
device = "/dev/sda1";
fsType = "vfat";
};
"/nas" = {
device = "nas:/";
fsType = "nfs4";
options = [
"ro"
"x-systemd.automount"
];
};
};
swapDevices = [ { device = "/dev/swap"; } ];
networking = {
useDHCP = false;
hostName = "host";
wireless = {
enable = true;
interfaces = [ "eth1" ];
};
interfaces = {
eth0.useDHCP = true;
eth1.useDHCP = true;
};
wg-quick.interfaces = {
wg0 = {
address = [ "2001:db8::1" ];
privateKeyFile = "/etc/secrets/wg0.key";
peers = [
{
publicKey = "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";
endpoint = "[2001:db8::2]:61021";
allowedIPs = [ "2001::db8:1::/64" ];
}
];
};
};
firewall.allowedUDPPorts = [ 4567 ];
};
i18n = {
defaultLocale = "en_US.UTF-8";
inputMethod.enabled = "ibus";
};
services = {
xserver = {
enable = true;
layout = "us";
xkbVariant = "altgr-intl";
xkbOptions = "ctrl:nocaps";
libinput.enable = true;
wacom.enable = true;
videoDrivers = [ "modesetting" ];
modules = [ pkgs.xf86_input_wacom ];
displayManager.sx.enable = true;
windowManager.i3.enable = true;
};
udev.extraHwdb = ''
# not like this mattered at all
# we're not running udev from here
'';
udev.extraRules = ''
# ACTION=="add", SUBSYSTEM=="input", ...
'';
};
sound.enable = true;
hardware.pulseaudio = {
enable = true;
package = pkgs.pulseaudioFull;
daemon.config = {
lock-memory = "yes";
realtime-scheduling = "yes";
rlimit-rtprio = "-1";
};
};
programs = {
light.enable = true;
wireshark = {
enable = true;
package = pkgs.wireshark-qt;
};
gnupg.agent = {
enable = true;
};
};
fonts.packages = with pkgs; [
font-awesome
noto-fonts
noto-fonts-cjk
noto-fonts-emoji
noto-fonts-extra
dejavu_fonts
powerline-fonts
source-code-pro
cantarell-fonts
];
users = {
mutableUsers = false;
users = {
user = {
isNormalUser = true;
group = "user";
extraGroups = [
"wheel"
"video"
"audio"
"dialout"
"users"
"kvm"
"wireshark"
];
password = "unimportant";
};
};
groups = {
user = { };
};
};
security = {
pam.loginLimits = [
{
domain = "@audio";
item = "memlock";
type = "-";
value = "unlimited";
}
{
domain = "@audio";
item = "rtprio";
type = "-";
value = "99";
}
{
domain = "@audio";
item = "nofile";
type = "soft";
value = "99999";
}
{
domain = "@audio";
item = "nofile";
type = "hard";
value = "99999";
}
];
sudo.extraRules = [
{
users = [ "user" ];
commands = [
{
command = "${pkgs.linuxPackages.cpupower}/bin/cpupower";
options = [ "NOPASSWD" ];
}
];
}
];
};
environment.systemPackages = with pkgs; [
a2jmidid
age
ardour
bemenu
blender
breeze-icons
breeze-qt5
bubblewrap
calf
claws-mail
darktable
duperemove
emacs
feh
file
firefox
fluidsynth
gnome3.adwaita-icon-theme
gnuplot
graphviz
helm
i3status-rust
inkscape
jack2
jq
krita
ldns
libqalculate
libreoffice
man-pages
nheko
nix-diff
nix-index
nix-output-monitor
open-music-kontrollers.patchmatrix
pamixer
pavucontrol
pciutils
picom
pwgen
redshift
ripgrep
rlwrap
silver-searcher
soundfont-fluid
whois
wol
xclip
xdot
xdotool
xorg.xkbcomp
yt-dlp
zathura
borgbackup
linuxPackages.cpupower
mtr
kitty
xf86_input_wacom
];
environment.pathsToLink = [ "/share/soundfonts" ];
systemd.user.services.run-python = {
after = [ "network-online.target" ];
script = ''
exec ${pkgs.python3}/bin/python
'';
serviceConfig = {
CapabilityBoundingSet = [ "" ];
KeyringMode = "private";
LockPersonality = true;
MemoryDenyWriteExecute = true;
NoNewPrivileges = true;
PrivateDevices = true;
PrivateTmp = true;
PrivateUsers = true;
ProcSubset = "pid";
ProtectClock = true;
ProtectControlGroups = true;
ProtectHome = true;
ProtectHostname = true;
ProtectKernelLogs = true;
ProtectKernelModules = true;
ProtectKernelTunables = true;
ProtectProc = "invisible";
ProtectSystem = "strict";
RestrictAddressFamilies = "AF_INET AF_INET6";
RestrictNamespaces = true;
RestrictRealtime = true;
RestrictSUIDSGID = true;
SystemCallArchitectures = "native";
SystemCallFilter = [
"@system-service"
"~ @resources @privileged"
];
UMask = "077";
};
};
system.stateVersion = "23.11";
}
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env -S jq -Mrf
def round3:
. * 1000 | round | . / 1000
;
def stats($first):
[
" mean: \(.mean | round3)s ± \(.stddev | round3)s",
" user: \(.user | round3)s | system: \(.system | round3)s",
" median: \(.median | round3)s",
" range: \(.min | round3)s ... \(.max | round3)s",
" relative: \(.mean / $first.mean | round3)"
]
| join("\n")
;
def fmt($first):
"\(.command)\n" + (. | stats($first))
;
[.results | .[0] as $first | .[] | fmt($first)] | join("\n\n") | (. + "\n\n---\n")
+1
View File
@@ -0,0 +1 @@
BasedOnStyle: llvm
+4
View File
@@ -0,0 +1,4 @@
# LLVM style code is 2-space indented
[*.{cc,hh}]
indent_style = space
indent_size = 2
+90
View File
@@ -0,0 +1,90 @@
#include "FixIncludes.hh"
#include <clang-tidy/ClangTidyCheck.h>
#include <clang/Basic/Diagnostic.h>
#include <clang/Basic/SourceManager.h>
#include <clang/Lex/PPCallbacks.h>
#include <clang/Lex/Preprocessor.h>
#include <llvm/ADT/StringRef.h>
#include <llvm/Support/Debug.h>
#include <memory>
#include <set>
#include <string>
namespace nix::clang_tidy {
using namespace clang;
using namespace clang::tidy;
class FixIncludesCallbacks : public PPCallbacks {
public:
ClangTidyCheck &Check;
Preprocessor &PP;
FixIncludesCallbacks(ClangTidyCheck &Check, Preprocessor &PP)
: Check(Check), PP(PP) {}
private:
bool Ignore = false;
virtual void LexedFileChanged(FileID FID, LexedFileChangeReason Reason,
SrcMgr::CharacteristicKind FileType,
FileID PrevFID, SourceLocation Loc) override;
virtual void InclusionDirective(SourceLocation HashLoc,
const Token &IncludeTok, StringRef FileName,
bool IsAngled, CharSourceRange FilenameRange,
OptionalFileEntryRef File,
StringRef SearchPath, StringRef RelativePath,
const Module *Imported,
SrcMgr::CharacteristicKind FileType) override;
};
void FixIncludesCallbacks::LexedFileChanged(FileID, LexedFileChangeReason,
SrcMgr::CharacteristicKind FileType,
FileID, SourceLocation) {
Ignore = FileType != SrcMgr::C_User;
}
void FixIncludesCallbacks::InclusionDirective(
SourceLocation, const Token &, StringRef FileName, bool IsAngled,
CharSourceRange FilenameRange, OptionalFileEntryRef File, StringRef,
StringRef, const Module *, SrcMgr::CharacteristicKind) {
if (Ignore)
return;
// FIXME: this is kinda evil, but this is a one-time fixup
const std::vector<std::string> SourceDirs = {"src/", "include/lix/"};
const auto Bracketize = [IsAngled](StringRef s) {
return IsAngled ? ("<" + s + ">").str() : ("\"" + s + "\"").str();
};
for (const auto &SourceDir : SourceDirs) {
const bool IsAlreadyFixed = FileName.starts_with("lix/lib");
if (File && File->getNameAsRequested().contains(SourceDir) &&
!IsAlreadyFixed) {
StringRef Name = File->getNameAsRequested();
auto Idx = Name.find(SourceDir);
assert(Idx != std::string::npos);
std::string Suffix = Name.drop_front(Idx + SourceDir.length()).str();
if (!Suffix.starts_with("lib")) {
llvm::dbgs() << "ignored: " << Suffix << "\n";
return;
}
Suffix = "lix/" + Suffix;
auto Diag = Check.diag(FilenameRange.getBegin(),
"include needs to specify the source subdir");
Diag << FilenameRange
<< FixItHint::CreateReplacement(FilenameRange, Bracketize(Suffix));
}
}
}
void FixIncludesCheck::registerPPCallbacks(const SourceManager &,
Preprocessor *PP, Preprocessor *) {
PP->addPPCallbacks(std::make_unique<FixIncludesCallbacks>(*this, *PP));
}
}; // namespace nix::clang_tidy
+21
View File
@@ -0,0 +1,21 @@
#pragma once
///@file
#include <clang-tidy/ClangTidyCheck.h>
#include <clang/ASTMatchers/ASTMatchFinder.h>
#include <llvm/ADT/StringRef.h>
namespace nix::clang_tidy {
using namespace clang;
using namespace clang::tidy;
class FixIncludesCheck : public ClangTidyCheck {
public:
FixIncludesCheck(StringRef Name, ClangTidyContext *Context)
: ClangTidyCheck(Name, Context) {}
void registerPPCallbacks(const SourceManager &SM, Preprocessor *PP, Preprocessor *ModuleExpanderPP) override;
};
};
+80
View File
@@ -0,0 +1,80 @@
#include "HasPrefixSuffix.hh"
#include <clang/AST/ASTTypeTraits.h>
#include <clang/AST/Expr.h>
#include <clang/AST/PrettyPrinter.h>
#include <clang/AST/Type.h>
#include <clang/ASTMatchers/ASTMatchers.h>
#include <clang/Basic/Diagnostic.h>
#include <clang/Frontend/FrontendAction.h>
#include <clang/Frontend/FrontendPluginRegistry.h>
#include <clang/Tooling/Transformer/SourceCode.h>
#include <clang/Tooling/Transformer/SourceCodeBuilders.h>
#include <iostream>
namespace nix::clang_tidy {
using namespace clang::ast_matchers;
using namespace clang;
void HasPrefixSuffixCheck::registerMatchers(ast_matchers::MatchFinder *Finder) {
Finder->addMatcher(
traverse(clang::TK_AsIs,
callExpr(callee(functionDecl(anyOf(hasName("hasPrefix"),
hasName("hasSuffix")))
.bind("callee-decl")),
optionally(hasArgument(
0, cxxConstructExpr(
hasDeclaration(functionDecl(hasParameter(
0, parmVarDecl(hasType(
asString("const char *")))))))
.bind("implicit-cast"))))
.bind("call")),
this);
}
void HasPrefixSuffixCheck::check(
const ast_matchers::MatchFinder::MatchResult &Result) {
const auto *CalleeDecl = Result.Nodes.getNodeAs<FunctionDecl>("callee-decl");
auto FuncName = std::string(CalleeDecl->getName());
std::string NewName;
if (FuncName == "hasPrefix") {
NewName = "starts_with";
} else if (FuncName == "hasSuffix") {
NewName = "ends_with";
} else {
llvm_unreachable("nix-has-prefix: invalid callee");
}
const auto *MatchedDecl = Result.Nodes.getNodeAs<CallExpr>("call");
const auto *ImplicitConvertArg =
Result.Nodes.getNodeAs<CXXConstructExpr>("implicit-cast");
const auto *Lhs = MatchedDecl->getArg(0);
const auto *Rhs = MatchedDecl->getArg(1);
auto Diag = diag(MatchedDecl->getExprLoc(), FuncName + " is deprecated");
std::string Text = "";
// Form possible cast to string_view, or nothing.
if (ImplicitConvertArg) {
Text = "std::string_view(";
Text.append(tooling::getText(*Lhs, *Result.Context));
Text.append(").");
} else {
Text.append(*tooling::buildAccess(*Lhs, *Result.Context));
}
// Call .starts_with.
Text.append(NewName);
Text.push_back('(');
Text.append(tooling::getText(*Rhs, *Result.Context));
Text.push_back(')');
Diag << FixItHint::CreateReplacement(MatchedDecl->getSourceRange(), Text);
// for (const auto *arg : MatchedDecl->arguments()) {
// arg->dumpColor();
// arg->getType().dump();
// }
}
}; // namespace nix::clang_tidy
+25
View File
@@ -0,0 +1,25 @@
#pragma once
///@file
/// This is an example of a clang-tidy automated refactoring against the Nix
/// codebase. The refactoring has been completed in
/// https://gerrit.lix.systems/c/lix/+/565 so this code is around as
/// an example.
#include <clang-tidy/ClangTidyCheck.h>
#include <clang/ASTMatchers/ASTMatchFinder.h>
#include <llvm/ADT/StringRef.h>
namespace nix::clang_tidy {
using namespace clang;
using namespace clang::tidy;
using namespace llvm;
class HasPrefixSuffixCheck : public ClangTidyCheck {
public:
HasPrefixSuffixCheck(StringRef Name, ClangTidyContext *Context)
: ClangTidyCheck(Name, Context) {}
void registerMatchers(ast_matchers::MatchFinder *Finder) override;
void check(const ast_matchers::MatchFinder::MatchResult &Result) override;
};
}; // namespace nix::clang_tidy
+19
View File
@@ -0,0 +1,19 @@
#include <clang-tidy/ClangTidyModule.h>
#include <clang-tidy/ClangTidyModuleRegistry.h>
#include "FixIncludes.hh"
#include "HasPrefixSuffix.hh"
namespace nix::clang_tidy {
using namespace clang;
using namespace clang::tidy;
class NixClangTidyChecks : public ClangTidyModule {
public:
void addCheckFactories(ClangTidyCheckFactories &CheckFactories) override {
CheckFactories.registerCheck<HasPrefixSuffixCheck>("lix-hasprefixsuffix");
CheckFactories.registerCheck<FixIncludesCheck>("lix-fixincludes");
}
};
static ClangTidyModuleRegistry::Add<NixClangTidyChecks> X("lix-module", "Adds lix specific checks");
};
+56
View File
@@ -0,0 +1,56 @@
# Clang tidy lints for Lix
This is a skeleton of a clang-tidy lints library for Lix.
Currently there is one check (which is already obsolete as it has served its
goal and is there as an example), `HasPrefixSuffixCheck`.
## Running fixes/checks
One file:
```
ninja -C build && clang-tidy --checks='-*,lix-*' --load=build/liblix-clang-tidy.so -p ../compile_commands.json -header-filter '\.\./src/.*\.h' --fix ../src/libcmd/installables.cc
```
Several files, in parallel:
```
ninja -C build && run-clang-tidy -checks='-*,lix-*' -load=build/liblix-clang-tidy.so -p .. -header-filter '\.\./src/.*\.h' -fix ../src | tee -a clang-tidy-result
```
## Resources
* https://firefox-source-docs.mozilla.org/code-quality/static-analysis/writing-new/clang-query.html
* https://clang.llvm.org/docs/LibASTMatchersReference.html
* https://devblogs.microsoft.com/cppblog/exploring-clang-tooling-part-3-rewriting-code-with-clang-tidy/
## Developing new checks
Put something like so in `myquery.txt`:
```
set traversal IgnoreUnlessSpelledInSource
# ^ Ignore implicit AST nodes. May need to use AsIs depending on how you are
# working.
set bind-root true
# ^ true unless you use any .bind("foo") commands
set print-matcher true
enable output dump
match callExpr(callee(functionDecl(hasName("hasPrefix"))), optionally(hasArgument( 0, cxxConstructExpr(hasDeclaration(functionDecl(hasParameter(0, parmVarDecl(hasType(asString("const char *"))).bind("meow2"))))))))
```
Then run, e.g. `clang-query --preload hasprefix.query -p compile_commands.json src/libcmd/installables.cc`.
With this you can iterate a query before writing it in C++ and suffering from
C++.
### Tips and tricks for the C++
There is a function `dump()` on many things that will dump to stderr. Also
`llvm::errs()` lets you print to stderr.
When I wrote `HasPrefixSuffixCheck`, I was not really able to figure out how
the structured replacement system was supposed to work. In principle you can
describe the replacement with a nice DSL. Look up the Stencil system in Clang
for details.
+13
View File
@@ -0,0 +1,13 @@
project('lix-clang-tidy', ['cpp', 'c'],
version : '0.1',
default_options : ['warning_level=3', 'cpp_std=c++20'])
llvm = dependency('Clang', version: '>= 14', modules: ['libclang'])
sources = files(
'HasPrefixSuffix.cc',
'LixClangTidyChecks.cc',
'FixIncludes.cc',
)
shared_module('lix-clang-tidy', sources,
dependencies: llvm)
+38
View File
@@ -0,0 +1,38 @@
#!/usr/bin/env nix-shell
#!nix-shell -i python3 -p python3 --pure
# To be used with `--trace-function-calls` and `flamegraph.pl`.
#
# For example:
#
# nix-instantiate --trace-function-calls '<nixpkgs>' -A hello 2> nix-function-calls.trace
# ./contrib/stack-collapse.py nix-function-calls.trace > nix-function-calls.folded
# nix-shell -p flamegraph --run "flamegraph.pl nix-function-calls.folded > nix-function-calls.svg"
import sys
from pprint import pprint
import fileinput
stack = []
timestack = []
for line in fileinput.input():
components = line.strip().split(" ", 2)
if components[0] != "function-trace":
continue
direction = components[1]
components = components[2].rsplit(" ", 2)
loc = components[0]
_at = components[1]
time = int(components[2])
if direction == "entered":
stack.append(loc)
timestack.append(time)
elif direction == "exited":
dur = time - timestack.pop()
vst = ";".join(stack)
print(f"{vst} {dur}")
stack.pop()
+9
View File
@@ -0,0 +1,9 @@
(import (
let
lock = builtins.fromJSON (builtins.readFile ./flake.lock);
in
fetchTarball {
url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
sha256 = lock.nodes.flake-compat.locked.narHash;
}
) { src = ./.; }).defaultNix
+3
View File
@@ -0,0 +1,3 @@
/doxygen.cfg
/html
/latex
+99
View File
@@ -0,0 +1,99 @@
# Doxyfile 1.9.5
# The PROJECT_NAME tag is a single word (or a sequence of words surrounded by
# double-quotes, unless you are using Doxywizard) that should identify the
# project for which the documentation is generated. This name is used in the
# title of most generated pages and in a few other places.
# The default value is: My Project.
PROJECT_NAME = "Nix"
# The PROJECT_NUMBER tag can be used to enter a project or revision number. This
# could be handy for archiving the generated documentation or if some version
# control system is used.
PROJECT_NUMBER = @PACKAGE_VERSION@
OUTPUT_DIRECTORY = @docdir@
# Using the PROJECT_BRIEF tag one can provide an optional one line description
# for a project that appears at the top of each page and should give viewer a
# quick idea about the purpose of the project. Keep the description short.
PROJECT_BRIEF = "Nix, the purely functional package manager; unstable internal interfaces"
# If the GENERATE_LATEX tag is set to YES, doxygen will generate LaTeX output.
# The default value is: YES.
GENERATE_LATEX = NO
# The INPUT tag is used to specify the files and/or directories that contain
# documented source files. You may enter file names like myfile.cpp or
# directories like /usr/src/myproject. Separate the files or directories with
# spaces. See also FILE_PATTERNS and EXTENSION_MAPPING
# Note: If this tag is empty the current directory is searched.
# FIXME Make this list more maintainable somehow. We could maybe generate this
# in the Makefile, but we would need to change how `.in` files are preprocessed
# so they can expand variables despite configure variables.
INPUT = \
src/libcmd \
src/libexpr \
src/libexpr/flake \
tests/unit/libexpr \
tests/unit/libexpr/value \
tests/unit/libexpr/test \
tests/unit/libexpr/test/value \
src/libexpr/value \
src/libfetchers \
src/libmain \
src/libstore \
src/libstore/build \
src/libstore/builtins \
tests/unit/libstore \
tests/unit/libstore/test \
src/libutil \
tests/unit/libutil \
tests/unit/libutil/test \
src/nix \
src/nix-env \
src/nix-store
# If the MACRO_EXPANSION tag is set to YES, doxygen will expand all macro names
# in the source code. If set to NO, only conditional compilation will be
# performed. Macro expansion can be done in a controlled way by setting
# EXPAND_ONLY_PREDEF to YES.
# The default value is: NO.
# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
MACRO_EXPANSION = YES
# If the EXPAND_ONLY_PREDEF and MACRO_EXPANSION tags are both set to YES then
# the macro expansion is limited to the macros specified with the PREDEFINED and
# EXPAND_AS_DEFINED tags.
# The default value is: NO.
# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
EXPAND_ONLY_PREDEF = YES
# The INCLUDE_PATH tag can be used to specify one or more directories that
# contain include files that are not input files but should be processed by the
# preprocessor. Note that the INCLUDE_PATH is not recursive, so the setting of
# RECURSIVE has no effect here.
# This tag requires that the tag SEARCH_INCLUDES is set to YES.
INCLUDE_PATH = @RAPIDCHECK_HEADERS@
# If the MACRO_EXPANSION and EXPAND_ONLY_PREDEF tags are set to YES then this
# tag can be used to specify a list of macro names that should be expanded. The
# macro definition that is found in the sources will be used. Use the PREDEFINED
# tag if you want to use a different macro definition that overrules the
# definition found in the source code.
# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.
EXPAND_AS_DEFINED = \
DECLARE_COMMON_SERIALISER \
DECLARE_WORKER_SERIALISER \
DECLARE_SERVE_SERIALISER \
LENGTH_PREFIXED_PROTO_HELPER
+34
View File
@@ -0,0 +1,34 @@
doxygen_cfg = configure_file(
input : 'doxygen.cfg.in',
output : 'doxygen.cfg',
configuration : {
'PACKAGE_VERSION': meson.project_version(),
'RAPIDCHECK_HEADERS': rapidcheck_meson.get_variable('includedir'),
'docdir' : meson.current_build_dir(),
},
)
internal_api_docs = custom_target(
'internal-api-docs',
command : [
bash,
# Meson can you please just give us a `workdir` argument to custom targets...
'-c',
# We have to prefix the doxygen_cfg path with the project build root
# because of the cd in front.
'cd @0@ && @1@ @2@/@INPUT0@'.format(
meson.project_source_root(),
doxygen.full_path(),
meson.project_build_root(),
),
],
input : [
doxygen_cfg,
],
output : 'html',
install : true,
install_dir : datadir / 'doc/nix/internal-api',
build_always_stale : true,
)
alias_target('internal-api-html', internal_api_docs)
+31
View File
@@ -0,0 +1,31 @@
"\\[\\]\\{#(?<anchor>[^\\}]+?)\\}" as $empty_anchor_regex |
"\\[(?<text>[^\\]]+?)\\]\\{#(?<anchor>[^\\}]+?)\\}" as $anchor_regex |
def transform_anchors_html:
. | gsub($empty_anchor_regex; "<a name=\"" + .anchor + "\"></a>")
| gsub($anchor_regex; "<a href=\"#" + .anchor + "\" id=\"" + .anchor + "\">" + .text + "</a>");
def transform_anchors_strip:
. | gsub($empty_anchor_regex; "")
| gsub($anchor_regex; .text);
def map_contents_recursively(transformer):
. + {
Chapter: (.Chapter + {
content: .Chapter.content | transformer,
sub_items: .Chapter.sub_items | map(map_contents_recursively(transformer)),
}),
};
def process_command:
.[0] as $context |
.[1] as $body |
$body + {
sections: $body.sections | map(map_contents_recursively(if $context.renderer == "html" then transform_anchors_html else transform_anchors_strip end)),
};
process_command
+42
View File
@@ -0,0 +1,42 @@
[book]
title = "Lix Reference Manual"
[build]
create-missing = false
[output.html]
additional-css = ["custom.css"]
additional-js = ["redirects.js"]
# Using our GitHub mirror enables easier typo fixes since there is no easy way
# to just submit a Gerrit CL by the web for trivial stuff.
edit-url-template = "https://github.com/lix-project/lix/tree/main/doc/manual/{path}"
git-repository-url = "https://git.lix.systems/lix-project/lix"
# Folding by default would prevent things like "Ctrl+F for nix-env" from working
# trivially, but the user should be able to fold if they want to.
fold.enable = true
fold.level = 30
# Handles replacing @docroot@ with a path to ./src relative to that markdown file,
# {{#include handlebars}}, and the @generated@ syntax used within these. it mostly
# but not entirely replaces the links preprocessor (which we cannot simply use due
# to @generated@ files living in a different directory to make meson happy). we do
# not want to disable the links preprocessor entirely though because that requires
# disabling *all* built-in preprocessors and selectively reenabling those we want.
[preprocessor.substitute]
command = "python3 doc/manual/substitute.py"
before = ["anchors", "links"]
[preprocessor.anchors]
renderers = ["html"]
command = "jq --from-file doc/manual/anchors.jq"
[output.markdown]
[output.linkcheck]
# no Internet during the build (in the sandbox)
follow-web-links = false
# mdbook-linkcheck does not understand [foo]{#bar} style links, resulting in
# excessive "Potential incomplete link" warnings. No other kind of warning was
# produced at the time of writing.
warning-policy = "ignore"
+123
View File
@@ -0,0 +1,123 @@
# This file is a mapping of metadata for change authors, sort of like
# maintainer-list.nix in nixpkgs.
#
# It's used for crediting people accurately in release notes. The release notes
# script will link to forgejo, then to GitHub if forgejo is not present.
#
# When adding someone from outside the Lix project, you generally want to simply link their GitHub profile without adding a display name unless they are well-known in the community by that display name.
#
# See doc/manual/src/contributing/hacking.md for more documentation on this file's format and typical usage.
9999years:
display_name: wiggles
forgejo: rbt
github: 9999years
Artturin:
github: Artturin
DavHau:
github: DavHau
Kha:
github: Kha
Lunaphied:
forgejo: Lunaphied
github: Lunaphied
Qyriad:
forgejo: Qyriad
github: Qyriad
SharzyL:
github: SharzyL
alois31:
forgejo: alois31
github: alois31
artemist:
display_name: Artemis Tosini
forgejo: artemist
cole-h:
display_name: Cole Helbling
github: cole-h
edolstra:
display_name: Eelco Dolstra
github: edolstra
ericson:
display_name: John Ericson
github: ericson2314
horrors:
display_name: eldritch horrors
forgejo: pennae
github: pennae
iFreilicht:
github: iFreilicht
jade:
forgejo: jade
github: lf-
kloenk:
forgejo: kloenk
github: kloenk
lovesegfault:
github: lovesegfault
ma27:
forgejo: ma27
github: ma27
matthewbauer:
github: matthewbauer
midnightveil:
display_name: julia
forgejo: midnightveil
github: midnightveil
ncfavier:
github: ncfavier
puck:
display_name: puck
forgejo: puck
github: puckipedia
r-vdp:
github: r-vdp
raito:
display_name: Raito Bezarius
forgejo: raito
github: RaitoBezarius
roberth:
display_name: Robert Hensing
github: roberth
thufschmitt:
display_name: Théophane Hufschmitt
github: thufschmitt
tomberek:
display_name: Tom Bereknyei
github: tomberek
valentin:
display_name: Valentin Gagarin
github: fricklerhandwerk
winter:
forgejo: winter
github: winterqt
yshui:
github: yshui
+11
View File
@@ -0,0 +1,11 @@
h1:not(:first-of-type) {
margin-top: 1.3em;
}
h2 {
margin-top: 1em;
}
.hljs-meta {
user-select: none;
}
+37
View File
@@ -0,0 +1,37 @@
let
inherit (builtins) concatStringsSep attrValues mapAttrs;
inherit (import ./utils.nix) optionalString squash;
in
builtinsInfo:
let
showBuiltin =
name:
{
doc,
type,
impure-only,
}:
let
type' = optionalString (type != null) " (${type})";
impureNotice = optionalString impure-only ''
> **Note**
>
> Not available in [pure evaluation mode](@docroot@/command-ref/conf-file.md#conf-pure-eval).
'';
in
squash ''
<dt id="builtins-${name}">
<a href="#builtins-${name}"><code>${name}</code></a>${type'}
</dt>
<dd>
${doc}
${impureNotice}
</dd>
'';
in
concatStringsSep "\n" (attrValues (mapAttrs showBuiltin builtinsInfo))
+35
View File
@@ -0,0 +1,35 @@
let
inherit (builtins) concatStringsSep attrValues mapAttrs;
inherit (import ./utils.nix) optionalString squash;
in
builtinsInfo:
let
showBuiltin =
name:
{
doc,
args,
arity,
experimental-feature,
}:
let
experimentalNotice = optionalString (experimental-feature != null) ''
This function is only available if the [${experimental-feature}](@docroot@/contributing/experimental-features.md#xp-feature-${experimental-feature}) experimental feature is enabled.
'';
in
squash ''
<dt id="builtins-${name}">
<a href="#builtins-${name}"><code>${name} ${listArgs args}</code></a>
</dt>
<dd>
${doc}
${experimentalNotice}
</dd>
'';
listArgs = args: concatStringsSep " " (map (s: "<var>${s}</var>") args);
in
concatStringsSep "\n" (attrValues (mapAttrs showBuiltin builtinsInfo))
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env python3
import glob
import sys
# meson expects makefile-style dependency declarations, i.e.
#
# target: dependency...
#
# meson seems to pass depfiles straight on to ninja even though
# it also parses the file itself (or at least has code to do so
# in its tree), so we must live by ninja's rules: only slashes,
# spaces and octothorpes can be escaped, anything else is taken
# literally. since the rules for these aren't even the same for
# all three we will just fail when we encounter any of them (if
# asserts are off for some reason the depfile will likely point
# to nonexistant paths, making everything phony and thus fine.)
for path in glob.glob(sys.argv[1] + '/**', recursive=True):
assert '\\' not in path
assert ' ' not in path
assert '#' not in path
print("ignored:", path)
+224
View File
@@ -0,0 +1,224 @@
let
inherit (builtins)
attrNames
attrValues
fromJSON
listToAttrs
mapAttrs
concatStringsSep
concatMap
length
lessThan
replaceStrings
sort
;
inherit (import ./utils.nix)
concatStrings
optionalString
filterAttrs
trim
squash
unique
showSettings
;
in
inlineHTML: commandDump:
let
commandInfo = fromJSON commandDump;
showCommand =
{
command,
details,
filename,
toplevel,
}:
let
result = ''
> **Warning** \
> This program is
> [**experimental**](@docroot@/contributing/experimental-features.md#xp-feature-nix-command)
> and its interface is subject to change.
# Name
`${command}` - ${details.description}
# Synopsis
${showSynopsis command details.args}
${maybeSubcommands}
${maybeStoreDocs}
${maybeOptions}
'';
showSynopsis =
command: args:
let
showArgument = arg: "*${arg.label}*" + optionalString (!arg ? arity) "...";
arguments = concatStringsSep " " (map showArgument args);
in
''
`${command}` [*option*...] ${arguments}
'';
maybeSubcommands = optionalString (details ? commands && details.commands != { }) ''
where *subcommand* is one of the following:
${subcommands}
'';
subcommands = if length categories > 1 then listCategories else listSubcommands details.commands;
categories = sort (x: y: x.id < y.id) (
unique (map (cmd: cmd.category) (attrValues details.commands))
);
listCategories = concatStrings (map showCategory categories);
showCategory = cat: ''
**${toString cat.description}:**
${listSubcommands (filterAttrs (n: v: v.category == cat) details.commands)}
'';
listSubcommands = cmds: concatStrings (attrValues (mapAttrs showSubcommand cmds));
showSubcommand = name: subcmd: ''
* [`${command} ${name}`](./${appendName filename name}.md) - ${subcmd.description}
'';
# TODO: move this confusing special case out of here when implementing #8496
maybeStoreDocs = optionalString (details ? doc) (
replaceStrings [ "@stores@" ] [ storeDocs ] details.doc
);
maybeOptions = optionalString (details.flags != { }) ''
# Options
${showOptions details.flags toplevel.flags}
> **Note**
>
> See [`man nix.conf`](@docroot@/command-ref/conf-file.md#command-line-flags) for overriding configuration settings with command line flags.
'';
showOptions =
options: commonOptions:
let
allOptions = options // commonOptions;
showCategory = cat: ''
${optionalString (cat != "") "**${cat}:**"}
${listOptions (filterAttrs (n: v: v.category == cat) allOptions)}
'';
listOptions = opts: concatStringsSep "\n" (attrValues (mapAttrs showOption opts));
showOption =
name: option:
let
result = trim ''
- ${item}
${option.description}
'';
item =
if inlineHTML then
''<span id="opt-${name}">[`--${name}`](#opt-${name})</span> ${shortName} ${labels}''
else
"`--${name}` ${shortName} ${labels}";
shortName = optionalString (option ? shortName) ("/ `-${option.shortName}`");
labels = optionalString (option ? labels) (concatStringsSep " " (map (s: "*${s}*") option.labels));
in
result;
categories = sort lessThan (unique (map (cmd: cmd.category) (attrValues allOptions)));
in
concatStrings (map showCategory categories);
in
squash result;
appendName = filename: name: (if filename == "nix" then "nix3" else filename) + "-" + name;
processCommand =
{
command,
details,
filename,
toplevel,
}:
let
cmd = {
inherit command;
name = filename + ".md";
value = showCommand {
inherit
command
details
filename
toplevel
;
};
};
subcommand =
subCmd:
processCommand {
command = command + " " + subCmd;
details = details.commands.${subCmd};
filename = appendName filename subCmd;
inherit toplevel;
};
in
[ cmd ] ++ concatMap subcommand (attrNames details.commands or { });
manpages = processCommand {
command = "nix";
details = commandInfo.args;
filename = "nix";
toplevel = commandInfo.args;
};
storeDocs =
let
showStore =
name:
{
settings,
doc,
experimentalFeature,
}:
let
experimentalFeatureNote = optionalString (experimentalFeature != null) ''
> **Warning**
> This store is part of an
> [experimental feature](@docroot@/contributing/experimental-features.md).
To use this store, you need to make sure the corresponding experimental feature,
[`${experimentalFeature}`](@docroot@/contributing/experimental-features.md#xp-feature-${experimentalFeature}),
is enabled.
For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md):
```
extra-experimental-features = ${experimentalFeature}
```
'';
in
''
## ${name}
${doc}
${experimentalFeatureNote}
**Settings**:
${showSettings { inherit inlineHTML; } settings}
'';
in
concatStrings (attrValues (mapAttrs showStore commandInfo.stores));
in
listToAttrs manpages
@@ -0,0 +1,9 @@
with builtins;
with import ./utils.nix;
let
showExperimentalFeature = name: doc: ''
- [`${name}`](@docroot@/contributing/experimental-features.md#xp-feature-${name})
'';
in
xps: indent " " (concatStrings (attrValues (mapAttrs showExperimentalFeature xps)))
+13
View File
@@ -0,0 +1,13 @@
with builtins;
with import ./utils.nix;
let
showExperimentalFeature =
name: doc:
squash ''
## [`${name}`]{#xp-feature-${name}}
${doc}
'';
in
xps: (concatStringsSep "\n" (attrValues (mapAttrs showExperimentalFeature xps)))
+61
View File
@@ -0,0 +1,61 @@
#!/usr/bin/env python3
"""
This script is a helper for this project's Meson buildsystem, to replace its
usage of `nix eval --write-to`. Writing a JSON object as a nested directory
tree is more generic, easier to maintain, and far, far less cursed. Nix
has 'good' support for JSON output. Let's just use it.
"""
import argparse
from pathlib import Path
import json
import sys
name = 'json-to-tree.py'
def log(*args, **kwargs):
kwargs['file'] = sys.stderr
return print(f'{name}:', *args, **kwargs)
def write_dict_to_directory(current_directory: Path, data: dict, files_written=0):
current_directory.mkdir(parents=True, exist_ok=True)
for key, value in data.items():
nested_path = current_directory / key
match value:
case dict(nested_data):
files_written += write_dict_to_directory(nested_path, nested_data)
case str(content):
nested_path.write_text(content)
files_written += 1
case rest:
assert False, \
f'should have been called on a dict or string, not {type(rest)=}\n\t{rest=}'
return files_written
def main():
parser = argparse.ArgumentParser(name)
parser.add_argument('-i', '--input', type=argparse.FileType('r'), default='-',
help='The JSON input to operate on and output as a directory tree',
)
parser.add_argument('-o', '--output', type=Path, required=True,
help='The place to put the directory tree',
)
args = parser.parse_args()
json_string = args.input.read()
try:
data = json.loads(json_string)
except json.JSONDecodeError:
log(f'could not decode JSON from input: {json_string}')
raise
files_written = write_dict_to_directory(args.output, data)
log(f'wrote {files_written} files')
sys.exit(main())
+353
View File
@@ -0,0 +1,353 @@
nix_env_for_docs = {
'HOME': '/dummy',
'NIX_CONF_DIR': '/dummy',
'NIX_SSL_CERT_FILE': '/dummy/no-ca-bundle.crt',
'NIX_STATE_DIR': '/dummy',
'NIX_CONFIG': 'cores = 0',
}
nix_for_docs = [ nix, '--experimental-features', 'nix-command' ]
nix_eval_for_docs_common = nix_for_docs + [
'eval',
'-I', 'nix/corepkgs=corepkgs',
'--store', 'dummy://',
'--impure',
]
nix_eval_for_docs = nix_eval_for_docs_common + '--raw'
conf_file_json = custom_target(
command : nix_for_docs + [ 'show-config', '--json' ],
capture : true,
output : 'conf-file.json',
env : nix_env_for_docs,
)
nix_conf_file_md_body = custom_target(
command : nix_eval_for_docs + [
'--expr',
'(import @INPUT0@).showSettings { inlineHTML = true; } (builtins.fromJSON (builtins.readFile @INPUT1@))',
],
capture : true,
input : [
'utils.nix',
conf_file_json,
],
output : 'conf-file.md.body',
env : nix_env_for_docs,
)
nix_conf_file_md = custom_target(
command : [ 'cat', '@INPUT@' ],
capture : true,
input : [
'src/command-ref/conf-file.md',
nix_conf_file_md_body,
],
output : 'conf-file.md',
)
nix_exp_features_json = custom_target(
command : [ nix, '__dump-xp-features' ],
capture : true,
output : 'xp-features.json',
)
language_json = custom_target(
command: [nix, '__dump-language'],
output : 'language.json',
capture : true,
env : nix_env_for_docs,
)
nix3_cli_json = custom_target(
command : [ nix, '__dump-cli' ],
capture : true,
output : 'nix.json',
env : nix_env_for_docs,
)
generate_manual_deps = files(
'generate-deps.py',
)
# Generates builtins.md and builtin-constants.md.
subdir('src/language')
# Generates new-cli pages, experimental-features-shortlist.md, and conf-file.md.
subdir('src/command-ref')
# Generates experimental-feature-descriptions.md.
subdir('src/contributing')
# Generates rl-next-generated.md.
subdir('src/release-notes')
manual = custom_target(
'manual',
command : [
bash,
'-euo', 'pipefail',
'-c',
'''
@0@ @INPUT0@ @CURRENT_SOURCE_DIR@ > @DEPFILE@
cd @SOURCE_ROOT@
@1@ build doc/manual -d @2@ | { grep -Fv "because fragment resolution isn't implemented" || :; }
rm -rf @2@/manual
mv @2@/html @2@/manual
find @2@/manual -iname meson.build -delete
'''.format(
python.full_path(),
mdbook.full_path(),
meson.current_build_dir(),
),
],
input : [
generate_manual_deps,
'book.toml',
'anchors.jq',
'custom.css',
nix3_cli_files,
experimental_features_shortlist_md,
experimental_feature_descriptions_md,
conf_file_md,
builtins_md,
builtin_constants_md,
rl_next_generated,
nix,
],
output : [
'manual',
'markdown',
],
depfile : 'manual.d',
env : {
'RUST_LOG': 'info',
'MDBOOK_SUBSTITUTE_SEARCH': meson.current_build_dir() / 'src',
},
)
manual_html = manual[0]
manual_md = manual[1]
install_subdir(
manual_html.full_path(),
install_dir : datadir / 'doc/nix',
)
nix_nested_manpages = [
[ 'nix-env',
[
'delete-generations',
'install',
'list-generations',
'query',
'rollback',
'set-flag',
'set',
'switch-generation',
'switch-profile',
'uninstall',
'upgrade',
],
],
[ 'nix-store',
[
'add-fixed',
'add',
'delete',
'dump-db',
'dump',
'export',
'gc',
'generate-binary-cache-key',
'import',
'load-db',
'optimise',
'print-env',
'query',
'read-log',
'realise',
'repair-path',
'restore',
'serve',
'verify',
'verify-path',
],
],
]
foreach command : nix_nested_manpages
foreach page : command[1]
title = command[0] + ' --' + page
section = '1'
custom_target(
command : [
'./render-manpage.sh',
'--out-no-smarty',
title,
section,
'@INPUT0@/command-ref' / command[0] / (page + '.md'),
'@OUTPUT0@',
],
input : [
manual_md,
nix,
],
output : command[0] + '-' + page + '.1',
install : true,
install_dir : mandir / 'man1',
)
endforeach
endforeach
nix3_manpages = [
'nix3-build',
'nix3-bundle',
'nix3-config',
'nix3-config-show',
'nix3-copy',
'nix3-daemon',
'nix3-derivation-add',
'nix3-derivation',
'nix3-derivation-show',
'nix3-develop',
'nix3-doctor',
'nix3-edit',
'nix3-eval',
'nix3-flake-archive',
'nix3-flake-check',
'nix3-flake-clone',
'nix3-flake-info',
'nix3-flake-init',
'nix3-flake-lock',
'nix3-flake',
'nix3-flake-metadata',
'nix3-flake-new',
'nix3-flake-prefetch',
'nix3-flake-show',
'nix3-flake-update',
'nix3-fmt',
'nix3-hash-file',
'nix3-hash',
'nix3-hash-path',
'nix3-hash-to-base16',
'nix3-hash-to-base32',
'nix3-hash-to-base64',
'nix3-hash-to-sri',
'nix3-help',
'nix3-help-stores',
'nix3-key-convert-secret-to-public',
'nix3-key-generate-secret',
'nix3-key',
'nix3-log',
'nix3-nar-cat',
'nix3-nar-dump-path',
'nix3-nar-ls',
'nix3-nar',
'nix3-path-info',
'nix3-print-dev-env',
'nix3-profile-diff-closures',
'nix3-profile-history',
'nix3-profile-install',
'nix3-profile-list',
'nix3-profile',
'nix3-profile-remove',
'nix3-profile-rollback',
'nix3-profile-upgrade',
'nix3-profile-wipe-history',
'nix3-realisation-info',
'nix3-realisation',
'nix3-registry-add',
'nix3-registry-list',
'nix3-registry',
'nix3-registry-pin',
'nix3-registry-remove',
'nix3-repl',
'nix3-run',
'nix3-search',
'nix3-shell',
'nix3-store-add-file',
'nix3-store-add-path',
'nix3-store-cat',
'nix3-store-copy-log',
'nix3-store-copy-sigs',
'nix3-store-delete',
'nix3-store-diff-closures',
'nix3-store-dump-path',
'nix3-store-gc',
'nix3-store-ls',
'nix3-store-make-content-addressed',
'nix3-store',
'nix3-store-optimise',
'nix3-store-path-from-hash-part',
'nix3-store-ping',
'nix3-store-prefetch-file',
'nix3-store-repair',
'nix3-store-sign',
'nix3-store-verify',
'nix3-upgrade-nix',
'nix3-why-depends',
'nix',
]
foreach page : nix3_manpages
section = '1'
custom_target(
command : [
bash,
'@INPUT0@',
page,
section,
'@INPUT1@/command-ref/new-cli/@0@.md'.format(page),
'@OUTPUT@',
],
input : [
'render-manpage.sh',
manual_md,
nix,
],
output : page + '.1',
install : true,
install_dir : mandir / 'man1',
)
endforeach
nix_manpages = [
[ 'nix-env', 1 ],
[ 'nix-store', 1 ],
[ 'nix-build', 1 ],
[ 'nix-shell', 1 ],
[ 'nix-instantiate', 1 ],
[ 'nix-collect-garbage', 1 ],
[ 'nix-prefetch-url', 1 ],
[ 'nix-channel', 1 ],
[ 'nix-hash', 1 ],
[ 'nix-copy-closure', 1 ],
[ 'nix.conf', 5, 'conf-file.md' ],
[ 'nix-daemon', 8 ],
[ 'nix-profiles', 5, 'files/profiles.md' ],
]
foreach entry : nix_manpages
title = entry[0]
# nix.conf.5 and nix-profiles.5 are based off of conf-file.md and files/profiles.md,
# rather than a stem identical to its mdbook source.
# Therefore we use an optional third element of this array to override the name pattern
md_file = entry.get(2, title + '.md')
section = entry[1].to_string()
custom_target(
command : [
bash,
'@INPUT0@',
title,
section,
'@INPUT1@/command-ref/@0@'.format(md_file),
'@OUTPUT@',
],
input : [
'render-manpage.sh',
manual_md,
entry.get(3, []),
nix,
],
output : '@0@.@1@'.format(entry[0], entry[1]),
install : true,
install_dir : mandir / 'man@0@'.format(entry[1]),
)
endforeach
+32
View File
@@ -0,0 +1,32 @@
#!/usr/bin/env bash
set -euo pipefail
infile="$1"
outfile="$2"
shift 2
# set a search path for includes. the old makefile-based system splorked
# everything into the source tree and was thus able to not have a search
# path, but the meson system generates intermediate files into dedicated
# directories separate from the source. we still retain the implicit old
# behavior for now as the base search path, once meson is the default we
# can revisit this and remove the implicit search path entry. it's fine.
set -- "$(dirname "$infile")" "$@"
# re-implement mdBook's include directive to make it usable for terminal output and for proper @docroot@ substitution
(grep '{{#include' "$infile" || true) | while read -r line; do
found=false
include="$(printf "$line" | sed 's/{{#include \(.*\)}}/\1/')"
for path in "$@"; do
filename="$path/$include"
if [ -e "$filename" ]; then
found=true
matchline="$(printf "$line" | sed 's|/|\\/|g')"
sed -i "/$matchline/r $filename" "$outfile"
sed -i "s/$matchline//" "$outfile"
break
fi
done
$found || ( echo "#include-d file '$filename' does not exist." >&2; exit 1; )
done
+40
View File
@@ -0,0 +1,40 @@
<?xml version="1.0"?>
<xsl:stylesheet
version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:str="http://exslt.org/strings"
extension-element-prefixes="str">
<xsl:output method="xml"/>
<xsl:template match="function|command|literal|varname|filename|option|quote">`<xsl:apply-templates/>'</xsl:template>
<xsl:template match="token"><xsl:text> </xsl:text><xsl:apply-templates /><xsl:text>
</xsl:text></xsl:template>
<xsl:template match="screen|programlisting">
<screen><xsl:apply-templates select="str:split(., '&#xA;')" /></screen>
</xsl:template>
<xsl:template match="section[following::section]">
<section>
<xsl:apply-templates />
<screen><xsl:text>
</xsl:text></screen>
</section>
</xsl:template>
<xsl:template match="*">
<xsl:element name="{name(.)}" namespace="{namespace-uri(.)}">
<xsl:copy-of select="namespace::*" />
<xsl:for-each select="@*">
<xsl:attribute name="{name(.)}" namespace="{namespace-uri(.)}">
<xsl:value-of select="."/>
</xsl:attribute>
</xsl:for-each>
<xsl:apply-templates/>
</xsl:element>
</xsl:template>
</xsl:stylesheet>
+8
View File
@@ -0,0 +1,8 @@
[lix-docs]
type = s3
provider = Other
env_auth = true
endpoint = https://s3.lix.systems
location_constraint = garage
region = garage
acl = private
+440
View File
@@ -0,0 +1,440 @@
// redirect rules for URL fragments (client-side) to prevent link rot.
// this must be done on the client side, as web servers do not see the fragment part of the URL.
// it will only work with JavaScript enabled in the browser, but this is the best we can do here.
// see ./_redirects for path redirects (client-side)
// redirects are declared as follows:
// each entry has as its key a path matching the requested URL path, relative to the mdBook document root.
//
// IMPORTANT: it must specify the full path with file name and suffix
//
// each entry is itself a set of key-value pairs, where
// - keys are anchors on the matched path.
// - values are redirection targets relative to the current path.
const redirects = {
"index.html": {
"part-advanced-topics": "advanced-topics/advanced-topics.html",
"chap-tuning-cores-and-jobs": "advanced-topics/cores-vs-jobs.html",
"chap-diff-hook": "advanced-topics/diff-hook.html",
"check-dirs-are-unregistered": "advanced-topics/diff-hook.html#check-dirs-are-unregistered",
"chap-distributed-builds": "advanced-topics/distributed-builds.html",
"chap-post-build-hook": "advanced-topics/post-build-hook.html",
"chap-post-build-hook-caveats": "advanced-topics/post-build-hook.html#implementation-caveats",
"chap-writing-nix-expressions": "language/index.html",
"part-command-ref": "command-ref/command-ref.html",
"conf-allow-import-from-derivation": "command-ref/conf-file.html#conf-allow-import-from-derivation",
"conf-allowed-uris": "command-ref/conf-file.html#conf-allowed-uris",
"conf-allowed-users": "command-ref/conf-file.html#conf-allowed-users",
"conf-auto-optimise-store": "command-ref/conf-file.html#conf-auto-optimise-store",
"conf-binary-cache-public-keys": "command-ref/conf-file.html#conf-binary-cache-public-keys",
"conf-binary-caches": "command-ref/conf-file.html#conf-binary-caches",
"conf-build-compress-log": "command-ref/conf-file.html#conf-build-compress-log",
"conf-build-cores": "command-ref/conf-file.html#conf-build-cores",
"conf-build-extra-chroot-dirs": "command-ref/conf-file.html#conf-build-extra-chroot-dirs",
"conf-build-extra-sandbox-paths": "command-ref/conf-file.html#conf-build-extra-sandbox-paths",
"conf-build-fallback": "command-ref/conf-file.html#conf-build-fallback",
"conf-build-max-jobs": "command-ref/conf-file.html#conf-build-max-jobs",
"conf-build-max-log-size": "command-ref/conf-file.html#conf-build-max-log-size",
"conf-build-max-silent-time": "command-ref/conf-file.html#conf-build-max-silent-time",
"conf-build-timeout": "command-ref/conf-file.html#conf-build-timeout",
"conf-build-use-chroot": "command-ref/conf-file.html#conf-build-use-chroot",
"conf-build-use-sandbox": "command-ref/conf-file.html#conf-build-use-sandbox",
"conf-build-use-substitutes": "command-ref/conf-file.html#conf-build-use-substitutes",
"conf-build-users-group": "command-ref/conf-file.html#conf-build-users-group",
"conf-builders": "command-ref/conf-file.html#conf-builders",
"conf-builders-use-substitutes": "command-ref/conf-file.html#conf-builders-use-substitutes",
"conf-compress-build-log": "command-ref/conf-file.html#conf-compress-build-log",
"conf-connect-timeout": "command-ref/conf-file.html#conf-connect-timeout",
"conf-cores": "command-ref/conf-file.html#conf-cores",
"conf-diff-hook": "command-ref/conf-file.html#conf-diff-hook",
"conf-env-keep-derivations": "command-ref/conf-file.html#conf-env-keep-derivations",
"conf-extra-binary-caches": "command-ref/conf-file.html#conf-extra-binary-caches",
"conf-extra-platforms": "command-ref/conf-file.html#conf-extra-platforms",
"conf-extra-sandbox-paths": "command-ref/conf-file.html#conf-extra-sandbox-paths",
"conf-extra-substituters": "command-ref/conf-file.html#conf-extra-substituters",
"conf-fallback": "command-ref/conf-file.html#conf-fallback",
"conf-fsync-metadata": "command-ref/conf-file.html#conf-fsync-metadata",
"conf-gc-keep-derivations": "command-ref/conf-file.html#conf-gc-keep-derivations",
"conf-gc-keep-outputs": "command-ref/conf-file.html#conf-gc-keep-outputs",
"conf-hashed-mirrors": "command-ref/conf-file.html#conf-hashed-mirrors",
"conf-http-connections": "command-ref/conf-file.html#conf-http-connections",
"conf-keep-build-log": "command-ref/conf-file.html#conf-keep-build-log",
"conf-keep-derivations": "command-ref/conf-file.html#conf-keep-derivations",
"conf-keep-env-derivations": "command-ref/conf-file.html#conf-keep-env-derivations",
"conf-keep-outputs": "command-ref/conf-file.html#conf-keep-outputs",
"conf-max-build-log-size": "command-ref/conf-file.html#conf-max-build-log-size",
"conf-max-free": "command-ref/conf-file.html#conf-max-free",
"conf-max-jobs": "command-ref/conf-file.html#conf-max-jobs",
"conf-max-silent-time": "command-ref/conf-file.html#conf-max-silent-time",
"conf-min-free": "command-ref/conf-file.html#conf-min-free",
"conf-narinfo-cache-negative-ttl": "command-ref/conf-file.html#conf-narinfo-cache-negative-ttl",
"conf-narinfo-cache-positive-ttl": "command-ref/conf-file.html#conf-narinfo-cache-positive-ttl",
"conf-netrc-file": "command-ref/conf-file.html#conf-netrc-file",
"conf-plugin-files": "command-ref/conf-file.html#conf-plugin-files",
"conf-post-build-hook": "command-ref/conf-file.html#conf-post-build-hook",
"conf-pre-build-hook": "command-ref/conf-file.html#conf-pre-build-hook",
"conf-require-sigs": "command-ref/conf-file.html#conf-require-sigs",
"conf-restrict-eval": "command-ref/conf-file.html#conf-restrict-eval",
"conf-run-diff-hook": "command-ref/conf-file.html#conf-run-diff-hook",
"conf-sandbox": "command-ref/conf-file.html#conf-sandbox",
"conf-sandbox-dev-shm-size": "command-ref/conf-file.html#conf-sandbox-dev-shm-size",
"conf-sandbox-paths": "command-ref/conf-file.html#conf-sandbox-paths",
"conf-secret-key-files": "command-ref/conf-file.html#conf-secret-key-files",
"conf-show-trace": "command-ref/conf-file.html#conf-show-trace",
"conf-stalled-download-timeout": "command-ref/conf-file.html#conf-stalled-download-timeout",
"conf-substitute": "command-ref/conf-file.html#conf-substitute",
"conf-substituters": "command-ref/conf-file.html#conf-substituters",
"conf-system": "command-ref/conf-file.html#conf-system",
"conf-system-features": "command-ref/conf-file.html#conf-system-features",
"conf-tarball-ttl": "command-ref/conf-file.html#conf-tarball-ttl",
"conf-timeout": "command-ref/conf-file.html#conf-timeout",
"conf-trace-function-calls": "command-ref/conf-file.html#conf-trace-function-calls",
"conf-trusted-binary-caches": "command-ref/conf-file.html#conf-trusted-binary-caches",
"conf-trusted-public-keys": "command-ref/conf-file.html#conf-trusted-public-keys",
"conf-trusted-substituters": "command-ref/conf-file.html#conf-trusted-substituters",
"conf-trusted-users": "command-ref/conf-file.html#conf-trusted-users",
"extra-sandbox-paths": "command-ref/conf-file.html#extra-sandbox-paths",
"sec-conf-file": "command-ref/conf-file.html",
"env-NIX_PATH": "command-ref/env-common.html#env-NIX_PATH",
"env-common": "command-ref/env-common.html",
"envar-remote": "command-ref/env-common.html#env-NIX_REMOTE",
"sec-common-env": "command-ref/env-common.html",
"ch-files": "command-ref/files.html",
"ch-main-commands": "command-ref/main-commands.html",
"opt-out-link": "command-ref/nix-build.html#opt-out-link",
"sec-nix-build": "command-ref/nix-build.html",
"sec-nix-channel": "command-ref/nix-channel.html",
"sec-nix-collect-garbage": "command-ref/nix-collect-garbage.html",
"sec-nix-copy-closure": "command-ref/nix-copy-closure.html",
"sec-nix-daemon": "command-ref/nix-daemon.html",
"refsec-nix-env-install-examples": "command-ref/nix-env.html#examples",
"rsec-nix-env-install": "command-ref/nix-env.html#operation---install",
"rsec-nix-env-set": "command-ref/nix-env.html#operation---set",
"rsec-nix-env-set-flag": "command-ref/nix-env.html#operation---set-flag",
"rsec-nix-env-upgrade": "command-ref/nix-env.html#operation---upgrade",
"sec-nix-env": "command-ref/nix-env.html",
"ssec-version-comparisons": "command-ref/nix-env.html#versions",
"sec-nix-hash": "command-ref/nix-hash.html",
"sec-nix-instantiate": "command-ref/nix-instantiate.html",
"sec-nix-prefetch-url": "command-ref/nix-prefetch-url.html",
"sec-nix-shell": "command-ref/nix-shell.html",
"ssec-nix-shell-shebang": "command-ref/nix-shell.html#use-as-a--interpreter",
"nixref-queries": "command-ref/nix-store.html#queries",
"opt-add-root": "command-ref/nix-store.html#opt-add-root",
"refsec-nix-store-dump": "command-ref/nix-store.html#operation---dump",
"refsec-nix-store-export": "command-ref/nix-store.html#operation---export",
"refsec-nix-store-import": "command-ref/nix-store.html#operation---import",
"refsec-nix-store-query": "command-ref/nix-store.html#operation---query",
"refsec-nix-store-verify": "command-ref/nix-store.html#operation---verify",
"rsec-nix-store-gc": "command-ref/nix-store.html#operation---gc",
"rsec-nix-store-generate-binary-cache-key": "command-ref/nix-store.html#operation---generate-binary-cache-key",
"rsec-nix-store-realise": "command-ref/nix-store.html#operation---realise",
"rsec-nix-store-serve": "command-ref/nix-store.html#operation---serve",
"sec-nix-store": "command-ref/nix-store.html",
"opt-I": "command-ref/opt-common.html#opt-I",
"opt-attr": "command-ref/opt-common.html#opt-attr",
"opt-common": "command-ref/opt-common.html",
"opt-cores": "command-ref/opt-common.html#opt-cores",
"opt-log-format": "command-ref/opt-common.html#opt-log-format",
"opt-max-jobs": "command-ref/opt-common.html#opt-max-jobs",
"opt-max-silent-time": "command-ref/opt-common.html#opt-max-silent-time",
"opt-timeout": "command-ref/opt-common.html#opt-timeout",
"sec-common-options": "command-ref/opt-common.html",
"ch-utilities": "command-ref/utilities.html",
"chap-hacking": "contributing/hacking.html",
"adv-attr-allowSubstitutes": "language/advanced-attributes.html#adv-attr-allowSubstitutes",
"adv-attr-allowedReferences": "language/advanced-attributes.html#adv-attr-allowedReferences",
"adv-attr-allowedRequisites": "language/advanced-attributes.html#adv-attr-allowedRequisites",
"adv-attr-disallowedReferences": "language/advanced-attributes.html#adv-attr-disallowedReferences",
"adv-attr-disallowedRequisites": "language/advanced-attributes.html#adv-attr-disallowedRequisites",
"adv-attr-exportReferencesGraph": "language/advanced-attributes.html#adv-attr-exportReferencesGraph",
"adv-attr-impureEnvVars": "language/advanced-attributes.html#adv-attr-impureEnvVars",
"adv-attr-outputHash": "language/advanced-attributes.html#adv-attr-outputHash",
"adv-attr-outputHashAlgo": "language/advanced-attributes.html#adv-attr-outputHashAlgo",
"adv-attr-outputHashMode": "language/advanced-attributes.html#adv-attr-outputHashMode",
"adv-attr-passAsFile": "language/advanced-attributes.html#adv-attr-passAsFile",
"adv-attr-preferLocalBuild": "language/advanced-attributes.html#adv-attr-preferLocalBuild",
"fixed-output-drvs": "language/advanced-attributes.html#adv-attr-outputHash",
"sec-advanced-attributes": "language/advanced-attributes.html",
"builtin-abort": "language/builtins.html#builtins-abort",
"builtin-add": "language/builtins.html#builtins-add",
"builtin-all": "language/builtins.html#builtins-all",
"builtin-any": "language/builtins.html#builtins-any",
"builtin-attrNames": "language/builtins.html#builtins-attrNames",
"builtin-attrValues": "language/builtins.html#builtins-attrValues",
"builtin-baseNameOf": "language/builtins.html#builtins-baseNameOf",
"builtin-bitAnd": "language/builtins.html#builtins-bitAnd",
"builtin-bitOr": "language/builtins.html#builtins-bitOr",
"builtin-bitXor": "language/builtins.html#builtins-bitXor",
"builtin-builtins": "language/builtins.html#builtins-builtins",
"builtin-compareVersions": "language/builtins.html#builtins-compareVersions",
"builtin-concatLists": "language/builtins.html#builtins-concatLists",
"builtin-concatStringsSep": "language/builtins.html#builtins-concatStringsSep",
"builtin-currentSystem": "language/builtins.html#builtins-currentSystem",
"builtin-deepSeq": "language/builtins.html#builtins-deepSeq",
"builtin-derivation": "language/builtins.html#builtins-derivation",
"builtin-dirOf": "language/builtins.html#builtins-dirOf",
"builtin-div": "language/builtins.html#builtins-div",
"builtin-elem": "language/builtins.html#builtins-elem",
"builtin-elemAt": "language/builtins.html#builtins-elemAt",
"builtin-fetchGit": "language/builtins.html#builtins-fetchGit",
"builtin-fetchTarball": "language/builtins.html#builtins-fetchTarball",
"builtin-fetchurl": "language/builtins.html#builtins-fetchurl",
"builtin-filterSource": "language/builtins.html#builtins-filterSource",
"builtin-foldl-prime": "language/builtins.html#builtins-foldl-prime",
"builtin-fromJSON": "language/builtins.html#builtins-fromJSON",
"builtin-functionArgs": "language/builtins.html#builtins-functionArgs",
"builtin-genList": "language/builtins.html#builtins-genList",
"builtin-getAttr": "language/builtins.html#builtins-getAttr",
"builtin-getEnv": "language/builtins.html#builtins-getEnv",
"builtin-hasAttr": "language/builtins.html#builtins-hasAttr",
"builtin-hashFile": "language/builtins.html#builtins-hashFile",
"builtin-hashString": "language/builtins.html#builtins-hashString",
"builtin-head": "language/builtins.html#builtins-head",
"builtin-import": "language/builtins.html#builtins-import",
"builtin-intersectAttrs": "language/builtins.html#builtins-intersectAttrs",
"builtin-isAttrs": "language/builtins.html#builtins-isAttrs",
"builtin-isBool": "language/builtins.html#builtins-isBool",
"builtin-isFloat": "language/builtins.html#builtins-isFloat",
"builtin-isFunction": "language/builtins.html#builtins-isFunction",
"builtin-isInt": "language/builtins.html#builtins-isInt",
"builtin-isList": "language/builtins.html#builtins-isList",
"builtin-isNull": "language/builtins.html#builtins-isNull",
"builtin-isString": "language/builtins.html#builtins-isString",
"builtin-length": "language/builtins.html#builtins-length",
"builtin-lessThan": "language/builtins.html#builtins-lessThan",
"builtin-listToAttrs": "language/builtins.html#builtins-listToAttrs",
"builtin-map": "language/builtins.html#builtins-map",
"builtin-match": "language/builtins.html#builtins-match",
"builtin-mul": "language/builtins.html#builtins-mul",
"builtin-parseDrvName": "language/builtins.html#builtins-parseDrvName",
"builtin-path": "language/builtins.html#builtins-path",
"builtin-pathExists": "language/builtins.html#builtins-pathExists",
"builtin-placeholder": "language/builtins.html#builtins-placeholder",
"builtin-readDir": "language/builtins.html#builtins-readDir",
"builtin-readFile": "language/builtins.html#builtins-readFile",
"builtin-removeAttrs": "language/builtins.html#builtins-removeAttrs",
"builtin-replaceStrings": "language/builtins.html#builtins-replaceStrings",
"builtin-seq": "language/builtins.html#builtins-seq",
"builtin-sort": "language/builtins.html#builtins-sort",
"builtin-split": "language/builtins.html#builtins-split",
"builtin-splitVersion": "language/builtins.html#builtins-splitVersion",
"builtin-stringLength": "language/builtins.html#builtins-stringLength",
"builtin-sub": "language/builtins.html#builtins-sub",
"builtin-substring": "language/builtins.html#builtins-substring",
"builtin-tail": "language/builtins.html#builtins-tail",
"builtin-throw": "language/builtins.html#builtins-throw",
"builtin-toFile": "language/builtins.html#builtins-toFile",
"builtin-toJSON": "language/builtins.html#builtins-toJSON",
"builtin-toPath": "language/builtins.html#builtins-toPath",
"builtin-toString": "language/builtins.html#builtins-toString",
"builtin-toXML": "language/builtins.html#builtins-toXML",
"builtin-trace": "language/builtins.html#builtins-trace",
"builtin-tryEval": "language/builtins.html#builtins-tryEval",
"builtin-typeOf": "language/builtins.html#builtins-typeOf",
"ssec-builtins": "language/builtins.html",
"attr-system": "language/derivations.html#attr-system",
"ssec-derivation": "language/derivations.html",
"ch-expression-language": "language/index.html",
"sec-constructs": "language/constructs.html",
"sect-let-language": "language/constructs.html#let-language",
"ss-functions": "language/constructs.html#functions",
"sec-language-operators": "language/operators.html",
"table-operators": "language/operators.html",
"ssec-values": "language/values.html",
"gloss-closure": "glossary.html#gloss-closure",
"gloss-derivation": "glossary.html#gloss-derivation",
"gloss-deriver": "glossary.html#gloss-deriver",
"gloss-nar": "glossary.html#gloss-nar",
"gloss-output-path": "glossary.html#gloss-output-path",
"gloss-profile": "glossary.html#gloss-profile",
"gloss-reachable": "glossary.html#gloss-reachable",
"gloss-reference": "glossary.html#gloss-reference",
"gloss-substitute": "glossary.html#gloss-substitute",
"gloss-user-env": "glossary.html#gloss-user-env",
"gloss-validity": "glossary.html#gloss-validity",
"part-glossary": "glossary.html",
"sec-building-source": "installation/building-source.html",
"ch-env-variables": "installation/env-variables.html",
"sec-installer-proxy-settings": "installation/env-variables.html#proxy-environment-variables",
"sec-nix-ssl-cert-file": "installation/env-variables.html#nix_ssl_cert_file",
"sec-nix-ssl-cert-file-with-nix-daemon-and-macos": "installation/env-variables.html#nix_ssl_cert_file-with-macos-and-the-nix-daemon",
"chap-installation": "installation/installation.html",
"ch-installing-binary": "installation/installing-binary.html",
"sect-macos-installation": "installation/installing-binary.html#macos-installation",
"sect-macos-installation-change-store-prefix": "installation/installing-binary.html#macos-installation",
"sect-macos-installation-encrypted-volume": "installation/installing-binary.html#macos-installation",
"sect-macos-installation-recommended-notes": "installation/installing-binary.html#macos-installation",
"sect-macos-installation-symlink": "installation/installing-binary.html#macos-installation",
"sect-multi-user-installation": "installation/installing-binary.html#multi-user-installation",
"sect-nix-install-binary-tarball": "installation/installing-binary.html#installing-from-a-binary-tarball",
"sect-nix-install-pinned-version-url": "installation/installing-binary.html#installing-a-pinned-nix-version-from-a-url",
"sect-single-user-installation": "installation/installing-binary.html#single-user-installation",
"ch-installing-source": "installation/installing-source.html",
"ssec-multi-user": "installation/multi-user.html",
"ch-nix-security": "installation/nix-security.html",
"sec-obtaining-source": "installation/obtaining-source.html",
"sec-prerequisites-source": "installation/prerequisites-source.html",
"sec-single-user": "installation/single-user.html",
"ch-supported-platforms": "installation/supported-platforms.html",
"ch-upgrading-nix": "installation/upgrading.html",
"ch-about-nix": "introduction.html",
"chap-introduction": "introduction.html",
"ch-basic-package-mgmt": "package-management/basic-package-mgmt.html",
"ssec-binary-cache-substituter": "package-management/binary-cache-substituter.html",
"sec-channels": "command-ref/nix-channel.html",
"ssec-copy-closure": "package-management/copy-closure.html",
"sec-garbage-collection": "package-management/garbage-collection.html",
"ssec-gc-roots": "package-management/garbage-collector-roots.html",
"chap-package-management": "package-management/package-management.html",
"sec-profiles": "package-management/profiles.html",
"ssec-s3-substituter": "package-management/s3-substituter.html",
"ssec-s3-substituter-anonymous-reads": "package-management/s3-substituter.html#anonymous-reads-to-your-s3-compatible-binary-cache",
"ssec-s3-substituter-authenticated-reads": "package-management/s3-substituter.html#authenticated-reads-to-your-s3-binary-cache",
"ssec-s3-substituter-authenticated-writes": "package-management/s3-substituter.html#authenticated-writes-to-your-s3-compatible-binary-cache",
"sec-sharing-packages": "package-management/sharing-packages.html",
"ssec-ssh-substituter": "package-management/ssh-substituter.html",
"chap-quick-start": "quick-start.html",
"sec-relnotes": "release-notes/release-notes.html",
"ch-relnotes-0.10.1": "release-notes/rl-0.10.1.html",
"ch-relnotes-0.10": "release-notes/rl-0.10.html",
"ssec-relnotes-0.11": "release-notes/rl-0.11.html",
"ssec-relnotes-0.12": "release-notes/rl-0.12.html",
"ssec-relnotes-0.13": "release-notes/rl-0.13.html",
"ssec-relnotes-0.14": "release-notes/rl-0.14.html",
"ssec-relnotes-0.15": "release-notes/rl-0.15.html",
"ssec-relnotes-0.16": "release-notes/rl-0.16.html",
"ch-relnotes-0.5": "release-notes/rl-0.5.html",
"ch-relnotes-0.6": "release-notes/rl-0.6.html",
"ch-relnotes-0.7": "release-notes/rl-0.7.html",
"ch-relnotes-0.8.1": "release-notes/rl-0.8.1.html",
"ch-relnotes-0.8": "release-notes/rl-0.8.html",
"ch-relnotes-0.9.1": "release-notes/rl-0.9.1.html",
"ch-relnotes-0.9.2": "release-notes/rl-0.9.2.html",
"ch-relnotes-0.9": "release-notes/rl-0.9.html",
"ssec-relnotes-1.0": "release-notes/rl-1.0.html",
"ssec-relnotes-1.1": "release-notes/rl-1.1.html",
"ssec-relnotes-1.10": "release-notes/rl-1.10.html",
"ssec-relnotes-1.11.10": "release-notes/rl-1.11.10.html",
"ssec-relnotes-1.11": "release-notes/rl-1.11.html",
"ssec-relnotes-1.2": "release-notes/rl-1.2.html",
"ssec-relnotes-1.3": "release-notes/rl-1.3.html",
"ssec-relnotes-1.4": "release-notes/rl-1.4.html",
"ssec-relnotes-1.5.1": "release-notes/rl-1.5.1.html",
"ssec-relnotes-1.5.2": "release-notes/rl-1.5.2.html",
"ssec-relnotes-1.5": "release-notes/rl-1.5.html",
"ssec-relnotes-1.6.1": "release-notes/rl-1.6.1.html",
"ssec-relnotes-1.6.0": "release-notes/rl-1.6.html",
"ssec-relnotes-1.7": "release-notes/rl-1.7.html",
"ssec-relnotes-1.8": "release-notes/rl-1.8.html",
"ssec-relnotes-1.9": "release-notes/rl-1.9.html",
"ssec-relnotes-2.0": "release-notes/rl-2.0.html",
"ssec-relnotes-2.1": "release-notes/rl-2.1.html",
"ssec-relnotes-2.2": "release-notes/rl-2.2.html",
"ssec-relnotes-2.3": "release-notes/rl-2.3.html",
},
"language/values.html": {
"simple-values": "#primitives",
"lists": "#list",
"strings": "#string",
"lists": "#list",
"attribute-sets": "#attribute-set",
},
"installation/installing-binary.html": {
"linux": "uninstall.html#linux",
"macos": "uninstall.html#macos",
"uninstalling": "uninstall.html",
}
"contributing/hacking.html": {
"nix-with-flakes": "#building-nix-with-flakes",
"classic-nix": "#building-nix",
"running-tests": "testing.html#running-tests",
"unit-tests": "testing.html#unit-tests",
"functional-tests": "testing.html#functional-tests",
"debugging-failing-functional-tests": "testing.html#debugging-failing-functional-tests",
"integration-tests": "testing.html#integration-tests",
"installer-tests": "testing.html#installer-tests",
"one-time-setup": "testing.html#one-time-setup",
"using-the-ci-generated-installer-for-manual-testing": "testing.html#using-the-ci-generated-installer-for-manual-testing",
}
};
// the following code matches the current page's URL against the set of redirects.
//
// it is written to minimize the latency between page load and redirect.
// therefore we avoid function calls, copying data, and unnecessary loops.
// IMPORTANT: we use stateful array operations and their order matters!
//
// matching URLs is more involved than it should be:
//
// 1. `document.location.pathname` can have an arbitrary prefix.
//
// 2. `path_to_root` is set by mdBook. it consists only of `../`s and
// determines the depth of `<path>` relative to the prefix:
//
// `document.location.pathname`
// |------------------------------|
// /<prefix>/<path>/[<file>[.html]][#<anchor>]
// |----|
// `path_to_root` has same number of path segments
//
// source: https://phaiax.github.io/mdBook/format/theme/index-hbs.html#data
//
// 3. the following paths are equivalent:
//
// /foo/bar/
// /foo/bar/index.html
// /foo/bar/index
//
// 4. the following paths are also equivalent:
//
// /foo/bar/baz
// /foo/bar/baz.html
//
let segments = document.location.pathname.split('/');
let file = segments.pop();
// normalize file name
if (file === '') { file = "index.html"; }
else if (!file.endsWith('.html')) { file = file + '.html'; }
segments.push(file);
// use `path_to_root` to discern prefix from path.
const depth = path_to_root.split('/').length;
// remove segments containing prefix. the following works because
// 1. the original `document.location.pathname` is absolute,
// hence first element of `segments` is always empty.
// 2. last element of splitting `path_to_root` is also always empty.
// 3. last element of `segments` is the file name.
//
// visual example:
//
// '/foo/bar/baz.html'.split('/') -> [ '', 'foo', 'bar', 'baz.html' ]
// '../'.split('/') -> [ '..', '' ]
//
// the following operations will then result in
//
// path = 'bar/baz.html'
//
segments.splice(0, segments.length - depth);
const path = segments.join('/');
// anchor starts with the hash character (`#`),
// but our redirect declarations don't, so we strip it.
// example:
// document.location.hash -> '#foo'
// document.location.hash.substring(1) -> 'foo'
const anchor = document.location.hash.substring(1);
const redirect = redirects[path];
if (redirect) {
const target = redirect[anchor];
if (target) {
document.location.href = target;
}
}
+25
View File
@@ -0,0 +1,25 @@
#!/usr/bin/env bash
set -euo pipefail
lowdown_args=
if [ "$1" = --out-no-smarty ]; then
lowdown_args=--out-no-smarty
shift
fi
[ "$#" = 4 ] || {
echo "wrong number of args passed" >&2
exit 1
}
title="$1"
section="$2"
infile="$3"
outfile="$4"
(
printf "Title: %s\n\n" "$title"
cat "$infile"
) | lowdown -sT man --nroff-nolinks $lowdown_args -M section="$section" -o "$outfile"
View File
+12
View File
@@ -0,0 +1,12 @@
---
synopsis: "Add a `build-dir` setting to set the backing directory for builds"
cls: 1514
credits: [roberth, tomberek]
category: Improvements
---
`build-dir` can now be set in the Nix configuration to choose the backing directory for the build sandbox.
This can be useful on systems with `/tmp` on tmpfs, or simply to relocate large builds to another disk.
Also, `XDG_RUNTIME_DIR` is no longer considered when selecting the default temporary directory,
as it's not intended to be used for large amounts of data.
@@ -0,0 +1,14 @@
---
synopsis: Add log formats `multiline` and `multiline-with-logs`
cls: [1369]
credits: [kloenk]
category: Improvements
---
Added two new log formats (`multiline` and `multiline-with-logs`) that display
current activities below each other for better visibility.
These formats attempt to use the maximum available lines
(defaulting to 25 if unable to determine) and print up to that many lines.
The status bar is displayed as the first line, with each subsequent
activity on its own line.
+12
View File
@@ -0,0 +1,12 @@
---
synopsis: "`nix copy` is now several times faster at `querying info about /nix/store/...`"
cls: [1462]
issues: [fj#366]
credits: [jade]
category: Fixes
---
We fixed a locking bug that serialized `querying info about /nix/store/...`
onto just one thread such that it was eating `O(paths to copy * latency)` time
while setting up to copy paths to s3 and other stores. It is now `nproc` times
faster.
@@ -0,0 +1,21 @@
---
synopsis: "Lix no longer speaks the Nix remote-build worker protocol to clients or servers older than CppNix 2.3"
cls: [1207, 1208, 1206, 1205, 1204, 1203, 1479]
issues: [fj#325]
credits: [jade]
category: Breaking Changes
---
CppNix 2.3 was released in 2019, and is the new oldest supported version. We
will increase our support baseline in the future up to a final version of CppNix
2.18 (which may happen soon given that it is the only still-packaged and thus
still-tested >2.3 version), but this step already removes a significant amount
of dead, untested, code paths.
Lix speaks the same version of the protocol as CppNix 2.18 and that fact will
never change in the future; the Lix plans to replace the protocol for evolution
will entail a complete incompatible replacement that will be supported in
parallel with the old protocol. Lix will thus retain remote build compatibility
with CppNix as long as CppNix maintains protocol compatibility with 2.18, and
as long as Lix retains legacy protocol support (which will likely be a long
time given that we plan to convert it to a frozen-in-time shim).
@@ -0,0 +1,8 @@
---
synopsis: "`nix repl` now allows tab-completing the special repl :colon commands"
cls: 1367
credits: Qyriad
category: Improvements
---
The REPL (`nix repl`) supports pressing `<TAB>` to complete a partial expression, but now also supports completing the special :colon commands as well (`:b`, `:edit`, `:doc`, etc), if the line starts with a colon.
+10
View File
@@ -0,0 +1,10 @@
---
synopsis: "Lix now supports building with UndefinedBehaviorSanitizer"
cls: [1483]
credits: [jade]
category: Development
---
You can now build Lix with the configuration option `-Db_sanitize=undefined` and it will both work and pass tests. AddressSanitizer support is also coming soon.
For a list of undefined behaviour fixed by sanitizer usage, see [the gerrit topic "undefined-behaviour"](https://gerrit.lix.systems/q/topic:%22undefined-behaviour%22).
+250
View File
@@ -0,0 +1,250 @@
# Table of Contents
- [Introduction](introduction.md)
- [Quick Start](quick-start.md)
- [Installation](installation/installation.md)
- [Supported Platforms](installation/supported-platforms.md)
- [Installing a Binary Distribution](installation/installing-binary.md)
- [Installing Lix from Source](installation/installing-source.md)
- [Prerequisites](installation/prerequisites-source.md)
- [Obtaining a Source Distribution](installation/obtaining-source.md)
- [Building Lix from Source](installation/building-source.md)
- [Using Lix within Docker](installation/installing-docker.md)
- [Security](installation/nix-security.md)
- [Single-User Mode](installation/single-user.md)
- [Multi-User Mode](installation/multi-user.md)
- [Environment Variables](installation/env-variables.md)
- [Upgrading Lix](installation/upgrading.md)
- [Uninstalling Lix](installation/uninstall.md)
- [Package Management](package-management/package-management.md)
- [Basic Package Management](package-management/basic-package-mgmt.md)
- [Profiles](package-management/profiles.md)
- [Garbage Collection](package-management/garbage-collection.md)
- [Garbage Collector Roots](package-management/garbage-collector-roots.md)
- [Sharing Packages Between Machines](package-management/sharing-packages.md)
- [Serving a Nix store via HTTP](package-management/binary-cache-substituter.md)
- [Copying Closures via SSH](package-management/copy-closure.md)
- [Serving a Nix store via SSH](package-management/ssh-substituter.md)
- [Serving a Nix store via S3](package-management/s3-substituter.md)
- [Nix Language](language/index.md)
- [Data Types](language/values.md)
- [Language Constructs](language/constructs.md)
- [String interpolation](language/string-interpolation.md)
- [Operators](language/operators.md)
- [Derivations](language/derivations.md)
- [Advanced Attributes](language/advanced-attributes.md)
- [Built-in Constants](language/builtin-constants.md)
- [Built-in Functions](language/builtins.md)
- [Advanced Topics](advanced-topics/advanced-topics.md)
- [Remote Builds](advanced-topics/distributed-builds.md)
- [Tuning Cores and Jobs](advanced-topics/cores-vs-jobs.md)
- [Verifying Build Reproducibility](advanced-topics/diff-hook.md)
- [Using the `post-build-hook`](advanced-topics/post-build-hook.md)
- [Command Reference](command-ref/command-ref.md)
- [Common Options](command-ref/opt-common.md)
- [Common Environment Variables](command-ref/env-common.md)
- [Main Commands](command-ref/main-commands.md)
- [nix-build](command-ref/nix-build.md)
- [nix-shell](command-ref/nix-shell.md)
- [nix-store](command-ref/nix-store.md)
- [nix-store --add-fixed](command-ref/nix-store/add-fixed.md)
- [nix-store --add](command-ref/nix-store/add.md)
- [nix-store --delete](command-ref/nix-store/delete.md)
- [nix-store --dump-db](command-ref/nix-store/dump-db.md)
- [nix-store --dump](command-ref/nix-store/dump.md)
- [nix-store --export](command-ref/nix-store/export.md)
- [nix-store --gc](command-ref/nix-store/gc.md)
- [nix-store --generate-binary-cache-key](command-ref/nix-store/generate-binary-cache-key.md)
- [nix-store --import](command-ref/nix-store/import.md)
- [nix-store --load-db](command-ref/nix-store/load-db.md)
- [nix-store --optimise](command-ref/nix-store/optimise.md)
- [nix-store --print-env](command-ref/nix-store/print-env.md)
- [nix-store --query](command-ref/nix-store/query.md)
- [nix-store --read-log](command-ref/nix-store/read-log.md)
- [nix-store --realise](command-ref/nix-store/realise.md)
- [nix-store --repair-path](command-ref/nix-store/repair-path.md)
- [nix-store --restore](command-ref/nix-store/restore.md)
- [nix-store --serve](command-ref/nix-store/serve.md)
- [nix-store --verify-path](command-ref/nix-store/verify-path.md)
- [nix-store --verify](command-ref/nix-store/verify.md)
- [nix-env](command-ref/nix-env.md)
- [nix-env --delete-generations](command-ref/nix-env/delete-generations.md)
- [nix-env --install](command-ref/nix-env/install.md)
- [nix-env --list-generations](command-ref/nix-env/list-generations.md)
- [nix-env --query](command-ref/nix-env/query.md)
- [nix-env --rollback](command-ref/nix-env/rollback.md)
- [nix-env --set-flag](command-ref/nix-env/set-flag.md)
- [nix-env --set](command-ref/nix-env/set.md)
- [nix-env --switch-generation](command-ref/nix-env/switch-generation.md)
- [nix-env --switch-profile](command-ref/nix-env/switch-profile.md)
- [nix-env --uninstall](command-ref/nix-env/uninstall.md)
- [nix-env --upgrade](command-ref/nix-env/upgrade.md)
- [Utilities](command-ref/utilities.md)
- [nix-channel](command-ref/nix-channel.md)
- [nix-collect-garbage](command-ref/nix-collect-garbage.md)
- [nix-copy-closure](command-ref/nix-copy-closure.md)
- [nix-daemon](command-ref/nix-daemon.md)
- [nix-hash](command-ref/nix-hash.md)
- [nix-instantiate](command-ref/nix-instantiate.md)
- [nix-prefetch-url](command-ref/nix-prefetch-url.md)
- [Experimental Commands](command-ref/experimental-commands.md)
- [nix](command-ref/new-cli/nix.md)
- [nix build](command-ref/new-cli/nix3-build.md)
- [nix bundle](command-ref/new-cli/nix3-bundle.md)
- [nix config](command-ref/new-cli/nix3-config.md)
- [nix config show](command-ref/new-cli/nix3-config-show.md)
- [nix copy](command-ref/new-cli/nix3-copy.md)
- [nix daemon](command-ref/new-cli/nix3-daemon.md)
- [nix derivation](command-ref/new-cli/nix3-derivation.md)
- [nix derivation add](command-ref/new-cli/nix3-derivation-add.md)
- [nix derivation show](command-ref/new-cli/nix3-derivation-show.md)
- [nix develop](command-ref/new-cli/nix3-develop.md)
- [nix doctor](command-ref/new-cli/nix3-doctor.md)
- [nix edit](command-ref/new-cli/nix3-edit.md)
- [nix eval](command-ref/new-cli/nix3-eval.md)
- [nix flake](command-ref/new-cli/nix3-flake.md)
- [nix flake archive](command-ref/new-cli/nix3-flake-archive.md)
- [nix flake check](command-ref/new-cli/nix3-flake-check.md)
- [nix flake clone](command-ref/new-cli/nix3-flake-clone.md)
- [nix flake info](command-ref/new-cli/nix3-flake-info.md)
- [nix flake init](command-ref/new-cli/nix3-flake-init.md)
- [nix flake lock](command-ref/new-cli/nix3-flake-lock.md)
- [nix flake metadata](command-ref/new-cli/nix3-flake-metadata.md)
- [nix flake new](command-ref/new-cli/nix3-flake-new.md)
- [nix flake prefetch](command-ref/new-cli/nix3-flake-prefetch.md)
- [nix flake show](command-ref/new-cli/nix3-flake-show.md)
- [nix flake update](command-ref/new-cli/nix3-flake-update.md)
- [nix fmt](command-ref/new-cli/nix3-fmt.md)
- [nix hash](command-ref/new-cli/nix3-hash.md)
- [nix hash file](command-ref/new-cli/nix3-hash-file.md)
- [nix hash path](command-ref/new-cli/nix3-hash-path.md)
- [nix hash to-base16](command-ref/new-cli/nix3-hash-to-base16.md)
- [nix hash to-base32](command-ref/new-cli/nix3-hash-to-base32.md)
- [nix hash to-base64](command-ref/new-cli/nix3-hash-to-base64.md)
- [nix hash to-sri](command-ref/new-cli/nix3-hash-to-sri.md)
- [nix help](command-ref/new-cli/nix3-help.md)
- [nix help-stores](command-ref/new-cli/nix3-help-stores.md)
- [nix key](command-ref/new-cli/nix3-key.md)
- [nix key convert-secret-to-public](command-ref/new-cli/nix3-key-convert-secret-to-public.md)
- [nix key generate-secret](command-ref/new-cli/nix3-key-generate-secret.md)
- [nix log](command-ref/new-cli/nix3-log.md)
- [nix nar](command-ref/new-cli/nix3-nar.md)
- [nix nar cat](command-ref/new-cli/nix3-nar-cat.md)
- [nix nar dump-path](command-ref/new-cli/nix3-nar-dump-path.md)
- [nix nar ls](command-ref/new-cli/nix3-nar-ls.md)
- [nix path-info](command-ref/new-cli/nix3-path-info.md)
- [nix print-dev-env](command-ref/new-cli/nix3-print-dev-env.md)
- [nix profile](command-ref/new-cli/nix3-profile.md)
- [nix profile diff-closures](command-ref/new-cli/nix3-profile-diff-closures.md)
- [nix profile history](command-ref/new-cli/nix3-profile-history.md)
- [nix profile install](command-ref/new-cli/nix3-profile-install.md)
- [nix profile list](command-ref/new-cli/nix3-profile-list.md)
- [nix profile remove](command-ref/new-cli/nix3-profile-remove.md)
- [nix profile rollback](command-ref/new-cli/nix3-profile-rollback.md)
- [nix profile upgrade](command-ref/new-cli/nix3-profile-upgrade.md)
- [nix profile wipe-history](command-ref/new-cli/nix3-profile-wipe-history.md)
- [nix realisation](command-ref/new-cli/nix3-realisation.md)
- [nix realisation info](command-ref/new-cli/nix3-realisation-info.md)
- [nix registry](command-ref/new-cli/nix3-registry.md)
- [nix registry add](command-ref/new-cli/nix3-registry-add.md)
- [nix registry list](command-ref/new-cli/nix3-registry-list.md)
- [nix registry pin](command-ref/new-cli/nix3-registry-pin.md)
- [nix registry remove](command-ref/new-cli/nix3-registry-remove.md)
- [nix repl](command-ref/new-cli/nix3-repl.md)
- [nix run](command-ref/new-cli/nix3-run.md)
- [nix search](command-ref/new-cli/nix3-search.md)
- [nix shell](command-ref/new-cli/nix3-shell.md)
- [nix store](command-ref/new-cli/nix3-store.md)
- [nix store add-file](command-ref/new-cli/nix3-store-add-file.md)
- [nix store add-path](command-ref/new-cli/nix3-store-add-path.md)
- [nix store cat](command-ref/new-cli/nix3-store-cat.md)
- [nix store copy-log](command-ref/new-cli/nix3-store-copy-log.md)
- [nix store copy-sigs](command-ref/new-cli/nix3-store-copy-sigs.md)
- [nix store delete](command-ref/new-cli/nix3-store-delete.md)
- [nix store diff-closures](command-ref/new-cli/nix3-store-diff-closures.md)
- [nix store dump-path](command-ref/new-cli/nix3-store-dump-path.md)
- [nix store gc](command-ref/new-cli/nix3-store-gc.md)
- [nix store ls](command-ref/new-cli/nix3-store-ls.md)
- [nix store make-content-addressed](command-ref/new-cli/nix3-store-make-content-addressed.md)
- [nix store optimise](command-ref/new-cli/nix3-store-optimise.md)
- [nix store path-from-hash-part](command-ref/new-cli/nix3-store-path-from-hash-part.md)
- [nix store ping](command-ref/new-cli/nix3-store-ping.md)
- [nix store prefetch-file](command-ref/new-cli/nix3-store-prefetch-file.md)
- [nix store repair](command-ref/new-cli/nix3-store-repair.md)
- [nix store sign](command-ref/new-cli/nix3-store-sign.md)
- [nix store verify](command-ref/new-cli/nix3-store-verify.md)
- [nix upgrade-nix](command-ref/new-cli/nix3-upgrade-nix.md)
- [nix why-depends](command-ref/new-cli/nix3-why-depends.md)
- [Files](command-ref/files.md)
- [nix.conf](command-ref/conf-file.md)
- [Profiles](command-ref/files/profiles.md)
- [manifest.nix](command-ref/files/manifest.nix.md)
- [manifest.json](command-ref/files/manifest.json.md)
- [Channels](command-ref/files/channels.md)
- [Default Nix expression](command-ref/files/default-nix-expression.md)
- [Architecture and Design](architecture/architecture.md)
- [File System Object](architecture/file-system-object.md)
- [Protocols](protocols/protocols.md)
- [Serving Tarball Flakes](protocols/tarball-fetcher.md)
- [Derivation "ATerm" file format](protocols/derivation-aterm.md)
- [Glossary](glossary.md)
- [Contributing](contributing/contributing.md)
- [Hacking](contributing/hacking.md)
- [Testing](contributing/testing.md)
- [Experimental Features](contributing/experimental-features.md)
- [CLI guideline](contributing/cli-guideline.md)
- [C++ style guide](contributing/cxx.md)
- [Release Notes](release-notes/release-notes.md)
- [Upcoming release](release-notes/rl-next.md)
<!-- RELENG-AUTO-INSERTION-MARKER (see releng/release_notes.py) -->
- [Lix 2.90 (FIXME date)](release-notes/rl-2.90.md)
- [Nix 2.18 (2023-09-20)](release-notes/rl-2.18.md)
- [Nix 2.17 (2023-07-24)](release-notes/rl-2.17.md)
- [Nix 2.16 (2023-05-31)](release-notes/rl-2.16.md)
- [Nix 2.15 (2023-04-11)](release-notes/rl-2.15.md)
- [Nix 2.14 (2023-02-28)](release-notes/rl-2.14.md)
- [Nix 2.13 (2023-01-17)](release-notes/rl-2.13.md)
- [Nix 2.12 (2022-12-06)](release-notes/rl-2.12.md)
- [Nix 2.11 (2022-08-25)](release-notes/rl-2.11.md)
- [Nix 2.10 (2022-07-11)](release-notes/rl-2.10.md)
- [Nix 2.9 (2022-05-30)](release-notes/rl-2.9.md)
- [Nix 2.8 (2022-04-19)](release-notes/rl-2.8.md)
- [Nix 2.7 (2022-03-07)](release-notes/rl-2.7.md)
- [Nix 2.6 (2022-01-24)](release-notes/rl-2.6.md)
- [Nix 2.5 (2021-12-13)](release-notes/rl-2.5.md)
- [Nix 2.4 (2021-11-01)](release-notes/rl-2.4.md)
- [Nix 2.3 (2019-09-04)](release-notes/rl-2.3.md)
- [Nix 2.2 (2019-01-11)](release-notes/rl-2.2.md)
- [Nix 2.1 (2018-09-02)](release-notes/rl-2.1.md)
- [Nix 2.0 (2018-02-22)](release-notes/rl-2.0.md)
- [Nix 1.11.10 (2017-06-12)](release-notes/rl-1.11.10.md)
- [Nix 1.11 (2016-01-19)](release-notes/rl-1.11.md)
- [Nix 1.10 (2015-09-03)](release-notes/rl-1.10.md)
- [Nix 1.9 (2015-06-12)](release-notes/rl-1.9.md)
- [Nix 1.8 (2014-12-14)](release-notes/rl-1.8.md)
- [Nix 1.7 (2014-04-11)](release-notes/rl-1.7.md)
- [Nix 1.6.1 (2013-10-28)](release-notes/rl-1.6.1.md)
- [Nix 1.6 (2013-09-10)](release-notes/rl-1.6.md)
- [Nix 1.5.2 (2013-05-13)](release-notes/rl-1.5.2.md)
- [Nix 1.5 (2013-02-27)](release-notes/rl-1.5.md)
- [Nix 1.4 (2013-02-26)](release-notes/rl-1.4.md)
- [Nix 1.3 (2013-01-04)](release-notes/rl-1.3.md)
- [Nix 1.2 (2012-12-06)](release-notes/rl-1.2.md)
- [Nix 1.1 (2012-07-18)](release-notes/rl-1.1.md)
- [Nix 1.0 (2012-05-11)](release-notes/rl-1.0.md)
- [Nix 0.16 (2010-08-17)](release-notes/rl-0.16.md)
- [Nix 0.15 (2010-03-17)](release-notes/rl-0.15.md)
- [Nix 0.14 (2010-02-04)](release-notes/rl-0.14.md)
- [Nix 0.13 (2009-11-05)](release-notes/rl-0.13.md)
- [Nix 0.12 (2008-11-20)](release-notes/rl-0.12.md)
- [Nix 0.11 (2007-12-31)](release-notes/rl-0.11.md)
- [Nix 0.10.1 (2006-10-11)](release-notes/rl-0.10.1.md)
- [Nix 0.10 (2006-10-06)](release-notes/rl-0.10.md)
- [Nix 0.9.2 (2005-09-21)](release-notes/rl-0.9.2.md)
- [Nix 0.9.1 (2005-09-20)](release-notes/rl-0.9.1.md)
- [Nix 0.9 (2005-09-16)](release-notes/rl-0.9.md)
- [Nix 0.8.1 (2005-04-13)](release-notes/rl-0.8.1.md)
- [Nix 0.8 (2005-04-11)](release-notes/rl-0.8.md)
- [Nix 0.7 (2005-01-12)](release-notes/rl-0.7.md)
- [Nix 0.6 (2004-11-14)](release-notes/rl-0.6.md)
- [Nix 0.5 and earlier](release-notes/rl-0.5.md)
+29
View File
@@ -0,0 +1,29 @@
# redirect rules for paths (server-side) to prevent link rot.
# see ./redirects.js for redirects based on URL fragments (client-side)
#
# concrete user story this supports:
# - user finds URL to the manual for Nix x.y
# - Nix x.z (z > y) is the most recent release
# - updating the version in the URL will show the right thing
#
# format documentation:
# - https://docs.netlify.com/routing/redirects/#syntax-for-the-redirects-file
# - https://docs.netlify.com/routing/redirects/redirect-options/
#
# conventions:
# - always force (<CODE>!) since this allows re-using file names
# - group related paths to ease readability
# - always append new redirects to the end of the file
# - redirects that should have been there but are missing can be inserted where they belong
/expressions/expression-language /language/ 301!
/expressions/language-values /language/values 301!
/expressions/language-constructs /language/constructs 301!
/expressions/language-operators /language/operators 301!
/expressions/* /language/:splat 301!
/package-management/basic-package-mgmt /command-ref/nix-env 301!
/package-management/channels* /command-ref/nix-channel 301!
/package-management/s3-substituter* /command-ref/new-cli/nix3-help-stores#s3-binary-cache-store 301!
@@ -0,0 +1 @@
This section lists advanced topics related to builds and builds performance
@@ -0,0 +1,39 @@
# Tuning Cores and Jobs
Lix has two relevant settings with regards to how your CPU cores will
be utilized: `cores` and `max-jobs`. This chapter will talk about what
they are, how they interact, and their configuration trade-offs.
- `max-jobs`\
Dictates how many separate derivations will be built at the same
time. If you set this to zero, the local machine will do no
builds. Lix will still substitute from binary caches, and build
remotely if remote builders are configured.
- `cores`\
Suggests how many cores each derivation should use. Similar to
`make -j`.
The `cores` setting determines the value of
`NIX_BUILD_CORES`. `NIX_BUILD_CORES` is equal to `cores`, unless
`cores` equals `0`, in which case `NIX_BUILD_CORES` will be the total
number of cores in the system.
The maximum number of consumed cores is a simple multiplication,
`max-jobs` \* `NIX_BUILD_CORES`.
The balance on how to set these two independent variables depends upon
each builder's workload and hardware. Here are a few example scenarios
on a machine with 24 cores:
| `max-jobs` | `cores` | `NIX_BUILD_CORES` | Maximum Processes | Result |
| --------------------- | ------------------ | ----------------- | ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 1 | 24 | 24 | 24 | One derivation will be built at a time, each one can use 24 cores. Undersold if a job cant use 24 cores. |
| 4 | 6 | 6 | 24 | Four derivations will be built at once, each given access to six cores. |
| 12 | 6 | 6 | 72 | 12 derivations will be built at once, each given access to six cores. This configuration is over-sold. If all 12 derivations being built simultaneously try to use all six cores, the machine's performance will be degraded due to extensive context switching between the 12 builds. |
| 24 | 1 | 1 | 24 | 24 derivations can build at the same time, each using a single core. Never oversold, but derivations which require many cores will be very slow to compile. |
| 24 | 0 | 24 | 576 | 24 derivations can build at the same time, each using all the available cores of the machine. Very likely to be oversold, and very likely to suffer context switches. |
It is up to the derivations' build script to respect host's requested
cores-per-build by following the value of the `NIX_BUILD_CORES`
environment variable.
+123
View File
@@ -0,0 +1,123 @@
# Verifying Build Reproducibility
You can use Lix's `diff-hook` setting to compare build results. Note
that this hook is only executed if the results differ; it is not used
for determining if the results are the same.
For purposes of demonstration, we'll use the following Nix file,
`deterministic.nix` for testing:
```nix
let
inherit (import <nixpkgs> {}) runCommand;
in {
stable = runCommand "stable" {} ''
touch $out
'';
unstable = runCommand "unstable" {} ''
echo $RANDOM > $out
'';
}
```
Additionally, `nix.conf` contains:
diff-hook = /etc/nix/my-diff-hook
run-diff-hook = true
where `/etc/nix/my-diff-hook` is an executable file containing:
```bash
#!/bin/sh
exec >&2
echo "For derivation $3:"
/run/current-system/sw/bin/diff -r "$1" "$2"
```
The diff hook is executed by the same user and group who ran the build.
However, the diff hook does not have write access to the store path just
built.
# Spot-Checking Build Determinism
Verify a path which already exists in the Nix store by passing `--check`
to the build command.
If the build passes and is deterministic, Lix will exit with a status
code of 0:
```console
$ nix-build ./deterministic.nix --attr stable
this derivation will be built:
/nix/store/z98fasz2jqy9gs0xbvdj939p27jwda38-stable.drv
building '/nix/store/z98fasz2jqy9gs0xbvdj939p27jwda38-stable.drv'...
/nix/store/yyxlzw3vqaas7wfp04g0b1xg51f2czgq-stable
$ nix-build ./deterministic.nix --attr stable --check
checking outputs of '/nix/store/z98fasz2jqy9gs0xbvdj939p27jwda38-stable.drv'...
/nix/store/yyxlzw3vqaas7wfp04g0b1xg51f2czgq-stable
```
If the build is not deterministic, Lix will exit with a status code of
1:
```console
$ nix-build ./deterministic.nix --attr unstable
this derivation will be built:
/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv
building '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv'...
/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable
$ nix-build ./deterministic.nix --attr unstable --check
checking outputs of '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv'...
error: derivation '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv' may
not be deterministic: output '/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable' differs
```
In the Lix daemon's log, we will now see:
```
For derivation /nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv:
1c1
< 8108
---
> 30204
```
Using `--check` with `--keep-failed` will cause Lix to keep the second
build's output in a special, `.check` path:
```console
$ nix-build ./deterministic.nix --attr unstable --check --keep-failed
checking outputs of '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv'...
note: keeping build directory '/tmp/nix-build-unstable.drv-0'
error: derivation '/nix/store/cgl13lbj1w368r5z8gywipl1ifli7dhk-unstable.drv' may
not be deterministic: output '/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable' differs
from '/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable.check'
```
In particular, notice the
`/nix/store/krpqk0l9ib0ibi1d2w52z293zw455cap-unstable.check` output. Lix
has copied the build results to that directory where you can examine it.
> []{#check-dirs-are-unregistered} **Note**
>
> Check paths are not protected against garbage collection, and this
> path will be deleted on the next garbage collection.
>
> The path is guaranteed to be alive for the duration of
> the `diff-hook`'s execution, but may be deleted any time after.
>
> If the comparison is performed as part of automated tooling, please
> use the diff-hook or author your tooling to handle the case where the
> build was not deterministic and also a check path does not exist.
`--check` is only usable if the derivation has been built on the system
already. If the derivation has not been built Lix will fail with the
error:
error: some outputs of '/nix/store/hzi1h60z2qf0nb85iwnpvrai3j2w7rr6-unstable.drv'
are not valid, so checking is not possible
Run the build without `--check`, and then try with `--check` again.
@@ -0,0 +1,159 @@
# Remote Builds
Lix supports remote builds, where a local Lix installation can forward
Nix builds to other machines. This allows multiple builds to be
performed in parallel and allows Lix to perform multi-platform builds in
a semi-transparent way. For instance, if you perform a build for a
`x86_64-darwin` on an `i686-linux` machine, Lix can automatically
forward the build to a `x86_64-darwin` machine, if available.
To forward a build to a remote machine, its required that the remote
machine is accessible via SSH and that it has Nix installed. You can
test whether connecting to the remote Nix instance works, e.g.
```console
$ nix store ping --store ssh://mac
```
will try to connect to the machine named `mac`. It is possible to
specify an SSH identity file as part of the remote store URI, e.g.
```console
$ nix store ping --store ssh://mac?ssh-key=/home/alice/my-key
```
Since builds should be non-interactive, the key should not have a
passphrase. Alternatively, you can load identities ahead of time into
`ssh-agent` or `gpg-agent`.
If you get the error
```console
bash: nix-store: command not found
error: cannot connect to 'mac'
```
then you need to ensure that the `PATH` of non-interactive login shells
contains Nix.
> **Warning**
>
> If you are building via the Lix daemon (default on Linux and macOS), it is the Lix daemon user account (that is, `root`) that should have SSH access to a user (not necessarily `root`) on the remote machine.
>
> Furthermore, `root` needs to have the public host keys for the remote system in its `.ssh/known_hosts`.
> To add them to `known_hosts` for root, do `ssh-keyscan USER@HOST | sudo tee -a ~root/.ssh/known_hosts`.
>
> If you cant or dont want to configure `root` to be able to access the remote machine, you can use a private Nix store instead by passing e.g. `--store ~/my-nix` when running a Nix command from the local machine.
The list of remote machines can be specified on the command line or in
the Lix configuration file. The former is convenient for testing. For
example, the following command allows you to build a derivation for
`x86_64-darwin` on a Linux machine:
```console
$ uname
Linux
$ nix build --impure \
--expr '(with import <nixpkgs> { system = "x86_64-darwin"; }; runCommand "foo" {} "uname > $out")' \
--builders 'ssh://mac x86_64-darwin'
[1/0/1 built, 0.0 MiB DL] building foo on ssh://mac
$ cat ./result
Darwin
```
It is possible to specify multiple builders separated by a semicolon or
a newline, e.g.
```console
--builders 'ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd'
```
Each machine specification consists of the following elements, separated
by spaces. Only the first element is required. To leave a field at its
default, set it to `-`.
1. The URI of the remote store in the format
`ssh://[username@]hostname`, e.g. `ssh://nix@mac` or `ssh://mac`.
For backward compatibility, `ssh://` may be omitted. The hostname
may be an alias defined in your `~/.ssh/config`.
2. A comma-separated list of Nix platform type identifiers, such as
`x86_64-darwin`. It is possible for a machine to support multiple
platform types, e.g., `i686-linux,x86_64-linux`. If omitted, this
defaults to the local platform type.
3. The SSH identity file to be used to log in to the remote machine. If
omitted, SSH will use its regular identities.
4. The maximum number of builds that Lix will execute in parallel on
the machine. Typically this should be equal to the number of CPU
cores. For instance, the machine `itchy` in the example will execute
up to 8 builds in parallel.
5. The “speed factor”, indicating the relative speed of the machine. If
there are multiple machines of the right type, Lix will prefer the
fastest, taking load into account.
6. A comma-separated list of *supported features*. If a derivation has
the `requiredSystemFeatures` attribute, then Lix will only perform
the derivation on a machine that has the specified features. For
instance, the attribute
```nix
requiredSystemFeatures = [ "kvm" ];
```
will cause the build to be performed on a machine that has the `kvm`
feature.
7. A comma-separated list of *mandatory features*. A machine will only
be used to build a derivation if all of the machines mandatory
features appear in the derivations `requiredSystemFeatures`
attribute.
8. The (base64-encoded) public host key of the remote machine. If omitted, SSH
will use its regular known-hosts file. Specifically, the field is calculated
via `base64 -w0 /etc/ssh/ssh_host_ed25519_key.pub`.
For example, the machine specification
nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm
nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2
nix@poochie.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 1 2 kvm benchmark
specifies several machines that can perform `i686-linux` builds.
However, `poochie` will only do builds that have the attribute
```nix
requiredSystemFeatures = [ "benchmark" ];
```
or
```nix
requiredSystemFeatures = [ "benchmark" "kvm" ];
```
`itchy` cannot do builds that require `kvm`, but `scratchy` does support
such builds. For regular builds, `itchy` will be preferred over
`scratchy` because it has a higher speed factor.
Remote builders can also be configured in `nix.conf`, e.g.
builders = ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd
Finally, remote builders can be configured in a separate configuration
file included in `builders` via the syntax `@file`. For example,
builders = @/etc/nix/machines
causes the list of machines in `/etc/nix/machines` to be included. (This
is the default.)
If you want the builders to use caches, you likely want to set the
option `builders-use-substitutes` in your local `nix.conf`.
To build only on remote builders and disable building on the local
machine, you can use the option `--max-jobs 0`.
@@ -0,0 +1,126 @@
# Using the `post-build-hook`
# Implementation Caveats
Here we use the post-build hook to upload to a binary cache. This is a
simple and working example, but it is not suitable for all use cases.
The post build hook program runs after each executed build, and blocks
the build loop. The build loop exits if the hook program fails.
Concretely, this implementation will make Lix slow or unusable when the
internet is slow or unreliable.
A more advanced implementation might pass the store paths to a
user-supplied daemon or queue for processing the store paths outside of
the build loop.
# Prerequisites
This tutorial assumes you have [configured an S3-compatible binary
cache](../package-management/s3-substituter.md), and that the `root`
user's default AWS profile can upload to the bucket.
# Set up a Signing Key
Use `nix-store --generate-binary-cache-key` to create our public and
private signing keys. We will sign paths with the private key, and
distribute the public key for verifying the authenticity of the paths.
```console
# nix-store --generate-binary-cache-key example-nix-cache-1 /etc/nix/key.private /etc/nix/key.public
# cat /etc/nix/key.public
example-nix-cache-1:1/cKDz3QCCOmwcztD2eV6Coggp6rqc9DGjWv7C0G+rM=
```
Then update [`nix.conf`](../command-ref/conf-file.md) on any machine that will access the cache.
Add the cache URL to [`substituters`](../command-ref/conf-file.md#conf-substituters) and the public key to [`trusted-public-keys`](../command-ref/conf-file.md#conf-trusted-public-keys):
substituters = https://cache.nixos.org/ s3://example-nix-cache
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= example-nix-cache-1:1/cKDz3QCCOmwcztD2eV6Coggp6rqc9DGjWv7C0G+rM=
Machines that build for the cache must sign derivations using the private key.
On those machines, add the path to the key file to the [`secret-key-files`](../command-ref/conf-file.md#conf-secret-key-files) field in their [`nix.conf`](../command-ref/conf-file.md):
secret-key-files = /etc/nix/key.private
We will restart the Nix daemon in a later step.
# Implementing the build hook
Write the following script to `/etc/nix/upload-to-cache.sh`:
```bash
#!/bin/sh
set -eu
set -f # disable globbing
export IFS=' '
echo "Uploading paths" $OUT_PATHS
exec nix copy --to "s3://example-nix-cache" $OUT_PATHS
```
> **Note**
>
> The `$OUT_PATHS` variable is a space-separated list of Nix store
> paths. In this case, we expect and want the shell to perform word
> splitting to make each output path its own argument to `nix
> store sign`. Nix guarantees the paths will not contain any spaces,
> however a store path might contain glob characters. The `set -f`
> disables globbing in the shell.
Then make sure the hook program is executable by the `root` user:
```console
# chmod +x /etc/nix/upload-to-cache.sh
```
# Updating Lix Configuration
Edit `/etc/nix/nix.conf` to run our hook, by adding the following
configuration snippet at the end:
post-build-hook = /etc/nix/upload-to-cache.sh
Then, restart the `nix-daemon`.
# Testing
Build any derivation, for example:
```console
$ nix-build --expr '(import <nixpkgs> {}).writeText "example" (builtins.toString builtins.currentTime)'
this derivation will be built:
/nix/store/s4pnfbkalzy5qz57qs6yybna8wylkig6-example.drv
building '/nix/store/s4pnfbkalzy5qz57qs6yybna8wylkig6-example.drv'...
running post-build-hook '/home/grahamc/projects/github.com/NixOS/nix/post-hook.sh'...
post-build-hook: Signing paths /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
post-build-hook: Uploading paths /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
/nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
```
Then delete the path from the store, and try substituting it from the
binary cache:
```console
$ rm ./result
$ nix-store --delete /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
```
Now, copy the path back from the cache:
```console
$ nix-store --realise /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example
copying path '/nix/store/m8bmqwrch6l3h8s0k3d673xpmipcdpsa-example from 's3://example-nix-cache'...
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/m8bmqwrch6l3h8s0k3d673xpmipcdpsa-example
```
# Conclusion
We now have a Lix installation configured to automatically sign and
upload every local build to a remote binary cache.
Before deploying this to production, be sure to consider the
[implementation caveats](#implementation-caveats).
+114
View File
@@ -0,0 +1,114 @@
# Architecture
This chapter describes how Nix works.
It should help users understand why Lix behaves as it does, and it should help developers understand how to modify Lix and how to write similar tools.
## Overview
Nix consists of [hierarchical layers].
[hierarchical layers]: https://en.wikipedia.org/wiki/Multitier_architecture#Layers
The following [concept map] shows its main components (rectangles), the objects they operate on (rounded rectangles), and their interactions (connecting phrases):
[concept map]: https://en.wikipedia.org/wiki/Concept_map
```
.----------------.
| Nix expression |----------.
'----------------' |
| passed to
| |
+----------|-------------------|--------------------------------+
| Nix impl.| V |
| (Lix) | +-------------------------+ |
| | | commmand line interface |------. |
| | +-------------------------+ | |
| | | | |
| evaluated by calls manages |
| | | | |
| | V | |
| | +--------------------+ | |
| '-------->| language evaluator | | |
| +--------------------+ | |
| | | |
| produces | |
| | V |
| +----------------------------|------------------------------+ |
| | store | | |
| | referenced by V builds | |
| | .-------------. .------------. .--------------. | |
| | | build input |----->| build plan |----->| build result | | |
| | '-------------' '------------' '--------------' | |
| +-------------------------------------------------|---------+ |
+---------------------------------------------------|-----------+
|
represented as
|
V
.---------------.
| file |
'---------------'
```
At the top is the [command line interface](../command-ref/command-ref.md) that drives the underlying layers.
The [Nix language](../language/index.md) evaluator transforms Nix expressions into self-contained *build plans*, which are used to derive *build results* from referenced *build inputs*.
The command line interface and Nix expressions are what users deal with most.
> **Note**
> The Nix language itself does not have a notion of *packages* or *configurations*.
> As far as we are concerned here, the inputs and results of a build plan are just data.
Underlying the command line interface and the Nix language evaluator is the [Nix store](../glossary.md#gloss-store), a mechanism to keep track of build plans, data, and references between them.
It can also execute build plans to produce new data, which are made available to the operating system as files.
A build plan itself is a series of *build tasks*, together with their build inputs.
> **Important**
> A build task in Nix is called [derivation](../glossary.md#gloss-derivation).
Each build task has a special build input executed as *build instructions* in order to perform the build.
The result of a build task can be input to another build task.
The following [data flow diagram] shows a build plan for illustration.
Build inputs used as instructions to a build task are marked accordingly:
[data flow diagram]: https://en.wikipedia.org/wiki/Data-flow_diagram
```
+--------------------------------------------------------------------+
| build plan |
| |
| .-------------. |
| | build input |---------. |
| '-------------' | |
| instructions |
| | |
| v |
| .-------------. .----------. |
| | build input |-->( build task )-------. |
| '-------------' '----------' | |
| instructions |
| | |
| v |
| .-------------. .----------. .--------------. |
| | build input |---------. ( build task )--->| build result | |
| '-------------' | '----------' '--------------' |
| instructions ^ |
| | | |
| v | |
| .-------------. .----------. | |
| | build input |-->( build task )-------' |
| '-------------' '----------' |
| ^ |
| | |
| | |
| .-------------. | |
| | build input |---------' |
| '-------------' |
| |
+--------------------------------------------------------------------+
```
@@ -0,0 +1,64 @@
# File System Object
Nix implementations use a simplified model of the file system, which consists of file system objects.
Every file system object is one of the following:
- File
- A possibly empty sequence of bytes for contents
- A single boolean representing the [executable](https://en.m.wikipedia.org/wiki/File-system_permissions#Permissions) permission
- Directory
Mapping of names to child file system objects
- [Symbolic link](https://en.m.wikipedia.org/wiki/Symbolic_link)
An arbitrary string.
Nix implementations do not assign any semantics to symbolic links.
File system objects and their children form a tree.
A bare file or symlink can be a root file system object.
Nix does not encode any other file system notions such as [hard links](https://en.m.wikipedia.org/wiki/Hard_link), [permissions](https://en.m.wikipedia.org/wiki/File-system_permissions), timestamps, or other metadata.
## Examples of file system objects
A plain file:
```
50 B, executable: false
```
An executable file:
```
122 KB, executable: true
```
A symlink:
```
-> /usr/bin/sh
```
A directory with contents:
```
├── bin
│   └── hello: 35 KB, executable: true
└── share
├── info
│   └── hello.info: 36 KB, executable: false
└── man
└── man1
└── hello.1.gz: 790 B, executable: false
```
A directory that contains a symlink and other directories:
```
├── bin -> share/go/bin
├── nix-support/
└── share/
```
@@ -0,0 +1,2 @@
This section lists commands and options that you can use when you work
with Lix.
+71
View File
@@ -0,0 +1,71 @@
# Name
`nix.conf` - Lix configuration file
# Description
Lix supports a variety of configuration settings, which are read from configuration files or taken as command line flags.
## Configuration file
By default Lix reads settings from the following places, in that order:
1. The system-wide configuration file `sysconfdir/nix/nix.conf` (i.e. `/etc/nix/nix.conf` on most systems), or `$NIX_CONF_DIR/nix.conf` if [`NIX_CONF_DIR`](./env-common.md#env-NIX_CONF_DIR) is set.
Values loaded in this file are not forwarded to the Nix daemon.
The client assumes that the daemon has already loaded them.
1. If [`NIX_USER_CONF_FILES`](./env-common.md#env-NIX_USER_CONF_FILES) is set, then each path separated by `:` will be loaded in reverse order.
Otherwise it will look for `nix/nix.conf` files in `XDG_CONFIG_DIRS` and `XDG_CONFIG_HOME`.
If unset, `XDG_CONFIG_DIRS` defaults to `/etc/xdg`, and `XDG_CONFIG_HOME` defaults to `$HOME/.config` as per [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html).
1. If [`NIX_CONFIG`](./env-common.md#env-NIX_CONFIG) is set, its contents are treated as the contents of a configuration file.
### File format
Configuration files consist of `name = value` pairs, one per line.
Comments start with a `#` character.
Example:
```
keep-outputs = true # Nice for developers
keep-derivations = true # Idem
```
Other files can be included with a line like `include <path>`, where `<path>` is interpreted relative to the current configuration file.
A missing file is an error unless `!include` is used instead.
A configuration setting usually overrides any previous value.
However, for settings that take a list of items, you can prefix the name of the setting by `extra-` to *append* to the previous value.
For instance,
```
substituters = a b
extra-substituters = c d
```
defines the `substituters` setting to be `a b c d`.
Unknown option names are not an error, and are simply ignored with a warning.
## Command line flags
Configuration options can be set on the command line, overriding the values set in the [configuration file](#configuration-file):
- Every configuration setting has corresponding command line flag (e.g. `--max-jobs 16`).
Boolean settings do not need an argument, and can be explicitly disabled with the `no-` prefix (e.g. `--keep-failed` and `--no-keep-failed`).
Unknown option names are invalid flags (unless there is already a flag with that name), and are rejected with an error.
- The flag `--option <name> <value>` is interpreted exactly like a `<name> = <value>` in a setting file.
Unknown option names are ignored with a warning.
The `extra-` prefix is supported for settings that take a list of items (e.g. `--extra-trusted users alice` or `--option extra-trusted-users alice`).
# Available settings
{{#include @generated@/command-ref/conf-file.md}}
+124
View File
@@ -0,0 +1,124 @@
# Common Environment Variables
Most commands in Lix interpret the following environment variables:
- <span id="env-IN_NIX_SHELL">[`IN_NIX_SHELL`](#env-IN_NIX_SHELL)</span>\
Indicator that tells if the current environment was set up by
`nix-shell`. It can have the values `pure` or `impure`.
- <span id="env-NIX_PATH">[`NIX_PATH`](#env-NIX_PATH)</span>\
A colon-separated list of directories used to look up the location of Nix
expressions using [paths](@docroot@/language/values.md#type-path)
enclosed in angle brackets (i.e., `<path>`),
e.g. `/home/eelco/Dev:/etc/nixos`. It can be extended using the
[`-I` option](@docroot@/command-ref/opt-common.md#opt-I).
If `NIX_PATH` is not set at all, Lix will fall back to the following list in [impure](@docroot@/command-ref/conf-file.md#conf-pure-eval) and [unrestricted](@docroot@/command-ref/conf-file.md#conf-restrict-eval) evaluation mode:
1. `$HOME/.nix-defexpr/channels`
2. `nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixpkgs`
3. `/nix/var/nix/profiles/per-user/root/channels`
If `NIX_PATH` is set to an empty string, resolving search paths will always fail.
For example, attempting to use `<nixpkgs>` will produce:
error: file 'nixpkgs' was not found in the Nix search path
- <span id="env-NIX_IGNORE_SYMLINK_STORE">[`NIX_IGNORE_SYMLINK_STORE`](#env-NIX_IGNORE_SYMLINK_STORE)</span>\
Normally, the Nix store directory (typically `/nix/store`) is not
allowed to contain any symlink components. This is to prevent
“impure” builds. Builders sometimes “canonicalise” paths by
resolving all symlink components. Thus, builds on different machines
(with `/nix/store` resolving to different locations) could yield
different results. This is generally not a problem, except when
builds are deployed to machines where `/nix/store` resolves
differently. If you are sure that youre not going to do that, you
can set `NIX_IGNORE_SYMLINK_STORE` to `1`.
Note that if youre symlinking the Nix store so that you can put it
on another file system than the root file system, on Linux youre
better off using `bind` mount points, e.g.,
```console
$ mkdir /nix
$ mount -o bind /mnt/otherdisk/nix /nix
```
Consult the mount 8 manual page for details.
- <span id="env-NIX_STORE_DIR">[`NIX_STORE_DIR`](#env-NIX_STORE_DIR)</span>\
Overrides the location of the Nix store (default `prefix/store`).
- <span id="env-NIX_DATA_DIR">[`NIX_DATA_DIR`](#env-NIX_DATA_DIR)</span>\
Overrides the location of the Lix static data directory (default
`prefix/share`).
- <span id="env-NIX_LOG_DIR">[`NIX_LOG_DIR`](#env-NIX_LOG_DIR)</span>\
Overrides the location of the Lix log directory (default
`prefix/var/log/nix`).
- <span id="env-NIX_STATE_DIR">[`NIX_STATE_DIR`](#env-NIX_STATE_DIR)</span>\
Overrides the location of the Lix state directory (default
`prefix/var/nix`).
- <span id="env-NIX_CONF_DIR">[`NIX_CONF_DIR`](#env-NIX_CONF_DIR)</span>\
Overrides the location of the system Lix configuration directory
(default `prefix/etc/nix`).
- <span id="env-NIX_CONFIG">[`NIX_CONFIG`](#env-NIX_CONFIG)</span>\
Applies settings from Lix configuration from the environment.
The content is treated as if it was read from a Lix configuration file.
Settings are separated by the newline character.
- <span id="env-NIX_USER_CONF_FILES">[`NIX_USER_CONF_FILES`](#env-NIX_USER_CONF_FILES)</span>\
Overrides the location of the Lix user configuration files to load from.
The default are the locations according to the [XDG Base Directory Specification].
See the [XDG Base Directories](#xdg-base-directories) sub-section for details.
The variable is treated as a list separated by the `:` token.
- <span id="env-TMPDIR">[`TMPDIR`](#env-TMPDIR)</span>\
Use the specified directory to store temporary files. In particular,
this includes temporary build directories; these can take up
substantial amounts of disk space. The default is `/tmp`.
- <span id="env-NIX_REMOTE">[`NIX_REMOTE`](#env-NIX_REMOTE)</span>\
This variable should be set to `daemon` if you want to use the Lix
daemon to execute Nix operations. This is necessary in [multi-user
Nix installations](@docroot@/installation/multi-user.md). If the Lix
daemon's Unix socket is at some non-standard path, this variable
should be set to `unix://path/to/socket`. Otherwise, it should be
left unset.
- <span id="env-NIX_SHOW_STATS">[`NIX_SHOW_STATS`](#env-NIX_SHOW_STATS)</span>\
If set to `1`, Lix will print some evaluation statistics, such as
the number of values allocated.
- <span id="env-NIX_COUNT_CALLS">[`NIX_COUNT_CALLS`](#env-NIX_COUNT_CALLS)</span>\
If set to `1`, Lix will print how often functions were called during
Nix expression evaluation. This is useful for profiling your Nix
expressions.
- <span id="env-GC_INITIAL_HEAP_SIZE">[`GC_INITIAL_HEAP_SIZE`](#env-GC_INITIAL_HEAP_SIZE)</span>\
If Nix has been configured to use the Boehm garbage collector, this
variable sets the initial size of the heap in bytes. It defaults to
384 MiB. Setting it to a low value reduces memory consumption, but
will increase runtime due to the overhead of garbage collection.
## XDG Base Directories
Lix follows the [XDG Base Directory Specification].
For backwards compatibility, commands in Lix will follow the standard only when [`use-xdg-base-directories`] is enabled.
[New Nix commands](@docroot@/command-ref/new-cli/nix.md) (experimental) conform to the standard by default.
The following environment variables are used to determine locations of various state and configuration files:
- `XDG_CONFIG_HOME` (default `~/.config`)
- `XDG_STATE_HOME` (default `~/.local/state`)
- `XDG_CACHE_HOME` (default `~/.cache`)
[XDG Base Directory Specification]: https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
[`use-xdg-base-directories`]: @docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories
@@ -0,0 +1,8 @@
# Experimental Commands
This section lists [experimental commands](@docroot@/contributing/experimental-features.md#xp-feature-nix-command).
> **Warning**
>
> These commands may be removed in the future, or their syntax may
> change in incompatible ways.
+4
View File
@@ -0,0 +1,4 @@
# Files
This section lists configuration files that you can use when you work
with Lix.
@@ -0,0 +1,26 @@
## Channels
A directory containing symlinks to Nix channels, managed by [`nix-channel`]:
- `$XDG_STATE_HOME/nix/profiles/channels` for regular users
- `$NIX_STATE_DIR/profiles/per-user/root/channels` for `root`
[`nix-channel`] uses a [profile](@docroot@/command-ref/files/profiles.md) to store channels.
This profile contains symlinks to the contents of those channels.
## Subscribed channels
The list of subscribed channels is stored in
- `~/.nix-channels`
- `$XDG_STATE_HOME/nix/channels` if [`use-xdg-base-directories`] is set to `true`
in the following format:
```
<url> <name>
...
```
[`nix-channel`]: @docroot@/command-ref/nix-channel.md
[`use-xdg-base-directories`]: @docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories
@@ -0,0 +1,52 @@
## Default Nix expression
The source for the default [Nix expressions](@docroot@/language/index.md) used by [`nix-env`]:
- `~/.nix-defexpr`
- `$XDG_STATE_HOME/nix/defexpr` if [`use-xdg-base-directories`] is set to `true`.
It is loaded as follows:
- If the default expression is a file, it is loaded as a Nix expression.
- If the default expression is a directory containing a `default.nix` file, that `default.nix` file is loaded as a Nix expression.
- If the default expression is a directory without a `default.nix` file, then its contents (both files and subdirectories) are loaded as Nix expressions.
The expressions are combined into a single attribute set, each expression under an attribute with the same name as the original file or subdirectory.
Subdirectories without a `default.nix` file are traversed recursively in search of more Nix expressions, but the names of these intermediate directories are not added to the attribute paths of the default Nix expression.
Then, the resulting expression is interpreted like this:
- If the expression is an attribute set, it is used as the default Nix expression.
- If the expression is a function, an empty set is passed as argument and the return value is used as the default Nix expression.
For example, if the default expression contains two files, `foo.nix` and `bar.nix`, then the default Nix expression will be equivalent to
```nix
{
foo = import ~/.nix-defexpr/foo.nix;
bar = import ~/.nix-defexpr/bar.nix;
}
```
The file [`manifest.nix`](@docroot@/command-ref/files/manifest.nix.md) is always ignored.
The command [`nix-channel`] places a symlink to the user's current [channels profile](@docroot@/command-ref/files/channels.md) in this directory.
This makes all subscribed channels available as attributes in the default expression.
## User channel link
A symlink that ensures that [`nix-env`] can find your channels:
- `~/.nix-defexpr/channels`
- `$XDG_STATE_HOME/defexpr/channels` if [`use-xdg-base-directories`] is set to `true`.
This symlink points to:
- `$XDG_STATE_HOME/profiles/channels` for regular users
- `$NIX_STATE_DIR/profiles/per-user/root/channels` for `root`
In a multi-user installation, you may also have `~/.nix-defexpr/channels_root`, which links to the channels of the root user.[`nix-env`]: ../nix-env.md
[`nix-env`]: @docroot@/command-ref/nix-env.md
[`nix-channel`]: @docroot@/command-ref/nix-channel.md
[`use-xdg-base-directories`]: @docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories
@@ -0,0 +1,45 @@
## `manifest.json`
The manifest file records the provenance of the packages that are installed in a [profile](./profiles.md) managed by [`nix profile`](@docroot@/command-ref/new-cli/nix3-profile.md) (experimental).
Here is an example of what the file might look like after installing `zoom-us` from Nixpkgs:
```json
{
"version": 1,
"elements": [
{
"active": true,
"attrPath": "legacyPackages.x86_64-linux.zoom-us",
"originalUrl": "flake:nixpkgs",
"storePaths": [
"/nix/store/wbhg2ga8f3h87s9h5k0slxk0m81m4cxl-zoom-us-5.3.469451.0927"
],
"uri": "github:NixOS/nixpkgs/13d0c311e3ae923a00f734b43fd1d35b47d8943a"
},
]
}
```
Each object in the array `elements` denotes an installed package and
has the following fields:
* `originalUrl`: The [flake reference](@docroot@/command-ref/new-cli/nix3-flake.md) specified by
the user at the time of installation (e.g. `nixpkgs`). This is also
the flake reference that will be used by `nix profile upgrade`.
* `uri`: The locked flake reference to which `originalUrl` resolved.
* `attrPath`: The flake output attribute that provided this
package. Note that this is not necessarily the attribute that the
user specified, but the one resulting from applying the default
attribute paths and prefixes; for instance, `hello` might resolve to
`packages.x86_64-linux.hello` and the empty string to
`packages.x86_64-linux.default`.
* `storePath`: The paths in the Nix store containing the package.
* `active`: Whether the profile contains symlinks to the files of this
package. If set to false, the package is kept in the Nix store, but
is not "visible" in the profile's symlink tree.
@@ -0,0 +1,128 @@
## `manifest.nix`
The manifest file records the provenance of the packages that are installed in a [profile](./profiles.md) managed by [`nix-env`](@docroot@/command-ref/nix-env.md).
Here is an example of how this file might look like after installing `hello` from Nixpkgs:
```nix
[{
meta = {
available = true;
broken = false;
changelog =
"https://git.savannah.gnu.org/cgit/hello.git/plain/NEWS?h=v2.12.1";
description = "A program that produces a familiar, friendly greeting";
homepage = "https://www.gnu.org/software/hello/manual/";
insecure = false;
license = {
deprecated = false;
free = true;
fullName = "GNU General Public License v3.0 or later";
redistributable = true;
shortName = "gpl3Plus";
spdxId = "GPL-3.0-or-later";
url = "https://spdx.org/licenses/GPL-3.0-or-later.html";
};
longDescription = ''
GNU Hello is a program that prints "Hello, world!" when you run it.
It is fully customizable.
'';
maintainers = [{
email = "edolstra+nixpkgs@gmail.com";
github = "edolstra";
githubId = 1148549;
name = "Eelco Dolstra";
}];
name = "hello-2.12.1";
outputsToInstall = [ "out" ];
platforms = [
"i686-cygwin"
"x86_64-cygwin"
"x86_64-darwin"
"i686-darwin"
"aarch64-darwin"
"armv7a-darwin"
"i686-freebsd13"
"x86_64-freebsd13"
"aarch64-genode"
"i686-genode"
"x86_64-genode"
"x86_64-solaris"
"js-ghcjs"
"aarch64-linux"
"armv5tel-linux"
"armv6l-linux"
"armv7a-linux"
"armv7l-linux"
"i686-linux"
"m68k-linux"
"microblaze-linux"
"microblazeel-linux"
"mipsel-linux"
"mips64el-linux"
"powerpc64-linux"
"powerpc64le-linux"
"riscv32-linux"
"riscv64-linux"
"s390-linux"
"s390x-linux"
"x86_64-linux"
"mmix-mmixware"
"aarch64-netbsd"
"armv6l-netbsd"
"armv7a-netbsd"
"armv7l-netbsd"
"i686-netbsd"
"m68k-netbsd"
"mipsel-netbsd"
"powerpc-netbsd"
"riscv32-netbsd"
"riscv64-netbsd"
"x86_64-netbsd"
"aarch64_be-none"
"aarch64-none"
"arm-none"
"armv6l-none"
"avr-none"
"i686-none"
"microblaze-none"
"microblazeel-none"
"msp430-none"
"or1k-none"
"m68k-none"
"powerpc-none"
"powerpcle-none"
"riscv32-none"
"riscv64-none"
"rx-none"
"s390-none"
"s390x-none"
"vc4-none"
"x86_64-none"
"i686-openbsd"
"x86_64-openbsd"
"x86_64-redox"
"wasm64-wasi"
"wasm32-wasi"
"x86_64-windows"
"i686-windows"
];
position =
"/nix/store/7niq32w715567hbph0q13m5lqna64c1s-nixos-unstable.tar.gz/nixos-unstable.tar.gz/pkgs/applications/misc/hello/default.nix:34";
unfree = false;
unsupported = false;
};
name = "hello-2.12.1";
out = {
outPath = "/nix/store/260q5867crm1xjs4khgqpl6vr9kywql1-hello-2.12.1";
};
outPath = "/nix/store/260q5867crm1xjs4khgqpl6vr9kywql1-hello-2.12.1";
outputs = [ "out" ];
system = "x86_64-linux";
type = "derivation";
}]
```
Each element in this list corresponds to an installed package.
It incorporates some attributes of the original derivation, including `meta`, `name`, `out`, `outPath`, `outputs`, `system`.
This information is used by Nix for querying and updating the package.
@@ -0,0 +1,74 @@
## Profiles
A directory that contains links to profiles managed by [`nix-env`] and [`nix profile`]:
- `$XDG_STATE_HOME/nix/profiles` for regular users
- `$NIX_STATE_DIR/profiles/per-user/root` if the user is `root`
A profile is a directory of symlinks to files in the Nix store.
### Filesystem layout
Profiles are versioned as follows. When using a profile named *path*, *path* is a symlink to *path*`-`*N*`-link`, where *N* is the version of the profile.
In turn, *path*`-`*N*`-link` is a symlink to a path in the Nix store.
For example:
```console
$ ls -l ~alice/.local/state/nix/profiles/profile*
lrwxrwxrwx 1 alice users 14 Nov 25 14:35 /home/alice/.local/state/nix/profiles/profile -> profile-7-link
lrwxrwxrwx 1 alice users 51 Oct 28 16:18 /home/alice/.local/state/nix/profiles/profile-5-link -> /nix/store/q69xad13ghpf7ir87h0b2gd28lafjj1j-profile
lrwxrwxrwx 1 alice users 51 Oct 29 13:20 /home/alice/.local/state/nix/profiles/profile-6-link -> /nix/store/6bvhpysd7vwz7k3b0pndn7ifi5xr32dg-profile
lrwxrwxrwx 1 alice users 51 Nov 25 14:35 /home/alice/.local/state/nix/profiles/profile-7-link -> /nix/store/mp0x6xnsg0b8qhswy6riqvimai4gm677-profile
```
Each of these symlinks is a root for the Lix garbage collector.
The contents of the store path corresponding to each version of the
profile is a tree of symlinks to the files of the installed packages,
e.g.
```console
$ ll -R ~eelco/.local/state/nix/profiles/profile-7-link/
/home/eelco/.local/state/nix/profiles/profile-7-link/:
total 20
dr-xr-xr-x 2 root root 4096 Jan 1 1970 bin
-r--r--r-- 2 root root 1402 Jan 1 1970 manifest.nix
dr-xr-xr-x 4 root root 4096 Jan 1 1970 share
/home/eelco/.local/state/nix/profiles/profile-7-link/bin:
total 20
lrwxrwxrwx 5 root root 79 Jan 1 1970 chromium -> /nix/store/ijm5k0zqisvkdwjkc77mb9qzb35xfi4m-chromium-86.0.4240.111/bin/chromium
lrwxrwxrwx 7 root root 87 Jan 1 1970 spotify -> /nix/store/w9182874m1bl56smps3m5zjj36jhp3rn-spotify-1.1.26.501.gbe11e53b-15/bin/spotify
lrwxrwxrwx 3 root root 79 Jan 1 1970 zoom-us -> /nix/store/wbhg2ga8f3h87s9h5k0slxk0m81m4cxl-zoom-us-5.3.469451.0927/bin/zoom-us
/home/eelco/.local/state/nix/profiles/profile-7-link/share/applications:
total 12
lrwxrwxrwx 4 root root 120 Jan 1 1970 chromium-browser.desktop -> /nix/store/4cf803y4vzfm3gyk3vzhzb2327v0kl8a-chromium-unwrapped-86.0.4240.111/share/applications/chromium-browser.desktop
lrwxrwxrwx 7 root root 110 Jan 1 1970 spotify.desktop -> /nix/store/w9182874m1bl56smps3m5zjj36jhp3rn-spotify-1.1.26.501.gbe11e53b-15/share/applications/spotify.desktop
lrwxrwxrwx 3 root root 107 Jan 1 1970 us.zoom.Zoom.desktop -> /nix/store/wbhg2ga8f3h87s9h5k0slxk0m81m4cxl-zoom-us-5.3.469451.0927/share/applications/us.zoom.Zoom.desktop
```
Each profile version contains a manifest file:
- [`manifest.nix`](@docroot@/command-ref/files/manifest.nix.md) used by [`nix-env`](@docroot@/command-ref/nix-env.md).
- [`manifest.json`](@docroot@/command-ref/files/manifest.json.md) used by [`nix profile`](@docroot@/command-ref/new-cli/nix3-profile.md) (experimental).
## User profile link
A symbolic link to the user's current profile:
- `~/.nix-profile`
- `$XDG_STATE_HOME/nix/profile` if [`use-xdg-base-directories`] is set to `true`.
By default, this symlink points to:
- `$XDG_STATE_HOME/nix/profiles/profile` for regular users
- `$NIX_STATE_DIR/profiles/per-user/root/profile` for `root`
The `PATH` environment variable should include `/bin` subdirectory of the profile link (e.g. `~/.nix-profile/bin`) for the user environment to be visible to the user.
The [installer](@docroot@/installation/installing-binary.md) sets this up by default, unless you enable [`use-xdg-base-directories`].
[`nix-env`]: @docroot@/command-ref/nix-env.md
[`nix profile`]: @docroot@/command-ref/new-cli/nix3-profile.md
[`use-xdg-base-directories`]: @docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories
@@ -0,0 +1,4 @@
# Main Commands
This section lists commands and options that you can use when you work
with Lix.
+66
View File
@@ -0,0 +1,66 @@
xp_features_json = custom_target(
command : [nix, '__dump-xp-features'],
capture : true,
output : 'xp-features.json',
)
experimental_features_shortlist_md = custom_target(
command : nix_eval_for_docs + [
'--expr',
'import @INPUT0@ (builtins.fromJSON (builtins.readFile @INPUT1@))',
],
input : [
'../../generate-xp-features-shortlist.nix',
xp_features_json,
],
capture : true,
output : 'experimental-features-shortlist.md',
env : nix_env_for_docs,
)
# Intermediate step for manpage generation.
# This splorks the output of generate-manpage.nix as JSON,
# which gets written as a directory tree below.
nix3_cli_files_json = custom_target(
command : nix_eval_for_docs_common + [
'--json',
'--expr',
'import @INPUT0@ true (builtins.readFile @INPUT1@)',
],
input : [
'../../generate-manpage.nix',
nix3_cli_json,
],
capture : true,
output : 'new-cli.json',
env : nix_env_for_docs,
)
nix3_cli_files = custom_target(
command : [
python,
'@INPUT0@',
'-i', '@INPUT1@',
'-o', '@OUTPUT@',
],
input : [
'../../json-to-tree.py',
nix3_cli_files_json,
],
output : 'new-cli',
)
conf_file_md = custom_target(
command : [
nix_eval_for_docs,
'--expr',
'(import @INPUT0@).showSettings { inlineHTML = true; } (builtins.fromJSON (builtins.readFile @INPUT1@))',
],
capture : true,
input : [
'../../utils.nix',
conf_file_json,
experimental_features_shortlist_md,
],
output : 'conf-file.md',
env : nix_env_for_docs,
)
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-build.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-bundle.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-config-show.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-config.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-copy.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-daemon.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-derivation-add.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-derivation-show.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-derivation.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-develop.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-doctor.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-edit.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-eval.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-archive.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-check.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-clone.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-info.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-init.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-lock.md}}
@@ -0,0 +1 @@
{{#include @generated@/command-ref/new-cli/nix3-flake-metadata.md}}

Some files were not shown because too many files have changed in this diff Show More