Compare commits

...
Author SHA1 Message Date
eldritch horrors 45623f077f libutil: drop Fs{Source,Sink}::good
setting this only on exceptions caused by actual fd access is not
sufficient to diagnose all errors (such as SerialisationError) in
some cases. this usually does not have any negative effects since
those errors will end up killing the process in another way. this
is not a reliable assumption though and we should be using proper
error handling (and closing connections more often, preferring to
close over keeping something open that might be in a weird state)

Change-Id: I1b792cd7ad8ba9ff0f6bd174945ab2575ff2208e
2024-03-31 16:42:40 +00:00
eldritch horrors f402c45cfa libutil: allow graceful dropping of Pool::Handle
not needed yet, but returning a resource from the exception handling
path that has ownership of a handle is currently not well-supported.
we could also add a default constructor to Handle, but then we would
also need to change the pool reference to a pointer. eventually that
should be done since now resources can be swapped between pools with
clever moves, but since that's not a problem yet we won't do it now.

Change-Id: I26eb06581f7be34569e9e67a33da736128d167af
2024-03-31 15:46:01 +00:00
jadeandGerrit Code Review 73507a7167 Merge changes Ib62d3d68,Ic3e7affe into main
* changes:
  Make things that can throw not noexcept anymore
  Fix various clang-tidy lints
2024-03-31 15:38:48 +00:00
jadeandGerrit Code Review 6165f21074 Merge "Add basic clang-tidy config" into main 2024-03-31 15:38:29 +00:00
eldritch horrors 2d3fd6b9ab libstore: using throwing finally in withFramedSink
the duplication of exception handling was added without justification,
so we can only assume that it was done like this because Finally could
not throw exceptions safely. since this has now been rectified we will
deduplicate this handler code again.

Change-Id: I40721f3378c0fd9f34e2914a16d383f6e2713b40
2024-03-31 14:23:15 +00:00
eldritch horrors dd06f9b792 libutil: make ~Finally noexcept(false)
this is supposed to act like a finally block does in other languages. a
finally block should be able to throw exceptions of its own rather than
just crashing the entire program when it throws it own exceptions. even
in the rare case of a finally throwing an unexpected exception it might
be better to report the exception from Finally instead of the original,
at least that can keep our program running instead of letting it crash.

Change-Id: Id42011e46b1df369152b4564938c0e93fa1acf32
2024-03-31 13:42:01 +00:00
eldritch horrors c777dcd1ae libstore: remove one Resource::good flag
usage of this flag previously kept connections open much longer than
necessary, and at the same time obscured that a connection was being
dropped when it *was* set. new variable names clarify this somewhat.

Change-Id: I11f6f08f37a5e4dc04ea6c6036ea589154b121c6
2024-03-31 00:52:41 +00:00
eldritch horrors 0b6d353474 libutil: remove Pool::Handle::bad
it was used incorrectly (not swapped on handle move), only used in one
place (that is now handled with exception handling detection in Handle
itself), and if ever reintroduced should be replaced with a different,
more understandable mechanism (like an explicit dropAsInvalid method).

Change-Id: Ie3e5d5cfa81d335429cb2ee5c3ad85c74a9df17b
2024-03-31 00:30:43 +00:00
eldritch horrors 862f20a4ba libutil: remove Pool::flushBad
this was never actually used, and bad design in the first place—why
should a bad resource be put back into the idle pool? just drop it.

Change-Id: Idab8774bee19dadae0209d404c4fb86dd4aeba1e
2024-03-31 00:07:09 +00:00
eldritch horrors 620de98d0c libutil: drop Pool resources on exceptional free
if a scope owning a resource does not gracefully drop that resource
while handling exceptions from deeper down the call stack we should
assume the resource is invalid state and drop it. currently it *is*
true that such cases do not cause resources to be freed, but thanks
to validator misuses this has so far not caused any larger problem.

Change-Id: Ie4f91bcd60a64d05c5ff9d22cc97954816d13b97
2024-03-30 23:40:45 +00:00
rebecca “wiggles” turnerandGerrit Code Review 4b730f328e Merge "Add pre-commit checks" into main 2024-03-30 22:26:59 +00:00
Qyriad 2041bdac2d perl: put perl dependencies in buildInputs instead of manually passing them to configure
Perl has an env hook[1]. Passing the paths manually without putting them
in buildInputs is harder to understand, plays less nicely with dev
shells, and is less build-generic.

Produced identical output on my x86_64-linux machine, and on my
aarch64-darwin machine was identical save for the derivation output path
which gets embedded into the .dylib Mach-O.

Change-Id: Ib313caa5a6f0b0e3154ce6f05379033920d0d290
2024-03-30 14:44:32 -06:00
eldritch horrors 5956f509b6 meson: install libexec binaries
Change-Id: I149892bf081e1569d7786f085e890bc3d2eb50e5
2024-03-30 14:34:22 +00:00
eldritch horrors 351c1a9c52 meson: build and install nix3 manpages
this notably does *not* install the `nix3-manpages` manpage the old
system generated, mostly because that page was empty and just a bug
with a coat of documentation paint.

Change-Id: I7a4248a72e7bb5e0cc925a6311a33b6b72589569
2024-03-30 14:26:47 +00:00
eldritch horrors bfbcc3352b meson: add manpages for old cli and config files
Change-Id: I93384ec774d1945a649f6aaf7cd967c3fb7197f9
2024-03-30 14:26:47 +00:00
eldritch horrors 65d544b1a6 build: extract make "functions" into scripts
we'll want to use these for the meson builds, and probably eventually
rewrite them in something that isn't plain shell. diffoscope confirms
that out/share and doc/share are equal before and after these changes

Change-Id: I49aa418fc8615cad86d67328e08c28a7405ec952
2024-03-30 14:26:47 +00:00
Rebecca Turner 2a98ba8b97 Add pre-commit checks
The big ones here are `trim-trailing-whitespace` and `end-of-file-fixer`
(which makes sure that every file ends with exactly one newline
character).

Change-Id: Idca73b640883188f068f9903e013cf0d82aa1123
2024-03-29 22:57:40 -07:00
Jade Lovelace 194a1b91af Make things that can throw not noexcept anymore
This does involve making a large number of destructors able to throw,
because we had to change it high in the class hierarchy. Oh well.

Change-Id: Ib62d3d6895b755f20322bb8acc9bf43daf0174b2
2024-03-29 20:26:38 -07:00
Jade Lovelace 99f159c536 Add basic clang-tidy config
This has not yet had all the warnings Obliterated, but it is a start and
is not *super* far away from being able to run the current configuration
in CI, which will catch some limited number of mistakes.

I tried the meson clang-tidy target and it seems to fail to find flags
for several files, which seems broken. Unsure what is up with that, but
we can use run-clang-tidy or other tooling instead.

We have an extremely annoying situation with the lexer table, which
means that the lexer probably must be moved to another directory with
its own .clang-tidy file to disable the lints in it, *or* write scuffed
code that prepends a disable comment to the top of the generated file.
None of the comment-based lint disabling features work since yacc dumps
a bunch of non compliant code at the top of the file before anything the
user can control.

Change-Id: I1d2aa6ec32deb1db1fbd581127334db1b972323c
2024-03-29 20:26:38 -07:00
Jade Lovelace 1fa6a3e335 Fix various clang-tidy lints
* some things that can throw are marked noexcept
  yet the linter seems to think not. Maybe they can't throw in practice.
  I would rather not have the UB possibility in pretty obvious cold
  paths.
* various default-case-missing complaints
* a fair pile of casts from integer to character, which are in fact
  deliberate.
* an instance of <https://clang.llvm.org/extra/clang-tidy/checks/bugprone/move-forwarding-reference.html>
* bugprone-not-null-terminated-result on handing a string to curl in
  chunks of bytes. our usage is fine.
* reassigning a unique_ptr by CRIMES instead of using release(), then
  using release() and ignoring the result. wild. let's use release() for
  its intended purpose.

Change-Id: Ic3e7affef12383576213a8a7c8145c27e662513d
2024-03-29 20:26:38 -07:00
jadeandGerrit Code Review a4f5bb951d Merge "Document ExprLambda fields" into main 2024-03-30 03:22:53 +00:00
Rebecca Turnerandjade 0b4e738944 Add a justfile to make meson friendly :)
Change-Id: Id6e4528392266c6f2444e030b67293abe297ed17
2024-03-30 03:03:46 +00:00
Jade Lovelace 817322fbfb Document ExprLambda fields
We got confused what formals did and had to briefly figure it out. We
should just have docs, so these are some.

Change-Id: If3e794a401e69d022785cbfa0b0c2e2284f41f58
2024-03-29 18:29:59 -07:00
Rebecca Turner a5a25894c1 Move escapeString to its own file
Change-Id: Ie5c954ec73c46c9d3c679ef99a83a29cc7a08352
2024-03-29 16:26:29 -07:00
eldritch horrors 5a54b0a20c meson: install systemd files
Change-Id: Idacf602fd379c82a051f00df2293cb02c8b286d4
2024-03-29 20:10:33 +00:00
eldritch horrors e28dc26084 meson: install shell files
Change-Id: I7c30690e5763d095cf7444333f7b687509051c5f
2024-03-29 20:10:33 +00:00
eldritch horrors 1da1f501fc meson: fix state-dir default value
the autoconf build system defaults to /nix/var, not /nix/var/nix. the
latter is only used in libstore, so we'll move the extra segment there.

Change-Id: Idfbc988ee302355982abdcd51d6d7b5d5d661c0d
2024-03-29 19:14:23 +00:00
Winter 6646b80396 meson: add missing explicit dependency on nlohmann_json
Without this, the Meson setup won't bail out if nlohmann_json is
missing, leading to subpar DX (and maybe worse, but I'm not entirely
sure).

Change-Id: I5913111060226b540dcf003257c99a08e84da0de
2024-03-29 14:16:58 -04:00
rebecca “wiggles” turnerandGerrit Code Review 9d97d1cb68 Merge "Set MAKEFLAGS=-j and GTEST_BRIEF in .envrc" into main 2024-03-29 17:09:13 +00:00
rebecca “wiggles” turnerandGerrit Code Review 877750b7c5 Merge "Move DebugChar into its own file" into main 2024-03-29 16:20:14 +00:00
eldritch horrors 6e5db5e4a2 meson: install missing/generated headers
one headers (args/root.hh) was simply missing, and the generated headers
were not installed. not all of them *should* be installed either, only a
select few (and sadly this needs a custom target for each one, it seems)

Change-Id: I37b25517895d0e5e521abc1202fa65624de57ed1
2024-03-29 02:45:48 +00:00
eldritch horrors 69bfd21e20 meson: install pkg-config files for libraries
Change-Id: I14b9d81d09f188eacfb9c68bcfb84751c18e3779
2024-03-29 02:45:48 +00:00
eldritch horrors 86b954a7af meson: increase functional test timeout
sometimes these fail with timeouts on loaded machines. let's up the
timeouts until we can pull the tests apart to more reasonable sizes

Change-Id: I2dfff2183cc1f3ff5e6107f43748ac046fe00d05
2024-03-29 02:19:36 +00:00
Rebecca Turner f9d5079c69 Set MAKEFLAGS=-j and GTEST_BRIEF in .envrc
- Enable parallel builds by default (and allow using environment
  variables to override `make` variables)
  - Hopefully we can get rid of this once we have Meson
- Set `GTEST_BRIEF=1`
  - This only shows failed tests, instead of listing every test on its
    own line.

```
$ GTEST_BRIEF=1 make check
[==========] 328 tests from 15 test suites ran. (37 ms total)
[  PASSED  ] 328 tests.
```

Change-Id: Id8103a8f24a9681be2be87e1b4df6fd5fdd7e4fd
2024-03-28 18:17:28 -07:00
wiggles dogandGerrit Code Review 236bc046ba Merge "Remove HintFmt::operator%" into main 2024-03-29 01:13:45 +00:00
Raito BezariusandGerrit Code Review 55350bd68d Merge "feat: unprivileged read-only open of SQLite DB" into main 2024-03-29 00:49:17 +00:00
Rebecca Turner 5ec2efb686 Move DebugChar into its own file
Change-Id: Ia40549e5d0b78ece8dd0722c3a5a032b9915f24b
2024-03-28 15:54:12 -07:00
wiggles dogandGerrit Code Review 62332c1250 Merge "Move shell_words into its own file" into main 2024-03-28 22:49:00 +00:00
QyriadandGerrit Code Review 81e50fef70 Merge "meson: implement functional tests" into main 2024-03-28 20:38:05 +00:00
jadeandGerrit Code Review 61d394e344 Merge "Build with traps on signed overflow" into main 2024-03-28 20:27:32 +00:00
jadeandGerrit Code Review ae065a992d Merge "progress-bar.cc: fix signed overflow" into main 2024-03-28 15:21:11 +00:00
Jade Lovelace 14207e1cf8 Build with traps on signed overflow
This is UB, we should not be doing it, and we can cheaply turn it into
crashes reliably. We would much rather have crashes than the program
doing something silly.

Benchmarks, but i wonder if they are nonsense because they get identical
times across compilers?!

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `result-clang/bin/nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix` | 375.5 ± 24.0 | 353.8 | 408.8 | 1.00 |
| `result-gcc/bin/nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix` | 407.9 ± 26.0 | 385.1 | 449.5 | 1.09 ± 0.10 |
| `result-clangsan/bin/nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix` | 382.2 ± 26.6 | 354.9 | 419.0 | 1.02 ± 0.10 |
| `result-gccsan/bin/nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix` | 408.6 ± 24.6 | 384.5 | 441.9 | 1.09 ± 0.10 |

| Command | Mean [s] | Min [s] | Max [s] | Relative |
|:---|---:|---:|---:|---:|
| `result-clang/bin/nix search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello` | 17.199 ± 0.167 | 16.930 | 17.499 | 1.01 ± 0.01 |
| `result-gcc/bin/nix search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello` | 17.409 ± 0.126 | 17.242 | 17.633 | 1.02 ± 0.01 |
| `result-clangsan/bin/nix search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello` | 17.080 ± 0.137 | 16.879 | 17.350 | 1.00 |
| `result-gccsan/bin/nix search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello` | 17.396 ± 0.160 | 17.131 | 17.660 | 1.02 ± 0.01 |

| Command | Mean [s] | Min [s] | Max [s] | Relative |
|:---|---:|---:|---:|---:|
| `result-clang/bin/nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 6.267 ± 0.069 | 6.197 | 6.415 | 1.02 ± 0.01 |
| `result-gcc/bin/nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 6.232 ± 0.045 | 6.180 | 6.311 | 1.01 ± 0.01 |
| `result-clangsan/bin/nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 6.162 ± 0.020 | 6.133 | 6.196 | 1.00 |
| `result-gccsan/bin/nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 6.229 ± 0.031 | 6.199 | 6.289 | 1.01 ± 0.01 |

| Command | Mean [s] | Min [s] | Max [s] | Relative |
|:---|---:|---:|---:|---:|
| `GC_INITIAL_HEAP_SIZE=10g result-clang/bin/nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 4.683 ± 0.044 | 4.630 | 4.761 | 1.00 |
| `GC_INITIAL_HEAP_SIZE=10g result-gcc/bin/nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 4.750 ± 0.041 | 4.680 | 4.812 | 1.01 ± 0.01 |
| `GC_INITIAL_HEAP_SIZE=10g result-clangsan/bin/nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 4.703 ± 0.040 | 4.640 | 4.760 | 1.00 ± 0.01 |
| `GC_INITIAL_HEAP_SIZE=10g result-gccsan/bin/nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'` | 4.766 ± 0.037 | 4.727 | 4.844 | 1.02 ± 0.01 |

Change-Id: I616ca3eab670317587d47b41870d8ac963c019ae
2024-03-27 23:54:04 -07:00
Jade Lovelace ffbad9b762 progress-bar.cc: fix signed overflow
this was caused by the use of std::chrono::duration::max() which gets
multiplied by some ratio to calculate nanoseconds to wait. then, it
explodes because that is a signed integer overflow. this was definitely
a bug.

error below:

/nix/store/fdiknsmnnczx6brsbppyljcs9hqckawk-gcc-12.3.0/include/c++/12.3.0/bits/chrono.h:225:38: runtime error: signed integer overflow: 9223372036854775807 * 1000000 cannot be represented in type 'long'
    #0 0x736d376b2b69 in std::chrono::duration<long, std::ratio<1l, 1000000000l>> std::chrono::__duration_cast_impl<std::chrono:
:duration<long, std::ratio<1l, 1000000000l>>, std::ratio<1000000l, 1l>, long, false, true>::__cast<long, std::ratio<1l, 1000l>>(
std::chrono::duration<long, std::ratio<1l, 1000l>> const&) /nix/store/fdiknsmnnczx6brsbppyljcs9hqckawk-gcc-12.3.0/include/c++/12
.3.0/bits/chrono.h:225:38
    #1 0x736d376b2b69 in std::enable_if<__is_duration<std::chrono::duration<long, std::ratio<1l, 1000000000l>>>::value, std::chr
ono::duration<long, std::ratio<1l, 1000000000l>>>::type std::chrono::duration_cast<std::chrono::duration<long, std::ratio<1l, 10
00000000l>>, long, std::ratio<1l, 1000l>>(std::chrono::duration<long, std::ratio<1l, 1000l>> const&) /nix/store/fdiknsmnnczx6brs
bppyljcs9hqckawk-gcc-12.3.0/include/c++/12.3.0/bits/chrono.h:270:9
    #2 0x736d376b2b69 in std::enable_if<__is_duration<std::chrono::duration<long, std::ratio<1l, 1000000000l>>>::value, std::chr
ono::duration<long, std::ratio<1l, 1000000000l>>>::type std::chrono::ceil<std::chrono::duration<long, std::ratio<1l, 1000000000l
>>, long, std::ratio<1l, 1000l>>(std::chrono::duration<long, std::ratio<1l, 1000l>> const&) /nix/store/fdiknsmnnczx6brsbppyljcs9
hqckawk-gcc-12.3.0/include/c++/12.3.0/bits/chrono.h:386:14
    #3 0x736d376b2b69 in std::cv_status std::condition_variable::wait_for<long, std::ratio<1l, 1000l>>(std::unique_lock<std::mut
ex>&, std::chrono::duration<long, std::ratio<1l, 1000l>> const&) /nix/store/fdiknsmnnczx6brsbppyljcs9hqckawk-gcc-12.3.0/include/
c++/12.3.0/condition_variable:164:6
    #4 0x736d376b1ee9 in std::cv_status nix::Sync<nix::ProgressBar::State, std::mutex>::Lock::wait_for<long, std::ratio<1l, 1000
l>>(std::condition_variable&, std::chrono::duration<long, std::ratio<1l, 1000l>> const&) /home/jade/lix/lix/src/libutil/sync.hh:
65:23
    #5 0x736d376b1ee9 in nix::ProgressBar::ProgressBar(bool)::'lambda'()::operator()() const /home/jade/lix/lix/src/libmain/prog
ress-bar.cc:99:27
    #6 0x736d36de25c2 in execute_native_thread_routine (/nix/store/a3zlvnswi1p8cg7i9w4lpnvaankc7dxx-gcc-12.3.0-lib/lib/libstdc++
.so.6+0xe05c2)
    #7 0x736d36b6b0e3 in start_thread (/nix/store/1zy01hjzwvvia6h9dq5xar88v77fgh9x-glibc-2.38-44/lib/libc.so.6+0x8b0e3) (BuildId
: 287831bffdbdde0ec25dbd021d12bdfc0ab9f5ff)
    #8 0x736d36bed5e3 in __clone (/nix/store/1zy01hjzwvvia6h9dq5xar88v77fgh9x-glibc-2.38-44/lib/libc.so.6+0x10d5e3) (BuildId: 28
7831bffdbdde0ec25dbd021d12bdfc0ab9f5ff)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /nix/store/fdiknsmnnczx6brsbppyljcs9hqckawk-gcc-12.3.0/include/c++/12.3.
0/bits/chrono.h:225:38 in

Change-Id: Ia0303242cdfd5d49385ae9e99718d709625a4633
2024-03-27 22:56:04 -07:00
Winter 80405d0626 Stop vendoring toml11
We don't apply any patches to it, and vendoring it locks users into
bugs (it hasn't been updated since its introduction in late 2021).

Closes https://git.lix.systems/lix-project/lix/issues/164

Change-Id: Ied071c841fc30b0dfb575151afd1e7f66970fdb9
2024-03-27 21:04:00 -04:00
Qyriad 038daad218 meson: implement functional tests
Functional tests can be run with
`meson test -C build --suite installcheck`.

Notably, functional tests must be run *after* running `meson install`
(Lix's derivation runs the installcheck suite in installCheckPhase so it
does this correctly), due to some quirks between Meson and the testing
system.

As far as I can tell the functional tests are meant to be run after
installing anyway, but unfortunately I can't transparently make
`meson test --suite installcheck` depend on the install targets.

The script that runs the functional tests, meson/run-test.py, checks
that `meson install` has happened and fails fast with a (hopefully)
helpful error message if any of the functional tests are run before
installing.

TODO: this change needs reflection in developer documentation

Change-Id: I8dcb5fdfc0b6cb17580973d24ad930abd57018f6
2024-03-27 18:37:50 -06:00
Jade Lovelace edba570664 HOT SALE: 15% off your build times!
This was achieved by running maintainers/buildtime_report.sh on the
build directory of a meson build, then asking "why the heck is json
eating our build times", and strategically moving the json using bits
out of widely included headers.

It turns out that putting literally any metrics whatsoever into the
build had immediate and predictable results.

Results are 1382.5s frontend time -> 1175.4s frontend time, back end
time approximately invariant.

Related: https://git.lix.systems/lix-project/lix/issues/159

Change-Id: I7edea95c8536203325c8bb4dae5f32d727a21b2d
2024-03-27 03:52:57 +00:00
Jade Lovelace 412a9c9f67 Enable clang build timing analysis
I didn't enable this by default for clang due to making the build time
10% worse or so. Unfortunate, but tbh devs for whom 10% of build time is
not *that* bad should probably simply enable this.

Change-Id: I8d1e5b6f3f76c649a4e2f115f534f7f97cee46e6
2024-03-27 03:52:57 +00:00
Jade Lovelace 50c6feeb77 Add release notes system for dev facing release notes
We keep changing dev stuff and we probably should keep the news up to
date?

Change-Id: I819da6a29f1c56c8ab8d758c159a9c96164cb04e
2024-03-27 03:52:57 +00:00
eldritch horrors 0436f4cfa6 flake: always build release notes in devshell
Change-Id: I0e02567fe8f102a8a8f1558aa094eefacdac9393
2024-03-27 03:09:14 +00:00
eldritch horrors 279e30e7ef build: replace changelog-d with local script
hacking changelog-d to support not just github but also forgejo and
gerrit is a lot more complicated than it's worth, even moreso since
the entire thing can just as well be done with ~60 lines of python.
this new script is also much cheaper to instantiate (being python),
so having it enabled in all shells is far less of a hassle.

we've also adjusted existing release notes that referenced a gerrit
cl to auto-link to the cl in question, making the diff a bit bigger

closes https://git.lix.systems/lix-project/lix/issues/176

Change-Id: I8ba7dd0070aad9ba4474401731215fcf5d9d2130
2024-03-27 03:09:14 +00:00
eldritch horrors 8fd02df90d manual: fix release notes
fix key spelling errors, type errors, things-should-not-be-comments errors

Change-Id: I3ce12873aa78002bca686bd88404771895b05d30
2024-03-27 03:09:14 +00:00
Rebecca Turner aee3d639b5 Move shell_words into its own file
Change-Id: I34c0ebfb6dcea49bf632d8880e04075335a132bf
2024-03-26 16:44:04 -07:00
Rebecca Turner 8e63eca912 Remove HintFmt::operator%
Change-Id: Ibcf1a7848b4b18ec9b0807628ff229079ae7a0fe
2024-03-26 15:40:05 -07:00
jadeandGerrit Code Review da22dbc333 Merge "Issue importer: do not notify" into main 2024-03-26 22:04:48 +00:00
Ilya KandGerrit Code Review fa3088a878 Merge "envrc: improve" into main 2024-03-26 20:49:40 +00:00
jadeandGerrit Code Review a5254186fd Merge "libstore/filetransfer: use Lix UA and unnix error message" into main 2024-03-26 20:46:27 +00:00
QyriadandGerrit Code Review 207d24da4e Merge "meson: implement unit tests" into main 2024-03-26 20:06:09 +00:00
K900 312cee142a envrc: improve
- add shellcheck hint
- load .envrc.local if one exists
- use a variable to allow choosing the shell variant

Change-Id: Iea34e5a800f5d463e5792020c5c293b8b3071ca5
2024-03-26 19:40:43 +03:00
Raito Bezarius 80b66b5065 libstore/filetransfer: use Lix UA and unnix error message
Once this commit lands, we are even more visible in analytics FWIW.

Change-Id: Id7e0c162315d0f191edbea9cb5fb82ce363704b9
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-26 16:06:27 +00:00
jadeandGerrit Code Review 531b8d0ab8 Merge "libmain: version printer uses Lix instead of Nix" into main 2024-03-26 16:06:15 +00:00
K900 7170e9f947 envrc: add
direnv good.

Change-Id: I81bd0086b847c6ee642a201fbc991a1f63fc7d7a
2024-03-26 15:19:37 +03:00
K900andjade a69f6e185a build-remote: fix format string shenanigans
HintFmt(string) invokes the HintFmt("%s", literal) constructor,
which is not what we want here. Add a constructor with a proper name
and call that.

Next step: rename all the other ones to HintFmt::literal(string).

Fixes: https://git.lix.systems/lix-project/lix/issues/178

Change-Id: If52d2eb8864ceb8663e05992e9d1fffef573d6b8
2024-03-26 07:58:24 +00:00
lunaphiedandGerrit Code Review 73624f7d9c Merge "Merge pull request #8817 from iFreilicht/flake-update-lock-overhaul" into main 2024-03-26 00:45:49 +00:00
Qyriadandeldritch horrors e1ffe56793 meson: implement unit tests
Unit tests can be run with `meson test -C build --suite check`.
`--suite check` is optional, as right now that's the only test suite,
but when functional tests are added those will be in a separate suite.

Change-Id: I7f22f1cde4b489b3cdb5f9a36a544f0c409fcc1f
2024-03-26 00:43:33 +01:00
Théophane HufschmittandLunaphied 86881226b0 Merge pull request #8817 from iFreilicht/flake-update-lock-overhaul
Overhaul `nix flake update` and `nix flake lock` UX

(cherry picked from commit 12a0ae73dbb37becefa5a442eb4532ff0de9ce65)
Change-Id: Iff3b4f4235ebb1948ec612036b39ab29e4ca22b2
2024-03-25 17:36:24 -06:00
eldritch horrors c3a5f937d3 flake.nix: linearize meson builds
parallel meson builds need too much ram. linearize them for now, and
hopefully we can remove the make build system and this hack soonish.

Change-Id: I42c092db8b0c63680e77da2263cdfe9e7f6575be
2024-03-25 21:48:55 +00:00
QyriadandGerrit Code Review 787c4397f1 Merge "issue template: use nix --version instead of nix-env --version" into main 2024-03-25 21:47:53 +00:00
Qyriad 1da1aa5045 issue template: use nix --version instead of nix-env --version
`nix --version` doesn't require `nix-command` experimental feature to
run and we could all do with less nix-env

Change-Id: I90748d591c574d96eda46591e9f9ce828311da29
2024-03-25 12:59:30 -06:00
Eelco Dolstraandlunaphied aa7653608d Minor cleanup in libexpr/flake/flake.cc
(cherry picked from commit 05316d401fa509557c71140e17bb19814412fcb8)
Change-Id: I6ba0b55709f5fe21beb4e9f3bf72ee28715d15f3
2024-03-25 15:30:36 +00:00
Eelco Dolstraandlunaphied b525d0f20c Input: Replace markFileChanged() by putFile()
Committing a lock file using markFileChanged() required the input to
be writable by the caller in the local filesystem (using the path
returned by getSourcePath()). putFile() abstracts over this.

(cherry picked from commit 95d657c8b3ae4282e24628ba7426edb90c8f3942)
Change-Id: Ie081c5d9eb4e923b229191c5e23ece85145557ff
2024-03-25 15:30:36 +00:00
John Ericsonandlunaphied 3d065192c0 Overhaul completions, redo #6693 (#8131)
As I complained in
https://github.com/NixOS/nix/pull/6784#issuecomment-1421777030 (a
comment on the wrong PR, sorry again!), #6693 introduced a second
completions mechanism to fix a bug. Having two completion mechanisms
isn't so nice.

As @thufschmitt also pointed out, it was a bummer to go from `FlakeRef`
to `std::string` when collecting flake refs. Now it is `FlakeRefs`
again.

The underlying issue that sought to work around was that completion of
arguments not at the end can still benefit from the information from
latter arguments.

To fix this better, we rip out that change and simply defer all
completion processing until after all the (regular, already-complete)
arguments have been passed.

In addition, I noticed the original completion logic used some global
variables. I do not like global variables, because even if they save
lines of code, they also obfuscate the architecture of the code.

I got rid of them  moved them to a new `RootArgs` class, which now has
`parseCmdline` instead of `Args`. The idea is that we have many argument
parsers from subcommands and what-not, but only one root args that owns
the other per actual parsing invocation. The state that was global is
now part of the root args instead.

This did, admittedly, add a bunch of new code. And I do feel bad about
that. So I went and added a lot of API docs to try to at least make the
current state of things clear to the next person.

--

This is needed for RFC 134 (tracking issue #7868). It was very hard to
modularize `Installable` parsing when there were two completion
arguments. I wouldn't go as far as to say it is *easy* now, but at least
it is less hard (and the completions test finally passed).

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Change-Id: If18cd5be78da4a70635e3fdcac6326dbfeea71a5
(cherry picked from commit 67eb37c1d0de28160cd25376e51d1ec1b1c8305b)
2024-03-25 15:30:36 +00:00
Tom Bereknyeiandlunaphied 4494f9097f feat: notation to refer to no attribute search prefix
An attrPath prefix of "." indicates no need to try default attrPath prefixes. For example `nixpkgs#legacyPackages.x86_64-linux.ERROR` searches through

```
trying flake output attribute 'packages.x86_64-linux.legacyPackages.x86_64-linux.ERROR'
using cached attrset attribute ''
trying flake output attribute 'legacyPackages.x86_64-linux.legacyPackages.x86_64-linux.ERROR'
using cached attrset attribute 'legacyPackages.x86_64-linux'
trying flake output attribute 'legacyPackages.x86_64-linux.ERROR'
using cached attrset attribute 'legacyPackages.x86_64-linux'
```

And there is no way to specify that one does not want the automatic
search behavior. Now one can specify
`nixpkgs#.legacyPackages.x86_64-linux.ERROR` to only refer to the rooted
attribute path without any default injection of attribute search path or
system.

Change-Id: Iac1334e1470137b7ce11dcf845513810230638ec
(cherry picked from commit d4aed18883b361133607296fb6cd789c47427a38)
2024-03-25 15:30:36 +00:00
lunaphiedandGerrit Code Review d3d7489571 Merge "Improve new CLI UX by supporting short -E flag for --expr" into main 2024-03-25 14:13:44 +00:00
Raito Bezariusandjade ad8a4b380e libmain: version printer uses Lix instead of Nix
Change-Id: I014ff24b900c0b9a48b7a63c8bb8b86cde3ebe54
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-25 08:04:31 +00:00
jadeandGerrit Code Review 5a1c35f907 Merge "Restore system-install profile files from the previous installer" into main 2024-03-25 04:17:36 +00:00
Jade Lovelace dee6b75702 Restore system-install profile files from the previous installer
These files are required to get Nix in PATH in existing multi-user installs using
the legacy installer. We really could use some tests.

Cc: https://git.lix.systems/lix-project/lix/issues/33

This partially reverts commit 93cc063344.

Fixes: https://git.lix.systems/lix-project/lix/issues/173

Change-Id: Iafb55280596732670a432f604b897f48562868e4
2024-03-25 04:17:14 +00:00
Lunaphied 185ecf1f45 Improve new CLI UX by supporting short -E flag for --expr
Change-Id: I55881c846da8416a92a14deedfa5bbbf09a122fb
2024-03-24 21:17:51 -06:00
eldritch horrors c856b82c2e libstore: despecialcase protocol version check
protocol versions are sent as u64. on the peer we read them as uint64,
check that the upper half is 0, and throw an exception if not. we then
read an arbitrary amount of data from the peer and dump it to the user
terminal. this is a little bit ridiculous, can never happen in correct
implementation, and is severly untested. let us just drop it entirely.

Change-Id: Ibd2f53a765341ed6439d40d9d1eac11e79c6b5e3
2024-03-24 18:45:22 +00:00
eldritch horrors 3e428f2289 libstore: un-inline copyNAR expansions
these are copies of copyNAR with only some variables renamed.

Change-Id: I98ddd7a98250fa5d304e18e1debf417e9f7768dd
2024-03-24 15:24:02 +01:00
Jade Lovelace 33da9c09c8 Issue importer: do not notify
This uses the forgejo patch we have for dont_notify on issue creation on
the api, and indeed does not notify, so we can simply run the script
safely :D

Fixes: https://git.lix.systems/lix-project/web-services/issues/38

Change-Id: I86bcbf9b4499b439b79b82af84ee7df0f8eb3298
2024-03-23 19:03:34 -07:00
Jade Lovelace 946fc12e4e Revert "Merge pull request #9476 from alois31/restore-progress-bar"
Observed to regress nix repl attrset printing with narrow windows.

This reverts commit a2d5e803cf.

Fixes: https://git.lix.systems/lix-project/lix/issues/168

Change-Id: I8e0031475b4ec26d6a71014357d973578b70815c
2024-03-23 18:04:29 -07:00
eldritch horrors 652f52f071 libutil: don't memset 64k in drainFD
this is not needed and introduces a bunch of memset calls, making up for
3% of valgrind cycle estimation *alone*. real-world impact is a lot
lower on our test machine, but we suspect that less powerful machines
would see an impact from dropping this.

Change-Id: Iad10e9d556e64fdeb0bee0059a4e52520058d11e
2024-03-23 22:17:46 +00:00
Raito Bezarius 8044540c42 feat: unprivileged read-only open of SQLite DB
If the state SQLite database is configured to use a write-ahead-log, it
creates WAL files in the state directory.

When the state SQLite database is closed by the `nix-daemon` after
builds, those files are removed.

When an unprivileged user would like to open _in read only_ that
database, they cannot do so because they would need to create those WAL
files and they do not have the permission to do so.

For this, SQLite offers a "persistent WAL" feature [1] to leave the WAL
files around, even after closing the database.

This CL enable the persistent WAL mode.

Fixes: https://github.com/NixOS/nix/issues/10300
[1]: https://www.sqlite.org/wal.html

Change-Id: Id8ae534d7d2290457af28782e5215222ae051fe5
Signed-off-by: Raito Bezarius <raito@lix.systems>
2024-03-23 15:07:48 +01:00
Qyriad b4d07656ff build: optionally build and install with meson
This commit adds several meson.build, which successfully build and
install Lix executables, libraries, and headers. Meson does not yet
build docs, Perl bindings, or run tests, which will be added in
following commits. As such, this commit does not remove the existing
build system, or make it the default, and also as such, this commit has
several FIXMEs and TODOs as notes for what should be done before the
existing autoconf + make buildsystem can be removed and Meson made the
default. This commit does not modify any source files.

A Meson-enabled build is also added as a Hydra job, and to
`nix flake check`.

Change-Id: I667c8685b13b7bab91e281053f807a11616ae3d4
2024-03-22 08:36:50 -06:00
jadeandGerrit Code Review a7161b6c0f Merge "clang-tidy check infrastructure" into main 2024-03-21 12:28:13 -06:00
Qyriad fab55aff0e flake: fix arm32 Linux cross devShell on macOS (fix nix flake check)
Change-Id: Iacac97de0b3d5f2df52c7bc985148624a351f45d
2024-03-21 08:22:38 -06:00
eldritch horrors 22e3f0e987 libexpr: unbreak PosTable performance
this was mostly an inconvenience for error reporting, but fully broke
the debugger (because the debugger does *a lot* of eager position
resolution). copying the line offsets into a local and filling that
local when empty without also storing the calculated offsets back does
kind of ... not cache anything.

fixes https://git.lix.systems/lix-project/lix/issues/165

Change-Id: Iccb0ba193ce2f15c832978daecf7b9bebbbe8585
2024-03-20 13:45:36 +01:00
jadeandGerrit Code Review 5a28d70d1e Merge "un-ups your start" into main 2024-03-19 14:29:57 -06:00
eldritch horrors d9a83886f9 libutil: remove exception handling workingness check
within lix itself this problem is caught by the test suite. outside of
lix itself three cases can be had: either the problem is fully inside
lix libs, fully inside user code, or it exists at the boundary. the
first is caught by the test suite, the second isn't caught at all, and
the third is something lix should not be responsible for.

Change-Id: I95aa35d8cb6f0ef5816a2941c467bc0c15916063
2024-03-19 06:09:42 -06:00
jadeandGerrit Code Review 4050245faa Merge changes I72c945ca,I2138bb4d,Ib96749f3 into main
* changes:
  Release notes for builtins.nixVersion change
  un-nixes ur lix, a little
  issue importer: list issues that are *not* closed when finding existing issues
2024-03-18 20:19:53 -06:00
Jade Lovelace 985bd5eb9f un-ups your start
We do not need upstart, it is so thoroughly obsolete that we should not
care about supporting it.

Change-Id: Ie0ca084740845555fddffacc899cd129c9a4c1fe
2024-03-18 18:28:08 -07:00
Jade Lovelace 20b4a97af3 Release notes for builtins.nixVersion change
Change-Id: I72c945cab464d26d73f5594ef0a4bb2184545da4
2024-03-18 18:26:52 -07:00
Jade Lovelace 30233d87f9 un-nixes ur lix, a little
I didn't really go attack the docs because we need to pull a bunch of
PRs. I went looking for strings in the code that called lix nix.

Change-Id: I2138bb4dd239096bc530946b281db7f875195b39
2024-03-18 18:20:24 -07:00
Jade Lovelace 81be5eb7c6 issue importer: list issues that are *not* closed when finding existing issues
Turns out also, you cannot set the queue to 0 with any success. So we
really should just like, prevent notifications in forgejo itself.

Filed a bug for that:
https://git.lix.systems/lix-project/web-services/issues/38

Change-Id: Ib96749f3159659182904963cab7b2ef88fc64442
2024-03-18 18:14:31 -07:00
Jade Lovelace 6b0020749d clang-tidy check infrastructure
This brings in infrastructure for developing new custom clang-tidy lints
and refactors for Lix.

Change-Id: I3df5f5855712ab4f97d4e84d771e5e818f81f881
2024-03-18 16:10:29 -07:00
eldritch horrors f38ae92a38 libutil: make AutoCloseFD a better resource
add a reset() method to close the wrapped fd instead of assigning magic
constants. also make the from-fd constructor explicit so you can't
accidentally assign the *wrong* magic constant, or even an unrelated
integer that also just happens to be an fd by pure chance.

Change-Id: I51311b0f6e040240886b5103d39d1794a6acc325
2024-03-18 15:42:52 -06:00
eldritch horrorsandGerrit Code Review 0f518f44e2 Merge "libexpr: associate let exprs with the correct StaticEnv" into main 2024-03-18 15:20:21 -06:00
eldritch horrors afb839a0c9 libexpr: associate let exprs with the correct StaticEnv
static env association is from expr to its enclosing scope, but let
exprs set their association to their *inner* scope. this skips one level
of envs and will cause segfaults if the parent is a with expr.

fixes #145

Change-Id: I1d22146110f071ede21b4eed7ed34b5850ef2ef3
2024-03-18 14:15:22 -07:00
Jade Lovelace 37c4b10c44 Add clang format configuration
Contemplate the configuration with: https://clang-format-configurator.site/

(cherry picked from commit 53fdcbca509b6c5dacaea3d3c465d86e49b0dd74)
Change-Id: I5446fd45de2bf644e34112f719afb3318a440b30
2024-03-18 13:31:39 -07:00
eldritch horrors b3599166ad libexpr: sort binding name in debugger
not doing this exposes the binding name order to the annoying
interference of parse order on symbol order, which wouldn't be so bad if
it didn't make the tests less reliable and, importantly, dependent on
linker behavior (due to primop initialization being done in static
initializer, and the order of static initializers being defined only
within a single translation unit).

fixes #143

Change-Id: I3cf417893fbcf19e9ad3ff8986deb7cbcf3ca511
2024-03-18 20:03:31 +01:00
jadeandGerrit Code Review 47a237f7ec Merge "Delete hasPrefix and hasSuffix from the codebase" into main 2024-03-18 12:01:39 -06:00
eldritch horrors 86a1121d16 use byte indexed locations for PosIdx
we now keep not a table of all positions, but a table of all origins and
their sizes. position indices are now direct pointers into the virtual
concatenation of all parsed contents. this slightly reduces memory usage
and time spent in the parser, at the cost of not being able to report
positions if the total input size exceeds 4GiB. this limit is not unique
to nix though, rustc and clang also limit their input to 4GiB (although
at least clang refuses to process inputs that are larger, we will not).

this new 4GiB limit probably will not cause any problems for quite a
while, all of nixpkgs together is less than 100MiB in size and already
needs over 700MiB of memory and multiple seconds just to parse. 4GiB
worth of input will easily take multiple minutes and over 30GiB of
memory without even evaluating anything. if problems *do* arise we can
probably recover the old table-based system by adding some tracking to
Pos::Origin (or increasing the size of PosIdx outright), but for time
being this looks like more complexity than it's worth.

since we now need to read the entire input again to determine the
line/column of a position we'll make unsafeGetAttrPos slightly lazy:
mostly the set it returns is only used to determine the file of origin
of an attribute, not its exact location. the thunks do not add
measurable runtime overhead.

notably this change is necessary to allow changing the parser since
apparently nothing supports nix's very idiosyncratic line ending choice
of "anything goes", making it very hard to calculate line/column
positions in the parser (while byte offsets are very easy).

(cherry picked from commit 5d9fdab3de0ee17c71369ad05806b9ea06dfceda)
Change-Id: Ie0b2430cb120c09097afa8c0101884d94f4bbf34
2024-03-18 16:12:46 +01:00
eldritch horrors c39150e6bb diagnose "unexpected EOF" at EOF
this needs a string comparison because there seems to be no other way to
get that information out of bison. usually the location info is going to
be correct (pointing at a bad token), but since EOF isn't a token as
such it'll be wrong in that this case.

this hasn't shown up much so far because a single line ending *is* a
token, so any file formatted in the usual manner (ie, ending in a line
ending) would have its EOF position reported correctly.

(cherry picked from commit 855fd5a1bb781e4f722c1d757ba43e866d370132)
Change-Id: I120c56a962f4286b1ae3b71da7b71ce8ec3e0535
2024-03-18 16:12:46 +01:00
eldritch horrors 4c072c7c5f match line endings used by parser and error reports
the parser treats a plain \r as a newline, error reports do not. this
can lead to interesting divergences if anything makes use of this
feature, with error reports pointing to wrong locations in the input (or
even outside the input altogether).

(cherry picked from commit 2be6b143289e5479cc4a2667bb84e879116c2447)
Change-Id: Ieb7f7655bac8cb0cf5734c60bd41723388f2973c
2024-03-18 16:12:46 +01:00
eldritch horrors 9cf92c012d report inherit attr errors at the duplicate name
previously we reported the error at the beginning of the binding
block (for plain inherits) or the beginning of the attr list (for
inherit-from), effectively hiding where exactly the error happened.

this also carries over to runtime positions of attributes in sets as
reported by unsafeGetAttrPos. we're not worried about this changing
observable eval behavior because it *is* marked unsafe, and the new
behavior is much more useful.

(cherry picked from commit 1edd6fada53553b89847ac3981ac28025857ca02)
Change-Id: I2f50eb9f3dc3977db4eb3e3da96f1cb37ccd5174
2024-03-18 16:12:45 +01:00
eldritch horrors d826427f02 normalize formal order on ExprLambda::show
we already normalize attr order to lexicographic, doing the same for
formals makes sense. doubly so because the order of formals would
otherwise depend on the context of the expression, which is not quite as
useful as one might expect.

(cherry picked from commit 4147ecfb1c51f3fe3b4adcbd4e753fd487dab645)
Change-Id: I3fd0dbdef3ac7447a3a03ff20bb514a0d0f23fb1
2024-03-18 07:56:34 -06:00
eldritch horrors 314f044c2b keep copies of parser inputs that are in-memory only
the parser modifies its inputs, which means that sharing them between
the error context reporting system and the parser itself can confuse the
reporting system. usually this led to early truncation of error context
reports which, while not dangerous, can be quite confusing.

(cherry picked from commit d384ecd553aa997270b79ee98d02f7cf7e1849e6)
Change-Id: I677646b5675b12b2faa787943646aa36dc6e6ee3
2024-03-18 07:56:23 -06:00
eldritch horrors 1f8b85786e libutil: remove vfork
vfork confers a large performance advantage over fork, measured locally
at 16µs per vfork agains 90µs per fork. however nix *almost always*
follows a vfork up with an execve-family call, melting the performance
advantage from 6x to only 15%. in most of those cases it's doing things
that are undefined behavior (like manipulating the heap, or even
throwing exceptions and trashing the parent process stack).

most notably the one place that could benefit from the vfork performance
improvement is linux derivation sandbox setup—which doesn't use vfork.

Change-Id: I2037b7384d5a4ca24da219a569e1b1f39531410e
2024-03-18 06:10:41 -06:00
jadeandGerrit Code Review 2890840b96 Merge "Forgejo issue importer" into main 2024-03-17 22:08:59 -06:00
Jade Lovelace 61e21b2557 Delete hasPrefix and hasSuffix from the codebase
These now have equivalents in the standard lib in C++20. This change was
performed with a custom clang-tidy check which I will submit later.
Executed like so:

ninja -C build && run-clang-tidy -checks='-*,nix-*' -load=build/libnix-clang-tidy.so -p .. -fix ../tests | tee -a clang-tidy-result

Change-Id: I62679e315ff9e7ce72a40b91b79c3e9fc01b27e9
2024-03-17 20:17:19 -07:00
jadeandGerrit Code Review 706cee5c49 Merge "builtins.nixVersion: return fixed fake version" into main 2024-03-17 12:13:21 -06:00
Qyriad 32d6e58069 flake: fix musl static stdenv devShell (fix nix flake check)
pkgs.pkgsStatic.glibcLocales is null, so the string coercion was failing
for devShells against static stdenvs

Change-Id: Iee8e1042a852133ce0432627d72a85e97c17055e
2024-03-17 10:05:18 -06:00
Jade Lovelace 886a418d23 builtins.nixVersion: return fixed fake version
This builtin is only going to cause us problems because we are not Nix,
so let's just falsify being in the 2.18 series, since that is the
closest target that has any meaning.

In future we might want to have a better feature detection mechanism,
for when we actually add stuff to some builtin's attr set argument. But
builtins.nixVersion is just going to be hopelessly broken and it should
be stubbed out.

Fixes https://git.lix.systems/lix-project/lix/issues/144

Change-Id: Id7390b32a29c6147f2977737d81846320de5d67e
2024-03-17 00:32:19 -07:00
eldritch horrors 11f35afa6f diagnose duplicated attrs at correct path
diagnose attr duplication at the path the duplication was detected, not
at the path the current attribute wanted to place. doing the latter is
only correct if a leaf attribute was duplicated, not if an attrpath was
set to a non-attrset in one binding and a (potentially implied) attrset
in another binding.

fixes #124

Change-Id: Ic4aa9cc12a9874d4e7897c6f64408f10aa36fc82
2024-03-16 22:12:49 +01:00
Jade Lovelace 3392020710 Forgejo issue importer
We needed a script to go yoink all the real NixOS/Nix issues from our
mirror into the Lix repo.

Change-Id: If8c8ebfb58634c675eae450454c0189288c6b18a
2024-03-16 00:22:33 -07:00
rbtandGerrit Code Review e257ff10fd Merge "Fix gc-small-vector.hh includes" into main 2024-03-15 16:20:40 -06:00
eldritch horrors 3c52344300 out with the -O3, in with the -O2
-O3 does not measurably improve performance of the resulting binaries,
neither with lto enabled nor with lto disabled. what it does to however
is cause gcc warning spew in libstdc++ that we can't do anything
about (and that upon inspection of libstdc++ source looks like a gcc
bug).

with lto, -O3:

Benchmark 1: GC_INITIAL_HEAP_SIZE=10g nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      4.608 s ±  0.027 s    [User: 3.866 s, System: 0.522 s]
    Range (min … max):    4.579 s …  4.640 s    10 runs

Benchmark 2:  nix eval -f <nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix>
    Time (mean ± σ):     408.1 ms ±  25.5 ms    [User: 360.0 ms, System: 28.1 ms]
    Range (min … max):   387.6 ms … 439.0 ms    10 runs

with lto, -O2:

Benchmark 1: GC_INITIAL_HEAP_SIZE=10g nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      4.632 s ±  0.044 s    [User: 3.874 s, System: 0.544 s]
    Range (min … max):    4.563 s …  4.673 s    10 runs

Benchmark 2:  nix eval -f <nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix>
    Time (mean ± σ):     394.0 ms ±  23.9 ms    [User: 351.2 ms, System: 27.6 ms]
    Range (min … max):   377.8 ms … 429.3 ms    10 runs

without lto, -O3:

Benchmark 1: GC_INITIAL_HEAP_SIZE=10g nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      4.700 s ±  0.024 s    [User: 3.906 s, System: 0.559 s]
    Range (min … max):    4.663 s …  4.717 s    10 runs

Benchmark 2:  nix eval -f <nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix>
    Time (mean ± σ):     400.4 ms ±  25.6 ms    [User: 353.7 ms, System: 26.8 ms]
    Range (min … max):   379.8 ms … 430.6 ms    10 runs

without lto, -O2:

Benchmark 1: GC_INITIAL_HEAP_SIZE=10g nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      4.724 s ±  0.030 s    [User: 3.924 s, System: 0.570 s]
    Range (min … max):    4.687 s …  4.749 s    10 runs

Benchmark 2:  nix eval -f <nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix>
    Time (mean ± σ):     392.4 ms ±  24.3 ms    [User: 350.9 ms, System: 26.4 ms]
    Range (min … max):   376.9 ms … 428.0 ms    10 runs

fixes #46

Change-Id: Ib8afad8a07c278f57f2e3317d00cce4f9ec0f338
2024-03-15 15:31:49 -06:00
Rebecca Turner 7abbce500b Fix gc-small-vector.hh includes
Change-Id: I4abc19029fb62712582761d4fc1895156b68803d
2024-03-15 13:39:32 -07:00
Jade Lovelace 0d85875c3a Allow dlopen of plugins to fail
It happens with some frequency that plugins that might be unimportant to
the evaluation at hand mismatch with the nix version, leading to
spurious load failures. Let's make these non fatal.

Change-Id: Iba10e951d171725ccf1a121bcd9be1e1d6ad69eb
2024-03-15 12:31:16 -07:00
Jade Lovelace 7d361f1a82 Test that :st does ... something
Change-Id: I97c00b5eb1288f68d8c2b484436cc185d040b8b2
2024-03-15 12:31:16 -07:00
Jade Lovelace af066af7f3 repl_characterization: Also verify the stack trace exists
Change-Id: I8b2d8211a24011fae1586a1182d7d0772a039cd7
2024-03-15 12:31:16 -07:00
Jade Lovelace 78513b1fc8 repl_characterization: eat newlines after commands and source-dir paths
This is because they are unrepresentable in the source files with
commentary but not in the output, so we should just eat them in
normalization. It's ok.

Change-Id: I2cb7e8b3fc7b00874885bb287cbaa200b41cb16b
2024-03-15 12:31:16 -07:00
Jade Lovelace 8a8715af89 Add regression tests for #9917, #9918
Change-Id: Ib0591e1499c5dba5e5a83ee75a899c9d16986827
2024-03-15 12:31:16 -07:00
Jade Lovelace 18ed6c3bdf Implement a repl characterization test system
This allows for automating using the repl without needing a PTY, with
very easy to write test files.

Change-Id: Ia8d7854edd91f93477638942cb6fc261354e6035
2024-03-15 12:31:16 -07:00
Jade Lovelace 38571c50e6 Implement a parser for a literate testing system for the repl
This parser can be reused for other purposes. It's inspired by
https://bitheap.org/cram/

Although eelco's impostor exists https://github.com/mobusoperandi/eelco,
it is not very nice to depend on out of tree testing frameworks with no
way to customize them.

Change-Id: Ifca50177e09730182baf0ebf829c3505bbb0274a
2024-03-14 14:30:38 -07:00
Cole HelblingandRebecca Turner 84727bebb4 Backport PR#9633 by cole-h: package: don't set sysconfdir in devShells
(cherry-picked from commit ba0087316acc2aba999cabe5e1a159da636b2569)

Change-Id: I5a50afb3b7b65516df798ee51b74f06727a91928
2024-03-14 14:06:31 -07:00
Rebecca Turner 9f242fae76 Only set LOCALE_ARCHIVE on Linux
`macOS` does not have `glibcLocales`:

    error:
       … while calling the 'derivationStrict' builtin

         at /derivation-internal.nix:9:12:

            8|
            9|   strict = derivationStrict drvAttrs;
             |            ^
           10|

       … while evaluating derivation 'nix-2.90.0'
         whose name attribute is located at /nix/store/y0c95bwyvs80pm69hdd4b11pyq2ghiwh-source
/pkgs/stdenv/generic/make-derivation.nix:348:7

       … while evaluating attribute 'LOCALE_ARCHIVE' of derivation 'nix-2.90.0'

         at /nix/store/ng5qzbyv4902b4pw7g35caqw5cnmryf9-source/flake.nix:331:15:

          330|               # Required to make non-NixOS Linux not complain about missing loc

Change-Id: I4464484a0eca12b5e073d49d900b6f25886245c1
2024-03-14 14:06:23 -07:00
Puck MeerburgandGerrit Code Review caded0d55e Merge "Delete the existing installer" into main 2024-03-14 13:14:21 -06:00
jadeandGerrit Code Review c79b5dca2a Merge "flake: dev shells should include the LOCALE_ARCHIVE so configure does not complain" into main 2024-03-14 12:22:35 -06:00
jadeandGerrit Code Review cc55da201b Merge "un-thumbs-up ur github templates" into main 2024-03-14 12:22:09 -06:00
Puck Meerburg 93cc063344 Delete the existing installer
We're not going to use it.

Fixes: #31
Change-Id: Ib17a2eb6cae1ecbbf9ad1062e576ba6107a3c13b
2024-03-14 18:15:46 +00:00
eldritch horrors c26599b143 libexpr: fix elided value counting in printer
using the total-attrs-printed and total-list-items-printed counters to
calculate how many attrs were elided only works properly if no nesting
is involved. once things do nest the global counter can exceed the size
of the currently printed object, leading to unsigned wrapping and great
overestimation of elided counts. counting locally in addition to global
counts fixes this.

these are functional tests because creating these objects requires the
evaluator to not be a huge amount of code, and we also want defaults to
be tested for cli usage.

fixes #14

Change-Id: Icb9a0cb21b2f4bacbc5e9dcdd8c0b9055b4088a7
2024-03-14 01:52:19 -06:00
Jade Lovelace 0278c03de5 un-thumbs-up ur github templates
These are not the way that we want to do things.

Change-Id: I5f3706cf50d007a6659edb96a6230d52e18a769a
2024-03-13 23:10:25 -07:00
jadeandGerrit Code Review 2a8f579c53 Merge "Backport PR#10204 by 9999years: Replace foo with __NIX_STR in cxx-big-literal" into main 2024-03-14 00:07:21 -06:00
Jade Lovelace ab5ff86917 flake: dev shells should include the LOCALE_ARCHIVE so configure does not complain
Change-Id: Id661bd8c90696cb663a25503878a697f596d1e77
2024-03-13 12:25:23 -07:00
eldritch horrors 06952cf7c4 support <program>_ENV variables
this lets us set per-test-program environment variables rather than only
a single, global default. this was supported in nix originally but
might've gone partially missing in the upstream backports process?

Change-Id: Iad0919841b1b6d11e0b7ebd3920449a62f544e77
2024-03-13 19:48:26 +01:00
Rebecca TurnerandJade Lovelace da2f165128 Backport PR#10204 by 9999years: Replace foo with __NIX_STR in cxx-big-literal
Looks a little nicer when you check the generated sources.

(cherry-picked from commit e65e9114d2797cc4380da218972979dda7395df6)

Change-Id: I91bd185bf12deef72d20fba36178ff42a686c518
Upstream-PR: https://github.com/NixOS/nix/pull/10204
2024-03-12 11:58:36 -07:00
Jade Lovelace 1b8662b85c import the revisions to the characterization test framework from cppnix
This has some Flaws for sure (like, it is going to be a bit stretched to
use for repl characterization), but it is a start.

Change-Id: I258c8beb3aee236f45818a03be83bcda858120c9
2024-03-11 14:14:43 -07:00
Jade Lovelace be2b87ed4d add automated usage mode to the repl
This is definitely not a stable thing, but it does feel slightly crimes
to put it as an experimental feature. Shrug, up for bikeshedding.

Change-Id: I6ef176e3dee6fb1cac9c0a7a60d553a2c63ea728
2024-03-11 14:14:43 -07:00
jadeandGerrit Code Review b06a392114 Merge "refactor: repl prompts are now the job of the interacter" into main 2024-03-11 15:12:09 -06:00
jadeandGerrit Code Review dd05106d1c Merge "refactor: move readline stuff into its own file" into main 2024-03-11 15:11:56 -06:00
jadeandGerrit Code Review df2723b972 Merge "finally.hh: delete copy constructor which is a bad idea" into main 2024-03-11 15:11:20 -06:00
jadeandGerrit Code Review d9367da027 Merge "Add box_ptr: nonnull unique_ptr with value semantics" into main 2024-03-11 15:11:15 -06:00
Qyriad SzekelyandGerrit Code Review f9701da8f6 Merge changes from topic "maint/flake-package.nix" into main
* changes:
  package: cleanup of all intermediaries
  package: migrate devShells
  package: migrate internal-api-docs
  package: migrate testNixVersions
  package: use pname, version, and dontBuild (first change with diff hash)
  package: refactor Nix out of flake.nix and into package.nix
2024-03-11 14:32:37 -06:00
jadeandGerrit Code Review 50c401b4c1 Merge "util.hh: split out signals stuff" into main 2024-03-11 11:14:24 -06:00
Qyriad 36a8d151e1 package: cleanup of all intermediaries
Change-Id: I0da5182de6b01c192cfcba407959d659d70c6dc9
2024-03-11 04:26:35 -06:00
Qyriad 529a01ade2 package: migrate devShells
Change-Id: Ic63721667edd4bef79aa699a0de8411639e5159b
2024-03-11 04:26:35 -06:00
Qyriad b072c069b7 package: migrate internal-api-docs
Change-Id: I344d73a412c2c6e4bb2eb14bd4859056324f1ba7
2024-03-11 04:26:35 -06:00
Qyriad 4ad3446311 package: migrate testNixVersions
Change-Id: I71845f8a6d7b77c3617d055e726ed4a28cd05fa3
2024-03-11 04:26:35 -06:00
Qyriad 875b76d0c7 package: use pname, version, and dontBuild (first change with diff hash)
The src fileset, preConfigure, and separateDebugInfo also respond to doBuild if its overridden

This commit is logically just a continuation of the previous commit's
refactor, but exists separately to delineate when the core Nix
derivation hash changed (this commit).

Change-Id: I67a61bc9608d91b6a833ebc5c3894b2d2e694050
2024-03-11 04:26:35 -06:00
Qyriad 15380b4c6e package: refactor Nix out of flake.nix and into package.nix
This series takes a somewhat different approach from the flake rework
done in NixOS/nix. The package.nix here does not provide callPackage
options for all the various settings in the build, and instead the other
places Nix derivations are used (like internal-api-docs) will .overrideAttrs
the normal Nix package derivation. This more closely matches how these
things were structured originally, and results in less churn and more
atomicity in these changes.

In the future, package.nix likely will migrate to have more build
options in the callPackage arguments, but we are also planning to
rewrite the build system anyway.

Change-Id: I170c4e5a4184bab62e1fd75e56db876d4ff116cf
2024-03-11 04:26:35 -06:00
Jade Lovelace 1758a6ef25 refactor: repl prompts are now the job of the interacter
Change-Id: I17c2873dfbbff303cdbdc7a8903deb8409ce3026
2024-03-11 01:04:52 -07:00
Jade Lovelace 95a87f2c2a refactor: move readline stuff into its own file
This is in direct preparation for an automation mode of nix repl.

Change-Id: I26e6ca88ef1c48aab11a2d1e939ff769f1770caa
2024-03-11 01:04:52 -07:00
Jade Lovelace 45f6e3521a finally.hh: delete copy constructor which is a bad idea
Change-Id: I6d0b5736893c44bddc6f5789b452b434f8671b9b
2024-03-11 01:04:52 -07:00
Jade Lovelace af515baf6e Add box_ptr: nonnull unique_ptr with value semantics
This solves the problem of collections of boxed subclasses with virtual
dispatch, which should still be treated as values, since the
indirection is only there due to the virtual dispatch.

Change-Id: I368daedd3f31298e99c6e56a15606337a55494c6
2024-03-11 01:04:52 -07:00
Jade Lovelace 8be7030299 util.hh: split out signals stuff
Copies part of the changes of ac89bb064aeea85a62b82a6daf0ecca7190a28b7

Change-Id: I9ce601875cd6d4db5eb1132d7835c5bab9f126d8
2024-03-11 00:52:09 -07:00
jadeandGerrit Code Review 6432bf9197 Merge "Print derivation paths in nix eval" into main 2024-03-10 16:12:32 -06:00
eldritch horrors 0a4737f519 add doc comment justifying ExprInheritFrom
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
(cherry picked from commit f24e445bc024cfd3c26be5f061280af549321c22)
Change-Id: I7acda5d5c34c0914a78adc2385d32782c4c275cd
2024-03-10 03:18:32 -06:00
eldritch horrors 06764118ea remove ExprAttrs::AttrDef::inherited
it's no longer widely used and has a rather confusing meaning now that
inherit-from is handled very differently.

(cherry picked from commit 1cd87b7042d14aae1fafa47b1c28db4c5bd20de7)
Change-Id: I90bbebddf06762960d8ca4f621cf042ce8ae83f9
2024-03-10 03:18:32 -06:00
eldritch horrors b667b4cded evaluate inherit (from) exprs only once per directive
desugaring inherit-from to syntactic duplication of the source expr also
duplicates side effects of the source expr (such as trace calls) and
expensive computations (such as derivationStrict).

(cherry picked from commit cefd0302b55b3360dbca59cfcb4bf6a750d6cdcf)
Change-Id: Iff519f991adef2e51683ba2c552d37a3df7a179e
2024-03-10 03:18:32 -06:00
eldritch horrors 71e0114708 remove getDerivations deduplication
deduplication does not currently work fully, showing derivations
multiple times if they have different underlying values. this can happen
by selecting the same derivation twice for two different attributes of a
set, using inherit-from (which reduces to the previous), importing
nixpkgs twice, or any other number of things.

since users already have to deal with duplicates for this reason it
won't hurt to add *more* duplicates. the alternative would be to
deduplicate fully, which would drop derivations that are currently
returned and those pose a regression risk.

Change-Id: I64b397351237e10375d270f1bddecb71f62aa131
2024-03-10 03:18:32 -06:00
eldritch horrors 2a84123631 group inherit by source during Expr::show
for plain inherits this is really just a stylistic choice, but for
inherit-from it actually fixes an exponential size increase problem
during expr printing (as may happen during assertion failure reporting,
on during duplicate attr detection in the parser)

(cherry picked from commit ecf8b12d60ad2929f9998666cf0966475b91e291)
Change-Id: Ie55f0cb01a37e766414c31f8d40f51c2c7d106b0
2024-03-10 03:18:32 -06:00
eldritch horrors bf19eebb9b use the same bindings print for ExprAttrs and ExprLet
this also has the effect of sorting let bindings lexicographically
rather than by symbol creation order as was previously done, giving a
better canonicalization in the process.

(cherry picked from commit 6c08fba533ef31cad2bdc03ba72ecf58dc8ee5a0)
Change-Id: Ia887f629305645bb8a165fbbc0d32e620912595a
2024-03-10 03:18:32 -06:00
eldritch horrors 1cf0fa0633 add ExprAttrs::AttrDef::chooseByKind
in place of inherited() — not quite useful yet since we don't
distinguish plain and inheritFrom attr kinds so far.

(cherry picked from commit 1f542adb3e18e7078e6a589182a53a47d971748a)
Change-Id: If948c9d43e875de18f213a73a06a36f7c335b536
2024-03-10 03:18:32 -06:00
eldritch horrors 03f852b2c6 preserve information about whether/how an attribute was inherited
(cherry picked from commit c66ee57edc6cac3571bfbf77d0c0ea4d25b4e805)
Change-Id: Ie8606a8b2f5946c87dd4d16b7b46203e199a4cc1
2024-03-10 03:18:32 -06:00
eldritch horrors 3e43f4aeff add test for inherit expr printing
(cherry picked from commit 73065a400d176b21f518c1f4ece90c31318b218d)
Change-Id: I9356d8084d241a7904b66554d7c4194f8433edf7
2024-03-10 03:18:32 -06:00
eldritch horrors 0c19e6b56c add test for inherit-from semantics
(cherry picked from commit 8669c02468994887be91072ac58b1ee43380d354)
Change-Id: If8513316bf4b4b559c5bb63842c856f016816802
2024-03-10 03:18:32 -06:00
eldritch horrorsandGerrit Code Review 7c882a5075 Merge "make the multi-node vm tests a bit more reliable" into main 2024-03-10 03:18:08 -06:00
eldritch horrors 34fb7a7e9d make the multi-node vm tests a bit more reliable
without these changes the tests will very repeatably (although not very
reliably) wedge in our runs. the ssh command starts, opens a sessions,
does something, the session closes again, but the test does not move on.
adding *just* the redirect and not the unit waits is not sufficient
either, it needs both. this feels like a bug in the nixos testing
framework somewhere, but digging that far is not in the cards right now.

Change-Id: Idab577b83a36cc4899bb5ffbb3d9adc04e83e51c
2024-03-10 10:10:52 +01:00
eldritch horrors a9b813cc3b Merge pull request #10066 from 9999years/print-all-frames
Do not skip any stack frames when `--show-trace` is given

(cherry picked from commit 0b47783d0a879875d558f0b56e49584f25ceb2d0)
Change-Id: Ia0f18266dbcf97543110110c655c219c7a3e3270
2024-03-09 10:17:26 -07:00
eldritch horrors f2e11ddce1 Merge pull request #9914 from 9999years/debugger-on-trace
Enter debugger on `builtins.trace` with an option

(cherry picked from commit 774e7ca5847ebc392eac2a124a8f12b24da4f65a)
Change-Id: If01e2110b3a128e639b05143227e365227d149f1
2024-03-09 10:17:26 -07:00
eldritch horrors 030c8aa833 Rename ProcessLineResult variants
(cherry picked from commit 8e71883e3f59100479e96aa1883ef52dbaa03fd3)
Change-Id: If7d8b75eaec623dac106ce2363fa148af37d150c
2024-03-09 10:17:26 -07:00
eldritch horrors 992d99592f :quit in the debugger should quit the whole program
(cherry picked from commit 2a8fe9a93837733e9dd9ed5c078734a35b203e14)
Change-Id: I71dadfef6b24d9272b206e9e2c408040559d8a1c
2024-03-09 10:17:26 -07:00
eldritch horrors 6b11c2cd70 Extract printSpace helper
(cherry picked from commit 403c90ddf58a3f16a44dfe1f20004b6baa4e5ce2)
Change-Id: I53c9824e6b1c4c619b4dfd8346d39e5289d92265
2024-03-09 07:20:23 -07:00
eldritch horrors 73cdaf44cf prettyPrint -> shouldPrettyPrint
(cherry picked from commit 1c5f5d4291df7bf80806e57c75d2ec67bced8616)
Change-Id: I7a517490e7baa5cef00716f6d6cfcbcbcdde11bf
2024-03-09 07:20:23 -07:00
eldritch horrors 4dabde0485 Add assertion for decreasing the indent
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
(cherry picked from commit a27651908fc1b5ef73a81e46434a408c5868fa7b)
Change-Id: I2ec78e234c1c6e982f7b05f81d8b8356daf6c274
2024-03-09 07:20:23 -07:00
eldritch horrors 1958152d14 Pretty-print values in the REPL
Pretty-print values in the REPL by printing each item in a list or
attrset on a separate line. When possible, single-item lists and
attrsets are printed on one line, as long as they don't contain a nested
list, attrset, or thunk.

Before:
```
{ attrs = { a = { b = { c = { }; }; }; }; list = [ 1 ]; list' = [ 1 2 3 ]; }
```

After:
```
{
  attrs = {
    a = {
      b = {
        c = { };
      };
    };
  };
  list = [ 1 ];
  list' = [
    1
    2
    3
  ];
}
```

(cherry picked from commit c0a15fb7d03dfb8f53bc6726c414bc88aa362592)
Change-Id: Ia2b41849165a5ddb63f7a8c272a2476b3e4292df
2024-03-09 07:20:23 -07:00
Rebecca TurnerandJade Lovelace 6ef9d8efba Print derivation paths in nix eval
`nix eval` forces values and prints derivations as attribute sets, so
commands that print derivations (e.g. `nix eval nixpkgs#bash`) will
infinitely loop and segfault.

Printing derivations as `.drv` paths makes `nix eval` complete as
expected. Further work is needed, but this is better than a segfault.

(cherry picked from commit 4910d74086a85876e093136a0e8ebc547b467af7)

Change-Id: I8e1cb39c05db812080759ec183ee7a131760e6ea
2024-03-08 23:46:16 -08:00
510 changed files with 8668 additions and 18594 deletions
+50
View File
@@ -0,0 +1,50 @@
---
BasedOnStyle: LLVM
AccessModifierOffset: -4
AlignAfterOpenBracket: BlockIndent
AlignEscapedNewlines: Left
AlignOperands: DontAlign
AllowShortBlocksOnASingleLine: Always
AllowShortFunctionsOnASingleLine: Empty
AllowShortIfStatementsOnASingleLine: WithoutElse
AlwaysBreakBeforeMultilineStrings: true
AlwaysBreakTemplateDeclarations: Yes
BinPackArguments: false
BinPackParameters: false
BitFieldColonSpacing: None
BraceWrapping:
AfterCaseLabel: false
AfterClass: true
AfterControlStatement: MultiLine
AfterEnum: false
AfterFunction: true
AfterNamespace: false
AfterObjCDeclaration: false
AfterStruct: true
AfterUnion: true
AfterExternBlock: false
BeforeCatch: false
BeforeElse: false
BeforeLambdaBody: false
BeforeWhile: false
IndentBraces: false
SplitEmptyFunction: true
SplitEmptyRecord: false
SplitEmptyNamespace: true
BreakAfterAttributes: Always
BreakBeforeBinaryOperators: NonAssignment
BreakBeforeBraces: Custom
BreakConstructorInitializers: BeforeComma
ColumnLimit: 100
EmptyLineAfterAccessModifier: Leave
EmptyLineBeforeAccessModifier: Leave
FixNamespaceComments: false
IndentWidth: 4
InsertBraces: true
InsertTrailingCommas: Wrapped
LambdaBodyIndentation: Signature
PackConstructorInitializers: CurrentLine
PointerAlignment: Middle
SortIncludes: Never
SpaceAfterCStyleCast: true
SpaceAfterTemplateKeyword: false
+18
View File
@@ -0,0 +1,18 @@
UseColor: true
Checks:
- -*
- bugprone-*
# too many warnings
- -bugprone-assignment-in-if-condition
# too many warnings
- -bugprone-narrowing-conversions
# kind of nonsense
- -bugprone-easily-swappable-parameters
# too many warnings for now
- -bugprone-implicit-widening-of-multiplication-result
# Lix's exception handling is Questionable
- -bugprone-empty-catch
# many warnings
- -bugprone-unchecked-optional-access
# many warnings, seems like a questionable lint
- -bugprone-branch-clone
+9
View File
@@ -0,0 +1,9 @@
# shellcheck shell=bash
source_env_if_exists .envrc.local
# TODO: `use flake .#native-clangStdenvPackages` on macOS?
use flake ".#${LIX_SHELL_VARIANT:-default}" "${LIX_SHELL_EXTRA_ARGS[@]}"
export MAKEFLAGS="$MAKEFLAGS -e"
if [[ -n "$NIX_BUILD_CORES" ]]; then
export MAKEFLAGS="$MAKEFLAGS -j $NIX_BUILD_CORES"
fi
export GTEST_BRIEF=1
+6 -10
View File
@@ -7,30 +7,26 @@ assignees: ''
---
**Describe the bug**
## Describe the bug
A clear and concise description of what the bug is.
If you have a problem with a specific package or NixOS,
you probably want to file an issue at https://github.com/NixOS/nixpkgs/issues.
you probably want to file an issue at https://github.com/NixOS/nixpkgs/issues.
**Steps To Reproduce**
## Steps To Reproduce
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
## Expected behavior
A clear and concise description of what you expected to happen.
**`nix-env --version` output**
## `nix --version` output
**Additional context**
## Additional context
Add any other context about the problem here.
**Priorities**
Add :+1: to [issues you find important](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc).
+8 -8
View File
@@ -7,18 +7,18 @@ assignees: ''
---
**Is your feature request related to a problem? Please describe.**
## Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
## Describe the solution you'd like
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
## Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
## Additional context
Add any other context or screenshots about the feature request here.
**Priorities**
Add :+1: to [issues you find important](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc).
-4
View File
@@ -30,7 +30,3 @@ assignees: ''
```
</details>
## Priorities
Add :+1: to [issues you find important](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc).
@@ -25,7 +25,3 @@ assignees: ''
[latest Nix manual]: https://nixos.org/manual/nix/unstable/
[source]: https://github.com/NixOS/nix/tree/master/doc/manual/src
[open documentation issues and pull requests]: https://github.com/NixOS/nix/labels/documentation
## Priorities
Add :+1: to [issues you find important](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc).
-4
View File
@@ -9,7 +9,3 @@
<!-- Invasive change: Discuss alternative designs or approaches you considered. -->
<!-- Large change: Provide instructions to reviewers how to read the diff. -->
# Priorities
Add :+1: to [pull requests you find important](https://github.com/NixOS/nix/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc).
-1
View File
@@ -17,4 +17,3 @@ jobs:
with:
fetch-depth: 0
- run: bash scripts/check-hydra-status.sh
+9
View File
@@ -146,9 +146,18 @@ result
result-*
.vscode/
.direnv/
.envrc.local
# clangd and possibly more
.cache/
# Mac OS
.DS_Store
# ClangBuildAnalyzer output, see maintainers/buildtime_report.sh
buildtime.bin
.envrc.local
# We generate this with a Nix shell hook
/.pre-commit-config.yaml
+10 -12
View File
@@ -55,7 +55,7 @@ modified by someone else and passed on, the recipients should know
that what they have is not the original version, so that the original
author's reputation will not be affected by problems that might be
introduced by others.
Finally, software patents pose a constant threat to the existence of
any free program. We wish to make sure that a company cannot
effectively restrict the users of a free program by obtaining a
@@ -111,7 +111,7 @@ modification follow. Pay close attention to the difference between a
"work based on the library" and a "work that uses the library". The
former contains code derived from the library, whereas the latter must
be combined with the library in order to run.
GNU LESSER GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
@@ -146,7 +146,7 @@ such a program is covered only if its contents constitute a work based
on the Library (independent of the use of the Library in a tool for
writing it). Whether that is true depends on what the Library does
and what the program that uses the Library does.
1. You may copy and distribute verbatim copies of the Library's
complete source code as you receive it, in any medium, provided that
you conspicuously and appropriately publish on each copy an
@@ -158,7 +158,7 @@ Library.
You may charge a fee for the physical act of transferring a copy,
and you may at your option offer warranty protection in exchange for a
fee.
2. You may modify your copy or copies of the Library or any portion
of it, thus forming a work based on the Library, and copy and
distribute such modifications or work under the terms of Section 1
@@ -216,7 +216,7 @@ instead of to this License. (If a newer version than version 2 of the
ordinary GNU General Public License has appeared, then you can specify
that version instead if you wish.) Do not make any other change in
these notices.
Once this change is made in a given copy, it is irreversible for
that copy, so the ordinary GNU General Public License applies to all
subsequent copies and derivative works made from that copy.
@@ -267,7 +267,7 @@ Library will still fall under Section 6.)
distribute the object code for the work under the terms of Section 6.
Any executables containing that work also fall under Section 6,
whether or not they are linked directly with the Library itself.
6. As an exception to the Sections above, you may also combine or
link a "work that uses the Library" with the Library to produce a
work containing portions of the Library, and distribute that work
@@ -329,7 +329,7 @@ restrictions of other proprietary libraries that do not normally
accompany the operating system. Such a contradiction means you cannot
use both them and the Library together in an executable that you
distribute.
7. You may place library facilities that are a work based on the
Library side-by-side in a single library together with other library
facilities not covered by this License, and distribute such a combined
@@ -370,7 +370,7 @@ subject to these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties with
this License.
11. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
@@ -422,7 +422,7 @@ conditions either of that version or of any later version published by
the Free Software Foundation. If the Library does not specify a
license version number, you may choose any version ever published by
the Free Software Foundation.
14. If you wish to incorporate parts of the Library into other free
programs whose distribution conditions are incompatible with these,
write to the author to ask for permission. For software which is
@@ -456,7 +456,7 @@ SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Libraries
If you develop a new library, and you want it to be of the greatest
@@ -500,5 +500,3 @@ necessary. Here is a sample; alter the names:
Ty Coon, President of Vice
That's all there is to it!
+3 -3
View File
@@ -20,8 +20,7 @@ makefiles = \
misc/fish/local.mk \
misc/zsh/local.mk \
misc/systemd/local.mk \
misc/launchd/local.mk \
misc/upstart/local.mk
misc/launchd/local.mk
endif
ifeq ($(ENABLE_BUILD)_$(ENABLE_TESTS), yes_yes)
@@ -41,6 +40,7 @@ makefiles += \
tests/functional/ca/local.mk \
tests/functional/dyn-drv/local.mk \
tests/functional/test-libstoreconsumer/local.mk \
tests/functional/repl_characterization/local.mk \
tests/functional/plugins/local.mk
else
makefiles += \
@@ -60,7 +60,7 @@ endif
OPTIMIZE = 1
ifeq ($(OPTIMIZE), 1)
GLOBAL_CXXFLAGS += -O3 $(CXXLTO)
GLOBAL_CXXFLAGS += -O2 $(CXXLTO)
GLOBAL_LDFLAGS += $(CXXLTO)
else
GLOBAL_CXXFLAGS += -O0 -U_FORTIFY_SOURCE
+2 -2
View File
@@ -9,7 +9,7 @@ index 0468aaec..b348d869 100644
+ size_t stack_limit;
if (!EXPECT(GC_thr_initialized, TRUE))
GC_thr_init();
@@ -411,6 +412,19 @@ GC_INNER void GC_push_all_stacks(void)
GC_push_all_stack_sections(lo, hi, p->traced_stack_sect);
}
@@ -37,7 +37,7 @@ index edab6c22..f2c61282 100644
@@ -2172,6 +2172,11 @@ GC_API void GC_CALL GC_win32_free_heap(void);
(*GC_amiga_allocwrapper_do)(a,GC_malloc_atomic_ignore_off_page)
#endif /* _AMIGA && !GC_AMIGA_MAKINGLIB */
+#if !__APPLE__
+/* Patch doesn't work on apple */
+#define NIX_BOEHM_PATCH_VERSION 1
+1 -1
View File
@@ -3,7 +3,7 @@ index 597c7f13..587286be 100644
--- a/include/gc_allocator.h
+++ b/include/gc_allocator.h
@@ -312,6 +312,7 @@ public:
template<>
class traceable_allocator<void> {
+public:
+1
View File
@@ -0,0 +1 @@
BasedOnStyle: llvm
+80
View File
@@ -0,0 +1,80 @@
#include "HasPrefixSuffix.hh"
#include <clang/AST/ASTTypeTraits.h>
#include <clang/AST/Expr.h>
#include <clang/AST/PrettyPrinter.h>
#include <clang/AST/Type.h>
#include <clang/ASTMatchers/ASTMatchers.h>
#include <clang/Basic/Diagnostic.h>
#include <clang/Frontend/FrontendAction.h>
#include <clang/Frontend/FrontendPluginRegistry.h>
#include <clang/Tooling/Transformer/SourceCode.h>
#include <clang/Tooling/Transformer/SourceCodeBuilders.h>
#include <iostream>
namespace nix::clang_tidy {
using namespace clang::ast_matchers;
using namespace clang;
void HasPrefixSuffixCheck::registerMatchers(ast_matchers::MatchFinder *Finder) {
Finder->addMatcher(
traverse(clang::TK_AsIs,
callExpr(callee(functionDecl(anyOf(hasName("hasPrefix"),
hasName("hasSuffix")))
.bind("callee-decl")),
optionally(hasArgument(
0, cxxConstructExpr(
hasDeclaration(functionDecl(hasParameter(
0, parmVarDecl(hasType(
asString("const char *")))))))
.bind("implicit-cast"))))
.bind("call")),
this);
}
void HasPrefixSuffixCheck::check(
const ast_matchers::MatchFinder::MatchResult &Result) {
const auto *CalleeDecl = Result.Nodes.getNodeAs<FunctionDecl>("callee-decl");
auto FuncName = std::string(CalleeDecl->getName());
std::string NewName;
if (FuncName == "hasPrefix") {
NewName = "starts_with";
} else if (FuncName == "hasSuffix") {
NewName = "ends_with";
} else {
llvm_unreachable("nix-has-prefix: invalid callee");
}
const auto *MatchedDecl = Result.Nodes.getNodeAs<CallExpr>("call");
const auto *ImplicitConvertArg =
Result.Nodes.getNodeAs<CXXConstructExpr>("implicit-cast");
const auto *Lhs = MatchedDecl->getArg(0);
const auto *Rhs = MatchedDecl->getArg(1);
auto Diag = diag(MatchedDecl->getExprLoc(), FuncName + " is deprecated");
std::string Text = "";
// Form possible cast to string_view, or nothing.
if (ImplicitConvertArg) {
Text = "std::string_view(";
Text.append(tooling::getText(*Lhs, *Result.Context));
Text.append(").");
} else {
Text.append(*tooling::buildAccess(*Lhs, *Result.Context));
}
// Call .starts_with.
Text.append(NewName);
Text.push_back('(');
Text.append(tooling::getText(*Rhs, *Result.Context));
Text.push_back(')');
Diag << FixItHint::CreateReplacement(MatchedDecl->getSourceRange(), Text);
// for (const auto *arg : MatchedDecl->arguments()) {
// arg->dumpColor();
// arg->getType().dump();
// }
}
}; // namespace nix::clang_tidy
+25
View File
@@ -0,0 +1,25 @@
#pragma once
///@file
/// This is an example of a clang-tidy automated refactoring against the Nix
/// codebase. The refactoring has been completed in
/// https://gerrit.lix.systems/c/lix/+/565 so this code is around as
/// an example.
#include <clang-tidy/ClangTidyCheck.h>
#include <clang/ASTMatchers/ASTMatchFinder.h>
#include <llvm/ADT/StringRef.h>
namespace nix::clang_tidy {
using namespace clang;
using namespace clang::tidy;
using namespace llvm;
class HasPrefixSuffixCheck : public ClangTidyCheck {
public:
HasPrefixSuffixCheck(StringRef Name, ClangTidyContext *Context)
: ClangTidyCheck(Name, Context) {}
void registerMatchers(ast_matchers::MatchFinder *Finder) override;
void check(const ast_matchers::MatchFinder::MatchResult &Result) override;
};
}; // namespace nix::clang_tidy
+17
View File
@@ -0,0 +1,17 @@
#include <clang-tidy/ClangTidyModule.h>
#include <clang-tidy/ClangTidyModuleRegistry.h>
#include "HasPrefixSuffix.hh"
namespace nix::clang_tidy {
using namespace clang;
using namespace clang::tidy;
class NixClangTidyChecks : public ClangTidyModule {
public:
void addCheckFactories(ClangTidyCheckFactories &CheckFactories) override {
CheckFactories.registerCheck<HasPrefixSuffixCheck>("nix-hasprefixsuffix");
}
};
static ClangTidyModuleRegistry::Add<NixClangTidyChecks> X("nix-module", "Adds nix specific checks");
};
+56
View File
@@ -0,0 +1,56 @@
# Clang tidy lints for Nix
This is a skeleton of a clang-tidy lints library for Nix.
Currently there is one check (which is already obsolete as it has served its
goal and is there as an example), `HasPrefixSuffixCheck`.
## Running fixes/checks
One file:
```
ninja -C build && clang-tidy --checks='-*,nix-*' --load=build/libnix-clang-tidy.so -p ../compile_commands.json --fix ../src/libcmd/installables.cc
```
Several files, in parallel:
```
ninja -C build && run-clang-tidy -checks='-*,nix-*' -load=build/libnix-clang-tidy.so -p .. -fix ../src | tee -a clang-tidy-result
```
## Resources
* https://firefox-source-docs.mozilla.org/code-quality/static-analysis/writing-new/clang-query.html
* https://clang.llvm.org/docs/LibASTMatchersReference.html
* https://devblogs.microsoft.com/cppblog/exploring-clang-tooling-part-3-rewriting-code-with-clang-tidy/
## Developing new checks
Put something like so in `myquery.txt`:
```
set traversal IgnoreUnlessSpelledInSource
# ^ Ignore implicit AST nodes. May need to use AsIs depending on how you are
# working.
set bind-root true
# ^ true unless you use any .bind("foo") commands
set print-matcher true
enable output dump
match callExpr(callee(functionDecl(hasName("hasPrefix"))), optionally(hasArgument( 0, cxxConstructExpr(hasDeclaration(functionDecl(hasParameter(0, parmVarDecl(hasType(asString("const char *"))).bind("meow2"))))))))
```
Then run, e.g. `clang-query --preload hasprefix.query -p compile_commands.json src/libcmd/installables.cc`.
With this you can iterate a query before writing it in C++ and suffering from
C++.
### Tips and tricks for the C++
There is a function `dump()` on many things that will dump to stderr. Also
`llvm::errs()` lets you print to stderr.
When I wrote `HasPrefixSuffixCheck`, I was not really able to figure out how
the structured replacement system was supposed to work. In principle you can
describe the replacement with a nice DSL. Look up the Stencil system in Clang
for details.
+8
View File
@@ -0,0 +1,8 @@
project('nix-clang-tidy', ['cpp', 'c'],
version : '0.1',
default_options : ['warning_level=3', 'cpp_std=c++20'])
llvm = dependency('Clang', version: '>= 14', modules: ['libclang'])
sources = ['HasPrefixSuffix.cc', 'NixClangTidyChecks.cc']
shared_module('nix-clang-tidy', sources,
dependencies: llvm)
+7
View File
@@ -342,6 +342,13 @@ AC_SUBST(doc_generate)
# Look for lowdown library.
PKG_CHECK_MODULES([LOWDOWN], [lowdown >= 0.9.0], [CXXFLAGS="$LOWDOWN_CFLAGS $CXXFLAGS"])
# Look for toml11, a required dependency.
AC_ARG_VAR([TOML11_HEADERS], [include path of toml11 headers])
AC_LANG_PUSH(C++)
[CXXFLAGS="-I $TOML11_HEADERS $CXXFLAGS"]
AC_CHECK_HEADER([toml.hpp], [], [AC_MSG_ERROR([toml11 is not found.])])
AC_LANG_POP(C++)
# Setuid installations.
AC_CHECK_FUNCS([setresuid setreuid lchown])
Executable → Regular
View File
+12 -49
View File
@@ -39,67 +39,30 @@ dummy-env = env -i \
nix-eval = $(dummy-env) $(doc_nix) eval --experimental-features nix-command -I nix/corepkgs=corepkgs --store dummy:// --impure --raw
# re-implement mdBook's include directive to make it usable for terminal output and for proper @docroot@ substitution
define process-includes
while read -r line; do \
set -euo pipefail; \
filename="$$(dirname $(1))/$$(sed 's/{{#include \(.*\)}}/\1/'<<< $$line)"; \
test -f "$$filename" || ( echo "#include-d file '$$filename' does not exist." >&2; exit 1; ); \
matchline="$$(sed 's|/|\\/|g' <<< $$line)"; \
sed -i "/$$matchline/r $$filename" $(2); \
sed -i "s/$$matchline//" $(2); \
done < <(grep '{{#include' $(1))
endef
$(d)/nix-env-%.1: $(d)/src/command-ref/nix-env/%.md
@printf "Title: %s\n\n" "$(subst nix-env-,nix-env --,$$(basename "$@" .1))" > $^.tmp
$(render-subcommand)
$(trace-gen) doc/manual/render-manpage.sh \
--unescape-dashes "$(subst nix-env-,nix-env --,$$(basename "$@" .1))" 1 $^ $^.tmp $@
$(d)/nix-store-%.1: $(d)/src/command-ref/nix-store/%.md
@printf -- 'Title: %s\n\n' "$(subst nix-store-,nix-store --,$$(basename "$@" .1))" > $^.tmp
$(render-subcommand)
# FIXME: there surely is some more deduplication to be achieved here with even darker Make magic
define render-subcommand
@cat $^ >> $^.tmp
@$(call process-includes,$^,$^.tmp)
$(trace-gen) lowdown -sT man --nroff-nolinks -M section=1 $^.tmp -o $@
@# fix up `lowdown`'s automatic escaping of `--`
@# https://github.com/kristapsdz/lowdown/blob/edca6ce6d5336efb147321a43c47a698de41bb7c/entity.c#L202
@sed -i 's/\e\[u2013\]/--/' $@
@rm $^.tmp
endef
$(trace-gen) doc/manual/render-manpage.sh \
--unescape-dashes "$(subst nix-store-,nix-store --,$$(basename "$@" .1))" 1 $^ $^.tmp $@
$(d)/%.1: $(d)/src/command-ref/%.md
@printf "Title: %s\n\n" "$$(basename $@ .1)" > $^.tmp
@cat $^ >> $^.tmp
@$(call process-includes,$^,$^.tmp)
$(trace-gen) lowdown -sT man --nroff-nolinks -M section=1 $^.tmp -o $@
@rm $^.tmp
$(trace-gen) doc/manual/render-manpage.sh "$$(basename $@ .1)" 1 $^ $^.tmp $@
$(d)/%.8: $(d)/src/command-ref/%.md
@printf "Title: %s\n\n" "$$(basename $@ .8)" > $^.tmp
@cat $^ >> $^.tmp
$(trace-gen) lowdown -sT man --nroff-nolinks -M section=8 $^.tmp -o $@
@rm $^.tmp
$(trace-gen) doc/manual/render-manpage.sh "$$(basename $@ .8)" 8 $^ $^.tmp $@
$(d)/nix.conf.5: $(d)/src/command-ref/conf-file.md
@printf "Title: %s\n\n" "$$(basename $@ .5)" > $^.tmp
@cat $^ >> $^.tmp
@$(call process-includes,$^,$^.tmp)
$(trace-gen) lowdown -sT man --nroff-nolinks -M section=5 $^.tmp -o $@
@rm $^.tmp
$(trace-gen) doc/manual/render-manpage.sh "$$(basename $@ .5)" 5 $^ $^.tmp $@
$(d)/nix-profiles.5: $(d)/src/command-ref/files/profiles.md
@printf "Title: %s\n\n" "$$(basename $@ .5)" > $^.tmp
@cat $^ >> $^.tmp
$(trace-gen) lowdown -sT man --nroff-nolinks -M section=5 $^.tmp -o $@
@rm $^.tmp
$(trace-gen) doc/manual/render-manpage.sh "$$(basename $@ .5)" 5 $^ $^.tmp $@
$(d)/src/SUMMARY.md: $(d)/src/SUMMARY.md.in $(d)/src/SUMMARY-rl-next.md $(d)/src/command-ref/new-cli $(d)/src/contributing/experimental-feature-descriptions.md
@cp $< $@
@$(call process-includes,$@,$@)
@doc/manual/process-includes.sh $@ $@
$(d)/src/command-ref/new-cli: $(d)/nix.json $(d)/utils.nix $(d)/generate-manpage.nix $(doc_nix)
@rm -rf $@ $@.tmp
@@ -151,9 +114,9 @@ $(d)/language.json: $(doc_nix)
# Generate "Upcoming release" notes (or clear it and remove from menu)
$(d)/src/release-notes/rl-next.md: $(d)/rl-next $(d)/rl-next/*
@if type -p changelog-d > /dev/null; then \
@if type -p build-release-notes > /dev/null; then \
echo " GEN " $@; \
changelog-d doc/manual/rl-next > $@; \
build-release-notes doc/manual/rl-next > $@; \
else \
echo " NULL " $@; \
true > $@; \
@@ -201,7 +164,7 @@ $(docdir)/manual/index.html: $(MANUAL_SRCS) $(d)/book.toml $(d)/anchors.jq $(d)/
tmp="$$(mktemp -d)"; \
cp -r doc/manual "$$tmp"; \
find "$$tmp" -name '*.md' | while read -r file; do \
$(call process-includes,$$file,$$file); \
doc/manual/process-includes.sh $$file $$file; \
done; \
find "$$tmp" -name '*.md' | while read -r file; do \
docroot="$$(realpath --relative-to="$$(dirname "$$file")" $$tmp/manual/src)"; \
+290
View File
@@ -0,0 +1,290 @@
nix_env_for_docs = {
'HOME': '/dummy',
'NIX_CONF_DIR': '/dummy',
'NIX_SSL_CERT_FILE': '/dummy/no-ca-bundle.crt',
'NIX_STATE_DIR': '/dummy',
'NIX_CONFIG': 'cores = 0',
}
nix_for_docs = [ nix, '--experimental-features', 'nix-command' ]
nix_eval_for_docs = nix_for_docs + [
'eval',
'-I', 'nix/corepkgs=corepkgs',
'--store', 'dummy://',
'--impure',
'--raw',
]
nix_conf_file_json = custom_target(
command : nix_for_docs + [ 'show-config', '--json' ],
capture : true,
output : 'conf-file.json',
env : nix_env_for_docs,
)
nix_conf_file_md_body = custom_target(
command : nix_eval_for_docs + [
'--expr',
'(import @INPUT0@).showSettings { inlineHTML = true; } (builtins.fromJSON (builtins.readFile @INPUT1@))',
],
capture : true,
input : [
'utils.nix',
nix_conf_file_json,
],
output : 'conf-file.md.body',
env : nix_env_for_docs,
)
nix_conf_file_md = custom_target(
command : [ 'cat', '@INPUT@' ],
capture : true,
input : [
'src/command-ref/conf-file-prefix.md',
nix_conf_file_md_body,
],
output : 'conf-file.md',
)
nix_exp_features_json = custom_target(
command : [ nix, '__dump-xp-features' ],
capture : true,
output : 'xp-features.json',
)
nix_exp_feature_shortlist = custom_target(
command : nix_eval_for_docs + [
'--expr',
'import @INPUT0@ (builtins.fromJSON (builtins.readFile @INPUT1@))',
],
input : [
'generate-xp-features-shortlist.nix',
nix_exp_features_json,
],
output : 'experimental-features-shortlist.md',
capture : true,
env : nix_env_for_docs,
)
nix_nested_manpages = [
[ 'nix-env',
[
'delete-generations',
'install',
'list-generations',
'query',
'rollback',
'set-flag',
'set',
'switch-generation',
'switch-profile',
'uninstall',
'upgrade',
],
],
[ 'nix-store',
[
'add-fixed',
'add',
'delete',
'dump-db',
'dump',
'export',
'gc',
'generate-binary-cache-key',
'import',
'load-db',
'optimise',
'print-env',
'query',
'read-log',
'realise',
'repair-path',
'restore',
'serve',
'verify',
'verify-path',
],
],
]
foreach command : nix_nested_manpages
foreach page : command[1]
custom_target(
command : [
'./render-manpage.sh',
'--unescape-dashes',
command[0] + ' --' + page,
'1',
'@INPUT@',
'@OUTPUT@.tmp',
'@OUTPUT@',
],
input : 'src/command-ref' / command[0] / (page + '.md'),
output : command[0] + '-' + page + '.1',
install : true,
install_dir : mandir / 'man1',
)
endforeach
endforeach
nix3_cli_json = custom_target(
command : [ nix, '__dump-cli' ],
capture : true,
output : 'nix.json',
)
nix3_cli_files = custom_target(
command : nix_eval_for_docs + [
'--write-to', '@OUTPUT@',
'--expr',
'import @INPUT1@ true (builtins.readFile @INPUT0@)',
],
input : [
nix3_cli_json,
'generate-manpage.nix',
'utils.nix',
],
output : 'new-cli',
)
nix3_manpages = [
'nix3-build',
'nix3-bundle',
'nix3-copy',
'nix3-daemon',
'nix3-derivation-add',
'nix3-derivation',
'nix3-derivation-show',
'nix3-develop',
'nix3-doctor',
'nix3-edit',
'nix3-eval',
'nix3-flake-archive',
'nix3-flake-check',
'nix3-flake-clone',
'nix3-flake-info',
'nix3-flake-init',
'nix3-flake-lock',
'nix3-flake',
'nix3-flake-metadata',
'nix3-flake-new',
'nix3-flake-prefetch',
'nix3-flake-show',
'nix3-flake-update',
'nix3-fmt',
'nix3-hash-file',
'nix3-hash',
'nix3-hash-path',
'nix3-hash-to-base16',
'nix3-hash-to-base32',
'nix3-hash-to-base64',
'nix3-hash-to-sri',
'nix3-help',
'nix3-help-stores',
'nix3-key-convert-secret-to-public',
'nix3-key-generate-secret',
'nix3-key',
'nix3-log',
'nix3-nar-cat',
'nix3-nar-dump-path',
'nix3-nar-ls',
'nix3-nar',
'nix3-path-info',
'nix3-print-dev-env',
'nix3-profile-diff-closures',
'nix3-profile-history',
'nix3-profile-install',
'nix3-profile-list',
'nix3-profile',
'nix3-profile-remove',
'nix3-profile-rollback',
'nix3-profile-upgrade',
'nix3-profile-wipe-history',
'nix3-realisation-info',
'nix3-realisation',
'nix3-registry-add',
'nix3-registry-list',
'nix3-registry',
'nix3-registry-pin',
'nix3-registry-remove',
'nix3-repl',
'nix3-run',
'nix3-search',
'nix3-shell',
'nix3-show-config',
'nix3-store-add-file',
'nix3-store-add-path',
'nix3-store-cat',
'nix3-store-copy-log',
'nix3-store-copy-sigs',
'nix3-store-delete',
'nix3-store-diff-closures',
'nix3-store-dump-path',
'nix3-store-gc',
'nix3-store-ls',
'nix3-store-make-content-addressed',
'nix3-store',
'nix3-store-optimise',
'nix3-store-path-from-hash-part',
'nix3-store-ping',
'nix3-store-prefetch-file',
'nix3-store-repair',
'nix3-store-sign',
'nix3-store-verify',
'nix3-upgrade-nix',
'nix3-why-depends',
'nix',
]
foreach page : nix3_manpages
custom_target(
command : [
'./render-manpage.sh',
page,
'1',
'@INPUT0@/' + page + '.md',
'@OUTPUT@.tmp',
'@OUTPUT@',
],
input : nix3_cli_files,
output : page + '.1',
install : true,
install_dir : mandir / 'man1',
)
endforeach
nix_manpages = [
[ 'nix-env', 1 ],
[ 'nix-store', 1 ],
[ 'nix-build', 1 ],
[ 'nix-shell', 1 ],
[ 'nix-instantiate', 1 ],
[ 'nix-collect-garbage', 1 ],
[ 'nix-prefetch-url', 1 ],
[ 'nix-channel', 1 ],
[ 'nix-hash', 1 ],
[ 'nix-copy-closure', 1 ],
[ 'nix.conf', 5, nix_conf_file_md, nix_exp_feature_shortlist ],
[ 'nix-daemon', 8 ],
[ 'nix-profiles', 5, 'src/command-ref/files/profiles.md', nix_exp_feature_shortlist ],
]
foreach entry : nix_manpages
custom_target(
command : [
'./render-manpage.sh',
entry[0],
entry[1].to_string(),
'@INPUT0@',
'@OUTPUT@.tmp',
'@OUTPUT@',
meson.current_build_dir(),
],
input : [
entry.get(2, 'src/command-ref' / (entry[0] + '.md')),
entry.get(3, []),
],
output : '@0@.@1@'.format(entry[0], entry[1]),
install : true,
install_dir : mandir / 'man@0@'.format(entry[1]),
)
endforeach
+32
View File
@@ -0,0 +1,32 @@
#!/bin/sh
set -euo pipefail
infile="$1"
outfile="$2"
shift 2
# set a search path for includes. the old makefile-based system splorked
# everything into the source tree and was thus able to not have a search
# path, but the meson system generates intermediate files into dedicated
# directories separate from the source. we still retain the implicit old
# behavior for now as the base search path, once meson is the default we
# can revisit this and remove the implicit search path entry. it's fine.
set -- "$(dirname "$infile")" "$@"
# re-implement mdBook's include directive to make it usable for terminal output and for proper @docroot@ substitution
(grep '{{#include' "$infile" || true) | while read -r line; do
found=false
include="$(printf "$line" | sed 's/{{#include \(.*\)}}/\1/')"
for path in "$@"; do
filename="$path/$include"
if [ -e "$filename" ]; then
found=true
matchline="$(printf "$line" | sed 's|/|\\/|g')"
sed -i "/$matchline/r $filename" "$outfile"
sed -i "s/$matchline//" "$outfile"
break
fi
done
$found || ( echo "#include-d file '$filename' does not exist." >&2; exit 1; )
done
+27
View File
@@ -0,0 +1,27 @@
#!/bin/sh
set -euo pipefail
unescape_dashes=
if [ "$1" = --unescape-dashes ]; then
unescape_dashes=yes
shift
fi
title="$1"
section="$2"
infile="$3"
tmpfile="$4"
outfile="$5"
printf "Title: %s\n\n" "$title" > "$tmpfile"
cat "$infile" >> "$tmpfile"
"$(dirname "$0")"/process-includes.sh "$infile" "$tmpfile"
lowdown -sT man --nroff-nolinks -M section="$section" "$tmpfile" -o "$outfile"
if [ -n "$unescape_dashes" ]; then
# fix up `lowdown`'s automatic escaping of `--`
# https://github.com/kristapsdz/lowdown/blob/edca6ce6d5336efb147321a43c47a698de41bb7c/entity.c#L202
sed -i 's/\e\[u2013\]/--/' "$outfile"
fi
rm "$tmpfile"
@@ -0,0 +1,15 @@
---
synopsis: Clang build timing analysis
cls: 587
---
We now have Clang build profiling available, which generates Chrome
tracing files for each compilation unit. To enable it, run `meson configure
build -Dprofile-build=enabled` then rerun the compilation.
If you want to make the build go faster, do a clang build with meson, then run
`maintainers/buildtime_report.sh build`, then contemplate how to improve the
build time.
You can also look at individual object files' traces in
<https://ui.perfetto.dev>.
@@ -37,4 +37,3 @@ error:
error: uh oh!
```
-2
View File
@@ -1,2 +0,0 @@
organization: NixOS
repository: nix
@@ -1,7 +1,7 @@
---
synopsis: "`--debugger` can now access bindings from `let` expressions"
prs: 9918
issues: 8827.
issues: 8827
---
Breakpoints and errors in the bindings of a `let` expression can now access
+9
View File
@@ -0,0 +1,9 @@
---
synopsis: Enter the `--debugger` when `builtins.trace` is called if `debugger-on-trace` is set
prs: 9914
---
If the `debugger-on-trace` option is set and `--debugger` is given,
`builtins.trace` calls will behave similarly to `builtins.break` and will enter
the debug REPL. This is useful for determining where warnings are being emitted
from.
@@ -0,0 +1,7 @@
---
synopsis: Stop vendoring toml11
cls: 675
---
We don't apply any patches to it, and vendoring it locks users into
bugs (it hasn't been updated since its introduction in late 2021).
+22
View File
@@ -0,0 +1,22 @@
---
synopsis: Duplicate attribute reports are more accurate
cls: 557
---
Duplicate attribute errors are now more accurate, showing the path at which an error was detected rather than the full, possibly longer, path that caused the error.
Error reports are now
```ShellSession
$ nix eval --expr '{ a.b = 1; a.b.c.d = 1; }'
error: attribute 'a.b' already defined at «string»:1:3
at «string»:1:12:
1| { a.b = 1; a.b.c.d = 1;
| ^
```
instead of
```ShellSession
$ nix eval --expr '{ a.b = 1; a.b.c.d = 1; }'
error: attribute 'a.b.c.d' already defined at «string»:1:3
at «string»:1:12:
1| { a.b = 1; a.b.c.d = 1;
| ^
```
+3 -5
View File
@@ -1,8 +1,6 @@
---
synopsis: Disallow empty search regex in `nix search`
prs: #9481
description: {
prs: 9481
---
[`nix search`](@docroot@/command-ref/new-cli/nix3-search.md) now requires a search regex to be passed. To show all packages, use `^`.
}
+7
View File
@@ -0,0 +1,7 @@
---
synopsis: consistent order of lambda formals in printed expressions
prs: 9874
---
Always print lambda formals in lexicographic order rather than the internal, creation-time based symbol order.
This makes printed formals independent of the context they appear in.
@@ -0,0 +1,6 @@
---
synopsis: fix duplicate attribute error positions for `inherit`
prs: 9874
---
When an inherit caused a duplicate attribute error the position of the error was not reported correctly, placing the error with the inherit itself or at the start of the bindings block instead of the offending attribute name.
@@ -0,0 +1,7 @@
---
synopsis: "`inherit (x) ...` evaluates `x` only once"
prs: 9847
---
`inherit (x) a b ...` now evaluates the expression `x` only once for all inherited attributes rather than once for each inherited attribute.
This does not usually have a measurable impact, but side-effects (such as `builtins.trace`) would be duplicated and expensive expressions (such as derivations) could cause a measurable slowdown.
+17
View File
@@ -0,0 +1,17 @@
---
synopsis: "`Overhaul `nix flake update` and `nix flake lock` UX"
prs: 8817
---
The interface for creating and updating lock files has been overhauled:
- [`nix flake lock`](@docroot@/command-ref/new-cli/nix3-flake-lock.md) only creates lock files and adds missing inputs now.
It will *never* update existing inputs.
- [`nix flake update`](@docroot@/command-ref/new-cli/nix3-flake-update.md) does the same, but *will* update inputs.
- Passing no arguments will update all inputs of the current flake, just like it already did.
- Passing input names as arguments will ensure only those are updated. This replaces the functionality of `nix flake lock --update-input`
- To operate on a flake outside the current directory, you must now pass `--flake path/to/flake`.
- The flake-specific flags `--recreate-lock-file` and `--update-input` have been removed from all commands operating on installables.
They are superceded by `nix flake update`.
+11
View File
@@ -0,0 +1,11 @@
---
synopsis: "`builtins.nixVersion` now returns a fixed value \"2.18.3-lix\""
cls: 558
---
`builtins.nixVersion` now returns a fixed value `"2.18.3-lix"`. This prevents
feature detection assuming that features that exist in Nix post-Lix-branch-off
might exist, even though the Lix version is greater than the Nix version.
In the future, check for builtins for feature detection. If a feature cannot be
detected by *those* means, please file a Lix bug.
@@ -1,7 +1,7 @@
---
synopsis: Coercion errors include the failing value
issues: #561
prs: #9754
issues: 561
prs: 9754
---
The `error: cannot coerce a <TYPE> to a string` message now includes the value
@@ -1,7 +1,7 @@
---
synopsis: Type errors include the failing value
issues: #561
prs: #9753
issues: 561
prs: 9753
---
In errors like `value is an integer while a list was expected`, the message now
+14
View File
@@ -0,0 +1,14 @@
---
synopsis: reintroduce shortened `-E` form for `--expr` to new CLI
cls: 605
---
In the past, it was possible to supply a shorter `-E` flag instead of fully
specifying `--expr` every time you wished to provide an expression that would
be evaluated to produce the given command's input. This was retained for the
`--file` flag when the new CLI utilities were written with `-f`, but `-E` was
dropped.
We now restore the `-E` short form for better UX. This is most useful for
`nix eval` but most any command that takes an Installable argument should benefit
from it as well.
+8
View File
@@ -0,0 +1,8 @@
---
synopsis: Upstart scripts removed
cls: 574
---
Upstart scripts have been removed from Lix, since Upstart is obsolete and has
not been shipped by any major distributions for many years. If these are
necessary to your use case, please back port them to your packaging.
-1
View File
@@ -27,4 +27,3 @@
/package-management/channels* /command-ref/nix-channel 301!
/package-management/s3-substituter* /command-ref/new-cli/nix3-help-stores#s3-binary-cache-store 301!
+4 -4
View File
@@ -102,13 +102,13 @@ In particular, notice the
has copied the build results to that directory where you can examine it.
> []{#check-dirs-are-unregistered} **Note**
>
>
> Check paths are not protected against garbage collection, and this
> path will be deleted on the next garbage collection.
>
>
> The path is guaranteed to be alive for the duration of
> the `diff-hook`'s execution, but may be deleted any time after.
>
>
> If the comparison is performed as part of automated tooling, please
> use the diff-hook or author your tooling to handle the case where the
> build was not deterministic and also a check path does not exist.
@@ -117,7 +117,7 @@ has copied the build results to that directory where you can examine it.
already. If the derivation has not been built Nix will fail with the
error:
error: some outputs of '/nix/store/hzi1h60z2qf0nb85iwnpvrai3j2w7rr6-unstable.drv'
error: some outputs of '/nix/store/hzi1h60z2qf0nb85iwnpvrai3j2w7rr6-unstable.drv'
are not valid, so checking is not possible
Run the build without `--check`, and then try with `--check` again.
@@ -112,4 +112,3 @@ Build inputs used as instructions to a build task are marked accordingly:
| |
+--------------------------------------------------------------------+
```
@@ -67,4 +67,3 @@ Configuration options can be set on the command line, overriding the values set
The `extra-` prefix is supported for settings that take a list of items (e.g. `--extra-trusted users alice` or `--option extra-trusted-users alice`).
# Available settings
@@ -184,4 +184,3 @@ channel:
```console
$ nix-env --file https://github.com/NixOS/nixpkgs/archive/nixos-14.12.tar.gz --install --attr firefox
```
@@ -30,4 +30,3 @@ $ nix-env --list-generations
97 2004-02-06 16:22:45
98 2004-02-06 16:24:33 (current)
```
@@ -212,4 +212,3 @@ To show all packages in the latest revision of the Nixpkgs repository:
```console
$ nix-env --file https://github.com/NixOS/nixpkgs/archive/master.tar.gz --query --available
```
@@ -79,4 +79,3 @@ To make files from `binutils` take precedence over files from `gcc`:
$ nix-env --set-flag priority 5 binutils
$ nix-env --set-flag priority 10 gcc
```
@@ -27,4 +27,3 @@ contain just Firefox:
```console
$ nix-env --profile /nix/var/nix/profiles/browser --set firefox
```
@@ -30,4 +30,3 @@ Switching will fail if the specified generation does not exist.
$ nix-env --switch-generation 42
switching from generation 50 to 42
```
@@ -138,4 +138,3 @@ This is illustrated by the following examples:
2.3a < 2.3c
2.3pre1 < 2.3c
2.3pre1 < 2.3q
@@ -32,4 +32,3 @@ This operation has the following options:
$ nix-store --add-fixed sha256 ./hello-2.10.tar.gz
/nix/store/3x7dwzq014bblazs7kq20p9hyzz0qh8g-hello-2.10.tar.gz
```
@@ -69,4 +69,3 @@ To delete at least 100 MiBs of unreachable paths:
```console
$ nix-store --gc --max-freed $((100 * 1024 * 1024))
```
@@ -33,4 +33,3 @@ The following options are allowed for all `nix-store` operations, but may not al
If there are multiple results, then multiple symlinks will be
created by sequentially numbering symlinks beyond the first one
(e.g., `foo`, `foo-2`, `foo-3`, and so on).
@@ -37,4 +37,3 @@ hashing files in `/nix/store/qhqx7l2f1kmwihc9bnxs7rc159hsxnf3-gcc-4.1.1'
541838819 bytes (516.74 MiB) freed by hard-linking 54143 files;
there are 114486 files with equal contents out of 215894 files in total
```
@@ -28,4 +28,3 @@ export stdenv; stdenv='/nix/store/7c8asx3yfrg5dg1gzhzyq2236zfgibnn-stdenv'
export system; system='x86_64-linux'
export _args; _args='-e /nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25c-default-builder.sh'
```
@@ -225,4 +225,3 @@ $ nix-store --query --roots $(which svn)
/nix/var/nix/profiles/default-82-link
/home/eelco/.local/state/nix/profiles/profile-97-link
```
@@ -35,4 +35,3 @@ ktorrent-2.2.1/
ktorrent-2.2.1/NEWS
...
```
@@ -32,4 +32,3 @@ $ nix-store --repair-path /nix/store/dj7a81wsm1ijwwpkks3725661h3263p5-glibc-2.13
fetching path `/nix/store/d7a81wsm1ijwwpkks3725661h3263p5-glibc-2.13'...
```
@@ -35,4 +35,3 @@ $ cat <<EOF >>/root/.ssh/authorized_keys
command="nice -n20 nix-store --serve --write" ssh-rsa AAAAB3NzaC1yc2EAAAA...
EOF
```
@@ -26,4 +26,3 @@ To verify the integrity of the `svn` command and all its dependencies:
```console
$ nix-store --verify-path $(nix-store --query --requisites $(which svn))
```
+1 -1
View File
@@ -37,7 +37,7 @@ Most Nix commands accept the following command-line options:
Print even more informational messages.
- `4` “Debug”
Print debug information.
- `5` “Vomit”
-3
View File
@@ -340,9 +340,6 @@ Significant changes should add the following header, which moves them to the top
significance: significant
```
<!-- Keep an eye on https://codeberg.org/fgaz/changelog-d/issues/1 -->
See also the [format documentation](https://github.com/haskell/cabal/blob/master/CONTRIBUTING.md#changelog).
### Build process
Releases have a precomputed `rl-MAJOR.MINOR.md`, and no `rl-next.md`.
-1
View File
@@ -289,4 +289,3 @@ search/replaced in it for each new build.
The installer now supports a `--tarball-url-prefix` flag which _may_ have
solved this need?
-->
@@ -21,7 +21,7 @@ Nix keeps its *store* (the place where packages are stored) in
`--with-store-dir=path`.
> **Warning**
>
>
> It is best *not* to change the Nix store from its default, since doing
> so makes it impossible to use pre-built binaries from the standard
> Nixpkgs channels — that is, all packages will need to be built from
+1 -1
View File
@@ -14,7 +14,7 @@ builds) are forwarded to a *Nix daemon* running under the owner of the
Nix store/database that performs the operation.
> **Note**
>
>
> Multi-user mode has one important limitation: only root and a set of
> trusted users specified in `nix.conf` can specify arbitrary binary
> caches. So while unprivileged users may install packages from
-1
View File
@@ -145,4 +145,3 @@ which you may remove.
> complete. macOS (Catalina+) directly controls root directories and its
> read-only root will prevent you from manually deleting the empty `/nix`
> mountpoint.
+14 -14
View File
@@ -146,65 +146,65 @@ three kinds of patterns:
- If a pattern is a single identifier, then the function matches any
argument. Example:
```nix
let negate = x: !x;
concat = x: y: x + y;
in if negate true then concat "foo" "bar" else ""
```
Note that `concat` is a function that takes one argument and returns
a function that takes another argument. This allows partial
parameterisation (i.e., only filling some of the arguments of a
function); e.g.,
```nix
map (concat "foo") [ "bar" "bla" "abc" ]
```
evaluates to `[ "foobar" "foobla" "fooabc" ]`.
- A *set pattern* of the form `{ name1, name2, …, nameN }` matches a
set containing the listed attributes, and binds the values of those
attributes to variables in the function body. For example, the
function
```nix
{ x, y, z }: z + y + x
```
can only be called with a set containing exactly the attributes `x`,
`y` and `z`. No other attributes are allowed. If you want to allow
additional arguments, you can use an ellipsis (`...`):
```nix
{ x, y, z, ... }: z + y + x
```
This works on any set that contains at least the three named
attributes.
It is possible to provide *default values* for attributes, in
which case they are allowed to be missing. A default value is
specified by writing `name ? e`, where *e* is an arbitrary
expression. For example,
```nix
{ x, y ? "foo", z ? "bar" }: z + y + x
```
specifies a function that only requires an attribute named `x`, but
optionally accepts `y` and `z`.
- An `@`-pattern provides a means of referring to the whole value
being matched:
```nix
args@{ x, y, z, ... }: z + y + x + args.a
```
but can also be written as:
```nix
{ x, y, z, ... } @ args: z + y + x + args.a
```
+19 -19
View File
@@ -24,22 +24,22 @@ the attributes of which specify the inputs of the build.
- Every attribute is passed as an environment variable to the builder.
Attribute values are translated to environment variables as follows:
- Strings and numbers are just passed verbatim.
- A *path* (e.g., `../foo/sources.tar`) causes the referenced file
to be copied to the store; its location in the store is put in
the environment variable. The idea is that all sources should
reside in the Nix store, since all inputs to a derivation should
reside in the Nix store.
- A *derivation* causes that derivation to be built prior to the
present derivation; its default output path is put in the
environment variable.
- Lists of the previous types are also allowed. They are simply
concatenated, separated by spaces.
- `true` is passed as the string `1`, `false` and `null` are
passed as an empty string.
@@ -56,36 +56,36 @@ the attributes of which specify the inputs of the build.
library doesnt need the header files and documentation at runtime,
and it doesnt need the documentation at build time. Thus, the
library package could specify:
```nix
outputs = [ "lib" "headers" "doc" ];
```
This will cause Nix to pass environment variables `lib`, `headers`
and `doc` to the builder containing the intended store paths of each
output. The builder would typically do something like
```bash
./configure \
--libdir=$lib/lib \
--includedir=$headers/include \
--docdir=$doc/share/doc
```
for an Autoconf-style package. You can refer to each output of a
derivation by selecting it as an attribute, e.g.
```nix
buildInputs = [ pkg.lib pkg.headers ];
```
The first element of `outputs` determines the *default output*.
Thus, you could also write
```nix
buildInputs = [ pkg pkg.headers ];
```
since `pkg` is equivalent to `pkg.lib`.
The function `mkDerivation` in the Nixpkgs standard environment is a
@@ -103,24 +103,24 @@ The builder is executed as follows:
specified above.
- In addition, the following variables are set:
- `NIX_BUILD_TOP` contains the path of the temporary directory for
this build.
- Also, `TMPDIR`, `TEMPDIR`, `TMP`, `TEMP` are set to point to the
temporary directory. This is to prevent the builder from
accidentally writing temporary files anywhere else. Doing so
might cause interference by other processes.
- `PATH` is set to `/path-not-set` to prevent shells from
initialising it to their built-in default value.
- `HOME` is set to `/homeless-shelter` to prevent programs from
using `/etc/passwd` or the like to find the user's home
directory, which could cause impurity. Usually, when `HOME` is
set, it is used as the location of the home directory, even if
it points to a non-existent path.
- `NIX_STORE` is set to the path of the top-level Nix store
directory (typically, `/nix/store`).
@@ -128,7 +128,7 @@ The builder is executed as follows:
is set to `true` for the dervation. A detailed explanation of this
behavior can be found in the
[section about structured attrs](./advanced-attributes.md#adv-attr-structuredAttrs).
- For each output declared in `outputs`, the corresponding
environment variable is set to point to the intended path in the
Nix store for that output. Each output path is a concatenation
-1
View File
@@ -161,4 +161,3 @@ All comparison operators are implemented in terms of `<`, and the following equi
Equivalent to `!`*b1* `||` *b2*.
[Logical implication]: #logical-implication
@@ -37,7 +37,7 @@ $ nix-channel --update
```
> **Note**
>
>
> On NixOS, youre automatically subscribed to a NixOS channel
> corresponding to your NixOS major release (e.g.
> <http://nixos.org/channels/nixos-21.11>). A NixOS channel is identical
@@ -120,7 +120,7 @@ will install the package called `subversion` from `nixpkgs` channel (which is, o
[Subversion version management system](http://subversion.tigris.org/)).
> **Note**
>
>
> When you ask Nix to install a package, it will first try to get it in
> pre-compiled form from a *binary cache*. By default, Nix will use the
> binary cache <https://cache.nixos.org>; it contains binaries for most
@@ -16,7 +16,7 @@ it will fall back to using the binary cache substituter, and then to
building from source.
> **Note**
>
>
> The SSH substituter currently does not allow you to enter an SSH
> passphrase interactively. Therefore, you should use `ssh-add` to load
> the decrypted private key into `ssh-agent`.
+33 -33
View File
@@ -1,67 +1,67 @@
# Release 0.10 (2006-10-06)
> **Note**
>
>
> This version of Nix uses Berkeley DB 4.4 instead of 4.3. The database
> is upgraded automatically, but you should be careful not to use old
> versions of Nix that still use Berkeley DB 4.3. In particular, if you
> use a Nix installed through Nix, you should run
>
>
> $ nix-store --clear-substitutes
>
>
> first.
> **Warning**
>
>
> Also, the database schema has changed slighted to fix a performance
> issue (see below). When you run any Nix 0.10 command for the first
> time, the database will be upgraded automatically. This is
> irreversible.
- `nix-env` usability improvements:
- An option `--compare-versions` (or `-c`) has been added to
`nix-env
--query` to allow you to compare installed versions of packages
to available versions, or vice versa. An easy way to see if you
are up to date with whats in your subscribed channels is
`nix-env -qc \*`.
- `nix-env --query` now takes as arguments a list of package names
about which to show information, just like `--install`, etc.:
for example, `nix-env -q gcc`. Note that to show all
derivations, you need to specify `\*`.
- `nix-env -i
pkgname` will now install the highest available version of
*pkgname*, rather than installing all available versions (which
would probably give collisions) (`NIX-31`).
- `nix-env (-i|-u) --dry-run` now shows exactly which missing
paths will be built or substituted.
- `nix-env -qa --description` shows human-readable descriptions of
packages, provided that they have a `meta.description` attribute
(which most packages in Nixpkgs dont have yet).
- New language features:
- Reference scanning (which happens after each build) is much
faster and takes a constant amount of memory.
- String interpolation. Expressions like
"--with-freetype2-library=" + freetype + "/lib"
can now be written as
"--with-freetype2-library=${freetype}/lib"
You can write arbitrary expressions within `${...}`, not just
identifiers.
- Multi-line string literals.
- String concatenations can now involve derivations, as in the
example `"--with-freetype2-library="
+ freetype + "/lib"`. This was not previously possible because
@@ -69,17 +69,17 @@
dependent on `freetype`. The evaluator now properly propagates
this information. Consequently, the subpath operator (`~`) has
been deprecated.
- Default values of function arguments can now refer to other
function arguments; that is, all arguments are in scope in the
default values (`NIX-45`).
- Lots of new built-in primitives, such as functions for list
manipulation and integer arithmetic. See the manual for a
complete list. All primops are now available in the set
`builtins`, allowing one to test for the availability of primop
in a backwards-compatible way.
- Real let-expressions: `let x = ...;
... z = ...; in ...`.
@@ -89,15 +89,15 @@
your machine and you want to copy it somewhere else.
- XML support:
- `nix-env -q --xml` prints the installed or available packages in
an XML representation for easy processing by other tools.
- `nix-instantiate --eval-only
--xml` prints an XML representation of the resulting term. (The
new flag `--strict` forces deep evaluation of the result,
i.e., list elements and attributes are evaluated recursively.)
- In Nix expressions, the primop `builtins.toXML` converts a term
to an XML representation. This is primarily useful for passing
structured information to builders.
@@ -133,21 +133,21 @@
something other than `result`.
- Platform support:
- Support for 64-bit platforms, provided a [suitably patched ATerm
library](http://bugzilla.sen.cwi.nl:8080/show_bug.cgi?id=606) is
used. Also, files larger than 2 GiB are now supported.
- Added support for Cygwin (Windows, `i686-cygwin`), Mac OS X on
Intel (`i686-darwin`) and Linux on PowerPC (`powerpc-linux`).
- Users of SMP and multicore machines will appreciate that the
number of builds to be performed in parallel can now be
specified in the configuration file in the `build-max-jobs`
setting.
- Garbage collector improvements:
- Open files (such as running programs) are now used as roots of
the garbage collector. This prevents programs that have been
uninstalled from being garbage collected while they are still
@@ -155,17 +155,17 @@
(`find-runtime-roots.pl`) is inherently system-specific, but it
should work on Linux and on all platforms that have the `lsof`
utility.
- `nix-store --gc` (a.k.a. `nix-collect-garbage`) prints out the
number of bytes freed on standard output. `nix-store
--gc --print-dead` shows how many bytes would be freed by an
actual garbage collection.
- `nix-collect-garbage -d` removes all old generations of *all*
profiles before calling the actual garbage collector (`nix-store
--gc`). This is an easy way to get rid of all old packages in
the Nix store.
- `nix-store` now has an operation `--delete` to delete specific
paths from the Nix store. It wont delete reachable
(non-garbage) paths unless `--ignore-liveness` is specified.
@@ -192,13 +192,13 @@
memoisation of path hashing.
- Lots of bug fixes, notably:
- Make sure that the garbage collector can run successfully when
the disk is full (`NIX-18`).
- `nix-env` now locks the profile to prevent races between
concurrent `nix-env` operations on the same profile (`NIX-7`).
- Removed misleading messages from `nix-env -i` (e.g.,
``installing
`foo'`` followed by ``uninstalling
+4 -4
View File
@@ -50,7 +50,7 @@ on Nix. Here is an (incomplete) list:
multiple packages with the same name, then pick the package with the
highest priority, and only use the version if there are multiple
packages with the same priority.
This makes it possible to mark specific versions/variant in Nixpkgs
more or less desirable than others. A typical example would be a
beta version of some package (e.g., `gcc-4.2.0rc1`) which should not
@@ -62,14 +62,14 @@ on Nix. Here is an (incomplete) list:
be modified. There are several attributes that can be usefully
modified, because they affect the behaviour of `nix-env` or the user
environment build script:
- `meta.priority` can be changed to resolve filename clashes (see
above).
- `meta.keep` can be set to `true` to prevent the package from
being upgraded or replaced. Useful if you want to hang on to an
older version of a package.
- `meta.active` can be set to `false` to “disable” the package.
That is, no symlinks will be generated to the files of the
package, but it remains part of the profile (so it wont be
+16 -16
View File
@@ -7,23 +7,23 @@
Nix process crashes; no write access is needed for read-only
operations; no more running out of Berkeley DB locks on certain
operations.
You still need to compile Nix with Berkeley DB support if you want
Nix to automatically convert your old Nix store to the new schema.
If you dont need this, you can build Nix with the `configure`
option `--disable-old-db-compat`.
After the automatic conversion to the new schema, you can delete the
old Berkeley DB files:
$ cd /nix/var/nix/db
$ rm __db* log.* derivers references referrers reserved validpaths DB_CONFIG
The new metadata is stored in the directories `/nix/var/nix/db/info`
and `/nix/var/nix/db/referrer`. Though the metadata is stored in
human-readable plain-text files, they are not intended to be
human-editable, as Nix is rather strict about the format.
The new storage schema may or may not require less disk space than
the Berkeley DB environment, mostly depending on the cluster size of
your file system. With 1 KiB clusters (which seems to be the `ext3`
@@ -50,11 +50,11 @@
last access time. This allows non-recently used stuff to be deleted.
The option `--max-atime time` specifies an upper limit to the last
accessed time of paths that may be deleted. For instance,
```
```
$ nix-store --gc -v --max-atime $(date +%s -d "2 months ago")
```
deletes everything that hasnt been accessed in two months.
- `nix-env` now uses optimistic profile locking when performing an
@@ -73,7 +73,7 @@
now always shown by `nix-env`, `nix-store -r` and `nix-build`. The
total download size of substitutable paths is now also shown. For
instance, a build will show something like
the following derivations will be built:
/nix/store/129sbxnk5n466zg6r1qmq1xjv9zymyy7-activate-configuration.sh.drv
/nix/store/7mzy971rdm8l566ch8hgxaf89x7lr7ik-upstart-jobs.drv
@@ -84,24 +84,24 @@
...
- Language features:
- @-patterns as in Haskell. For instance, in a function definition
f = args @ {x, y, z}: ...;
`args` refers to the argument as a whole, which is further
pattern-matched against the attribute set pattern `{x, y, z}`.
- “`...`” (ellipsis) patterns. An attribute set pattern can now
say `...` at the end of the attribute name list to specify that
the function takes *at least* the listed attributes, while
ignoring additional attributes. For instance,
{stdenv, fetchurl, fuse, ...}: ...
defines a function that accepts any attribute set that includes
at least the three listed attributes.
- New primops: `builtins.parseDrvName` (split a package name
string like `"nix-0.12pre12876"` into its name and version
components, e.g. `"nix"` and `"0.12pre12876"`),
+4 -4
View File
@@ -3,7 +3,7 @@
This is primarily a bug fix release. It has some new features:
- Syntactic sugar for writing nested attribute sets. Instead of
{
foo = {
bar = 123;
@@ -11,15 +11,15 @@ This is primarily a bug fix release. It has some new features:
};
a = { b = { c = "d"; }; };
}
you can write
{
foo.bar = 123;
foo.xyzzy = true;
a.b.c = "d";
}
This is useful, for instance, in NixOS configuration files.
- Support for Nix channels generated by Hydra, the Nix-based
+12 -12
View File
@@ -1,15 +1,15 @@
# Release 0.6 (2004-11-14)
- Rewrite of the normalisation engine.
- Multiple builds can now be performed in parallel (option `-j`).
- Distributed builds. Nix can now call a shell script to forward
builds to Nix installations on remote machines, which may or may
not be of the same platform type.
- Option `--fallback` allows recovery from broken substitutes.
- Option `--keep-going` causes building of other (unaffected)
derivations to continue if one failed.
@@ -28,32 +28,32 @@
- Manual updates.
- `nix-env` changes:
- Derivations for other platforms are filtered out (which can be
overridden using `--system-filter`).
- `--install` by default now uninstall previous derivations with
the same name.
- `--upgrade` allows upgrading to a specific version.
- New operation `--delete-generations` to remove profile
generations (necessary for effective garbage collection).
- Nicer output (sorted, columnised).
- More sensible verbosity levels all around (builder output is now
shown always, unless `-Q` is given).
- Nix expression language changes:
- New language construct: `with
E1;
E2` brings all attributes defined in the attribute set *E1* in
scope in *E2*.
- Added a `map` function.
- Various new operators (e.g., string concatenation).
- Expression evaluation is much faster.
+24 -24
View File
@@ -39,29 +39,29 @@ Nix 0.8 has the following improvements:
notion of “closure store expressions” is gone (and so is the notion
of “successors”); the file system references of a store path are now
just stored in the database.
For instance, given any store path, you can query its closure:
$ nix-store -qR $(which firefox)
... lots of paths ...
Also, Nix now remembers for each store path the derivation that
built it (the “deriver”):
$ nix-store -qR $(which firefox)
/nix/store/4b0jx7vq80l9aqcnkszxhymsf1ffa5jd-firefox-1.0.1.drv
So to see the build-time dependencies, you can do
$ nix-store -qR $(nix-store -qd $(which firefox))
or, in a nicer format:
$ nix-store -q --tree $(nix-store -qd $(which firefox))
File system references are also stored in reverse. For instance, you
can query all paths that directly or indirectly use a certain Glibc:
$ nix-store -q --referrers-closure \
/nix/store/8lz9yc6zgmc0vlqmn2ipcpkjlmbi51vv-glibc-2.3.4
@@ -92,28 +92,28 @@ Nix 0.8 has the following improvements:
- `nix-channel` has new operations `--list` and `--remove`.
- New ways of installing components into user environments:
- Copy from another user environment:
$ nix-env -i --from-profile .../other-profile firefox
- Install a store derivation directly (bypassing the Nix
expression language entirely):
$ nix-env -i /nix/store/z58v41v21xd3...-aterm-2.3.1.drv
(This is used to implement `nix-install-package`, which is
therefore immune to evolution in the Nix expression language.)
- Install an already built store path directly:
$ nix-env -i /nix/store/hsyj5pbn0d9i...-aterm-2.3.1
- Install the result of a Nix expression specified as a
command-line argument:
$ nix-env -f .../i686-linux.nix -i -E 'x: x.firefoxWrapper'
The difference with the normal installation mode is that `-E`
does not use the `name` attributes of derivations. Therefore,
this can be used to disambiguate multiple derivations with the
@@ -127,7 +127,7 @@ Nix 0.8 has the following improvements:
- Implemented a concurrent garbage collector. It is now always safe to
run the garbage collector, even if other Nix operations are
happening simultaneously.
However, there can still be GC races if you use `nix-instantiate`
and `nix-store
--realise` directly to build things. To prevent races, use the
@@ -147,13 +147,13 @@ Nix 0.8 has the following improvements:
- The behaviour of the garbage collector can be changed globally by
setting options in `/nix/etc/nix/nix.conf`.
- `gc-keep-derivations` specifies whether deriver links should be
followed when searching for live paths.
- `gc-keep-outputs` specifies whether outputs of derivations
should be followed when searching for live paths.
- `env-keep-derivations` specifies whether user environments
should store the paths of derivations when they are added (thus
keeping the derivations alive).
+7 -7
View File
@@ -18,36 +18,36 @@ first.
derivations can mutually refer to each other (as long as there are
no data dependencies on the `outPath` and `drvPath` attributes
computed by `derivation`).
For example, the expression `derivation
attrs` now evaluates to (essentially)
attrs // {
type = "derivation";
outPath = derivation! attrs;
drvPath = derivation! attrs;
}
where `derivation!` is a primop that does the actual derivation
instantiation (i.e., it does what `derivation` used to do). The
advantage is that it allows commands such as `nix-env -qa` and
`nix-env -i` to be much faster since they no longer need to
instantiate all derivations, just the `name` attribute.
Also, it allows derivations to cyclically reference each other, for
example,
webServer = derivation {
...
hostName = "svn.cs.uu.nl";
services = [svnService];
};
svnService = derivation {
...
hostName = webServer.hostName;
};
Previously, this would yield a black hole (infinite recursion).
- `nix-build` now defaults to using `./default.nix` if no Nix
+5 -5
View File
@@ -5,26 +5,26 @@ This release has the following improvements:
- On Linux, when doing a chroot build, Nix now uses various namespace
features provided by the Linux kernel to improve build isolation.
Namely:
- The private network namespace ensures that builders cannot talk
to the outside world (or vice versa): each build only sees a
private loopback interface. This also means that two concurrent
builds can listen on the same port (e.g. as part of a test)
without conflicting with each other.
- The PID namespace causes each build to start as PID 1. Processes
outside of the chroot are not visible to those on the inside. On
the other hand, processes inside the chroot *are* visible from
the outside (though with different PIDs).
- The IPC namespace prevents the builder from communicating with
outside processes using SysV IPC mechanisms (shared memory,
message queues, semaphores). It also ensures that all IPC
objects are destroyed when the builder exits.
- The UTS namespace ensures that builders see a hostname of
`localhost` rather than the actual hostname.
- The private mount namespace was already used by Nix to ensure
that the bind-mounts used to set up the chroot are cleaned up
automatically.
+8 -8
View File
@@ -5,23 +5,23 @@ features:
- `nix-prefetch-url` can now download URLs specified in a Nix
expression. For example,
$ nix-prefetch-url -A hello.src
will prefetch the file specified by the `fetchurl` call in the
attribute `hello.src` from the Nix expression in the current
directory, and print the cryptographic hash of the resulting file on
stdout. This differs from `nix-build -A
hello.src` in that it doesn't verify the hash, and is thus useful
when youre updating a Nix expression.
You can also prefetch the result of functions that unpack a tarball,
such as `fetchFromGitHub`. For example:
$ nix-prefetch-url --unpack https://github.com/NixOS/patchelf/archive/0.8.tar.gz
or from a Nix expression:
$ nix-prefetch-url -A nix-repl.src
- The builtin function `<nix/fetchurl.nix>` now supports downloading
@@ -49,12 +49,12 @@ features:
same derivation and `-K` is given, will make the output of the other
run available under `store-path-check`. This makes it easier to
investigate the non-determinism using tools like `diffoscope`, e.g.,
$ nix-build pkgs/stdenv/linux -A stage1.pkgs.zlib --check -K
error: derivation /nix/store/l54i8wlw2265…-zlib-1.2.8.drv may not
be deterministic: output /nix/store/11a27shh6n2i…-zlib-1.2.8
differs from /nix/store/11a27shh6n2i…-zlib-1.2.8-check
$ diffoscope /nix/store/11a27shh6n2i…-zlib-1.2.8 /nix/store/11a27shh6n2i…-zlib-1.2.8-check
├── lib/libz.a
+10 -10
View File
@@ -8,9 +8,9 @@ This release has the following improvements and changes:
binary caches to see if any of them has a pre-built binary of that
path. The configuration setting `binary-caches` contains a list of
URLs of binary caches. For instance, doing
$ nix-env -i thunderbird --option binary-caches http://cache.nixos.org
will install Thunderbird and its dependencies, using the available
pre-built binaries in <http://cache.nixos.org>. The main advantage
over the old “manifest”-based method of getting pre-built binaries
@@ -19,12 +19,12 @@ This release has the following improvements and changes:
need to run `nix-pull` to update your manifest. Its also more
scalable because you dont need to redownload a giant manifest file
every time.
A Nix channel can provide a binary cache URL that will be used
automatically if you subscribe to that channel. If you use the
Nixpkgs or NixOS channels (<http://nixos.org/channels>) you
automatically get the cache <http://cache.nixos.org>.
Binary caches are created using `nix-push`. For details on the
operation and format of binary caches, see the `nix-push` manpage.
More details are provided in [this nix-dev
@@ -33,16 +33,16 @@ This release has the following improvements and changes:
- Multiple output support should now be usable. A derivation can
declare that it wants to produce multiple store paths by saying
something like
outputs = [ "lib" "headers" "doc" ];
This will cause Nix to pass the intended store path of each output
to the builder through the environment variables `lib`, `headers`
and `doc`. Other packages can refer to a specific output by
referring to `pkg.output`, e.g.
buildInputs = [ pkg.lib pkg.headers ];
If you install a package with multiple outputs using `nix-env`, each
output path will be symlinked into the user environment.
@@ -60,10 +60,10 @@ This release has the following improvements and changes:
Instead, the recommended way to guard the Nix store against
accidental modification on Linux is to make it a read-only bind
mount, like this:
$ mount --bind /nix/store /nix/store
$ mount -o remount,ro,bind /nix/store
Nix will automatically make `/nix/store` writable as needed (using a
private mount namespace) to allow modifications.
+6 -6
View File
@@ -25,28 +25,28 @@ features:
declarative package management similar to NixOSs
`environment.systemPackages`. For instance, if you have a
specification `my-packages.nix` like this:
with import <nixpkgs> {};
[ thunderbird
geeqie
...
]
then after any change to this file, you can run:
$ nix-env -f my-packages.nix -ir
to update your profile to match the specification.
- The `with` language construct is now more lazy. It only evaluates
its argument if a variable might actually refer to an attribute in
the argument. For instance, this now works:
let
pkgs = with pkgs; { foo = "old"; bar = foo; } // overrides;
overrides = { foo = "new"; };
in pkgs.bar
This evaluates to `"new"`, while previously it gave an “infinite
recursion” error.
+12 -12
View File
@@ -25,7 +25,7 @@ features:
builds a previously built derivation again, and prints an error
message if the output is not exactly the same. This helps to verify
whether a derivation is truly deterministic. For example:
$ nix-build '<nixpkgs>' -A patchelf
$ nix-build '<nixpkgs>' -A patchelf --check
@@ -43,31 +43,31 @@ features:
'1 + 2'` will print `3`.
- `nix-shell` improvements:
- It has a new flag, `--packages` (or `-p`), that sets up a build
environment containing the specified packages from Nixpkgs. For
example, the command
$ nix-shell -p sqlite xorg.libX11 hello
will start a shell in which the given packages are present.
- It now uses `shell.nix` as the default expression, falling back
to `default.nix` if the former doesnt exist. This makes it
convenient to have a `shell.nix` in your project to set up a
nice development environment.
- It evaluates the derivation attribute `shellHook`, if set. Since
`stdenv` does not normally execute this hook, it allows you to
do `nix-shell`-specific setup.
- It preserves the users timezone setting.
- In chroots, Nix now sets up a `/dev` containing only a minimal set
of devices (such as `/dev/null`). Note that it only does this if you
*dont* have `/dev` listed in your `build-chroot-dirs` setting;
otherwise, it will bind-mount the `/dev` from outside the chroot.
Similarly, if you dont have `/dev/pts` listed in
`build-chroot-dirs`, Nix will mount a private `devpts` filesystem on
the chroots `/dev/pts`.
@@ -82,7 +82,7 @@ features:
such as attribute sets.
- The `-A` flag now allows attribute names with dots in them, e.g.
$ nix-instantiate --eval '<nixos>' -A 'config.systemd.units."nscd.service".text'
- The `--max-freed` option to `nix-store --gc` now accepts a unit
@@ -106,7 +106,7 @@ features:
`unsafeGetAttrPos`, that returns the position of an attribute. This
is used by Nixpkgs to provide location information in error
messages, e.g.
$ nix-build '<nixpkgs>' -A libreoffice --argstr system x86_64-darwin
error: the package libreoffice-4.0.5.2 in .../applications/office/libreoffice/default.nix:263
is not supported on x86_64-darwin
@@ -116,12 +116,12 @@ features:
- The binary tarball installer has been improved. You can now install
Nix by running:
$ bash <(curl -L https://nixos.org/nix/install)
- More evaluation errors include position information. For instance,
selecting a missing attribute will print something like
error: attribute `nixUnstabl' missing, at /etc/nixos/configurations/misc/eelco/mandark.nix:216:15
- The command `nix-setuid-helper` is gone.
+6 -6
View File
@@ -10,24 +10,24 @@
- `nix-env` selectors are now regular expressions. For instance, you
can do
$ nix-env -qa '.*zip.*'
to query all packages with a name containing `zip`.
- `nix-store --read-log` can now fetch remote build logs. If a build
log is not available locally, then nix-store -l will now try to
download it from the servers listed in the log-servers option in
nix.conf. For instance, if you have the configuration option
log-servers = http://hydra.nixos.org/log
then it will try to get logs from `http://hydra.nixos.org/log/base
name of the
store path`. This allows you to do things like:
$ nix-store -l $(which xterm)
and get a log even if `xterm` wasn't built locally.
- New builtin functions: `attrValues`, `deepSeq`, `fromJSON`,
+27 -27
View File
@@ -5,17 +5,17 @@ features:
- Signed binary cache support. You can enable signature checking by
adding the following to `nix.conf`:
signed-binary-caches = *
binary-cache-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
This will prevent Nix from downloading any binary from the cache
that is not signed by one of the keys listed in
`binary-cache-public-keys`.
Signature checking is only supported if you built Nix with the
`libsodium` package.
Note that while Nix has had experimental support for signed binary
caches since version 1.7, this release changes the signature format
in a backwards-incompatible way.
@@ -24,79 +24,79 @@ features:
you can now specify the URL of a tarball containing Nix expressions
(such as Nixpkgs), which will be downloaded and unpacked
automatically. For example:
- In `nix-env`:
$ nix-env -f https://github.com/NixOS/nixpkgs-channels/archive/nixos-14.12.tar.gz -iA firefox
This installs Firefox from the latest tested and built revision
of the NixOS 14.12 channel.
- In `nix-build` and `nix-shell`:
$ nix-build https://github.com/NixOS/nixpkgs/archive/master.tar.gz -A hello
This builds GNU Hello from the latest revision of the Nixpkgs
master branch.
- In the Nix search path (as specified via `NIX_PATH` or `-I`).
For example, to start a shell containing the Pan package from a
specific version of Nixpkgs:
$ nix-shell -p pan -I nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/8a3eea054838b55aca962c3fbde9c83c102b8bf2.tar.gz
- In `nixos-rebuild` (on NixOS):
$ nixos-rebuild test -I nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/nixos-unstable.tar.gz
- In Nix expressions, via the new builtin function `fetchTarball`:
with import (fetchTarball https://github.com/NixOS/nixpkgs-channels/archive/nixos-14.12.tar.gz) {}; …
(This is not allowed in restricted mode.)
- `nix-shell` improvements:
- `nix-shell` now has a flag `--run` to execute a command in the
`nix-shell` environment, e.g. `nix-shell --run make`. This is
like the existing `--command` flag, except that it uses a
non-interactive shell (ensuring that hitting Ctrl-C wont drop
you into the child shell).
- `nix-shell` can now be used as a `#!`-interpreter. This allows
you to write scripts that dynamically fetch their own
dependencies. For example, here is a Haskell script that, when
invoked, first downloads GHC and the Haskell packages on which
it depends:
#! /usr/bin/env nix-shell
#! nix-shell -i runghc -p haskellPackages.ghc haskellPackages.HTTP
import Network.HTTP
main = do
resp <- Network.HTTP.simpleHTTP (getRequest "http://nixos.org/")
body <- getResponseBody resp
print (take 100 body)
Of course, the dependencies are cached in the Nix store, so the
second invocation of this script will be much faster.
- Chroot improvements:
- Chroot builds are now supported on Mac OS X (using its sandbox
mechanism).
- If chroots are enabled, they are now used for all derivations,
including fixed-output derivations (such as `fetchurl`). The
latter do have network access, but can no longer access the host
filesystem. If you need the old behaviour, you can set the
option `build-use-chroot` to `relaxed`.
- On Linux, if chroots are enabled, builds are performed in a
private PID namespace once again. (This functionality was lost
in Nix 1.8.)
- Store paths listed in `build-chroot-dirs` are now automatically
expanded to their closure. For instance, if you want
`/nix/store/…-bash/bin/sh` mounted in your chroot as `/bin/sh`,
+72 -72
View File
@@ -8,13 +8,13 @@ The following incompatible changes have been made:
It has been superseded by the binary cache substituter mechanism
since several years. As a result, the following programs have been
removed:
- `nix-pull`
- `nix-generate-patches`
- `bsdiff`
- `bspatch`
- The “copy from other stores” substituter mechanism
@@ -58,26 +58,26 @@ This release has the following new features:
`nix-build`, `nix-shell -p`, `nix-env -qa`, `nix-instantiate
--eval`, `nix-push` and `nix-copy-closure`. It has the following
major features:
- Unlike the legacy commands, it has a consistent way to refer to
packages and package-like arguments (like store paths). For
example, the following commands all copy the GNU Hello package
to a remote machine:
nix copy --to ssh://machine nixpkgs.hello
nix copy --to ssh://machine /nix/store/0i2jd68mp5g6h2sa5k9c85rb80sn8hi9-hello-2.10
nix copy --to ssh://machine '(with import <nixpkgs> {}; hello)'
By contrast, `nix-copy-closure` only accepted store paths as
arguments.
- It is self-documenting: `--help` shows all available
command-line arguments. If `--help` is given after a subcommand,
it shows examples for that subcommand. `nix
--help-config` shows all configuration options.
- It is much less verbose. By default, it displays a single-line
progress indicator that shows how many packages are left to be
built or downloaded, and (if there are running builds) the most
@@ -85,7 +85,7 @@ This release has the following new features:
last few lines of builder output. The full build log can be
retrieved using `nix
log`.
- It
[provides](https://github.com/NixOS/nix/commit/b8283773bd64d7da6859ed520ee19867742a03ba)
all `nix.conf` configuration options as command line flags. For
@@ -93,122 +93,122 @@ This release has the following new features:
http-connections 100` you can write `--http-connections 100`.
Boolean options can be written as `--foo` or `--no-foo` (e.g.
`--no-auto-optimise-store`).
- Many subcommands have a `--json` flag to write results to stdout
in JSON format.
> **Warning**
>
>
> Please note that the `nix` command is a work in progress and the
> interface is subject to change.
It provides the following high-level (“porcelain”) subcommands:
- `nix build` is a replacement for `nix-build`.
- `nix run` executes a command in an environment in which the
specified packages are available. It is (roughly) a replacement
for `nix-shell
-p`. Unlike that command, it does not execute the command in a
shell, and has a flag (`-c`) that specifies the unquoted command
line to be executed.
It is particularly useful in conjunction with chroot stores,
allowing Linux users who do not have permission to install Nix
in `/nix/store` to still use binary substitutes that assume
`/nix/store`. For example,
nix run --store ~/my-nix nixpkgs.hello -c hello --greeting 'Hi everybody!'
downloads (or if not substitutes are available, builds) the GNU
Hello package into `~/my-nix/nix/store`, then runs `hello` in a
mount namespace where `~/my-nix/nix/store` is mounted onto
`/nix/store`.
- `nix search` replaces `nix-env
-qa`. It searches the available packages for occurrences of a
search string in the attribute name, package name or
description. Unlike `nix-env -qa`, it has a cache to speed up
subsequent searches.
- `nix copy` copies paths between arbitrary Nix stores,
generalising `nix-copy-closure` and `nix-push`.
- `nix repl` replaces the external program `nix-repl`. It provides
an interactive environment for evaluating and building Nix
expressions. Note that it uses `linenoise-ng` instead of GNU
Readline.
- `nix upgrade-nix` upgrades Nix to the latest stable version.
This requires that Nix is installed in a profile. (Thus it wont
work on NixOS, or if its installed outside of the Nix store.)
- `nix verify` checks whether store paths are unmodified and/or
“trusted” (see below). It replaces `nix-store --verify` and
`nix-store
--verify-path`.
- `nix log` shows the build log of a package or path. If the
build log is not available locally, it will try to obtain it
from the configured substituters (such as
[cache.nixos.org](https://cache.nixos.org/), which now
provides build logs).
- `nix edit` opens the source code of a package in your editor.
- `nix eval` replaces `nix-instantiate --eval`.
- `nix
why-depends` shows why one store path has another in its
closure. This is primarily useful to finding the causes of
closure bloat. For example,
nix why-depends nixpkgs.vlc nixpkgs.libdrm.dev
shows a chain of files and fragments of file contents that cause
the VLC package to have the “dev” output of `libdrm` in its
closure — an undesirable situation.
- `nix path-info` shows information about store paths, replacing
`nix-store -q`. A useful feature is the option `--closure-size`
(`-S`). For example, the following command show the closure
sizes of every path in the current NixOS system closure, sorted
by size:
nix path-info -rS /run/current-system | sort -nk2
- `nix optimise-store` replaces `nix-store --optimise`. The main
difference is that it has a progress indicator.
A number of low-level (“plumbing”) commands are also available:
- `nix ls-store` and `nix
ls-nar` list the contents of a store path or NAR file. The
former is primarily useful in conjunction with remote stores,
e.g.
nix ls-store --store https://cache.nixos.org/ -lR /nix/store/0i2jd68mp5g6h2sa5k9c85rb80sn8hi9-hello-2.10
lists the contents of path in a binary cache.
- `nix cat-store` and `nix
cat-nar` allow extracting a file from a store path or NAR file.
- `nix dump-path` writes the contents of a store path to stdout in
NAR format. This replaces `nix-store --dump`.
- `nix
show-derivation` displays a store derivation in JSON format.
This is an alternative to `pp-aterm`.
- `nix
add-to-store` replaces `nix-store
--add`.
- `nix sign-paths` signs store paths.
- `nix copy-sigs` copies signatures from one store to another.
- `nix show-config` shows all configuration options and their
current values.
@@ -224,11 +224,11 @@ This release has the following new features:
`nix-copy-closure`, `nix-push` and substitution are all instances
of the general notion of copying paths between different kinds of
Nix stores.
Stores are specified using an URI-like syntax, e.g.
<https://cache.nixos.org/> or <ssh://machine>. The following store
types are supported:
- `LocalStore` (stori URI `local` or an absolute path) and the
misnamed `RemoteStore` (`daemon`) provide access to a local Nix
store, the latter via the Nix daemon. You can use `auto` or the
@@ -236,63 +236,63 @@ This release has the following new features:
whether you have write permission to the Nix store. It is no
longer necessary to set the `NIX_REMOTE` environment variable to
use the Nix daemon.
As noted above, `LocalStore` now supports chroot builds,
allowing the “physical” location of the Nix store (e.g.
`/home/alice/nix/store`) to differ from its “logical” location
(typically `/nix/store`). This allows non-root users to use Nix
while still getting the benefits from prebuilt binaries from
[cache.nixos.org](https://cache.nixos.org/).
- `BinaryCacheStore` is the abstract superclass of all binary
cache stores. It supports writing build logs and NAR content
listings in JSON format.
- `HttpBinaryCacheStore` (`http://`, `https://`) supports binary
caches via HTTP or HTTPS. If the server supports `PUT` requests,
it supports uploading store paths via commands such as `nix
copy`.
- `LocalBinaryCacheStore` (`file://`) supports binary caches in
the local filesystem.
- `S3BinaryCacheStore` (`s3://`) supports binary caches stored in
Amazon S3, if enabled at compile time.
- `LegacySSHStore` (`ssh://`) is used to implement remote builds
and `nix-copy-closure`.
- `SSHStore` (`ssh-ng://`) supports arbitrary Nix operations on a
remote machine via the same protocol used by `nix-daemon`.
- Security has been improved in various ways:
- Nix now stores signatures for local store paths. When paths are
copied between stores (e.g., copied from a binary cache to a
local store), signatures are propagated.
Locally-built paths are signed automatically using the secret
keys specified by the `secret-key-files` store option.
Secret/public key pairs can be generated using `nix-store
--generate-binary-cache-key`.
In addition, locally-built store paths are marked as “ultimately
trusted”, but this bit is not propagated when paths are copied
between stores.
- Content-addressable store paths no longer require signatures —
they can be imported into a store by unprivileged users even if
they lack signatures.
- The command `nix verify` checks whether the specified paths are
trusted, i.e., have a certain number of trusted signatures, are
ultimately trusted, or are content-addressed.
- Substitutions from binary caches
[now](https://github.com/NixOS/nix/commit/ecbc3fedd3d5bdc5a0e1a0a51b29062f2874ac8b)
require signatures by default. This was already the case on
NixOS.
- In Linux sandbox builds, we
[now](https://github.com/NixOS/nix/commit/eba840c8a13b465ace90172ff76a0db2899ab11b)
use `/build` instead of `/tmp` as the temporary build directory.
@@ -309,7 +309,7 @@ This release has the following new features:
hash or commit hash is specified. For example, calls to
`builtins.fetchGit` are only allowed if a `rev` attribute is
specified.
The goal of this feature is to enable true reproducibility and
traceability of builds (including NixOS system configurations) at
the evaluation level. For example, in the future, `nixos-rebuild`
@@ -367,21 +367,21 @@ This release has the following new features:
log will be shown if a build fails.
- Networking has been improved:
- HTTP/2 is now supported. This makes binary cache lookups [much
more
efficient](https://github.com/NixOS/nix/commit/90ad02bf626b885a5dd8967894e2eafc953bdf92).
- We now retry downloads on many HTTP errors, making binary caches
substituters more resilient to temporary failures.
- HTTP credentials can now be configured via the standard `netrc`
mechanism.
- If S3 support is enabled at compile time, <s3://> URIs are
[supported](https://github.com/NixOS/nix/commit/9ff9c3f2f80ba4108e9c945bbfda2c64735f987b)
in all places where Nix allows URIs.
- Brotli compression is now supported. In particular,
[cache.nixos.org](https://cache.nixos.org/) build logs are now compressed
using Brotli.
@@ -431,9 +431,9 @@ The Nix language has the following new features:
- Derivation attributes can now reference the outputs of the
derivation using the `placeholder` builtin function. For example,
the attribute
configureFlags = "--prefix=${placeholder "out"} --includedir=${placeholder "dev"}";
will cause the `configureFlags` environment variable to contain the
actual store paths corresponding to the `out` and `dev` outputs.
@@ -444,7 +444,7 @@ The following builtin functions are new or extended:
Nixpkgs, which fetches at build time and cannot be used to fetch Nix
expressions during evaluation. A typical use case is to import
external NixOS modules from your configuration, e.g.
imports = [ (builtins.fetchGit https://github.com/edolstra/dwarffs + "/module.nix") ];
- Similarly, `builtins.fetchMercurial` allows you to fetch Mercurial
@@ -485,7 +485,7 @@ The Nix build environment has the following changes:
builder via the file `.attrs.json` in the builders temporary
directory. This obviates the need for `passAsFile` since JSON files
have no size restrictions, unlike process environments.
[As a convenience to Bash
builders](https://github.com/NixOS/nix/commit/2d5b1b24bf70a498e4c0b378704cfdb6471cc699),
Nix writes a script named `.attrs.sh` to the builders directory
+9 -9
View File
@@ -6,15 +6,15 @@ This is primarily a bug fix release. It also has the following changes:
the `__structuredAttrs` attribute to `true` to cause all attributes
to be passed to the builder in JSON format), you can now specify
closure checks per output, e.g.:
outputChecks."out" = {
# The closure of 'out' must not be larger than 256 MiB.
maxClosureSize = 256 * 1024 * 1024;
# It must not refer to C compiler or to the 'dev' output.
disallowedRequisites = [ stdenv.cc "dev" ];
};
outputChecks."dev" = {
# The 'dev' output must not be larger than 128 KiB.
maxSize = 128 * 1024;
@@ -24,7 +24,7 @@ This is primarily a bug fix release. It also has the following changes:
for local builds, and not just to route builds to remote builders.
The supported features of a machine can be specified through the
configuration setting `system-features`.
By default, `system-features` includes `kvm` if `/dev/kvm` exists.
For compatibility, it also includes the pseudo-features
`nixos-test`, `benchmark` and `big-parallel` which are used by
@@ -45,22 +45,22 @@ This is primarily a bug fix release. It also has the following changes:
- Nix now supports [SRI hashes](https://www.w3.org/TR/SRI/), allowing
the hash algorithm and hash to be specified in a single string. For
example, you can write:
import <nix/fetchurl.nix> {
url = https://nixos.org/releases/nix/nix-2.1.3/nix-2.1.3.tar.xz;
hash = "sha256-XSLa0FjVyADWWhFfkZ2iKTjFDda6mMXjoYMXLRSYQKQ=";
};
instead of
import <nix/fetchurl.nix> {
url = https://nixos.org/releases/nix/nix-2.1.3/nix-2.1.3.tar.xz;
sha256 = "5d22dad058d5c800d65a115f919da22938c50dd6ba98c5e3a183172d149840a4";
};
In fixed-output derivations, the `outputHashAlgo` attribute is no
longer mandatory if `outputHash` specifies the hash.
`nix hash-file` and `nix
hash-path` now print hashes in SRI format by default. They also use
SHA-256 by default instead of SHA-512 because that's what we use
File diff suppressed because one or more lines are too long
Generated
+91 -7
View File
@@ -3,11 +3,11 @@
"flake-compat": {
"flake": false,
"locked": {
"lastModified": 1673956053,
"narHash": "sha256-4gtG9iQuiKITOjNQQeQIpoIB6b16fm+504Ch3sNKLd8=",
"lastModified": 1696426674,
"narHash": "sha256-kvjfFW7WAETZlt09AgDn1MrtKzP7t90Vf7vypd3OL1U=",
"owner": "edolstra",
"repo": "flake-compat",
"rev": "35bb57c0c8d8b62bbfd284272c928ceb64ddbde9",
"rev": "0f9255e01c2351cc7d116c072cb317785dd33b33",
"type": "github"
},
"original": {
@@ -16,13 +16,52 @@
"type": "github"
}
},
"flake-utils": {
"inputs": {
"systems": "systems"
},
"locked": {
"lastModified": 1710146030,
"narHash": "sha256-SZ5L6eA7HJ/nmkzGG7/ISclqe6oZdOZTNoesiInkXPQ=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "b1d9ab70662946ef0850d488da1c9019f3a9752a",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "flake-utils",
"type": "github"
}
},
"gitignore": {
"inputs": {
"nixpkgs": [
"pre-commit-hooks",
"nixpkgs"
]
},
"locked": {
"lastModified": 1709087332,
"narHash": "sha256-HG2cCnktfHsKV0s4XW83gU3F57gaTljL9KNSuG6bnQs=",
"owner": "hercules-ci",
"repo": "gitignore.nix",
"rev": "637db329424fd7e46cf4185293b9cc8c88c95394",
"type": "github"
},
"original": {
"owner": "hercules-ci",
"repo": "gitignore.nix",
"type": "github"
}
},
"nixpkgs": {
"locked": {
"lastModified": 1709884566,
"narHash": "sha256-NSYJg2sfdO/XS3L8XN/59Zhzn0dqWm7XtVnKI2mHq3w=",
"lastModified": 1711481231,
"narHash": "sha256-J/fW3Xhm3WsJPNd8ksZmfMnol5aOG2qEMDPbOnNNdTQ=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "2be119add7b37dc535da2dd4cba68e2cf8d1517e",
"rev": "9d6ddb13cee3cc1192e4430277708c732685f38a",
"type": "github"
},
"original": {
@@ -48,11 +87,56 @@
"type": "github"
}
},
"pre-commit-hooks": {
"inputs": {
"flake-compat": [
"flake-compat"
],
"flake-utils": "flake-utils",
"gitignore": "gitignore",
"nixpkgs": [
"nixpkgs"
],
"nixpkgs-stable": [
"nixpkgs"
]
},
"locked": {
"lastModified": 1711562502,
"narHash": "sha256-SLo1yf/lfozph818VME9Y56uhaZWY0juZcj/qJZPPrs=",
"owner": "9999years",
"repo": "git-hooks.nix",
"rev": "d2f6f9376c38fb0c48526720793a17f10cd3ac10",
"type": "github"
},
"original": {
"owner": "9999years",
"ref": "add-default-pre-commit-hooks",
"repo": "git-hooks.nix",
"type": "github"
}
},
"root": {
"inputs": {
"flake-compat": "flake-compat",
"nixpkgs": "nixpkgs",
"nixpkgs-regression": "nixpkgs-regression"
"nixpkgs-regression": "nixpkgs-regression",
"pre-commit-hooks": "pre-commit-hooks"
}
},
"systems": {
"locked": {
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "default",
"type": "github"
}
}
},
+218 -441
View File
@@ -1,11 +1,23 @@
{
description = "The purely functional package manager";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.11-small";
inputs.nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
inputs.flake-compat = { url = "github:edolstra/flake-compat"; flake = false; };
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.11-small";
nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
pre-commit-hooks = {
# Can go back to `cachix/git-hooks.nix` when this is merged:
# https://github.com/cachix/git-hooks.nix/pull/401
url = "github:9999years/git-hooks.nix/add-default-pre-commit-hooks";
inputs = {
flake-compat.follows = "flake-compat";
nixpkgs.follows = "nixpkgs";
nixpkgs-stable.follows = "nixpkgs";
};
};
flake-compat = { url = "github:edolstra/flake-compat"; flake = false; };
};
outputs = { self, nixpkgs, nixpkgs-regression, flake-compat }:
outputs = { self, nixpkgs, nixpkgs-regression, pre-commit-hooks, flake-compat }:
let
inherit (nixpkgs) lib;
@@ -48,50 +60,6 @@
})
stdenvs);
baseFiles =
# .gitignore has already been processed, so any changes in it are irrelevant
# at this point. It is not represented verbatim for test purposes because
# that would interfere with repo semantics.
fileset.fileFilter (f: f.name != ".gitignore") ./.;
configureFiles = fileset.unions [
./.version
./configure.ac
./m4
# TODO: do we really need README.md? It doesn't seem used in the build.
./README.md
];
topLevelBuildFiles = fileset.unions [
./local.mk
./Makefile
./Makefile.config.in
./mk
];
functionalTestFiles = fileset.unions [
./tests/functional
./tests/unit
(fileset.fileFilter (f: lib.strings.hasPrefix "nix-profile" f.name) ./scripts)
];
nixSrc = fileset.toSource {
root = ./.;
fileset = fileset.intersection baseFiles (fileset.unions [
configureFiles
topLevelBuildFiles
./boehmgc-coroutine-sp-fallback.diff
./doc
./misc
./precompiled-headers.h
./src
./unit-test-data
./COPYING
./scripts/local.mk
functionalTestFiles
]);
};
# Memoize nixpkgs for different platforms for efficiency.
nixpkgsFor = forAllSystems
(system: let
@@ -118,205 +86,45 @@
cross = forAllCrossSystems (crossSystem: make-pkgs crossSystem "stdenv");
});
commonDeps =
{ pkgs
, isStatic ? pkgs.stdenv.hostPlatform.isStatic
}:
with pkgs; rec {
# Use "busybox-sandbox-shell" if present,
# if not (legacy) fallback and hope it's sufficient.
sh = pkgs.busybox-sandbox-shell or (busybox.override {
useMusl = true;
enableStatic = true;
enableMinimal = true;
extraConfig = ''
CONFIG_FEATURE_FANCY_ECHO y
CONFIG_FEATURE_SH_MATH y
CONFIG_FEATURE_SH_MATH_64 y
testNixVersions = pkgs: client: daemon: let
nix = pkgs.callPackage ./package.nix {
pname =
"nix-tests"
+ lib.optionalString
(lib.versionAtLeast daemon.version "2.4pre20211005" &&
lib.versionAtLeast client.version "2.4pre20211005")
"-${client.version}-against-${daemon.version}";
CONFIG_ASH y
CONFIG_ASH_OPTIMIZE_FOR_SIZE y
CONFIG_ASH_ALIAS y
CONFIG_ASH_BASH_COMPAT y
CONFIG_ASH_CMDCMD y
CONFIG_ASH_ECHO y
CONFIG_ASH_GETOPTS y
CONFIG_ASH_INTERNAL_GLOB y
CONFIG_ASH_JOB_CONTROL y
CONFIG_ASH_PRINTF y
CONFIG_ASH_TEST y
'';
});
configureFlags =
lib.optionals stdenv.isLinux [
"--with-boost=${boost}/lib"
"--with-sandbox-shell=${sh}/bin/busybox"
]
++ lib.optionals (stdenv.isLinux && !(isStatic && stdenv.system == "aarch64-linux")) [
"LDFLAGS=-fuse-ld=gold"
];
testConfigureFlags = [
"RAPIDCHECK_HEADERS=${lib.getDev rapidcheck}/extras/gtest/include"
];
internalApiDocsConfigureFlags = [
"--enable-internal-api-docs"
];
changelog-d = pkgs.buildPackages.callPackage ./misc/changelog-d.nix { };
nativeBuildDeps =
[
buildPackages.bison
buildPackages.flex
(lib.getBin buildPackages.lowdown)
buildPackages.mdbook
buildPackages.mdbook-linkcheck
buildPackages.autoconf-archive
buildPackages.autoreconfHook
buildPackages.pkg-config
# Tests
buildPackages.git
buildPackages.mercurial # FIXME: remove? only needed for tests
buildPackages.jq # Also for custom mdBook preprocessor.
]
++ lib.optionals stdenv.hostPlatform.isLinux [(buildPackages.util-linuxMinimal or buildPackages.utillinuxMinimal)]
# Official releases don't have rl-next, so we don't need to compile a changelog
++ lib.optional (!officialRelease && buildUnreleasedNotes) changelog-d
;
buildDeps =
[ curl
bzip2 xz brotli editline
openssl sqlite
libarchive
boost
lowdown
libsodium
]
++ lib.optionals stdenv.isLinux [libseccomp]
++ lib.optional stdenv.hostPlatform.isx86_64 libcpuid;
checkDeps = [
gtest
rapidcheck
];
internalApiDocsDeps = [
buildPackages.doxygen
];
awsDeps = lib.optional (stdenv.isLinux || stdenv.isDarwin)
(aws-sdk-cpp.override {
apis = ["s3" "transfer"];
customMemoryManagement = false;
});
propagatedDeps =
[ ((boehmgc.override {
enableLargeConfig = true;
}).overrideAttrs(o: {
patches = (o.patches or []) ++ [
./boehmgc-coroutine-sp-fallback.diff
# https://github.com/ivmai/bdwgc/pull/586
./boehmgc-traceable_allocator-public.diff
];
})
)
nlohmann_json
];
};
installScriptFor = systems:
with nixpkgsFor.x86_64-linux.native;
runCommand "installer-script"
{ buildInputs = [ nix ];
}
''
mkdir -p $out/nix-support
# Converts /nix/store/50p3qk8k...-nix-2.4pre20201102_550e11f/bin/nix to 50p3qk8k.../bin/nix.
tarballPath() {
# Remove the store prefix
local path=''${1#${builtins.storeDir}/}
# Get the path relative to the derivation root
local rest=''${path#*/}
# Get the derivation hash
local drvHash=''${path%%-*}
echo "$drvHash/$rest"
}
substitute ${./scripts/install.in} $out/install \
${pkgs.lib.concatMapStrings
(system: let
tarball = if builtins.elem system crossSystems then self.hydraJobs.binaryTarballCross.x86_64-linux.${system} else self.hydraJobs.binaryTarball.${system};
in '' \
--replace '@tarballHash_${system}@' $(nix --experimental-features nix-command hash-file --base16 --type sha256 ${tarball}/*.tar.xz) \
--replace '@tarballPath_${system}@' $(tarballPath ${tarball}/*.tar.xz) \
''
)
systems
} --replace '@nixVersion@' ${version}
echo "file installer $out/install" >> $out/nix-support/hydra-build-products
'';
testNixVersions = pkgs: client: daemon: with commonDeps { inherit pkgs; }; with pkgs.lib; pkgs.stdenv.mkDerivation {
inherit fileset;
};
in nix.overrideAttrs (prevAttrs: {
NIX_DAEMON_PACKAGE = daemon;
NIX_CLIENT_PACKAGE = client;
name =
"nix-tests"
+ optionalString
(versionAtLeast daemon.version "2.4pre20211005" &&
versionAtLeast client.version "2.4pre20211005")
"-${client.version}-against-${daemon.version}";
inherit version;
src = fileset.toSource {
root = ./.;
fileset = fileset.intersection baseFiles (fileset.unions [
configureFiles
topLevelBuildFiles
functionalTestFiles
]);
};
VERSION_SUFFIX = versionSuffix;
nativeBuildInputs = nativeBuildDeps;
buildInputs = buildDeps ++ awsDeps ++ checkDeps;
propagatedBuildInputs = propagatedDeps;
enableParallelBuilding = true;
configureFlags =
testConfigureFlags # otherwise configure fails
++ [ "--disable-build" ];
dontBuild = true;
doInstallCheck = true;
configureFlags = prevAttrs.configureFlags ++ [
# We don't need the actual build here.
"--disable-build"
];
installPhase = ''
mkdir -p $out
'';
installCheckPhase = (optionalString pkgs.stdenv.hostPlatform.isDarwin ''
installCheckPhase = lib.optionalString pkgs.stdenv.hostPlatform.isDarwin ''
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
'') + ''
'' + ''
mkdir -p src/nix-channel
make installcheck -j$NIX_BUILD_CORES -l$NIX_BUILD_CORES
'';
};
});
binaryTarball = nix: pkgs:
let
inherit (pkgs) buildPackages;
inherit (pkgs) cacert;
installerClosureInfo = buildPackages.closureInfo { rootPaths = [ nix cacert ]; };
installerClosureInfo = buildPackages.closureInfo { rootPaths = [ nix ]; };
in
buildPackages.runCommand "nix-binary-tarball-${version}"
@@ -325,45 +133,7 @@
}
''
cp ${installerClosureInfo}/registration $TMPDIR/reginfo
cp ${./scripts/create-darwin-volume.sh} $TMPDIR/create-darwin-volume.sh
substitute ${./scripts/install-nix-from-closure.sh} $TMPDIR/install \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
substitute ${./scripts/install-darwin-multi-user.sh} $TMPDIR/install-darwin-multi-user.sh \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
substitute ${./scripts/install-systemd-multi-user.sh} $TMPDIR/install-systemd-multi-user.sh \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
substitute ${./scripts/install-multi-user.sh} $TMPDIR/install-multi-user \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
if type -p shellcheck; then
# SC1090: Don't worry about not being able to find
# $nix/etc/profile.d/nix.sh
shellcheck --exclude SC1090 $TMPDIR/install
shellcheck $TMPDIR/create-darwin-volume.sh
shellcheck $TMPDIR/install-darwin-multi-user.sh
shellcheck $TMPDIR/install-systemd-multi-user.sh
# SC1091: Don't panic about not being able to source
# /etc/profile
# SC2002: Ignore "useless cat" "error", when loading
# .reginfo, as the cat is a much cleaner
# implementation, even though it is "useless"
# SC2116: Allow ROOT_HOME=$(echo ~root) for resolving
# root's home directory
shellcheck --external-sources \
--exclude SC1091,SC2002,SC2116 $TMPDIR/install-multi-user
fi
chmod +x $TMPDIR/install
chmod +x $TMPDIR/create-darwin-volume.sh
chmod +x $TMPDIR/install-darwin-multi-user.sh
chmod +x $TMPDIR/install-systemd-multi-user.sh
chmod +x $TMPDIR/install-multi-user
dir=nix-${version}-${pkgs.system}
fn=$out/$dir.tar.xz
mkdir -p $out/nix-support
@@ -373,123 +143,69 @@
--mtime='1970-01-01' \
--absolute-names \
--hard-dereference \
--transform "s,$TMPDIR/install,$dir/install," \
--transform "s,$TMPDIR/create-darwin-volume.sh,$dir/create-darwin-volume.sh," \
--transform "s,$TMPDIR/reginfo,$dir/.reginfo," \
--transform "s,$NIX_STORE,$dir/store,S" \
$TMPDIR/install \
$TMPDIR/create-darwin-volume.sh \
$TMPDIR/install-darwin-multi-user.sh \
$TMPDIR/install-systemd-multi-user.sh \
$TMPDIR/install-multi-user \
$TMPDIR/reginfo \
$(cat ${installerClosureInfo}/store-paths)
'';
overlayFor = getStdenv: final: prev:
let currentStdenv = getStdenv final; in
{
nixStable = prev.nix;
nix =
with final;
with commonDeps {
let
currentStdenv = getStdenv final;
comDeps = with final; commonDeps {
inherit pkgs;
inherit (currentStdenv.hostPlatform) isStatic;
};
let
canRunInstalled = currentStdenv.buildPlatform.canExecute currentStdenv.hostPlatform;
in currentStdenv.mkDerivation (finalAttrs: {
name = "nix-${version}";
inherit version;
in {
nixStable = prev.nix;
src = nixSrc;
VERSION_SUFFIX = versionSuffix;
# Forward from the previous stage as we dont want it to pick the lowdown override
nixUnstable = prev.nixUnstable;
outputs = [ "out" "dev" "doc" ];
build-release-notes =
final.buildPackages.callPackage ./maintainers/build-release-notes.nix { };
clangbuildanalyzer = final.buildPackages.callPackage ./misc/clangbuildanalyzer.nix { };
boehmgc-nix = (final.boehmgc.override {
enableLargeConfig = true;
}).overrideAttrs (o: {
patches = (o.patches or [ ]) ++ [
./boehmgc-coroutine-sp-fallback.diff
nativeBuildInputs = nativeBuildDeps;
buildInputs = buildDeps
# There have been issues building these dependencies
++ lib.optionals (currentStdenv.hostPlatform == currentStdenv.buildPlatform) awsDeps
++ lib.optionals finalAttrs.doCheck checkDeps;
propagatedBuildInputs = propagatedDeps;
disallowedReferences = [ boost ];
preConfigure = lib.optionalString (! currentStdenv.hostPlatform.isStatic)
''
# Copy libboost_context so we don't get all of Boost in our closure.
# https://github.com/NixOS/nixpkgs/issues/45462
mkdir -p $out/lib
cp -pd ${boost}/lib/{libboost_context*,libboost_thread*,libboost_system*} $out/lib
rm -f $out/lib/*.a
${lib.optionalString currentStdenv.hostPlatform.isLinux ''
chmod u+w $out/lib/*.so.*
patchelf --set-rpath $out/lib:${currentStdenv.cc.cc.lib}/lib $out/lib/libboost_thread.so.*
''}
${lib.optionalString currentStdenv.hostPlatform.isDarwin ''
for LIB in $out/lib/*.dylib; do
chmod u+w $LIB
install_name_tool -id $LIB $LIB
install_name_tool -delete_rpath ${boost}/lib/ $LIB || true
done
install_name_tool -change ${boost}/lib/libboost_system.dylib $out/lib/libboost_system.dylib $out/lib/libboost_thread.dylib
''}
'';
configureFlags = configureFlags ++
[ "--sysconfdir=/etc" ] ++
lib.optional stdenv.hostPlatform.isStatic "--enable-embedded-sandbox-shell" ++
[ (lib.enableFeature finalAttrs.doCheck "tests") ] ++
lib.optionals finalAttrs.doCheck testConfigureFlags ++
lib.optional (!canRunInstalled) "--disable-doc-gen";
enableParallelBuilding = true;
makeFlags = "profiledir=$(out)/etc/profile.d PRECOMPILE_HEADERS=1";
doCheck = true;
installFlags = "sysconfdir=$(out)/etc";
postInstall = ''
mkdir -p $doc/nix-support
echo "doc manual $doc/share/doc/nix/manual" >> $doc/nix-support/hydra-build-products
${lib.optionalString currentStdenv.hostPlatform.isStatic ''
mkdir -p $out/nix-support
echo "file binary-dist $out/bin/nix" >> $out/nix-support/hydra-build-products
''}
${lib.optionalString currentStdenv.isDarwin ''
install_name_tool \
-change ${boost}/lib/libboost_context.dylib \
$out/lib/libboost_context.dylib \
$out/lib/libnixutil.dylib
''}
'';
doInstallCheck = finalAttrs.doCheck;
installCheckFlags = "sysconfdir=$(out)/etc";
installCheckTarget = "installcheck"; # work around buggy detection in stdenv
preInstallCheck = lib.optionalString stdenv.hostPlatform.isDarwin ''
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
'';
separateDebugInfo = !currentStdenv.hostPlatform.isStatic;
strictDeps = true;
hardeningDisable = lib.optional stdenv.hostPlatform.isStatic "pie";
passthru.perl-bindings = final.callPackage ./perl {
inherit fileset;
stdenv = currentStdenv;
};
meta.platforms = lib.platforms.unix;
# https://github.com/ivmai/bdwgc/pull/586
./boehmgc-traceable_allocator-public.diff
];
});
default-busybox-sandbox-shell = final.busybox.override {
useMusl = true;
enableStatic = true;
enableMinimal = true;
extraConfig = ''
CONFIG_FEATURE_FANCY_ECHO y
CONFIG_FEATURE_SH_MATH y
CONFIG_FEATURE_SH_MATH_64 y
CONFIG_ASH y
CONFIG_ASH_OPTIMIZE_FOR_SIZE y
CONFIG_ASH_ALIAS y
CONFIG_ASH_BASH_COMPAT y
CONFIG_ASH_CMDCMD y
CONFIG_ASH_ECHO y
CONFIG_ASH_GETOPTS y
CONFIG_ASH_INTERNAL_GLOB y
CONFIG_ASH_JOB_CONTROL y
CONFIG_ASH_PRINTF y
CONFIG_ASH_TEST y
'';
};
nix = final.callPackage ./package.nix {
inherit versionSuffix fileset;
stdenv = currentStdenv;
boehmgc = final.boehmgc-nix;
busybox-sandbox-shell = final.busybox-sandbox-shell or final.default-busybox-sandbox-shell;
};
};
in {
@@ -502,43 +218,53 @@
# Binary package for various platforms.
build = forAllSystems (system: self.packages.${system}.nix);
# FIXME(Qyriad): remove this when the migration to Meson has been completed.
# NOTE: mesonBuildClang depends on mesonBuild depends on build to avoid OOMs
# on aarch64 builders caused by too many parallel compiler/linker processes.
mesonBuild = forAllSystems (system:
(self.packages.${system}.nix.override {
buildWithMeson = true;
}).overrideAttrs (prev: {
buildInputs = prev.buildInputs ++ [ self.packages.${system}.nix ];
}));
mesonBuildClang = forAllSystems (system:
(nixpkgsFor.${system}.stdenvs.clangStdenvPackages.nix.override {
buildWithMeson = true;
}).overrideAttrs (prev: {
buildInputs = prev.buildInputs ++ [ self.hydraJobs.mesonBuild.${system} ];
})
);
# Perl bindings for various platforms.
perlBindings = forAllSystems (system: nixpkgsFor.${system}.native.nix.perl-bindings);
# Binary tarball for various platforms, containing a Nix store
# with the closure of 'nix' package, and the second half of
# the installation script.
# with the closure of 'nix' package.
binaryTarball = forAllSystems (system: binaryTarball nixpkgsFor.${system}.native.nix nixpkgsFor.${system}.native);
# docker image with Nix inside
dockerImage = lib.genAttrs linux64BitSystems (system: self.packages.${system}.dockerImage);
# API docs for Nix's unstable internal C++ interfaces.
internal-api-docs =
with nixpkgsFor.x86_64-linux.native;
with commonDeps { inherit pkgs; };
internal-api-docs = let
nixpkgs = nixpkgsFor.x86_64-linux.native;
inherit (nixpkgs) pkgs;
stdenv.mkDerivation {
pname = "nix-internal-api-docs";
inherit version;
src = nixSrc;
configureFlags = testConfigureFlags ++ internalApiDocsConfigureFlags;
nativeBuildInputs = nativeBuildDeps;
buildInputs = buildDeps ++ propagatedDeps
++ awsDeps ++ checkDeps ++ internalApiDocsDeps;
dontBuild = true;
installTargets = [ "internal-api-html" ];
postInstall = ''
mkdir -p $out/nix-support
echo "doc internal-api-docs $out/share/doc/nix/internal-api/html" >> $out/nix-support/hydra-build-products
'';
nix = pkgs.callPackage ./package.nix {
inherit versionSuffix fileset officialRelease buildUnreleasedNotes;
inherit (pkgs) build-release-notes;
internalApiDocs = true;
boehmgc = pkgs.boehmgc-nix;
busybox-sandbox-shell = pkgs.busybox-sandbox-shell;
};
in
nix.overrideAttrs (prev: {
# This Hydra job is just for the internal API docs.
# We don't need the build artifacts here.
dontBuild = true;
doCheck = false;
doInstallCheck = false;
});
# System tests.
tests = import ./tests/nixos { inherit lib nixpkgs nixpkgsFor; } // {
@@ -552,7 +278,7 @@
type -p nix-env
# Note: we're filtering out nixos-install-tools because https://github.com/NixOS/nixpkgs/pull/153594#issuecomment-1020530593.
time nix-env --store dummy:// -f ${nixpkgs-regression} -qaP --drv-path | sort | grep -v nixos-install-tools > packages
[[ $(sha1sum < packages | cut -c1-40) = ff451c521e61e4fe72bdbe2d0ca5d1809affa733 ]]
[[ $(sha1sum < packages | cut -c1-40) = 402242fca90874112b34718b8199d844e8b03d12 ]]
mkdir $out
'';
@@ -565,31 +291,66 @@
);
};
installTests = forAllSystems (system:
let pkgs = nixpkgsFor.${system}.native; in
pkgs.runCommand "install-tests" {
againstSelf = testNixVersions pkgs pkgs.nix pkgs.pkgs.nix;
againstCurrentUnstable =
# FIXME: temporarily disable this on macOS because of #3605.
if system == "x86_64-linux"
then testNixVersions pkgs pkgs.nix pkgs.nixUnstable
else null;
# Disabled because the latest stable version doesn't handle
# `NIX_DAEMON_SOCKET_PATH` which is required for the tests to work
# againstLatestStable = testNixVersions pkgs pkgs.nix pkgs.nixStable;
} "touch $out");
pre-commit = builtins.mapAttrs (system: pre-commit-lib: pre-commit-lib.run {
src = self;
hooks = {
no-commit-to-branch = {
enable = true;
settings.branch = ["main"];
};
check-case-conflicts.enable = true;
check-executables-have-shebangs = {
enable = true;
stages = [ "commit" ];
};
check-shebang-scripts-are-executable = {
enable = true;
stages = [ "commit" ];
};
check-symlinks = {
enable = true;
excludes = [ "^tests/functional/lang/symlink-resolution/broken$" ];
};
check-merge-conflicts.enable = true;
end-of-file-fixer = {
enable = true;
excludes = [
"\\.drv$"
"^tests/functional/lang/"
];
};
mixed-line-endings = {
enable = true;
excludes = [ "^tests/functional/lang/" ];
};
# TODO: Once the test suite is nicer, clean up and start
# enforcing trailing whitespace on tests that don't explicitly
# check for it.
trim-trailing-whitespace = {
enable = true;
stages = [ "commit" ];
excludes = [ "^tests/functional/lang/" ];
};
};
}) pre-commit-hooks.lib;
};
checks = forAllSystems (system: {
checks = forAllSystems (system: let
rl-next-check = name: dir:
let pkgs = nixpkgsFor.${system}.native;
in pkgs.buildPackages.runCommand "test-${name}-release-notes" { } ''
LANG=C.UTF-8 ${lib.getExe pkgs.build-release-notes} ${dir} >$out
'';
in {
# FIXME(Qyriad): remove this when the migration to Meson has been completed.
mesonBuild = self.hydraJobs.mesonBuild.${system};
mesonBuildClang = self.hydraJobs.mesonBuildClang.${system};
binaryTarball = self.hydraJobs.binaryTarball.${system};
perlBindings = self.hydraJobs.perlBindings.${system};
installTests = self.hydraJobs.installTests.${system};
nixpkgsLibTests = self.hydraJobs.tests.nixpkgsLibTests.${system};
rl-next =
let pkgs = nixpkgsFor.${system}.native;
in pkgs.buildPackages.runCommand "test-rl-next-release-notes" { } ''
LANG=C.UTF-8 ${(commonDeps { inherit pkgs; }).changelog-d}/bin/changelog-d ${./doc/manual/rl-next} >$out
'';
rl-next = rl-next-check "rl-next" ./doc/manual/rl-next;
rl-next-dev = rl-next-check "rl-next-dev" ./doc/manual/rl-next-dev;
pre-commit = self.hydraJobs.pre-commit.${system};
} // (lib.optionalAttrs (builtins.elem system linux64BitSystems)) {
dockerImage = self.hydraJobs.dockerImage.${system};
});
@@ -629,44 +390,60 @@
devShells = let
makeShell = pkgs: stdenv:
let
canRunInstalled = stdenv.buildPlatform.canExecute stdenv.hostPlatform;
nix = pkgs.callPackage ./package.nix {
inherit stdenv versionSuffix fileset;
boehmgc = pkgs.boehmgc-nix;
busybox-sandbox-shell = pkgs.busybox-sandbox-shell or pkgs.default-busybox-sandbox;
forDevShell = true;
};
pre-commit = self.hydraJobs.pre-commit.${pkgs.system} or {};
in
with commonDeps { inherit pkgs; };
stdenv.mkDerivation {
name = "nix";
(nix.override {
buildUnreleasedNotes = true;
officialRelease = false;
}).overrideAttrs (prev: {
# Required for clang-tidy checks
buildInputs = prev.buildInputs
++ [ pkgs.just ]
++ lib.optional (pre-commit ? enabledPackages) pre-commit.enabledPackages
++ lib.optionals (stdenv.cc.isClang) [ pkgs.llvmPackages.llvm pkgs.llvmPackages.clang-unwrapped.dev ];
nativeBuildInputs = prev.nativeBuildInputs
++ lib.optional (stdenv.cc.isClang && !stdenv.buildPlatform.isDarwin) pkgs.buildPackages.bear
# Required for clang-tidy checks
++ lib.optionals (stdenv.cc.isClang) [ pkgs.buildPackages.cmake pkgs.buildPackages.ninja pkgs.buildPackages.llvmPackages.llvm.dev ]
++ lib.optional
(stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform)
# for some reason that seems accidental and was changed in
# NixOS 24.05-pre, clang-tools is pinned to LLVM 14 when
# default LLVM is newer.
(pkgs.buildPackages.clang-tools.override { inherit (pkgs.buildPackages) llvmPackages; })
++ [
# FIXME(Qyriad): remove once the migration to Meson is complete.
pkgs.buildPackages.meson
pkgs.buildPackages.ninja
outputs = [ "out" "dev" "doc" ];
pkgs.buildPackages.clangbuildanalyzer
];
nativeBuildInputs = nativeBuildDeps
++ lib.optional (stdenv.cc.isClang && !stdenv.buildPlatform.isDarwin) pkgs.buildPackages.bear
++ lib.optional
(stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform)
pkgs.buildPackages.clang-tools
# We want changelog-d in the shell even if the current build doesn't need it
++ lib.optional (officialRelease || ! buildUnreleasedNotes) changelog-d
;
src = null;
buildInputs = buildDeps ++ propagatedDeps
++ awsDeps ++ checkDeps ++ internalApiDocsDeps;
installFlags = "sysconfdir=$(out)/etc";
strictDeps = false;
configureFlags = configureFlags
++ testConfigureFlags ++ internalApiDocsConfigureFlags
++ lib.optional (!canRunInstalled) "--disable-doc-gen";
enableParallelBuilding = true;
installFlags = "sysconfdir=$(out)/etc";
shellHook =
''
shellHook = ''
PATH=$prefix/bin:$PATH
unset PYTHONPATH
export MANPATH=$out/share/man:$MANPATH
# Make bash completion work.
XDG_DATA_DIRS+=:$out/share
${lib.optionalString (pre-commit ? shellHook) pre-commit.shellHook}
'';
};
} // lib.optionalAttrs (stdenv.buildPlatform.isLinux && pkgs.glibcLocales != null) {
# Required to make non-NixOS Linux not complain about missing locale files during configure in a dev shell
LOCALE_ARCHIVE = "${lib.getLib pkgs.glibcLocales}/lib/locale/locale-archive";
});
in
forAllSystems (system:
let
+19
View File
@@ -0,0 +1,19 @@
# https://just.systems/man/en/
clean:
rm -rf build
setup:
meson setup build --prefix="$PWD/outputs/out"
build:
meson compile -C build
compile:
just build
install:
meson install -C build
test *OPTIONS:
meson test -C build --print-errorlogs --quiet {{ OPTIONS }}
+8 -1
View File
@@ -1,4 +1,11 @@
GLOBAL_CXXFLAGS += -Wno-deprecated-declarations -Werror=switch
# 2024-03-24: jade benchmarked the default sanitize reporting in clang and got
# a regression of about 10% on hackage-packages.nix with clang. So we are trapping instead.
#
# This has an overhead of 0-4% on gcc and unmeasurably little on clang, in
# Nix evaluation benchmarks.
DEFAULT_SANITIZE_FLAGS = -fsanitize=signed-integer-overflow -fsanitize-undefined-trap-on-error
GLOBAL_CXXFLAGS += -Wno-deprecated-declarations -Werror=switch $(DEFAULT_SANITIZE_FLAGS)
GLOBAL_LDFLAGS += $(DEFAULT_SANITIZE_FLAGS)
# Allow switch-enum to be overridden for files that do not support it, usually because of dependency headers.
ERROR_SWITCH_ENUM = -Werror=switch-enum
+6
View File
@@ -0,0 +1,6 @@
{ lib, python3, writeShellScriptBin }:
writeShellScriptBin "build-release-notes" ''
exec ${lib.getExe (python3.withPackages (p: [ p.python-frontmatter ]))} \
${./build-release-notes.py} "$@"
''
+66
View File
@@ -0,0 +1,66 @@
import frontmatter
import sys
import pathlib
import textwrap
GH_BASE = "https://github.com/NixOS/nix"
FORGEJO_BASE = "https://git.lix.systems/lix-project/lix"
GERRIT_BASE = "https://gerrit.lix.systems/c/lix/+"
SIGNIFICANCECES = {
None: 0,
'significant': 10,
}
def format_link(ident: str, gh_part: str, fj_part: str) -> str:
# FIXME: deprecate github as default
if ident.isdigit():
num, link, base = int(ident), f"#{ident}", f"{GH_BASE}/{gh_part}"
elif ident.startswith("gh#"):
num, link, base = int(ident[3:]), ident, f"{GH_BASE}/{gh_part}"
elif ident.startswith("fj#"):
num, link, base = int(ident[3:]), ident, f"{FORGEJO_BASE}/{fj_part}"
else:
raise Exception("unrecognized reference format", ident)
return f"[{link}]({base}/{num})"
def format_issue(issue: str) -> str:
return format_link(issue, "issues", "issues")
def format_pr(pr: str) -> str:
return format_link(pr, "pull", "pulls")
def format_cl(cl: str) -> str:
clid = int(cl)
return f"[cl/{clid}]({GERRIT_BASE}/{clid})"
paths = pathlib.Path(sys.argv[1]).glob('*.md')
entries = []
for p in paths:
try:
e = frontmatter.load(p)
if 'synopsis' not in e.metadata:
raise Exception('missing synposis')
unknownKeys = set(e.metadata.keys()) - set(('synopsis', 'cls', 'issues', 'prs', 'significance'))
if unknownKeys:
raise Exception('unknown keys', unknownKeys)
entries.append((p, e))
except Exception as e:
e.add_note(f"in {p}")
raise
for p, entry in sorted(entries, key=lambda e: (-SIGNIFICANCECES[e[1].metadata.get('significance')], e[0])):
try:
header = entry.metadata['synopsis']
links = []
links += map(format_issue, str(entry.metadata.get('issues', "")).split())
links += map(format_pr, str(entry.metadata.get('prs', "")).split())
links += map(format_cl, str(entry.metadata.get('cls', "")).split())
if links != []:
header += " " + " ".join(links)
if header:
print(f"- {header}")
print()
print(textwrap.indent(entry.content, ' '))
print()
except Exception as e:
e.add_note(f"in {p}")
raise
+20
View File
@@ -0,0 +1,20 @@
#!/bin/sh
# Generates a report of build time based on a meson build using -ftime-trace in
# Clang.
if [ $# -lt 1 ]; then
echo "usage: $0 BUILD-DIR [filename]" >&2
exit 1
fi
scriptdir=$(cd "$(dirname -- "$0")" || exit ; pwd -P)
filename=${2:-$scriptdir/../buildtime.bin}
if [ "$(meson introspect "$1" --buildoptions | jq -r '.[] | select(.name == "profile-build") | .value')" != enabled ]; then
echo 'This build was not done with profile-build enabled, so cannot generate a report' >&2
# shellcheck disable=SC2016
echo 'Run `meson configure build -Dprofile-build=enabled` then rebuild, first' >&2
exit 1
fi
ClangBuildAnalyzer --all "$1" "$filename" && ClangBuildAnalyzer --analyze "$filename"
+148
View File
@@ -0,0 +1,148 @@
import requests
import textwrap
import dataclasses
import logging
import re
import os
API_BASE = 'https://git.lix.systems/api/v1'
API_KEY = os.environ['FORGEJO_API_KEY']
log = logging.getLogger(__name__)
log.setLevel(logging.INFO)
fmt = logging.Formatter('{asctime} {levelname} {name}: {message}',
datefmt='%b %d %H:%M:%S',
style='{')
if not any(isinstance(h, logging.StreamHandler) for h in log.handlers):
hand = logging.StreamHandler()
hand.setFormatter(fmt)
log.addHandler(hand)
# These are erring in the direction of re-triage, rather than necessarily
# mapping all metadata of the issue
LABEL_MAPPING = {
'lix-import': 153, # 'imported',
'contributor-experience': 148, # 'devx',
'bug': 150, # 'bug',
'UX': 149, # 'ux',
'error-messages': 149, # 'ux',
'lix-stability': 146, # 'stability',
'performance': 147, # 'performance',
'tests': 121, # 'tests',
}
def api(method, endpoint: str, resp_json=True, **kwargs):
log.info('http %s %s', method, endpoint)
if not endpoint.startswith('https'):
endpoint = API_BASE + endpoint
resp = requests.request(method,
endpoint,
headers={'Authorization': f'Bearer {API_KEY}'},
**kwargs)
resp.raise_for_status()
if resp_json:
return resp.json()
else:
return resp
def paginate(method: str, url: str):
while True:
resp = api(method, url, resp_json=False)
yield from resp.json()
next_one = resp.links.get('next')
if not next_one:
return
url = next_one.get('url')
if not url:
return
class DataClassUnpack:
"""Taken from: https://stackoverflow.com/a/72164665"""
classFieldCache = {}
@classmethod
def instantiate(cls, classToInstantiate, argDict):
if classToInstantiate not in cls.classFieldCache:
cls.classFieldCache[classToInstantiate] = {
f.name
for f in getattr(classToInstantiate, dataclasses._FIELDS).values() if f._field_type is not dataclasses._FIELD_CLASSVAR # type: ignore
}
fieldSet = cls.classFieldCache[classToInstantiate]
filteredArgDict = {k: v for k, v in argDict.items() if k in fieldSet}
return classToInstantiate(**filteredArgDict)
@dataclasses.dataclass
class Label:
name: str
description: str
@dataclasses.dataclass
class Issue:
number: int
url: str
html_url: str
title: str
body: str
labels: dataclasses.InitVar[list[dict]]
labels_clean: list[Label] = dataclasses.field(init=False)
def __post_init__(self, labels):
self.labels_clean = [DataClassUnpack.instantiate(Label, l) for l in labels]
def issues_to_import():
yield from paginate('GET', '/repos/nixos/nix/issues?state=open&labels=lix-import')
def issues_already_imported():
yield from paginate('GET', '/repos/lix-project/lix/issues?state=all&labels=imported')
UPSTREAM_ISSUE_RE = re.compile(r'^Upstream-Issue: https://git\.lix\.systems/NixOS/nix/issues/(\d+)$', re.MULTILINE)
def make_already_imported():
d = {}
for issue in issues_already_imported():
iss = DataClassUnpack.instantiate(Issue, issue)
print(iss)
match = UPSTREAM_ISSUE_RE.search(iss.body)
if match:
d[int(match.group(1))] = iss
return d
def new_issue(title, body, labels):
api('POST', '/repos/lix-project/lix/issues', resp_json=True, json={
'labels': labels,
'body': body,
'title': title,
'dont_notify': True,
})
already_imported = make_already_imported()
def import_issue(iss: Issue):
if iss.number in already_imported:
log.info('Skipping already imported %d', iss.number)
return
new_body = textwrap.dedent('''
Upstream-Issue: {iss}
{original_body}
''').format(iss=iss.html_url, original_body=iss.body)
new_labels = [LABEL_MAPPING[l.name] for l in iss.labels_clean if l.name in LABEL_MAPPING]
new_title = '[Nix#{num}] {title}'.format(num=iss.number, title=iss.title)
log.info('%s', f'create issue with: {new_labels} {new_title} {new_body}')
new_issue(new_title, new_body, new_labels)
def go():
log.info('Importing issues!')
for issue in issues_to_import():
import_issue(DataClassUnpack.instantiate(Issue, issue))
if __name__ == '__main__':
go()

Some files were not shown because too many files have changed in this diff Show More