Compare commits
91
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f82a2a9aaa | ||
|
|
e1059bfa34 | ||
|
|
d5c670ad01 | ||
|
|
16ea19ced8 | ||
|
|
7be0d237e0 | ||
|
|
e715e5fd31 | ||
|
|
068576042b | ||
|
|
d194939ff5 | ||
|
|
74fb2e8c47 | ||
|
|
ac28cff28f | ||
|
|
1d6fd94cf9 | ||
|
|
d0b28f0e74 | ||
|
|
38e4e69633 | ||
|
|
248ecb11af | ||
|
|
73898cad0e | ||
|
|
59b5965bbf | ||
|
|
211f79d4a2 | ||
|
|
5f6eb6eb44 | ||
|
|
d9345d8836 | ||
|
|
6939ffc9f9 | ||
|
|
479055aee8 | ||
|
|
f7b6552699 | ||
|
|
3c0434999e | ||
|
|
f46194faa2 | ||
|
|
8a3d063a49 | ||
|
|
f432e464dd | ||
|
|
a986a8dfa1 | ||
|
|
8a09465c3a | ||
|
|
82dc712d93 | ||
|
|
ce71d0e9ab | ||
|
|
9aeb314e6a | ||
|
|
4392d89eea | ||
|
|
9bb7fb8f69 | ||
|
|
7dfa2a761e | ||
|
|
ff95b980d4 | ||
|
|
28a079f841 | ||
|
|
9281a12532 | ||
|
|
a05de58ebd | ||
|
|
4f94531209 | ||
|
|
98e8475147 | ||
|
|
bdf1b264ad | ||
|
|
c32a01f9eb | ||
|
|
ec768df004 | ||
|
|
611b1de441 | ||
|
|
9c77c62e73 | ||
|
|
24057dcb6a | ||
|
|
1659404626 | ||
|
|
e0748377dc | ||
|
|
766e718f67 | ||
|
|
06e65e537b | ||
|
|
8f9bcd20eb | ||
|
|
609b721425 | ||
|
|
6e59b4b407 | ||
|
|
a3256a9375 | ||
|
|
72d85acba4 | ||
|
|
5d3910330d | ||
|
|
cc3674ea93 | ||
|
|
c55e93ca23 | ||
|
|
d374a9908f | ||
|
|
b8cb7abcf0 | ||
|
|
5312e60be6 | ||
|
|
e54d4c9381 | ||
|
|
c7ca87461d | ||
|
|
7081889faa | ||
|
|
adedac70fa | ||
|
|
82de36f77a | ||
|
|
a75d7a5777 | ||
|
|
ff08d95420 | ||
|
|
713cd7e9e7 | ||
|
|
d4b7e6baca | ||
|
|
ac78c1dcd5 | ||
|
|
0c6cb34de6 | ||
|
|
cf756fdf3c | ||
|
|
a6b33cb3b2 | ||
|
|
0f99ed43f1 | ||
|
|
e6e5cacabe | ||
|
|
3df013597d | ||
|
|
2f104bbe3b | ||
|
|
533d469875 | ||
|
|
260db1ea64 | ||
|
|
c161687b5f | ||
|
|
18aa3e1d57 | ||
|
|
53d40888ff | ||
|
|
3d78b4847e | ||
|
|
6abac7aacc | ||
|
|
da95bf8c82 | ||
|
|
6475793678 | ||
|
|
7575db522e | ||
|
|
285bc67318 | ||
|
|
26b3a1b9ce | ||
|
|
d8e452a91b |
@@ -1,18 +0,0 @@
|
||||
# Pull requests concerning the listed files will automatically invite the respective maintainers as reviewers.
|
||||
# This file is not used for denoting any kind of ownership, but is merely a tool for handling notifications.
|
||||
#
|
||||
# Merge permissions are required for maintaining an entry in this file.
|
||||
# For documentation on this mechanism, see https://help.github.com/articles/about-codeowners/
|
||||
|
||||
# Default reviewers if nothing else matches
|
||||
* @edolstra
|
||||
|
||||
# This file
|
||||
.github/CODEOWNERS @edolstra
|
||||
|
||||
# Public documentation
|
||||
/doc @fricklerhandwerk
|
||||
*.md @fricklerhandwerk
|
||||
|
||||
# Libstore layer
|
||||
/src/libstore @thufschmitt
|
||||
@@ -9,7 +9,7 @@ assignees: ''
|
||||
|
||||
## Platform
|
||||
|
||||
<!-- select the platform on which you tried to install Nix -->
|
||||
<!-- select the platform on which you tried to install Lix -->
|
||||
|
||||
- [ ] Linux: <!-- state your distribution, e.g. Arch Linux, Ubuntu, ... -->
|
||||
- [ ] macOS
|
||||
|
||||
@@ -19,9 +19,10 @@ assignees: ''
|
||||
|
||||
<!-- make sure this issue is not redundant or obsolete -->
|
||||
|
||||
- [ ] checked [latest Nix manual] \([source])
|
||||
- [ ] checked [open documentation issues and pull requests] for possible duplicates
|
||||
- [ ] checked [latest Lix manual] \([source]\)
|
||||
- [ ] checked [documentation issues] and [recent documentation changes] for possible duplicates
|
||||
|
||||
[latest Nix manual]: https://nixos.org/manual/nix/unstable/
|
||||
[source]: https://github.com/NixOS/nix/tree/master/doc/manual/src
|
||||
[open documentation issues and pull requests]: https://github.com/NixOS/nix/labels/documentation
|
||||
[latest Nix manual]: https://docs.lix.systems/manual/lix/nightly
|
||||
[source]: https://git.lix.systems/lix-project/lix/src/main/doc/manual/src
|
||||
[documentation issues]: https://git.lix.systems/lix-project/lix/issues?labels=151&state=all
|
||||
[recent documentation changes]: https://gerrit.lix.systems/q/p:lix+path:%22%5Edoc/manual/.*%22
|
||||
|
||||
@@ -1,35 +0,0 @@
|
||||
# Stale bot information
|
||||
|
||||
- Thanks for your contribution!
|
||||
- To remove the stale label, just leave a new comment.
|
||||
- _How to find the right people to ping?_ → [`git blame`](https://git-scm.com/docs/git-blame) to the rescue! (or GitHub's history and blame buttons.)
|
||||
- You can always ask for help on [our Discourse Forum](https://discourse.nixos.org/) or on [Matrix - #nix:nixos.org](https://matrix.to/#/#nix:nixos.org).
|
||||
|
||||
## Suggestions for PRs
|
||||
|
||||
1. GitHub sometimes doesn't notify people who commented / reviewed a PR previously, when you (force) push commits. If you have addressed the reviews you can [officially ask for a review](https://docs.github.com/en/free-pro-team@latest/github/collaborating-with-issues-and-pull-requests/requesting-a-pull-request-review) from those who commented to you or anyone else.
|
||||
2. If it is unfinished but you plan to finish it, please mark it as a draft.
|
||||
3. If you don't expect to work on it any time soon, closing it with a short comment may encourage someone else to pick up your work.
|
||||
4. To get things rolling again, rebase the PR against the target branch and address valid comments.
|
||||
5. If you need a review to move forward, ask in [the Discourse thread for PRs that need help](https://discourse.nixos.org/t/prs-in-distress/3604).
|
||||
6. If all you need is a merge, check the git history to find and [request reviews](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/requesting-a-pull-request-review) from people who usually merge related contributions.
|
||||
|
||||
## Suggestions for issues
|
||||
|
||||
1. If it is resolved (either for you personally, or in general), please consider closing it.
|
||||
2. If this might still be an issue, but you are not interested in promoting its resolution, please consider closing it while encouraging others to take over and reopen an issue if they care enough.
|
||||
3. If you still have interest in resolving it, try to ping somebody who you believe might have an interest in the topic. Consider discussing the problem in [our Discourse Forum](https://discourse.nixos.org/).
|
||||
4. As with all open source projects, your best option is to submit a Pull Request that addresses this issue. We :heart: this attitude!
|
||||
|
||||
**Memorandum on closing issues**
|
||||
|
||||
Don't be afraid to close an issue that holds valuable information. Closed issues stay in the system for people to search, read, cross-reference, or even reopen--nothing is lost! Closing obsolete issues is an important way to help maintainers focus their time and effort.
|
||||
|
||||
## Useful GitHub search queries
|
||||
|
||||
- [Open PRs with any stale-bot interaction](https://github.com/NixOS/nix/pulls?q=is%3Apr+is%3Aopen+commenter%3Aapp%2Fstale+)
|
||||
- [Open PRs with any stale-bot interaction and `stale`](https://github.com/NixOS/nix/pulls?q=is%3Apr+is%3Aopen+commenter%3Aapp%2Fstale+label%3A%22stale%22)
|
||||
- [Open PRs with any stale-bot interaction and NOT `stale`](https://github.com/NixOS/nix/pulls?q=is%3Apr+is%3Aopen+commenter%3Aapp%2Fstale+-label%3A%22stale%22+)
|
||||
- [Open Issues with any stale-bot interaction](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+commenter%3Aapp%2Fstale+)
|
||||
- [Open Issues with any stale-bot interaction and `stale`](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+commenter%3Aapp%2Fstale+label%3A%22stale%22+)
|
||||
- [Open Issues with any stale-bot interaction and NOT `stale`](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+commenter%3Aapp%2Fstale+-label%3A%22stale%22+)
|
||||
@@ -1,6 +0,0 @@
|
||||
version: 2
|
||||
updates:
|
||||
- package-ecosystem: "github-actions"
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
@@ -1,23 +0,0 @@
|
||||
"documentation":
|
||||
- doc/manual/*
|
||||
- src/nix/**/*.md
|
||||
|
||||
"store":
|
||||
- src/libstore/store-api.*
|
||||
- src/libstore/*-store.*
|
||||
|
||||
"fetching":
|
||||
- src/libfetchers/**/*
|
||||
|
||||
"repl":
|
||||
- src/libcmd/repl.*
|
||||
- src/nix/repl.*
|
||||
|
||||
"new-cli":
|
||||
- src/nix/**/*
|
||||
|
||||
"with-tests":
|
||||
# Unit tests
|
||||
- src/*/tests/**/*
|
||||
# Functional and integration tests
|
||||
- tests/functional/**/*
|
||||
@@ -1,9 +0,0 @@
|
||||
# Configuration for probot-stale - https://github.com/probot/stale
|
||||
daysUntilStale: 180
|
||||
daysUntilClose: false
|
||||
exemptLabels:
|
||||
- "critical"
|
||||
- "never-stale"
|
||||
staleLabel: "stale"
|
||||
markComment: false
|
||||
closeComment: false
|
||||
+1
-129
@@ -1,128 +1,5 @@
|
||||
Makefile.config
|
||||
perl/Makefile.config
|
||||
|
||||
# /
|
||||
/aclocal.m4
|
||||
/autom4te.cache
|
||||
/precompiled-headers.h.gch
|
||||
/config.*
|
||||
/configure
|
||||
/stamp-h1
|
||||
/svn-revision
|
||||
/libtool
|
||||
/config
|
||||
|
||||
# /doc/manual/
|
||||
/doc/manual/*.1
|
||||
/doc/manual/*.5
|
||||
/doc/manual/*.8
|
||||
/doc/manual/generated/*
|
||||
/doc/manual/nix.json
|
||||
/doc/manual/conf-file.json
|
||||
/doc/manual/language.json
|
||||
/doc/manual/xp-features.json
|
||||
/doc/manual/src/command-ref/experimental-features-shortlist.md
|
||||
/doc/manual/src/contributing/experimental-feature-descriptions.md
|
||||
/doc/manual/src/release-notes/rl-next-generated.md
|
||||
|
||||
# /scripts/
|
||||
/scripts/nix-profile.sh
|
||||
/scripts/nix-profile-daemon.sh
|
||||
/scripts/nix-profile.fish
|
||||
/scripts/nix-profile-daemon.fish
|
||||
|
||||
# /src/libexpr/
|
||||
/src/libexpr/lexer-tab.cc
|
||||
/src/libexpr/lexer-tab.hh
|
||||
/src/libexpr/parser-tab.cc
|
||||
/src/libexpr/parser-tab.hh
|
||||
/src/libexpr/parser-tab.output
|
||||
/src/libexpr/nix.tbl
|
||||
/src/libexpr/tests
|
||||
/tests/unit/libexpr/libnixexpr-tests
|
||||
|
||||
# /src/libstore/
|
||||
*.gen.*
|
||||
/src/libstore/tests
|
||||
/tests/unit/libstore/libnixstore-tests
|
||||
|
||||
# /src/libutil/
|
||||
/src/libutil/tests
|
||||
/tests/unit/libutil/libnixutil-tests
|
||||
|
||||
/src/nix/nix
|
||||
|
||||
/src/nix/doc
|
||||
|
||||
# /src/nix-env/
|
||||
/src/nix-env/nix-env
|
||||
|
||||
# /src/nix-instantiate/
|
||||
/src/nix-instantiate/nix-instantiate
|
||||
|
||||
# /src/nix-store/
|
||||
/src/nix-store/nix-store
|
||||
|
||||
/src/nix-prefetch-url/nix-prefetch-url
|
||||
|
||||
/src/nix-collect-garbage/nix-collect-garbage
|
||||
|
||||
# /src/nix-channel/
|
||||
/src/nix-channel/nix-channel
|
||||
|
||||
# /src/nix-build/
|
||||
/src/nix-build/nix-build
|
||||
|
||||
/src/nix-copy-closure/nix-copy-closure
|
||||
|
||||
/src/error-demo/error-demo
|
||||
|
||||
/src/build-remote/build-remote
|
||||
|
||||
# /tests/functional/
|
||||
/tests/functional/test-tmp
|
||||
/tests/functional/common/vars-and-functions.sh
|
||||
/tests/functional/result*
|
||||
/tests/functional/restricted-innocent
|
||||
/tests/functional/shell
|
||||
/tests/functional/shell.drv
|
||||
/tests/functional/config.nix
|
||||
/tests/functional/ca/config.nix
|
||||
/tests/functional/dyn-drv/config.nix
|
||||
/tests/functional/repl-result-out
|
||||
/tests/functional/debugger-test-out
|
||||
/tests/functional/test-libstoreconsumer/test-libstoreconsumer
|
||||
|
||||
# /tests/functional/lang/
|
||||
/tests/functional/lang/*.out
|
||||
/tests/functional/lang/*.out.xml
|
||||
/tests/functional/lang/*.err
|
||||
/tests/functional/lang/*.ast
|
||||
|
||||
/perl/lib/Nix/Config.pm
|
||||
/perl/lib/Nix/Store.cc
|
||||
|
||||
/misc/systemd/nix-daemon.service
|
||||
/misc/systemd/nix-daemon.socket
|
||||
/misc/systemd/nix-daemon.conf
|
||||
/misc/upstart/nix-daemon.conf
|
||||
|
||||
/src/resolve-system-dependencies/resolve-system-dependencies
|
||||
|
||||
outputs/
|
||||
|
||||
*.a
|
||||
*.o
|
||||
*.o.tmp
|
||||
*.so
|
||||
*.dylib
|
||||
*.dll
|
||||
*.exe
|
||||
*.dep
|
||||
*~
|
||||
*.pc
|
||||
*.plist
|
||||
|
||||
# GNU Global
|
||||
GPATH
|
||||
GRTAGS
|
||||
@@ -132,17 +9,11 @@ GTAGS
|
||||
# ccls
|
||||
/.ccls-cache
|
||||
|
||||
# auto-generated compilation database
|
||||
compile_commands.json
|
||||
|
||||
nix-rust/target
|
||||
|
||||
result
|
||||
result-*
|
||||
|
||||
.vscode/
|
||||
.direnv/
|
||||
.envrc.local
|
||||
|
||||
# clangd and possibly more
|
||||
.cache/
|
||||
@@ -157,3 +28,4 @@ buildtime.bin
|
||||
# We generate this with a Nix shell hook
|
||||
/.pre-commit-config.yaml
|
||||
/.nocontribmsg
|
||||
/release
|
||||
|
||||
@@ -6,14 +6,14 @@ Read more about us at https://lix.systems.
|
||||
|
||||
## Installation
|
||||
|
||||
On Linux and macOS the easiest way to install Nix is to run the following shell command
|
||||
On Linux and macOS the easiest way to install Lix is to run the following shell command
|
||||
(as a user other than root):
|
||||
|
||||
```console
|
||||
$ curl -sSf -L https://install.lix.systems/lix | sh -s -- install
|
||||
```
|
||||
|
||||
For systems that **already have Nix installed**, such as NixOS systems, read our [install page](https://lix.systems/install)
|
||||
For systems that **already have a Nix implementation installed**, such as NixOS systems, read our [install page](https://lix.systems/install)
|
||||
|
||||
## Building And Developing
|
||||
|
||||
|
||||
@@ -1,56 +1,8 @@
|
||||
diff --git a/darwin_stop_world.c b/darwin_stop_world.c
|
||||
index 0468aaec..b348d869 100644
|
||||
--- a/darwin_stop_world.c
|
||||
+++ b/darwin_stop_world.c
|
||||
@@ -356,6 +356,7 @@ GC_INNER void GC_push_all_stacks(void)
|
||||
int nthreads = 0;
|
||||
word total_size = 0;
|
||||
mach_msg_type_number_t listcount = (mach_msg_type_number_t)THREAD_TABLE_SZ;
|
||||
+ size_t stack_limit;
|
||||
if (!EXPECT(GC_thr_initialized, TRUE))
|
||||
GC_thr_init();
|
||||
|
||||
@@ -411,6 +412,19 @@ GC_INNER void GC_push_all_stacks(void)
|
||||
GC_push_all_stack_sections(lo, hi, p->traced_stack_sect);
|
||||
}
|
||||
if (altstack_lo) {
|
||||
+ // When a thread goes into a coroutine, we lose its original sp until
|
||||
+ // control flow returns to the thread.
|
||||
+ // While in the coroutine, the sp points outside the thread stack,
|
||||
+ // so we can detect this and push the entire thread stack instead,
|
||||
+ // as an approximation.
|
||||
+ // We assume that the coroutine has similarly added its entire stack.
|
||||
+ // This could be made accurate by cooperating with the application
|
||||
+ // via new functions and/or callbacks.
|
||||
+ stack_limit = pthread_get_stacksize_np(p->id);
|
||||
+ if (altstack_lo >= altstack_hi || altstack_lo < altstack_hi - stack_limit) { // sp outside stack
|
||||
+ altstack_lo = altstack_hi - stack_limit;
|
||||
+ }
|
||||
+
|
||||
total_size += altstack_hi - altstack_lo;
|
||||
GC_push_all_stack(altstack_lo, altstack_hi);
|
||||
}
|
||||
diff --git a/include/gc.h b/include/gc.h
|
||||
index edab6c22..f2c61282 100644
|
||||
--- a/include/gc.h
|
||||
+++ b/include/gc.h
|
||||
@@ -2172,6 +2172,11 @@ GC_API void GC_CALL GC_win32_free_heap(void);
|
||||
(*GC_amiga_allocwrapper_do)(a,GC_malloc_atomic_ignore_off_page)
|
||||
#endif /* _AMIGA && !GC_AMIGA_MAKINGLIB */
|
||||
|
||||
+#if !__APPLE__
|
||||
+/* Patch doesn't work on apple */
|
||||
+#define NIX_BOEHM_PATCH_VERSION 1
|
||||
+#endif
|
||||
+
|
||||
#ifdef __cplusplus
|
||||
} /* extern "C" */
|
||||
#endif
|
||||
diff --git a/pthread_stop_world.c b/pthread_stop_world.c
|
||||
index b5d71e62..aed7b0bf 100644
|
||||
index 2b45489..0e6d8ef 100644
|
||||
--- a/pthread_stop_world.c
|
||||
+++ b/pthread_stop_world.c
|
||||
@@ -768,6 +768,8 @@ STATIC void GC_restart_handler(int sig)
|
||||
@@ -776,6 +776,8 @@ STATIC void GC_restart_handler(int sig)
|
||||
/* world is stopped. Should not fail if it isn't. */
|
||||
GC_INNER void GC_push_all_stacks(void)
|
||||
{
|
||||
@@ -59,20 +11,23 @@ index b5d71e62..aed7b0bf 100644
|
||||
GC_bool found_me = FALSE;
|
||||
size_t nthreads = 0;
|
||||
int i;
|
||||
@@ -851,6 +853,37 @@ GC_INNER void GC_push_all_stacks(void)
|
||||
hi = p->altstack + p->altstack_size;
|
||||
@@ -868,6 +870,40 @@ GC_INNER void GC_push_all_stacks(void)
|
||||
hi = p->altstack + p->altstack_size;
|
||||
# endif
|
||||
/* FIXME: Need to scan the normal stack too, but how ? */
|
||||
/* FIXME: Assume stack grows down */
|
||||
+ } else {
|
||||
+#ifdef HAVE_PTHREAD_ATTR_GET_NP
|
||||
+ if (!pthread_attr_init(&pattr)
|
||||
+ || !pthread_attr_get_np(p->id, &pattr))
|
||||
+#else /* HAVE_PTHREAD_GETATTR_NP */
|
||||
+ if (pthread_getattr_np(p->id, &pattr))
|
||||
+#endif
|
||||
+ {
|
||||
+ #ifdef HAVE_PTHREAD_ATTR_GET_NP
|
||||
+ if (pthread_attr_init(&pattr) != 0) {
|
||||
+ ABORT("GC_push_all_stacks: pthread_attr_init failed!");
|
||||
+ }
|
||||
+ if (pthread_attr_get_np(p->id, &pattr) != 0) {
|
||||
+ ABORT("GC_push_all_stacks: pthread_attr_get_np failed!");
|
||||
+ }
|
||||
+ #else
|
||||
+ if (pthread_getattr_np(p->id, &pattr)) {
|
||||
+ ABORT("GC_push_all_stacks: pthread_getattr_np failed!");
|
||||
+ }
|
||||
+ #endif
|
||||
+ if (pthread_attr_getstacksize(&pattr, &stack_limit)) {
|
||||
+ ABORT("GC_push_all_stacks: pthread_attr_getstacksize failed!");
|
||||
+ }
|
||||
@@ -95,5 +50,5 @@ index b5d71e62..aed7b0bf 100644
|
||||
+ #error "STACK_GROWS_UP not supported in boost_coroutine2 (as of june 2021), so we don't support it in Nix."
|
||||
+ #endif
|
||||
}
|
||||
GC_push_all_stack_sections(lo, hi, traced_stack_sect);
|
||||
# ifdef STACK_GROWS_UP
|
||||
# ifdef STACKPTR_CORRECTOR_AVAILABLE
|
||||
if (GC_sp_corrector != 0)
|
||||
|
||||
@@ -1,12 +0,0 @@
|
||||
diff --git a/include/gc_allocator.h b/include/gc_allocator.h
|
||||
index 597c7f13..587286be 100644
|
||||
--- a/include/gc_allocator.h
|
||||
+++ b/include/gc_allocator.h
|
||||
@@ -312,6 +312,7 @@ public:
|
||||
|
||||
template<>
|
||||
class traceable_allocator<void> {
|
||||
+public:
|
||||
typedef size_t size_type;
|
||||
typedef ptrdiff_t difference_type;
|
||||
typedef void* pointer;
|
||||
+8
-1
@@ -1,11 +1,13 @@
|
||||
---
|
||||
synopsis: Clang build timing analysis
|
||||
cls: 587
|
||||
category: Development
|
||||
---
|
||||
|
||||
We now have Clang build profiling available, which generates Chrome
|
||||
tracing files for each compilation unit. To enable it, run `meson configure
|
||||
build -Dprofile-build=enabled` then rerun the compilation.
|
||||
build -Dprofile-build=enabled` in a Clang stdenv (`nix develop
|
||||
.#native-clangStdenvPackages`) then rerun the compilation.
|
||||
|
||||
If you want to make the build go faster, do a clang build with meson, then run
|
||||
`maintainers/buildtime_report.sh build`, then contemplate how to improve the
|
||||
@@ -13,3 +15,8 @@ build time.
|
||||
|
||||
You can also look at individual object files' traces in
|
||||
<https://ui.perfetto.dev>.
|
||||
|
||||
See [the wiki page][improving-build-times-wiki] for more details on how to do
|
||||
this.
|
||||
|
||||
[improving-build-times-wiki]: https://wiki.lix.systems/link/8#bkmrk-page-title
|
||||
@@ -1,5 +1,8 @@
|
||||
---
|
||||
synopsis: Show all FOD errors with `nix build --keep-going`
|
||||
credits: [ma27]
|
||||
category: Improvements
|
||||
cls: [1108]
|
||||
---
|
||||
|
||||
`nix build --keep-going` now behaves consistently with `nix-build --keep-going`. This means
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
---
|
||||
synopsis: Lix is built with meson
|
||||
# and many more
|
||||
cls: [580, 627, 628, 707, 711, 712, 719]
|
||||
credits: [Qyriad, horrors, jade, 9999years, winter]
|
||||
category: Packaging
|
||||
---
|
||||
|
||||
Lix is built exclusively with the meson build system thanks to a huge team-wide
|
||||
effort, and the legacy `make`/`autoconf` based build system has been removed
|
||||
altogether. This improves maintainability of Lix, enables things like saving
|
||||
20% of compile times with precompiled headers, and generally makes the build
|
||||
less able to produce obscure incremental compilation bugs.
|
||||
|
||||
Non-Nix-based downstream packaging needs rewriting accordingly.
|
||||
@@ -137,7 +137,7 @@ You can also build Lix for one of the [supported platforms](#platforms).
|
||||
|
||||
Lix can be built for various platforms, as specified in [`flake.nix`]:
|
||||
|
||||
[`flake.nix`]: https://github.com/nixos/nix/blob/master/flake.nix
|
||||
[`flake.nix`]: https://git.lix.systems/lix-project/lix/src/branch/main/flake.nix
|
||||
|
||||
- `x86_64-linux`
|
||||
- `x86_64-darwin`
|
||||
|
||||
@@ -12,44 +12,51 @@ The unit tests are defined using the [googletest] and [rapidcheck] frameworks.
|
||||
> An example of some files, demonstrating much of what is described below
|
||||
>
|
||||
> ```
|
||||
> src
|
||||
> ├── libexpr
|
||||
> │ ├── value/context.hh
|
||||
> │ ├── value/context.cc
|
||||
> │ │
|
||||
> │ …
|
||||
> └── tests
|
||||
> │ ├── value/context.hh
|
||||
> │ ├── value/context.cc
|
||||
> │ │
|
||||
> │ …
|
||||
> │
|
||||
> ├── unit-test-data
|
||||
> │ ├── libstore
|
||||
> │ │ ├── worker-protocol/content-address.bin
|
||||
> │ │ …
|
||||
> │ …
|
||||
> …
|
||||
> ├── src
|
||||
> │ ├── libexpr
|
||||
> │ │ ├── …
|
||||
> │ │ ├── value
|
||||
> │ │ │ ├── context.cc
|
||||
> │ │ │ └── context.hh
|
||||
> │ … …
|
||||
> ├── tests
|
||||
> │ …
|
||||
> │ └── unit
|
||||
> │ ├── libcmd
|
||||
> │ │ └── args.cc
|
||||
> │ ├── libexpr
|
||||
> │ │ ├── …
|
||||
> │ │ └── value
|
||||
> │ │ ├── context.cc
|
||||
> │ │ └── print.cc
|
||||
> │ ├── libexpr-support
|
||||
> │ │ └── tests
|
||||
> │ │ ├── libexpr.hh
|
||||
> │ │ └── value
|
||||
> │ │ ├── context.cc
|
||||
> │ │ └── context.hh
|
||||
> │ ├── libstore
|
||||
> │ │ ├── common-protocol.cc
|
||||
> │ │ ├── data
|
||||
> │ │ │ ├── libstore
|
||||
> │ │ │ │ ├── common-protocol
|
||||
> │ │ │ │ │ ├── content-address.bin
|
||||
> │ │ │ │ │ ├── drv-output.bin
|
||||
> … … … … … …
|
||||
> ```
|
||||
|
||||
<!-- FIXME(Lix): this might get renamed to liblixexpr, etc? -->
|
||||
The unit tests for each Lix library (`liblixexpr`, `liblixstore`, etc..) live inside a directory `src/${library_shortname}/tests` within the directory for the library (`src/${library_shortname}`).
|
||||
|
||||
The unit tests for each Lix library (`libnixexpr`, `libnixstore`, etc..) live inside a directory `src/${library_shortname}/tests` within the directory for the library (`src/${library_shortname}`).
|
||||
|
||||
The data is in `unit-test-data`, with one subdir per library, with the same name as where the code goes.
|
||||
For example, `libnixstore` code is in `src/libstore`, and its test data is in `unit-test-data/libstore`.
|
||||
The path to the `unit-test-data` directory is passed to the unit test executable with the environment variable `_NIX_TEST_UNIT_DATA`.
|
||||
|
||||
> **Note**
|
||||
> Due to the way googletest works, downstream unit test executables will actually include and re-run upstream library tests.
|
||||
> Therefore it is important that the same value for `_NIX_TEST_UNIT_DATA` be used with the tests for each library.
|
||||
> That is why we have the test data nested within a single `unit-test-data` directory.
|
||||
The data is in `tests/unit/LIBNAME/data/LIBNAME`, with one subdir per library, with the same name as where the code goes.
|
||||
For example, `liblixstore` code is in `src/libstore`, and its test data is in `tests/unit/libstore/data/libstore`.
|
||||
The path to the unit test data directory is passed to the unit test executable with the environment variable `_NIX_TEST_UNIT_DATA`.
|
||||
|
||||
### Running tests
|
||||
|
||||
You can run the whole testsuite with `just test` (see justfile for exact invocation of meson), and if you want to run just one test suite, use `just test --suite installcheck functional-init` where `installcheck` is the name of the test suite in this case and `functional-init` is the name of the test.
|
||||
|
||||
To get a list of tests, use `meson test -C build --list`.
|
||||
To get a list of tests, use `meson test -C build --list` (or `just test --list` for short).
|
||||
|
||||
For `installcheck` specifically, first run `just install` before running the test suite (this is due to meson limitations that don't let us put a dependency on installing before doing the test).
|
||||
|
||||
@@ -74,20 +81,27 @@ See [below](#characterization-testing-1) for a broader discussion of characteriz
|
||||
Like with the functional characterization, `_NIX_TEST_ACCEPT=1` is also used.
|
||||
For example:
|
||||
```shell-session
|
||||
$ _NIX_TEST_ACCEPT=1 make libstore-tests-exe_RUN
|
||||
$ _NIX_TEST_ACCEPT=1 just test --suite check libstore-unit-tests
|
||||
...
|
||||
[ SKIPPED ] WorkerProtoTest.string_read
|
||||
[ SKIPPED ] WorkerProtoTest.string_write
|
||||
[ SKIPPED ] WorkerProtoTest.storePath_read
|
||||
[ SKIPPED ] WorkerProtoTest.storePath_write
|
||||
../tests/unit/libstore/common-protocol.cc:27: Skipped
|
||||
Cannot read golden master because another test is also updating it
|
||||
|
||||
../tests/unit/libstore/common-protocol.cc:62: Skipped
|
||||
Updating golden master
|
||||
|
||||
../tests/unit/libstore/common-protocol.cc:27: Skipped
|
||||
Cannot read golden master because another test is also updating it
|
||||
|
||||
../tests/unit/libstore/common-protocol.cc:62: Skipped
|
||||
Updating golden master
|
||||
...
|
||||
```
|
||||
will regenerate the "golden master" expected result for the `libnixstore` characterization tests.
|
||||
will regenerate the "golden master" expected result for the `liblixstore` characterization tests.
|
||||
The characterization tests will mark themselves "skipped" since they regenerated the expected result instead of actually testing anything.
|
||||
|
||||
## Functional tests
|
||||
|
||||
The functional tests reside under the `tests/functional` directory and are listed in `tests/functional/local.mk`.
|
||||
The functional tests reside under the `tests/functional` directory and are listed in `tests/functional/meson.build`.
|
||||
Each test is a bash script.
|
||||
|
||||
### Running the whole test suite
|
||||
@@ -185,7 +199,7 @@ edit it like so:
|
||||
```
|
||||
|
||||
<div class="warning">
|
||||
FIXME(meson): the command here may be incorrect for meson.
|
||||
FIXME(meson): the command here is incorrect for meson and this whole functionality may need rebuilding.
|
||||
</div>
|
||||
|
||||
Then, running the test with `./mk/debug-test.sh` will drop you into GDB once the script reaches that point:
|
||||
@@ -209,15 +223,11 @@ This technique is to include the exact output/behavior of a former version of Ni
|
||||
|
||||
For example, this technique is used for the language tests, to check both the printed final value if evaluation was successful, and any errors and warnings encountered.
|
||||
|
||||
<div class="warning">
|
||||
FIXME(meson): this is incorrect for meson. `_NIX_TEST_ACCEPT=1` is still valid but the test invocation needs to change.
|
||||
</div>
|
||||
|
||||
It is frequently useful to regenerate the expected output.
|
||||
To do that, rerun the failed test(s) with `_NIX_TEST_ACCEPT=1`.
|
||||
For example:
|
||||
```bash
|
||||
_NIX_TEST_ACCEPT=1 make tests/functional/lang.sh.test
|
||||
_NIX_TEST_ACCEPT=1 just test --suite installcheck -v functional-lang
|
||||
```
|
||||
|
||||
An interesting situation to document is the case when these tests are "overfitted".
|
||||
@@ -320,3 +330,135 @@ solved this need?
|
||||
~~>
|
||||
|
||||
-->
|
||||
|
||||
## Magic environment variables
|
||||
|
||||
FIXME: maybe this section should be moved elsewhere or turned partially into user docs, but I just need a complete index for now.
|
||||
I actually want to ban people calling getenv without writing documentation, and produce a comprehensive list of env-vars used by Lix and enforce it.
|
||||
|
||||
This is a non-exhaustive list of almost all environment variables, magic or not, accepted or used by various parts of the test suite as well as Lix itself.
|
||||
Please add more if you find them.
|
||||
|
||||
I looked for these in the testsuite with the following bad regexes:
|
||||
|
||||
```
|
||||
rg '(?:[^A-Za-z]|^)(_[A-Z][^-\[ }/:");$(]+)' -r '$1' --no-filename --only-matching tests | sort -u > vars.txt
|
||||
rg '\$\{?([A-Z][^-\[ }/:");]+)' -r '$1' --no-filename --only-matching tests | sort -u > vars.txt
|
||||
```
|
||||
|
||||
I grepped `src/` for `get[eE]nv\("` to find the mentions in Lix code.
|
||||
|
||||
### Used by Lix testing support code
|
||||
|
||||
- `_NIX_TEST_ACCEPT` (optional) - Writes out the result of a characterization test as the new expected value.
|
||||
**Expected value**: 1
|
||||
|
||||
- `_NIX_TEST_UNIT_DATA` - The path to the directory for the data for a given unit test suite.
|
||||
|
||||
**Expected value**: `tests/unit/libstore/data/libstore` or similar
|
||||
|
||||
|
||||
### Used by Lix
|
||||
|
||||
- `_NIX_FORCE_HTTP` - Forces file URIs to be treated as remote ones.
|
||||
|
||||
Used by `src/libfetchers/git.cc`, `src/libstore/http-binary-cache-store.cc`,
|
||||
`src/libstore/local-binary-cache-store.cc`. Seems to be for forcing Git
|
||||
clones of `git+file://` URLs, making the HTTP binary
|
||||
cache store accept `file://` URLs (presumably passing them to curl?), and
|
||||
unknown reasons for the local binary cache.
|
||||
|
||||
FIXME(jade): is this obscuring a bug in https://git.lix.systems/lix-project/lix/issues/200?
|
||||
|
||||
**Expected value**: 1
|
||||
- `NIX_ATTRS_SH_FILE`, `NIX_ATTRS_JSON_FILE` (output) - Set by Lix builders; see
|
||||
`structuredAttrs` documentation.
|
||||
- `NIX_BIN_DIR`, `NIX_STORE_DIR` (or its inconsistently-used old alias `NIX_STORE`), `NIX_DATA_DIR`,
|
||||
`NIX_LOG_DIR`, `NIX_LOG_DIR`, `NIX_STATE_DIR`, `NIX_CONF_DIR` -
|
||||
Overrides compile-time configuration of various locations used by Lix. See `src/libstore/globals.cc`.
|
||||
|
||||
**Expected value**: a directory
|
||||
- `NIX_DAEMON_SOCKET_PATH` (optional) - Overrides the daemon socket path from `$NIX_STATE_DIR/daemon-socket/socket`.
|
||||
|
||||
**Expected value**: path to a socket
|
||||
- `NIX_LOG_FD` (output) - An FD number for logs in `internal-json` format to be sent to.
|
||||
Used for, mostly, "setPhase" in nixpkgs setup.sh, but can also be creatively used to print verbose log messages from derivations.
|
||||
|
||||
**Provided value**: number corresponding to an FD in the builder
|
||||
- `NIX_PATH` - Search path for `<whatever>`. Documented elsewhere in the manual.
|
||||
|
||||
**Expected value**: `:` separated list of things that are not necessarily pointing to filesystem paths
|
||||
- `NIX_REMOTE` - The default value of the Lix setting `store`.
|
||||
|
||||
**Expected value**: "daemon", usually. Could be "auto" or any other value acceptable in `store`.
|
||||
- `NIX_BUILD_SHELL` - Documented elsewhere; the shell to invoke with `nix-shell` but not `nix develop`/`nix shell`.
|
||||
The latter ignoring it altogether seems like a bug.
|
||||
|
||||
**Expected value**: the path to an executable shell
|
||||
- `PRINT_PATH` - Undocumented. Used by `nix-prefetch-url` as an alternative form of `--print-path`. Why???
|
||||
- `_NIX_IN_TEST` - If present with any value, makes `fetchClosure` accept file URLs in addition to HTTP ones. Why is this not `_NIX_FORCE_HTTP`??
|
||||
|
||||
Not used anywhere else.
|
||||
- `NIX_ALLOW_EVAL` - Used by eval-cache tests to block evaluation if set to `0`.
|
||||
|
||||
**Expected value**: 1 or 0
|
||||
- `EDITOR` - Used by `editorFor()`, which has some extremely sketchy editor-detection code for jumping to line numbers.
|
||||
- `LISTEN_FDS` and `LISTEN_PID` - Used for systemd socket activation using the systemd socket activation protocol.
|
||||
- `NIX_PAGER` (alternatively, `PAGER`) - Used to select a pager for Lix output. Why does this not use libutil `getEnv()`?
|
||||
- `LESS` (output) - Sets the pager settings for `less` when invoked by Lix.
|
||||
- `NIX_IGNORE_SYMLINK_STORE` - When set, Lix allows the store to be a symlink. Why do we support this?
|
||||
|
||||
Apparently [someone was using it enough to fix it](https://github.com/NixOS/nix/pull/4038).
|
||||
- `NIX_SSL_CERT_FILE` (alternatively, `SSL_CERT_FILE`) - Used to set CA certificates for libcurl.
|
||||
|
||||
**Expected value**: "/etc/ssl/certs/ca-certificates.crt" or similar
|
||||
- `NIX_REMOTE_SYSTEMS` - Used to set `builders`. Can we please deprecate this?
|
||||
- `NIX_USER_CONF_FILES` - `:` separated list of config files to load before
|
||||
`/nix/nix.conf` under each of `XDG_CONFIG_DIRS`.
|
||||
- `NIX_CONFIG` - Newline separated configuration to load into Lix.
|
||||
- `NIX_GET_COMPLETIONS` - Returns completions.
|
||||
Unsure of the exact format, someone should document it; either way my shell never had any completions.
|
||||
|
||||
**Expected value**: number of completions to return.
|
||||
- `IN_SYSTEMD` - Used to switch the logging format so that systemd gets the correct log levels. I think.
|
||||
- `NIX_HELD_LOCKS` - Not used, what is this for?? We should surely remove it right after searching github?
|
||||
- `GC_INITIAL_HEAP_SIZE` - Used to set the initial heap size, processed by boehmgc.
|
||||
- `NIX_COUNT_CALLS` - Documented elsewhere; prints call counts for profiling purposes.
|
||||
- `NIX_SHOW_STATS` - Documented elsewhere; prints various evaluation statistics like function calls, gc info, and similar.
|
||||
- `NIX_SHOW_STATS_PATH` - Writes those statistics into a file at the given path instead of stdout. Undocumented.
|
||||
- `NIX_SHOW_SYMBOLS` - Dumps the symbol table into the show-stats json output.
|
||||
- `TERM` - If `dumb` or unset, disables ANSI colour output.
|
||||
- `NO_COLOR`, `NOCOLOR` - Disables ANSI colour output.
|
||||
- `_NIX_DEVELOPER_SHOW_UNKNOWN_LOCATIONS` - Highlights unknown locations in errors.
|
||||
- `NIX_PROFILE` - Selects which profile `nix-env` will operate on. Documented elsewhere.
|
||||
- `NIX_SSHOPTS` - Options passed to `ssh(1)` when using a ssh remote store.
|
||||
Incorrectly documented on `nix-copy-closure` which is *surely* not the only place they are used??
|
||||
- `_NIX_TEST_NO_LSOF` - Used on non-Linux, non-macOS platforms to disable using `lsof` when finding gc roots.
|
||||
|
||||
Since https://git.lix.systems/lix-project/lix/issues/156 was fixed, this should probably just be removed as it was a bad workaround for a macOS issue.
|
||||
- `_NIX_TEST_GC_SYNC_1` - Path to a pipe that is used to block the GC briefly to validate invariants from the test suite.
|
||||
- `_NIX_TEST_GC_SYNC_2` - Path to a pipe that is used to block the GC briefly to validate invariants from the test suite.
|
||||
- `_NIX_TEST_FREE_SPACE_FILE` - Path to a file containing a decimal number with the free space that the GC is to believe it has.
|
||||
- Various XDG vars
|
||||
- `NIX_DEBUG_SQLITE_TRACES` - Dump all sqlite queries to the log at `notice` level.
|
||||
- `_NIX_TEST_NO_SANDBOX` - Disables actually setting up the sandbox on macOS while leaving other logic the same. Unused on other platforms.
|
||||
- `_NIX_TRACE_BUILT_OUTPUTS` - Dumps all the derivation paths alongside their outputs as lines into a file of the given name.
|
||||
|
||||
### Used by the functional test framework
|
||||
|
||||
- `NIX_DAEMON_PACKAGE` - Runs the test suite against an alternate Nix daemon with the current client.
|
||||
|
||||
**Expected value**: something like `/nix/store/...-nix-2.18.2`
|
||||
- `NIX_CLIENT_PACKAGE` - Runs the test suite against an alternate Nix client with the current daemon.
|
||||
|
||||
**Expected value**: something like `/nix/store/...-nix-2.18.2`
|
||||
- `NIX_TESTS_CA_BY_DEFAULT` - Pass `__contentAddressed`, `outputHashMode` and `outputHashAlgo` to builds of some input-addressed derivations in the test suite.
|
||||
|
||||
**Expected value**: 1
|
||||
- `TEST_DATA` - Not an environment variable! This is used in repl characterization tests to refer to `tests/functional/repl_characterization/data`.
|
||||
More specifically, that path is replaced with the string `$TEST_DATA` in output for reproducibility.
|
||||
- `TEST_HOME` (output) - Set to the temporary directory that is set as `$HOME` inside the tests, underneath `$TEST_ROOT`.
|
||||
- `TEST_ROOT` (output) - Set to the temporary directory that is created for each test to mess with.
|
||||
- `_NIX_TEST_DAEMON_PID` (output) - Used to track the daemon pid to be able to kill it.
|
||||
|
||||
**Provided value**: Daemon pid as a base-10 integer, e.g. 2345
|
||||
|
||||
@@ -1,64 +1,62 @@
|
||||
# Using Lix within Docker
|
||||
|
||||
Currently the Lix project doesn't ship docker images. However, we have the infrastructure to do it, it's just not yet been done. See https://git.lix.systems/lix-project/lix/issues/252
|
||||
|
||||
<!--
|
||||
Lix is available on the following two container registries:
|
||||
- [ghcr.io/lix-project/lix](https://ghcr.io/lix-project/lix)
|
||||
- [git.lix.systems/lix-project/lix](https://git.lix.systems/lix-project/-/packages/container/lix)
|
||||
|
||||
To run the latest stable release of Lix with Docker run the following command:
|
||||
|
||||
```console
|
||||
$ docker run -ti nixos/nix
|
||||
Unable to find image 'nixos/nix:latest' locally
|
||||
latest: Pulling from nixos/nix
|
||||
5843afab3874: Pull complete
|
||||
b52bf13f109c: Pull complete
|
||||
1e2415612aa3: Pull complete
|
||||
Digest: sha256:27f6e7f60227e959ee7ece361f75d4844a40e1cc6878b6868fe30140420031ff
|
||||
Status: Downloaded newer image for nixos/nix:latest
|
||||
35ca4ada6e96:/# nix --version
|
||||
nix (Nix) 2.3.12
|
||||
35ca4ada6e96:/# exit
|
||||
~ » sudo podman run -it ghcr.io/lix-project/lix:latest
|
||||
Trying to pull ghcr.io/lix-project/lix:latest...
|
||||
|
||||
bash-5.2# nix --version
|
||||
nix (Lix, like Nix) 2.90.0
|
||||
```
|
||||
|
||||
# What is included in Lix's Docker image?
|
||||
|
||||
The official Docker image is created using `pkgs.dockerTools.buildLayeredImage`
|
||||
The official Docker image is created using [nix2container]
|
||||
(and not with `Dockerfile` as it is usual with Docker images). You can still
|
||||
base your custom Docker image on it as you would do with any other Docker
|
||||
image.
|
||||
|
||||
The Docker image is also not based on any other image and includes minimal set
|
||||
of runtime dependencies that are required to use Lix:
|
||||
[nix2container]: https://github.com/nlewo/nix2container
|
||||
|
||||
- pkgs.nix
|
||||
- pkgs.bashInteractive
|
||||
- pkgs.coreutils-full
|
||||
- pkgs.gnutar
|
||||
- pkgs.gzip
|
||||
- pkgs.gnugrep
|
||||
- pkgs.which
|
||||
- pkgs.curl
|
||||
- pkgs.less
|
||||
- pkgs.wget
|
||||
- pkgs.man
|
||||
- pkgs.cacert.out
|
||||
- pkgs.findutils
|
||||
The Docker image is also not based on any other image and includes the nixpkgs
|
||||
that Lix was built with along with a minimal set of tools in the system profile:
|
||||
|
||||
- bashInteractive
|
||||
- cacert.out
|
||||
- coreutils-full
|
||||
- curl
|
||||
- findutils
|
||||
- gitMinimal
|
||||
- gnugrep
|
||||
- gnutar
|
||||
- gzip
|
||||
- iana-etc
|
||||
- less
|
||||
- libxml2
|
||||
- lix
|
||||
- man
|
||||
- openssh
|
||||
- sqlite
|
||||
- wget
|
||||
- which
|
||||
|
||||
# Docker image with the latest development version of Lix
|
||||
|
||||
To get the latest image that was built by [Hydra](https://hydra.nixos.org) run
|
||||
the following command:
|
||||
FIXME: There are not currently images of development versions of Lix. Tracking issue: https://git.lix.systems/lix-project/lix/issues/381
|
||||
|
||||
You can build a Docker image from source yourself and copy it to either:
|
||||
|
||||
Podman: `nix run '.#dockerImage.copyTo' containers-storage:lix`
|
||||
|
||||
Docker: `nix run '.#dockerImage.copyToDockerDaemon'`
|
||||
|
||||
Then:
|
||||
|
||||
```console
|
||||
$ curl -L https://hydra.nixos.org/job/nix/master/dockerImage.x86_64-linux/latest/download/1 | docker load
|
||||
$ docker run -ti nix:2.5pre20211105
|
||||
$ docker run -ti lix
|
||||
```
|
||||
|
||||
You can also build a Docker image from source yourself:
|
||||
|
||||
```console
|
||||
$ nix build ./\#hydraJobs.dockerImage.x86_64-linux
|
||||
$ docker load -i ./result/image.tar.gz
|
||||
$ docker run -ti nix:2.5pre20211105
|
||||
```
|
||||
-->
|
||||
|
||||
@@ -1,32 +1,121 @@
|
||||
# Multi-User Mode
|
||||
|
||||
To allow a Nix store to be shared safely among multiple users, it is
|
||||
important that users are not able to run builders that modify the Nix
|
||||
store or database in arbitrary ways, or that interfere with builds
|
||||
started by other users. If they could do so, they could install a Trojan
|
||||
horse in some package and compromise the accounts of other users.
|
||||
To allow a Nix store to be shared safely among multiple users, it is important that users cannot meaningfully influence the execution of derivation builds such that they could inject malicious code into them without changing their (either input- or output- addressed) hash.
|
||||
If they could do so, they could install a Trojan horse in some package and compromise the accounts of other users.
|
||||
|
||||
To prevent this, the Nix store and database are owned by some privileged
|
||||
user (usually `root`) and builders are executed under special user
|
||||
accounts (usually named `nixbld1`, `nixbld2`, etc.). When a unprivileged
|
||||
user runs a Nix command, actions that operate on the Nix store (such as
|
||||
builds) are forwarded to a *Nix daemon* running under the owner of the
|
||||
Nix store/database that performs the operation.
|
||||
To prevent this, the Nix store and database are owned by some privileged user (usually `root`) and builders are executed under unprivileged system user accounts (usually named `nixbld1`, `nixbld2`, etc.).
|
||||
When an unprivileged user runs a Nix command, actions that operate on the Nix store (such as builds) are forwarded to a *Nix daemon* running under the owner of the Nix store/database that performs the operation.
|
||||
|
||||
> **Note**
|
||||
>
|
||||
> Multi-user mode has one important limitation: only root and a set of
|
||||
> trusted users specified in `nix.conf` can specify arbitrary binary
|
||||
> caches. So while unprivileged users may install packages from
|
||||
> arbitrary Nix expressions, they may not get pre-built binaries.
|
||||
The buried lede in the above sentence is that *currently*, even in multi-user mode using a daemon, if executing as the user that owns the store, Lix directly manipulates the store unless `--store daemon` is specified.
|
||||
[We intend to change this in the future][multi-user-should-not-be-root].
|
||||
|
||||
## Setting up the build users
|
||||
<div class="warning">
|
||||
The Lix team considers the goal of the sandbox to be primarily for preventing reproducibility mistakes, and does not consider multi-user mode to be a strong security boundary between users.
|
||||
|
||||
Do not evaluate or build untrusted, potentially-malicious, Nix language code on machines that you care deeply about maintaining user isolation on.
|
||||
|
||||
Although we would consider any sandbox escapes to be serious security bugs and we intend to fix them, we are not confident enough in the daemon's security to call the daemon a security boundary.
|
||||
</div>
|
||||
|
||||
[multi-user-should-not-be-root]: https://git.lix.systems/lix-project/lix/issues/18
|
||||
|
||||
## Trust model
|
||||
|
||||
There are two categories of users of the Lix daemon: trusted users and untrusted users.
|
||||
The Lix daemon only allows connections from users that are either trusted users, or are specified in, or are members of groups specified in, [`allowed-users`](../command-ref/conf-file.md#conf-allowed-users) in `nix.conf`.
|
||||
Trusted users are users and users of groups specified in [`trusted-users`](../command-ref/conf-file.md#conf-trusted-users) in `nix.conf`.
|
||||
|
||||
All users of the Lix daemon may do the following to bring things into the Nix store:
|
||||
|
||||
- Users may load derivations and output-addressed files into the store with `nix-store --add` or through Nix language code.
|
||||
- Users may locally build derivations, either of the output-addressed or input-addressed variety, creating output paths.
|
||||
|
||||
Note that [fixed-output derivations only consider name and hash](https://github.com/NixOS/nix/issues/969), so it is possible to write a fixed-output derivation for something important with a bogus hash and have it resolve to something else already built in the store.
|
||||
|
||||
On systems with `sandbox` enabled (default on Linux; [not *yet* on macOS][sandbox-enable-macos]), derivations are either:
|
||||
- Input-addressed, so they are run in the sandbox with no network access, with the following exceptions:
|
||||
|
||||
- The (poorly named, since it is not *just* about chroot) property `__noChroot` is set on the derivation and `sandbox` is set to `relaxed`.
|
||||
- On macOS, the derivation property `__darwinAllowLocalNetworking` allows network access to localhost from input-addressed derivations regardless of the `sandbox` setting value. This property exists with such semantics because macOS has no network namespace equivalent to isolate individual processes' localhost networking.
|
||||
- Output-addressed, so they are run with network access but their result must match an expected hash.
|
||||
|
||||
Trusted users may set any setting, including `sandbox = false`, so the sandbox state can be different at runtime from what is described in `nix.conf` for builds invoked with such settings.
|
||||
- Users may copy appropriately-signed derivation outputs into the store.
|
||||
|
||||
By default, any paths *copied into a store* (such as by substitution) must have signatures from [`trusted-public-keys`](../command-ref/conf-file.md#conf-trusted-public-keys) unless they are [output-addressed](../glossary.md#gloss-output-addressed-store-object).
|
||||
|
||||
Unsigned paths may be copied into a store if [`require-sigs`](../command-ref/conf-file.md#conf-require-sigs) is disabled in the daemon's configuration (not default), or if the client is a trusted user and passed `--no-check-sigs` to `nix copy`.
|
||||
- Users may request that the daemon substitutes appropriately-signed derivation outputs from a binary cache in the daemon's [`substituters`](../command-ref/conf-file.md#conf-substituters) list.
|
||||
|
||||
Untrusted clients may also specify additional values for `substituters` (via e.g. `--extra-substituters` on a Nix command) that are listed in [`trusted-substituters`](../command-ref/conf-file.md#conf-trusted-substituters).
|
||||
|
||||
A client could in principle substitute such paths itself then copy them to the daemon (see clause above) if they are appropriately signed but are *not* from a trusted substituter, however this is not implemented in the current Lix client to our knowledge, at the time of writing.
|
||||
This probably means that `trusted-substituters` is a redundant setting except insofar as such substitution would have to be done on the client rather than as root on the daemon; and it is highly defensible to not allow random usage of our HTTP client running as root.
|
||||
|
||||
[sandbox-enable-macos]: https://git.lix.systems/lix-project/lix/issues/386
|
||||
|
||||
### The Lix daemon as a security non-boundary
|
||||
|
||||
The Lix team and wider community does not consider the Lix daemon to be a *security boundary* against malicious Nix language code.
|
||||
|
||||
Although we do our best to make it secure, we do not recommend sharing a Lix daemon with potentially malicious users.
|
||||
That means that public continuous integration (CI) builds of untrusted Nix code should not share builders with CI that writes into a cache used by trusted infrastructure.
|
||||
|
||||
For example, [hydra.nixos.org], which is the builder for [cache.nixos.org], does not execute untrusted Nix language code; a separate system, [ofborg] is used for CI of nixpkgs pull requests.
|
||||
The build output of pull request CI is never pushed to [cache.nixos.org], and those systems are considered entirely untrusted.
|
||||
|
||||
This is because, among other things, the Lix sandbox is *more* susceptible to kernel exploits than Docker, which, unlike Lix, blocks nested user namespaces via `seccomp` in its default policy, and there have been many kernel bugs only exposed to unprivileged users via user namespaces allowing otherwise-root-only system calls.
|
||||
In general, the Lix sandbox is set up to be relatively unrestricted while maintaining its goals of building useful, reproducible software; security is not its primary goal.
|
||||
|
||||
The Lix sandbox is a custom *non-rootless* Linux container implementation that has not been audited to nearly the same degree as Docker and similar systems.
|
||||
Also, the Lix daemon is a complex and historied C++ executable running as root with very little privilege separation.
|
||||
All of this means that a security hole in the Lix daemon gives immediate root access.
|
||||
Systems like Docker (especially non-rootless Docker) should *themselves* probably not be used in a multi-tenant manner with mutually distrusting tenants, but the Lix daemon *especially* should not be used as such as of this writing.
|
||||
|
||||
The primary purpose of the sandbox is to strongly encourage packages to be reproducible, a goal which it is generally quite successful at.
|
||||
|
||||
[hydra.nixos.org]: https://hydra.nixos.org
|
||||
[ofborg]: https://github.com/NixOS/ofborg
|
||||
[cache.nixos.org]: https://cache.nixos.org
|
||||
|
||||
### Trusted users
|
||||
|
||||
Trusted users are permitted to set any setting and bypass security restrictions on the daemon.
|
||||
They are currently in widespread use for a couple of reasons such as remote builds (which we [intend to fix](https://git.lix.systems/lix-project/lix/issues/171)).
|
||||
|
||||
Trusted users are effectively root on Nix daemons running as root (the default configuration) for *at least* the following reasons, and should be thus thought of as equivalent to passwordless sudo.
|
||||
This is not a comprehensive list.
|
||||
|
||||
- They may copy an unsigned malicious built output into the store for `systemd` or anything else that will run as root, then when the system is upgraded, that path will be used from the local store rather than substituted.
|
||||
- They may set the following settings that are commands the daemon will run as root:
|
||||
- `build-hook`
|
||||
- `diff-hook`
|
||||
- `pre-build-hook`
|
||||
- `post-build-hook`
|
||||
- They may set `build-users-group`.
|
||||
|
||||
In particular, they may set it to empty string, which runs builds as root with respect to the rest of the system (!!).
|
||||
We, too, [think that is absurd and intend to not accept such a configuration](https://git.lix.systems/lix-project/lix/issues/242).
|
||||
It is then simply an exercise to the reader to find a daemon that does `SCM_CREDENTIALS` over a `unix(7)` socket and lets you run commands as root, and mount it into the sandbox with `extra-sandbox-paths`.
|
||||
|
||||
At the very least, the Lix daemon itself (since `root` is a trusted user by default) and probably `systemd` qualify for this.
|
||||
- They may set the `builders` list, which will have ssh run as root.
|
||||
We aren't sure if there is a way to abuse this for command execution but it's plausible.
|
||||
|
||||
Note that setting `accept-flake-config` allows arbitrary Nix flakes to set Nix settings in the `nixConfig` stanza.
|
||||
Do not set this setting or pass `--accept-flake-config` while executing untrusted Nix language code as a trusted user for the reasons above!
|
||||
|
||||
## Build users
|
||||
|
||||
The *build users* are the special UIDs under which builds are performed.
|
||||
They should all be members of the *build users group* `nixbld`. This
|
||||
group should have no other members. The build users should not be
|
||||
members of any other group. On Linux, you can create the group and users
|
||||
as follows:
|
||||
A build user is selected for a build by looking in the group specified by [`build-users-group`](../command-ref/conf-file.md#conf-build-users-group), by default, `nixbld`, then a member of that group not currently executing a build is selected for the build.
|
||||
The build users should not be members of any other group.
|
||||
|
||||
There can never be more concurrent builds than the number of build users, unless using [`auto-allocate-uids`](../command-ref/conf-file.md#conf-auto-allocate-uids) ([tracking issue][auto-allocate-uids-issue]).
|
||||
|
||||
[auto-allocate-uids-issue]: https://git.lix.systems/lix-project/lix/issues/387
|
||||
|
||||
If, for some reason, you need to create such users manually, the following command will create 10 build users on Linux:
|
||||
|
||||
```console
|
||||
$ groupadd -r nixbld
|
||||
@@ -35,43 +124,12 @@ $ for n in $(seq 1 10); do useradd -c "Nix build user $n" \
|
||||
nixbld$n; done
|
||||
```
|
||||
|
||||
This creates 10 build users. There can never be more concurrent builds
|
||||
than the number of build users, so you may want to increase this if you
|
||||
expect to do many builds at the same time.
|
||||
|
||||
## Running the daemon
|
||||
|
||||
The [Nix daemon](../command-ref/nix-daemon.md) should be started as
|
||||
follows (as `root`):
|
||||
The [Nix daemon](../command-ref/nix-daemon.md) can be started manually as follows (as `root`):
|
||||
|
||||
```console
|
||||
$ nix-daemon
|
||||
# nix-daemon
|
||||
```
|
||||
|
||||
You’ll want to put that line somewhere in your system’s boot scripts.
|
||||
|
||||
To let unprivileged users use the daemon, they should set the
|
||||
[`NIX_REMOTE` environment variable](../command-ref/env-common.md) to
|
||||
`daemon`. So you should put a line like
|
||||
|
||||
```console
|
||||
export NIX_REMOTE=daemon
|
||||
```
|
||||
|
||||
into the users’ login scripts.
|
||||
|
||||
## Restricting access
|
||||
|
||||
To limit which users can perform Nix operations, you can use the
|
||||
permissions on the directory `/nix/var/nix/daemon-socket`. For instance,
|
||||
if you want to restrict the use of Nix to the members of a group called
|
||||
`nix-users`, do
|
||||
|
||||
```console
|
||||
$ chgrp nix-users /nix/var/nix/daemon-socket
|
||||
$ chmod ug=rwx,o= /nix/var/nix/daemon-socket
|
||||
```
|
||||
|
||||
This way, users who are not in the `nix-users` group cannot connect to
|
||||
the Unix domain socket `/nix/var/nix/daemon-socket/socket`, so they
|
||||
cannot perform Nix operations.
|
||||
In standard installations of Lix, the daemon is started by a `systemd` unit (Linux) or `launchd` service (macOS).
|
||||
|
||||
@@ -7,9 +7,8 @@ management operations. All other users can then use the installed
|
||||
packages, but they cannot perform package management operations
|
||||
themselves.
|
||||
|
||||
Alternatively, you can configure Lix in “multi-user mode”. In this
|
||||
model, all users can perform package management operations — for
|
||||
instance, every user can install software without requiring root
|
||||
privileges. Lix ensures that this is secure. For instance, it’s not
|
||||
possible for one user to overwrite a package used by another user with a
|
||||
Trojan horse.
|
||||
Alternatively, you can configure Lix in “multi-user mode”. In this model, all users can perform package management operations — for instance, every user can install software for themselves without requiring root privileges.
|
||||
Lix does its best to ensure that this is secure.
|
||||
For instance, it would be considered a serious security bug for one untrusted user to be able to overwrite a package used by another user with a Trojan horse.
|
||||
|
||||
Nevertheless, the Lix team does not consider multi-user mode a strong security boundary, and does not recommend running untrusted user-supplied Nix language code on privileged machines, even if it is secure to the best of our knowledge at any moment in time.
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
# Obtaining the Source
|
||||
|
||||
The most recent sources of Nix can be obtained from its [Git
|
||||
repository](https://github.com/NixOS/nix). For example, the following
|
||||
The most recent sources of Lix can be obtained from its [Git
|
||||
repository](https://git.lix.systems/lix-project/lix). For example, the following
|
||||
command will check out the latest revision into a directory called
|
||||
`nix`:
|
||||
|
||||
```console
|
||||
$ git clone https://github.com/NixOS/nix
|
||||
$ git clone https://git.lix.systems/lix-project/lix
|
||||
```
|
||||
|
||||
Likewise, specific releases can be obtained from the
|
||||
[tags](https://github.com/NixOS/nix/tags) of the repository.
|
||||
[tags](https://git.lix.systems/lix-project/lix/tags) of the repository.
|
||||
|
||||
@@ -7,3 +7,14 @@ These constants are built into the Nix language evaluator:
|
||||
{{#include @generated@/language/builtin-constants.md}}
|
||||
|
||||
</dl>
|
||||
|
||||
## Things which might be mistaken for constants
|
||||
|
||||
<dl>
|
||||
<dt><code>__curPos</code></dt>
|
||||
<dd>
|
||||
|
||||
This is not a constant but a [context-dependent keyword](@docroot@/language/constructs.md#keywords-__curPos)
|
||||
|
||||
</dd>
|
||||
</dl>
|
||||
|
||||
@@ -380,3 +380,70 @@ let a = 1; in let a = 2; in let a = 3; in let a = 4; in ...
|
||||
|
||||
Comments can be single-line, started with a `#` character, or
|
||||
inline/multi-line, enclosed within `/* ... */`.
|
||||
|
||||
## Context-dependent keywords
|
||||
|
||||
<dl>
|
||||
<dt id="keywords-__curPos">
|
||||
<a href="#keywords-__curPos"><code>__curPos</code></a>
|
||||
</dt>
|
||||
<dd>
|
||||
|
||||
A quasi-constant which will be replaced with an attribute set describing
|
||||
the location where `__curPos` was used, with attributes `file`, `line`,
|
||||
and `column`. For example, `import ./file.nix` will result in
|
||||
|
||||
```nix
|
||||
{
|
||||
column = 1;
|
||||
file = "/path/to/some/file.nix";
|
||||
line = 1;
|
||||
}
|
||||
```
|
||||
|
||||
assuming `file.nix` contains nothing but `__curPos`.
|
||||
|
||||
In context without a source file (such as `nix-repl`), it will always
|
||||
be replaced with `null`:
|
||||
|
||||
```nix-repl
|
||||
nix-repl> __curPos
|
||||
null
|
||||
```
|
||||
|
||||
While it may vaguely look like a builtin, this is a very different beast
|
||||
that is handled directly by the parser. It thus cannot be shadowed,
|
||||
bound to a different name, and is also not available under
|
||||
[`builtins`](@docroot@/language/builtin-constants.md#builtins-builtins).
|
||||
|
||||
```nix-repl
|
||||
nix-repl> let __curPos = "no"; in __curPos
|
||||
null
|
||||
```
|
||||
|
||||
Despite this `__curPos`, much like `or`, may still be used as an identifier,
|
||||
it is only treated specially when it appears as an unqualified name:
|
||||
|
||||
```nix-repl
|
||||
nix-repl> { __curPos = 1; }.__curPos
|
||||
1
|
||||
```
|
||||
|
||||
</dd>
|
||||
|
||||
<dt id="keywords-or">
|
||||
<a href="#keywords-or"><code>or</code></a>
|
||||
</dt>
|
||||
<dd>
|
||||
|
||||
`or` is used in [Attribute selection](@docroot@/language/operators.html#attribute-selection),
|
||||
where it is a keyword.
|
||||
|
||||
However, it is not a keyword in some other contexts, and can be used as
|
||||
a binding name in attribute sets, let-bindings, non-initial function
|
||||
application position, and as a label in attribute paths.
|
||||
|
||||
Its use as anything other than a keyword is discouraged.
|
||||
|
||||
</dd>
|
||||
</dl>
|
||||
|
||||
@@ -22,7 +22,7 @@ to subsequent chapters.
|
||||
$ curl -sSf -L https://install.lix.systems/lix | sh -s -- install
|
||||
```
|
||||
|
||||
For systems that **already have Nix installed**, such as NixOS systems, read our [install page](https://lix.systems/install)
|
||||
For systems that **already have a Nix implementation installed**, such as NixOS systems, read our [install page](https://lix.systems/install)
|
||||
|
||||
The install script will use `sudo`, so make sure you have sufficient rights.
|
||||
|
||||
|
||||
@@ -229,7 +229,7 @@ This release has the following new features:
|
||||
<https://cache.nixos.org/> or <ssh://machine>. The following store
|
||||
types are supported:
|
||||
|
||||
- `LocalStore` (stori URI `local` or an absolute path) and the
|
||||
- `LocalStore` (store URI `local` or an absolute path) and the
|
||||
misnamed `RemoteStore` (`daemon`) provide access to a local Nix
|
||||
store, the latter via the Nix daemon. You can use `auto` or the
|
||||
empty string to auto-select a local or daemon store depending on
|
||||
|
||||
+151
-65
@@ -1,7 +1,10 @@
|
||||
{
|
||||
pkgs ? import <nixpkgs> { },
|
||||
# Git commit ID, if available
|
||||
lixRevision ? null,
|
||||
nix2container,
|
||||
lib ? pkgs.lib,
|
||||
name ? "nix",
|
||||
name ? "lix",
|
||||
tag ? "latest",
|
||||
bundleNixpkgs ? true,
|
||||
channelName ? "nixpkgs",
|
||||
@@ -12,26 +15,51 @@
|
||||
flake-registry ? null,
|
||||
}:
|
||||
let
|
||||
layerContents = with pkgs; [
|
||||
# pulls in glibc and openssl, about 60MB
|
||||
{ contents = [ coreutils-full ]; }
|
||||
# some stuff that is low in the closure graph and small ish, mostly to make
|
||||
# incremental lix updates cheaper
|
||||
{
|
||||
contents = [
|
||||
curl
|
||||
libxml2
|
||||
sqlite
|
||||
];
|
||||
}
|
||||
# 50MB of git
|
||||
{ contents = [ gitMinimal ]; }
|
||||
# 144MB of nixpkgs
|
||||
{
|
||||
contents = [ channel ];
|
||||
inProfile = false;
|
||||
}
|
||||
];
|
||||
|
||||
# These packages are left to be auto layered by nix2container, since it is
|
||||
# less critical that they get layered sensibly and they tend to not be deps
|
||||
# of anything in particular
|
||||
autoLayered = with pkgs; [
|
||||
bashInteractive
|
||||
gnutar
|
||||
gzip
|
||||
gnugrep
|
||||
which
|
||||
less
|
||||
wget
|
||||
man
|
||||
cacert.out
|
||||
findutils
|
||||
iana-etc
|
||||
openssh
|
||||
nix
|
||||
];
|
||||
|
||||
defaultPkgs =
|
||||
with pkgs;
|
||||
[
|
||||
nix
|
||||
bashInteractive
|
||||
coreutils-full
|
||||
gnutar
|
||||
gzip
|
||||
gnugrep
|
||||
which
|
||||
curl
|
||||
less
|
||||
wget
|
||||
man
|
||||
cacert.out
|
||||
findutils
|
||||
iana-etc
|
||||
git
|
||||
openssh
|
||||
]
|
||||
lib.lists.flatten (
|
||||
map (x: if !(x ? inProfile) || x.inProfile then x.contents else [ ]) layerContents
|
||||
)
|
||||
++ autoLayered
|
||||
++ extraPkgs;
|
||||
|
||||
users =
|
||||
@@ -139,16 +167,17 @@ let
|
||||
))
|
||||
+ "\n";
|
||||
|
||||
nixpkgs = pkgs.path;
|
||||
channel = pkgs.runCommand "channel-nixpkgs" { } ''
|
||||
mkdir $out
|
||||
${lib.optionalString bundleNixpkgs ''
|
||||
ln -s ${nixpkgs} $out/nixpkgs
|
||||
echo "[]" > $out/manifest.nix
|
||||
''}
|
||||
'';
|
||||
|
||||
baseSystem =
|
||||
let
|
||||
nixpkgs = pkgs.path;
|
||||
channel = pkgs.runCommand "channel-nixos" { inherit bundleNixpkgs; } ''
|
||||
mkdir $out
|
||||
if [ "$bundleNixpkgs" ]; then
|
||||
ln -s ${nixpkgs} $out/nixpkgs
|
||||
echo "[]" > $out/manifest.nix
|
||||
fi
|
||||
'';
|
||||
rootEnv = pkgs.buildPackages.buildEnv {
|
||||
name = "root-profile-env";
|
||||
paths = defaultPkgs;
|
||||
@@ -187,7 +216,7 @@ let
|
||||
profile = pkgs.buildPackages.runCommand "user-environment" { } ''
|
||||
mkdir $out
|
||||
cp -a ${rootEnv}/* $out/
|
||||
ln -s ${manifest} $out/manifest.nix
|
||||
ln -sf ${manifest} $out/manifest.nix
|
||||
'';
|
||||
flake-registry-path =
|
||||
if (flake-registry == null) then
|
||||
@@ -236,6 +265,7 @@ let
|
||||
ln -s /nix/var/nix/profiles/share $out/usr/
|
||||
|
||||
mkdir -p $out/nix/var/nix/gcroots
|
||||
ln -s /nix/var/nix/profiles $out/nix/var/nix/gcroots/profiles
|
||||
|
||||
mkdir $out/tmp
|
||||
|
||||
@@ -248,14 +278,14 @@ let
|
||||
mkdir -p $out/nix/var/nix/profiles/per-user/root
|
||||
|
||||
ln -s ${profile} $out/nix/var/nix/profiles/default-1-link
|
||||
ln -s $out/nix/var/nix/profiles/default-1-link $out/nix/var/nix/profiles/default
|
||||
ln -s /nix/var/nix/profiles/default-1-link $out/nix/var/nix/profiles/default
|
||||
ln -s /nix/var/nix/profiles/default $out/root/.nix-profile
|
||||
|
||||
ln -s ${channel} $out/nix/var/nix/profiles/per-user/root/channels-1-link
|
||||
ln -s $out/nix/var/nix/profiles/per-user/root/channels-1-link $out/nix/var/nix/profiles/per-user/root/channels
|
||||
ln -s /nix/var/nix/profiles/per-user/root/channels-1-link $out/nix/var/nix/profiles/per-user/root/channels
|
||||
|
||||
mkdir -p $out/root/.nix-defexpr
|
||||
ln -s $out/nix/var/nix/profiles/per-user/root/channels $out/root/.nix-defexpr/channels
|
||||
ln -s /nix/var/nix/profiles/per-user/root/channels $out/root/.nix-defexpr/channels
|
||||
echo "${channelURL} ${channelName}" > $out/root/.nix-channels
|
||||
|
||||
mkdir -p $out/bin $out/usr/bin
|
||||
@@ -273,43 +303,99 @@ let
|
||||
ln -s $globalFlakeRegistryPath $out/nix/var/nix/gcroots/auto/$rootName
|
||||
'')
|
||||
);
|
||||
in
|
||||
pkgs.dockerTools.buildLayeredImageWithNixDb {
|
||||
|
||||
inherit name tag maxLayers;
|
||||
layers = builtins.foldl' (
|
||||
layersList: el:
|
||||
let
|
||||
layer = nix2container.buildLayer {
|
||||
deps = el.contents;
|
||||
layers = layersList;
|
||||
};
|
||||
in
|
||||
layersList ++ [ layer ]
|
||||
) [ ] layerContents;
|
||||
|
||||
contents = [ baseSystem ];
|
||||
image = nix2container.buildImage {
|
||||
|
||||
extraCommands = ''
|
||||
rm -rf nix-support
|
||||
ln -s /nix/var/nix/profiles nix/var/nix/gcroots/profiles
|
||||
'';
|
||||
fakeRootCommands = ''
|
||||
chmod 1777 tmp
|
||||
chmod 1777 var/tmp
|
||||
'';
|
||||
inherit name tag maxLayers;
|
||||
|
||||
config = {
|
||||
Cmd = [ "/root/.nix-profile/bin/bash" ];
|
||||
Env = [
|
||||
"USER=root"
|
||||
"PATH=${
|
||||
lib.concatStringsSep ":" [
|
||||
"/root/.nix-profile/bin"
|
||||
"/nix/var/nix/profiles/default/bin"
|
||||
"/nix/var/nix/profiles/default/sbin"
|
||||
]
|
||||
}"
|
||||
"MANPATH=${
|
||||
lib.concatStringsSep ":" [
|
||||
"/root/.nix-profile/share/man"
|
||||
"/nix/var/nix/profiles/default/share/man"
|
||||
]
|
||||
}"
|
||||
"SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
|
||||
"GIT_SSL_CAINFO=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
|
||||
"NIX_SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
|
||||
"NIX_PATH=/nix/var/nix/profiles/per-user/root/channels:/root/.nix-defexpr/channels"
|
||||
inherit layers;
|
||||
|
||||
copyToRoot = [ baseSystem ];
|
||||
|
||||
initializeNixDatabase = true;
|
||||
|
||||
perms = [
|
||||
{
|
||||
path = baseSystem;
|
||||
regex = "(/var)?/tmp";
|
||||
mode = "1777";
|
||||
}
|
||||
];
|
||||
|
||||
config = {
|
||||
Cmd = [ "/root/.nix-profile/bin/bash" ];
|
||||
Env = [
|
||||
"USER=root"
|
||||
"PATH=${
|
||||
lib.concatStringsSep ":" [
|
||||
"/root/.nix-profile/bin"
|
||||
"/nix/var/nix/profiles/default/bin"
|
||||
"/nix/var/nix/profiles/default/sbin"
|
||||
]
|
||||
}"
|
||||
"MANPATH=${
|
||||
lib.concatStringsSep ":" [
|
||||
"/root/.nix-profile/share/man"
|
||||
"/nix/var/nix/profiles/default/share/man"
|
||||
]
|
||||
}"
|
||||
"SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
|
||||
"GIT_SSL_CAINFO=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
|
||||
"NIX_SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
|
||||
"NIX_PATH=/nix/var/nix/profiles/per-user/root/channels:/root/.nix-defexpr/channels"
|
||||
];
|
||||
|
||||
Labels = {
|
||||
"org.opencontainers.image.title" = "Lix";
|
||||
"org.opencontainers.image.source" = "https://git.lix.systems/lix-project/lix";
|
||||
"org.opencontainers.image.vendor" = "Lix project";
|
||||
"org.opencontainers.image.version" = pkgs.nix.version;
|
||||
"org.opencontainers.image.description" = "Minimal Lix container image, with some batteries included.";
|
||||
} // lib.optionalAttrs (lixRevision != null) { "org.opencontainers.image.revision" = lixRevision; };
|
||||
};
|
||||
|
||||
meta = {
|
||||
description = "Docker image for Lix. This is built with nix2container; see that project's README for details";
|
||||
longDescription = ''
|
||||
Docker image for Lix, built with nix2container.
|
||||
To copy it to your docker daemon, nix run .#dockerImage.copyToDockerDaemon
|
||||
To copy it to podman, nix run .#dockerImage.copyTo containers-storage:lix
|
||||
'';
|
||||
};
|
||||
};
|
||||
in
|
||||
image
|
||||
// {
|
||||
# We don't ship the tarball as the default output because it is a strange thing to want imo
|
||||
tarball =
|
||||
pkgs.buildPackages.runCommand "docker-image-tarball-${pkgs.nix.version}"
|
||||
{
|
||||
nativeBuildInputs = [ pkgs.buildPackages.bubblewrap ];
|
||||
meta.description = "Docker image tarball with Lix for ${pkgs.system}";
|
||||
}
|
||||
''
|
||||
mkdir -p $out/nix-support
|
||||
image=$out/image.tar
|
||||
# bwrap for foolish temp dir selection code that forces /var/tmp:
|
||||
# https://github.com/containers/skopeo.git/blob/60ee543f7f7c242f46cc3a7541d9ac8ab1c89168/vendor/github.com/containers/image/v5/internal/tmpdir/tmpdir.go#L15-L18
|
||||
mkdir -p $TMPDIR/fake-var/tmp
|
||||
args=(--unshare-user --bind "$TMPDIR/fake-var" /var)
|
||||
for dir in /*; do
|
||||
args+=(--dev-bind "/$dir" "/$dir")
|
||||
done
|
||||
bwrap ''${args[@]} -- ${lib.getExe image.copyTo} docker-archive:$image
|
||||
gzip $image
|
||||
echo "file binary-dist $image" >> $out/nix-support/hydra-build-products
|
||||
'';
|
||||
}
|
||||
|
||||
Generated
+21
-4
@@ -16,18 +16,34 @@
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"nix2container": {
|
||||
"flake": false,
|
||||
"locked": {
|
||||
"lastModified": 1712990762,
|
||||
"narHash": "sha256-hO9W3w7NcnYeX8u8cleHiSpK2YJo7ecarFTUlbybl7k=",
|
||||
"owner": "nlewo",
|
||||
"repo": "nix2container",
|
||||
"rev": "20aad300c925639d5d6cbe30013c8357ce9f2a2e",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "nlewo",
|
||||
"repo": "nix2container",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"nixpkgs": {
|
||||
"locked": {
|
||||
"lastModified": 1715123187,
|
||||
"narHash": "sha256-0czuu757t53lK6uWeo1a5/jJbCd9t4sOtLDFpts60DM=",
|
||||
"lastModified": 1718111384,
|
||||
"narHash": "sha256-7tSst0S5FOmcgvNtfy6cjZX5w8CabCVAfAeCkhY4OVg=",
|
||||
"owner": "NixOS",
|
||||
"repo": "nixpkgs",
|
||||
"rev": "0c592f9a288bdf764b6f24c757277c0e49757a46",
|
||||
"rev": "a508a44af0c1b1b57785c34d8b54783536273eeb",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "NixOS",
|
||||
"ref": "nixos-23.11-small",
|
||||
"ref": "nixos-24.05-small",
|
||||
"repo": "nixpkgs",
|
||||
"type": "github"
|
||||
}
|
||||
@@ -67,6 +83,7 @@
|
||||
"root": {
|
||||
"inputs": {
|
||||
"flake-compat": "flake-compat",
|
||||
"nix2container": "nix2container",
|
||||
"nixpkgs": "nixpkgs",
|
||||
"nixpkgs-regression": "nixpkgs-regression",
|
||||
"pre-commit-hooks": "pre-commit-hooks"
|
||||
|
||||
@@ -2,12 +2,16 @@
|
||||
description = "The purely functional package manager";
|
||||
|
||||
inputs = {
|
||||
nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.11-small";
|
||||
nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.05-small";
|
||||
nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
|
||||
pre-commit-hooks = {
|
||||
url = "github:cachix/git-hooks.nix";
|
||||
flake = false;
|
||||
};
|
||||
nix2container = {
|
||||
url = "github:nlewo/nix2container";
|
||||
flake = false;
|
||||
};
|
||||
flake-compat = {
|
||||
url = "github:edolstra/flake-compat";
|
||||
flake = false;
|
||||
@@ -20,6 +24,7 @@
|
||||
nixpkgs,
|
||||
nixpkgs-regression,
|
||||
pre-commit-hooks,
|
||||
nix2container,
|
||||
flake-compat,
|
||||
}:
|
||||
|
||||
@@ -54,12 +59,11 @@
|
||||
(Run `touch .nocontribmsg` to hide this message.)
|
||||
'';
|
||||
|
||||
officialRelease = false;
|
||||
officialRelease = true;
|
||||
|
||||
# Set to true to build the release notes for the next release.
|
||||
buildUnreleasedNotes = true;
|
||||
|
||||
version = lib.fileContents ./.version + versionSuffix;
|
||||
versionSuffix =
|
||||
if officialRelease then
|
||||
""
|
||||
@@ -83,10 +87,11 @@
|
||||
crossSystems = [
|
||||
"armv6l-linux"
|
||||
"armv7l-linux"
|
||||
# FIXME: doesn't evaluate, plausibly fixed in >=24.05, so recheck when
|
||||
# we update to 24.05
|
||||
# "x86_64-freebsd13"
|
||||
"x86_64-netbsd"
|
||||
"riscv64-linux"
|
||||
# FIXME: still broken in 24.05: fails to build rustc(??) due to missing -lstdc++ dep
|
||||
# "x86_64-freebsd"
|
||||
# FIXME: broken dev shell due to python
|
||||
# "x86_64-netbsd"
|
||||
];
|
||||
|
||||
stdenvs = [
|
||||
@@ -131,13 +136,11 @@
|
||||
{
|
||||
system = crossSystem;
|
||||
}
|
||||
// lib.optionalAttrs (crossSystem == "x86_64-freebsd13") { useLLVM = true; };
|
||||
// lib.optionalAttrs (crossSystem == "x86_64-freebsd") { useLLVM = true; };
|
||||
overlays = [
|
||||
(overlayFor (p: p.${stdenv}))
|
||||
(final: prev: { nixfmt = final.callPackage ./nix-support/nixfmt.nix { }; })
|
||||
];
|
||||
|
||||
config.permittedInsecurePackages = [ "nix-2.13.6" ];
|
||||
};
|
||||
stdenvs = forAllStdenvs (make-pkgs null);
|
||||
native = stdenvs.stdenvPackages;
|
||||
@@ -149,9 +152,6 @@
|
||||
}
|
||||
);
|
||||
|
||||
binaryTarball =
|
||||
nix: pkgs: pkgs.callPackage ./nix-support/binary-tarball.nix { inherit nix version; };
|
||||
|
||||
overlayFor =
|
||||
getStdenv: final: prev:
|
||||
let
|
||||
@@ -164,7 +164,6 @@
|
||||
nixUnstable = prev.nixUnstable;
|
||||
|
||||
check-headers = final.buildPackages.callPackage ./maintainers/check-headers.nix { };
|
||||
clangbuildanalyzer = final.buildPackages.callPackage ./misc/clangbuildanalyzer.nix { };
|
||||
|
||||
default-busybox-sandbox-shell = final.busybox.override {
|
||||
useMusl = true;
|
||||
@@ -191,7 +190,7 @@
|
||||
};
|
||||
|
||||
nix = final.callPackage ./package.nix {
|
||||
inherit versionSuffix;
|
||||
inherit versionSuffix officialRelease;
|
||||
stdenv = currentStdenv;
|
||||
busybox-sandbox-shell = final.busybox-sandbox-shell or final.default-busybox-sandbox-shell;
|
||||
};
|
||||
@@ -209,10 +208,14 @@
|
||||
overlays.default = overlayFor (p: p.stdenv);
|
||||
|
||||
hydraJobs = {
|
||||
|
||||
# Binary package for various platforms.
|
||||
build = forAllSystems (system: self.packages.${system}.nix);
|
||||
|
||||
devShell = forAllSystems (system: {
|
||||
default = self.devShells.${system}.default;
|
||||
clang = self.devShells.${system}.native-clangStdenvPackages;
|
||||
});
|
||||
|
||||
rl-next = forAllSystems (
|
||||
system:
|
||||
let
|
||||
@@ -227,20 +230,17 @@
|
||||
in
|
||||
{
|
||||
user = rl-next-check "rl-next" ./doc/manual/rl-next;
|
||||
dev = rl-next-check "rl-next-dev" ./doc/manual/rl-next-dev;
|
||||
}
|
||||
);
|
||||
|
||||
# Perl bindings for various platforms.
|
||||
perlBindings = forAllSystems (system: nixpkgsFor.${system}.native.nix.perl-bindings);
|
||||
perlBindings = forAllSystems (system: nixpkgsFor.${system}.native.nix.passthru.perl-bindings);
|
||||
|
||||
# Binary tarball for various platforms, containing a Nix store
|
||||
# with the closure of 'nix' package.
|
||||
binaryTarball = forAllSystems (
|
||||
system: binaryTarball nixpkgsFor.${system}.native.nix nixpkgsFor.${system}.native
|
||||
);
|
||||
binaryTarball = forAllSystems (system: nixpkgsFor.${system}.native.nix.passthru.binaryTarball);
|
||||
|
||||
# docker image with Nix inside
|
||||
# docker image with Lix inside
|
||||
dockerImage = lib.genAttrs linux64BitSystems (system: self.packages.${system}.dockerImage);
|
||||
|
||||
# API docs for Nix's unstable internal C++ interfaces.
|
||||
@@ -281,9 +281,20 @@
|
||||
|
||||
nixpkgsLibTests = forAllSystems (
|
||||
system:
|
||||
import (nixpkgs + "/lib/tests/release.nix") {
|
||||
let
|
||||
inherit (self.packages.${system}) nix;
|
||||
pkgs = nixpkgsFor.${system}.native;
|
||||
nixVersions = [ self.packages.${system}.nix ];
|
||||
testWithNix = import (nixpkgs + "/lib/tests/test-with-nix.nix") { inherit pkgs lib nix; };
|
||||
in
|
||||
pkgs.symlinkJoin {
|
||||
name = "nixpkgs-lib-tests";
|
||||
paths =
|
||||
[ testWithNix ]
|
||||
# FIXME: This is disabled on darwin due to a nixpkgs bug https://github.com/NixOS/nixpkgs/issues/319147
|
||||
# After that is fixed, it should be restored to use lib/tests/release.nix as before, rather than this reimplementation.
|
||||
++ lib.optionals pkgs.stdenv.isLinux [
|
||||
(import (nixpkgs + "/pkgs/test/release") { inherit pkgs lib nix; })
|
||||
];
|
||||
}
|
||||
);
|
||||
};
|
||||
@@ -300,16 +311,23 @@
|
||||
);
|
||||
};
|
||||
|
||||
release-jobs = import ./releng/release-jobs.nix {
|
||||
inherit (self) hydraJobs;
|
||||
pkgs = nixpkgsFor.x86_64-linux.native;
|
||||
};
|
||||
|
||||
# NOTE *do not* add fresh derivations to checks, always add them to
|
||||
# hydraJobs first (so CI will pick them up) and only link them here
|
||||
checks = forAvailableSystems (
|
||||
system:
|
||||
{
|
||||
# devShells and packages already get checked by nix flake check, so
|
||||
# this is just jobs that are special
|
||||
|
||||
binaryTarball = self.hydraJobs.binaryTarball.${system};
|
||||
perlBindings = self.hydraJobs.perlBindings.${system};
|
||||
nixpkgsLibTests = self.hydraJobs.tests.nixpkgsLibTests.${system};
|
||||
rl-next = self.hydraJobs.rl-next.${system}.user;
|
||||
rl-next-dev = self.hydraJobs.rl-next.${system}.dev;
|
||||
# Will be empty attr set on i686-linux, and filtered out by forAvailableSystems.
|
||||
pre-commit = self.hydraJobs.pre-commit.${system};
|
||||
}
|
||||
@@ -330,19 +348,13 @@
|
||||
dockerImage =
|
||||
let
|
||||
pkgs = nixpkgsFor.${system}.native;
|
||||
image = import ./docker.nix {
|
||||
inherit pkgs;
|
||||
tag = version;
|
||||
};
|
||||
nix2container' = import nix2container { inherit pkgs system; };
|
||||
in
|
||||
pkgs.runCommand "docker-image-tarball-${version}"
|
||||
{ meta.description = "Docker image with Nix for ${system}"; }
|
||||
''
|
||||
mkdir -p $out/nix-support
|
||||
image=$out/image.tar.gz
|
||||
ln -s ${image} $image
|
||||
echo "file binary-dist $image" >> $out/nix-support/hydra-build-products
|
||||
'';
|
||||
import ./docker.nix {
|
||||
inherit pkgs;
|
||||
nix2container = nix2container'.nix2container;
|
||||
tag = pkgs.nix.version;
|
||||
};
|
||||
}
|
||||
// builtins.listToAttrs (
|
||||
map (crossSystem: {
|
||||
@@ -365,7 +377,7 @@
|
||||
pkgs: stdenv:
|
||||
let
|
||||
nix = pkgs.callPackage ./package.nix {
|
||||
inherit stdenv versionSuffix;
|
||||
inherit stdenv officialRelease versionSuffix;
|
||||
busybox-sandbox-shell = pkgs.busybox-sandbox-shell or pkgs.default-busybox-sandbox;
|
||||
internalApiDocs = true;
|
||||
};
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
from collections import defaultdict
|
||||
import frontmatter
|
||||
import sys
|
||||
import pathlib
|
||||
import textwrap
|
||||
from typing import Any, Tuple
|
||||
@@ -27,6 +26,7 @@ CATEGORIES = [
|
||||
'Improvements',
|
||||
'Fixes',
|
||||
'Packaging',
|
||||
'Development',
|
||||
'Miscellany',
|
||||
]
|
||||
|
||||
@@ -143,7 +143,7 @@ def run_on_dir(author_info: AuthorInfoDB, d):
|
||||
|
||||
for category in CATEGORIES:
|
||||
if entries[category]:
|
||||
print('\n#', category)
|
||||
print('\n##', category)
|
||||
do_category(author_info, entries[category])
|
||||
|
||||
def main():
|
||||
|
||||
+33
-25
@@ -17,6 +17,19 @@
|
||||
#
|
||||
# Finally, src/nix/meson.build defines the Nix command itself, relying on all prior meson files.
|
||||
#
|
||||
# libstore, libexpr, and libfetchers have some special handling to make static builds work.
|
||||
# Their use static constructors for dynamic registration of primops, store backends, etc
|
||||
# gets borked during static link. We can't simply wholesale apply `link_whole :` either,
|
||||
# because these libraries get linked multiple times since Lix's components are transitively
|
||||
# dependent. So instead, each of those libraries have two dependency objects:
|
||||
# liblix{store,expr,fetchers,util} and liblix{store,expr,fetchers,util}_mstatic ("maybe static").
|
||||
# The _mstatic versions should be used in the `dependencies :` arguments to ALL EXECUTABLES
|
||||
# but executables ONLY. When we are not building statically (default_library != 'static'),
|
||||
# they are equivalent. When we are building statically, the _mstatic version will be
|
||||
# `link_whole :` rather than `link_with :`.
|
||||
# FIXME: This hack should be removed when https://git.lix.systems/lix-project/lix/issues/359
|
||||
# is fixed.
|
||||
#
|
||||
# Unit tests are setup in tests/unit/meson.build, under the test suite "check".
|
||||
#
|
||||
# Functional tests are a bit more complicated. Generally they're defined in
|
||||
@@ -26,7 +39,7 @@
|
||||
# in the build directory.
|
||||
|
||||
project('lix', 'cpp',
|
||||
version : run_command('bash', '-c', 'echo -n $(cat ./.version)$VERSION_SUFFIX', check : true).stdout().strip(),
|
||||
version : run_command('bash', '-c', 'echo -n $(jq -r .version < ./version.json)$VERSION_SUFFIX', check : true).stdout().strip(),
|
||||
default_options : [
|
||||
'cpp_std=c++2a',
|
||||
# TODO(Qyriad): increase the warning level
|
||||
@@ -79,6 +92,8 @@ if not fs.is_absolute(sysconfdir)
|
||||
sysconfdir = '/' / sysconfdir
|
||||
endif
|
||||
|
||||
is_static = get_option('default_library') == 'static'
|
||||
|
||||
# All of this has to go before the rest of the dependency checking,
|
||||
# so that internal-api-docs can be built with -Denable-build=false
|
||||
|
||||
@@ -114,6 +129,20 @@ endif
|
||||
|
||||
cxx = meson.get_compiler('cpp')
|
||||
|
||||
|
||||
# clangd breaks when GCC is using precompiled headers lmao
|
||||
# https://git.lix.systems/lix-project/lix/issues/374
|
||||
should_pch = get_option('enable-pch-std')
|
||||
summary('PCH C++ stdlib', should_pch, bool_yn : true)
|
||||
if should_pch
|
||||
# Unlike basically everything else that takes a file, Meson requires the arguments to
|
||||
# cpp_pch : to be strings and doesn't accept files(). So absolute path it is.
|
||||
cpp_pch = [meson.project_source_root() / 'src/pch/precompiled-headers.hh']
|
||||
else
|
||||
cpp_pch = []
|
||||
endif
|
||||
|
||||
|
||||
# Translate some historical and Mesony CPU names to Lixy CPU names.
|
||||
# FIXME(Qyriad): the 32-bit x86 code is not tested right now, because cross compilation for Lix
|
||||
# to those architectures is currently broken for other reasons, namely:
|
||||
@@ -153,23 +182,18 @@ elif is_linux
|
||||
# Clang sanitizers on Linux.
|
||||
# FIXME(Qyriad): is that true?
|
||||
endif
|
||||
deps = [ ]
|
||||
configdata = { }
|
||||
|
||||
#
|
||||
# Dependencies
|
||||
#
|
||||
|
||||
boehm = dependency('bdw-gc', required : get_option('gc'))
|
||||
if boehm.found()
|
||||
deps += boehm
|
||||
endif
|
||||
boehm = dependency('bdw-gc', required : get_option('gc'), version : '>=8.2.6')
|
||||
configdata += {
|
||||
'HAVE_BOEHMGC': boehm.found().to_int(),
|
||||
}
|
||||
|
||||
boost = dependency('boost', required : true, modules : ['context', 'coroutine', 'container'])
|
||||
deps += boost
|
||||
|
||||
# cpuid only makes sense on x86_64
|
||||
cpuid_required = is_x64 ? get_option('cpuid') : false
|
||||
@@ -177,7 +201,6 @@ cpuid = dependency('libcpuid', 'cpuid', required : cpuid_required)
|
||||
configdata += {
|
||||
'HAVE_LIBCPUID': cpuid.found().to_int(),
|
||||
}
|
||||
deps += cpuid
|
||||
|
||||
# seccomp only makes sense on Linux
|
||||
seccomp_required = is_linux ? get_option('seccomp-sandboxing') : false
|
||||
@@ -190,20 +213,17 @@ configdata += {
|
||||
}
|
||||
|
||||
libarchive = dependency('libarchive', required : true)
|
||||
deps += libarchive
|
||||
|
||||
brotli = [
|
||||
dependency('libbrotlicommon', required : true),
|
||||
dependency('libbrotlidec', required : true),
|
||||
dependency('libbrotlienc', required : true),
|
||||
]
|
||||
deps += brotli
|
||||
|
||||
openssl = dependency('libcrypto', 'openssl', required : true)
|
||||
deps += openssl
|
||||
|
||||
aws_sdk = dependency('aws-cpp-sdk-core', required : false)
|
||||
aws_sdk_transfer = dependency('aws-cpp-sdk-transfer', required : aws_sdk.found())
|
||||
aws_sdk_transfer = dependency('aws-cpp-sdk-transfer', required : aws_sdk.found(), fallback : ['aws_sdk', 'aws_cpp_sdk_transfer_dep'])
|
||||
if aws_sdk.found()
|
||||
# The AWS pkg-config adds -std=c++11.
|
||||
# https://github.com/aws/aws-sdk-cpp/issues/2673
|
||||
@@ -214,7 +234,6 @@ if aws_sdk.found()
|
||||
links : true,
|
||||
sources : true,
|
||||
)
|
||||
deps += aws_sdk
|
||||
s = aws_sdk.version().split('.')
|
||||
configdata += {
|
||||
'AWS_VERSION_MAJOR': s[0].to_int(),
|
||||
@@ -230,7 +249,7 @@ if aws_sdk.found()
|
||||
)
|
||||
endif
|
||||
|
||||
aws_s3 = dependency('aws-cpp-sdk-s3', required : false)
|
||||
aws_s3 = dependency('aws-cpp-sdk-s3', required : aws_sdk.found(), fallback : ['aws_sdk', 'aws_cpp_sdk_s3_dep'])
|
||||
if aws_s3.found()
|
||||
# The AWS pkg-config adds -std=c++11.
|
||||
# https://github.com/aws/aws-sdk-cpp/issues/2673
|
||||
@@ -241,7 +260,6 @@ if aws_s3.found()
|
||||
links : true,
|
||||
sources : true,
|
||||
)
|
||||
deps += aws_s3
|
||||
endif
|
||||
|
||||
configdata += {
|
||||
@@ -249,26 +267,20 @@ configdata += {
|
||||
}
|
||||
|
||||
sqlite = dependency('sqlite3', 'sqlite', version : '>=3.6.19', required : true)
|
||||
deps += sqlite
|
||||
|
||||
sodium = dependency('libsodium', 'sodium', required : true)
|
||||
deps += sodium
|
||||
|
||||
curl = dependency('libcurl', 'curl', required : true)
|
||||
deps += curl
|
||||
|
||||
editline = dependency('libeditline', 'editline', version : '>=1.14', required : true)
|
||||
deps += editline
|
||||
|
||||
lowdown = dependency('lowdown', version : '>=0.9.0', required : true)
|
||||
deps += lowdown
|
||||
|
||||
# HACK(Qyriad): rapidcheck's pkg-config doesn't include the libs lol
|
||||
# Note: technically we 'check' for rapidcheck twice, for the internal-api-docs handling above,
|
||||
# but Meson will cache the result of the first one, and the required : arguments are different.
|
||||
rapidcheck_meson = dependency('rapidcheck', required : enable_tests)
|
||||
rapidcheck = declare_dependency(dependencies : rapidcheck_meson, link_args : ['-lrapidcheck'])
|
||||
deps += rapidcheck
|
||||
|
||||
gtest = [
|
||||
dependency('gtest', required : enable_tests),
|
||||
@@ -276,13 +288,10 @@ gtest = [
|
||||
dependency('gmock', required : enable_tests),
|
||||
dependency('gmock_main', required : enable_tests),
|
||||
]
|
||||
deps += gtest
|
||||
|
||||
toml11 = dependency('toml11', version : '>=3.7.0', required : true, method : 'cmake')
|
||||
deps += toml11
|
||||
|
||||
nlohmann_json = dependency('nlohmann_json', required : true)
|
||||
deps += nlohmann_json
|
||||
|
||||
# lix-doc is a Rust project provided via buildInputs and unfortunately doesn't have any way to be detected.
|
||||
# Just declare it manually to resolve this.
|
||||
@@ -290,7 +299,6 @@ deps += nlohmann_json
|
||||
# FIXME: build this with meson in the future after we drop Make (with which we
|
||||
# *absolutely* are not going to make it work)
|
||||
lix_doc = declare_dependency(link_args : [ '-llix_doc' ])
|
||||
deps += lix_doc
|
||||
|
||||
#
|
||||
# Build-time tools
|
||||
|
||||
@@ -64,3 +64,7 @@ option('internal-api-docs', type : 'feature', value : 'auto',
|
||||
option('profile-dir', type : 'string', value : 'etc/profile.d',
|
||||
description : 'the path to install shell profile files',
|
||||
)
|
||||
|
||||
option('enable-pch-std', type : 'boolean', value : true,
|
||||
description : 'whether to use precompiled headers for C++\'s standard library (breaks clangd if you\'re using GCC)',
|
||||
)
|
||||
|
||||
@@ -1,33 +0,0 @@
|
||||
# Upstreaming here, can be deleted once it's upstreamed:
|
||||
# https://github.com/NixOS/nixpkgs/pull/297102
|
||||
{
|
||||
stdenv,
|
||||
lib,
|
||||
cmake,
|
||||
fetchFromGitHub,
|
||||
}:
|
||||
stdenv.mkDerivation (finalAttrs: {
|
||||
pname = "clangbuildanalyzer";
|
||||
version = "1.5.0";
|
||||
|
||||
src = fetchFromGitHub {
|
||||
owner = "aras-p";
|
||||
repo = "ClangBuildAnalyzer";
|
||||
rev = "v${finalAttrs.version}";
|
||||
sha256 = "sha256-kmgdk634zM0W0OoRoP/RzepArSipa5bNqdVgdZO9gxo=";
|
||||
};
|
||||
|
||||
nativeBuildInputs = [ cmake ];
|
||||
|
||||
meta = {
|
||||
description = "Tool for analyzing Clang's -ftrace-time files";
|
||||
homepage = "https://github.com/aras-p/ClangBuildAnalyzer";
|
||||
maintainers = with lib.maintainers; [ lf- ];
|
||||
license = lib.licenses.unlicense;
|
||||
platforms = lib.platforms.unix;
|
||||
# `long long int` != `size_t`
|
||||
# There's no convenient lib.platforms.32bit or anything, but it's easy enough to do ourselves.
|
||||
badPlatforms = lib.filter (plat: (lib.systems.elaborate plat).is32bit) lib.platforms.all;
|
||||
mainProgram = "ClangBuildAnalyzer";
|
||||
};
|
||||
})
|
||||
+6
-5
@@ -63,7 +63,7 @@ pre-commit-run {
|
||||
files = ''^doc/manual/(change-authors\.yml|rl-next(-dev)?)'';
|
||||
pass_filenames = false;
|
||||
entry = ''
|
||||
${lib.getExe pkgs.build-release-notes} --change-authors doc/manual/change-authors.yml doc/manual/rl-next doc/manual/rl-next-dev
|
||||
${lib.getExe pkgs.build-release-notes} --change-authors doc/manual/change-authors.yml doc/manual/rl-next
|
||||
'';
|
||||
};
|
||||
change-authors-sorted = {
|
||||
@@ -87,11 +87,12 @@ pre-commit-run {
|
||||
"file"
|
||||
"header"
|
||||
];
|
||||
# generated files; these will never actually be seen by this
|
||||
# check, and are left here as documentation
|
||||
excludes = [
|
||||
"(parser|lexer)-tab\\.hh$"
|
||||
"\\.gen\\.hh$"
|
||||
''^src/pch/.*$''
|
||||
# generated files; these will never actually be seen by this
|
||||
# check, and are left here as documentation
|
||||
''(parser|lexer)-tab\.hh$''
|
||||
''\.gen\.hh$''
|
||||
];
|
||||
entry = lib.getExe pkgs.check-headers;
|
||||
};
|
||||
|
||||
@@ -3,7 +3,6 @@
|
||||
cacert,
|
||||
nix,
|
||||
system,
|
||||
version,
|
||||
}:
|
||||
let
|
||||
installerClosureInfo = buildPackages.closureInfo {
|
||||
@@ -13,12 +12,12 @@ let
|
||||
];
|
||||
};
|
||||
|
||||
meta.description = "Distribution-independent Nix bootstrap binaries for ${system}";
|
||||
meta.description = "Distribution-independent Lix bootstrap binaries for ${system}";
|
||||
in
|
||||
buildPackages.runCommand "nix-binary-tarball-${version}" { inherit meta; } ''
|
||||
buildPackages.runCommand "lix-binary-tarball-${nix.version}" { inherit meta; } ''
|
||||
cp ${installerClosureInfo}/registration $TMPDIR/reginfo
|
||||
|
||||
dir=nix-${version}-${system}
|
||||
dir=lix-${nix.version}-${system}
|
||||
fn=$out/$dir.tar.xz
|
||||
mkdir -p $out/nix-support
|
||||
echo "file binary-dist $fn" >> $out/nix-support/hydra-build-products
|
||||
|
||||
+56
-23
@@ -50,7 +50,7 @@
|
||||
# internal fork of nix-doc providing :doc in the repl
|
||||
lix-doc ? __forDefaults.lix-doc,
|
||||
|
||||
pname ? "nix",
|
||||
pname ? "lix",
|
||||
versionSuffix ? "",
|
||||
officialRelease ? false,
|
||||
# Set to true to build the release notes for the next release.
|
||||
@@ -69,8 +69,6 @@
|
||||
# `boehmgc-nix` then this will almost certainly have duplicate patches, which means
|
||||
# the patches won't apply and we'll get a build failure.
|
||||
./boehmgc-coroutine-sp-fallback.diff
|
||||
# https://github.com/ivmai/bdwgc/pull/586
|
||||
./boehmgc-traceable_allocator-public.diff
|
||||
];
|
||||
};
|
||||
|
||||
@@ -87,7 +85,8 @@ let
|
||||
inherit (lib) fileset;
|
||||
inherit (stdenv) hostPlatform buildPlatform;
|
||||
|
||||
version = lib.fileContents ./.version + versionSuffix;
|
||||
versionJson = builtins.fromJSON (builtins.readFile ./version.json);
|
||||
version = versionJson.version + versionSuffix;
|
||||
|
||||
aws-sdk-cpp-nix = aws-sdk-cpp.override {
|
||||
apis = [
|
||||
@@ -137,13 +136,14 @@ let
|
||||
# that would interfere with repo semantics.
|
||||
baseFiles = fileset.fileFilter (f: f.name != ".gitignore") ./.;
|
||||
|
||||
configureFiles = fileset.unions [ ./.version ];
|
||||
configureFiles = fileset.unions [ ./version.json ];
|
||||
|
||||
topLevelBuildFiles = fileset.unions ([
|
||||
./meson.build
|
||||
./meson.options
|
||||
./meson
|
||||
./scripts/meson.build
|
||||
./subprojects
|
||||
]);
|
||||
|
||||
functionalTestFiles = fileset.unions [
|
||||
@@ -168,7 +168,6 @@ stdenv.mkDerivation (finalAttrs: {
|
||||
./boehmgc-coroutine-sp-fallback.diff
|
||||
./doc
|
||||
./misc
|
||||
./precompiled-headers.h
|
||||
./src
|
||||
./COPYING
|
||||
]
|
||||
@@ -259,7 +258,7 @@ stdenv.mkDerivation (finalAttrs: {
|
||||
++ lib.optional internalApiDocs rapidcheck
|
||||
++ lib.optional hostPlatform.isx86_64 libcpuid
|
||||
# There have been issues building these dependencies
|
||||
++ lib.optional (hostPlatform == buildPlatform) aws-sdk-cpp-nix
|
||||
++ lib.optional (hostPlatform.canExecute buildPlatform) aws-sdk-cpp-nix
|
||||
++ lib.optionals (finalAttrs.dontBuild) maybePropagatedInputs;
|
||||
|
||||
checkInputs = [
|
||||
@@ -377,25 +376,31 @@ stdenv.mkDerivation (finalAttrs: {
|
||||
platforms = lib.platforms.unix;
|
||||
};
|
||||
|
||||
passthru.perl-bindings = pkgs.callPackage ./perl { inherit fileset stdenv; };
|
||||
|
||||
# Export the patched version of boehmgc.
|
||||
# flake.nix exports that into its overlay.
|
||||
passthru = {
|
||||
inherit (__forDefaults) boehmgc-nix editline-lix build-release-notes;
|
||||
|
||||
inherit officialRelease;
|
||||
|
||||
# The collection of dependency logic for this derivation is complicated enough that
|
||||
# it's easier to parameterize the devShell off an already called package.nix.
|
||||
mkDevShell =
|
||||
{
|
||||
mkShell,
|
||||
just,
|
||||
nixfmt,
|
||||
glibcLocales,
|
||||
pre-commit-checks,
|
||||
|
||||
bashInteractive,
|
||||
clang-tools,
|
||||
llvmPackages,
|
||||
clangbuildanalyzer,
|
||||
glibcLocales,
|
||||
just,
|
||||
llvmPackages,
|
||||
nixfmt,
|
||||
skopeo,
|
||||
xonsh,
|
||||
|
||||
# Lix specific packages
|
||||
pre-commit-checks,
|
||||
contribNotice,
|
||||
}:
|
||||
let
|
||||
@@ -403,10 +408,17 @@ stdenv.mkDerivation (finalAttrs: {
|
||||
# Required to make non-NixOS Linux not complain about missing locale files during configure in a dev shell
|
||||
LOCALE_ARCHIVE = "${lib.getLib pkgs.glibcLocales}/lib/locale/locale-archive";
|
||||
};
|
||||
# for some reason that seems accidental and was changed in
|
||||
# NixOS 24.05-pre, clang-tools is pinned to LLVM 14 when
|
||||
# default LLVM is newer.
|
||||
clang-tools_llvm = clang-tools.override { inherit llvmPackages; };
|
||||
|
||||
pythonPackages = (
|
||||
p: [
|
||||
p.yapf
|
||||
p.python-frontmatter
|
||||
p.requests
|
||||
p.xdg-base-dirs
|
||||
(p.toPythonModule xonsh.passthru.unwrapped)
|
||||
]
|
||||
);
|
||||
pythonEnv = python3.withPackages pythonPackages;
|
||||
|
||||
# pkgs.mkShell uses pkgs.stdenv by default, regardless of inputsFrom.
|
||||
actualMkShell = mkShell.override { inherit stdenv; };
|
||||
@@ -424,13 +436,25 @@ stdenv.mkDerivation (finalAttrs: {
|
||||
# For Meson to find Boost.
|
||||
env = finalAttrs.env;
|
||||
|
||||
# I guess this is necessary because mesonFlags to mkDerivation doesn't propagate in inputsFrom,
|
||||
# which only propagates stuff set in hooks? idk.
|
||||
inherit (finalAttrs) mesonFlags;
|
||||
mesonFlags =
|
||||
# I guess this is necessary because mesonFlags to mkDerivation doesn't propagate in inputsFrom,
|
||||
# which only propagates stuff set in hooks? idk.
|
||||
finalAttrs.mesonFlags
|
||||
# Clangd breaks when GCC is using precompiled headers, so for the devshell specifically
|
||||
# we make precompiled C++ stdlib conditional on using Clang.
|
||||
# https://git.lix.systems/lix-project/lix/issues/374
|
||||
++ [ (lib.mesonBool "enable-pch-std" stdenv.cc.isClang) ];
|
||||
|
||||
packages =
|
||||
lib.optional (stdenv.cc.isClang && hostPlatform == buildPlatform) clang-tools_llvm
|
||||
lib.optional (stdenv.cc.isClang && hostPlatform == buildPlatform) clang-tools
|
||||
++ [
|
||||
# Why are we providing a bashInteractive? Well, when you run
|
||||
# `bash` from inside `nix develop`, say, because you are using it
|
||||
# via direnv, you will by default get bash (unusable edition).
|
||||
bashInteractive
|
||||
pythonEnv
|
||||
# docker image tool
|
||||
skopeo
|
||||
just
|
||||
nixfmt
|
||||
# Load-bearing order. Must come before clang-unwrapped below, but after clang_tools above.
|
||||
@@ -448,7 +472,10 @@ stdenv.mkDerivation (finalAttrs: {
|
||||
shellHook = ''
|
||||
# don't re-run the hook in (other) nested nix-shells
|
||||
function lixShellHook() {
|
||||
if [[ $name != lix-shell-env ]]; then
|
||||
# n.b. how the heck does this become -env-env? well, `nix develop` does it:
|
||||
# https://git.lix.systems/lix-project/lix/src/commit/7575db522e9008685c4009423398f6900a16bcce/src/nix/develop.cc#L240-L241
|
||||
# this is, of course, absurd.
|
||||
if [[ $name != lix-shell-env && $name != lix-shell-env-env ]]; then
|
||||
return;
|
||||
fi
|
||||
|
||||
@@ -480,5 +507,11 @@ stdenv.mkDerivation (finalAttrs: {
|
||||
'';
|
||||
}
|
||||
);
|
||||
|
||||
perl-bindings = pkgs.callPackage ./perl { inherit fileset stdenv; };
|
||||
|
||||
binaryTarball = pkgs.callPackage ./nix-support/binary-tarball.nix {
|
||||
nix = finalAttrs.finalPackage;
|
||||
};
|
||||
};
|
||||
})
|
||||
|
||||
+1
-1
@@ -29,7 +29,7 @@ NEED_PROG(xz, xz)
|
||||
AC_MSG_CHECKING([whether Perl is recent enough])
|
||||
if ! $perl -e 'open(FOO, "-|", "true"); while (<FOO>) { print; }; close FOO or die;'; then
|
||||
AC_MSG_RESULT(no)
|
||||
AC_MSG_ERROR([Your Perl version is too old. Nix requires Perl 5.8.0 or newer.])
|
||||
AC_MSG_ERROR([Your Perl version is too old. Lix requires Perl 5.8.0 or newer.])
|
||||
fi
|
||||
AC_MSG_RESULT(yes)
|
||||
|
||||
|
||||
+1
-1
@@ -23,7 +23,7 @@ perl.pkgs.toPerlModule (
|
||||
src = fileset.toSource {
|
||||
root = ../.;
|
||||
fileset = fileset.unions ([
|
||||
../.version
|
||||
../version.json
|
||||
./lib
|
||||
./meson.build
|
||||
]);
|
||||
|
||||
+1
-1
@@ -1,5 +1,5 @@
|
||||
project('lix-perl', 'cpp',
|
||||
version : run_command('bash', '-c', 'echo -n $(cat ../.version)$VERSION_SUFFIX', check : true).stdout().strip(),
|
||||
version : run_command('bash', '-c', 'echo -n $(jq -r .version < ../version.json)$VERSION_SUFFIX', check : true).stdout().strip(),
|
||||
default_options : [
|
||||
'cpp_std=c++2a',
|
||||
# TODO(Qyriad): increase the warning level
|
||||
|
||||
@@ -0,0 +1,132 @@
|
||||
# Release engineering
|
||||
|
||||
This directory contains the release engineering scripts for Lix.
|
||||
|
||||
## Release process
|
||||
|
||||
### Prerequisites
|
||||
|
||||
* FIXME: validation via misc tests in nixpkgs, among other things? What other
|
||||
validation do we need before we can actually release?
|
||||
* Have a release post ready to go as a PR on the website repo.
|
||||
* No [release-blocker bugs][release-blockers].
|
||||
|
||||
[release-blockers]: https://git.lix.systems/lix-project/lix/issues?q=&type=all&sort=&labels=145&state=open&milestone=0&project=0&assignee=0&poster=0
|
||||
|
||||
### Process
|
||||
|
||||
The following process can be done either against the staging environment or the
|
||||
live environment.
|
||||
|
||||
For staging, the buckets are `staging-releases`, `staging-cache`, etc.
|
||||
|
||||
FIXME: obtainment of signing key for signing cache paths?
|
||||
|
||||
First, we prepare the release. `python -m releng prepare` is used for this.
|
||||
|
||||
* Gather everything in `doc/manual/rl-next` and put it in
|
||||
`doc/manual/src/release-notes/rl-MAJOR.md`.
|
||||
|
||||
Then we tag the release with `python -m releng tag`:
|
||||
|
||||
* Git HEAD is detached.
|
||||
* `officialRelease = true` is set in `flake.nix`, this is committed, and a
|
||||
release is tagged.
|
||||
* The tag is merged back into the last branch (either `main` for new releases
|
||||
or `release-MAJOR` for maintenance releases) with `git merge -s ours VERSION`
|
||||
creating a history link but ignoring the tree of the release tag.
|
||||
* Git HEAD is once again detached onto the release tag.
|
||||
|
||||
Then, we build the release artifacts with `python -m releng build`:
|
||||
|
||||
* Source tarball is generated with `git archive`, then checksummed.
|
||||
* Manifest for `nix upgrade-nix` is produced and put in `s3://releases` at
|
||||
`/manifest.nix` and `/lix/lix-VERSION`.
|
||||
* Release is built: `hydraJobs.binaryTarball` jobs are built, and joined into a
|
||||
derivation that depends on all of them and adds checksum files. This and the
|
||||
sources go into `s3://releases/lix/lix-VERSION`.
|
||||
|
||||
At this point we have a `release/artifacts` and `release/manual` directory
|
||||
which are ready to publish, and tags ready for publication. No keys are
|
||||
required to do this part.
|
||||
|
||||
Next, we do the publication with `python -m releng upload`:
|
||||
|
||||
* Artifacts for this release are uploaded:
|
||||
* s3://releases/manifest.nix, changing the default version of Lix for
|
||||
`nix upgrade-nix`.
|
||||
* s3://releases/lix/lix-VERSION/ gets the following contents
|
||||
* Binary tarballs
|
||||
* Docs: `manual/` (FIXME: should we actually do this? what about putting it
|
||||
on docs.lix.systems? I think doing both is correct, since the Web site
|
||||
should not be an archive of random old manuals)
|
||||
* Docs as tarball in addition to web.
|
||||
* Source tarball
|
||||
* Docker image (FIXME: upload to forgejo registry and github registry [in the future][upload-docker])
|
||||
* s3://docs/manual/lix/MAJOR
|
||||
* s3://docs/manual/lix/stable
|
||||
|
||||
* The tag is uploaded to the remote repo.
|
||||
* **Manually** build the installer using the scripts in the installer repo and upload.
|
||||
|
||||
FIXME: This currently requires a local Apple Macintosh® aarch64 computer, but
|
||||
we could possibly automate doing it from the aarch64-darwin builder.
|
||||
* **Manually** Push the main/release branch directly to gerrit.
|
||||
* If this is a new major release, branch-off to `release-MAJOR` and push *that* branch
|
||||
directly to gerrit as well (FIXME: special creds for doing this as a service
|
||||
account so we don't need to have the gerrit perms to shoot ourselves in the
|
||||
foot by default because pushing to main is bad?).
|
||||
|
||||
FIXME: automate branch-off to `release-*` branch.
|
||||
* **Manually** (FIXME?) switch back to the release branch, which now has the
|
||||
correct revision.
|
||||
* Post!!
|
||||
* Merge release blog post to [lix-website].
|
||||
* Toot about it! https://chaos.social/@lix_project
|
||||
* Tweet about it! https://twitter.com/lixproject
|
||||
|
||||
[lix-website]: https://git.lix.systems/lix-project/lix-website
|
||||
|
||||
[upload-docker]: https://git.lix.systems/lix-project/lix/issues/252
|
||||
|
||||
### Installer
|
||||
|
||||
The installer is cross-built to several systems from a Mac using
|
||||
`build-all.xsh` and `upload-to-lix.xsh` in the installer repo (FIXME: currently
|
||||
at least; maybe this should be moved here?) .
|
||||
|
||||
It installs a binary tarball (FIXME: [it should be taught to substitute from
|
||||
cache instead][installer-substitute])
|
||||
from some URL; this is the `hydraJobs.binaryTarball`. The default URLs differ
|
||||
by architecture and are [configured here][tarball-urls].
|
||||
|
||||
[installer-substitute]: https://git.lix.systems/lix-project/lix-installer/issues/13
|
||||
[tarball-urls]: https://git.lix.systems/lix-project/lix-installer/src/commit/693592ed10d421a885bec0a9dd45e87ab87eb90a/src/settings.rs#L14-L28
|
||||
|
||||
## Infrastructure summary
|
||||
|
||||
* releases.lix.systems (`s3://releases`):
|
||||
* Each release gets a directory: https://releases.lix.systems/lix/lix-2.90-beta.1
|
||||
* Binary tarballs: `nix-2.90.0-beta.1-x86_64-linux.tar.xz`, from `hydraJobs.binaryTarball`
|
||||
* Manifest: `manifest.nix`, an attrset of the store paths by architecture.
|
||||
* Manifest for `nix upgrade-nix` to the latest release at `/manifest.nix`.
|
||||
* cache.lix.systems (`s3://cache`):
|
||||
* Receives all artifacts for released versions of Lix; is a plain HTTP binary cache.
|
||||
* install.lix.systems (`s3://install`):
|
||||
```
|
||||
~ » aws s3 ls s3://install/lix/
|
||||
PRE lix-2.90-beta.0/
|
||||
PRE lix-2.90-beta.1/
|
||||
PRE lix-2.90.0pre20240411/
|
||||
PRE lix-2.90.0pre20240412/
|
||||
2024-05-05 18:59:11 6707344 lix-installer-aarch64-darwin
|
||||
2024-05-05 18:59:16 7479768 lix-installer-aarch64-linux
|
||||
2024-05-05 18:59:14 7982208 lix-installer-x86_64-darwin
|
||||
2024-05-05 18:59:17 8978352 lix-installer-x86_64-linux
|
||||
|
||||
~ » aws s3 ls s3://install/lix/lix-2.90-beta.1/
|
||||
2024-05-05 18:59:01 6707344 lix-installer-aarch64-darwin
|
||||
2024-05-05 18:59:06 7479768 lix-installer-aarch64-linux
|
||||
2024-05-05 18:59:03 7982208 lix-installer-x86_64-darwin
|
||||
2024-05-05 18:59:07 8978352 lix-installer-x86_64-linux
|
||||
```
|
||||
@@ -0,0 +1,79 @@
|
||||
from xonsh.main import setup
|
||||
|
||||
setup()
|
||||
del setup
|
||||
|
||||
import logging
|
||||
import sys
|
||||
|
||||
import xonsh.base_shell
|
||||
|
||||
from . import environment
|
||||
from . import create_release
|
||||
from . import keys
|
||||
from . import version
|
||||
from . import cli
|
||||
from . import docker
|
||||
from . import docker_assemble
|
||||
from . import gitutils
|
||||
|
||||
|
||||
def setup_logging():
|
||||
"""
|
||||
Sets up logging to work properly. The following are intended to work:
|
||||
- ipython/xonsh configuration files adding log handlers out of band
|
||||
- Reloading the module in xonsh/ipython not causing Bonus Loggers (which is
|
||||
why we check if there is already a handler. This also helps the previous
|
||||
case)
|
||||
- Importing the releng module from xonsh and poking at it interactively
|
||||
"""
|
||||
|
||||
LEVELS = {
|
||||
# Root logger must be DEBUG so that anything else can be DEBUG
|
||||
None: logging.DEBUG,
|
||||
# Everything in releng
|
||||
__name__: logging.DEBUG,
|
||||
# Log spam caused by prompt_toolkit
|
||||
'asyncio': logging.INFO,
|
||||
}
|
||||
|
||||
for name, level in LEVELS.items():
|
||||
logger = logging.getLogger(name)
|
||||
logger.setLevel(level)
|
||||
|
||||
root_logger = logging.getLogger()
|
||||
|
||||
fmt = logging.Formatter('{asctime} {levelname} {name}: {message}',
|
||||
datefmt='%b %d %H:%M:%S',
|
||||
style='{')
|
||||
|
||||
if not any(
|
||||
isinstance(h, logging.StreamHandler) for h in root_logger.handlers):
|
||||
stderr = sys.stderr
|
||||
# XXX: Horrible hack required by the virtual stderr xonsh uses for each entered
|
||||
# command getting closed after the command is run: we need to pull out
|
||||
# the real stderr because this survives across multiple command runs.
|
||||
#
|
||||
# This only applies when running xonsh in interactive mode and importing releng.
|
||||
if isinstance(sys.stderr, xonsh.base_shell._TeeStd):
|
||||
stderr = stderr.std # type: ignore
|
||||
|
||||
hand = logging.StreamHandler(stream=stderr)
|
||||
hand.set_name('releng root handler')
|
||||
hand.setFormatter(fmt)
|
||||
root_logger.addHandler(hand)
|
||||
|
||||
|
||||
setup_logging()
|
||||
|
||||
|
||||
def reload():
|
||||
import importlib
|
||||
importlib.reload(environment)
|
||||
importlib.reload(create_release)
|
||||
importlib.reload(keys)
|
||||
importlib.reload(version)
|
||||
importlib.reload(cli)
|
||||
importlib.reload(docker)
|
||||
importlib.reload(docker_assemble)
|
||||
importlib.reload(gitutils)
|
||||
@@ -0,0 +1,3 @@
|
||||
from . import cli
|
||||
|
||||
cli.main()
|
||||
+116
@@ -0,0 +1,116 @@
|
||||
from . import create_release
|
||||
from . import docker
|
||||
from .environment import RelengEnvironment
|
||||
from . import environment
|
||||
import argparse
|
||||
import sys
|
||||
|
||||
|
||||
def do_build(args):
|
||||
if args.target == 'all':
|
||||
create_release.build_artifacts(args.profile, no_check_git=args.no_check_git)
|
||||
elif args.target == 'manual':
|
||||
# n.b. args.profile does nothing here, you will just get the x86_64-linux manual no matter what.
|
||||
eval_result = create_release.eval_jobs(args.profile)
|
||||
create_release.build_manual(eval_result)
|
||||
else:
|
||||
raise ValueError('invalid target, unreachable')
|
||||
|
||||
|
||||
def do_tag(args):
|
||||
create_release.do_tag_merge(force_tag=args.force_tag,
|
||||
no_check_git=args.no_check_git)
|
||||
|
||||
|
||||
def do_upload(env: RelengEnvironment, args):
|
||||
create_release.setup_creds(env)
|
||||
if args.target == 'all':
|
||||
docker.check_all_logins(env)
|
||||
create_release.upload_artifacts(env,
|
||||
force_push_tag=args.force_push_tag,
|
||||
noconfirm=args.noconfirm,
|
||||
no_check_git=args.no_check_git)
|
||||
elif args.target == 'manual':
|
||||
create_release.upload_manual(env)
|
||||
else:
|
||||
raise ValueError('invalid target, unreachable')
|
||||
|
||||
|
||||
def do_prepare(args):
|
||||
create_release.prepare_release_notes()
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description='*Lix ur release engineering*')
|
||||
|
||||
def fail(args):
|
||||
ap.print_usage()
|
||||
sys.exit(1)
|
||||
|
||||
ap.set_defaults(cmd=fail)
|
||||
|
||||
sps = ap.add_subparsers()
|
||||
|
||||
prepare = sps.add_parser(
|
||||
'prepare',
|
||||
help='Prepares for a release by moving the release notes over.')
|
||||
prepare.set_defaults(cmd=do_prepare)
|
||||
|
||||
tag = sps.add_parser(
|
||||
'tag',
|
||||
help=
|
||||
'Create the tag for the current release in .version and merge it back to the current branch, then switch to it'
|
||||
)
|
||||
tag.add_argument('--no-check-git',
|
||||
action='store_true',
|
||||
help="Don't check git state before tagging. For testing.")
|
||||
tag.add_argument('--force-tag',
|
||||
action='store_true',
|
||||
help='Overwrite the existing tag. For testing.')
|
||||
tag.set_defaults(cmd=do_tag)
|
||||
|
||||
build = sps.add_parser(
|
||||
'build',
|
||||
help=
|
||||
'Build an artifacts/ directory with the things that would be released')
|
||||
build.add_argument(
|
||||
'--no-check-git',
|
||||
action='store_true',
|
||||
help="Don't check git state before building. For testing.")
|
||||
build.add_argument('--target',
|
||||
choices=['manual', 'all'],
|
||||
help='Whether to build everything or just the manual')
|
||||
build.add_argument('--profile',
|
||||
default='all',
|
||||
choices=('all', 'x86_64-linux-only'),
|
||||
help='Which systems to build targets for.')
|
||||
build.set_defaults(cmd=do_build)
|
||||
|
||||
upload = sps.add_parser(
|
||||
'upload', help='Upload artifacts to cache and releases bucket')
|
||||
upload.add_argument(
|
||||
'--no-check-git',
|
||||
action='store_true',
|
||||
help="Don't check git state before uploading. For testing.")
|
||||
upload.add_argument('--force-push-tag',
|
||||
action='store_true',
|
||||
help='Force push the tag. For testing.')
|
||||
upload.add_argument(
|
||||
'--target',
|
||||
choices=['manual', 'all'],
|
||||
default='all',
|
||||
help='Whether to upload a release or just the nightly/otherwise manual'
|
||||
)
|
||||
upload.add_argument(
|
||||
'--noconfirm',
|
||||
action='store_true',
|
||||
help="Don't ask for confirmation. For testing/automation.")
|
||||
upload.add_argument('--environment',
|
||||
choices=list(environment.ENVIRONMENTS.keys()),
|
||||
default='staging',
|
||||
help='Environment to release to')
|
||||
upload.set_defaults(cmd=lambda args: do_upload(
|
||||
environment.ENVIRONMENTS[args.environment], args))
|
||||
|
||||
args = ap.parse_args()
|
||||
args.cmd(args)
|
||||
@@ -0,0 +1,312 @@
|
||||
import json
|
||||
import subprocess
|
||||
import itertools
|
||||
import textwrap
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
import hashlib
|
||||
import datetime
|
||||
from . import environment
|
||||
from .environment import RelengEnvironment
|
||||
from . import keys
|
||||
from . import docker
|
||||
from .version import VERSION, RELEASE_NAME, MAJOR
|
||||
from .gitutils import verify_are_on_tag, git_preconditions
|
||||
|
||||
$RAISE_SUBPROC_ERROR = True
|
||||
$XONSH_SHOW_TRACEBACK = True
|
||||
|
||||
GCROOTS_DIR = Path('./release/gcroots')
|
||||
BUILT_GCROOTS_DIR = Path('./release/gcroots-build')
|
||||
DRVS_TXT = Path('./release/drvs.txt')
|
||||
ARTIFACTS = Path('./release/artifacts')
|
||||
MANUAL = Path('./release/manual')
|
||||
|
||||
RELENG_MSG = "Release created with releng/create_release.xsh"
|
||||
|
||||
BUILD_CORES = 16
|
||||
MAX_JOBS = 2
|
||||
|
||||
|
||||
def setup_creds(env: RelengEnvironment):
|
||||
key = keys.get_ephemeral_key(env)
|
||||
$AWS_SECRET_ACCESS_KEY = key.secret_key
|
||||
$AWS_ACCESS_KEY_ID = key.id
|
||||
$AWS_DEFAULT_REGION = 'garage'
|
||||
$AWS_ENDPOINT_URL = environment.S3_ENDPOINT
|
||||
|
||||
|
||||
def official_release_commit_tag(force_tag=False):
|
||||
print('[+] Setting officialRelease in flake.nix and tagging')
|
||||
prev_branch = $(git symbolic-ref --short HEAD).strip()
|
||||
|
||||
git switch --detach
|
||||
sed -i 's/officialRelease = false/officialRelease = true/' flake.nix
|
||||
git add flake.nix
|
||||
message = f'release: {VERSION} "{RELEASE_NAME}"\n\nRelease produced with releng/create_release.xsh'
|
||||
git commit -m @(message)
|
||||
git tag @(['-f'] if force_tag else []) -a -m @(message) @(VERSION)
|
||||
|
||||
return prev_branch
|
||||
|
||||
|
||||
def merge_to_release(prev_branch):
|
||||
git switch @(prev_branch)
|
||||
# Create a merge back into the release branch so that git tools understand
|
||||
# that the release branch contains the tag, without the release commit
|
||||
# actually influencing the tree.
|
||||
merge_msg = textwrap.dedent("""\
|
||||
release: merge release {VERSION} back to mainline
|
||||
|
||||
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
|
||||
{RELENG_MSG}
|
||||
""").format(VERSION=VERSION, RELENG_MSG=RELENG_MSG)
|
||||
git merge -m @(merge_msg) -s ours @(VERSION)
|
||||
|
||||
|
||||
def realise(paths: list[str]):
|
||||
args = [
|
||||
'--realise',
|
||||
'--max-jobs',
|
||||
MAX_JOBS,
|
||||
'--cores',
|
||||
BUILD_CORES,
|
||||
'--log-format',
|
||||
'bar-with-logs',
|
||||
'--add-root',
|
||||
BUILT_GCROOTS_DIR
|
||||
]
|
||||
nix-store @(args) @(paths)
|
||||
|
||||
|
||||
def eval_jobs(build_profile):
|
||||
nej_output = $(nix-eval-jobs --workers 4 --gc-roots-dir @(GCROOTS_DIR) --force-recurse --flake f'.#release-jobs.{build_profile}')
|
||||
return [json.loads(s) for s in nej_output.strip().split('\n')]
|
||||
|
||||
|
||||
def upload_drv_paths_and_outputs(env: RelengEnvironment, paths: list[str]):
|
||||
proc = subprocess.Popen([
|
||||
'nix',
|
||||
'copy',
|
||||
'-v',
|
||||
'--to',
|
||||
env.cache_store_uri(),
|
||||
'--stdin',
|
||||
],
|
||||
stdin=subprocess.PIPE,
|
||||
env=__xonsh__.env.detype(),
|
||||
)
|
||||
|
||||
proc.stdin.write('\n'.join(itertools.chain(paths, x + '^*' for x in paths)).encode())
|
||||
proc.stdin.close()
|
||||
rv = proc.wait()
|
||||
if rv != 0:
|
||||
raise subprocess.CalledProcessError(rv, proc.args)
|
||||
|
||||
|
||||
def make_manifest(eval_result):
|
||||
manifest = {vs['system']: vs['outputs']['out'] for vs in eval_result}
|
||||
def manifest_line(system, out):
|
||||
return f' {system} = "{out}";'
|
||||
|
||||
manifest_text = textwrap.dedent("""\
|
||||
# This file was generated by releng/create_release.xsh in Lix
|
||||
{{
|
||||
{lines}
|
||||
}}
|
||||
""").format(lines='\n'.join(manifest_line(s, p) for (s, p) in manifest.items()))
|
||||
|
||||
return manifest_text
|
||||
|
||||
|
||||
def make_git_tarball(to: Path):
|
||||
git archive --verbose --prefix=lix-@(VERSION)/ --format=tar.gz -o @(to) @(VERSION)
|
||||
|
||||
|
||||
def confirm(prompt, expected):
|
||||
resp = input(prompt)
|
||||
|
||||
if resp != expected:
|
||||
raise ValueError('Unconfirmed')
|
||||
|
||||
|
||||
def sha256_file(f: Path):
|
||||
hasher = hashlib.sha256()
|
||||
|
||||
with open(f, 'rb') as h:
|
||||
while data := h.read(1024 * 1024):
|
||||
hasher.update(data)
|
||||
|
||||
return hasher.hexdigest()
|
||||
|
||||
|
||||
def make_artifacts_dir(eval_result, d: Path):
|
||||
d.mkdir(exist_ok=True, parents=True)
|
||||
version_dir = d / 'lix' / f'lix-{VERSION}'
|
||||
version_dir.mkdir(exist_ok=True, parents=True)
|
||||
|
||||
tarballs_drv = next(p for p in eval_result if p['attr'] == 'tarballs')
|
||||
cp --no-preserve=mode -r @(tarballs_drv['outputs']['out'])/* @(version_dir)
|
||||
|
||||
# FIXME: upgrade-nix searches for manifest.nix at root, which is rather annoying
|
||||
with open(d / 'manifest.nix', 'w') as h:
|
||||
h.write(make_manifest(eval_result))
|
||||
|
||||
with open(version_dir / 'manifest.nix', 'w') as h:
|
||||
h.write(make_manifest(eval_result))
|
||||
|
||||
print('[+] Make sources tarball')
|
||||
|
||||
filename = f'lix-{VERSION}.tar.gz'
|
||||
git_tarball = version_dir / filename
|
||||
make_git_tarball(git_tarball)
|
||||
|
||||
file_hash = sha256_file(git_tarball)
|
||||
|
||||
print(f'Hash: {file_hash}')
|
||||
with open(version_dir / f'{filename}.sha256', 'w') as h:
|
||||
h.write(file_hash)
|
||||
|
||||
|
||||
def prepare_release_notes():
|
||||
print('[+] Preparing release notes')
|
||||
RELEASE_NOTES_PATH = Path('doc/manual/rl-next')
|
||||
|
||||
if RELEASE_NOTES_PATH.isdir():
|
||||
notes_body = subprocess.check_output(['build-release-notes', '--change-authors', 'doc/manual/change-authors.yml', 'doc/manual/rl-next']).decode()
|
||||
else:
|
||||
# I guess nobody put release notes on their changes?
|
||||
print('[-] Warning: seemingly missing any release notes, not worrying about it')
|
||||
notes_body = ''
|
||||
|
||||
rl_path = Path(f'doc/manual/src/release-notes/rl-{MAJOR}.md')
|
||||
|
||||
existing_rl = ''
|
||||
try:
|
||||
with open(rl_path, 'r') as fh:
|
||||
existing_rl = fh.read()
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
|
||||
date = datetime.datetime.now().strftime('%Y-%m-%d')
|
||||
|
||||
minor_header = f'# Lix {VERSION} ({date})'
|
||||
|
||||
header = f'# Lix {MAJOR} "{RELEASE_NAME}"'
|
||||
if existing_rl.startswith(header):
|
||||
# strip the header off for minor releases
|
||||
lines = existing_rl.splitlines()
|
||||
header = lines[0]
|
||||
existing_rl = '\n'.join(lines[1:])
|
||||
else:
|
||||
header += f' ({date})\n\n'
|
||||
|
||||
header += '\n' + minor_header + '\n'
|
||||
|
||||
notes = header
|
||||
notes += notes_body
|
||||
notes += "\n\n"
|
||||
notes += existing_rl
|
||||
|
||||
# make pre-commit happy about one newline
|
||||
notes = notes.rstrip()
|
||||
notes += "\n"
|
||||
|
||||
with open(rl_path, 'w') as fh:
|
||||
fh.write(notes)
|
||||
|
||||
commit_msg = textwrap.dedent("""\
|
||||
release: release notes for {VERSION}
|
||||
|
||||
{RELENG_MSG}
|
||||
""").format(VERSION=VERSION, RELENG_MSG=RELENG_MSG)
|
||||
|
||||
git add @(rl_path)
|
||||
git rm doc/manual/rl-next/*.md
|
||||
|
||||
git commit -m @(commit_msg)
|
||||
|
||||
|
||||
def upload_artifacts(env: RelengEnvironment, noconfirm=False, no_check_git=False, force_push_tag=False):
|
||||
if not no_check_git:
|
||||
verify_are_on_tag()
|
||||
git_preconditions()
|
||||
assert 'AWS_SECRET_ACCESS_KEY' in __xonsh__.env
|
||||
|
||||
tree @(ARTIFACTS)
|
||||
|
||||
env_part = f'environment {env.name}'
|
||||
not noconfirm and confirm(
|
||||
f'Would you like to release {ARTIFACTS} as {VERSION} in {env.colour(env_part)}? Type "I want to release this to {env.name}" to confirm\n',
|
||||
f'I want to release this to {env.name}'
|
||||
)
|
||||
|
||||
docker_images = list((ARTIFACTS / f'lix/lix-{VERSION}').glob(f'lix-{VERSION}-docker-image-*.tar.gz'))
|
||||
assert docker_images
|
||||
|
||||
print('[+] Upload to cache')
|
||||
with open(DRVS_TXT) as fh:
|
||||
upload_drv_paths_and_outputs(env, [x.strip() for x in fh.readlines() if x])
|
||||
|
||||
print('[+] Upload docker images')
|
||||
for target in env.docker_targets:
|
||||
docker.upload_docker_images(target, docker_images)
|
||||
|
||||
print('[+] Upload to release bucket')
|
||||
aws s3 cp --recursive @(ARTIFACTS)/ @(env.releases_bucket)/
|
||||
print('[+] Upload manual')
|
||||
upload_manual(env)
|
||||
|
||||
print('[+] git push tag')
|
||||
git push @(['-f'] if force_push_tag else []) @(env.git_repo) f'{VERSION}:refs/tags/{VERSION}'
|
||||
|
||||
|
||||
def do_tag_merge(force_tag=False, no_check_git=False):
|
||||
if not no_check_git:
|
||||
git_preconditions()
|
||||
prev_branch = official_release_commit_tag(force_tag=force_tag)
|
||||
merge_to_release(prev_branch)
|
||||
git switch --detach @(VERSION)
|
||||
|
||||
|
||||
def build_manual(eval_result):
|
||||
(drv, manual) = next((x['drvPath'], x['outputs']['doc']) for x in eval_result if x['attr'] == 'build.x86_64-linux')
|
||||
print('[+] Building manual')
|
||||
realise([drv])
|
||||
|
||||
cp --no-preserve=mode -T -vr @(manual)/share/doc/nix/manual @(MANUAL)
|
||||
|
||||
|
||||
def upload_manual(env: RelengEnvironment):
|
||||
stable = json.loads($(nix eval --json '.#nix.officialRelease'))
|
||||
if stable:
|
||||
version = MAJOR
|
||||
else:
|
||||
version = 'nightly'
|
||||
|
||||
print('[+] aws s3 sync manual')
|
||||
aws s3 sync @(MANUAL)/ @(env.docs_bucket)/manual/lix/@(version)/
|
||||
if stable:
|
||||
aws s3 sync @(MANUAL)/ @(env.docs_bucket)/manual/lix/stable/
|
||||
|
||||
|
||||
def build_artifacts(build_profile, no_check_git=False):
|
||||
rm -rf release/
|
||||
if not no_check_git:
|
||||
verify_are_on_tag()
|
||||
git_preconditions()
|
||||
|
||||
print('[+] Evaluating')
|
||||
eval_result = eval_jobs(build_profile)
|
||||
drv_paths = [x['drvPath'] for x in eval_result]
|
||||
|
||||
print('[+] Building')
|
||||
realise(drv_paths)
|
||||
build_manual(eval_result)
|
||||
|
||||
with open(DRVS_TXT, 'w') as fh:
|
||||
# don't bother putting the release tarballs themselves because they are duplicate and huge
|
||||
fh.write('\n'.join(x['drvPath'] for x in eval_result if x['attr'] != 'lix-release-tarballs'))
|
||||
|
||||
make_artifacts_dir(eval_result, ARTIFACTS)
|
||||
print(f'[+] Done! See {ARTIFACTS}')
|
||||
@@ -0,0 +1,83 @@
|
||||
import json
|
||||
import logging
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
|
||||
import requests
|
||||
|
||||
from .environment import DockerTarget, RelengEnvironment
|
||||
from .version import VERSION, MAJOR
|
||||
from . import gitutils
|
||||
from .docker_assemble import Registry, OCIIndex, OCIIndexItem
|
||||
from . import docker_assemble
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
log.setLevel(logging.INFO)
|
||||
|
||||
def check_all_logins(env: RelengEnvironment):
|
||||
for target in env.docker_targets:
|
||||
check_login(target)
|
||||
|
||||
def check_login(target: DockerTarget):
|
||||
log.info('Checking login for %s', target.registry_name)
|
||||
skopeo login @(target.registry_name())
|
||||
|
||||
def upload_docker_images(target: DockerTarget, paths: list[Path]):
|
||||
if not paths: return
|
||||
|
||||
sess = requests.Session()
|
||||
sess.headers['User-Agent'] = 'lix-releng'
|
||||
|
||||
tag_names = [DockerTarget.resolve(tag, version=VERSION, major=MAJOR) for tag in target.tags]
|
||||
|
||||
# latest only gets tagged for the current release branch of Lix
|
||||
if not gitutils.is_maintenance_branch('HEAD'):
|
||||
tag_names.append('latest')
|
||||
|
||||
meta = {}
|
||||
|
||||
reg = docker_assemble.Registry(sess)
|
||||
manifests = []
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
tmp = Path(tmp)
|
||||
|
||||
for path in paths:
|
||||
digest_file = tmp / (path.name + '.digest')
|
||||
tmp_image = tmp / 'tmp-image.tar.gz'
|
||||
|
||||
# insecure-policy: we don't have any signature policy, we are just uploading an image
|
||||
#
|
||||
# Absurd: we copy it into an OCI image first so we can get the hash
|
||||
# we need to upload it untagged, because skopeo has no "don't tag
|
||||
# this" option.
|
||||
# The reason for this is that forgejo's container registry throws
|
||||
# away old versions of tags immediately, so we cannot use a temp
|
||||
# tag, and it *does* reduce confusion to not upload tags that
|
||||
# should not be used.
|
||||
#
|
||||
# Workaround for: https://github.com/containers/skopeo/issues/2354
|
||||
log.info('skopeo copy to temp oci-archive %s', tmp_image)
|
||||
skopeo --insecure-policy copy --format oci --all --digestfile @(digest_file) docker-archive:@(path) oci-archive:@(tmp_image)
|
||||
|
||||
inspection = json.loads($(skopeo inspect oci-archive:@(tmp_image)))
|
||||
|
||||
docker_arch = inspection['Architecture']
|
||||
docker_os = inspection['Os']
|
||||
meta = inspection['Labels']
|
||||
|
||||
log.info('Pushing image %s for %s to %s', path, docker_arch, target.registry_path)
|
||||
|
||||
digest = digest_file.read_text().strip()
|
||||
skopeo --insecure-policy copy --preserve-digests --all oci-archive:@(tmp_image) f'docker://{target.registry_path}@{digest}'
|
||||
|
||||
# skopeo doesn't give us the manifest size directly, so we just ask the registry
|
||||
metadata = reg.image_info(target.registry_path, digest)
|
||||
|
||||
manifests.append(OCIIndexItem(metadata=metadata, architecture=docker_arch, os=docker_os))
|
||||
|
||||
log.info('Pushed images to %r, building a bigger and more menacing manifest from %r with metadata %r', target, manifests, meta)
|
||||
# send the multiarch manifest to each tag
|
||||
index = OCIIndex(manifests=manifests, annotations=meta)
|
||||
for tag in tag_names:
|
||||
reg.upload_index(target.registry_path, tag, index)
|
||||
@@ -0,0 +1,391 @@
|
||||
from typing import Any, Literal, Optional
|
||||
import re
|
||||
from pathlib import Path
|
||||
import json
|
||||
import dataclasses
|
||||
import time
|
||||
from urllib.parse import unquote
|
||||
import urllib.request
|
||||
import logging
|
||||
|
||||
import requests.auth
|
||||
import requests
|
||||
import xdg_base_dirs
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
log.setLevel(logging.INFO)
|
||||
|
||||
DEBUG_REQUESTS = False
|
||||
if DEBUG_REQUESTS:
|
||||
urllib3_logger = logging.getLogger('requests.packages.urllib3')
|
||||
urllib3_logger.setLevel(logging.DEBUG)
|
||||
urllib3_logger.propagate = True
|
||||
|
||||
# So, there is a bunch of confusing stuff happening in this file. The gist of why it's Like This is:
|
||||
#
|
||||
# nix2container does not concern itself with tags (reasonably enough):
|
||||
# https://github.com/nlewo/nix2container/issues/59
|
||||
#
|
||||
# This is fine. But then we noticed: docker images don't play nice if you have
|
||||
# multiple architectures you want to abstract over if you don't do special
|
||||
# things. Those special things are images with manifests containing multiple
|
||||
# images.
|
||||
#
|
||||
# Docker has a data model vaguely analogous to git: you have higher level
|
||||
# objects referring to a bunch of content-addressed blobs.
|
||||
#
|
||||
# A multiarch image is more or less just a manifest that refers to more
|
||||
# manifests; in OCI it is an Index.
|
||||
#
|
||||
# See the API spec here: https://github.com/opencontainers/distribution-spec/blob/v1.0.1/spec.md#definitions
|
||||
# And the Index spec here: https://github.com/opencontainers/image-spec/blob/v1.0.1/image-index.md
|
||||
#
|
||||
# skopeo doesn't *know* how to make multiarch *manifests*:
|
||||
# https://github.com/containers/skopeo/issues/1136
|
||||
#
|
||||
# There is a tool called manifest-tool that is supposed to do this
|
||||
# (https://github.com/estesp/manifest-tool) but it doesn't support putting in
|
||||
# annotations on the outer image, and I *really* didn't want to write golang to
|
||||
# fix that. Thus, a little bit of homebrew containers code.
|
||||
#
|
||||
# Essentially what we are doing in here is splatting a bunch of images into the
|
||||
# registry without tagging them (with a silly workaround to skopeo issues),
|
||||
# then simply sending a new composite manifest ourselves.
|
||||
|
||||
DockerArchitecture = Literal['amd64'] | Literal['arm64']
|
||||
MANIFEST_MIME = 'application/vnd.oci.image.manifest.v1+json'
|
||||
INDEX_MIME = 'application/vnd.oci.image.index.v1+json'
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True, order=True)
|
||||
class ImageMetadata:
|
||||
size: int
|
||||
digest: str
|
||||
"""sha256:SOMEHEX"""
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True, order=True)
|
||||
class OCIIndexItem:
|
||||
"""Information about an untagged uploaded image."""
|
||||
|
||||
metadata: ImageMetadata
|
||||
|
||||
architecture: DockerArchitecture
|
||||
|
||||
os: str = 'linux'
|
||||
|
||||
def serialize(self):
|
||||
return {
|
||||
'mediaType': MANIFEST_MIME,
|
||||
'size': self.metadata.size,
|
||||
'digest': self.metadata.digest,
|
||||
'platform': {
|
||||
'architecture': self.architecture,
|
||||
'os': self.os,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class OCIIndex:
|
||||
manifests: list[OCIIndexItem]
|
||||
|
||||
annotations: dict[str, str]
|
||||
|
||||
def serialize(self):
|
||||
return {
|
||||
'schemaVersion': 2,
|
||||
'manifests': [item.serialize() for item in sorted(self.manifests)],
|
||||
'annotations': self.annotations
|
||||
}
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class TaggingOperation:
|
||||
manifest: OCIIndex
|
||||
tags: list[str]
|
||||
"""Tags this image is uploaded under"""
|
||||
|
||||
|
||||
runtime_dir = xdg_base_dirs.xdg_runtime_dir()
|
||||
config_dir = xdg_base_dirs.xdg_config_home()
|
||||
|
||||
AUTH_FILES = ([runtime_dir / 'containers/auth.json'] if runtime_dir else []) + \
|
||||
[config_dir / 'containers/auth.json', Path.home() / '.docker/config.json']
|
||||
|
||||
|
||||
# Copied from Werkzeug https://github.com/pallets/werkzeug/blob/62e3ea45846d06576199a2f8470be7fe44c867c1/src/werkzeug/http.py#L300-L325
|
||||
def parse_list_header(value: str) -> list[str]:
|
||||
"""Parse a header value that consists of a list of comma separated items according
|
||||
to `RFC 9110 <https://httpwg.org/specs/rfc9110.html#abnf.extension>`__.
|
||||
|
||||
This extends :func:`urllib.request.parse_http_list` to remove surrounding quotes
|
||||
from values.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
parse_list_header('token, "quoted value"')
|
||||
['token', 'quoted value']
|
||||
|
||||
This is the reverse of :func:`dump_header`.
|
||||
|
||||
:param value: The header value to parse.
|
||||
"""
|
||||
result = []
|
||||
|
||||
for item in urllib.request.parse_http_list(value):
|
||||
if len(item) >= 2 and item[0] == item[-1] == '"':
|
||||
item = item[1:-1]
|
||||
|
||||
result.append(item)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
# https://www.rfc-editor.org/rfc/rfc2231#section-4
|
||||
_charset_value_re = re.compile(
|
||||
r"""
|
||||
([\w!#$%&*+\-.^`|~]*)' # charset part, could be empty
|
||||
[\w!#$%&*+\-.^`|~]*' # don't care about language part, usually empty
|
||||
([\w!#$%&'*+\-.^`|~]+) # one or more token chars with percent encoding
|
||||
""",
|
||||
re.ASCII | re.VERBOSE,
|
||||
)
|
||||
|
||||
|
||||
# Copied from: https://github.com/pallets/werkzeug/blob/62e3ea45846d06576199a2f8470be7fe44c867c1/src/werkzeug/http.py#L327-L394
|
||||
def parse_dict_header(value: str) -> dict[str, str | None]:
|
||||
"""Parse a list header using :func:`parse_list_header`, then parse each item as a
|
||||
``key=value`` pair.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
parse_dict_header('a=b, c="d, e", f')
|
||||
{"a": "b", "c": "d, e", "f": None}
|
||||
|
||||
This is the reverse of :func:`dump_header`.
|
||||
|
||||
If a key does not have a value, it is ``None``.
|
||||
|
||||
This handles charsets for values as described in
|
||||
`RFC 2231 <https://www.rfc-editor.org/rfc/rfc2231#section-3>`__. Only ASCII, UTF-8,
|
||||
and ISO-8859-1 charsets are accepted, otherwise the value remains quoted.
|
||||
|
||||
:param value: The header value to parse.
|
||||
|
||||
.. versionchanged:: 3.0
|
||||
Passing bytes is not supported.
|
||||
|
||||
.. versionchanged:: 3.0
|
||||
The ``cls`` argument is removed.
|
||||
|
||||
.. versionchanged:: 2.3
|
||||
Added support for ``key*=charset''value`` encoded items.
|
||||
|
||||
.. versionchanged:: 0.9
|
||||
The ``cls`` argument was added.
|
||||
"""
|
||||
result: dict[str, str | None] = {}
|
||||
|
||||
for item in parse_list_header(value):
|
||||
key, has_value, value = item.partition("=")
|
||||
key = key.strip()
|
||||
|
||||
if not has_value:
|
||||
result[key] = None
|
||||
continue
|
||||
|
||||
value = value.strip()
|
||||
encoding: str | None = None
|
||||
|
||||
if key[-1] == "*":
|
||||
# key*=charset''value becomes key=value, where value is percent encoded
|
||||
# adapted from parse_options_header, without the continuation handling
|
||||
key = key[:-1]
|
||||
match = _charset_value_re.match(value)
|
||||
|
||||
if match:
|
||||
# If there is a charset marker in the value, split it off.
|
||||
encoding, value = match.groups()
|
||||
assert encoding
|
||||
encoding = encoding.lower()
|
||||
|
||||
# A safe list of encodings. Modern clients should only send ASCII or UTF-8.
|
||||
# This list will not be extended further. An invalid encoding will leave the
|
||||
# value quoted.
|
||||
if encoding in {"ascii", "us-ascii", "utf-8", "iso-8859-1"}:
|
||||
# invalid bytes are replaced during unquoting
|
||||
value = unquote(value, encoding=encoding)
|
||||
|
||||
if len(value) >= 2 and value[0] == value[-1] == '"':
|
||||
value = value[1:-1]
|
||||
|
||||
result[key] = value
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def parse_www_authenticate(www_authenticate):
|
||||
scheme, _, rest = www_authenticate.partition(' ')
|
||||
scheme = scheme.lower()
|
||||
rest = rest.strip()
|
||||
|
||||
parsed = parse_dict_header(rest.rstrip('='))
|
||||
return parsed
|
||||
|
||||
|
||||
class AuthState:
|
||||
|
||||
def __init__(self, auth_files: list[Path] = AUTH_FILES):
|
||||
self.auth_map: dict[str, str] = {}
|
||||
for f in auth_files:
|
||||
self.auth_map.update(AuthState.load_auth_file(f))
|
||||
self.token_cache: dict[str, str] = {}
|
||||
|
||||
@staticmethod
|
||||
def load_auth_file(path: Path) -> dict[str, str]:
|
||||
if path.exists():
|
||||
with path.open() as fh:
|
||||
try:
|
||||
json_obj = json.load(fh)
|
||||
return {k: v['auth'] for k, v in json_obj['auths'].items()}
|
||||
except (json.JSONDecodeError, KeyError) as e:
|
||||
log.exception('JSON decode error in %s', path, exc_info=e)
|
||||
return {}
|
||||
|
||||
def get_token(self, hostname: str) -> Optional[str]:
|
||||
return self.token_cache.get(hostname)
|
||||
|
||||
def obtain_token(self, session: requests.Session, token_endpoint: str,
|
||||
scope: str, service: str, image_path: str) -> str:
|
||||
authority, _, _ = image_path.partition('/')
|
||||
if tok := self.get_token(authority):
|
||||
return tok
|
||||
|
||||
creds = self.find_credential_for(image_path)
|
||||
if not creds:
|
||||
raise ValueError('No credentials available for ' + image_path)
|
||||
|
||||
resp = session.get(token_endpoint,
|
||||
params={
|
||||
'client_id': 'lix-releng',
|
||||
'scope': scope,
|
||||
'service': service,
|
||||
},
|
||||
headers={
|
||||
'Authorization': 'Basic ' + creds
|
||||
}).json()
|
||||
token = resp['token']
|
||||
self.token_cache[authority] = token
|
||||
return token
|
||||
|
||||
def find_credential_for(self, image_path: str):
|
||||
trails = image_path.split('/')
|
||||
for i in range(len(trails)):
|
||||
prefix = '/'.join(trails[:len(trails) - i])
|
||||
if prefix in self.auth_map:
|
||||
return self.auth_map[prefix]
|
||||
|
||||
return None
|
||||
|
||||
|
||||
class RegistryAuthenticator(requests.auth.AuthBase):
|
||||
"""Authenticates to an OCI compliant registry"""
|
||||
|
||||
def __init__(self, auth_state: AuthState, session: requests.Session,
|
||||
image: str):
|
||||
self.auth_map: dict[str, str] = {}
|
||||
self.image = image
|
||||
self.session = session
|
||||
self.auth_state = auth_state
|
||||
|
||||
def response_hook(self, r: requests.Response,
|
||||
**kwargs: Any) -> requests.Response:
|
||||
if r.status_code == 401:
|
||||
www_authenticate = r.headers.get('www-authenticate', '').lower()
|
||||
parsed = parse_www_authenticate(www_authenticate)
|
||||
assert parsed
|
||||
|
||||
tok = self.auth_state.obtain_token(
|
||||
self.session,
|
||||
parsed['realm'], # type: ignore
|
||||
parsed['scope'], # type: ignore
|
||||
parsed['service'], # type: ignore
|
||||
self.image)
|
||||
|
||||
new_req = r.request.copy()
|
||||
new_req.headers['Authorization'] = 'Bearer ' + tok
|
||||
|
||||
return self.session.send(new_req)
|
||||
else:
|
||||
return r
|
||||
|
||||
def __call__(self,
|
||||
r: requests.PreparedRequest) -> requests.PreparedRequest:
|
||||
authority, _, _ = self.image.partition('/')
|
||||
auth_may = self.auth_state.get_token(authority)
|
||||
|
||||
if auth_may:
|
||||
r.headers['Authorization'] = 'Bearer ' + auth_may
|
||||
|
||||
r.register_hook('response', self.response_hook)
|
||||
return r
|
||||
|
||||
|
||||
class Registry:
|
||||
|
||||
def __init__(self, session: requests.Session):
|
||||
self.auth_state = AuthState()
|
||||
self.session = session
|
||||
|
||||
def image_info(self, image_path: str, manifest_id: str) -> ImageMetadata:
|
||||
authority, _, path = image_path.partition('/')
|
||||
resp = self.session.head(
|
||||
f'https://{authority}/v2/{path}/manifests/{manifest_id}',
|
||||
headers={'Accept': MANIFEST_MIME},
|
||||
auth=RegistryAuthenticator(self.auth_state, self.session,
|
||||
image_path))
|
||||
resp.raise_for_status()
|
||||
return ImageMetadata(int(resp.headers['content-length']),
|
||||
resp.headers['docker-content-digest'])
|
||||
|
||||
def delete_tag(self, image_path: str, tag: str):
|
||||
authority, _, path = image_path.partition('/')
|
||||
resp = self.session.delete(
|
||||
f'https://{authority}/v2/{path}/manifests/{tag}',
|
||||
headers={'Content-Type': INDEX_MIME},
|
||||
auth=RegistryAuthenticator(self.auth_state, self.session,
|
||||
image_path))
|
||||
resp.raise_for_status()
|
||||
|
||||
def _upload_index(self, image_path: str, tag: str, index: OCIIndex):
|
||||
authority, _, path = image_path.partition('/')
|
||||
body = json.dumps(index.serialize(),
|
||||
separators=(',', ':'),
|
||||
sort_keys=True)
|
||||
|
||||
resp = self.session.put(
|
||||
f'https://{authority}/v2/{path}/manifests/{tag}',
|
||||
data=body,
|
||||
headers={'Content-Type': INDEX_MIME},
|
||||
auth=RegistryAuthenticator(self.auth_state, self.session,
|
||||
image_path))
|
||||
resp.raise_for_status()
|
||||
|
||||
return resp.headers['Location']
|
||||
|
||||
def upload_index(self,
|
||||
image_path: str,
|
||||
tag: str,
|
||||
index: OCIIndex,
|
||||
retries=20,
|
||||
retry_delay=1):
|
||||
# eventual consistency lmao
|
||||
for _ in range(retries):
|
||||
try:
|
||||
return self._upload_index(image_path, tag, index)
|
||||
except requests.HTTPError as e:
|
||||
if e.response.status_code != 404:
|
||||
raise
|
||||
|
||||
time.sleep(retry_delay)
|
||||
@@ -0,0 +1,134 @@
|
||||
from typing import Callable
|
||||
import urllib.parse
|
||||
import re
|
||||
import functools
|
||||
import subprocess
|
||||
import dataclasses
|
||||
|
||||
S3_HOST = 's3.lix.systems'
|
||||
S3_ENDPOINT = 'https://s3.lix.systems'
|
||||
|
||||
DEFAULT_STORE_URI_BITS = {
|
||||
'region': 'garage',
|
||||
'endpoint': 's3.lix.systems',
|
||||
'want-mass-query': 'true',
|
||||
'write-nar-listing': 'true',
|
||||
'ls-compression': 'zstd',
|
||||
'narinfo-compression': 'zstd',
|
||||
'compression': 'zstd',
|
||||
'parallel-compression': 'true',
|
||||
}
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class DockerTarget:
|
||||
registry_path: str
|
||||
"""Registry path without the tag, e.g. ghcr.io/lix-project/lix"""
|
||||
|
||||
tags: list[str]
|
||||
"""List of tags this image should take. There must be at least one."""
|
||||
|
||||
@staticmethod
|
||||
def resolve(item: str, version: str, major: str) -> str:
|
||||
"""
|
||||
Applies templates:
|
||||
- version: the Lix version e.g. 2.90.0
|
||||
- major: the major Lix version e.g. 2.90
|
||||
"""
|
||||
return item.format(version=version, major=major)
|
||||
|
||||
def registry_name(self) -> str:
|
||||
[a, _, _] = self.registry_path.partition('/')
|
||||
return a
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class RelengEnvironment:
|
||||
name: str
|
||||
colour: Callable[[str], str]
|
||||
|
||||
cache_store_overlay: dict[str, str]
|
||||
cache_bucket: str
|
||||
releases_bucket: str
|
||||
docs_bucket: str
|
||||
git_repo: str
|
||||
|
||||
docker_targets: list[DockerTarget]
|
||||
|
||||
def cache_store_uri(self):
|
||||
qs = DEFAULT_STORE_URI_BITS.copy()
|
||||
qs.update(self.cache_store_overlay)
|
||||
return self.cache_bucket + "?" + urllib.parse.urlencode(qs)
|
||||
|
||||
|
||||
SGR = '\x1b['
|
||||
RED = '31;1m'
|
||||
GREEN = '32;1m'
|
||||
RESET = '0m'
|
||||
|
||||
|
||||
def sgr(colour: str, text: str) -> str:
|
||||
return f'{SGR}{colour}{text}{SGR}{RESET}'
|
||||
|
||||
|
||||
STAGING = RelengEnvironment(
|
||||
name='staging',
|
||||
colour=functools.partial(sgr, GREEN),
|
||||
docs_bucket='s3://staging-docs',
|
||||
cache_bucket='s3://staging-cache',
|
||||
cache_store_overlay={'secret-key': 'staging.key'},
|
||||
releases_bucket='s3://staging-releases',
|
||||
git_repo='ssh://git@git.lix.systems/lix-project/lix-releng-staging',
|
||||
docker_targets=[
|
||||
# latest will be auto tagged if appropriate
|
||||
DockerTarget('git.lix.systems/lix-project/lix-releng-staging',
|
||||
tags=['{version}', '{major}']),
|
||||
DockerTarget('ghcr.io/lix-project/lix-releng-staging',
|
||||
tags=['{version}', '{major}']),
|
||||
],
|
||||
)
|
||||
|
||||
GERRIT_REMOTE_RE = re.compile(r'^ssh://(\w+@)?gerrit.lix.systems:2022/lix$')
|
||||
|
||||
|
||||
def guess_gerrit_remote():
|
||||
"""
|
||||
Deals with people having unknown gerrit username.
|
||||
"""
|
||||
out = [
|
||||
x.split()[1] for x in subprocess.check_output(
|
||||
['git', 'remote', '-v']).decode().splitlines()
|
||||
]
|
||||
return next(x for x in out if GERRIT_REMOTE_RE.match(x))
|
||||
|
||||
|
||||
PROD = RelengEnvironment(
|
||||
name='production',
|
||||
colour=functools.partial(sgr, RED),
|
||||
docs_bucket='s3://docs',
|
||||
cache_bucket='s3://cache',
|
||||
# FIXME: we should decrypt this with age into a tempdir in the future, but
|
||||
# the issue is how to deal with the recipients file. For now, we should
|
||||
# just delete it after doing a release.
|
||||
cache_store_overlay={'secret-key': 'prod.key'},
|
||||
releases_bucket='s3://releases',
|
||||
git_repo=guess_gerrit_remote(),
|
||||
docker_targets=[
|
||||
# latest will be auto tagged if appropriate
|
||||
DockerTarget('git.lix.systems/lix-project/lix',
|
||||
tags=['{version}', '{major}']),
|
||||
DockerTarget('ghcr.io/lix-project/lix', tags=['{version}', '{major}']),
|
||||
],
|
||||
)
|
||||
|
||||
ENVIRONMENTS = {
|
||||
'staging': STAGING,
|
||||
'production': PROD,
|
||||
}
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class S3Credentials:
|
||||
name: str
|
||||
id: str
|
||||
secret_key: str
|
||||
@@ -0,0 +1,39 @@
|
||||
import subprocess
|
||||
import json
|
||||
|
||||
from .version import VERSION
|
||||
|
||||
|
||||
def version_compare(v1: str, v2: str):
|
||||
return json.loads($(nix-instantiate --eval --json --argstr v1 @(v1) --argstr v2 @(v2) --expr '{v1, v2}: builtins.compareVersions v1 v2'))
|
||||
|
||||
|
||||
def latest_tag_on_branch(branch: str) -> str:
|
||||
return $(git describe --abbrev=0 @(branch) e>/dev/null).strip()
|
||||
|
||||
|
||||
def is_maintenance_branch(branch: str) -> bool:
|
||||
try:
|
||||
main_tag = latest_tag_on_branch('main')
|
||||
current_tag = latest_tag_on_branch(branch)
|
||||
|
||||
return version_compare(current_tag, main_tag) < 0
|
||||
except subprocess.CalledProcessError:
|
||||
# This is the case before Lix releases 2.90, since main *has* no
|
||||
# release tag on it.
|
||||
# FIXME: delete this case after 2.91
|
||||
return False
|
||||
|
||||
|
||||
def verify_are_on_tag():
|
||||
current_tag = $(git describe --tag).strip()
|
||||
assert current_tag == VERSION
|
||||
|
||||
|
||||
def git_preconditions():
|
||||
# verify there is nothing in index ready to stage
|
||||
proc = !(git diff-index --quiet --cached HEAD --)
|
||||
assert proc.rtn == 0
|
||||
# verify there is nothing *stageable* and tracked
|
||||
proc = !(git diff-files --quiet)
|
||||
assert proc.rtn == 0
|
||||
@@ -0,0 +1,19 @@
|
||||
import subprocess
|
||||
import json
|
||||
from . import environment
|
||||
|
||||
|
||||
def get_ephemeral_key(
|
||||
env: environment.RelengEnvironment) -> environment.S3Credentials:
|
||||
output = subprocess.check_output([
|
||||
'ssh', '-l', 'root', environment.S3_HOST, 'garage-ephemeral-key',
|
||||
'new', '--name', f'releng-{env.name}', '--read', '--write',
|
||||
'--age-secs', '3600',
|
||||
env.releases_bucket.removeprefix('s3://'),
|
||||
env.cache_bucket.removeprefix('s3://'),
|
||||
env.docs_bucket.removeprefix('s3://'),
|
||||
])
|
||||
d = json.loads(output.decode())
|
||||
return environment.S3Credentials(name=d['name'],
|
||||
id=d['id'],
|
||||
secret_key=d['secret_key'])
|
||||
@@ -0,0 +1,92 @@
|
||||
{ hydraJobs, pkgs }:
|
||||
let
|
||||
inherit (pkgs) lib;
|
||||
lix = hydraJobs.build.x86_64-linux;
|
||||
|
||||
# This is all so clumsy because we can't use arguments to functions in
|
||||
# flakes, and certainly not with n-e-j.
|
||||
profiles = {
|
||||
# Used for testing
|
||||
x86_64-linux-only = {
|
||||
systems = [ "x86_64-linux" ];
|
||||
dockerSystems = [ "x86_64-linux" ];
|
||||
};
|
||||
all = {
|
||||
systems = [
|
||||
"x86_64-linux"
|
||||
"aarch64-linux"
|
||||
"aarch64-darwin"
|
||||
"x86_64-darwin"
|
||||
];
|
||||
dockerSystems = [
|
||||
"x86_64-linux"
|
||||
"aarch64-linux"
|
||||
];
|
||||
};
|
||||
};
|
||||
|
||||
doTarball =
|
||||
{
|
||||
target,
|
||||
targetName,
|
||||
rename ? null,
|
||||
}:
|
||||
''
|
||||
echo "doing: ${target}"
|
||||
# expand wildcard
|
||||
filename=$(echo ${target}/${targetName})
|
||||
basename="$(basename $filename)"
|
||||
|
||||
echo $filename $basename
|
||||
cp -v "$filename" "$out"
|
||||
${lib.optionalString (rename != null) ''
|
||||
mv "$out/$basename" "$out/${rename}"
|
||||
basename="${rename}"
|
||||
''}
|
||||
sha256sum --binary $filename | cut -f1 -d' ' > $out/$basename.sha256
|
||||
'';
|
||||
|
||||
targetsFor =
|
||||
{ systems, dockerSystems }:
|
||||
builtins.map (system: {
|
||||
target = hydraJobs.binaryTarball.${system};
|
||||
targetName = "*.tar.xz";
|
||||
}) systems
|
||||
++ builtins.map (system: {
|
||||
target = hydraJobs.dockerImage.${system}.tarball;
|
||||
targetName = "image.tar.gz";
|
||||
rename = "lix-${lix.version}-docker-image-${system}.tar.gz";
|
||||
}) dockerSystems;
|
||||
|
||||
manualTar = pkgs.runCommand "lix-manual-tarball" { } ''
|
||||
mkdir -p $out
|
||||
cp -r ${lix.doc}/share/doc/nix/manual lix-${lix.version}-manual
|
||||
tar -cvzf "$out/lix-${lix.version}-manual.tar.gz" lix-${lix.version}-manual
|
||||
'';
|
||||
|
||||
tarballsFor =
|
||||
{ systems, dockerSystems }:
|
||||
pkgs.runCommand "lix-release-tarballs" { } ''
|
||||
mkdir -p $out
|
||||
${lib.concatMapStringsSep "\n" doTarball (targetsFor {
|
||||
inherit systems dockerSystems;
|
||||
})}
|
||||
${doTarball {
|
||||
target = manualTar;
|
||||
targetName = "lix-*.tar.gz";
|
||||
}}
|
||||
cp -r ${lix.doc}/share/doc/nix/manual $out
|
||||
'';
|
||||
in
|
||||
(builtins.mapAttrs (
|
||||
_:
|
||||
{ systems, dockerSystems }:
|
||||
{
|
||||
build = lib.filterAttrs (x: _: builtins.elem x systems) hydraJobs.build;
|
||||
tarballs = tarballsFor { inherit systems dockerSystems; };
|
||||
}
|
||||
) profiles)
|
||||
// {
|
||||
inherit (hydraJobs) build;
|
||||
inherit tarballsFor;
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
import json
|
||||
|
||||
version_json = json.load(open('version.json'))
|
||||
VERSION = version_json['version']
|
||||
MAJOR = '.'.join(VERSION.split('.')[:2])
|
||||
RELEASE_NAME = version_json['release_name']
|
||||
@@ -3,6 +3,7 @@
|
||||
#include <algorithm>
|
||||
#include <set>
|
||||
#include <memory>
|
||||
#include <string_view>
|
||||
#include <tuple>
|
||||
#include <iomanip>
|
||||
#if __APPLE__
|
||||
@@ -20,9 +21,9 @@
|
||||
#include "local-store.hh"
|
||||
#include "legacy.hh"
|
||||
#include "experimental-features.hh"
|
||||
#include "hash.hh"
|
||||
|
||||
using namespace nix;
|
||||
using std::cin;
|
||||
|
||||
static void handleAlarm(int sig) {
|
||||
}
|
||||
@@ -35,9 +36,19 @@ std::string escapeUri(std::string uri)
|
||||
|
||||
static std::string currentLoad;
|
||||
|
||||
static std::string makeLockFilename(const std::string & storeUri) {
|
||||
// We include 48 bytes of escaped URI to give an idea of what the lock
|
||||
// is on, then 16 bytes of hash to disambiguate.
|
||||
// This avoids issues with the escaped URI being very long and causing
|
||||
// path too long errors, while also avoiding any possibility of collision
|
||||
// caused by simple truncation.
|
||||
auto hash = hashString(HashType::htSHA256, storeUri).to_string(Base::Base32, false);
|
||||
return escapeUri(storeUri).substr(0, 48) + "-" + hash.substr(0, 16);
|
||||
}
|
||||
|
||||
static AutoCloseFD openSlotLock(const Machine & m, uint64_t slot)
|
||||
{
|
||||
return openLockFile(fmt("%s/%s-%d", currentLoad, escapeUri(m.storeUri), slot), true);
|
||||
return openLockFile(fmt("%s/%s-%d", currentLoad, makeLockFilename(m.storeUri), slot), true);
|
||||
}
|
||||
|
||||
static bool allSupportedLocally(Store & store, const std::set<std::string>& requiredFeatures) {
|
||||
@@ -263,7 +274,9 @@ connected:
|
||||
auto inputs = readStrings<PathSet>(source);
|
||||
auto wantedOutputs = readStrings<StringSet>(source);
|
||||
|
||||
AutoCloseFD uploadLock = openLockFile(currentLoad + "/" + escapeUri(storeUri) + ".upload-lock", true);
|
||||
auto lockFileName = currentLoad + "/" + makeLockFilename(storeUri) + ".upload-lock";
|
||||
|
||||
AutoCloseFD uploadLock = openLockFile(lockFileName, true);
|
||||
|
||||
{
|
||||
Activity act(*logger, lvlTalkative, actUnknown, fmt("waiting for the upload lock to '%s'", storeUri));
|
||||
|
||||
@@ -45,7 +45,7 @@ MixEvalArgs::MixEvalArgs()
|
||||
-I /etc/nixos
|
||||
```
|
||||
|
||||
will cause Nix to look for paths relative to `/home/eelco/Dev` and
|
||||
will cause Lix to look for paths relative to `/home/eelco/Dev` and
|
||||
`/etc/nixos`, in that order. This is equivalent to setting the
|
||||
`NIX_PATH` environment variable to
|
||||
|
||||
@@ -61,7 +61,7 @@ MixEvalArgs::MixEvalArgs()
|
||||
-I /etc/nixos
|
||||
```
|
||||
|
||||
will cause Nix to search for `<nixpkgs/path>` in
|
||||
will cause Lix to search for `<nixpkgs/path>` in
|
||||
`/home/eelco/Dev/nixpkgs-branch/path` and `/etc/nixos/nixpkgs/path`.
|
||||
|
||||
If a path in the Nix search path starts with `http://` or `https://`,
|
||||
@@ -73,7 +73,7 @@ MixEvalArgs::MixEvalArgs()
|
||||
-I nixpkgs=https://github.com/NixOS/nixpkgs/archive/master.tar.gz
|
||||
```
|
||||
|
||||
tells Nix to download and use the current contents of the `master`
|
||||
tells Lix to download and use the current contents of the `master`
|
||||
branch in the `nixpkgs` repository.
|
||||
|
||||
The URLs of the tarballs from the official `nixos.org` channels
|
||||
|
||||
@@ -54,6 +54,7 @@ libcmd = library(
|
||||
nlohmann_json,
|
||||
lix_doc
|
||||
],
|
||||
cpp_pch : cpp_pch,
|
||||
install : true,
|
||||
# FIXME(Qyriad): is this right?
|
||||
install_rpath : libdir,
|
||||
|
||||
@@ -64,15 +64,15 @@ struct EvalSettings : Config
|
||||
Pure evaluation mode ensures that the result of Nix expressions is fully determined by explicitly declared inputs, and not influenced by external state:
|
||||
|
||||
- Restrict file system and network access to files specified by cryptographic hash
|
||||
- Disable [`bultins.currentSystem`](@docroot@/language/builtin-constants.md#builtins-currentSystem) and [`builtins.currentTime`](@docroot@/language/builtin-constants.md#builtins-currentTime)
|
||||
- Disable [`builtins.currentSystem`](@docroot@/language/builtin-constants.md#builtins-currentSystem) and [`builtins.currentTime`](@docroot@/language/builtin-constants.md#builtins-currentTime)
|
||||
)"
|
||||
};
|
||||
|
||||
Setting<bool> enableImportFromDerivation{
|
||||
this, true, "allow-import-from-derivation",
|
||||
R"(
|
||||
By default, Nix allows you to `import` from a derivation, allowing
|
||||
building at evaluation time. With this option set to false, Nix will
|
||||
By default, Lix allows you to `import` from a derivation, allowing
|
||||
building at evaluation time. With this option set to false, Lix will
|
||||
throw an error when evaluating an expression that uses this feature,
|
||||
allowing users to ensure their evaluation will not require any
|
||||
builds to take place.
|
||||
|
||||
+1
-1
@@ -22,6 +22,7 @@
|
||||
#include <algorithm>
|
||||
#include <chrono>
|
||||
#include <iostream>
|
||||
#include <sstream>
|
||||
#include <cstring>
|
||||
#include <optional>
|
||||
#include <unistd.h>
|
||||
@@ -29,7 +30,6 @@
|
||||
#include <sys/resource.h>
|
||||
#include <fstream>
|
||||
#include <functional>
|
||||
#include <iostream>
|
||||
|
||||
#include <sys/resource.h>
|
||||
#include <nlohmann/json.hpp>
|
||||
|
||||
+5
-5
@@ -26,10 +26,9 @@
|
||||
#pragma clang diagnostic ignored "-Wimplicit-fallthrough"
|
||||
#endif
|
||||
|
||||
#include <boost/lexical_cast.hpp>
|
||||
|
||||
#include "nixexpr.hh"
|
||||
#include "parser-tab.hh"
|
||||
#include "strings.hh"
|
||||
|
||||
using namespace nix;
|
||||
|
||||
@@ -132,9 +131,10 @@ or { return OR_KW; }
|
||||
|
||||
{ID} { yylval->id = {yytext, (size_t) yyleng}; return ID; }
|
||||
{INT} { errno = 0;
|
||||
try {
|
||||
yylval->n = boost::lexical_cast<int64_t>(yytext);
|
||||
} catch (const boost::bad_lexical_cast &) {
|
||||
std::optional<int64_t> numMay = string2Int<int64_t>(yytext);
|
||||
if (numMay.has_value()) {
|
||||
yylval->n = *numMay;
|
||||
} else {
|
||||
throw ParseError(ErrorInfo{
|
||||
.msg = HintFmt("invalid integer '%1%'", yytext),
|
||||
.pos = state->positions[CUR_POS],
|
||||
|
||||
+11
-14
@@ -86,25 +86,11 @@ libexpr_sources = files(
|
||||
'flake/flake.cc',
|
||||
'flake/flakeref.cc',
|
||||
'flake/lockfile.cc',
|
||||
'primops/arithmetic.cc',
|
||||
'primops/attrset.cc',
|
||||
'primops/context.cc',
|
||||
'primops/control.cc',
|
||||
'primops/debug.cc',
|
||||
'primops/derivation.cc',
|
||||
'primops/hash.cc',
|
||||
'primops/fetchClosure.cc',
|
||||
'primops/fetchMercurial.cc',
|
||||
'primops/fetchTree.cc',
|
||||
'primops/fromTOML.cc',
|
||||
'primops/import.cc',
|
||||
'primops/json.cc',
|
||||
'primops/list.cc',
|
||||
'primops/path.cc',
|
||||
'primops/string.cc',
|
||||
'primops/system.cc',
|
||||
'primops/toXML.cc',
|
||||
'primops/types.cc',
|
||||
'value/context.cc',
|
||||
)
|
||||
|
||||
@@ -159,6 +145,7 @@ libexpr = library(
|
||||
include_directories : [
|
||||
'../libmain',
|
||||
],
|
||||
cpp_pch : cpp_pch,
|
||||
install : true,
|
||||
# FIXME(Qyriad): is this right?
|
||||
install_rpath : libdir,
|
||||
@@ -175,6 +162,16 @@ liblixexpr = declare_dependency(
|
||||
link_with : libexpr,
|
||||
)
|
||||
|
||||
# FIXME: remove when https://git.lix.systems/lix-project/lix/issues/359 is fixed.
|
||||
if is_static
|
||||
liblixexpr_mstatic = declare_dependency(
|
||||
include_directories : include_directories('.'),
|
||||
link_whole : libexpr,
|
||||
)
|
||||
else
|
||||
liblixexpr_mstatic = liblixexpr
|
||||
endif
|
||||
|
||||
# FIXME: not using the pkg-config module because it creates way too many deps
|
||||
# while meson migration is in progress, and we want to not include boost here
|
||||
configure_file(
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
#include "escape-string.hh"
|
||||
|
||||
#include <cstdlib>
|
||||
#include <sstream>
|
||||
|
||||
namespace nix {
|
||||
|
||||
|
||||
+4261
-339
File diff suppressed because it is too large
Load Diff
+3
-141
@@ -1,10 +1,9 @@
|
||||
#pragma once
|
||||
///@file
|
||||
|
||||
#include "derivations.hh"
|
||||
#include "eval.hh"
|
||||
|
||||
#include <regex>
|
||||
#include <tuple>
|
||||
#include <vector>
|
||||
|
||||
namespace nix {
|
||||
@@ -45,150 +44,13 @@ struct RegisterPrimOp
|
||||
/**
|
||||
* Load a ValueInitializer from a DSO and return whatever it initializes
|
||||
*/
|
||||
void prim_importNative(EvalState & state, const PosIdx pos, Value ** args, Value & v);
|
||||
void prim_importNative(EvalState & state, const PosIdx pos, Value * * args, Value & v);
|
||||
|
||||
/**
|
||||
* Execute a program and parse its output
|
||||
*/
|
||||
void prim_exec(EvalState & state, const PosIdx pos, Value ** args, Value & v);
|
||||
|
||||
void prim_lessThan(EvalState & state, const PosIdx pos, Value ** args, Value & v);
|
||||
|
||||
void prim_importNative(EvalState & state, const PosIdx pos, Value ** args, Value & v);
|
||||
|
||||
void prim_trace(EvalState & state, const PosIdx pos, Value ** args, Value & v);
|
||||
|
||||
void prim_second(EvalState & state, const PosIdx pos, Value ** args, Value & v);
|
||||
void prim_exec(EvalState & state, const PosIdx pos, Value * * args, Value & v);
|
||||
|
||||
void makePositionThunks(EvalState & state, const PosIdx pos, Value & line, Value & column);
|
||||
|
||||
void mkOutputString(
|
||||
EvalState & state,
|
||||
BindingsBuilder & attrs,
|
||||
const StorePath & drvPath,
|
||||
const std::pair<std::string, DerivationOutput> & o
|
||||
);
|
||||
|
||||
#if HAVE_BOEHMGC
|
||||
typedef std::list<Value *, gc_allocator<Value *>> ValueList;
|
||||
#else
|
||||
typedef std::list<Value *> ValueList;
|
||||
#endif
|
||||
|
||||
/**
|
||||
* getAttr wrapper
|
||||
*/
|
||||
|
||||
Bindings::iterator
|
||||
getAttr(EvalState & state, Symbol attrSym, Bindings * attrSet, std::string_view errorCtx);
|
||||
|
||||
/**
|
||||
* Struct definitions
|
||||
*/
|
||||
|
||||
struct RegexCache
|
||||
{
|
||||
// TODO use C++20 transparent comparison when available
|
||||
std::unordered_map<std::string_view, std::regex> cache;
|
||||
std::list<std::string> keys;
|
||||
|
||||
std::regex get(std::string_view re)
|
||||
{
|
||||
auto it = cache.find(re);
|
||||
if (it != cache.end()) {
|
||||
return it->second;
|
||||
}
|
||||
keys.emplace_back(re);
|
||||
return cache.emplace(keys.back(), std::regex(keys.back(), std::regex::extended))
|
||||
.first->second;
|
||||
}
|
||||
};
|
||||
|
||||
struct CompareValues
|
||||
{
|
||||
EvalState & state;
|
||||
const PosIdx pos;
|
||||
const std::string_view errorCtx;
|
||||
|
||||
CompareValues(EvalState & state, const PosIdx pos, const std::string_view && errorCtx)
|
||||
: state(state)
|
||||
, pos(pos)
|
||||
, errorCtx(errorCtx){};
|
||||
|
||||
bool operator()(Value * v1, Value * v2) const
|
||||
{
|
||||
return (*this)(v1, v2, errorCtx);
|
||||
}
|
||||
|
||||
bool operator()(Value * v1, Value * v2, std::string_view errorCtx) const
|
||||
{
|
||||
try {
|
||||
if (v1->type() == nFloat && v2->type() == nInt) {
|
||||
return v1->fpoint < v2->integer;
|
||||
}
|
||||
if (v1->type() == nInt && v2->type() == nFloat) {
|
||||
return v1->integer < v2->fpoint;
|
||||
}
|
||||
if (v1->type() != v2->type()) {
|
||||
state.error<EvalError>("cannot compare %s with %s", showType(*v1), showType(*v2))
|
||||
.debugThrow();
|
||||
}
|
||||
// Allow selecting a subset of enum values
|
||||
#pragma GCC diagnostic push
|
||||
#pragma GCC diagnostic ignored "-Wswitch-enum"
|
||||
switch (v1->type()) {
|
||||
case nInt:
|
||||
return v1->integer < v2->integer;
|
||||
case nFloat:
|
||||
return v1->fpoint < v2->fpoint;
|
||||
case nString:
|
||||
return strcmp(v1->string.s, v2->string.s) < 0;
|
||||
case nPath:
|
||||
return strcmp(v1->_path, v2->_path) < 0;
|
||||
case nList:
|
||||
// Lexicographic comparison
|
||||
for (size_t i = 0;; i++) {
|
||||
if (i == v2->listSize()) {
|
||||
return false;
|
||||
} else if (i == v1->listSize()) {
|
||||
return true;
|
||||
} else if (!state.eqValues(
|
||||
*v1->listElems()[i], *v2->listElems()[i], pos, errorCtx
|
||||
))
|
||||
{
|
||||
return (*this)(
|
||||
v1->listElems()[i],
|
||||
v2->listElems()[i],
|
||||
"while comparing two list elements"
|
||||
);
|
||||
}
|
||||
}
|
||||
default:
|
||||
state
|
||||
.error<EvalError>(
|
||||
"cannot compare %s with %s; values of that type are incomparable",
|
||||
showType(*v1),
|
||||
showType(*v2)
|
||||
)
|
||||
.debugThrow();
|
||||
#pragma GCC diagnostic pop
|
||||
}
|
||||
} catch (Error & e) {
|
||||
if (!errorCtx.empty()) {
|
||||
e.addTrace(nullptr, errorCtx);
|
||||
}
|
||||
throw;
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
struct RealisePathFlags
|
||||
{
|
||||
// Whether to check that the path is allowed in pure eval mode
|
||||
bool checkForPureEval = true;
|
||||
};
|
||||
|
||||
SourcePath
|
||||
realisePath(EvalState & state, const PosIdx pos, Value & v, const RealisePathFlags flags = {});
|
||||
|
||||
}
|
||||
|
||||
@@ -1,312 +0,0 @@
|
||||
#include <math.h> // for ceil, floor
|
||||
#include <limits> // for numeric_limits
|
||||
#include "eval-error.hh" // for EvalError, EvalErrorBuilder
|
||||
#include "eval.hh" // for EvalState, PrimOp
|
||||
#include "pos-idx.hh" // for PosIdx, noPos
|
||||
#include "primops.hh" // for CompareValues, prim_lessThan
|
||||
#include "value.hh" // for Value, nFloat, NixInt, NixFloat
|
||||
|
||||
namespace nix {
|
||||
|
||||
/**
|
||||
* builtins.add
|
||||
*/
|
||||
|
||||
static void prim_add(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceValue(*args[0], pos);
|
||||
state.forceValue(*args[1], pos);
|
||||
if (args[0]->type() == nFloat || args[1]->type() == nFloat) {
|
||||
v.mkFloat(
|
||||
state.forceFloat(*args[0], pos, "while evaluating the first argument of the addition")
|
||||
+ state.forceFloat(
|
||||
*args[1], pos, "while evaluating the second argument of the addition"
|
||||
)
|
||||
);
|
||||
} else {
|
||||
v.mkInt(
|
||||
state.forceInt(*args[0], pos, "while evaluating the first argument of the addition")
|
||||
+ state.forceInt(*args[1], pos, "while evaluating the second argument of the addition")
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_add({
|
||||
.name = "__add",
|
||||
.args = {"e1", "e2"},
|
||||
.doc = R"(
|
||||
Return the sum of the numbers *e1* and *e2*.
|
||||
Return a float if either *e1* or *e2* is a float, otherwise
|
||||
return an integer.
|
||||
)",
|
||||
.fun = prim_add,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.bitAnd
|
||||
*/
|
||||
|
||||
static void prim_bitAnd(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
v.mkInt(
|
||||
state.forceInt(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.bitAnd"
|
||||
)
|
||||
& state.forceInt(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.bitAnd"
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
PrimOp primop_bitAnd({
|
||||
.name = "__bitAnd",
|
||||
.args = {"e1", "e2"},
|
||||
.doc = R"(
|
||||
Return the bitwise AND of the integers *e1* and *e2*.
|
||||
)",
|
||||
.fun = prim_bitAnd,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.bitOr
|
||||
*/
|
||||
|
||||
static void prim_bitOr(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
v.mkInt(
|
||||
state.forceInt(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.bitOr"
|
||||
)
|
||||
| state.forceInt(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.bitOr"
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
PrimOp primop_bitOr({
|
||||
.name = "__bitOr",
|
||||
.args = {"e1", "e2"},
|
||||
.doc = R"(
|
||||
Return the bitwise OR of the integers *e1* and *e2*.
|
||||
)",
|
||||
.fun = prim_bitOr,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.bitXor
|
||||
*/
|
||||
|
||||
static void prim_bitXor(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
v.mkInt(
|
||||
state.forceInt(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.bitXor"
|
||||
)
|
||||
^ state.forceInt(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.bitXor"
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
PrimOp primop_bitXor({
|
||||
.name = "__bitXor",
|
||||
.args = {"e1", "e2"},
|
||||
.doc = R"(
|
||||
Return the bitwise XOR of the integers *e1* and *e2*.
|
||||
)",
|
||||
.fun = prim_bitXor,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.ceil
|
||||
*/
|
||||
|
||||
static void prim_ceil(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto value = state.forceFloat(
|
||||
*args[0],
|
||||
args[0]->determinePos(pos),
|
||||
"while evaluating the first argument passed to builtins.ceil"
|
||||
);
|
||||
v.mkInt(ceil(value));
|
||||
}
|
||||
|
||||
PrimOp primop_ceil({
|
||||
.name = "__ceil",
|
||||
.args = {"double"},
|
||||
.doc = R"(
|
||||
Converts an IEEE-754 double-precision floating-point number (*double*) to
|
||||
the next higher integer.
|
||||
|
||||
If the datatype is neither an integer nor a "float", an evaluation error will be
|
||||
thrown.
|
||||
)",
|
||||
.fun = prim_ceil,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.div
|
||||
*/
|
||||
|
||||
static void prim_div(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceValue(*args[0], pos);
|
||||
state.forceValue(*args[1], pos);
|
||||
|
||||
NixFloat f2 =
|
||||
state.forceFloat(*args[1], pos, "while evaluating the second operand of the division");
|
||||
if (f2 == 0) {
|
||||
state.error<EvalError>("division by zero").atPos(pos).debugThrow();
|
||||
}
|
||||
|
||||
if (args[0]->type() == nFloat || args[1]->type() == nFloat) {
|
||||
v.mkFloat(
|
||||
state.forceFloat(*args[0], pos, "while evaluating the first operand of the division")
|
||||
/ f2
|
||||
);
|
||||
} else {
|
||||
NixInt i1 =
|
||||
state.forceInt(*args[0], pos, "while evaluating the first operand of the division");
|
||||
NixInt i2 =
|
||||
state.forceInt(*args[1], pos, "while evaluating the second operand of the division");
|
||||
/* Avoid division overflow as it might raise SIGFPE. */
|
||||
if (i1 == std::numeric_limits<NixInt>::min() && i2 == -1) {
|
||||
state.error<EvalError>("overflow in integer division").atPos(pos).debugThrow();
|
||||
}
|
||||
|
||||
v.mkInt(i1 / i2);
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_div({
|
||||
.name = "__div",
|
||||
.args = {"e1", "e2"},
|
||||
.doc = R"(
|
||||
Return the quotient of the numbers *e1* and *e2*.
|
||||
)",
|
||||
.fun = prim_div,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.floor
|
||||
*/
|
||||
|
||||
static void prim_floor(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto value = state.forceFloat(
|
||||
*args[0],
|
||||
args[0]->determinePos(pos),
|
||||
"while evaluating the first argument passed to builtins.floor"
|
||||
);
|
||||
v.mkInt(floor(value));
|
||||
}
|
||||
|
||||
PrimOp primop_floor({
|
||||
.name = "__floor",
|
||||
.args = {"double"},
|
||||
.doc = R"(
|
||||
Converts an IEEE-754 double-precision floating-point number (*double*) to
|
||||
the next lower integer.
|
||||
|
||||
If the datatype is neither an integer nor a "float", an evaluation error will be
|
||||
thrown.
|
||||
)",
|
||||
.fun = prim_floor,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.lessThan
|
||||
*/
|
||||
|
||||
void prim_lessThan(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceValue(*args[0], pos);
|
||||
state.forceValue(*args[1], pos);
|
||||
// pos is exact here, no need for a message.
|
||||
CompareValues comp(state, noPos, "");
|
||||
v.mkBool(comp(args[0], args[1]));
|
||||
}
|
||||
|
||||
PrimOp primop_lessThan({
|
||||
.name = "__lessThan",
|
||||
.args = {"e1", "e2"},
|
||||
.doc = R"(
|
||||
Return `true` if the number *e1* is less than the number *e2*, and
|
||||
`false` otherwise. Evaluation aborts if either *e1* or *e2* does not
|
||||
evaluate to a number.
|
||||
)",
|
||||
.fun = prim_lessThan,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.mul
|
||||
*/
|
||||
|
||||
static void prim_mul(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceValue(*args[0], pos);
|
||||
state.forceValue(*args[1], pos);
|
||||
if (args[0]->type() == nFloat || args[1]->type() == nFloat) {
|
||||
v.mkFloat(
|
||||
state.forceFloat(*args[0], pos, "while evaluating the first of the multiplication")
|
||||
* state.forceFloat(
|
||||
*args[1], pos, "while evaluating the second argument of the multiplication"
|
||||
)
|
||||
);
|
||||
} else {
|
||||
v.mkInt(
|
||||
state.forceInt(
|
||||
*args[0], pos, "while evaluating the first argument of the multiplication"
|
||||
)
|
||||
* state.forceInt(
|
||||
*args[1], pos, "while evaluating the second argument of the multiplication"
|
||||
)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_mul({
|
||||
.name = "__mul",
|
||||
.args = {"e1", "e2"},
|
||||
.doc = R"(
|
||||
Return the product of the numbers *e1* and *e2*.
|
||||
)",
|
||||
.fun = prim_mul,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.sub
|
||||
*/
|
||||
|
||||
static void prim_sub(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceValue(*args[0], pos);
|
||||
state.forceValue(*args[1], pos);
|
||||
if (args[0]->type() == nFloat || args[1]->type() == nFloat) {
|
||||
v.mkFloat(
|
||||
state.forceFloat(
|
||||
*args[0], pos, "while evaluating the first argument of the subtraction"
|
||||
)
|
||||
- state.forceFloat(
|
||||
*args[1], pos, "while evaluating the second argument of the subtraction"
|
||||
)
|
||||
);
|
||||
} else {
|
||||
v.mkInt(
|
||||
state.forceInt(*args[0], pos, "while evaluating the first argument of the subtraction")
|
||||
- state.forceInt(
|
||||
*args[1], pos, "while evaluating the second argument of the subtraction"
|
||||
)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_sub({
|
||||
.name = "__sub",
|
||||
.args = {"e1", "e2"},
|
||||
.doc = R"(
|
||||
Return the difference between the numbers *e1* and *e2*.
|
||||
)",
|
||||
.fun = prim_sub,
|
||||
});
|
||||
}
|
||||
@@ -1,698 +0,0 @@
|
||||
#include <string.h> // for size_t, memcpy, strcmp
|
||||
#include <algorithm> // for sort, min, set_difference
|
||||
#include <boost/container/small_vector.hpp> // for small_vector
|
||||
#include <boost/container/vector.hpp> // for operator+, operator-
|
||||
#include <iterator> // for back_insert_iterator
|
||||
#include <map> // for map, _Rb_tree_iterator
|
||||
#include <set> // for set
|
||||
#include <string_view> // for string_view, operator<=>
|
||||
#include <utility> // for pair
|
||||
#include <vector> // for vector
|
||||
#include "attr-set.hh" // for Attr, Bindings, Bindings...
|
||||
#include "eval-error.hh" // for TypeError, EvalErrorBuilder
|
||||
#include "eval.hh" // for EvalState, PrimOp, prim_...
|
||||
#include "gc-small-vector.hh" // for SmallValueVector
|
||||
#include "pos-idx.hh" // for PosIdx, noPos
|
||||
#include "primops.hh" // for CompareValues, ValueList
|
||||
#include "symbol-table.hh" // for Symbol, SymbolTable, Sym...
|
||||
#include "value.hh" // for Value, ValueVector, Valu...
|
||||
|
||||
namespace nix {
|
||||
|
||||
Bindings::iterator
|
||||
getAttr(EvalState & state, Symbol attrSym, Bindings * attrSet, std::string_view errorCtx)
|
||||
{
|
||||
Bindings::iterator value = attrSet->find(attrSym);
|
||||
if (value == attrSet->end()) {
|
||||
state.error<TypeError>("attribute '%s' missing", state.symbols[attrSym])
|
||||
.withTrace(noPos, errorCtx)
|
||||
.debugThrow();
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
/**
|
||||
* builtins.attrNames
|
||||
*/
|
||||
|
||||
static void prim_attrNames(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceAttrs(*args[0], pos, "while evaluating the argument passed to builtins.attrNames");
|
||||
|
||||
state.mkList(v, args[0]->attrs->size());
|
||||
|
||||
size_t n = 0;
|
||||
for (auto & i : *args[0]->attrs) {
|
||||
(v.listElems()[n++] = state.allocValue())->mkString(state.symbols[i.name]);
|
||||
}
|
||||
|
||||
std::sort(v.listElems(), v.listElems() + n, [](Value * v1, Value * v2) {
|
||||
return strcmp(v1->string.s, v2->string.s) < 0;
|
||||
});
|
||||
}
|
||||
|
||||
PrimOp primop_attrNames({
|
||||
.name = "__attrNames",
|
||||
.args = {"set"},
|
||||
.doc = R"(
|
||||
Return the names of the attributes in the set *set* in an
|
||||
alphabetically sorted list. For instance, `builtins.attrNames { y
|
||||
= 1; x = "foo"; }` evaluates to `[ "x" "y" ]`.
|
||||
)",
|
||||
.fun = prim_attrNames,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.attrNames
|
||||
*/
|
||||
|
||||
static void prim_attrValues(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceAttrs(*args[0], pos, "while evaluating the argument passed to builtins.attrValues");
|
||||
|
||||
state.mkList(v, args[0]->attrs->size());
|
||||
|
||||
unsigned int n = 0;
|
||||
for (auto & i : *args[0]->attrs) {
|
||||
v.listElems()[n++] = (Value *) &i;
|
||||
}
|
||||
|
||||
std::sort(v.listElems(), v.listElems() + n, [&](Value * v1, Value * v2) {
|
||||
std::string_view s1 = state.symbols[((Attr *) v1)->name],
|
||||
s2 = state.symbols[((Attr *) v2)->name];
|
||||
return s1 < s2;
|
||||
});
|
||||
|
||||
for (unsigned int i = 0; i < n; ++i) {
|
||||
v.listElems()[i] = ((Attr *) v.listElems()[i])->value;
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_attrValues({
|
||||
.name = "__attrValues",
|
||||
.args = {"set"},
|
||||
.doc = R"(
|
||||
Return the values of the attributes in the set *set* in the order
|
||||
corresponding to the sorted attribute names.
|
||||
)",
|
||||
.fun = prim_attrValues,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.catAttrs
|
||||
*/
|
||||
|
||||
static void prim_catAttrs(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto attrName = state.symbols.create(state.forceStringNoCtx(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.catAttrs"
|
||||
));
|
||||
state.forceList(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.catAttrs"
|
||||
);
|
||||
|
||||
SmallValueVector<nonRecursiveStackReservation> res(args[1]->listSize());
|
||||
size_t found = 0;
|
||||
|
||||
for (auto v2 : args[1]->listItems()) {
|
||||
state.forceAttrs(
|
||||
*v2,
|
||||
pos,
|
||||
"while evaluating an element in the list passed as second argument to builtins.catAttrs"
|
||||
);
|
||||
Bindings::iterator i = v2->attrs->find(attrName);
|
||||
if (i != v2->attrs->end()) {
|
||||
res[found++] = i->value;
|
||||
}
|
||||
}
|
||||
|
||||
state.mkList(v, found);
|
||||
for (unsigned int n = 0; n < found; ++n) {
|
||||
v.listElems()[n] = res[n];
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_catAttrs({
|
||||
.name = "__catAttrs",
|
||||
.args = {"attr", "list"},
|
||||
.doc = R"(
|
||||
Collect each attribute named *attr* from a list of attribute
|
||||
sets. Attrsets that don't contain the named attribute are
|
||||
ignored. For example,
|
||||
|
||||
```nix
|
||||
builtins.catAttrs "a" [{a = 1;} {b = 0;} {a = 2;}]
|
||||
```
|
||||
|
||||
evaluates to `[1 2]`.
|
||||
)",
|
||||
.fun = prim_catAttrs,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.genericClosure
|
||||
*/
|
||||
|
||||
static void prim_genericClosure(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceAttrs(
|
||||
*args[0], noPos, "while evaluating the first argument passed to builtins.genericClosure"
|
||||
);
|
||||
|
||||
/* Get the start set. */
|
||||
Bindings::iterator startSet = getAttr(
|
||||
state,
|
||||
state.sStartSet,
|
||||
args[0]->attrs,
|
||||
"in the attrset passed as argument to builtins.genericClosure"
|
||||
);
|
||||
|
||||
state.forceList(
|
||||
*startSet->value,
|
||||
noPos,
|
||||
"while evaluating the 'startSet' attribute passed as argument to builtins.genericClosure"
|
||||
);
|
||||
|
||||
ValueList workSet;
|
||||
for (auto elem : startSet->value->listItems()) {
|
||||
workSet.push_back(elem);
|
||||
}
|
||||
|
||||
if (startSet->value->listSize() == 0) {
|
||||
v = *startSet->value;
|
||||
return;
|
||||
}
|
||||
|
||||
/* Get the operator. */
|
||||
Bindings::iterator op = getAttr(
|
||||
state,
|
||||
state.sOperator,
|
||||
args[0]->attrs,
|
||||
"in the attrset passed as argument to builtins.genericClosure"
|
||||
);
|
||||
state.forceFunction(
|
||||
*op->value,
|
||||
noPos,
|
||||
"while evaluating the 'operator' attribute passed as argument to builtins.genericClosure"
|
||||
);
|
||||
|
||||
/* Construct the closure by applying the operator to elements of
|
||||
`workSet', adding the result to `workSet', continuing until
|
||||
no new elements are found. */
|
||||
ValueList res;
|
||||
// `doneKeys' doesn't need to be a GC root, because its values are
|
||||
// reachable from res.
|
||||
auto cmp = CompareValues(
|
||||
state, noPos, "while comparing the `key` attributes of two genericClosure elements"
|
||||
);
|
||||
std::set<Value *, decltype(cmp)> doneKeys(cmp);
|
||||
while (!workSet.empty()) {
|
||||
Value * e = *(workSet.begin());
|
||||
workSet.pop_front();
|
||||
|
||||
state.forceAttrs(
|
||||
*e,
|
||||
noPos,
|
||||
"while evaluating one of the elements generated by (or initially passed to) "
|
||||
"builtins.genericClosure"
|
||||
);
|
||||
|
||||
Bindings::iterator key = getAttr(
|
||||
state,
|
||||
state.sKey,
|
||||
e->attrs,
|
||||
"in one of the attrsets generated by (or initially passed to) builtins.genericClosure"
|
||||
);
|
||||
state.forceValue(*key->value, noPos);
|
||||
|
||||
if (!doneKeys.insert(key->value).second) {
|
||||
continue;
|
||||
}
|
||||
res.push_back(e);
|
||||
|
||||
/* Call the `operator' function with `e' as argument. */
|
||||
Value newElements;
|
||||
state.callFunction(*op->value, 1, &e, newElements, noPos);
|
||||
state.forceList(
|
||||
newElements,
|
||||
noPos,
|
||||
"while evaluating the return value of the `operator` passed to builtins.genericClosure"
|
||||
);
|
||||
|
||||
/* Add the values returned by the operator to the work set. */
|
||||
for (auto elem : newElements.listItems()) {
|
||||
state.forceValue(*elem, noPos); // "while evaluating one one of the elements returned by
|
||||
// the `operator` passed to builtins.genericClosure");
|
||||
workSet.push_back(elem);
|
||||
}
|
||||
}
|
||||
|
||||
/* Create the result list. */
|
||||
state.mkList(v, res.size());
|
||||
unsigned int n = 0;
|
||||
for (auto & i : res) {
|
||||
v.listElems()[n++] = i;
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_genericClosure(PrimOp{
|
||||
.name = "__genericClosure",
|
||||
.args = {"attrset"},
|
||||
.arity = 1,
|
||||
.doc = R"(
|
||||
Take an *attrset* with values named `startSet` and `operator` in order to
|
||||
return a *list of attrsets* by starting with the `startSet` and recursively
|
||||
applying the `operator` function to each `item`. The *attrsets* in the
|
||||
`startSet` and the *attrsets* produced by `operator` must contain a value
|
||||
named `key` which is comparable. The result is produced by calling `operator`
|
||||
for each `item` with a value for `key` that has not been called yet including
|
||||
newly produced `item`s. The function terminates when no new `item`s are
|
||||
produced. The resulting *list of attrsets* contains only *attrsets* with a
|
||||
unique key. For example,
|
||||
|
||||
```
|
||||
builtins.genericClosure {
|
||||
startSet = [ {key = 5;} ];
|
||||
operator = item: [{
|
||||
key = if (item.key / 2 ) * 2 == item.key
|
||||
then item.key / 2
|
||||
else 3 * item.key + 1;
|
||||
}];
|
||||
}
|
||||
```
|
||||
evaluates to
|
||||
```
|
||||
[ { key = 5; } { key = 16; } { key = 8; } { key = 4; } { key = 2; } { key = 1; } ]
|
||||
```
|
||||
)",
|
||||
.fun = prim_genericClosure,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.getAttr
|
||||
*/
|
||||
|
||||
void prim_getAttr(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto attr = state.forceStringNoCtx(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.getAttr"
|
||||
);
|
||||
state.forceAttrs(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.getAttr"
|
||||
);
|
||||
Bindings::iterator i = getAttr(
|
||||
state,
|
||||
state.symbols.create(attr),
|
||||
args[1]->attrs,
|
||||
"in the attribute set under consideration"
|
||||
);
|
||||
// !!! add to stack trace?
|
||||
if (state.countCalls && i->pos) {
|
||||
state.attrSelects[i->pos]++;
|
||||
}
|
||||
state.forceValue(*i->value, pos);
|
||||
v = *i->value;
|
||||
}
|
||||
|
||||
PrimOp primop_getAttr({
|
||||
.name = "__getAttr",
|
||||
.args = {"s", "set"},
|
||||
.doc = R"(
|
||||
`getAttr` returns the attribute named *s* from *set*. Evaluation
|
||||
aborts if the attribute doesn’t exist. This is a dynamic version of
|
||||
the `.` operator, since *s* is an expression rather than an
|
||||
identifier.
|
||||
)",
|
||||
.fun = prim_getAttr,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.groupBy
|
||||
*/
|
||||
|
||||
static void prim_groupBy(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceFunction(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.groupBy"
|
||||
);
|
||||
state.forceList(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.groupBy"
|
||||
);
|
||||
|
||||
ValueVectorMap attrs;
|
||||
|
||||
for (auto vElem : args[1]->listItems()) {
|
||||
Value res;
|
||||
state.callFunction(*args[0], *vElem, res, pos);
|
||||
auto name = state.forceStringNoCtx(
|
||||
res,
|
||||
pos,
|
||||
"while evaluating the return value of the grouping function passed to builtins.groupBy"
|
||||
);
|
||||
auto sym = state.symbols.create(name);
|
||||
auto vector = attrs.try_emplace(sym, ValueVector()).first;
|
||||
vector->second.push_back(vElem);
|
||||
}
|
||||
|
||||
auto attrs2 = state.buildBindings(attrs.size());
|
||||
|
||||
for (auto & i : attrs) {
|
||||
auto & list = attrs2.alloc(i.first);
|
||||
auto size = i.second.size();
|
||||
state.mkList(list, size);
|
||||
memcpy(list.listElems(), i.second.data(), sizeof(Value *) * size);
|
||||
}
|
||||
|
||||
v.mkAttrs(attrs2.alreadySorted());
|
||||
}
|
||||
|
||||
PrimOp primop_groupBy({
|
||||
.name = "__groupBy",
|
||||
.args = {"f", "list"},
|
||||
.doc = R"(
|
||||
Groups elements of *list* together by the string returned from the
|
||||
function *f* called on each element. It returns an attribute set
|
||||
where each attribute value contains the elements of *list* that are
|
||||
mapped to the same corresponding attribute name returned by *f*.
|
||||
|
||||
For example,
|
||||
|
||||
```nix
|
||||
builtins.groupBy (builtins.substring 0 1) ["foo" "bar" "baz"]
|
||||
```
|
||||
|
||||
evaluates to
|
||||
|
||||
```nix
|
||||
{ b = [ "bar" "baz" ]; f = [ "foo" ]; }
|
||||
```
|
||||
)",
|
||||
.fun = prim_groupBy,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.hasAttr
|
||||
*/
|
||||
|
||||
static void prim_hasAttr(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto attr = state.forceStringNoCtx(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.hasAttr"
|
||||
);
|
||||
state.forceAttrs(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.hasAttr"
|
||||
);
|
||||
v.mkBool(args[1]->attrs->find(state.symbols.create(attr)) != args[1]->attrs->end());
|
||||
}
|
||||
|
||||
PrimOp primop_hasAttr({
|
||||
.name = "__hasAttr",
|
||||
.args = {"s", "set"},
|
||||
.doc = R"(
|
||||
`hasAttr` returns `true` if *set* has an attribute named *s*, and
|
||||
`false` otherwise. This is a dynamic version of the `?` operator,
|
||||
since *s* is an expression rather than an identifier.
|
||||
)",
|
||||
.fun = prim_hasAttr,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.intersectAttrs
|
||||
*/
|
||||
|
||||
static void prim_intersectAttrs(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceAttrs(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.intersectAttrs"
|
||||
);
|
||||
state.forceAttrs(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.intersectAttrs"
|
||||
);
|
||||
|
||||
Bindings & left = *args[0]->attrs;
|
||||
Bindings & right = *args[1]->attrs;
|
||||
|
||||
auto attrs = state.buildBindings(std::min(left.size(), right.size()));
|
||||
|
||||
// The current implementation has good asymptotic complexity and is reasonably
|
||||
// simple. Further optimization may be possible, but does not seem productive,
|
||||
// considering the state of eval performance in 2022.
|
||||
//
|
||||
// I have looked for reusable and/or standard solutions and these are my
|
||||
// findings:
|
||||
//
|
||||
// STL
|
||||
// ===
|
||||
// std::set_intersection is not suitable, as it only performs a simultaneous
|
||||
// linear scan; not taking advantage of random access. This is O(n + m), so
|
||||
// linear in the largest set, which is not acceptable for callPackage in Nixpkgs.
|
||||
//
|
||||
// Simultaneous scan, with alternating simple binary search
|
||||
// ===
|
||||
// One alternative algorithm scans the attrsets simultaneously, jumping
|
||||
// forward using `lower_bound` in case of inequality. This should perform
|
||||
// well on very similar sets, having a local and predictable access pattern.
|
||||
// On dissimilar sets, it seems to need more comparisons than the current
|
||||
// algorithm, as few consecutive attrs match. `lower_bound` could take
|
||||
// advantage of the decreasing remaining search space, but this causes
|
||||
// the medians to move, which can mean that they don't stay in the cache
|
||||
// like they would with the current naive `find`.
|
||||
//
|
||||
// Double binary search
|
||||
// ===
|
||||
// The optimal algorithm may be "Double binary search", which doesn't
|
||||
// scan at all, but rather divides both sets simultaneously.
|
||||
// See "Fast Intersection Algorithms for Sorted Sequences" by Baeza-Yates et al.
|
||||
// https://cs.uwaterloo.ca/~ajsaling/papers/intersection_alg_app10.pdf
|
||||
// The only downsides I can think of are not having a linear access pattern
|
||||
// for similar sets, and having to maintain a more intricate algorithm.
|
||||
//
|
||||
// Adaptive
|
||||
// ===
|
||||
// Finally one could run try a simultaneous scan, count misses and fall back
|
||||
// to double binary search when the counter hit some threshold and/or ratio.
|
||||
|
||||
if (left.size() < right.size()) {
|
||||
for (auto & l : left) {
|
||||
Bindings::iterator r = right.find(l.name);
|
||||
if (r != right.end()) {
|
||||
attrs.insert(*r);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
for (auto & r : right) {
|
||||
Bindings::iterator l = left.find(r.name);
|
||||
if (l != left.end()) {
|
||||
attrs.insert(r);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
v.mkAttrs(attrs.alreadySorted());
|
||||
}
|
||||
|
||||
PrimOp primop_intersectAttrs({
|
||||
.name = "__intersectAttrs",
|
||||
.args = {"e1", "e2"},
|
||||
.doc = R"(
|
||||
Return a set consisting of the attributes in the set *e2* which have the
|
||||
same name as some attribute in *e1*.
|
||||
|
||||
Performs in O(*n* log *m*) where *n* is the size of the smaller set and *m* the larger set's size.
|
||||
)",
|
||||
.fun = prim_intersectAttrs,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.mapAttrs
|
||||
*/
|
||||
|
||||
static void prim_mapAttrs(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceAttrs(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.mapAttrs"
|
||||
);
|
||||
|
||||
auto attrs = state.buildBindings(args[1]->attrs->size());
|
||||
|
||||
for (auto & i : *args[1]->attrs) {
|
||||
Value * vName = state.allocValue();
|
||||
Value * vFun2 = state.allocValue();
|
||||
vName->mkString(state.symbols[i.name]);
|
||||
vFun2->mkApp(args[0], vName);
|
||||
attrs.alloc(i.name).mkApp(vFun2, i.value);
|
||||
}
|
||||
|
||||
v.mkAttrs(attrs.alreadySorted());
|
||||
}
|
||||
|
||||
PrimOp primop_mapAttrs({
|
||||
.name = "__mapAttrs",
|
||||
.args = {"f", "attrset"},
|
||||
.doc = R"(
|
||||
Apply function *f* to every element of *attrset*. For example,
|
||||
|
||||
```nix
|
||||
builtins.mapAttrs (name: value: value * 10) { a = 1; b = 2; }
|
||||
```
|
||||
|
||||
evaluates to `{ a = 10; b = 20; }`.
|
||||
)",
|
||||
.fun = prim_mapAttrs,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.mapAttrs
|
||||
*/
|
||||
|
||||
static void prim_removeAttrs(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceAttrs(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.removeAttrs"
|
||||
);
|
||||
state.forceList(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.removeAttrs"
|
||||
);
|
||||
|
||||
/* Get the attribute names to be removed.
|
||||
We keep them as Attrs instead of Symbols so std::set_difference
|
||||
can be used to remove them from attrs[0]. */
|
||||
// 64: large enough to fit the attributes of a derivation
|
||||
boost::container::small_vector<Attr, 64> names;
|
||||
names.reserve(args[1]->listSize());
|
||||
for (auto elem : args[1]->listItems()) {
|
||||
state.forceStringNoCtx(
|
||||
*elem,
|
||||
pos,
|
||||
"while evaluating the values of the second argument passed to builtins.removeAttrs"
|
||||
);
|
||||
names.emplace_back(state.symbols.create(elem->string.s), nullptr);
|
||||
}
|
||||
std::sort(names.begin(), names.end());
|
||||
|
||||
/* Copy all attributes not in that set. Note that we don't need
|
||||
to sort v.attrs because it's a subset of an already sorted
|
||||
vector. */
|
||||
auto attrs = state.buildBindings(args[0]->attrs->size());
|
||||
std::set_difference(
|
||||
args[0]->attrs->begin(),
|
||||
args[0]->attrs->end(),
|
||||
names.begin(),
|
||||
names.end(),
|
||||
std::back_inserter(attrs)
|
||||
);
|
||||
v.mkAttrs(attrs.alreadySorted());
|
||||
}
|
||||
|
||||
PrimOp primop_removeAttrs({
|
||||
.name = "removeAttrs",
|
||||
.args = {"set", "list"},
|
||||
.doc = R"(
|
||||
Remove the attributes listed in *list* from *set*. The attributes
|
||||
don’t have to exist in *set*. For instance,
|
||||
|
||||
```nix
|
||||
removeAttrs { x = 1; y = 2; z = 3; } [ "a" "x" "z" ]
|
||||
```
|
||||
|
||||
evaluates to `{ y = 2; }`.
|
||||
)",
|
||||
.fun = prim_removeAttrs,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.zipAttrsWith
|
||||
*/
|
||||
|
||||
static void prim_zipAttrsWith(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
// we will first count how many values are present for each given key.
|
||||
// we then allocate a single attrset and pre-populate it with lists of
|
||||
// appropriate sizes, stash the pointers to the list elements of each,
|
||||
// and populate the lists. after that we replace the list in the every
|
||||
// attribute with the merge function application. this way we need not
|
||||
// use (slightly slower) temporary storage the GC does not know about.
|
||||
|
||||
std::map<Symbol, std::pair<size_t, Value **>> attrsSeen;
|
||||
|
||||
state.forceFunction(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.zipAttrsWith"
|
||||
);
|
||||
state.forceList(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.zipAttrsWith"
|
||||
);
|
||||
const auto listSize = args[1]->listSize();
|
||||
const auto listElems = args[1]->listElems();
|
||||
|
||||
for (unsigned int n = 0; n < listSize; ++n) {
|
||||
Value * vElem = listElems[n];
|
||||
state.forceAttrs(
|
||||
*vElem,
|
||||
noPos,
|
||||
"while evaluating a value of the list passed as second argument to "
|
||||
"builtins.zipAttrsWith"
|
||||
);
|
||||
for (auto & attr : *vElem->attrs) {
|
||||
attrsSeen[attr.name].first++;
|
||||
}
|
||||
}
|
||||
|
||||
auto attrs = state.buildBindings(attrsSeen.size());
|
||||
for (auto & [sym, elem] : attrsSeen) {
|
||||
auto & list = attrs.alloc(sym);
|
||||
state.mkList(list, elem.first);
|
||||
elem.second = list.listElems();
|
||||
}
|
||||
v.mkAttrs(attrs.alreadySorted());
|
||||
|
||||
for (unsigned int n = 0; n < listSize; ++n) {
|
||||
Value * vElem = listElems[n];
|
||||
for (auto & attr : *vElem->attrs) {
|
||||
*attrsSeen[attr.name].second++ = attr.value;
|
||||
}
|
||||
}
|
||||
|
||||
for (auto & attr : *v.attrs) {
|
||||
auto name = state.allocValue();
|
||||
name->mkString(state.symbols[attr.name]);
|
||||
auto call1 = state.allocValue();
|
||||
call1->mkApp(args[0], name);
|
||||
auto call2 = state.allocValue();
|
||||
call2->mkApp(call1, attr.value);
|
||||
attr.value = call2;
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_zipAttrsWith({
|
||||
.name = "__zipAttrsWith",
|
||||
.args = {"f", "list"},
|
||||
.doc = R"(
|
||||
Transpose a list of attribute sets into an attribute set of lists,
|
||||
then apply `mapAttrs`.
|
||||
|
||||
`f` receives two arguments: the attribute name and a non-empty
|
||||
list of all values encountered for that attribute name.
|
||||
|
||||
The result is an attribute set where the attribute names are the
|
||||
union of the attribute names in each element of `list`. The attribute
|
||||
values are the return values of `f`.
|
||||
|
||||
```nix
|
||||
builtins.zipAttrsWith
|
||||
(name: values: { inherit name values; })
|
||||
[ { a = "x"; } { a = "y"; b = "z"; } ]
|
||||
```
|
||||
|
||||
evaluates to
|
||||
|
||||
```
|
||||
{
|
||||
a = { name = "a"; values = [ "x" "y" ]; };
|
||||
b = { name = "b"; values = [ "z" ]; };
|
||||
}
|
||||
```
|
||||
)",
|
||||
.fun = prim_zipAttrsWith,
|
||||
});
|
||||
|
||||
}
|
||||
+211
-442
@@ -1,190 +1,142 @@
|
||||
#include <map> // for map, _Rb_tree_const_iterator
|
||||
#include <memory> // for shared_ptr
|
||||
#include <string> // for basic_string, operator<=>, char...
|
||||
#include <utility> // for move, pair
|
||||
#include <variant> // for visit, get_if
|
||||
#include <vector> // for vector
|
||||
#include "attr-set.hh" // for Attr, Bindings, BindingsBuilder
|
||||
#include "config.hh" // for Setting, ExperimentalFeatureSet...
|
||||
#include "derivations.hh" // for isDerivation
|
||||
#include "derived-path.hh" // for DerivedPath, makeConstantStoreP...
|
||||
#include "downstream-placeholder.hh" // for DownstreamPlaceholder
|
||||
#include "error.hh" // for Error
|
||||
#include "eval-error.hh" // for EvalError, EvalErrorBuilder
|
||||
#include "eval-settings.hh" // for EvalSettings, evalSettings
|
||||
#include "eval.hh" // for EvalState, PrimOp
|
||||
#include "experimental-features.hh" // for ExperimentalFeature, Xp
|
||||
#include "fmt.hh" // for HintFmt
|
||||
#include "globals.hh" // for Settings, settings
|
||||
#include "outputs-spec.hh" // for OutputsSpec
|
||||
#include "path.hh" // for StorePath, StorePathSet
|
||||
#include "pos-idx.hh" // for PosIdx, noPos
|
||||
#include "ref.hh" // for ref
|
||||
#include "store-api.hh" // for Store, resolveDerivedPath, copy...
|
||||
#include "symbol-table.hh" // for SymbolStr, Symbol, SymbolTable
|
||||
#include "types.hh" // for StringMap, BackedStringView
|
||||
#include "value.hh" // for Value
|
||||
#include "value/context.hh" // for NixStringContextElem, NixString...
|
||||
#include "primops.hh"
|
||||
#include "eval-inline.hh"
|
||||
#include "derivations.hh"
|
||||
#include "store-api.hh"
|
||||
|
||||
namespace nix {
|
||||
|
||||
StringMap EvalState::realiseContext(const NixStringContext & context)
|
||||
{
|
||||
std::vector<DerivedPath::Built> drvs;
|
||||
StringMap res;
|
||||
|
||||
for (auto & c : context) {
|
||||
auto ensureValid = [&](const StorePath & p) {
|
||||
if (!store->isValidPath(p)) {
|
||||
error<InvalidPathError>(store->printStorePath(p)).debugThrow();
|
||||
}
|
||||
};
|
||||
std::visit(
|
||||
overloaded{
|
||||
[&](const NixStringContextElem::Built & b) {
|
||||
drvs.push_back(DerivedPath::Built{
|
||||
.drvPath = b.drvPath,
|
||||
.outputs = OutputsSpec::Names{b.output},
|
||||
});
|
||||
ensureValid(b.drvPath->getBaseStorePath());
|
||||
},
|
||||
[&](const NixStringContextElem::Opaque & o) {
|
||||
auto ctxS = store->printStorePath(o.path);
|
||||
res.insert_or_assign(ctxS, ctxS);
|
||||
ensureValid(o.path);
|
||||
},
|
||||
[&](const NixStringContextElem::DrvDeep & d) {
|
||||
/* Treat same as Opaque */
|
||||
auto ctxS = store->printStorePath(d.drvPath);
|
||||
res.insert_or_assign(ctxS, ctxS);
|
||||
ensureValid(d.drvPath);
|
||||
},
|
||||
},
|
||||
c.raw
|
||||
);
|
||||
}
|
||||
|
||||
if (drvs.empty()) {
|
||||
return {};
|
||||
}
|
||||
|
||||
if (!evalSettings.enableImportFromDerivation) {
|
||||
error<EvalError>(
|
||||
"cannot build '%1%' during evaluation because the option "
|
||||
"'allow-import-from-derivation' is disabled",
|
||||
drvs.begin()->to_string(*store)
|
||||
)
|
||||
.debugThrow();
|
||||
}
|
||||
|
||||
/* Build/substitute the context. */
|
||||
std::vector<DerivedPath> buildReqs;
|
||||
for (auto & d : drvs) {
|
||||
buildReqs.emplace_back(DerivedPath{d});
|
||||
}
|
||||
buildStore->buildPaths(buildReqs, bmNormal, store);
|
||||
|
||||
StorePathSet outputsToCopyAndAllow;
|
||||
|
||||
for (auto & drv : drvs) {
|
||||
auto outputs = resolveDerivedPath(*buildStore, drv, &*store);
|
||||
for (auto & [outputName, outputPath] : outputs) {
|
||||
outputsToCopyAndAllow.insert(outputPath);
|
||||
|
||||
/* Get all the output paths corresponding to the placeholders we had */
|
||||
if (experimentalFeatureSettings.isEnabled(Xp::CaDerivations)) {
|
||||
res.insert_or_assign(
|
||||
DownstreamPlaceholder::fromSingleDerivedPathBuilt(SingleDerivedPath::Built{
|
||||
.drvPath = drv.drvPath,
|
||||
.output = outputName,
|
||||
})
|
||||
.render(),
|
||||
buildStore->printStorePath(outputPath)
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (store != buildStore) {
|
||||
copyClosure(*buildStore, *store, outputsToCopyAndAllow);
|
||||
}
|
||||
if (allowedPaths) {
|
||||
for (auto & outputPath : outputsToCopyAndAllow) {
|
||||
/* Add the output of this derivations to the allowed
|
||||
paths. */
|
||||
allowPath(outputPath);
|
||||
}
|
||||
}
|
||||
|
||||
return res;
|
||||
}
|
||||
|
||||
/**
|
||||
* builtins.addDrvOutputDependencies
|
||||
*/
|
||||
|
||||
static void
|
||||
prim_addDrvOutputDependencies(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
static void prim_unsafeDiscardStringContext(EvalState & state, const PosIdx pos, Value * * args, Value & v)
|
||||
{
|
||||
NixStringContext context;
|
||||
auto s = state.coerceToString(
|
||||
pos,
|
||||
*args[0],
|
||||
context,
|
||||
"while evaluating the argument passed to builtins.addDrvOutputDependencies"
|
||||
);
|
||||
auto s = state.coerceToString(pos, *args[0], context, "while evaluating the argument passed to builtins.unsafeDiscardStringContext");
|
||||
v.mkString(*s);
|
||||
}
|
||||
|
||||
auto contextSize = context.size();
|
||||
if (contextSize != 1) {
|
||||
state
|
||||
.error<EvalError>(
|
||||
"context of string '%s' must have exactly one element, but has %d", *s, contextSize
|
||||
)
|
||||
.atPos(pos)
|
||||
.debugThrow();
|
||||
static RegisterPrimOp primop_unsafeDiscardStringContext({
|
||||
.name = "__unsafeDiscardStringContext",
|
||||
.arity = 1,
|
||||
.fun = prim_unsafeDiscardStringContext
|
||||
});
|
||||
|
||||
|
||||
static void prim_hasContext(EvalState & state, const PosIdx pos, Value * * args, Value & v)
|
||||
{
|
||||
NixStringContext context;
|
||||
state.forceString(*args[0], context, pos, "while evaluating the argument passed to builtins.hasContext");
|
||||
v.mkBool(!context.empty());
|
||||
}
|
||||
|
||||
static RegisterPrimOp primop_hasContext({
|
||||
.name = "__hasContext",
|
||||
.args = {"s"},
|
||||
.doc = R"(
|
||||
Return `true` if string *s* has a non-empty context.
|
||||
The context can be obtained with
|
||||
[`getContext`](#builtins-getContext).
|
||||
|
||||
> **Example**
|
||||
>
|
||||
> Many operations require a string context to be empty because they are intended only to work with "regular" strings, and also to help users avoid unintentionally losing track of string context elements.
|
||||
> `builtins.hasContext` can help create better domain-specific errors in those case.
|
||||
>
|
||||
> ```nix
|
||||
> name: meta:
|
||||
>
|
||||
> if builtins.hasContext name
|
||||
> then throw "package name cannot contain string context"
|
||||
> else { ${name} = meta; }
|
||||
> ```
|
||||
)",
|
||||
.fun = prim_hasContext
|
||||
});
|
||||
|
||||
|
||||
static void prim_unsafeDiscardOutputDependency(EvalState & state, const PosIdx pos, Value * * args, Value & v)
|
||||
{
|
||||
NixStringContext context;
|
||||
auto s = state.coerceToString(pos, *args[0], context, "while evaluating the argument passed to builtins.unsafeDiscardOutputDependency");
|
||||
|
||||
NixStringContext context2;
|
||||
for (auto && c : context) {
|
||||
if (auto * ptr = std::get_if<NixStringContextElem::DrvDeep>(&c.raw)) {
|
||||
context2.emplace(NixStringContextElem::Opaque {
|
||||
.path = ptr->drvPath
|
||||
});
|
||||
} else {
|
||||
/* Can reuse original item */
|
||||
context2.emplace(std::move(c).raw);
|
||||
}
|
||||
}
|
||||
NixStringContext context2{
|
||||
(NixStringContextElem{std::visit(
|
||||
overloaded{
|
||||
[&](const NixStringContextElem::Opaque & c) -> NixStringContextElem::DrvDeep {
|
||||
if (!c.path.isDerivation()) {
|
||||
state
|
||||
.error<EvalError>(
|
||||
"path '%s' is not a derivation", state.store->printStorePath(c.path)
|
||||
)
|
||||
.atPos(pos)
|
||||
.debugThrow();
|
||||
}
|
||||
return NixStringContextElem::DrvDeep{
|
||||
.drvPath = c.path,
|
||||
};
|
||||
},
|
||||
[&](const NixStringContextElem::Built & c) -> NixStringContextElem::DrvDeep {
|
||||
state
|
||||
.error<EvalError>(
|
||||
"`addDrvOutputDependencies` can only act on derivations, not on a "
|
||||
"derivation output such as '%1%'",
|
||||
c.output
|
||||
)
|
||||
.atPos(pos)
|
||||
.debugThrow();
|
||||
},
|
||||
[&](const NixStringContextElem::DrvDeep & c) -> NixStringContextElem::DrvDeep {
|
||||
/* Reuse original item because we want this to be idempotent. */
|
||||
return std::move(c);
|
||||
},
|
||||
|
||||
v.mkString(*s, context2);
|
||||
}
|
||||
|
||||
static RegisterPrimOp primop_unsafeDiscardOutputDependency({
|
||||
.name = "__unsafeDiscardOutputDependency",
|
||||
.args = {"s"},
|
||||
.doc = R"(
|
||||
Create a copy of the given string where every "derivation deep" string context element is turned into a constant string context element.
|
||||
|
||||
This is the opposite of [`builtins.addDrvOutputDependencies`](#builtins-addDrvOutputDependencies).
|
||||
|
||||
This is unsafe because it allows us to "forget" store objects we would have otherwise refered to with the string context,
|
||||
whereas Nix normally tracks all dependencies consistently.
|
||||
Safe operations "grow" but never "shrink" string contexts.
|
||||
[`builtins.addDrvOutputDependencies`] in contrast is safe because "derivation deep" string context element always refers to the underlying derivation (among many more things).
|
||||
Replacing a constant string context element with a "derivation deep" element is a safe operation that just enlargens the string context without forgetting anything.
|
||||
|
||||
[`builtins.addDrvOutputDependencies`]: #builtins-addDrvOutputDependencies
|
||||
)",
|
||||
.fun = prim_unsafeDiscardOutputDependency
|
||||
});
|
||||
|
||||
|
||||
static void prim_addDrvOutputDependencies(EvalState & state, const PosIdx pos, Value * * args, Value & v)
|
||||
{
|
||||
NixStringContext context;
|
||||
auto s = state.coerceToString(pos, *args[0], context, "while evaluating the argument passed to builtins.addDrvOutputDependencies");
|
||||
|
||||
auto contextSize = context.size();
|
||||
if (contextSize != 1) {
|
||||
state.error<EvalError>(
|
||||
"context of string '%s' must have exactly one element, but has %d",
|
||||
*s,
|
||||
contextSize
|
||||
).atPos(pos).debugThrow();
|
||||
}
|
||||
NixStringContext context2 {
|
||||
(NixStringContextElem { std::visit(overloaded {
|
||||
[&](const NixStringContextElem::Opaque & c) -> NixStringContextElem::DrvDeep {
|
||||
if (!c.path.isDerivation()) {
|
||||
state.error<EvalError>(
|
||||
"path '%s' is not a derivation",
|
||||
state.store->printStorePath(c.path)
|
||||
).atPos(pos).debugThrow();
|
||||
}
|
||||
return NixStringContextElem::DrvDeep {
|
||||
.drvPath = c.path,
|
||||
};
|
||||
},
|
||||
context.begin()->raw
|
||||
)}),
|
||||
[&](const NixStringContextElem::Built & c) -> NixStringContextElem::DrvDeep {
|
||||
state.error<EvalError>(
|
||||
"`addDrvOutputDependencies` can only act on derivations, not on a derivation output such as '%1%'",
|
||||
c.output
|
||||
).atPos(pos).debugThrow();
|
||||
},
|
||||
[&](const NixStringContextElem::DrvDeep & c) -> NixStringContextElem::DrvDeep {
|
||||
/* Reuse original item because we want this to be idempotent. */
|
||||
return std::move(c);
|
||||
},
|
||||
}, context.begin()->raw) }),
|
||||
};
|
||||
|
||||
v.mkString(*s, context2);
|
||||
}
|
||||
|
||||
PrimOp primop_addDrvOutputDependencies(
|
||||
{.name = "__addDrvOutputDependencies",
|
||||
.args = {"s"},
|
||||
.doc = R"(
|
||||
static RegisterPrimOp primop_addDrvOutputDependencies({
|
||||
.name = "__addDrvOutputDependencies",
|
||||
.args = {"s"},
|
||||
.doc = R"(
|
||||
Create a copy of the given string where a single constant string context element is turned into a "derivation deep" string context element.
|
||||
|
||||
The store path that is the constant string context element should point to a valid derivation, and end in `.drv`.
|
||||
@@ -194,153 +146,9 @@ PrimOp primop_addDrvOutputDependencies(
|
||||
|
||||
This is the opposite of [`builtins.unsafeDiscardOutputDependency`](#builtins-unsafeDiscardOutputDependency).
|
||||
)",
|
||||
.fun = prim_addDrvOutputDependencies}
|
||||
);
|
||||
|
||||
/**
|
||||
* builtins.addErrorContext
|
||||
*/
|
||||
|
||||
static void prim_addErrorContext(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
try {
|
||||
state.forceValue(*args[1], pos);
|
||||
v = *args[1];
|
||||
} catch (Error & e) {
|
||||
NixStringContext context;
|
||||
auto message =
|
||||
state
|
||||
.coerceToString(
|
||||
pos,
|
||||
*args[0],
|
||||
context,
|
||||
"while evaluating the error message passed to builtins.addErrorContext",
|
||||
false,
|
||||
false
|
||||
)
|
||||
.toOwned();
|
||||
e.addTrace(nullptr, HintFmt(message));
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_addErrorContext(PrimOp{
|
||||
.name = "__addErrorContext",
|
||||
.arity = 2,
|
||||
.fun = prim_addErrorContext,
|
||||
.fun = prim_addDrvOutputDependencies
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.appendContext
|
||||
*/
|
||||
|
||||
static void prim_appendContext(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
NixStringContext context;
|
||||
auto orig = state.forceString(
|
||||
*args[0],
|
||||
context,
|
||||
noPos,
|
||||
"while evaluating the first argument passed to builtins.appendContext"
|
||||
);
|
||||
|
||||
state.forceAttrs(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.appendContext"
|
||||
);
|
||||
|
||||
auto sPath = state.symbols.create("path");
|
||||
auto sAllOutputs = state.symbols.create("allOutputs");
|
||||
for (auto & i : *args[1]->attrs) {
|
||||
const auto & name = state.symbols[i.name];
|
||||
if (!state.store->isStorePath(name)) {
|
||||
state.error<EvalError>("context key '%s' is not a store path", name)
|
||||
.atPos(i.pos)
|
||||
.debugThrow();
|
||||
}
|
||||
auto namePath = state.store->parseStorePath(name);
|
||||
if (!settings.readOnlyMode) {
|
||||
state.store->ensurePath(namePath);
|
||||
}
|
||||
state.forceAttrs(*i.value, i.pos, "while evaluating the value of a string context");
|
||||
auto iter = i.value->attrs->find(sPath);
|
||||
if (iter != i.value->attrs->end()) {
|
||||
if (state.forceBool(
|
||||
*iter->value,
|
||||
iter->pos,
|
||||
"while evaluating the `path` attribute of a string context"
|
||||
))
|
||||
{
|
||||
context.emplace(NixStringContextElem::Opaque{
|
||||
.path = namePath,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
iter = i.value->attrs->find(sAllOutputs);
|
||||
if (iter != i.value->attrs->end()) {
|
||||
if (state.forceBool(
|
||||
*iter->value,
|
||||
iter->pos,
|
||||
"while evaluating the `allOutputs` attribute of a string context"
|
||||
))
|
||||
{
|
||||
if (!isDerivation(name)) {
|
||||
state
|
||||
.error<EvalError>(
|
||||
"tried to add all-outputs context of %s, which is not a derivation, to "
|
||||
"a string",
|
||||
name
|
||||
)
|
||||
.atPos(i.pos)
|
||||
.debugThrow();
|
||||
}
|
||||
context.emplace(NixStringContextElem::DrvDeep{
|
||||
.drvPath = namePath,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
iter = i.value->attrs->find(state.sOutputs);
|
||||
if (iter != i.value->attrs->end()) {
|
||||
state.forceList(
|
||||
*iter->value,
|
||||
iter->pos,
|
||||
"while evaluating the `outputs` attribute of a string context"
|
||||
);
|
||||
if (iter->value->listSize() && !isDerivation(name)) {
|
||||
state
|
||||
.error<EvalError>(
|
||||
"tried to add derivation output context of %s, which is not a derivation, "
|
||||
"to a string",
|
||||
name
|
||||
)
|
||||
.atPos(i.pos)
|
||||
.debugThrow();
|
||||
}
|
||||
for (auto elem : iter->value->listItems()) {
|
||||
auto outputName = state.forceStringNoCtx(
|
||||
*elem, iter->pos, "while evaluating an output name within a string context"
|
||||
);
|
||||
context.emplace(NixStringContextElem::Built{
|
||||
.drvPath = makeConstantStorePathRef(namePath),
|
||||
.output = std::string{outputName},
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
v.mkString(orig, context);
|
||||
}
|
||||
|
||||
PrimOp primop_appendContext({
|
||||
.name = "__appendContext",
|
||||
.arity = 2,
|
||||
.fun = prim_appendContext,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.getContext
|
||||
*/
|
||||
|
||||
/* Extract the context of a string as a structured Nix value.
|
||||
|
||||
@@ -361,37 +169,31 @@ PrimOp primop_appendContext({
|
||||
Note that for a given path any combination of the above attributes
|
||||
may be present.
|
||||
*/
|
||||
static void prim_getContext(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
static void prim_getContext(EvalState & state, const PosIdx pos, Value * * args, Value & v)
|
||||
{
|
||||
struct ContextInfo
|
||||
{
|
||||
struct ContextInfo {
|
||||
bool path = false;
|
||||
bool allOutputs = false;
|
||||
Strings outputs;
|
||||
};
|
||||
NixStringContext context;
|
||||
state.forceString(
|
||||
*args[0], context, pos, "while evaluating the argument passed to builtins.getContext"
|
||||
);
|
||||
state.forceString(*args[0], context, pos, "while evaluating the argument passed to builtins.getContext");
|
||||
auto contextInfos = std::map<StorePath, ContextInfo>();
|
||||
for (auto && i : context) {
|
||||
std::visit(
|
||||
overloaded{
|
||||
[&](NixStringContextElem::DrvDeep && d) {
|
||||
contextInfos[std::move(d.drvPath)].allOutputs = true;
|
||||
},
|
||||
[&](NixStringContextElem::Built && b) {
|
||||
// FIXME should eventually show string context as is, no
|
||||
// resolving here.
|
||||
auto drvPath = resolveDerivedPath(*state.store, *b.drvPath);
|
||||
contextInfos[std::move(drvPath)].outputs.emplace_back(std::move(b.output));
|
||||
},
|
||||
[&](NixStringContextElem::Opaque && o) {
|
||||
contextInfos[std::move(o.path)].path = true;
|
||||
},
|
||||
std::visit(overloaded {
|
||||
[&](NixStringContextElem::DrvDeep && d) {
|
||||
contextInfos[std::move(d.drvPath)].allOutputs = true;
|
||||
},
|
||||
((NixStringContextElem &&) i).raw
|
||||
);
|
||||
[&](NixStringContextElem::Built && b) {
|
||||
// FIXME should eventually show string context as is, no
|
||||
// resolving here.
|
||||
auto drvPath = resolveDerivedPath(*state.store, *b.drvPath);
|
||||
contextInfos[std::move(drvPath)].outputs.emplace_back(std::move(b.output));
|
||||
},
|
||||
[&](NixStringContextElem::Opaque && o) {
|
||||
contextInfos[std::move(o.path)].path = true;
|
||||
},
|
||||
}, ((NixStringContextElem &&) i).raw);
|
||||
}
|
||||
|
||||
auto attrs = state.buildBindings(contextInfos.size());
|
||||
@@ -400,18 +202,15 @@ static void prim_getContext(EvalState & state, const PosIdx pos, Value ** args,
|
||||
auto sAllOutputs = state.symbols.create("allOutputs");
|
||||
for (const auto & info : contextInfos) {
|
||||
auto infoAttrs = state.buildBindings(3);
|
||||
if (info.second.path) {
|
||||
if (info.second.path)
|
||||
infoAttrs.alloc(sPath).mkBool(true);
|
||||
}
|
||||
if (info.second.allOutputs) {
|
||||
if (info.second.allOutputs)
|
||||
infoAttrs.alloc(sAllOutputs).mkBool(true);
|
||||
}
|
||||
if (!info.second.outputs.empty()) {
|
||||
auto & outputsVal = infoAttrs.alloc(state.sOutputs);
|
||||
state.mkList(outputsVal, info.second.outputs.size());
|
||||
for (const auto & [i, output] : enumerate(info.second.outputs)) {
|
||||
for (const auto & [i, output] : enumerate(info.second.outputs))
|
||||
(outputsVal.listElems()[i] = state.allocValue())->mkString(output);
|
||||
}
|
||||
}
|
||||
attrs.alloc(state.store->printStorePath(info.first)).mkAttrs(infoAttrs);
|
||||
}
|
||||
@@ -419,7 +218,7 @@ static void prim_getContext(EvalState & state, const PosIdx pos, Value ** args,
|
||||
v.mkAttrs(attrs);
|
||||
}
|
||||
|
||||
PrimOp primop_getContext({
|
||||
static RegisterPrimOp primop_getContext({
|
||||
.name = "__getContext",
|
||||
.args = {"s"},
|
||||
.doc = R"(
|
||||
@@ -441,114 +240,84 @@ PrimOp primop_getContext({
|
||||
{ "/nix/store/arhvjaf6zmlyn8vh8fgn55rpwnxq0n7l-a.drv" = { outputs = [ "out" ]; }; }
|
||||
```
|
||||
)",
|
||||
.fun = prim_getContext,
|
||||
.fun = prim_getContext
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.hasContext
|
||||
*/
|
||||
|
||||
static void prim_hasContext(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
/* Append the given context to a given string.
|
||||
|
||||
See the commentary above unsafeGetContext for details of the
|
||||
context representation.
|
||||
*/
|
||||
static void prim_appendContext(EvalState & state, const PosIdx pos, Value * * args, Value & v)
|
||||
{
|
||||
NixStringContext context;
|
||||
state.forceString(
|
||||
*args[0], context, pos, "while evaluating the argument passed to builtins.hasContext"
|
||||
);
|
||||
v.mkBool(!context.empty());
|
||||
}
|
||||
auto orig = state.forceString(*args[0], context, noPos, "while evaluating the first argument passed to builtins.appendContext");
|
||||
|
||||
PrimOp primop_hasContext(
|
||||
{.name = "__hasContext",
|
||||
.args = {"s"},
|
||||
.doc = R"(
|
||||
Return `true` if string *s* has a non-empty context.
|
||||
The context can be obtained with
|
||||
[`getContext`](#builtins-getContext).
|
||||
state.forceAttrs(*args[1], pos, "while evaluating the second argument passed to builtins.appendContext");
|
||||
|
||||
> **Example**
|
||||
>
|
||||
> Many operations require a string context to be empty because they are intended only to work with "regular" strings, and also to help users avoid unintentionally losing track of string context elements.
|
||||
> `builtins.hasContext` can help create better domain-specific errors in those case.
|
||||
>
|
||||
> ```nix
|
||||
> name: meta:
|
||||
>
|
||||
> if builtins.hasContext name
|
||||
> then throw "package name cannot contain string context"
|
||||
> else { ${name} = meta; }
|
||||
> ```
|
||||
)",
|
||||
.fun = prim_hasContext}
|
||||
);
|
||||
auto sPath = state.symbols.create("path");
|
||||
auto sAllOutputs = state.symbols.create("allOutputs");
|
||||
for (auto & i : *args[1]->attrs) {
|
||||
const auto & name = state.symbols[i.name];
|
||||
if (!state.store->isStorePath(name))
|
||||
state.error<EvalError>(
|
||||
"context key '%s' is not a store path",
|
||||
name
|
||||
).atPos(i.pos).debugThrow();
|
||||
auto namePath = state.store->parseStorePath(name);
|
||||
if (!settings.readOnlyMode)
|
||||
state.store->ensurePath(namePath);
|
||||
state.forceAttrs(*i.value, i.pos, "while evaluating the value of a string context");
|
||||
auto iter = i.value->attrs->find(sPath);
|
||||
if (iter != i.value->attrs->end()) {
|
||||
if (state.forceBool(*iter->value, iter->pos, "while evaluating the `path` attribute of a string context"))
|
||||
context.emplace(NixStringContextElem::Opaque {
|
||||
.path = namePath,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* builtins.unsafeDiscardOutputDependency
|
||||
*/
|
||||
iter = i.value->attrs->find(sAllOutputs);
|
||||
if (iter != i.value->attrs->end()) {
|
||||
if (state.forceBool(*iter->value, iter->pos, "while evaluating the `allOutputs` attribute of a string context")) {
|
||||
if (!isDerivation(name)) {
|
||||
state.error<EvalError>(
|
||||
"tried to add all-outputs context of %s, which is not a derivation, to a string",
|
||||
name
|
||||
).atPos(i.pos).debugThrow();
|
||||
}
|
||||
context.emplace(NixStringContextElem::DrvDeep {
|
||||
.drvPath = namePath,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
static void
|
||||
prim_unsafeDiscardOutputDependency(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
NixStringContext context;
|
||||
auto s = state.coerceToString(
|
||||
pos,
|
||||
*args[0],
|
||||
context,
|
||||
"while evaluating the argument passed to builtins.unsafeDiscardOutputDependency"
|
||||
);
|
||||
|
||||
NixStringContext context2;
|
||||
for (auto && c : context) {
|
||||
if (auto * ptr = std::get_if<NixStringContextElem::DrvDeep>(&c.raw)) {
|
||||
context2.emplace(NixStringContextElem::Opaque{.path = ptr->drvPath});
|
||||
} else {
|
||||
/* Can reuse original item */
|
||||
context2.emplace(std::move(c).raw);
|
||||
iter = i.value->attrs->find(state.sOutputs);
|
||||
if (iter != i.value->attrs->end()) {
|
||||
state.forceList(*iter->value, iter->pos, "while evaluating the `outputs` attribute of a string context");
|
||||
if (iter->value->listSize() && !isDerivation(name)) {
|
||||
state.error<EvalError>(
|
||||
"tried to add derivation output context of %s, which is not a derivation, to a string",
|
||||
name
|
||||
).atPos(i.pos).debugThrow();
|
||||
}
|
||||
for (auto elem : iter->value->listItems()) {
|
||||
auto outputName = state.forceStringNoCtx(*elem, iter->pos, "while evaluating an output name within a string context");
|
||||
context.emplace(NixStringContextElem::Built {
|
||||
.drvPath = makeConstantStorePathRef(namePath),
|
||||
.output = std::string { outputName },
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
v.mkString(*s, context2);
|
||||
v.mkString(orig, context);
|
||||
}
|
||||
|
||||
PrimOp primop_unsafeDiscardOutputDependency(
|
||||
{.name = "__unsafeDiscardOutputDependency",
|
||||
.args = {"s"},
|
||||
.doc = R"(
|
||||
Create a copy of the given string where every "derivation deep" string context element is turned into a constant string context element.
|
||||
|
||||
This is the opposite of [`builtins.addDrvOutputDependencies`](#builtins-addDrvOutputDependencies).
|
||||
|
||||
This is unsafe because it allows us to "forget" store objects we would have otherwise refered to with the string context,
|
||||
whereas Nix normally tracks all dependencies consistently.
|
||||
Safe operations "grow" but never "shrink" string contexts.
|
||||
[`builtins.addDrvOutputDependencies`] in contrast is safe because "derivation deep" string context element always refers to the underlying derivation (among many more things).
|
||||
Replacing a constant string context element with a "derivation deep" element is a safe operation that just enlargens the string context without forgetting anything.
|
||||
|
||||
[`builtins.addDrvOutputDependencies`]: #builtins-addDrvOutputDependencies
|
||||
)",
|
||||
.fun = prim_unsafeDiscardOutputDependency}
|
||||
);
|
||||
|
||||
/**
|
||||
* builtins. unsafeDiscardStringContext
|
||||
*/
|
||||
|
||||
static void
|
||||
prim_unsafeDiscardStringContext(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
NixStringContext context;
|
||||
auto s = state.coerceToString(
|
||||
pos,
|
||||
*args[0],
|
||||
context,
|
||||
"while evaluating the argument passed to builtins.unsafeDiscardStringContext"
|
||||
);
|
||||
v.mkString(*s);
|
||||
}
|
||||
|
||||
PrimOp primop_unsafeDiscardStringContext({
|
||||
.name = "__unsafeDiscardStringContext",
|
||||
.arity = 1,
|
||||
.fun = prim_unsafeDiscardStringContext,
|
||||
static RegisterPrimOp primop_appendContext({
|
||||
.name = "__appendContext",
|
||||
.arity = 2,
|
||||
.fun = prim_appendContext
|
||||
});
|
||||
|
||||
}
|
||||
|
||||
@@ -1,164 +0,0 @@
|
||||
#include <list> // for list
|
||||
#include <string> // for basic_string
|
||||
#include "attr-set.hh" // for BindingsBuilder
|
||||
#include "error.hh" // for Trace, Error, lvlInfo, ErrorInfo
|
||||
#include "eval-error.hh" // for Abort, ThrownError, EvalErrorBuilder
|
||||
#include "eval-settings.hh" // for EvalSettings, evalSettings
|
||||
#include "eval.hh" // for EvalState, PrimOp, DebugTrace, ValMap
|
||||
#include "fmt.hh" // for HintFmt
|
||||
#include "pos-idx.hh" // for PosIdx
|
||||
#include "pos-table.hh" // for PosTable
|
||||
#include "ref.hh" // for ref
|
||||
#include "symbol-table.hh" // for Symbol
|
||||
#include "types.hh" // for BackedStringView, MaintainCount
|
||||
#include "value.hh" // for Value
|
||||
#include "value/context.hh" // for NixStringContext
|
||||
|
||||
namespace nix {
|
||||
|
||||
enum class ReplExitStatus;
|
||||
|
||||
/**
|
||||
* builtins.abort
|
||||
*/
|
||||
|
||||
static void prim_abort(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
NixStringContext context;
|
||||
auto s = state
|
||||
.coerceToString(
|
||||
pos,
|
||||
*args[0],
|
||||
context,
|
||||
"while evaluating the error message passed to builtins.abort"
|
||||
)
|
||||
.toOwned();
|
||||
state.error<Abort>("evaluation aborted with the following error message: '%1%'", s)
|
||||
.debugThrow();
|
||||
}
|
||||
|
||||
PrimOp primop_abort({
|
||||
.name = "abort",
|
||||
.args = {"s"},
|
||||
.doc = R"(
|
||||
Abort Nix expression evaluation and print the error message *s*.
|
||||
)",
|
||||
.fun = prim_abort,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.break
|
||||
*/
|
||||
|
||||
static void prim_break(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
if (state.debugRepl && !state.debugTraces.empty()) {
|
||||
auto error = Error(ErrorInfo{
|
||||
.level = lvlInfo,
|
||||
.msg = HintFmt("breakpoint reached"),
|
||||
.pos = state.positions[pos],
|
||||
});
|
||||
|
||||
auto & dt = state.debugTraces.front();
|
||||
state.runDebugRepl(&error, dt.env, dt.expr);
|
||||
}
|
||||
|
||||
// Return the value we were passed.
|
||||
v = *args[0];
|
||||
}
|
||||
|
||||
PrimOp primop_break({
|
||||
.name = "break",
|
||||
.args = {"v"},
|
||||
.doc = R"(
|
||||
In debug mode (enabled using `--debugger`), pause Nix expression evaluation and enter the REPL.
|
||||
Otherwise, return the argument `v`.
|
||||
)",
|
||||
.fun = prim_break,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.throw
|
||||
*/
|
||||
|
||||
static void prim_throw(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
NixStringContext context;
|
||||
auto s =
|
||||
state
|
||||
.coerceToString(
|
||||
pos, *args[0], context, "while evaluating the error message passed to builtin.throw"
|
||||
)
|
||||
.toOwned();
|
||||
state.error<ThrownError>(s).debugThrow();
|
||||
}
|
||||
|
||||
PrimOp primop_throw({
|
||||
.name = "throw",
|
||||
.args = {"s"},
|
||||
.doc = R"(
|
||||
Throw an error message *s*. This usually aborts Nix expression
|
||||
evaluation, but in `nix-env -qa` and other commands that try to
|
||||
evaluate a set of derivations to get information about those
|
||||
derivations, a derivation that throws an error is silently skipped
|
||||
(which is not the case for `abort`).
|
||||
)",
|
||||
.fun = prim_throw,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.tryEval
|
||||
*/
|
||||
|
||||
static void prim_tryEval(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto attrs = state.buildBindings(2);
|
||||
|
||||
/* increment state.trylevel, and decrement it when this function returns. */
|
||||
MaintainCount trylevel(state.trylevel);
|
||||
|
||||
ReplExitStatus (*savedDebugRepl)(ref<EvalState> es, const ValMap & extraEnv) = nullptr;
|
||||
if (state.debugRepl && evalSettings.ignoreExceptionsDuringTry) {
|
||||
/* to prevent starting the repl from exceptions withing a tryEval, null it. */
|
||||
savedDebugRepl = state.debugRepl;
|
||||
state.debugRepl = nullptr;
|
||||
}
|
||||
|
||||
try {
|
||||
state.forceValue(*args[0], pos);
|
||||
attrs.insert(state.sValue, args[0]);
|
||||
attrs.alloc("success").mkBool(true);
|
||||
} catch (AssertionError & e) {
|
||||
attrs.alloc(state.sValue).mkBool(false);
|
||||
attrs.alloc("success").mkBool(false);
|
||||
}
|
||||
|
||||
// restore the debugRepl pointer if we saved it earlier.
|
||||
if (savedDebugRepl) {
|
||||
state.debugRepl = savedDebugRepl;
|
||||
}
|
||||
|
||||
v.mkAttrs(attrs);
|
||||
}
|
||||
|
||||
PrimOp primop_tryEval({
|
||||
.name = "__tryEval",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Try to shallowly evaluate *e*. Return a set containing the
|
||||
attributes `success` (`true` if *e* evaluated successfully,
|
||||
`false` if an error was thrown) and `value`, equalling *e* if
|
||||
successful and `false` otherwise. `tryEval` will only prevent
|
||||
errors created by `throw` or `assert` from being thrown.
|
||||
Errors `tryEval` will not catch are for example those created
|
||||
by `abort` and type errors generated by builtins. Also note that
|
||||
this doesn't evaluate *e* deeply, so `let e = { x = throw ""; };
|
||||
in (builtins.tryEval e).success` will be `true`. Using
|
||||
`builtins.deepSeq` one can get the expected result:
|
||||
`let e = { x = throw ""; }; in
|
||||
(builtins.tryEval (builtins.deepSeq e e)).success` will be
|
||||
`false`.
|
||||
)",
|
||||
.fun = prim_tryEval,
|
||||
});
|
||||
}
|
||||
@@ -1,181 +0,0 @@
|
||||
#include <list> // for list
|
||||
#include <string> // for basic_string
|
||||
#include "attr-set.hh" // for Bindings
|
||||
#include "eval-settings.hh" // for EvalSettings, evalSettings
|
||||
#include "eval.hh" // for EvalState, PrimOp, DebugTrace
|
||||
#include "logging.hh" // for Logger, printError
|
||||
#include "pos-idx.hh" // for PosIdx
|
||||
#include "pos-table.hh" // for PosTable
|
||||
#include "position.hh" // for Pos
|
||||
#include "primops.hh" // for makePositionThunks, prim_second, prim_trace
|
||||
#include "print.hh" // for ValuePrinter
|
||||
#include "symbol-table.hh" // for SymbolTable
|
||||
#include "value.hh" // for Value, nString
|
||||
|
||||
namespace nix {
|
||||
|
||||
// access to exact position information (ie, line and colum numbers) is deferred
|
||||
// due to the cost associated with calculating that information and how rarely
|
||||
// it is used in practice. this is achieved by creating thunks to otherwise
|
||||
// inaccessible primops that are not exposed as __op or under builtins to turn
|
||||
// the internal PosIdx back into a line and column number, respectively. exposing
|
||||
// these primops in any way would at best be not useful and at worst create wildly
|
||||
// indeterministic eval results depending on parse order of files.
|
||||
//
|
||||
// in a simpler world this would instead be implemented as another kind of thunk,
|
||||
// but each type of thunk has an associated runtime cost in the current evaluator.
|
||||
// as with black holes this cost is too high to justify another thunk type to check
|
||||
// for in the very hot path that is forceValue.
|
||||
static struct LazyPosAcessors
|
||||
{
|
||||
PrimOp primop_lineOfPos{
|
||||
.arity = 1,
|
||||
.fun = [](EvalState & state, PosIdx pos, Value ** args, Value & v
|
||||
) { v.mkInt(state.positions[PosIdx(args[0]->integer)].line); }
|
||||
};
|
||||
PrimOp primop_columnOfPos{
|
||||
.arity = 1,
|
||||
.fun = [](EvalState & state, PosIdx pos, Value ** args, Value & v
|
||||
) { v.mkInt(state.positions[PosIdx(args[0]->integer)].column); }
|
||||
};
|
||||
|
||||
Value lineOfPos, columnOfPos;
|
||||
|
||||
LazyPosAcessors()
|
||||
{
|
||||
lineOfPos.mkPrimOp(&primop_lineOfPos);
|
||||
columnOfPos.mkPrimOp(&primop_columnOfPos);
|
||||
}
|
||||
|
||||
void operator()(EvalState & state, const PosIdx pos, Value & line, Value & column)
|
||||
{
|
||||
Value * posV = state.allocValue();
|
||||
posV->mkInt(pos.id);
|
||||
line.mkApp(&lineOfPos, posV);
|
||||
column.mkApp(&columnOfPos, posV);
|
||||
}
|
||||
} makeLazyPosAccessors;
|
||||
|
||||
void makePositionThunks(EvalState & state, const PosIdx pos, Value & line, Value & column)
|
||||
{
|
||||
makeLazyPosAccessors(state, pos, line, column);
|
||||
}
|
||||
|
||||
/**
|
||||
* Takes two arguments and evaluates to the second one. Used as the
|
||||
* builtins.traceVerbose implementation when --trace-verbose is not enabled
|
||||
*/
|
||||
|
||||
void prim_second(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceValue(*args[1], pos);
|
||||
v = *args[1];
|
||||
}
|
||||
|
||||
/**
|
||||
* builtins.deepSeq
|
||||
*/
|
||||
|
||||
static void prim_deepSeq(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceValueDeep(*args[0]);
|
||||
state.forceValue(*args[1], pos);
|
||||
v = *args[1];
|
||||
}
|
||||
|
||||
PrimOp primop_deepSeq({
|
||||
.name = "__deepSeq",
|
||||
.args = {"e1", "e2"},
|
||||
.doc = R"(
|
||||
This is like `seq e1 e2`, except that *e1* is evaluated *deeply*:
|
||||
if it’s a list or set, its elements or attributes are also
|
||||
evaluated recursively.
|
||||
)",
|
||||
.fun = prim_deepSeq,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.seq
|
||||
*/
|
||||
|
||||
static void prim_seq(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceValue(*args[0], pos);
|
||||
state.forceValue(*args[1], pos);
|
||||
v = *args[1];
|
||||
}
|
||||
|
||||
PrimOp primop_seq({
|
||||
.name = "__seq",
|
||||
.args = {"e1", "e2"},
|
||||
.doc = R"(
|
||||
Evaluate *e1*, then evaluate and return *e2*. This ensures that a
|
||||
computation is strict in the value of *e1*.
|
||||
)",
|
||||
.fun = prim_seq,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.trace
|
||||
*/
|
||||
|
||||
void prim_trace(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceValue(*args[0], pos);
|
||||
if (args[0]->type() == nString) {
|
||||
printError("trace: %1%", args[0]->string.s);
|
||||
} else {
|
||||
printError("trace: %1%", ValuePrinter(state, *args[0]));
|
||||
}
|
||||
if (evalSettings.builtinsTraceDebugger && state.debugRepl && !state.debugTraces.empty()) {
|
||||
const DebugTrace & last = state.debugTraces.front();
|
||||
state.runDebugRepl(nullptr, last.env, last.expr);
|
||||
}
|
||||
state.forceValue(*args[1], pos);
|
||||
v = *args[1];
|
||||
}
|
||||
|
||||
PrimOp primop_trace({
|
||||
.name = "__trace",
|
||||
.args = {"e1", "e2"},
|
||||
.doc = R"(
|
||||
Evaluate *e1* and print its abstract syntax representation on
|
||||
standard error. Then return *e2*. This function is useful for
|
||||
debugging.
|
||||
|
||||
If the
|
||||
[`debugger-on-trace`](@docroot@/command-ref/conf-file.md#conf-debugger-on-trace)
|
||||
option is set to `true` and the `--debugger` flag is given, the
|
||||
interactive debugger will be started when `trace` is called (like
|
||||
[`break`](@docroot@/language/builtins.md#builtins-break)).
|
||||
)",
|
||||
.fun = prim_trace,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.unsafeGetAttrPos
|
||||
*/
|
||||
|
||||
static void prim_unsafeGetAttrPos(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto attr = state.forceStringNoCtx(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.unsafeGetAttrPos"
|
||||
);
|
||||
state.forceAttrs(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.unsafeGetAttrPos"
|
||||
);
|
||||
Bindings::iterator i = args[1]->attrs->find(state.symbols.create(attr));
|
||||
if (i == args[1]->attrs->end()) {
|
||||
v.mkNull();
|
||||
} else {
|
||||
state.mkPos(v, i->pos);
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_unsafeGetAttrPos(PrimOp{
|
||||
.name = "__unsafeGetAttrPos",
|
||||
.arity = 2,
|
||||
.fun = prim_unsafeGetAttrPos,
|
||||
});
|
||||
|
||||
}
|
||||
@@ -1,429 +0,0 @@
|
||||
#include <map> // for _Rb_tree_const_ite...
|
||||
#include <memory> // for allocator, shared_ptr
|
||||
#include <nlohmann/json.hpp> // for basic_json
|
||||
#include <nlohmann/json_fwd.hpp> // for json
|
||||
#include <optional> // for optional, operator==
|
||||
#include <set> // for set
|
||||
#include <string> // for char_traits, basic...
|
||||
#include <string_view> // for operator==, basic_...
|
||||
#include <utility> // for move, pair
|
||||
#include <variant> // for visit
|
||||
#include "attr-set.hh" // for Attr, Bindings
|
||||
#include "config.hh" // for ExperimentalFeatur...
|
||||
#include "content-address.hh" // for ContentAddressMethod
|
||||
#include "derivations.hh" // for Derivation, Deriva...
|
||||
#include "derived-path-map.hh" // for DerivedPathMap
|
||||
#include "error.hh" // for Error, lvlChatty
|
||||
#include "eval-error.hh" // for EvalError, EvalErr...
|
||||
#include "eval.hh" // for EvalState, PrimOp
|
||||
#include "experimental-features.hh" // for ExperimentalFeature
|
||||
#include "fmt.hh" // for HintFmt
|
||||
#include "hash.hh" // for parseHashTypeOpt
|
||||
#include "logging.hh" // for Logger, printMsg
|
||||
#include "path.hh" // for StorePath, StorePa...
|
||||
#include "pos-idx.hh" // for PosIdx, noPos
|
||||
#include "pos-table.hh" // for PosTable
|
||||
#include "position.hh" // for Pos, operator<<
|
||||
#include "primops.hh" // for getAttr, mkOutputS...
|
||||
#include "ref.hh" // for ref
|
||||
#include "store-api.hh" // for Store
|
||||
#include "strings.hh" // for tokenizeString
|
||||
#include "symbol-table.hh" // for Symbol, SymbolStr
|
||||
#include "sync.hh" // for Sync
|
||||
#include "types.hh" // for Strings, BackedStr...
|
||||
#include "value-to-json.hh" // for printValueAsJSON
|
||||
#include "value.hh" // for Value, nNull
|
||||
#include "value/context.hh" // for NixStringContextElem
|
||||
|
||||
namespace nix {
|
||||
|
||||
static void derivationStrictInternal(EvalState & state, const std::string &
|
||||
drvName, Bindings * attrs, Value & v)
|
||||
{
|
||||
/* Check whether attributes should be passed as a JSON file. */
|
||||
using nlohmann::json;
|
||||
std::optional<json> jsonObject;
|
||||
auto pos = v.determinePos(noPos);
|
||||
auto attr = attrs->find(state.sStructuredAttrs);
|
||||
if (attr != attrs->end() &&
|
||||
state.forceBool(*attr->value, pos,
|
||||
"while evaluating the `__structuredAttrs` "
|
||||
"attribute passed to builtins.derivationStrict"))
|
||||
jsonObject = json::object();
|
||||
|
||||
/* Check whether null attributes should be ignored. */
|
||||
bool ignoreNulls = false;
|
||||
attr = attrs->find(state.sIgnoreNulls);
|
||||
if (attr != attrs->end())
|
||||
ignoreNulls = state.forceBool(*attr->value, pos, "while evaluating the `__ignoreNulls` attribute " "passed to builtins.derivationStrict");
|
||||
|
||||
/* Build the derivation expression by processing the attributes. */
|
||||
Derivation drv;
|
||||
drv.name = drvName;
|
||||
|
||||
NixStringContext context;
|
||||
|
||||
bool contentAddressed = false;
|
||||
bool isImpure = false;
|
||||
std::optional<std::string> outputHash;
|
||||
std::string outputHashAlgo;
|
||||
std::optional<ContentAddressMethod> ingestionMethod;
|
||||
|
||||
StringSet outputs;
|
||||
outputs.insert("out");
|
||||
|
||||
for (auto & i : attrs->lexicographicOrder(state.symbols)) {
|
||||
if (i->name == state.sIgnoreNulls) continue;
|
||||
const std::string & key = state.symbols[i->name];
|
||||
vomit("processing attribute '%1%'", key);
|
||||
|
||||
auto handleHashMode = [&](const std::string_view s) {
|
||||
if (s == "recursive") ingestionMethod = FileIngestionMethod::Recursive;
|
||||
else if (s == "flat") ingestionMethod = FileIngestionMethod::Flat;
|
||||
else if (s == "text") {
|
||||
experimentalFeatureSettings.require(Xp::DynamicDerivations);
|
||||
ingestionMethod = TextIngestionMethod {};
|
||||
} else
|
||||
state.error<EvalError>(
|
||||
"invalid value '%s' for 'outputHashMode' attribute", s
|
||||
).atPos(v).debugThrow();
|
||||
};
|
||||
|
||||
auto handleOutputs = [&](const Strings & ss) {
|
||||
outputs.clear();
|
||||
for (auto & j : ss) {
|
||||
if (outputs.find(j) != outputs.end())
|
||||
state.error<EvalError>("duplicate derivation output '%1%'", j)
|
||||
.atPos(v)
|
||||
.debugThrow();
|
||||
/* !!! Check whether j is a valid attribute
|
||||
name. */
|
||||
/* Derivations cannot be named ‘drv’, because
|
||||
then we'd have an attribute ‘drvPath’ in
|
||||
the resulting set. */
|
||||
if (j == "drv")
|
||||
state.error<EvalError>("invalid derivation output name 'drv'")
|
||||
.atPos(v)
|
||||
.debugThrow();
|
||||
outputs.insert(j);
|
||||
}
|
||||
if (outputs.empty())
|
||||
state.error<EvalError>("derivation cannot have an empty set of outputs")
|
||||
.atPos(v)
|
||||
.debugThrow();
|
||||
};
|
||||
|
||||
try {
|
||||
// This try-catch block adds context for most errors.
|
||||
// Use this empty error context to signify that we defer to it.
|
||||
const std::string_view context_below("");
|
||||
|
||||
if (ignoreNulls) {
|
||||
state.forceValue(*i->value, pos);
|
||||
if (i->value->type() == nNull) continue;
|
||||
}
|
||||
|
||||
if (i->name == state.sContentAddressed && state.forceBool(*i->value, pos, context_below)) {
|
||||
contentAddressed = true;
|
||||
experimentalFeatureSettings.require(Xp::CaDerivations);
|
||||
}
|
||||
|
||||
else if (i->name == state.sImpure && state.forceBool(*i->value, pos, context_below)) {
|
||||
isImpure = true;
|
||||
experimentalFeatureSettings.require(Xp::ImpureDerivations);
|
||||
}
|
||||
|
||||
/* The `args' attribute is special: it supplies the
|
||||
command-line arguments to the builder. */
|
||||
else if (i->name == state.sArgs) {
|
||||
state.forceList(*i->value, pos, context_below);
|
||||
for (auto elem : i->value->listItems()) {
|
||||
auto s = state.coerceToString(pos, *elem, context,
|
||||
"while evaluating an element of the argument list",
|
||||
true).toOwned();
|
||||
drv.args.push_back(s);
|
||||
}
|
||||
}
|
||||
|
||||
/* All other attributes are passed to the builder through
|
||||
the environment. */
|
||||
else {
|
||||
|
||||
if (jsonObject) {
|
||||
|
||||
if (i->name == state.sStructuredAttrs) continue;
|
||||
|
||||
(*jsonObject)[key] = printValueAsJSON(state, true, *i->value, pos, context);
|
||||
|
||||
if (i->name == state.sBuilder)
|
||||
drv.builder = state.forceString(*i->value, context, pos, context_below);
|
||||
else if (i->name == state.sSystem)
|
||||
drv.platform = state.forceStringNoCtx(*i->value, pos, context_below);
|
||||
else if (i->name == state.sOutputHash)
|
||||
outputHash = state.forceStringNoCtx(*i->value, pos, context_below);
|
||||
else if (i->name == state.sOutputHashAlgo)
|
||||
outputHashAlgo = state.forceStringNoCtx(*i->value, pos, context_below);
|
||||
else if (i->name == state.sOutputHashMode)
|
||||
handleHashMode(state.forceStringNoCtx(*i->value, pos, context_below));
|
||||
else if (i->name == state.sOutputs) {
|
||||
/* Require ‘outputs’ to be a list of strings. */
|
||||
state.forceList(*i->value, pos, context_below);
|
||||
Strings ss;
|
||||
for (auto elem : i->value->listItems())
|
||||
ss.emplace_back(state.forceStringNoCtx(*elem, pos, context_below));
|
||||
handleOutputs(ss);
|
||||
}
|
||||
|
||||
} else {
|
||||
auto s = state.coerceToString(pos, *i->value, context, context_below, true).toOwned();
|
||||
drv.env.emplace(key, s);
|
||||
if (i->name == state.sBuilder) drv.builder = std::move(s);
|
||||
else if (i->name == state.sSystem) drv.platform = std::move(s);
|
||||
else if (i->name == state.sOutputHash) outputHash = std::move(s);
|
||||
else if (i->name == state.sOutputHashAlgo) outputHashAlgo = std::move(s);
|
||||
else if (i->name == state.sOutputHashMode) handleHashMode(s);
|
||||
else if (i->name == state.sOutputs)
|
||||
handleOutputs(tokenizeString<Strings>(s));
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
} catch (Error & e) {
|
||||
e.addTrace(state.positions[i->pos],
|
||||
HintFmt("while evaluating attribute '%1%' of derivation '%2%'", key, drvName));
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
if (jsonObject) {
|
||||
drv.env.emplace("__json", jsonObject->dump());
|
||||
jsonObject.reset();
|
||||
}
|
||||
|
||||
/* Everything in the context of the strings in the derivation
|
||||
attributes should be added as dependencies of the resulting
|
||||
derivation. */
|
||||
for (auto & c : context) {
|
||||
std::visit(overloaded {
|
||||
/* Since this allows the builder to gain access to every
|
||||
path in the dependency graph of the derivation (including
|
||||
all outputs), all paths in the graph must be added to
|
||||
this derivation's list of inputs to ensure that they are
|
||||
available when the builder runs. */
|
||||
[&](const NixStringContextElem::DrvDeep & d) {
|
||||
/* !!! This doesn't work if readOnlyMode is set. */
|
||||
StorePathSet refs;
|
||||
state.store->computeFSClosure(d.drvPath, refs);
|
||||
for (auto & j : refs) {
|
||||
drv.inputSrcs.insert(j);
|
||||
if (j.isDerivation()) {
|
||||
drv.inputDrvs.map[j].value = state.store->readDerivation(j).outputNames();
|
||||
}
|
||||
}
|
||||
},
|
||||
[&](const NixStringContextElem::Built & b) {
|
||||
drv.inputDrvs.ensureSlot(*b.drvPath).value.insert(b.output);
|
||||
},
|
||||
[&](const NixStringContextElem::Opaque & o) {
|
||||
drv.inputSrcs.insert(o.path);
|
||||
},
|
||||
}, c.raw);
|
||||
}
|
||||
|
||||
/* Do we have all required attributes? */
|
||||
if (drv.builder == "")
|
||||
state.error<EvalError>("required attribute 'builder' missing")
|
||||
.atPos(v)
|
||||
.debugThrow();
|
||||
|
||||
if (drv.platform == "")
|
||||
state.error<EvalError>("required attribute 'system' missing")
|
||||
.atPos(v)
|
||||
.debugThrow();
|
||||
|
||||
/* Check whether the derivation name is valid. */
|
||||
if (isDerivation(drvName) &&
|
||||
!(ingestionMethod == ContentAddressMethod { TextIngestionMethod { } } &&
|
||||
outputs.size() == 1 &&
|
||||
*(outputs.begin()) == "out"))
|
||||
{
|
||||
state.error<EvalError>(
|
||||
"derivation names are allowed to end in '%s' only if they produce a single derivation file",
|
||||
drvExtension
|
||||
).atPos(v).debugThrow();
|
||||
}
|
||||
|
||||
if (outputHash) {
|
||||
/* Handle fixed-output derivations.
|
||||
|
||||
Ignore `__contentAddressed` because fixed output derivations are
|
||||
already content addressed. */
|
||||
if (outputs.size() != 1 || *(outputs.begin()) != "out")
|
||||
state.error<EvalError>(
|
||||
"multiple outputs are not supported in fixed-output derivations"
|
||||
).atPos(v).debugThrow();
|
||||
|
||||
auto h = newHashAllowEmpty(*outputHash, parseHashTypeOpt(outputHashAlgo));
|
||||
|
||||
auto method = ingestionMethod.value_or(FileIngestionMethod::Flat);
|
||||
|
||||
DerivationOutput::CAFixed dof {
|
||||
.ca = ContentAddress {
|
||||
.method = std::move(method),
|
||||
.hash = std::move(h),
|
||||
},
|
||||
};
|
||||
|
||||
drv.env["out"] = state.store->printStorePath(dof.path(*state.store, drvName, "out"));
|
||||
drv.outputs.insert_or_assign("out", std::move(dof));
|
||||
}
|
||||
|
||||
else if (contentAddressed || isImpure) {
|
||||
if (contentAddressed && isImpure)
|
||||
state.error<EvalError>("derivation cannot be both content-addressed and impure")
|
||||
.atPos(v).debugThrow();
|
||||
|
||||
auto ht = parseHashTypeOpt(outputHashAlgo).value_or(htSHA256);
|
||||
auto method = ingestionMethod.value_or(FileIngestionMethod::Recursive);
|
||||
|
||||
for (auto & i : outputs) {
|
||||
drv.env[i] = hashPlaceholder(i);
|
||||
if (isImpure)
|
||||
drv.outputs.insert_or_assign(i,
|
||||
DerivationOutput::Impure {
|
||||
.method = method,
|
||||
.hashType = ht,
|
||||
});
|
||||
else
|
||||
drv.outputs.insert_or_assign(i,
|
||||
DerivationOutput::CAFloating {
|
||||
.method = method,
|
||||
.hashType = ht,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
else {
|
||||
/* Compute a hash over the "masked" store derivation, which is
|
||||
the final one except that in the list of outputs, the
|
||||
output paths are empty strings, and the corresponding
|
||||
environment variables have an empty value. This ensures
|
||||
that changes in the set of output names do get reflected in
|
||||
the hash. */
|
||||
for (auto & i : outputs) {
|
||||
drv.env[i] = "";
|
||||
drv.outputs.insert_or_assign(i,
|
||||
DerivationOutput::Deferred { });
|
||||
}
|
||||
|
||||
auto hashModulo = hashDerivationModulo(*state.store, Derivation(drv), true);
|
||||
switch (hashModulo.kind) {
|
||||
case DrvHash::Kind::Regular:
|
||||
for (auto & i : outputs) {
|
||||
auto h = get(hashModulo.hashes, i);
|
||||
if (!h)
|
||||
state.error<AssertionError>(
|
||||
"derivation produced no hash for output '%s'",
|
||||
i
|
||||
).atPos(v).debugThrow();
|
||||
auto outPath = state.store->makeOutputPath(i, *h, drvName);
|
||||
drv.env[i] = state.store->printStorePath(outPath);
|
||||
drv.outputs.insert_or_assign(
|
||||
i,
|
||||
DerivationOutput::InputAddressed {
|
||||
.path = std::move(outPath),
|
||||
});
|
||||
}
|
||||
break;
|
||||
;
|
||||
case DrvHash::Kind::Deferred:
|
||||
for (auto & i : outputs) {
|
||||
drv.outputs.insert_or_assign(i, DerivationOutput::Deferred {});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* Write the resulting term into the Nix store directory. */
|
||||
auto drvPath = writeDerivation(*state.store, drv, state.repair);
|
||||
auto drvPathS = state.store->printStorePath(drvPath);
|
||||
|
||||
printMsg(lvlChatty, "instantiated '%1%' -> '%2%'", drvName, drvPathS);
|
||||
|
||||
/* Optimisation, but required in read-only mode! because in that
|
||||
case we don't actually write store derivations, so we can't
|
||||
read them later. */
|
||||
{
|
||||
auto h = hashDerivationModulo(*state.store, drv, false);
|
||||
drvHashes.lock()->insert_or_assign(drvPath, h);
|
||||
}
|
||||
|
||||
auto result = state.buildBindings(1 + drv.outputs.size());
|
||||
result.alloc(state.sDrvPath).mkString(drvPathS, {
|
||||
NixStringContextElem::DrvDeep { .drvPath = drvPath },
|
||||
});
|
||||
for (auto & i : drv.outputs)
|
||||
mkOutputString(state, result, drvPath, i);
|
||||
|
||||
v.mkAttrs(result);
|
||||
}
|
||||
|
||||
/* Construct (as a unobservable side effect) a Nix derivation
|
||||
expression that performs the derivation described by the argument
|
||||
set. Returns the original set extended with the following
|
||||
attributes: `outPath' containing the primary output path of the
|
||||
derivation; `drvPath' containing the path of the Nix expression;
|
||||
and `type' set to `derivation' to indicate that this is a
|
||||
derivation. */
|
||||
static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * * args, Value & v)
|
||||
{
|
||||
state.forceAttrs(*args[0], pos, "while evaluating the argument passed to builtins.derivationStrict");
|
||||
|
||||
Bindings * attrs = args[0]->attrs;
|
||||
|
||||
/* Figure out the name first (for stack backtraces). */
|
||||
Bindings::iterator nameAttr = getAttr(state, state.sName, attrs, "in the attrset passed as argument to builtins.derivationStrict");
|
||||
|
||||
std::string drvName;
|
||||
try {
|
||||
drvName = state.forceStringNoCtx(*nameAttr->value, pos, "while evaluating the `name` attribute passed to builtins.derivationStrict");
|
||||
} catch (Error & e) {
|
||||
e.addTrace(state.positions[nameAttr->pos], "while evaluating the derivation attribute 'name'");
|
||||
throw;
|
||||
}
|
||||
|
||||
try {
|
||||
derivationStrictInternal(state, drvName, attrs, v);
|
||||
} catch (Error & e) {
|
||||
Pos pos = state.positions[nameAttr->pos];
|
||||
/*
|
||||
* Here we make two abuses of the error system
|
||||
*
|
||||
* 1. We print the location as a string to avoid a code snippet being
|
||||
* printed. While the location of the name attribute is a good hint, the
|
||||
* exact code there is irrelevant.
|
||||
*
|
||||
* 2. We mark this trace as a frame trace, meaning that we stop printing
|
||||
* less important traces from now on. In particular, this prevents the
|
||||
* display of the automatic "while calling builtins.derivationStrict"
|
||||
* trace, which is of little use for the public we target here.
|
||||
*
|
||||
* Please keep in mind that error reporting is done on a best-effort
|
||||
* basis in nix. There is no accurate location for a derivation, as it
|
||||
* often results from the composition of several functions
|
||||
* (derivationStrict, derivation, mkDerivation, mkPythonModule, etc.)
|
||||
*/
|
||||
e.addTrace(nullptr, HintFmt(
|
||||
"while evaluating derivation '%s'\n"
|
||||
" whose name attribute is located at %s",
|
||||
drvName, pos));
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_derivationStrict(PrimOp {
|
||||
.name = "derivationStrict",
|
||||
.arity = 1,
|
||||
.fun = prim_derivationStrict,
|
||||
});
|
||||
|
||||
}
|
||||
@@ -1,26 +1,7 @@
|
||||
#include <boost/optional/optional.hpp> // for get_pointer
|
||||
#include <map> // for map
|
||||
#include <optional> // for optional
|
||||
#include <string> // for allocator, char_tr...
|
||||
#include "attr-set.hh" // for Attr, Bindings
|
||||
#include "environment-variables.hh" // for getEnv
|
||||
#include "error.hh" // for Error, Trace
|
||||
#include "eval.hh" // for EvalState, PrimOp
|
||||
#include "experimental-features.hh" // for ExperimentalFeature
|
||||
#include "fmt.hh" // for HintFmt
|
||||
#include "make-content-addressed.hh" // for makeContentAddressed
|
||||
#include "path-info.hh" // for ValidPathInfo
|
||||
#include "path.hh" // for StorePath
|
||||
#include "pos-idx.hh" // for PosIdx
|
||||
#include "pos-table.hh" // for PosTable
|
||||
#include "realisation.hh" // for RealisedPath
|
||||
#include "ref.hh" // for ref
|
||||
#include "store-api.hh" // for Store, copyClosure
|
||||
#include "strings.hh" // for operator+
|
||||
#include "symbol-table.hh" // for SymbolStr, operator<<
|
||||
#include "url.hh" // for ParsedURL, parseURL
|
||||
#include "value.hh" // for Value, nString
|
||||
#include "value/context.hh" // for NixStringContext
|
||||
#include "primops.hh"
|
||||
#include "store-api.hh"
|
||||
#include "make-content-addressed.hh"
|
||||
#include "url.hh"
|
||||
|
||||
namespace nix {
|
||||
|
||||
@@ -70,7 +51,7 @@ static void runFetchClosureWithRewrite(EvalState & state, const PosIdx pos, Stor
|
||||
throw Error({
|
||||
.msg = HintFmt(
|
||||
"The 'toPath' value '%s' is input-addressed, so it can't possibly be the result of rewriting to a content-addressed path.\n\n"
|
||||
"Set 'toPath' to an empty string to make Nix report the correct content-addressed path.",
|
||||
"Set 'toPath' to an empty string to make Lix report the correct content-addressed path.",
|
||||
state.store->printStorePath(toPath)),
|
||||
.pos = state.positions[pos]
|
||||
});
|
||||
@@ -226,7 +207,7 @@ static void prim_fetchClosure(EvalState & state, const PosIdx pos, Value * * arg
|
||||
runFetchClosureWithContentAddressedPath(state, pos, *fromStore, *fromPath, v);
|
||||
}
|
||||
|
||||
PrimOp primop_fetchClosure({
|
||||
static RegisterPrimOp primop_fetchClosure({
|
||||
.name = "__fetchClosure",
|
||||
.args = {"args"},
|
||||
.doc = R"(
|
||||
@@ -290,7 +271,7 @@ PrimOp primop_fetchClosure({
|
||||
```
|
||||
|
||||
It is possible to fetch an [input-addressed store path](@docroot@/glossary.md#gloss-input-addressed-store-object) and return it as is.
|
||||
However, this is the least preferred way of invoking `fetchClosure`, because it requires that the input-addressed paths are trusted by the Nix configuration.
|
||||
However, this is the least preferred way of invoking `fetchClosure`, because it requires that the input-addressed paths are trusted by the Lix configuration.
|
||||
|
||||
**`builtins.storePath`**
|
||||
|
||||
|
||||
@@ -1,23 +1,10 @@
|
||||
#include <optional> // for optional
|
||||
#include <regex> // for regex_match
|
||||
#include <string> // for allocator, char_traits, basic_string
|
||||
#include <string_view> // for operator==, basic_string_view, string_view
|
||||
#include <utility> // for move
|
||||
#include "attr-set.hh" // for Attr, BindingsBuilder, Bindings
|
||||
#include "attrs.hh" // for Attrs
|
||||
#include "error.hh" // for Error
|
||||
#include "eval-error.hh" // for EvalError, EvalErrorBuilder
|
||||
#include "eval-settings.hh" // for EvalSettings, evalSettings
|
||||
#include "eval.hh" // for EvalState, PrimOp
|
||||
#include "fetchers.hh" // for Input, Tree
|
||||
#include "hash.hh" // for Hash, HashType
|
||||
#include "pos-idx.hh" // for PosIdx
|
||||
#include "ref.hh" // for ref
|
||||
#include "symbol-table.hh" // for Symbol, SymbolStr, SymbolTable
|
||||
#include "types.hh" // for BackedStringView
|
||||
#include "url-parts.hh" // for revRegex
|
||||
#include "value.hh" // for Value, nAttrs
|
||||
#include "value/context.hh" // for NixStringContext
|
||||
#include "primops.hh"
|
||||
#include "eval-inline.hh"
|
||||
#include "eval-settings.hh"
|
||||
#include "store-api.hh"
|
||||
#include "fetchers.hh"
|
||||
#include "url.hh"
|
||||
#include "url-parts.hh"
|
||||
|
||||
namespace nix {
|
||||
|
||||
@@ -96,7 +83,7 @@ static void prim_fetchMercurial(EvalState & state, const PosIdx pos, Value * * a
|
||||
state.allowPath(tree.storePath);
|
||||
}
|
||||
|
||||
PrimOp primop_fetchMercurial({
|
||||
static RegisterPrimOp r_fetchMercurial({
|
||||
.name = "fetchMercurial",
|
||||
.arity = 1,
|
||||
.fun = prim_fetchMercurial
|
||||
|
||||
@@ -1,36 +1,15 @@
|
||||
#include <assert.h> // for assert
|
||||
#include <stdint.h> // for uint64_t
|
||||
#include <ctime> // for gmtime
|
||||
#include <iomanip> // for put_time, operator<<
|
||||
#include <map> // for operator==, map
|
||||
#include <optional> // for optional, operator==
|
||||
#include <regex> // for regex_match, regex...
|
||||
#include <string> // for char_traits, basic...
|
||||
#include <string_view> // for operator==, basic_...
|
||||
#include <utility> // for move, pair
|
||||
#include "attr-set.hh" // for Attr, BindingsBuilder
|
||||
#include "attrs.hh" // for maybeGetStrAttr
|
||||
#include "config.hh" // for Setting, Experimen...
|
||||
#include "content-address.hh" // for FileIngestionMethod
|
||||
#include "eval-error.hh" // for EvalError, EvalErr...
|
||||
#include "eval-settings.hh" // for EvalSettings, eval...
|
||||
#include "eval.hh" // for EvalState, PrimOp
|
||||
#include "experimental-features.hh" // for ExperimentalFeature
|
||||
#include "fetchers.hh" // for Input, Tree, downl...
|
||||
#include "file-system.hh" // for baseNameOf
|
||||
#include "fmt.hh" // for fmt
|
||||
#include "hash.hh" // for Hash, HashType, Base
|
||||
#include "path-info.hh" // for ValidPathInfo
|
||||
#include "path.hh" // for StorePath
|
||||
#include "pos-idx.hh" // for PosIdx
|
||||
#include "ref.hh" // for ref
|
||||
#include "registry.hh" // for lookupInRegistries
|
||||
#include "store-api.hh" // for Store
|
||||
#include "symbol-table.hh" // for Symbol, SymbolTable
|
||||
#include "types.hh" // for BackedStringView
|
||||
#include "url.hh" // for ParsedURL
|
||||
#include "value.hh" // for Value, nAttrs, nBool
|
||||
#include "value/context.hh" // for NixStringContext
|
||||
#include "primops.hh"
|
||||
#include "eval-inline.hh"
|
||||
#include "eval-settings.hh"
|
||||
#include "store-api.hh"
|
||||
#include "fetchers.hh"
|
||||
#include "filetransfer.hh"
|
||||
#include "registry.hh"
|
||||
#include "url.hh"
|
||||
|
||||
#include <ctime>
|
||||
#include <iomanip>
|
||||
#include <regex>
|
||||
|
||||
namespace nix {
|
||||
|
||||
@@ -219,7 +198,7 @@ static void prim_fetchTree(EvalState & state, const PosIdx pos, Value * * args,
|
||||
}
|
||||
|
||||
// FIXME: document
|
||||
PrimOp primop_fetchTree({
|
||||
static RegisterPrimOp primop_fetchTree({
|
||||
.name = "fetchTree",
|
||||
.arity = 1,
|
||||
.fun = prim_fetchTree
|
||||
@@ -311,7 +290,7 @@ static void prim_fetchurl(EvalState & state, const PosIdx pos, Value * * args, V
|
||||
fetch(state, pos, args, v, "fetchurl", false, "");
|
||||
}
|
||||
|
||||
PrimOp primop_fetchurl({
|
||||
static RegisterPrimOp primop_fetchurl({
|
||||
.name = "__fetchurl",
|
||||
.args = {"url"},
|
||||
.doc = R"(
|
||||
@@ -327,7 +306,7 @@ static void prim_fetchTarball(EvalState & state, const PosIdx pos, Value * * arg
|
||||
fetch(state, pos, args, v, "fetchTarball", true, "source");
|
||||
}
|
||||
|
||||
PrimOp primop_fetchTarball({
|
||||
static RegisterPrimOp primop_fetchTarball({
|
||||
.name = "fetchTarball",
|
||||
.args = {"args"},
|
||||
.doc = R"(
|
||||
@@ -377,7 +356,7 @@ static void prim_fetchGit(EvalState & state, const PosIdx pos, Value * * args, V
|
||||
fetchTree(state, pos, args, v, "git", FetchTreeParams { .emptyRevFallback = true, .allowNameArgument = true });
|
||||
}
|
||||
|
||||
PrimOp primop_fetchGit({
|
||||
static RegisterPrimOp primop_fetchGit({
|
||||
.name = "fetchGit",
|
||||
.args = {"args"},
|
||||
.doc = R"(
|
||||
|
||||
@@ -1,27 +1,8 @@
|
||||
#include <stddef.h> // for size_t
|
||||
#include <stdint.h> // for int64_t
|
||||
#include <exception> // for exception
|
||||
#include <functional> // for function
|
||||
#include <sstream> // for char_traits, basic_ostringstream
|
||||
#include <stdexcept> // for runtime_error
|
||||
#include <string> // for basic_string, operator<<, operat...
|
||||
#include <toml/comments.hpp> // for operator<<
|
||||
#include <toml/datetime.hpp> // for operator<<
|
||||
#include <toml/get.hpp> // for get
|
||||
#include <toml/parser.hpp> // for parse
|
||||
#include <toml/serializer.hpp> // for operator<<
|
||||
#include <toml/types.hpp> // for value_t
|
||||
#include <toml/value.hpp> // for value, table
|
||||
#include <unordered_map> // for unordered_map, _Node_iterator
|
||||
#include <utility> // for pair
|
||||
#include <vector> // for vector
|
||||
#include "attr-set.hh" // for BindingsBuilder
|
||||
#include "config.hh" // for ExperimentalFeatureSettings, exp...
|
||||
#include "eval-error.hh" // for EvalError, EvalErrorBuilder
|
||||
#include "eval.hh" // for EvalState, PrimOp
|
||||
#include "experimental-features.hh" // for ExperimentalFeature, Xp
|
||||
#include "pos-idx.hh" // for PosIdx
|
||||
#include "value.hh" // for Value, NixFloat
|
||||
#include "primops.hh"
|
||||
#include "eval-inline.hh"
|
||||
|
||||
#include <sstream>
|
||||
#include <toml.hpp>
|
||||
|
||||
namespace nix {
|
||||
|
||||
@@ -105,7 +86,7 @@ static void prim_fromTOML(EvalState & state, const PosIdx pos, Value * * args, V
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_fromTOML({
|
||||
static RegisterPrimOp primop_fromTOML({
|
||||
.name = "fromTOML",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
|
||||
@@ -1,75 +0,0 @@
|
||||
#include "hash.hh"
|
||||
#include "eval-error.hh" // for EvalError, EvalErrorBuilder
|
||||
#include "eval.hh" // for EvalState, PrimOp
|
||||
#include "pos-idx.hh" // for PosIdx
|
||||
#include "primops.hh" // for realisePath
|
||||
#include "source-path.hh" // for SourcePath
|
||||
#include "value.hh" // for Value
|
||||
#include "value/context.hh" // for NixStringContext
|
||||
|
||||
namespace nix {
|
||||
|
||||
/**
|
||||
* builtins.hashFile
|
||||
*/
|
||||
|
||||
static void prim_hashFile(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto type = state.forceStringNoCtx(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.hashFile"
|
||||
);
|
||||
std::optional<HashType> ht = parseHashType(type);
|
||||
if (!ht) {
|
||||
state.error<EvalError>("unknown hash type '%1%'", type).atPos(pos).debugThrow();
|
||||
}
|
||||
|
||||
auto path = realisePath(state, pos, *args[1]);
|
||||
|
||||
v.mkString(hashString(*ht, path.readFile()).to_string(Base16, false));
|
||||
}
|
||||
|
||||
PrimOp primop_hashFile({
|
||||
.name = "__hashFile",
|
||||
.args = {"type", "p"},
|
||||
.doc = R"(
|
||||
Return a base-16 representation of the cryptographic hash of the
|
||||
file at path *p*. The hash algorithm specified by *type* must be one
|
||||
of `"md5"`, `"sha1"`, `"sha256"` or `"sha512"`.
|
||||
)",
|
||||
.fun = prim_hashFile,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.hashString
|
||||
*/
|
||||
|
||||
static void prim_hashString(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto type = state.forceStringNoCtx(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.hashString"
|
||||
);
|
||||
std::optional<HashType> ht = parseHashType(type);
|
||||
if (!ht) {
|
||||
state.error<EvalError>("unknown hash algorithm '%1%'", type).atPos(pos).debugThrow();
|
||||
}
|
||||
|
||||
NixStringContext context; // discarded
|
||||
auto s = state.forceString(
|
||||
*args[1], context, pos, "while evaluating the second argument passed to builtins.hashString"
|
||||
);
|
||||
|
||||
v.mkString(hashString(*ht, s).to_string(Base16, false));
|
||||
}
|
||||
|
||||
PrimOp primop_hashString({
|
||||
.name = "__hashString",
|
||||
.args = {"type", "s"},
|
||||
.doc = R"(
|
||||
Return a base-16 representation of the cryptographic hash of string
|
||||
*s*. The hash algorithm specified by *type* must be one of `"md5"`,
|
||||
`"sha1"`, `"sha256"` or `"sha512"`.
|
||||
)",
|
||||
.fun = prim_hashString,
|
||||
});
|
||||
|
||||
}
|
||||
@@ -1,290 +0,0 @@
|
||||
#include <dlfcn.h> // for dlerror, dlopen, dlsym, RTLD_LAZY, RTLD_...
|
||||
#include <memory> // for allocator, __shared_ptr_access, shared_ptr
|
||||
#include <optional> // for optional, nullopt, nullopt_t
|
||||
#include <string> // for basic_string, char_traits, string, opera...
|
||||
#include <utility> // for pair
|
||||
#include "attr-set.hh" // for BindingsBuilder, Bindings, Attr
|
||||
#include "canon-path.hh" // for CanonPath
|
||||
#include "derivations.hh" // for Derivation, DerivationOutput, isDerivation
|
||||
#include "derived-path.hh" // for makeConstantStorePathRef, SingleDerivedPath
|
||||
#include "eval-error.hh" // for EvalError, EvalErrorBuilder
|
||||
#include "eval.hh" // for EvalState, PrimOp, Env, resolveExprPath
|
||||
#include "logging.hh" // for Logger, debug
|
||||
#include "nixexpr.hh" // for StaticEnv, Expr
|
||||
#include "path.hh" // for StorePath
|
||||
#include "pos-idx.hh" // for PosIdx
|
||||
#include "primops.hh" // for realisePath, prim_importNative, mkOutput...
|
||||
#include "ref.hh" // for ref
|
||||
#include "source-path.hh" // for SourcePath
|
||||
#include "store-api.hh" // for Store
|
||||
#include "symbol-table.hh" // for Symbol
|
||||
#include "types.hh" // for enumerate
|
||||
#include "value.hh" // for Value, allocRootValue
|
||||
#include "value/context.hh" // for NixStringContextElem
|
||||
|
||||
namespace nix {
|
||||
|
||||
/* Want reasonable symbol names, so extern C */
|
||||
/* !!! Should we pass the Pos or the file name too? */
|
||||
extern "C" typedef void (*ValueInitializer)(EvalState & state, Value & v);
|
||||
|
||||
/**
|
||||
* Add and attribute to the given attribute map from the output name to
|
||||
* the output path, or a placeholder.
|
||||
*
|
||||
* Where possible the path is used, but for floating CA derivations we
|
||||
* may not know it. For sake of determinism we always assume we don't
|
||||
* and instead put in a place holder. In either case, however, the
|
||||
* string context will contain the drv path and output name, so
|
||||
* downstream derivations will have the proper dependency, and in
|
||||
* addition, before building, the placeholder will be rewritten to be
|
||||
* the actual path.
|
||||
*
|
||||
* The 'drv' and 'drvPath' outputs must correspond.
|
||||
*/
|
||||
|
||||
void mkOutputString(
|
||||
EvalState & state,
|
||||
BindingsBuilder & attrs,
|
||||
const StorePath & drvPath,
|
||||
const std::pair<std::string, DerivationOutput> & o
|
||||
)
|
||||
{
|
||||
state.mkOutputString(
|
||||
attrs.alloc(o.first),
|
||||
SingleDerivedPath::Built{
|
||||
.drvPath = makeConstantStorePathRef(drvPath),
|
||||
.output = o.first,
|
||||
},
|
||||
o.second.path(*state.store, Derivation::nameFromPath(drvPath), o.first)
|
||||
);
|
||||
}
|
||||
|
||||
/* Load and evaluate an expression from path specified by the
|
||||
argument. */
|
||||
static void import(EvalState & state, const PosIdx pos, Value & vPath, Value * vScope, Value & v)
|
||||
{
|
||||
auto path = realisePath(state, pos, vPath);
|
||||
auto path2 = path.path.abs();
|
||||
|
||||
// FIXME
|
||||
auto isValidDerivationInStore = [&]() -> std::optional<StorePath> {
|
||||
if (!state.store->isStorePath(path2)) {
|
||||
return std::nullopt;
|
||||
}
|
||||
auto storePath = state.store->parseStorePath(path2);
|
||||
if (!(state.store->isValidPath(storePath) && isDerivation(path2))) {
|
||||
return std::nullopt;
|
||||
}
|
||||
return storePath;
|
||||
};
|
||||
|
||||
if (auto storePath = isValidDerivationInStore()) {
|
||||
Derivation drv = state.store->readDerivation(*storePath);
|
||||
auto attrs = state.buildBindings(3 + drv.outputs.size());
|
||||
attrs.alloc(state.sDrvPath)
|
||||
.mkString(
|
||||
path2,
|
||||
{
|
||||
NixStringContextElem::DrvDeep{.drvPath = *storePath},
|
||||
}
|
||||
);
|
||||
attrs.alloc(state.sName).mkString(drv.env["name"]);
|
||||
auto & outputsVal = attrs.alloc(state.sOutputs);
|
||||
state.mkList(outputsVal, drv.outputs.size());
|
||||
|
||||
for (const auto & [i, o] : enumerate(drv.outputs)) {
|
||||
mkOutputString(state, attrs, *storePath, o);
|
||||
(outputsVal.listElems()[i] = state.allocValue())->mkString(o.first);
|
||||
}
|
||||
|
||||
auto w = state.allocValue();
|
||||
w->mkAttrs(attrs);
|
||||
|
||||
if (!state.vImportedDrvToDerivation) {
|
||||
state.vImportedDrvToDerivation = allocRootValue(state.allocValue());
|
||||
state.eval(
|
||||
state.parseExprFromString(
|
||||
#include "imported-drv-to-derivation.nix.gen.hh"
|
||||
, CanonPath::root
|
||||
),
|
||||
**state.vImportedDrvToDerivation
|
||||
);
|
||||
}
|
||||
|
||||
state.forceFunction(
|
||||
**state.vImportedDrvToDerivation,
|
||||
pos,
|
||||
"while evaluating imported-drv-to-derivation.nix.gen.hh"
|
||||
);
|
||||
v.mkApp(*state.vImportedDrvToDerivation, w);
|
||||
state.forceAttrs(v, pos, "while calling imported-drv-to-derivation.nix.gen.hh");
|
||||
}
|
||||
|
||||
else if (path2 == corepkgsPrefix + "fetchurl.nix")
|
||||
{
|
||||
state.eval(
|
||||
state.parseExprFromString(
|
||||
#include "fetchurl.nix.gen.hh"
|
||||
, CanonPath::root
|
||||
),
|
||||
v
|
||||
);
|
||||
}
|
||||
|
||||
else
|
||||
{
|
||||
if (!vScope) {
|
||||
state.evalFile(path, v);
|
||||
} else {
|
||||
state.forceAttrs(
|
||||
*vScope, pos, "while evaluating the first argument passed to builtins.scopedImport"
|
||||
);
|
||||
|
||||
Env * env = &state.allocEnv(vScope->attrs->size());
|
||||
env->up = &state.baseEnv;
|
||||
|
||||
auto staticEnv = std::make_shared<StaticEnv>(
|
||||
nullptr, state.staticBaseEnv.get(), vScope->attrs->size()
|
||||
);
|
||||
|
||||
unsigned int displ = 0;
|
||||
for (auto & attr : *vScope->attrs) {
|
||||
staticEnv->vars.emplace_back(attr.name, displ);
|
||||
env->values[displ++] = attr.value;
|
||||
}
|
||||
|
||||
// No need to call staticEnv.sort(), because
|
||||
// args[0]->attrs is already sorted.
|
||||
|
||||
debug("evaluating file '%1%'", path);
|
||||
Expr * e = state.parseExprFromFile(resolveExprPath(path), staticEnv);
|
||||
|
||||
e->eval(state, *env, v);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static void prim_import(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
import(state, pos, *args[0], nullptr, v);
|
||||
}
|
||||
|
||||
PrimOp primop_import({
|
||||
.name = "import",
|
||||
.args = {"path"},
|
||||
// TODO turn "normal path values" into link below
|
||||
.doc = R"(
|
||||
Load, parse and return the Nix expression in the file *path*.
|
||||
|
||||
The value *path* can be a path, a string, or an attribute set with an
|
||||
`__toString` attribute or a `outPath` attribute (as derivations or flake
|
||||
inputs typically have).
|
||||
|
||||
If *path* is a directory, the file `default.nix` in that directory
|
||||
is loaded.
|
||||
|
||||
Evaluation aborts if the file doesn’t exist or contains
|
||||
an incorrect Nix expression. `import` implements Nix’s module
|
||||
system: you can put any Nix expression (such as a set or a
|
||||
function) in a separate file, and use it from Nix expressions in
|
||||
other files.
|
||||
|
||||
> **Note**
|
||||
>
|
||||
> Unlike some languages, `import` is a regular function in Nix.
|
||||
> Paths using the angle bracket syntax (e.g., `import` *\<foo\>*)
|
||||
> are normal [path values](@docroot@/language/values.md#type-path).
|
||||
|
||||
A Nix expression loaded by `import` must not contain any *free
|
||||
variables* (identifiers that are not defined in the Nix expression
|
||||
itself and are not built-in). Therefore, it cannot refer to
|
||||
variables that are in scope at the call site. For instance, if you
|
||||
have a calling expression
|
||||
|
||||
```nix
|
||||
rec {
|
||||
x = 123;
|
||||
y = import ./foo.nix;
|
||||
}
|
||||
```
|
||||
|
||||
then the following `foo.nix` will give an error:
|
||||
|
||||
```nix
|
||||
x + 456
|
||||
```
|
||||
|
||||
since `x` is not in scope in `foo.nix`. If you want `x` to be
|
||||
available in `foo.nix`, you should pass it as a function argument:
|
||||
|
||||
```nix
|
||||
rec {
|
||||
x = 123;
|
||||
y = import ./foo.nix x;
|
||||
}
|
||||
```
|
||||
|
||||
and
|
||||
|
||||
```nix
|
||||
x: x + 456
|
||||
```
|
||||
|
||||
(The function argument doesn’t have to be called `x` in `foo.nix`;
|
||||
any name would work.)
|
||||
)",
|
||||
.fun = prim_import,
|
||||
});
|
||||
|
||||
/* Load a ValueInitializer from a DSO and return whatever it initializes */
|
||||
void prim_importNative(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto path = realisePath(state, pos, *args[0]);
|
||||
|
||||
std::string sym(state.forceStringNoCtx(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.importNative"
|
||||
));
|
||||
|
||||
void * handle = dlopen(path.path.c_str(), RTLD_LAZY | RTLD_LOCAL);
|
||||
if (!handle) {
|
||||
state.error<EvalError>("could not open '%1%': %2%", path, dlerror()).debugThrow();
|
||||
}
|
||||
|
||||
dlerror();
|
||||
ValueInitializer func = (ValueInitializer) dlsym(handle, sym.c_str());
|
||||
if (!func) {
|
||||
char * message = dlerror();
|
||||
if (message) {
|
||||
state
|
||||
.error<EvalError>("could not load symbol '%1%' from '%2%': %3%", sym, path, message)
|
||||
.debugThrow();
|
||||
} else {
|
||||
state
|
||||
.error<EvalError>(
|
||||
"symbol '%1%' from '%2%' resolved to NULL when a function pointer was expected",
|
||||
sym,
|
||||
path
|
||||
)
|
||||
.debugThrow();
|
||||
}
|
||||
}
|
||||
|
||||
(func)(state, v);
|
||||
|
||||
/* We don't dlclose because v may be a primop referencing a function in the shared object file
|
||||
*/
|
||||
}
|
||||
|
||||
static void prim_scopedImport(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
import(state, pos, *args[1], args[0], v);
|
||||
}
|
||||
|
||||
PrimOp primop_scopedImport(PrimOp{
|
||||
.name = "scopedImport",
|
||||
.arity = 2,
|
||||
.fun = prim_scopedImport,
|
||||
});
|
||||
|
||||
}
|
||||
@@ -1,70 +0,0 @@
|
||||
#include <sstream> // for basic_ostringstream, char_traits, ostrin...
|
||||
#include "eval.hh" // for PrimOp, EvalState
|
||||
#include "json-to-value.hh" // for JSONParseError, parseJSON
|
||||
#include "pos-idx.hh" // for PosIdx
|
||||
#include "pos-table.hh" // for PosTable
|
||||
#include "value-to-json.hh" // for printValueAsJSON
|
||||
#include "value.hh" // for Value
|
||||
#include "value/context.hh" // for NixStringContext
|
||||
|
||||
namespace nix {
|
||||
|
||||
/**
|
||||
* builtins.fromJSON
|
||||
*/
|
||||
|
||||
static void prim_fromJSON(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto s = state.forceStringNoCtx(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.fromJSON"
|
||||
);
|
||||
try {
|
||||
parseJSON(state, s, v);
|
||||
} catch (JSONParseError & e) {
|
||||
e.addTrace(state.positions[pos], "while decoding a JSON string");
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_fromJSON({
|
||||
.name = "__fromJSON",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Convert a JSON string to a Nix value. For example,
|
||||
|
||||
```nix
|
||||
builtins.fromJSON ''{"x": [1, 2, 3], "y": null}''
|
||||
```
|
||||
|
||||
returns the value `{ x = [ 1 2 3 ]; y = null; }`.
|
||||
)",
|
||||
.fun = prim_fromJSON,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.toJSON
|
||||
*/
|
||||
|
||||
static void prim_toJSON(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
std::ostringstream out;
|
||||
NixStringContext context;
|
||||
printValueAsJSON(state, true, *args[0], pos, out, context);
|
||||
v.mkString(out.str(), context);
|
||||
}
|
||||
|
||||
PrimOp primop_toJSON({
|
||||
.name = "__toJSON",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Return a string containing a JSON representation of *e*. Strings,
|
||||
integers, floats, booleans, nulls and lists are mapped to their JSON
|
||||
equivalents. Sets (except derivations) are represented as objects.
|
||||
Derivations are translated to a JSON string containing the
|
||||
derivation’s output path. Paths are copied to the store and
|
||||
represented as a JSON string of the resulting store path.
|
||||
)",
|
||||
.fun = prim_toJSON,
|
||||
});
|
||||
|
||||
}
|
||||
@@ -1,694 +0,0 @@
|
||||
#include <string.h> // for memcpy, size_t
|
||||
#include <algorithm> // for stable_sort
|
||||
#include <boost/container/vector.hpp> // for vector
|
||||
#include <functional> // for function
|
||||
#include <set> // for set
|
||||
#include <string> // for basic_string, allocator, opera...
|
||||
#include <utility> // for pair
|
||||
#include "attr-set.hh" // for BindingsBuilder, Bindings
|
||||
#include "eval-error.hh" // for EvalError, EvalErrorBuilder
|
||||
#include "eval.hh" // for EvalState, PrimOp
|
||||
#include "gc-small-vector.hh" // for SmallTemporaryValueVector, Sma...
|
||||
#include "pos-idx.hh" // for PosIdx, noPos
|
||||
#include "primops.hh" // for getAttr, CompareValues, prim_l...
|
||||
#include "symbol-table.hh" // for Symbol, SymbolTable
|
||||
#include "types.hh" // for enumerate
|
||||
#include "value.hh" // for Value, ValueVector
|
||||
|
||||
namespace nix {
|
||||
|
||||
static void anyOrAll(bool any, EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
const std::string fun = any ? "builtins.any" : "builtins.all";
|
||||
state.forceFunction(
|
||||
*args[0], pos, std::string("while evaluating the first argument passed to " + fun)
|
||||
);
|
||||
state.forceList(
|
||||
*args[1], pos, std::string("while evaluating the second argument passed to " + fun)
|
||||
);
|
||||
Value vTmp;
|
||||
for (auto elem : args[1]->listItems()) {
|
||||
state.callFunction(*args[0], *elem, vTmp, pos);
|
||||
bool res = state.forceBool(
|
||||
vTmp, pos, "while evaluating the return value of the function passed to " + fun
|
||||
);
|
||||
if (res == any) {
|
||||
v.mkBool(any);
|
||||
return;
|
||||
}
|
||||
}
|
||||
v.mkBool(!any);
|
||||
}
|
||||
|
||||
static void elemAt(EvalState & state, const PosIdx pos, Value & list, int n, Value & v)
|
||||
{
|
||||
state.forceList(list, pos, "while evaluating the first argument passed to builtins.elemAt");
|
||||
if (n < 0 || (unsigned int) n >= list.listSize()) {
|
||||
state.error<EvalError>("list index %1% is out of bounds", n).atPos(pos).debugThrow();
|
||||
}
|
||||
state.forceValue(*list.listElems()[n], pos);
|
||||
v = *list.listElems()[n];
|
||||
}
|
||||
|
||||
/**
|
||||
* builtins.all
|
||||
*/
|
||||
|
||||
static void prim_all(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
anyOrAll(false, state, pos, args, v);
|
||||
}
|
||||
|
||||
PrimOp primop_all({
|
||||
.name = "__all",
|
||||
.args = {"pred", "list"},
|
||||
.doc = R"(
|
||||
Return `true` if the function *pred* returns `true` for all elements
|
||||
of *list*, and `false` otherwise.
|
||||
)",
|
||||
.fun = prim_all,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.any
|
||||
*/
|
||||
|
||||
static void prim_any(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
anyOrAll(true, state, pos, args, v);
|
||||
}
|
||||
|
||||
PrimOp primop_any({
|
||||
.name = "__any",
|
||||
.args = {"pred", "list"},
|
||||
.doc = R"(
|
||||
Return `true` if the function *pred* returns `true` for at least one
|
||||
element of *list*, and `false` otherwise.
|
||||
)",
|
||||
.fun = prim_any,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.concatLists
|
||||
*/
|
||||
|
||||
static void prim_concatLists(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceList(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.concatLists"
|
||||
);
|
||||
state.concatLists(
|
||||
v,
|
||||
args[0]->listSize(),
|
||||
args[0]->listElems(),
|
||||
pos,
|
||||
"while evaluating a value of the list passed to builtins.concatLists"
|
||||
);
|
||||
}
|
||||
|
||||
PrimOp primop_concatLists({
|
||||
.name = "__concatLists",
|
||||
.args = {"lists"},
|
||||
.doc = R"(
|
||||
Concatenate a list of lists into a single list.
|
||||
)",
|
||||
.fun = prim_concatLists,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.concatMap
|
||||
*/
|
||||
|
||||
static void prim_concatMap(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceFunction(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.concatMap"
|
||||
);
|
||||
state.forceList(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.concatMap"
|
||||
);
|
||||
auto nrLists = args[1]->listSize();
|
||||
|
||||
// List of returned lists before concatenation. References to these Values must NOT be
|
||||
// persisted.
|
||||
SmallTemporaryValueVector<conservativeStackReservation> lists(nrLists);
|
||||
size_t len = 0;
|
||||
|
||||
for (unsigned int n = 0; n < nrLists; ++n) {
|
||||
Value * vElem = args[1]->listElems()[n];
|
||||
state.callFunction(*args[0], *vElem, lists[n], pos);
|
||||
state.forceList(
|
||||
lists[n],
|
||||
lists[n].determinePos(args[0]->determinePos(pos)),
|
||||
"while evaluating the return value of the function passed to builtins.concatMap"
|
||||
);
|
||||
len += lists[n].listSize();
|
||||
}
|
||||
|
||||
state.mkList(v, len);
|
||||
auto out = v.listElems();
|
||||
for (unsigned int n = 0, pos = 0; n < nrLists; ++n) {
|
||||
auto l = lists[n].listSize();
|
||||
if (l) {
|
||||
memcpy(out + pos, lists[n].listElems(), l * sizeof(Value *));
|
||||
}
|
||||
pos += l;
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_concatMap({
|
||||
.name = "__concatMap",
|
||||
.args = {"f", "list"},
|
||||
.doc = R"(
|
||||
This function is equivalent to `builtins.concatLists (map f list)`
|
||||
but is more efficient.
|
||||
)",
|
||||
.fun = prim_concatMap,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.elem
|
||||
*/
|
||||
|
||||
static void prim_elem(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
bool res = false;
|
||||
state.forceList(*args[1], pos, "while evaluating the second argument passed to builtins.elem");
|
||||
for (auto elem : args[1]->listItems()) {
|
||||
if (state.eqValues(
|
||||
*args[0],
|
||||
*elem,
|
||||
pos,
|
||||
"while searching for the presence of the given element in the list"
|
||||
))
|
||||
{
|
||||
res = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
v.mkBool(res);
|
||||
}
|
||||
|
||||
PrimOp primop_elem({
|
||||
.name = "__elem",
|
||||
.args = {"x", "xs"},
|
||||
.doc = R"(
|
||||
Return `true` if a value equal to *x* occurs in the list *xs*, and
|
||||
`false` otherwise.
|
||||
)",
|
||||
.fun = prim_elem,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.elemAt
|
||||
*/
|
||||
|
||||
static void prim_elemAt(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
elemAt(
|
||||
state,
|
||||
pos,
|
||||
*args[0],
|
||||
state.forceInt(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.elemAt"
|
||||
),
|
||||
v
|
||||
);
|
||||
}
|
||||
|
||||
PrimOp primop_elemAt({
|
||||
.name = "__elemAt",
|
||||
.args = {"xs", "n"},
|
||||
.doc = R"(
|
||||
Return element *n* from the list *xs*. Elements are counted starting
|
||||
from 0. A fatal error occurs if the index is out of bounds.
|
||||
)",
|
||||
.fun = prim_elemAt,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.filter
|
||||
*/
|
||||
|
||||
static void prim_filter(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceList(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.filter"
|
||||
);
|
||||
|
||||
if (args[1]->listSize() == 0) {
|
||||
v = *args[1];
|
||||
return;
|
||||
}
|
||||
|
||||
state.forceFunction(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.filter"
|
||||
);
|
||||
|
||||
SmallValueVector<nonRecursiveStackReservation> vs(args[1]->listSize());
|
||||
size_t k = 0;
|
||||
|
||||
bool same = true;
|
||||
for (unsigned int n = 0; n < args[1]->listSize(); ++n) {
|
||||
Value res;
|
||||
state.callFunction(*args[0], *args[1]->listElems()[n], res, noPos);
|
||||
if (state.forceBool(
|
||||
res,
|
||||
pos,
|
||||
"while evaluating the return value of the filtering function passed to "
|
||||
"builtins.filter"
|
||||
))
|
||||
{
|
||||
vs[k++] = args[1]->listElems()[n];
|
||||
} else {
|
||||
same = false;
|
||||
}
|
||||
}
|
||||
|
||||
if (same) {
|
||||
v = *args[1];
|
||||
} else {
|
||||
state.mkList(v, k);
|
||||
for (unsigned int n = 0; n < k; ++n) {
|
||||
v.listElems()[n] = vs[n];
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* builtins.foldl'
|
||||
*/
|
||||
|
||||
static void prim_foldlStrict(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceFunction(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.foldlStrict"
|
||||
);
|
||||
state.forceList(
|
||||
*args[2], pos, "while evaluating the third argument passed to builtins.foldlStrict"
|
||||
);
|
||||
if (args[2]->listSize()) {
|
||||
Value * vCur = args[1];
|
||||
for (auto [n, elem] : enumerate(args[2]->listItems())) {
|
||||
Value * vs[]{vCur, elem};
|
||||
vCur = n == args[2]->listSize() - 1 ? &v : state.allocValue();
|
||||
state.callFunction(*args[0], 2, vs, *vCur, pos);
|
||||
}
|
||||
state.forceValue(v, pos);
|
||||
} else {
|
||||
state.forceValue(*args[1], pos);
|
||||
v = *args[1];
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_foldlStrict({
|
||||
.name = "__foldl'",
|
||||
.args = {"op", "nul", "list"},
|
||||
.doc = R"(
|
||||
Reduce a list by applying a binary operator, from left to right,
|
||||
e.g. `foldl' op nul [x0 x1 x2 ...] = op (op (op nul x0) x1) x2)
|
||||
...`. For example, `foldl' (x: y: x + y) 0 [1 2 3]` evaluates to 6.
|
||||
The return value of each application of `op` is evaluated immediately,
|
||||
even for intermediate values.
|
||||
)",
|
||||
.fun = prim_foldlStrict,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.genList
|
||||
*/
|
||||
|
||||
static void prim_genList(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto len = state.forceInt(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.genList"
|
||||
);
|
||||
if (len < 0) {
|
||||
state.error<EvalError>("cannot create list of size %1%", len).atPos(pos).debugThrow();
|
||||
}
|
||||
// More strict than striclty (!) necessary, but acceptable
|
||||
// as evaluating map without accessing any values makes little sense.
|
||||
state.forceFunction(
|
||||
*args[0], noPos, "while evaluating the first argument passed to builtins.genList"
|
||||
);
|
||||
state.mkList(v, len);
|
||||
for (unsigned int n = 0; n < (unsigned int) len; ++n) {
|
||||
auto arg = state.allocValue();
|
||||
arg->mkInt(n);
|
||||
(v.listElems()[n] = state.allocValue())->mkApp(args[0], arg);
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_genList({
|
||||
.name = "__genList",
|
||||
.args = {"generator", "length"},
|
||||
.doc = R"(
|
||||
Generate list of size *length*, with each element *i* equal to the
|
||||
value returned by *generator* `i`. For example,
|
||||
```nix
|
||||
builtins.genList (x: x * x) 5
|
||||
```
|
||||
returns the list `[ 0 1 4 9 16 ]`.
|
||||
)",
|
||||
.fun = prim_genList,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.head
|
||||
*/
|
||||
|
||||
static void prim_head(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
elemAt(state, pos, *args[0], 0, v);
|
||||
}
|
||||
|
||||
PrimOp primop_head({
|
||||
.name = "__head",
|
||||
.args = {"list"},
|
||||
.doc = R"(
|
||||
Return the first element of a list; abort evaluation if the argument
|
||||
isn’t a list or is an empty list. You can test whether a list is
|
||||
empty by comparing it with `[]`.
|
||||
)",
|
||||
.fun = prim_head,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.length
|
||||
*/
|
||||
|
||||
static void prim_length(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceList(*args[0], pos, "while evaluating the first argument passed to builtins.length");
|
||||
v.mkInt(args[0]->listSize());
|
||||
}
|
||||
PrimOp primop_length({
|
||||
.name = "__length",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Return the length of the list *e*.
|
||||
)",
|
||||
.fun = prim_length,
|
||||
});
|
||||
|
||||
PrimOp primop_filter({
|
||||
.name = "__filter",
|
||||
.args = {"f", "list"},
|
||||
.doc = R"(
|
||||
Return a list consisting of the elements of *list* for which the
|
||||
function *f* returns `true`.
|
||||
)",
|
||||
.fun = prim_filter,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.listToAttrs
|
||||
*/
|
||||
|
||||
static void prim_listToAttrs(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceList(*args[0], pos, "while evaluating the argument passed to builtins.listToAttrs");
|
||||
|
||||
auto attrs = state.buildBindings(args[0]->listSize());
|
||||
|
||||
std::set<Symbol> seen;
|
||||
|
||||
for (auto v2 : args[0]->listItems()) {
|
||||
state.forceAttrs(
|
||||
*v2, pos, "while evaluating an element of the list passed to builtins.listToAttrs"
|
||||
);
|
||||
|
||||
Bindings::iterator j =
|
||||
getAttr(state, state.sName, v2->attrs, "in a {name=...; value=...;} pair");
|
||||
|
||||
auto name = state.forceStringNoCtx(
|
||||
*j->value,
|
||||
j->pos,
|
||||
"while evaluating the `name` attribute of an element of the list passed to "
|
||||
"builtins.listToAttrs"
|
||||
);
|
||||
|
||||
auto sym = state.symbols.create(name);
|
||||
if (seen.insert(sym).second) {
|
||||
Bindings::iterator j2 =
|
||||
getAttr(state, state.sValue, v2->attrs, "in a {name=...; value=...;} pair");
|
||||
attrs.insert(sym, j2->value, j2->pos);
|
||||
}
|
||||
}
|
||||
|
||||
v.mkAttrs(attrs);
|
||||
}
|
||||
|
||||
PrimOp primop_listToAttrs({
|
||||
.name = "__listToAttrs",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Construct a set from a list specifying the names and values of each
|
||||
attribute. Each element of the list should be a set consisting of a
|
||||
string-valued attribute `name` specifying the name of the attribute,
|
||||
and an attribute `value` specifying its value.
|
||||
|
||||
In case of duplicate occurrences of the same name, the first
|
||||
takes precedence.
|
||||
|
||||
Example:
|
||||
|
||||
```nix
|
||||
builtins.listToAttrs
|
||||
[ { name = "foo"; value = 123; }
|
||||
{ name = "bar"; value = 456; }
|
||||
{ name = "bar"; value = 420; }
|
||||
]
|
||||
```
|
||||
|
||||
evaluates to
|
||||
|
||||
```nix
|
||||
{ foo = 123; bar = 456; }
|
||||
```
|
||||
)",
|
||||
.fun = prim_listToAttrs,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.map
|
||||
*/
|
||||
|
||||
static void prim_map(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceList(*args[1], pos, "while evaluating the second argument passed to builtins.map");
|
||||
|
||||
if (args[1]->listSize() == 0) {
|
||||
v = *args[1];
|
||||
return;
|
||||
}
|
||||
|
||||
state.forceFunction(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.map"
|
||||
);
|
||||
|
||||
state.mkList(v, args[1]->listSize());
|
||||
for (unsigned int n = 0; n < v.listSize(); ++n) {
|
||||
(v.listElems()[n] = state.allocValue())->mkApp(args[0], args[1]->listElems()[n]);
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_map({
|
||||
.name = "map",
|
||||
.args = {"f", "list"},
|
||||
.doc = R"(
|
||||
Apply the function *f* to each element in the list *list*. For
|
||||
example,
|
||||
|
||||
```nix
|
||||
map (x: "foo" + x) [ "bar" "bla" "abc" ]
|
||||
```
|
||||
|
||||
evaluates to `[ "foobar" "foobla" "fooabc" ]`.
|
||||
)",
|
||||
.fun = prim_map,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.partition
|
||||
*/
|
||||
|
||||
static void prim_partition(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceFunction(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.partition"
|
||||
);
|
||||
state.forceList(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.partition"
|
||||
);
|
||||
|
||||
auto len = args[1]->listSize();
|
||||
|
||||
ValueVector right, wrong;
|
||||
|
||||
for (unsigned int n = 0; n < len; ++n) {
|
||||
auto vElem = args[1]->listElems()[n];
|
||||
state.forceValue(*vElem, pos);
|
||||
Value res;
|
||||
state.callFunction(*args[0], *vElem, res, pos);
|
||||
if (state.forceBool(
|
||||
res,
|
||||
pos,
|
||||
"while evaluating the return value of the partition function passed to "
|
||||
"builtins.partition"
|
||||
))
|
||||
{
|
||||
right.push_back(vElem);
|
||||
} else {
|
||||
wrong.push_back(vElem);
|
||||
}
|
||||
}
|
||||
|
||||
auto attrs = state.buildBindings(2);
|
||||
|
||||
auto & vRight = attrs.alloc(state.sRight);
|
||||
auto rsize = right.size();
|
||||
state.mkList(vRight, rsize);
|
||||
if (rsize) {
|
||||
memcpy(vRight.listElems(), right.data(), sizeof(Value *) * rsize);
|
||||
}
|
||||
|
||||
auto & vWrong = attrs.alloc(state.sWrong);
|
||||
auto wsize = wrong.size();
|
||||
state.mkList(vWrong, wsize);
|
||||
if (wsize) {
|
||||
memcpy(vWrong.listElems(), wrong.data(), sizeof(Value *) * wsize);
|
||||
}
|
||||
|
||||
v.mkAttrs(attrs);
|
||||
}
|
||||
|
||||
PrimOp primop_partition({
|
||||
.name = "__partition",
|
||||
.args = {"pred", "list"},
|
||||
.doc = R"(
|
||||
Given a predicate function *pred*, this function returns an
|
||||
attrset containing a list named `right`, containing the elements
|
||||
in *list* for which *pred* returned `true`, and a list named
|
||||
`wrong`, containing the elements for which it returned
|
||||
`false`. For example,
|
||||
|
||||
```nix
|
||||
builtins.partition (x: x > 10) [1 23 9 3 42]
|
||||
```
|
||||
|
||||
evaluates to
|
||||
|
||||
```nix
|
||||
{ right = [ 23 42 ]; wrong = [ 1 9 3 ]; }
|
||||
```
|
||||
)",
|
||||
.fun = prim_partition,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.sort
|
||||
*/
|
||||
|
||||
static void prim_sort(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceList(*args[1], pos, "while evaluating the second argument passed to builtins.sort");
|
||||
|
||||
auto len = args[1]->listSize();
|
||||
if (len == 0) {
|
||||
v = *args[1];
|
||||
return;
|
||||
}
|
||||
|
||||
state.forceFunction(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.sort"
|
||||
);
|
||||
|
||||
state.mkList(v, len);
|
||||
for (unsigned int n = 0; n < len; ++n) {
|
||||
state.forceValue(*args[1]->listElems()[n], pos);
|
||||
v.listElems()[n] = args[1]->listElems()[n];
|
||||
}
|
||||
|
||||
auto comparator = [&](Value * a, Value * b) {
|
||||
/* Optimization: if the comparator is lessThan, bypass
|
||||
callFunction. */
|
||||
/* TODO: (layus) this is absurd. An optimisation like this
|
||||
should be outside the lambda creation */
|
||||
if (args[0]->isPrimOp()) {
|
||||
auto ptr = args[0]->primOp->fun.target<decltype(&prim_lessThan)>();
|
||||
if (ptr && *ptr == prim_lessThan) {
|
||||
return CompareValues(state, noPos, "while evaluating the ordering function passed to builtins.sort")(a, b);
|
||||
}
|
||||
}
|
||||
|
||||
Value * vs[] = {a, b};
|
||||
Value vBool;
|
||||
state.callFunction(*args[0], 2, vs, vBool, noPos);
|
||||
return state.forceBool(
|
||||
vBool,
|
||||
pos,
|
||||
"while evaluating the return value of the sorting function passed to builtins.sort"
|
||||
);
|
||||
};
|
||||
|
||||
/* FIXME: std::sort can segfault if the comparator is not a strict
|
||||
weak ordering. What to do? std::stable_sort() seems more
|
||||
resilient, but no guarantees... */
|
||||
std::stable_sort(v.listElems(), v.listElems() + len, comparator);
|
||||
}
|
||||
|
||||
PrimOp primop_sort({
|
||||
.name = "__sort",
|
||||
.args = {"comparator", "list"},
|
||||
.doc = R"(
|
||||
Return *list* in sorted order. It repeatedly calls the function
|
||||
*comparator* with two elements. The comparator should return `true`
|
||||
if the first element is less than the second, and `false` otherwise.
|
||||
For example,
|
||||
|
||||
```nix
|
||||
builtins.sort builtins.lessThan [ 483 249 526 147 42 77 ]
|
||||
```
|
||||
|
||||
produces the list `[ 42 77 147 249 483 526 ]`.
|
||||
|
||||
This is a stable sort: it preserves the relative order of elements
|
||||
deemed equal by the comparator.
|
||||
)",
|
||||
.fun = prim_sort,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.tail
|
||||
*/
|
||||
|
||||
static void prim_tail(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceList(*args[0], pos, "while evaluating the first argument passed to builtins.tail");
|
||||
if (args[0]->listSize() == 0) {
|
||||
state.error<EvalError>("'tail' called on an empty list").atPos(pos).debugThrow();
|
||||
}
|
||||
|
||||
state.mkList(v, args[0]->listSize() - 1);
|
||||
for (unsigned int n = 0; n < v.listSize(); ++n) {
|
||||
v.listElems()[n] = args[0]->listElems()[n + 1];
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_tail({
|
||||
.name = "__tail",
|
||||
.args = {"list"},
|
||||
.doc = R"(
|
||||
Return the second to last elements of a list; abort evaluation if
|
||||
the argument isn’t a list or is an empty list.
|
||||
|
||||
> **Warning**
|
||||
>
|
||||
> This function should generally be avoided since it's inefficient:
|
||||
> unlike Haskell's `tail`, it takes O(n) time, so recursing over a
|
||||
> list by repeatedly calling `tail` takes O(n^2) time.
|
||||
)",
|
||||
.fun = prim_tail,
|
||||
});
|
||||
}
|
||||
@@ -1,967 +0,0 @@
|
||||
#include <sys/stat.h> // for stat, S_ISDIR, S_I...
|
||||
#include <list> // for list
|
||||
#include <optional> // for optional, operator!=
|
||||
#include <set> // for _Rb_tree_const_ite...
|
||||
#include <string> // for basic_string, char...
|
||||
#include <string_view> // for string_view
|
||||
#include <utility> // for move, pair
|
||||
#include <variant> // for get_if
|
||||
#include "attr-set.hh" // for Attr, Bindings
|
||||
#include "canon-path.hh" // for CanonPath
|
||||
#include "content-address.hh" // for FileIngestionMethod
|
||||
#include "derivations.hh" // for hashPlaceholder
|
||||
#include "derived-path.hh" // for SingleDerivedPath
|
||||
#include "error.hh" // for Error, SysError
|
||||
#include "eval-error.hh" // for EvalError, EvalErr...
|
||||
#include "eval-settings.hh" // for EvalSettings, eval...
|
||||
#include "eval.hh" // for EvalState, PrimOp
|
||||
#include "experimental-features.hh" // for ExperimentalFeature
|
||||
#include "fetch-to-store.hh" // for fetchToStore
|
||||
#include "file-system.hh" // for canonPath, dirOf
|
||||
#include "globals.hh" // for Settings, settings
|
||||
#include "hash.hh" // for Hash, newHashAllow...
|
||||
#include "input-accessor.hh" // for InputAccessor
|
||||
#include "outputs-spec.hh" // for OutputNameView
|
||||
#include "path-info.hh" // for ValidPathInfo
|
||||
#include "path-references.hh" // for PathRefScanSink
|
||||
#include "path.hh" // for StorePath, StorePa...
|
||||
#include "pos-idx.hh" // for PosIdx, noPos
|
||||
#include "pos-table.hh" // for PosTable
|
||||
#include "primops.hh" // for RealisePathFlags
|
||||
#include "ref.hh" // for ref, make_ref
|
||||
#include "search-path.hh" // for SearchPath
|
||||
#include "serialise.hh" // for operator<<, Sink
|
||||
#include "source-path.hh" // for SourcePath, Symlin...
|
||||
#include "store-api.hh" // for Store
|
||||
#include "strings.hh" // for rewriteStrings
|
||||
#include "symbol-table.hh" // for Symbol, SymbolStr
|
||||
#include "types.hh" // for BackedStringView
|
||||
#include "value.hh" // for Value, nPath, nString
|
||||
#include "value/context.hh" // for NixStringContext
|
||||
|
||||
namespace nix {
|
||||
|
||||
// TODO: Use a const map instead ?
|
||||
static std::string_view fileTypeToString(InputAccessor::Type type)
|
||||
{
|
||||
return type == InputAccessor::Type::tRegular ? "regular"
|
||||
: type == InputAccessor::Type::tDirectory ? "directory"
|
||||
: type == InputAccessor::Type::tSymlink ? "symlink"
|
||||
: "unknown";
|
||||
}
|
||||
|
||||
static void addPath(
|
||||
EvalState & state,
|
||||
const PosIdx pos,
|
||||
std::string_view name,
|
||||
Path path,
|
||||
Value * filterFun,
|
||||
FileIngestionMethod method,
|
||||
const std::optional<Hash> expectedHash,
|
||||
Value & v,
|
||||
const NixStringContext & context
|
||||
)
|
||||
{
|
||||
try {
|
||||
// FIXME: handle CA derivation outputs (where path needs to
|
||||
// be rewritten to the actual output).
|
||||
auto rewrites = state.realiseContext(context);
|
||||
path = state.toRealPath(rewriteStrings(path, rewrites), context);
|
||||
|
||||
StorePathSet refs;
|
||||
|
||||
if (state.store->isInStore(path)) {
|
||||
try {
|
||||
auto [storePath, subPath] = state.store->toStorePath(path);
|
||||
// FIXME: we should scanForReferences on the path before adding it
|
||||
refs = state.store->queryPathInfo(storePath)->references;
|
||||
path = state.store->toRealPath(storePath) + subPath;
|
||||
} catch (Error &) { // FIXME: should be InvalidPathError
|
||||
}
|
||||
}
|
||||
|
||||
path = evalSettings.pureEval && expectedHash
|
||||
? path
|
||||
: state.checkSourcePath(CanonPath(path)).path.abs();
|
||||
|
||||
PathFilter filter = filterFun ? ([&](const Path & path) {
|
||||
auto st = lstat(path);
|
||||
|
||||
/* Call the filter function. The first argument is the path,
|
||||
the second is a string indicating the type of the file. */
|
||||
Value arg1;
|
||||
arg1.mkString(path);
|
||||
|
||||
Value arg2;
|
||||
arg2.mkString(
|
||||
S_ISREG(st.st_mode) ? "regular"
|
||||
: S_ISDIR(st.st_mode) ? "directory"
|
||||
: S_ISLNK(st.st_mode) ? "symlink"
|
||||
: "unknown" /* not supported, will fail! */
|
||||
);
|
||||
|
||||
Value * args[]{&arg1, &arg2};
|
||||
Value res;
|
||||
state.callFunction(*filterFun, 2, args, res, pos);
|
||||
|
||||
return state.forceBool(
|
||||
res, pos, "while evaluating the return value of the path filter function"
|
||||
);
|
||||
})
|
||||
: defaultPathFilter;
|
||||
|
||||
std::optional<StorePath> expectedStorePath;
|
||||
if (expectedHash) {
|
||||
expectedStorePath = state.store->makeFixedOutputPath(
|
||||
name,
|
||||
FixedOutputInfo{
|
||||
.method = method,
|
||||
.hash = *expectedHash,
|
||||
.references = {},
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
if (!expectedHash || !state.store->isValidPath(*expectedStorePath)) {
|
||||
auto dstPath = fetchToStore(
|
||||
*state.store, state.rootPath(CanonPath(path)), name, method, &filter, state.repair
|
||||
);
|
||||
if (expectedHash && expectedStorePath != dstPath) {
|
||||
state
|
||||
.error<EvalError>(
|
||||
"store path mismatch in (possibly filtered) path added from '%s'", path
|
||||
)
|
||||
.atPos(pos)
|
||||
.debugThrow();
|
||||
}
|
||||
state.allowAndSetStorePathString(dstPath, v);
|
||||
} else {
|
||||
state.allowAndSetStorePathString(*expectedStorePath, v);
|
||||
}
|
||||
} catch (Error & e) {
|
||||
e.addTrace(state.positions[pos], "while adding path '%s'", path);
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
SourcePath realisePath(EvalState & state, const PosIdx pos, Value & v, const RealisePathFlags flags)
|
||||
{
|
||||
NixStringContext context;
|
||||
|
||||
auto path = state.coerceToPath(noPos, v, context, "while realising the context of a path");
|
||||
|
||||
try {
|
||||
StringMap rewrites = state.realiseContext(context);
|
||||
|
||||
auto realPath = state.rootPath(CanonPath(state.toRealPath(rewriteStrings(path.path.abs(), rewrites), context)));
|
||||
|
||||
return flags.checkForPureEval
|
||||
? state.checkSourcePath(realPath)
|
||||
: realPath;
|
||||
} catch (Error & e) {
|
||||
e.addTrace(state.positions[pos], "while realising the context of path '%s'", path);
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* builtins.dirOf
|
||||
*/
|
||||
|
||||
static void prim_dirOf(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceValue(*args[0], pos);
|
||||
if (args[0]->type() == nPath) {
|
||||
auto path = args[0]->path();
|
||||
v.mkPath(path.path.isRoot() ? path : path.parent());
|
||||
} else {
|
||||
NixStringContext context;
|
||||
auto path = state.coerceToString(
|
||||
pos,
|
||||
*args[0],
|
||||
context,
|
||||
"while evaluating the first argument passed to 'builtins.dirOf'",
|
||||
false,
|
||||
false
|
||||
);
|
||||
auto dir = dirOf(*path);
|
||||
v.mkString(dir, context);
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_dirOf({
|
||||
.name = "dirOf",
|
||||
.args = {"s"},
|
||||
.doc = R"(
|
||||
Return the directory part of the string *s*, that is, everything
|
||||
before the final slash in the string. This is similar to the GNU
|
||||
`dirname` command.
|
||||
)",
|
||||
.fun = prim_dirOf,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.filterSource
|
||||
*/
|
||||
|
||||
static void prim_filterSource(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
NixStringContext context;
|
||||
auto path = state.coerceToPath(
|
||||
pos,
|
||||
*args[1],
|
||||
context,
|
||||
"while evaluating the second argument (the path to filter) passed to builtins.filterSource"
|
||||
);
|
||||
state.forceFunction(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.filterSource"
|
||||
);
|
||||
addPath(
|
||||
state,
|
||||
pos,
|
||||
path.baseName(),
|
||||
path.path.abs(),
|
||||
args[0],
|
||||
FileIngestionMethod::Recursive,
|
||||
std::nullopt,
|
||||
v,
|
||||
context
|
||||
);
|
||||
}
|
||||
|
||||
PrimOp primop_filterSource({
|
||||
.name = "__filterSource",
|
||||
.args = {"e1", "e2"},
|
||||
.doc = R"(
|
||||
> **Warning**
|
||||
>
|
||||
> `filterSource` should not be used to filter store paths. Since
|
||||
> `filterSource` uses the name of the input directory while naming
|
||||
> the output directory, doing so will produce a directory name in
|
||||
> the form of `<hash2>-<hash>-<name>`, where `<hash>-<name>` is
|
||||
> the name of the input directory. Since `<hash>` depends on the
|
||||
> unfiltered directory, the name of the output directory will
|
||||
> indirectly depend on files that are filtered out by the
|
||||
> function. This will trigger a rebuild even when a filtered out
|
||||
> file is changed. Use `builtins.path` instead, which allows
|
||||
> specifying the name of the output directory.
|
||||
|
||||
This function allows you to copy sources into the Nix store while
|
||||
filtering certain files. For instance, suppose that you want to use
|
||||
the directory `source-dir` as an input to a Nix expression, e.g.
|
||||
|
||||
```nix
|
||||
stdenv.mkDerivation {
|
||||
...
|
||||
src = ./source-dir;
|
||||
}
|
||||
```
|
||||
|
||||
However, if `source-dir` is a Subversion working copy, then all
|
||||
those annoying `.svn` subdirectories will also be copied to the
|
||||
store. Worse, the contents of those directories may change a lot,
|
||||
causing lots of spurious rebuilds. With `filterSource` you can
|
||||
filter out the `.svn` directories:
|
||||
|
||||
```nix
|
||||
src = builtins.filterSource
|
||||
(path: type: type != "directory" || baseNameOf path != ".svn")
|
||||
./source-dir;
|
||||
```
|
||||
|
||||
Thus, the first argument *e1* must be a predicate function that is
|
||||
called for each regular file, directory or symlink in the source
|
||||
tree *e2*. If the function returns `true`, the file is copied to the
|
||||
Nix store, otherwise it is omitted. The function is called with two
|
||||
arguments. The first is the full path of the file. The second is a
|
||||
string that identifies the type of the file, which is either
|
||||
`"regular"`, `"directory"`, `"symlink"` or `"unknown"` (for other
|
||||
kinds of files such as device nodes or fifos — but note that those
|
||||
cannot be copied to the Nix store, so if the predicate returns
|
||||
`true` for them, the copy will fail). If you exclude a directory,
|
||||
the entire corresponding subtree of *e2* will be excluded.
|
||||
)",
|
||||
.fun = prim_filterSource,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.findFile
|
||||
*/
|
||||
|
||||
static void prim_findFile(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceList(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.findFile"
|
||||
);
|
||||
|
||||
SearchPath searchPath;
|
||||
|
||||
for (auto v2 : args[0]->listItems()) {
|
||||
state.forceAttrs(
|
||||
*v2, pos, "while evaluating an element of the list passed to builtins.findFile"
|
||||
);
|
||||
|
||||
std::string prefix;
|
||||
Bindings::iterator i = v2->attrs->find(state.sPrefix);
|
||||
if (i != v2->attrs->end()) {
|
||||
prefix = state.forceStringNoCtx(
|
||||
*i->value,
|
||||
pos,
|
||||
"while evaluating the `prefix` attribute of an element of the list passed to "
|
||||
"builtins.findFile"
|
||||
);
|
||||
}
|
||||
|
||||
i = getAttr(state, state.sPath, v2->attrs, "in an element of the __nixPath");
|
||||
|
||||
NixStringContext context;
|
||||
auto path = state
|
||||
.coerceToString(
|
||||
pos,
|
||||
*i->value,
|
||||
context,
|
||||
"while evaluating the `path` attribute of an element of the list "
|
||||
"passed to builtins.findFile",
|
||||
false,
|
||||
false
|
||||
)
|
||||
.toOwned();
|
||||
|
||||
try {
|
||||
auto rewrites = state.realiseContext(context);
|
||||
path = rewriteStrings(path, rewrites);
|
||||
} catch (InvalidPathError & e) {
|
||||
state.error<EvalError>("cannot find '%1%', since path '%2%' is not valid", path, e.path)
|
||||
.atPos(pos)
|
||||
.debugThrow();
|
||||
}
|
||||
|
||||
searchPath.elements.emplace_back(SearchPath::Elem{
|
||||
.prefix = SearchPath::Prefix{.s = prefix},
|
||||
.path = SearchPath::Path{.s = path},
|
||||
});
|
||||
}
|
||||
|
||||
auto path = state.forceStringNoCtx(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.findFile"
|
||||
);
|
||||
|
||||
v.mkPath(state.checkSourcePath(state.findFile(searchPath, path, pos)));
|
||||
}
|
||||
|
||||
PrimOp primop_findFile(PrimOp{
|
||||
.name = "__findFile",
|
||||
.args = {"search path", "lookup path"},
|
||||
.doc = R"(
|
||||
Look up the given path with the given search path.
|
||||
|
||||
A search path is represented list of [attribute sets](./values.md#attribute-set) with two attributes, `prefix`, and `path`.
|
||||
`prefix` is a relative path.
|
||||
`path` denotes a file system location; the exact syntax depends on the command line interface.
|
||||
|
||||
Examples of search path attribute sets:
|
||||
|
||||
- ```
|
||||
{
|
||||
prefix = "nixos-config";
|
||||
path = "/etc/nixos/configuration.nix";
|
||||
}
|
||||
```
|
||||
|
||||
- ```
|
||||
{
|
||||
prefix = "";
|
||||
path = "/nix/var/nix/profiles/per-user/root/channels";
|
||||
}
|
||||
```
|
||||
|
||||
The lookup algorithm checks each entry until a match is found, returning a [path value](@docroot@/language/values.html#type-path) of the match.
|
||||
|
||||
This is the process for each entry:
|
||||
If the lookup path matches `prefix`, then the remainder of the lookup path (the "suffix") is searched for within the directory denoted by `patch`.
|
||||
Note that the `path` may need to be downloaded at this point to look inside.
|
||||
If the suffix is found inside that directory, then the entry is a match;
|
||||
the combined absolute path of the directory (now downloaded if need be) and the suffix is returned.
|
||||
|
||||
The syntax
|
||||
|
||||
```nix
|
||||
<nixpkgs>
|
||||
```
|
||||
|
||||
is equivalent to:
|
||||
|
||||
```nix
|
||||
builtins.findFile builtins.nixPath "nixpkgs"
|
||||
```
|
||||
)",
|
||||
.fun = prim_findFile,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.outputOf
|
||||
*/
|
||||
|
||||
static void prim_outputOf(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
SingleDerivedPath drvPath = state.coerceToSingleDerivedPath(
|
||||
pos, *args[0], "while evaluating the first argument to builtins.outputOf"
|
||||
);
|
||||
|
||||
OutputNameView outputName = state.forceStringNoCtx(
|
||||
*args[1], pos, "while evaluating the second argument to builtins.outputOf"
|
||||
);
|
||||
|
||||
state.mkSingleDerivedPathString(
|
||||
SingleDerivedPath::Built{
|
||||
.drvPath = make_ref<SingleDerivedPath>(drvPath),
|
||||
.output = std::string{outputName},
|
||||
},
|
||||
v
|
||||
);
|
||||
}
|
||||
|
||||
PrimOp primop_outputOf({
|
||||
.name = "__outputOf",
|
||||
.args = {"derivation-reference", "output-name"},
|
||||
.doc = R"(
|
||||
Return the output path of a derivation, literally or using a placeholder if needed.
|
||||
|
||||
If the derivation has a statically-known output path (i.e. the derivation output is input-addressed, or fixed content-addresed), the output path will just be returned.
|
||||
But if the derivation is content-addressed or if the derivation is itself not-statically produced (i.e. is the output of another derivation), a placeholder will be returned instead.
|
||||
|
||||
*`derivation reference`* must be a string that may contain a regular store path to a derivation, or may be a placeholder reference. If the derivation is produced by a derivation, you must explicitly select `drv.outPath`.
|
||||
This primop can be chained arbitrarily deeply.
|
||||
For instance,
|
||||
|
||||
```nix
|
||||
builtins.outputOf
|
||||
(builtins.outputOf myDrv "out)
|
||||
"out"
|
||||
```
|
||||
|
||||
will return a placeholder for the output of the output of `myDrv`.
|
||||
|
||||
This primop corresponds to the `^` sigil for derivable paths, e.g. as part of installable syntax on the command line.
|
||||
)",
|
||||
.fun = prim_outputOf,
|
||||
.experimentalFeature = Xp::DynamicDerivations,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.path
|
||||
*/
|
||||
|
||||
static void prim_path(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
std::optional<SourcePath> path;
|
||||
std::string name;
|
||||
Value * filterFun = nullptr;
|
||||
auto method = FileIngestionMethod::Recursive;
|
||||
std::optional<Hash> expectedHash;
|
||||
NixStringContext context;
|
||||
|
||||
state.forceAttrs(*args[0], pos, "while evaluating the argument passed to 'builtins.path'");
|
||||
|
||||
for (auto & attr : *args[0]->attrs) {
|
||||
auto n = state.symbols[attr.name];
|
||||
if (n == "path") {
|
||||
path.emplace(state.coerceToPath(
|
||||
attr.pos,
|
||||
*attr.value,
|
||||
context,
|
||||
"while evaluating the 'path' attribute passed to 'builtins.path'"
|
||||
));
|
||||
} else if (attr.name == state.sName) {
|
||||
name = state.forceStringNoCtx(
|
||||
*attr.value,
|
||||
attr.pos,
|
||||
"while evaluating the `name` attribute passed to builtins.path"
|
||||
);
|
||||
} else if (n == "filter") {
|
||||
state.forceFunction(
|
||||
*(filterFun = attr.value),
|
||||
attr.pos,
|
||||
"while evaluating the `filter` parameter passed to builtins.path"
|
||||
);
|
||||
} else if (n == "recursive") {
|
||||
method = FileIngestionMethod{state.forceBool(
|
||||
*attr.value,
|
||||
attr.pos,
|
||||
"while evaluating the `recursive` attribute passed to builtins.path"
|
||||
)};
|
||||
} else if (n == "sha256") {
|
||||
expectedHash = newHashAllowEmpty(
|
||||
state.forceStringNoCtx(
|
||||
*attr.value,
|
||||
attr.pos,
|
||||
"while evaluating the `sha256` attribute passed to builtins.path"
|
||||
),
|
||||
htSHA256
|
||||
);
|
||||
} else {
|
||||
state
|
||||
.error<EvalError>(
|
||||
"unsupported argument '%1%' to 'addPath'", state.symbols[attr.name]
|
||||
)
|
||||
.atPos(attr.pos)
|
||||
.debugThrow();
|
||||
}
|
||||
}
|
||||
if (!path) {
|
||||
state
|
||||
.error<EvalError>(
|
||||
"missing required 'path' attribute in the first argument to builtins.path"
|
||||
)
|
||||
.atPos(pos)
|
||||
.debugThrow();
|
||||
}
|
||||
if (name.empty()) {
|
||||
name = path->baseName();
|
||||
}
|
||||
|
||||
addPath(state, pos, name, path->path.abs(), filterFun, method, expectedHash, v, context);
|
||||
}
|
||||
|
||||
PrimOp primop_path({
|
||||
.name = "__path",
|
||||
.args = {"args"},
|
||||
.doc = R"(
|
||||
An enrichment of the built-in path type, based on the attributes
|
||||
present in *args*. All are optional except `path`:
|
||||
|
||||
- path\
|
||||
The underlying path.
|
||||
|
||||
- name\
|
||||
The name of the path when added to the store. This can used to
|
||||
reference paths that have nix-illegal characters in their names,
|
||||
like `@`.
|
||||
|
||||
- filter\
|
||||
A function of the type expected by `builtins.filterSource`,
|
||||
with the same semantics.
|
||||
|
||||
- recursive\
|
||||
When `false`, when `path` is added to the store it is with a
|
||||
flat hash, rather than a hash of the NAR serialization of the
|
||||
file. Thus, `path` must refer to a regular file, not a
|
||||
directory. This allows similar behavior to `fetchurl`. Defaults
|
||||
to `true`.
|
||||
|
||||
- sha256\
|
||||
When provided, this is the expected hash of the file at the
|
||||
path. Evaluation will fail if the hash is incorrect, and
|
||||
providing a hash allows `builtins.path` to be used even when the
|
||||
`pure-eval` nix config option is on.
|
||||
)",
|
||||
.fun = prim_path,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.pathExists
|
||||
*/
|
||||
|
||||
static void prim_pathExists(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto & arg = *args[0];
|
||||
|
||||
/* We don’t check the path right now, because we don’t want to
|
||||
throw if the path isn’t allowed, but just return false (and we
|
||||
can’t just catch the exception here because we still want to
|
||||
throw if something in the evaluation of `arg` tries to
|
||||
access an unauthorized path). */
|
||||
auto path = realisePath(state, pos, arg, {.checkForPureEval = false});
|
||||
|
||||
/* SourcePath doesn't know about trailing slash. */
|
||||
auto mustBeDir =
|
||||
arg.type() == nString && (arg.str().ends_with("/") || arg.str().ends_with("/."));
|
||||
|
||||
try {
|
||||
auto checked = state.checkSourcePath(path).resolveSymlinks(
|
||||
mustBeDir ? SymlinkResolution::Full : SymlinkResolution::Ancestors
|
||||
);
|
||||
|
||||
auto st = checked.maybeLstat();
|
||||
auto exists = st && (!mustBeDir || st->type == InputAccessor::tDirectory);
|
||||
v.mkBool(exists);
|
||||
} catch (SysError & e) {
|
||||
/* Don't give away info from errors while canonicalising
|
||||
‘path’ in restricted mode. */
|
||||
v.mkBool(false);
|
||||
} catch (RestrictedPathError & e) {
|
||||
v.mkBool(false);
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_pathExists({
|
||||
.name = "__pathExists",
|
||||
.args = {"path"},
|
||||
.doc = R"(
|
||||
Return `true` if the path *path* exists at evaluation time, and
|
||||
`false` otherwise.
|
||||
)",
|
||||
.fun = prim_pathExists,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.palceholder
|
||||
*/
|
||||
|
||||
/* Return a placeholder string for the specified output that will be
|
||||
substituted by the corresponding output path at build time. For
|
||||
example, 'placeholder "out"' returns the string
|
||||
/1rz4g4znpzjwh1xymhjpm42vipw92pr73vdgl6xs1hycac8kf2n9. At build
|
||||
time, any occurrence of this string in an derivation attribute will
|
||||
be replaced with the concrete path in the Nix store of the output
|
||||
‘out’. */
|
||||
static void prim_placeholder(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
v.mkString(hashPlaceholder(state.forceStringNoCtx(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.placeholder"
|
||||
)));
|
||||
}
|
||||
|
||||
PrimOp primop_placeholder({
|
||||
.name = "placeholder",
|
||||
.args = {"output"},
|
||||
.doc = R"(
|
||||
Return a placeholder string for the specified *output* that will be
|
||||
substituted by the corresponding output path at build time. Typical
|
||||
outputs would be `"out"`, `"bin"` or `"dev"`.
|
||||
)",
|
||||
.fun = prim_placeholder,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.toPath
|
||||
* WARNING: deprecated
|
||||
*/
|
||||
|
||||
static void prim_toPath(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
NixStringContext context;
|
||||
auto path = state.coerceToPath(
|
||||
pos, *args[0], context, "while evaluating the first argument passed to builtins.toPath"
|
||||
);
|
||||
v.mkString(path.path.abs(), context);
|
||||
}
|
||||
|
||||
PrimOp primop_toPath({
|
||||
.name = "__toPath",
|
||||
.args = {"s"},
|
||||
.doc = R"(
|
||||
**DEPRECATED.** Use `/. + "/path"` to convert a string into an absolute
|
||||
path. For relative paths, use `./. + "/path"`.
|
||||
)",
|
||||
.fun = prim_toPath,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.readDir
|
||||
*/
|
||||
|
||||
static void prim_readDir(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto path = realisePath(state, pos, *args[0]);
|
||||
|
||||
// Retrieve directory entries for all nodes in a directory.
|
||||
// This is similar to `getFileType` but is optimized to reduce system calls
|
||||
// on many systems.
|
||||
auto entries = path.readDirectory();
|
||||
auto attrs = state.buildBindings(entries.size());
|
||||
|
||||
// If we hit unknown directory entry types we may need to fallback to
|
||||
// using `getFileType` on some systems.
|
||||
// In order to reduce system calls we make each lookup lazy by using
|
||||
// `builtins.readFileType` application.
|
||||
Value * readFileType = nullptr;
|
||||
|
||||
for (auto & [name, type] : entries) {
|
||||
auto & attr = attrs.alloc(name);
|
||||
if (!type) {
|
||||
// Some filesystems or operating systems may not be able to return
|
||||
// detailed node info quickly in this case we produce a thunk to
|
||||
// query the file type lazily.
|
||||
auto epath = state.allocValue();
|
||||
epath->mkPath(path + name);
|
||||
if (!readFileType) {
|
||||
readFileType = &state.getBuiltin("readFileType");
|
||||
}
|
||||
attr.mkApp(readFileType, epath);
|
||||
} else {
|
||||
// This branch of the conditional is much more likely.
|
||||
// Here we just stringize the directory entry type.
|
||||
attr.mkString(fileTypeToString(*type));
|
||||
}
|
||||
}
|
||||
|
||||
v.mkAttrs(attrs);
|
||||
}
|
||||
|
||||
PrimOp primop_readDir({
|
||||
.name = "__readDir",
|
||||
.args = {"path"},
|
||||
.doc = R"(
|
||||
Return the contents of the directory *path* as a set mapping
|
||||
directory entries to the corresponding file type. For instance, if
|
||||
directory `A` contains a regular file `B` and another directory
|
||||
`C`, then `builtins.readDir ./A` will return the set
|
||||
|
||||
```nix
|
||||
{ B = "regular"; C = "directory"; }
|
||||
```
|
||||
|
||||
The possible values for the file type are `"regular"`,
|
||||
`"directory"`, `"symlink"` and `"unknown"`.
|
||||
)",
|
||||
.fun = prim_readDir,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.readFile
|
||||
*/
|
||||
|
||||
static void prim_readFile(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto path = realisePath(state, pos, *args[0]);
|
||||
auto s = path.readFile();
|
||||
if (s.find((char) 0) != std::string::npos) {
|
||||
state
|
||||
.error<EvalError>(
|
||||
"the contents of the file '%1%' cannot be represented as a Nix string", path
|
||||
)
|
||||
.atPos(pos)
|
||||
.debugThrow();
|
||||
}
|
||||
StorePathSet refs;
|
||||
if (state.store->isInStore(path.path.abs())) {
|
||||
try {
|
||||
refs = state.store->queryPathInfo(state.store->toStorePath(path.path.abs()).first)
|
||||
->references;
|
||||
} catch (Error &) { // FIXME: should be InvalidPathError
|
||||
}
|
||||
// Re-scan references to filter down to just the ones that actually occur in the file.
|
||||
auto refsSink = PathRefScanSink::fromPaths(refs);
|
||||
refsSink << s;
|
||||
refs = refsSink.getResultPaths();
|
||||
}
|
||||
NixStringContext context;
|
||||
for (auto && p : std::move(refs)) {
|
||||
context.insert(NixStringContextElem::Opaque{
|
||||
.path = std::move((StorePath &&) p),
|
||||
});
|
||||
}
|
||||
v.mkString(s, context);
|
||||
}
|
||||
|
||||
PrimOp primop_readFile({
|
||||
.name = "__readFile",
|
||||
.args = {"path"},
|
||||
.doc = R"(
|
||||
Return the contents of the file *path* as a string.
|
||||
)",
|
||||
.fun = prim_readFile,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.readFileType
|
||||
*/
|
||||
|
||||
static void prim_readFileType(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto path = realisePath(state, pos, *args[0]);
|
||||
/* Retrieve the directory entry type and stringize it. */
|
||||
v.mkString(fileTypeToString(path.lstat().type));
|
||||
}
|
||||
|
||||
PrimOp primop_readFileType({
|
||||
.name = "__readFileType",
|
||||
.args = {"p"},
|
||||
.doc = R"(
|
||||
Determine the directory entry type of a filesystem node, being
|
||||
one of "directory", "regular", "symlink", or "unknown".
|
||||
)",
|
||||
.fun = prim_readFileType,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.storePath
|
||||
*/
|
||||
|
||||
static void prim_storePath(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
if (evalSettings.pureEval) {
|
||||
state.error<EvalError>("'%s' is not allowed in pure evaluation mode", "builtins.storePath")
|
||||
.atPos(pos)
|
||||
.debugThrow();
|
||||
}
|
||||
|
||||
NixStringContext context;
|
||||
auto path = state
|
||||
.checkSourcePath(state.coerceToPath(
|
||||
pos,
|
||||
*args[0],
|
||||
context,
|
||||
"while evaluating the first argument passed to builtins.storePath"
|
||||
))
|
||||
.path;
|
||||
/* Resolve symlinks in ‘path’, unless ‘path’ itself is a symlink
|
||||
directly in the store. The latter condition is necessary so
|
||||
e.g. nix-push does the right thing. */
|
||||
if (!state.store->isStorePath(path.abs())) {
|
||||
path = CanonPath(canonPath(path.abs(), true));
|
||||
}
|
||||
if (!state.store->isInStore(path.abs())) {
|
||||
state.error<EvalError>("path '%1%' is not in the Nix store", path).atPos(pos).debugThrow();
|
||||
}
|
||||
auto path2 = state.store->toStorePath(path.abs()).first;
|
||||
if (!settings.readOnlyMode) {
|
||||
state.store->ensurePath(path2);
|
||||
}
|
||||
context.insert(NixStringContextElem::Opaque{.path = path2});
|
||||
v.mkString(path.abs(), context);
|
||||
}
|
||||
|
||||
PrimOp primop_storePath({
|
||||
.name = "__storePath",
|
||||
.args = {"path"},
|
||||
.doc = R"(
|
||||
This function allows you to define a dependency on an already
|
||||
existing store path. For example, the derivation attribute `src
|
||||
= builtins.storePath /nix/store/f1d18v1y…-source` causes the
|
||||
derivation to depend on the specified path, which must exist or
|
||||
be substitutable. Note that this differs from a plain path
|
||||
(e.g. `src = /nix/store/f1d18v1y…-source`) in that the latter
|
||||
causes the path to be *copied* again to the Nix store, resulting
|
||||
in a new path (e.g. `/nix/store/ld01dnzc…-source-source`).
|
||||
|
||||
Not available in [pure evaluation mode](@docroot@/command-ref/conf-file.md#conf-pure-eval).
|
||||
|
||||
See also [`builtins.fetchClosure`](#builtins-fetchClosure).
|
||||
)",
|
||||
.fun = prim_storePath,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.toFile
|
||||
*/
|
||||
|
||||
static void prim_toFile(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
NixStringContext context;
|
||||
std::string name(state.forceStringNoCtx(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.toFile"
|
||||
));
|
||||
std::string contents(state.forceString(
|
||||
*args[1], context, pos, "while evaluating the second argument passed to builtins.toFile"
|
||||
));
|
||||
|
||||
StorePathSet refs;
|
||||
|
||||
for (auto c : context) {
|
||||
if (auto p = std::get_if<NixStringContextElem::Opaque>(&c.raw)) {
|
||||
refs.insert(p->path);
|
||||
} else {
|
||||
state
|
||||
.error<EvalError>(
|
||||
"files created by %1% may not reference derivations, but %2% references %3%",
|
||||
"builtins.toFile",
|
||||
name,
|
||||
c.to_string()
|
||||
)
|
||||
.atPos(pos)
|
||||
.debugThrow();
|
||||
}
|
||||
}
|
||||
|
||||
auto storePath = settings.readOnlyMode
|
||||
? state.store->computeStorePathForText(name, contents, refs)
|
||||
: state.store->addTextToStore(name, contents, refs, state.repair);
|
||||
|
||||
/* Note: we don't need to add `context' to the context of the
|
||||
result, since `storePath' itself has references to the paths
|
||||
used in args[1]. */
|
||||
|
||||
/* Add the output of this to the allowed paths. */
|
||||
state.allowAndSetStorePathString(storePath, v);
|
||||
}
|
||||
|
||||
PrimOp primop_toFile({
|
||||
.name = "__toFile",
|
||||
.args = {"name", "s"},
|
||||
.doc = R"(
|
||||
Store the string *s* in a file in the Nix store and return its
|
||||
path. The file has suffix *name*. This file can be used as an
|
||||
input to derivations. One application is to write builders
|
||||
“inline”. For instance, the following Nix expression combines the
|
||||
Nix expression for GNU Hello and its build script into one file:
|
||||
|
||||
```nix
|
||||
{ stdenv, fetchurl, perl }:
|
||||
|
||||
stdenv.mkDerivation {
|
||||
name = "hello-2.1.1";
|
||||
|
||||
builder = builtins.toFile "builder.sh" "
|
||||
source $stdenv/setup
|
||||
|
||||
PATH=$perl/bin:$PATH
|
||||
|
||||
tar xvfz $src
|
||||
cd hello-*
|
||||
./configure --prefix=$out
|
||||
make
|
||||
make install
|
||||
";
|
||||
|
||||
src = fetchurl {
|
||||
url = "http://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz";
|
||||
sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465";
|
||||
};
|
||||
inherit perl;
|
||||
}
|
||||
```
|
||||
|
||||
It is even possible for one file to refer to another, e.g.,
|
||||
|
||||
```nix
|
||||
builder = let
|
||||
configFile = builtins.toFile "foo.conf" "
|
||||
# This is some dummy configuration file.
|
||||
...
|
||||
";
|
||||
in builtins.toFile "builder.sh" "
|
||||
source $stdenv/setup
|
||||
...
|
||||
cp ${configFile} $out/etc/foo.conf
|
||||
";
|
||||
```
|
||||
|
||||
Note that `${configFile}` is a
|
||||
[string interpolation](@docroot@/language/values.md#type-string), so the result of the
|
||||
expression `configFile`
|
||||
(i.e., a path like `/nix/store/m7p7jfny445k...-foo.conf`) will be
|
||||
spliced into the resulting string.
|
||||
|
||||
It is however *not* allowed to have files mutually referring to each
|
||||
other, like so:
|
||||
|
||||
```nix
|
||||
let
|
||||
foo = builtins.toFile "foo" "...${bar}...";
|
||||
bar = builtins.toFile "bar" "...${foo}...";
|
||||
in foo
|
||||
```
|
||||
|
||||
This is not allowed because it would cause a cyclic dependency in
|
||||
the computation of the cryptographic hashes for `foo` and `bar`.
|
||||
|
||||
It is also not possible to reference the result of a derivation. If
|
||||
you are using Nixpkgs, the `writeTextFile` function is able to do
|
||||
that.
|
||||
)",
|
||||
.fun = prim_toFile,
|
||||
});
|
||||
|
||||
}
|
||||
@@ -1,641 +0,0 @@
|
||||
#include <assert.h> // for assert
|
||||
#include <stddef.h> // for size_t
|
||||
#include <iterator> // for distance
|
||||
#include <memory> // for shared_ptr, __shared_ptr_access, make_sh...
|
||||
#include <regex> // for regex_error, match_results, cregex_iterator
|
||||
#include <string> // for basic_string, char_traits, string
|
||||
#include <string_view> // for basic_string_view
|
||||
#include <unordered_map> // for unordered_map, operator==, _Node_iterator
|
||||
#include <utility> // for move, pair
|
||||
#include <vector> // for vector
|
||||
#include "attr-set.hh" // for BindingsBuilder
|
||||
#include "eval-error.hh" // for EvalError, EvalErrorBuilder
|
||||
#include "eval.hh" // for EvalState, PrimOp, makeRegexCache, prim_...
|
||||
#include "file-system.hh" // for baseNameOf
|
||||
#include "names.hh" // for DrvName, compareVersions, nextComponent
|
||||
#include "pos-idx.hh" // for PosIdx
|
||||
#include "primops.hh" // for RegexCache
|
||||
#include "symbol-table.hh" // for Symbol
|
||||
#include "types.hh" // for BackedStringView, enumerate, Strings
|
||||
#include "value.hh" // for Value, nString
|
||||
#include "value/context.hh" // for NixStringContext
|
||||
|
||||
namespace nix {
|
||||
|
||||
std::shared_ptr<RegexCache> makeRegexCache()
|
||||
{
|
||||
return std::make_shared<RegexCache>();
|
||||
}
|
||||
|
||||
/**
|
||||
* builtins.baseNameOf
|
||||
*/
|
||||
|
||||
static void prim_baseNameOf(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
NixStringContext context;
|
||||
v.mkString(
|
||||
baseNameOf(*state.coerceToString(
|
||||
pos,
|
||||
*args[0],
|
||||
context,
|
||||
"while evaluating the first argument passed to builtins.baseNameOf",
|
||||
false,
|
||||
false
|
||||
)),
|
||||
context
|
||||
);
|
||||
}
|
||||
|
||||
PrimOp primop_baseNameOf({
|
||||
.name = "baseNameOf",
|
||||
.args = {"s"},
|
||||
.doc = R"(
|
||||
Return the *base name* of the string *s*, that is, everything
|
||||
following the final slash in the string. This is similar to the GNU
|
||||
`basename` command.
|
||||
)",
|
||||
.fun = prim_baseNameOf,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.compareVersions
|
||||
*/
|
||||
|
||||
static void prim_compareVersions(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto version1 = state.forceStringNoCtx(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.compareVersions"
|
||||
);
|
||||
auto version2 = state.forceStringNoCtx(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.compareVersions"
|
||||
);
|
||||
v.mkInt(compareVersions(version1, version2));
|
||||
}
|
||||
|
||||
PrimOp primop_compareVersions({
|
||||
.name = "__compareVersions",
|
||||
.args = {"s1", "s2"},
|
||||
.doc = R"(
|
||||
Compare two strings representing versions and return `-1` if
|
||||
version *s1* is older than version *s2*, `0` if they are the same,
|
||||
and `1` if *s1* is newer than *s2*. The version comparison
|
||||
algorithm is the same as the one used by [`nix-env
|
||||
-u`](../command-ref/nix-env.md#operation---upgrade).
|
||||
)",
|
||||
.fun = prim_compareVersions,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.concatStringsSep
|
||||
*/
|
||||
|
||||
static void prim_concatStringsSep(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
NixStringContext context;
|
||||
|
||||
auto sep = state.forceString(
|
||||
*args[0],
|
||||
context,
|
||||
pos,
|
||||
"while evaluating the first argument (the separator string) passed to "
|
||||
"builtins.concatStringsSep"
|
||||
);
|
||||
state.forceList(
|
||||
*args[1],
|
||||
pos,
|
||||
"while evaluating the second argument (the list of strings to concat) passed to "
|
||||
"builtins.concatStringsSep"
|
||||
);
|
||||
|
||||
std::string res;
|
||||
res.reserve((args[1]->listSize() + 32) * sep.size());
|
||||
bool first = true;
|
||||
|
||||
for (auto elem : args[1]->listItems()) {
|
||||
if (first) {
|
||||
first = false;
|
||||
} else {
|
||||
res += sep;
|
||||
}
|
||||
res += *state.coerceToString(
|
||||
pos,
|
||||
*elem,
|
||||
context,
|
||||
"while evaluating one element of the list of strings to concat passed to "
|
||||
"builtins.concatStringsSep"
|
||||
);
|
||||
}
|
||||
|
||||
v.mkString(res, context);
|
||||
}
|
||||
|
||||
PrimOp primop_concatStringsSep(
|
||||
{.name = "__concatStringsSep",
|
||||
.args = {"separator", "list"},
|
||||
.doc = R"(
|
||||
Concatenate a list of strings with a separator between each
|
||||
element, e.g. `concatStringsSep "/" ["usr" "local" "bin"] ==
|
||||
"usr/local/bin"`.
|
||||
)",
|
||||
.fun = prim_concatStringsSep}
|
||||
);
|
||||
|
||||
/**
|
||||
* builtins.match
|
||||
*/
|
||||
|
||||
void prim_match(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto re = state.forceStringNoCtx(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.match"
|
||||
);
|
||||
|
||||
try {
|
||||
|
||||
auto regex = state.regexCache->get(re);
|
||||
|
||||
NixStringContext context;
|
||||
const auto str = state.forceString(
|
||||
*args[1], context, pos, "while evaluating the second argument passed to builtins.match"
|
||||
);
|
||||
|
||||
std::cmatch match;
|
||||
if (!std::regex_match(str.begin(), str.end(), match, regex)) {
|
||||
v.mkNull();
|
||||
return;
|
||||
}
|
||||
|
||||
// the first match is the whole string
|
||||
const size_t len = match.size() - 1;
|
||||
state.mkList(v, len);
|
||||
for (size_t i = 0; i < len; ++i) {
|
||||
if (!match[i + 1].matched) {
|
||||
(v.listElems()[i] = state.allocValue())->mkNull();
|
||||
} else {
|
||||
(v.listElems()[i] = state.allocValue())->mkString(match[i + 1].str());
|
||||
}
|
||||
}
|
||||
|
||||
} catch (std::regex_error & e) {
|
||||
if (e.code() == std::regex_constants::error_space) {
|
||||
// limit is _GLIBCXX_REGEX_STATE_LIMIT for libstdc++
|
||||
state.error<EvalError>("memory limit exceeded by regular expression '%s'", re)
|
||||
.atPos(pos)
|
||||
.debugThrow();
|
||||
} else {
|
||||
state.error<EvalError>("invalid regular expression '%s'", re).atPos(pos).debugThrow();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_match({
|
||||
.name = "__match",
|
||||
.args = {"regex", "str"},
|
||||
.doc = R"s(
|
||||
Returns a list if the [extended POSIX regular
|
||||
expression](http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_04)
|
||||
*regex* matches *str* precisely, otherwise returns `null`. Each item
|
||||
in the list is a regex group.
|
||||
|
||||
```nix
|
||||
builtins.match "ab" "abc"
|
||||
```
|
||||
|
||||
Evaluates to `null`.
|
||||
|
||||
```nix
|
||||
builtins.match "abc" "abc"
|
||||
```
|
||||
|
||||
Evaluates to `[ ]`.
|
||||
|
||||
```nix
|
||||
builtins.match "a(b)(c)" "abc"
|
||||
```
|
||||
|
||||
Evaluates to `[ "b" "c" ]`.
|
||||
|
||||
```nix
|
||||
builtins.match "[[:space:]]+([[:upper:]]+)[[:space:]]+" " FOO "
|
||||
```
|
||||
|
||||
Evaluates to `[ "FOO" ]`.
|
||||
)s",
|
||||
.fun = prim_match,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.parseDrvName
|
||||
*/
|
||||
|
||||
static void prim_parseDrvName(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto name = state.forceStringNoCtx(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.parseDrvName"
|
||||
);
|
||||
DrvName parsed(name);
|
||||
auto attrs = state.buildBindings(2);
|
||||
attrs.alloc(state.sName).mkString(parsed.name);
|
||||
attrs.alloc("version").mkString(parsed.version);
|
||||
v.mkAttrs(attrs);
|
||||
}
|
||||
|
||||
PrimOp primop_parseDrvName({
|
||||
.name = "__parseDrvName",
|
||||
.args = {"s"},
|
||||
.doc = R"(
|
||||
Split the string *s* into a package name and version. The package
|
||||
name is everything up to but not including the first dash not followed
|
||||
by a letter, and the version is everything following that dash. The
|
||||
result is returned in a set `{ name, version }`. Thus,
|
||||
`builtins.parseDrvName "nix-0.12pre12876"` returns `{ name =
|
||||
"nix"; version = "0.12pre12876"; }`.
|
||||
)",
|
||||
.fun = prim_parseDrvName,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.replaceStrings
|
||||
*/
|
||||
|
||||
static void prim_replaceStrings(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceList(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.replaceStrings"
|
||||
);
|
||||
state.forceList(
|
||||
*args[1], pos, "while evaluating the second argument passed to builtins.replaceStrings"
|
||||
);
|
||||
if (args[0]->listSize() != args[1]->listSize()) {
|
||||
state
|
||||
.error<EvalError>(
|
||||
"'from' and 'to' arguments passed to builtins.replaceStrings have different lengths"
|
||||
)
|
||||
.atPos(pos)
|
||||
.debugThrow();
|
||||
}
|
||||
|
||||
std::vector<std::string> from;
|
||||
from.reserve(args[0]->listSize());
|
||||
for (auto elem : args[0]->listItems()) {
|
||||
from.emplace_back(state.forceString(
|
||||
*elem,
|
||||
pos,
|
||||
"while evaluating one of the strings to replace passed to builtins.replaceStrings"
|
||||
));
|
||||
}
|
||||
|
||||
std::unordered_map<size_t, std::string> cache;
|
||||
auto to = args[1]->listItems();
|
||||
|
||||
NixStringContext context;
|
||||
auto s = state.forceString(
|
||||
*args[2],
|
||||
context,
|
||||
pos,
|
||||
"while evaluating the third argument passed to builtins.replaceStrings"
|
||||
);
|
||||
|
||||
std::string res;
|
||||
// Loops one past last character to handle the case where 'from' contains an empty string.
|
||||
for (size_t p = 0; p <= s.size();) {
|
||||
bool found = false;
|
||||
auto i = from.begin();
|
||||
auto j = to.begin();
|
||||
size_t j_index = 0;
|
||||
for (; i != from.end(); ++i, ++j, ++j_index) {
|
||||
if (s.compare(p, i->size(), *i) == 0) {
|
||||
found = true;
|
||||
auto v = cache.find(j_index);
|
||||
if (v == cache.end()) {
|
||||
NixStringContext ctx;
|
||||
auto ts = state.forceString(
|
||||
**j,
|
||||
ctx,
|
||||
pos,
|
||||
"while evaluating one of the replacement strings passed to "
|
||||
"builtins.replaceStrings"
|
||||
);
|
||||
v = (cache.emplace(j_index, ts)).first;
|
||||
for (auto & path : ctx) {
|
||||
context.insert(path);
|
||||
}
|
||||
}
|
||||
res += v->second;
|
||||
if (i->empty()) {
|
||||
if (p < s.size()) {
|
||||
res += s[p];
|
||||
}
|
||||
p++;
|
||||
} else {
|
||||
p += i->size();
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (!found) {
|
||||
if (p < s.size()) {
|
||||
res += s[p];
|
||||
}
|
||||
p++;
|
||||
}
|
||||
}
|
||||
|
||||
v.mkString(res, context);
|
||||
}
|
||||
|
||||
PrimOp primop_replaceStrings({
|
||||
.name = "__replaceStrings",
|
||||
.args = {"from", "to", "s"},
|
||||
.doc = R"(
|
||||
Given string *s*, replace every occurrence of the strings in *from*
|
||||
with the corresponding string in *to*.
|
||||
|
||||
The argument *to* is lazy, that is, it is only evaluated when its corresponding pattern in *from* is matched in the string *s*
|
||||
|
||||
Example:
|
||||
|
||||
```nix
|
||||
builtins.replaceStrings ["oo" "a"] ["a" "i"] "foobar"
|
||||
```
|
||||
|
||||
evaluates to `"fabir"`.
|
||||
)",
|
||||
.fun = prim_replaceStrings,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.split
|
||||
*/
|
||||
|
||||
void prim_split(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto re = state.forceStringNoCtx(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.split"
|
||||
);
|
||||
|
||||
try {
|
||||
|
||||
auto regex = state.regexCache->get(re);
|
||||
|
||||
NixStringContext context;
|
||||
const auto str = state.forceString(
|
||||
*args[1], context, pos, "while evaluating the second argument passed to builtins.split"
|
||||
);
|
||||
|
||||
auto begin = std::cregex_iterator(str.begin(), str.end(), regex);
|
||||
auto end = std::cregex_iterator();
|
||||
|
||||
// Any matches results are surrounded by non-matching results.
|
||||
const size_t len = std::distance(begin, end);
|
||||
state.mkList(v, 2 * len + 1);
|
||||
size_t idx = 0;
|
||||
|
||||
if (len == 0) {
|
||||
v.listElems()[idx++] = args[1];
|
||||
return;
|
||||
}
|
||||
|
||||
for (auto i = begin; i != end; ++i) {
|
||||
assert(idx <= 2 * len + 1 - 3);
|
||||
auto match = *i;
|
||||
|
||||
// Add a string for non-matched characters.
|
||||
(v.listElems()[idx++] = state.allocValue())->mkString(match.prefix().str());
|
||||
|
||||
// Add a list for matched substrings.
|
||||
const size_t slen = match.size() - 1;
|
||||
auto elem = v.listElems()[idx++] = state.allocValue();
|
||||
|
||||
// Start at 1, beacause the first match is the whole string.
|
||||
state.mkList(*elem, slen);
|
||||
for (size_t si = 0; si < slen; ++si) {
|
||||
if (!match[si + 1].matched) {
|
||||
(elem->listElems()[si] = state.allocValue())->mkNull();
|
||||
} else {
|
||||
(elem->listElems()[si] = state.allocValue())->mkString(match[si + 1].str());
|
||||
}
|
||||
}
|
||||
|
||||
// Add a string for non-matched suffix characters.
|
||||
if (idx == 2 * len) {
|
||||
(v.listElems()[idx++] = state.allocValue())->mkString(match.suffix().str());
|
||||
}
|
||||
}
|
||||
|
||||
assert(idx == 2 * len + 1);
|
||||
|
||||
} catch (std::regex_error & e) {
|
||||
if (e.code() == std::regex_constants::error_space) {
|
||||
// limit is _GLIBCXX_REGEX_STATE_LIMIT for libstdc++
|
||||
state.error<EvalError>("memory limit exceeded by regular expression '%s'", re)
|
||||
.atPos(pos)
|
||||
.debugThrow();
|
||||
} else {
|
||||
state.error<EvalError>("invalid regular expression '%s'", re).atPos(pos).debugThrow();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_split({
|
||||
.name = "__split",
|
||||
.args = {"regex", "str"},
|
||||
.doc = R"s(
|
||||
Returns a list composed of non matched strings interleaved with the
|
||||
lists of the [extended POSIX regular
|
||||
expression](http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_04)
|
||||
*regex* matches of *str*. Each item in the lists of matched
|
||||
sequences is a regex group.
|
||||
|
||||
```nix
|
||||
builtins.split "(a)b" "abc"
|
||||
```
|
||||
|
||||
Evaluates to `[ "" [ "a" ] "c" ]`.
|
||||
|
||||
```nix
|
||||
builtins.split "([ac])" "abc"
|
||||
```
|
||||
|
||||
Evaluates to `[ "" [ "a" ] "b" [ "c" ] "" ]`.
|
||||
|
||||
```nix
|
||||
builtins.split "(a)|(c)" "abc"
|
||||
```
|
||||
|
||||
Evaluates to `[ "" [ "a" null ] "b" [ null "c" ] "" ]`.
|
||||
|
||||
```nix
|
||||
builtins.split "([[:upper:]]+)" " FOO "
|
||||
```
|
||||
|
||||
Evaluates to `[ " " [ "FOO" ] " " ]`.
|
||||
)s",
|
||||
.fun = prim_split,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.splitVersion
|
||||
*/
|
||||
|
||||
static void prim_splitVersion(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
auto version = state.forceStringNoCtx(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.splitVersion"
|
||||
);
|
||||
auto iter = version.cbegin();
|
||||
Strings components;
|
||||
while (iter != version.cend()) {
|
||||
auto component = nextComponent(iter, version.cend());
|
||||
if (component.empty()) {
|
||||
break;
|
||||
}
|
||||
components.emplace_back(component);
|
||||
}
|
||||
state.mkList(v, components.size());
|
||||
for (const auto & [n, component] : enumerate(components)) {
|
||||
(v.listElems()[n] = state.allocValue())->mkString(std::move(component));
|
||||
}
|
||||
}
|
||||
|
||||
PrimOp primop_splitVersion({
|
||||
.name = "__splitVersion",
|
||||
.args = {"s"},
|
||||
.doc = R"(
|
||||
Split a string representing a version into its components, by the
|
||||
same version splitting logic underlying the version comparison in
|
||||
[`nix-env -u`](../command-ref/nix-env.md#operation---upgrade).
|
||||
)",
|
||||
.fun = prim_splitVersion,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.stringLength
|
||||
*/
|
||||
|
||||
static void prim_stringLength(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
NixStringContext context;
|
||||
auto s = state.coerceToString(
|
||||
pos, *args[0], context, "while evaluating the argument passed to builtins.stringLength"
|
||||
);
|
||||
v.mkInt(s->size());
|
||||
}
|
||||
|
||||
PrimOp primop_stringLength({
|
||||
.name = "__stringLength",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Return the length of the string *e*. If *e* is not a string,
|
||||
evaluation is aborted.
|
||||
)",
|
||||
.fun = prim_stringLength,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.substring
|
||||
*/
|
||||
|
||||
static void prim_substring(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
int start = state.forceInt(
|
||||
*args[0],
|
||||
pos,
|
||||
"while evaluating the first argument (the start offset) passed to builtins.substring"
|
||||
);
|
||||
|
||||
if (start < 0) {
|
||||
state.error<EvalError>("negative start position in 'substring'").atPos(pos).debugThrow();
|
||||
}
|
||||
|
||||
int len = state.forceInt(
|
||||
*args[1],
|
||||
pos,
|
||||
"while evaluating the second argument (the substring length) passed to builtins.substring"
|
||||
);
|
||||
|
||||
// Special-case on empty substring to avoid O(n) strlen
|
||||
// This allows for the use of empty substrings to efficently capture string context
|
||||
if (len == 0) {
|
||||
state.forceValue(*args[2], pos);
|
||||
if (args[2]->type() == nString) {
|
||||
v.mkString("", args[2]->string.context);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
NixStringContext context;
|
||||
auto s = state.coerceToString(
|
||||
pos,
|
||||
*args[2],
|
||||
context,
|
||||
"while evaluating the third argument (the string) passed to builtins.substring"
|
||||
);
|
||||
|
||||
v.mkString((unsigned int) start >= s->size() ? "" : s->substr(start, len), context);
|
||||
}
|
||||
|
||||
PrimOp primop_substring({
|
||||
.name = "__substring",
|
||||
.args = {"start", "len", "s"},
|
||||
.doc = R"(
|
||||
Return the substring of *s* from character position *start*
|
||||
(zero-based) up to but not including *start + len*. If *start* is
|
||||
greater than the length of the string, an empty string is returned,
|
||||
and if *start + len* lies beyond the end of the string, only the
|
||||
substring up to the end of the string is returned. *start* must be
|
||||
non-negative. For example,
|
||||
|
||||
```nix
|
||||
builtins.substring 0 3 "nixos"
|
||||
```
|
||||
|
||||
evaluates to `"nix"`.
|
||||
)",
|
||||
.fun = prim_substring,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.toString
|
||||
*/
|
||||
|
||||
static void prim_toString(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
NixStringContext context;
|
||||
auto s = state.coerceToString(
|
||||
pos,
|
||||
*args[0],
|
||||
context,
|
||||
"while evaluating the first argument passed to builtins.toString",
|
||||
true,
|
||||
false
|
||||
);
|
||||
v.mkString(*s, context);
|
||||
}
|
||||
|
||||
PrimOp primop_toString({
|
||||
.name = "toString",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Convert the expression *e* to a string. *e* can be:
|
||||
|
||||
- A string (in which case the string is returned unmodified).
|
||||
|
||||
- A path (e.g., `toString /foo/bar` yields `"/foo/bar"`.
|
||||
|
||||
- A set containing `{ __toString = self: ...; }` or `{ outPath = ...; }`.
|
||||
|
||||
- An integer.
|
||||
|
||||
- A list, in which case the string representations of its elements
|
||||
are joined with spaces.
|
||||
|
||||
- A Boolean (`false` yields `""`, `true` yields `"1"`).
|
||||
|
||||
- `null`, which yields the empty string.
|
||||
)",
|
||||
.fun = prim_toString,
|
||||
});
|
||||
|
||||
}
|
||||
@@ -1,116 +0,0 @@
|
||||
#include <string> // for basic_string, char...
|
||||
#include <utility> // for move
|
||||
#include "canon-path.hh" // for CanonPath
|
||||
#include "environment-variables.hh"
|
||||
#include "error.hh" // for Error
|
||||
#include "eval-error.hh" // for EvalError, EvalErr...
|
||||
#include "eval-settings.hh" // for EvalSettings, eval...
|
||||
#include "eval.hh" // for EvalState, PrimOp
|
||||
#include "pos-idx.hh" // for PosIdx
|
||||
#include "pos-table.hh" // for PosTable
|
||||
#include "primops.hh" // for prim_exec
|
||||
#include "processes.hh"
|
||||
#include "types.hh" // for BackedStringView
|
||||
#include "value.hh" // for Value
|
||||
#include "value/context.hh" // for NixStringContext
|
||||
|
||||
namespace nix {
|
||||
|
||||
struct Expr;
|
||||
|
||||
/**
|
||||
* builtins.exec
|
||||
*/
|
||||
|
||||
void prim_exec(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceList(*args[0], pos, "while evaluating the first argument passed to builtins.exec");
|
||||
auto elems = args[0]->listElems();
|
||||
auto count = args[0]->listSize();
|
||||
if (count == 0) {
|
||||
state.error<EvalError>("at least one argument to 'exec' required").atPos(pos).debugThrow();
|
||||
}
|
||||
NixStringContext context;
|
||||
auto program =
|
||||
state
|
||||
.coerceToString(
|
||||
pos,
|
||||
*elems[0],
|
||||
context,
|
||||
"while evaluating the first element of the argument passed to builtins.exec",
|
||||
false,
|
||||
false
|
||||
)
|
||||
.toOwned();
|
||||
Strings commandArgs;
|
||||
for (unsigned int i = 1; i < args[0]->listSize(); ++i) {
|
||||
commandArgs.push_back(
|
||||
state
|
||||
.coerceToString(
|
||||
pos,
|
||||
*elems[i],
|
||||
context,
|
||||
"while evaluating an element of the argument passed to builtins.exec",
|
||||
false,
|
||||
false
|
||||
)
|
||||
.toOwned()
|
||||
);
|
||||
}
|
||||
try {
|
||||
auto _ = state.realiseContext(context); // FIXME: Handle CA derivations
|
||||
} catch (InvalidPathError & e) {
|
||||
state
|
||||
.error<EvalError>(
|
||||
"cannot execute '%1%', since path '%2%' is not valid", program, e.path
|
||||
)
|
||||
.atPos(pos)
|
||||
.debugThrow();
|
||||
}
|
||||
|
||||
auto output = runProgram(program, true, commandArgs);
|
||||
Expr * parsed;
|
||||
try {
|
||||
parsed = state.parseExprFromString(std::move(output), state.rootPath(CanonPath::root));
|
||||
} catch (Error & e) {
|
||||
e.addTrace(state.positions[pos], "while parsing the output from '%1%'", program);
|
||||
throw;
|
||||
}
|
||||
try {
|
||||
state.eval(parsed, v);
|
||||
} catch (Error & e) {
|
||||
e.addTrace(state.positions[pos], "while evaluating the output from '%1%'", program);
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* builtins.getEnv
|
||||
*/
|
||||
|
||||
static void prim_getEnv(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
std::string name(state.forceStringNoCtx(
|
||||
*args[0], pos, "while evaluating the first argument passed to builtins.getEnv"
|
||||
));
|
||||
v.mkString(evalSettings.restrictEval || evalSettings.pureEval ? "" : getEnv(name).value_or(""));
|
||||
}
|
||||
|
||||
PrimOp primop_getEnv({
|
||||
.name = "__getEnv",
|
||||
.args = {"s"},
|
||||
.doc = R"(
|
||||
`getEnv` returns the value of the environment variable *s*, or an
|
||||
empty string if the variable doesn’t exist. This function should be
|
||||
used with care, as it can introduce all sorts of nasty environment
|
||||
dependencies in your Nix expression.
|
||||
|
||||
`getEnv` is used in Nix Packages to locate the file
|
||||
`~/.nixpkgs/config.nix`, which contains user-local settings for Nix
|
||||
Packages. (That is, it does a `getEnv "HOME"` to locate the user’s
|
||||
home directory.)
|
||||
)",
|
||||
.fun = prim_getEnv,
|
||||
});
|
||||
|
||||
}
|
||||
@@ -1,119 +0,0 @@
|
||||
#include <sstream> // for basic_ostringstream, basic_ios, basic_os...
|
||||
#include "eval.hh" // for PrimOp, EvalState (ptr only)
|
||||
#include "pos-idx.hh" // for PosIdx
|
||||
#include "value.hh" // for Value
|
||||
#include "value-to-xml.hh" // for printValueAsXML
|
||||
#include "value/context.hh" // for NixStringContext
|
||||
|
||||
namespace nix {
|
||||
|
||||
/**
|
||||
* builtins.toXML
|
||||
*/
|
||||
|
||||
static void prim_toXML(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
std::ostringstream out;
|
||||
NixStringContext context;
|
||||
printValueAsXML(state, true, false, *args[0], out, context, pos);
|
||||
v.mkString(out.str(), context);
|
||||
}
|
||||
|
||||
PrimOp primop_toXML({
|
||||
.name = "__toXML",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Return a string containing an XML representation of *e*. The main
|
||||
application for `toXML` is to communicate information with the
|
||||
builder in a more structured format than plain environment
|
||||
variables.
|
||||
|
||||
Here is an example where this is the case:
|
||||
|
||||
```nix
|
||||
{ stdenv, fetchurl, libxslt, jira, uberwiki }:
|
||||
|
||||
stdenv.mkDerivation (rec {
|
||||
name = "web-server";
|
||||
|
||||
buildInputs = [ libxslt ];
|
||||
|
||||
builder = builtins.toFile "builder.sh" "
|
||||
source $stdenv/setup
|
||||
mkdir $out
|
||||
echo "$servlets" | xsltproc ${stylesheet} - > $out/server-conf.xml ①
|
||||
";
|
||||
|
||||
stylesheet = builtins.toFile "stylesheet.xsl" ②
|
||||
"<?xml version='1.0' encoding='UTF-8'?>
|
||||
<xsl:stylesheet xmlns:xsl='http://www.w3.org/1999/XSL/Transform' version='1.0'>
|
||||
<xsl:template match='/'>
|
||||
<Configure>
|
||||
<xsl:for-each select='/expr/list/attrs'>
|
||||
<Call name='addWebApplication'>
|
||||
<Arg><xsl:value-of select=\"attr[@name = 'path']/string/@value\" /></Arg>
|
||||
<Arg><xsl:value-of select=\"attr[@name = 'war']/path/@value\" /></Arg>
|
||||
</Call>
|
||||
</xsl:for-each>
|
||||
</Configure>
|
||||
</xsl:template>
|
||||
</xsl:stylesheet>
|
||||
";
|
||||
|
||||
servlets = builtins.toXML [ ③
|
||||
{ path = "/bugtracker"; war = jira + "/lib/atlassian-jira.war"; }
|
||||
{ path = "/wiki"; war = uberwiki + "/uberwiki.war"; }
|
||||
];
|
||||
})
|
||||
```
|
||||
|
||||
The builder is supposed to generate the configuration file for a
|
||||
[Jetty servlet container](http://jetty.mortbay.org/). A servlet
|
||||
container contains a number of servlets (`*.war` files) each
|
||||
exported under a specific URI prefix. So the servlet configuration
|
||||
is a list of sets containing the `path` and `war` of the servlet
|
||||
(①). This kind of information is difficult to communicate with the
|
||||
normal method of passing information through an environment
|
||||
variable, which just concatenates everything together into a
|
||||
string (which might just work in this case, but wouldn’t work if
|
||||
fields are optional or contain lists themselves). Instead the Nix
|
||||
expression is converted to an XML representation with `toXML`,
|
||||
which is unambiguous and can easily be processed with the
|
||||
appropriate tools. For instance, in the example an XSLT stylesheet
|
||||
(at point ②) is applied to it (at point ①) to generate the XML
|
||||
configuration file for the Jetty server. The XML representation
|
||||
produced at point ③ by `toXML` is as follows:
|
||||
|
||||
```xml
|
||||
<?xml version='1.0' encoding='utf-8'?>
|
||||
<expr>
|
||||
<list>
|
||||
<attrs>
|
||||
<attr name="path">
|
||||
<string value="/bugtracker" />
|
||||
</attr>
|
||||
<attr name="war">
|
||||
<path value="/nix/store/d1jh9pasa7k2...-jira/lib/atlassian-jira.war" />
|
||||
</attr>
|
||||
</attrs>
|
||||
<attrs>
|
||||
<attr name="path">
|
||||
<string value="/wiki" />
|
||||
</attr>
|
||||
<attr name="war">
|
||||
<path value="/nix/store/y6423b1yi4sx...-uberwiki/uberwiki.war" />
|
||||
</attr>
|
||||
</attrs>
|
||||
</list>
|
||||
</expr>
|
||||
```
|
||||
|
||||
Note that we used the `toFile` built-in to write the builder and
|
||||
the stylesheet “inline” in the Nix expression. The path of the
|
||||
stylesheet is spliced into the builder using the syntax `xsltproc
|
||||
${stylesheet}`.
|
||||
)",
|
||||
.fun = prim_toXML,
|
||||
});
|
||||
|
||||
}
|
||||
@@ -1,245 +0,0 @@
|
||||
#include <stdlib.h> // for abort
|
||||
#include <string> // for basic_string, char_traits, string
|
||||
#include "attr-set.hh" // for BindingsBuilder
|
||||
#include "eval-error.hh" // for TypeError, EvalErrorBuilder
|
||||
#include "eval.hh" // for PrimOp, EvalState
|
||||
#include "nixexpr.hh" // for ExprLambda, Formal, Formals
|
||||
#include "pos-idx.hh" // for PosIdx
|
||||
#include "symbol-table.hh" // for Symbol
|
||||
#include "value.hh" // for Value, nAttrs, nBool, nFloat, nFunction
|
||||
|
||||
namespace nix {
|
||||
|
||||
/**
|
||||
* Generic isType function
|
||||
*/
|
||||
|
||||
static inline auto prim_isType(auto n)
|
||||
{
|
||||
return [n](EvalState & state, const PosIdx pos, Value ** args, Value & v) {
|
||||
state.forceValue(*args[0], pos);
|
||||
v.mkBool(args[0]->type() == n);
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* builtins.functionArgs
|
||||
*/
|
||||
|
||||
static void prim_functionArgs(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceValue(*args[0], pos);
|
||||
if (args[0]->isPrimOpApp() || args[0]->isPrimOp()) {
|
||||
v.mkAttrs(&state.emptyBindings);
|
||||
return;
|
||||
}
|
||||
if (!args[0]->isLambda()) {
|
||||
state.error<TypeError>("'functionArgs' requires a function").atPos(pos).debugThrow();
|
||||
}
|
||||
|
||||
if (!args[0]->lambda.fun->hasFormals()) {
|
||||
v.mkAttrs(&state.emptyBindings);
|
||||
return;
|
||||
}
|
||||
|
||||
auto attrs = state.buildBindings(args[0]->lambda.fun->formals->formals.size());
|
||||
for (auto & i : args[0]->lambda.fun->formals->formals) {
|
||||
// !!! should optimise booleans (allocate only once)
|
||||
attrs.alloc(i.name, i.pos).mkBool(i.def);
|
||||
}
|
||||
v.mkAttrs(attrs);
|
||||
}
|
||||
|
||||
PrimOp primop_functionArgs({
|
||||
.name = "__functionArgs",
|
||||
.args = {"f"},
|
||||
.doc = R"(
|
||||
Return a set containing the names of the formal arguments expected
|
||||
by the function *f*. The value of each attribute is a Boolean
|
||||
denoting whether the corresponding argument has a default value. For
|
||||
instance, `functionArgs ({ x, y ? 123}: ...) = { x = false; y =
|
||||
true; }`.
|
||||
|
||||
"Formal argument" here refers to the attributes pattern-matched by
|
||||
the function. Plain lambdas are not included, e.g. `functionArgs (x:
|
||||
...) = { }`.
|
||||
)",
|
||||
.fun = prim_functionArgs,
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.isAttrs
|
||||
*/
|
||||
|
||||
PrimOp primop_isAttrs({
|
||||
.name = "__isAttrs",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Return `true` if *e* evaluates to a set, and `false` otherwise.
|
||||
)",
|
||||
.fun = prim_isType(nAttrs),
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.isBool
|
||||
*/
|
||||
|
||||
PrimOp primop_isBool({
|
||||
.name = "__isBool",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Return `true` if *e* evaluates to a bool, and `false` otherwise.
|
||||
)",
|
||||
.fun = prim_isType(nBool),
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.Float
|
||||
*/
|
||||
|
||||
PrimOp primop_isFloat({
|
||||
.name = "__isFloat",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Return `true` if *e* evaluates to a float, and `false` otherwise.
|
||||
)",
|
||||
.fun = prim_isType(nFloat),
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.isFunction
|
||||
*/
|
||||
|
||||
PrimOp primop_isFunction({
|
||||
.name = "__isFunction",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Return `true` if *e* evaluates to a function, and `false` otherwise.
|
||||
)",
|
||||
.fun = prim_isType(nFunction),
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.isInt
|
||||
*/
|
||||
|
||||
PrimOp primop_isInt({
|
||||
.name = "__isInt",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Return `true` if *e* evaluates to an integer, and `false` otherwise.
|
||||
)",
|
||||
.fun = prim_isType(nInt),
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.isList
|
||||
*/
|
||||
|
||||
PrimOp primop_isList({
|
||||
.name = "__isList",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Return `true` if *e* evaluates to a list, and `false` otherwise.
|
||||
)",
|
||||
.fun = prim_isType(nList),
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.isNull
|
||||
*/
|
||||
|
||||
PrimOp primop_isNull({
|
||||
.name = "isNull",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Return `true` if *e* evaluates to `null`, and `false` otherwise.
|
||||
|
||||
This is equivalent to `e == null`.
|
||||
)",
|
||||
.fun = prim_isType(nNull),
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.isPath
|
||||
*/
|
||||
|
||||
PrimOp primop_isPath({
|
||||
.name = "__isPath",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Return `true` if *e* evaluates to a path, and `false` otherwise.
|
||||
)",
|
||||
.fun = prim_isType(nPath),
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.isString
|
||||
*/
|
||||
|
||||
PrimOp primop_isString({
|
||||
.name = "__isString",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Return `true` if *e* evaluates to a string, and `false` otherwise.
|
||||
)",
|
||||
.fun = prim_isType(nString),
|
||||
});
|
||||
|
||||
/**
|
||||
* builtins.typeOf
|
||||
*/
|
||||
|
||||
static void prim_typeOf(EvalState & state, const PosIdx pos, Value ** args, Value & v)
|
||||
{
|
||||
state.forceValue(*args[0], pos);
|
||||
std::string t;
|
||||
switch (args[0]->type()) {
|
||||
case nInt:
|
||||
t = "int";
|
||||
break;
|
||||
case nBool:
|
||||
t = "bool";
|
||||
break;
|
||||
case nString:
|
||||
t = "string";
|
||||
break;
|
||||
case nPath:
|
||||
t = "path";
|
||||
break;
|
||||
case nNull:
|
||||
t = "null";
|
||||
break;
|
||||
case nAttrs:
|
||||
t = "set";
|
||||
break;
|
||||
case nList:
|
||||
t = "list";
|
||||
break;
|
||||
case nFunction:
|
||||
t = "lambda";
|
||||
break;
|
||||
case nExternal:
|
||||
t = args[0]->external->typeOf();
|
||||
break;
|
||||
case nFloat:
|
||||
t = "float";
|
||||
break;
|
||||
case nThunk:
|
||||
abort();
|
||||
}
|
||||
v.mkString(t);
|
||||
}
|
||||
|
||||
PrimOp primop_typeOf({
|
||||
.name = "__typeOf",
|
||||
.args = {"e"},
|
||||
.doc = R"(
|
||||
Return a string representing the type of the value *e*, namely
|
||||
`"int"`, `"bool"`, `"string"`, `"path"`, `"null"`, `"set"`,
|
||||
`"list"`, `"lambda"` or `"float"`.
|
||||
)",
|
||||
.fun = prim_typeOf,
|
||||
});
|
||||
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
#include <limits>
|
||||
#include <span>
|
||||
#include <unordered_set>
|
||||
#include <sstream>
|
||||
|
||||
#include "escape-string.hh"
|
||||
#include "print.hh"
|
||||
|
||||
@@ -87,7 +87,17 @@ struct FetchSettings : public Config
|
||||
{}, true, Xp::Flakes};
|
||||
|
||||
Setting<bool> acceptFlakeConfig{this, false, "accept-flake-config",
|
||||
"Whether to accept nix configuration from a flake without prompting.",
|
||||
R"(
|
||||
Whether to accept Lix configuration from the `nixConfig` attribute of
|
||||
a flake without prompting. This is almost always a very bad idea.
|
||||
|
||||
Setting this setting as a trusted user allows Nix flakes to gain root
|
||||
access on your machine if they set one of the several
|
||||
trusted-user-only settings that execute commands as root.
|
||||
|
||||
See [multi-user installations](@docroot@/installation/multi-user.md)
|
||||
for more details on the Lix security model.
|
||||
)",
|
||||
{}, true, Xp::Flakes};
|
||||
|
||||
Setting<std::string> commitLockFileSummary{
|
||||
|
||||
@@ -322,9 +322,15 @@ struct GitLabInputScheme : GitArchiveInputScheme
|
||||
readFile(
|
||||
store->toRealPath(
|
||||
downloadFile(store, url, "source", false, headers).storePath)));
|
||||
auto rev = Hash::parseAny(std::string(json[0]["id"]), htSHA1);
|
||||
debug("HEAD revision for '%s' is %s", url, rev.gitRev());
|
||||
return rev;
|
||||
if (json.is_array() && json.size() >= 1 && json[0]["id"] != nullptr) {
|
||||
auto rev = Hash::parseAny(std::string(json[0]["id"]), htSHA1);
|
||||
debug("HEAD revision for '%s' is %s", url, rev.gitRev());
|
||||
return rev;
|
||||
} else if (json.is_array() && json.size() == 0) {
|
||||
throw Error("No commits returned by GitLab API -- does the ref really exist?");
|
||||
} else {
|
||||
throw Error("Didn't know what to do with response from GitLab: %s", json);
|
||||
}
|
||||
}
|
||||
|
||||
DownloadUrl getDownloadUrl(const Input & input) const override
|
||||
|
||||
@@ -30,6 +30,7 @@ libfetchers = library(
|
||||
liblixutil,
|
||||
nlohmann_json,
|
||||
],
|
||||
cpp_pch : cpp_pch,
|
||||
install : true,
|
||||
# FIXME(Qyriad): is this right?
|
||||
install_rpath : libdir,
|
||||
@@ -55,3 +56,13 @@ liblixfetchers = declare_dependency(
|
||||
include_directories : include_directories('.'),
|
||||
link_with : libfetchers,
|
||||
)
|
||||
|
||||
# FIXME: remove when https://git.lix.systems/lix-project/lix/issues/359 is fixed.
|
||||
if is_static
|
||||
liblixfetchers_mstatic = declare_dependency(
|
||||
include_directories : include_directories('.'),
|
||||
link_whole : libfetchers,
|
||||
)
|
||||
else
|
||||
liblixfetchers_mstatic = liblixfetchers
|
||||
endif
|
||||
|
||||
@@ -33,7 +33,7 @@ MixCommonArgs::MixCommonArgs(const std::string & programName)
|
||||
|
||||
addFlag({
|
||||
.longName = "option",
|
||||
.description = "Set the Nix configuration setting *name* to *value* (overriding `nix.conf`).",
|
||||
.description = "Set the Lix configuration setting *name* to *value* (overriding `nix.conf`).",
|
||||
.category = miscCategory,
|
||||
.labels = {"name", "value"},
|
||||
.handler = {[this](std::string name, std::string value) {
|
||||
|
||||
@@ -20,6 +20,7 @@ libmain = library(
|
||||
liblixutil,
|
||||
liblixstore,
|
||||
],
|
||||
cpp_pch : cpp_pch,
|
||||
install : true,
|
||||
# FIXME(Qyriad): is this right?
|
||||
install_rpath : libdir,
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
#include <atomic>
|
||||
#include <map>
|
||||
#include <thread>
|
||||
#include <sstream>
|
||||
#include <iostream>
|
||||
#include <chrono>
|
||||
|
||||
|
||||
@@ -124,7 +124,7 @@ void detectStackOverflow();
|
||||
*
|
||||
* This is called by the handler installed by detectStackOverflow().
|
||||
*
|
||||
* This gives Nix library consumers a limit opportunity to report the error
|
||||
* This gives Lix library consumers a limit opportunity to report the error
|
||||
* condition. The handler should exit the process.
|
||||
* See defaultStackOverflowHandler() for a reference implementation.
|
||||
*
|
||||
|
||||
@@ -16,6 +16,7 @@
|
||||
#include <future>
|
||||
#include <regex>
|
||||
#include <fstream>
|
||||
#include <sstream>
|
||||
|
||||
#include <nlohmann/json.hpp>
|
||||
|
||||
|
||||
@@ -64,10 +64,13 @@ extern "C" int sandbox_init_with_parameters(const char *profile, uint64_t flags,
|
||||
|
||||
namespace nix {
|
||||
|
||||
namespace {
|
||||
/**
|
||||
* The system for which Nix is compiled.
|
||||
*/
|
||||
constexpr std::string_view nativeSystem = SYSTEM;
|
||||
[[gnu::unused]]
|
||||
constexpr const std::string_view nativeSystem = SYSTEM;
|
||||
}
|
||||
|
||||
void handleDiffHook(
|
||||
uid_t uid, uid_t gid,
|
||||
@@ -1844,8 +1847,12 @@ void LocalDerivationGoal::runChild()
|
||||
copyFile(path, chrootRootDir + path, { .followSymlinks = true });
|
||||
}
|
||||
|
||||
if (settings.caFile != "")
|
||||
pathsInChroot.try_emplace("/etc/ssl/certs/ca-certificates.crt", settings.caFile, true);
|
||||
if (settings.caFile != "" && pathExists(settings.caFile)) {
|
||||
// For the same reasons as above, copy the CA certificates file too.
|
||||
// It should be even less likely to change during the build than resolv.conf.
|
||||
createDirs(chrootRootDir + "/etc/ssl/certs");
|
||||
copyFile(settings.caFile, chrootRootDir + "/etc/ssl/certs/ca-certificates.crt", { .followSymlinks = true });
|
||||
}
|
||||
}
|
||||
|
||||
for (auto & i : ss) pathsInChroot.emplace(i, i);
|
||||
|
||||
@@ -14,6 +14,8 @@
|
||||
#include "derivations.hh"
|
||||
#include "args.hh"
|
||||
|
||||
#include <sstream>
|
||||
|
||||
namespace nix::daemon {
|
||||
|
||||
Sink & operator << (Sink & sink, const Logger::Fields & fields)
|
||||
@@ -241,7 +243,7 @@ struct ClientSettings
|
||||
} else if (name == settings.pluginFiles.name) {
|
||||
if (tokenizeString<Paths>(value) != settings.pluginFiles.get())
|
||||
warn("Ignoring the client-specified plugin-files.\n"
|
||||
"The client specifying plugins to the daemon never made sense, and was removed in Nix >=2.14.");
|
||||
"The client specifying plugins to the daemon never made sense, and was removed in Nix.");
|
||||
}
|
||||
else if (trusted
|
||||
|| name == settings.buildTimeout.name
|
||||
|
||||
@@ -208,7 +208,7 @@ struct DerivationType {
|
||||
/**
|
||||
* Impure derivation type
|
||||
*
|
||||
* This is similar at buil-time to the content addressed, not standboxed, not fixed
|
||||
* This is similar at build-time to the content addressed, not sandboxed, not fixed
|
||||
* type, but has some restrictions on its usage.
|
||||
*/
|
||||
struct Impure {
|
||||
|
||||
@@ -196,7 +196,6 @@ struct curlFileTransfer : public FileTransfer
|
||||
result.immutableUrl = match.str(1);
|
||||
else
|
||||
debug("got invalid link header '%s'", value);
|
||||
warn("foo %s", value);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -657,7 +656,7 @@ struct curlFileTransfer : public FileTransfer
|
||||
/* Ugly hack to support s3:// URIs. */
|
||||
if (request.uri.starts_with("s3://")) {
|
||||
// FIXME: do this on a worker thread
|
||||
return std::async(std::launch::deferred, [uri{request.uri}] {
|
||||
return std::async(std::launch::deferred, [uri{request.uri}]() -> FileTransferResult {
|
||||
#if ENABLE_S3
|
||||
auto [bucketName, key, params] = parseS3Uri(uri);
|
||||
|
||||
|
||||
@@ -40,12 +40,12 @@ struct FileTransferSettings : Config
|
||||
this, 300, "stalled-download-timeout",
|
||||
R"(
|
||||
The timeout (in seconds) for receiving data from servers
|
||||
during download. Nix cancels idle downloads after this
|
||||
during download. Lix cancels idle downloads after this
|
||||
timeout's duration.
|
||||
)"};
|
||||
|
||||
Setting<unsigned int> tries{this, 5, "download-attempts",
|
||||
"How often Nix will attempt to download a file before giving up."};
|
||||
"How often Lix will attempt to download a file before giving up."};
|
||||
};
|
||||
|
||||
extern FileTransferSettings fileTransferSettings;
|
||||
|
||||
@@ -87,7 +87,7 @@ Settings::Settings()
|
||||
|
||||
/* Set the build hook location
|
||||
|
||||
For builds we perform a self-invocation, so Nix has to be self-aware.
|
||||
For builds we perform a self-invocation, so Lix has to be self-aware.
|
||||
That is, it has to know where it is installed. We don't think it's sentient.
|
||||
|
||||
Normally, nix is installed according to `nixBinDir`, which is set at compile time,
|
||||
@@ -101,7 +101,7 @@ Settings::Settings()
|
||||
current executable, using `getSelfExe()`.
|
||||
|
||||
As a last resort, we resort to `PATH`. Hopefully we find a `nix` there that's compatible.
|
||||
If you're porting Nix to a new platform, that might be good enough for a while, but
|
||||
If you're porting Lix to a new platform, that might be good enough for a while, but
|
||||
you'll want to improve `getSelfExe()` to work on your platform.
|
||||
*/
|
||||
std::string nixExePath = nixBinDir + "/nix";
|
||||
|
||||
+55
-48
@@ -131,7 +131,7 @@ public:
|
||||
Setting<bool> tryFallback{
|
||||
this, false, "fallback",
|
||||
R"(
|
||||
If set to `true`, Nix will fall back to building from source if a
|
||||
If set to `true`, Lix will fall back to building from source if a
|
||||
binary substitute fails. This is equivalent to the `--fallback`
|
||||
flag. The default is `false`.
|
||||
)",
|
||||
@@ -149,9 +149,9 @@ public:
|
||||
MaxBuildJobsSetting maxBuildJobs{
|
||||
this, 1, "max-jobs",
|
||||
R"(
|
||||
This option defines the maximum number of jobs that Nix will try to
|
||||
This option defines the maximum number of jobs that Lix will try to
|
||||
build in parallel. The default is `1`. The special value `auto`
|
||||
causes Nix to use the number of CPUs in your system. `0` is useful
|
||||
causes Lix to use the number of CPUs in your system. `0` is useful
|
||||
when using remote builders to prevent any local builds (except for
|
||||
`preferLocalBuild` derivation attribute which executes locally
|
||||
regardless). It can be overridden using the `--max-jobs` (`-j`)
|
||||
@@ -195,11 +195,11 @@ public:
|
||||
Setting<std::string> thisSystem{
|
||||
this, SYSTEM, "system",
|
||||
R"(
|
||||
The system type of the current Nix installation.
|
||||
Nix will only build a given [derivation](@docroot@/language/derivations.md) locally when its `system` attribute equals any of the values specified here or in [`extra-platforms`](#conf-extra-platforms).
|
||||
The system type of the current Lix installation.
|
||||
Lix will only build a given [derivation](@docroot@/language/derivations.md) locally when its `system` attribute equals any of the values specified here or in [`extra-platforms`](#conf-extra-platforms).
|
||||
|
||||
The default value is set when Nix itself is compiled for the system it will run on.
|
||||
The following system types are widely used, as [Nix is actively supported on these platforms](@docroot@/contributing/hacking.md#platforms):
|
||||
The default value is set when Lix itself is compiled for the system it will run on.
|
||||
The following system types are widely used, as [Lix is actively supported on these platforms](@docroot@/contributing/hacking.md#platforms):
|
||||
|
||||
- `x86_64-linux`
|
||||
- `x86_64-darwin`
|
||||
@@ -210,7 +210,7 @@ public:
|
||||
- `armv7l-linux`
|
||||
|
||||
In general, you do not have to modify this setting.
|
||||
While you can force Nix to run a Darwin-specific `builder` executable on a Linux machine, the result would obviously be wrong.
|
||||
While you can force Lix to run a Darwin-specific `builder` executable on a Linux machine, the result would obviously be wrong.
|
||||
|
||||
This value is available in the Nix language as
|
||||
[`builtins.currentSystem`](@docroot@/language/builtin-constants.md#builtins-currentSystem)
|
||||
@@ -252,8 +252,8 @@ public:
|
||||
R"(
|
||||
The path to the helper program that executes remote builds.
|
||||
|
||||
Nix communicates with the build hook over `stdio` using a custom protocol to request builds that cannot be performed directly by the Nix daemon.
|
||||
The default value is the internal Nix binary that implements remote building.
|
||||
Lix communicates with the build hook over `stdio` using a custom protocol to request builds that cannot be performed directly by the Nix daemon.
|
||||
The default value is the internal Lix binary that implements remote building.
|
||||
|
||||
> **Important**
|
||||
>
|
||||
@@ -270,7 +270,7 @@ public:
|
||||
Setting<bool> alwaysAllowSubstitutes{
|
||||
this, false, "always-allow-substitutes",
|
||||
R"(
|
||||
If set to `true`, Nix will ignore the `allowSubstitutes` attribute in
|
||||
If set to `true`, Lix will ignore the `allowSubstitutes` attribute in
|
||||
derivations and always attempt to use available substituters.
|
||||
For more information on `allowSubstitutes`, see [the manual chapter on advanced attributes](../language/advanced-attributes.md).
|
||||
)"};
|
||||
@@ -278,7 +278,7 @@ public:
|
||||
Setting<bool> buildersUseSubstitutes{
|
||||
this, false, "builders-use-substitutes",
|
||||
R"(
|
||||
If set to `true`, Nix will instruct remote build machines to use
|
||||
If set to `true`, Lix will instruct remote build machines to use
|
||||
their own binary substitutes if available. In practical terms, this
|
||||
means that remote hosts will fetch as many build dependencies as
|
||||
possible from their own substitutes (e.g, from `cache.nixos.org`),
|
||||
@@ -317,7 +317,7 @@ public:
|
||||
Setting<bool> useSubstitutes{
|
||||
this, true, "substitute",
|
||||
R"(
|
||||
If set to `true` (default), Nix will use binary substitutes if
|
||||
If set to `true` (default), Lix will use binary substitutes if
|
||||
available. This option can be disabled to force building from
|
||||
source.
|
||||
)",
|
||||
@@ -326,23 +326,23 @@ public:
|
||||
Setting<std::string> buildUsersGroup{
|
||||
this, "", "build-users-group",
|
||||
R"(
|
||||
This options specifies the Unix group containing the Nix build user
|
||||
accounts. In multi-user Nix installations, builds should not be
|
||||
performed by the Nix account since that would allow users to
|
||||
This options specifies the Unix group containing the Lix build user
|
||||
accounts. In multi-user Lix installations, builds should not be
|
||||
performed by the Lix account since that would allow users to
|
||||
arbitrarily modify the Nix store and database by supplying specially
|
||||
crafted builders; and they cannot be performed by the calling user
|
||||
since that would allow him/her to influence the build result.
|
||||
since that would allow them to influence the build result.
|
||||
|
||||
Therefore, if this option is non-empty and specifies a valid group,
|
||||
builds will be performed under the user accounts that are a member
|
||||
of the group specified here (as listed in `/etc/group`). Those user
|
||||
accounts should not be used for any other purpose\!
|
||||
|
||||
Nix will never run two builds under the same user account at the
|
||||
Lix will never run two builds under the same user account at the
|
||||
same time. This is to prevent an obvious security hole: a malicious
|
||||
user writing a Nix expression that modifies the build result of a
|
||||
legitimate Nix expression being built by another user. Therefore it
|
||||
is good to have as many Nix build user accounts as you can spare.
|
||||
is good to have as many Lix build user accounts as you can spare.
|
||||
(Remember: uids are cheap.)
|
||||
|
||||
The build users should have permission to create files in the Nix
|
||||
@@ -351,11 +351,18 @@ public:
|
||||
and its mode should be `1775`.
|
||||
|
||||
If the build users group is empty, builds will be performed under
|
||||
the uid of the Nix process (that is, the uid of the caller if
|
||||
`NIX_REMOTE` is empty, the uid under which the Nix daemon runs if
|
||||
`NIX_REMOTE` is `daemon`). Obviously, this should not be used
|
||||
the uid of the Lix process (that is, the uid of the caller if
|
||||
both `NIX_REMOTE` is either empty or `auto` and the Nix store is
|
||||
owned by that user, or, alternatively, the uid under which the Nix
|
||||
daemon runs if `NIX_REMOTE` is `daemon` or if it is `auto` and the
|
||||
store is not owned by the caller). Obviously, this should not be used
|
||||
with a nix daemon accessible to untrusted clients.
|
||||
|
||||
For the avoidance of doubt, explicitly setting this to *empty* with a
|
||||
Lix daemon running as root means that builds will be executed as root
|
||||
with respect to the rest of the system.
|
||||
We intend to fix this: https://git.lix.systems/lix-project/lix/issues/242
|
||||
|
||||
Defaults to `nixbld` when running as root, *empty* otherwise.
|
||||
)",
|
||||
{}, false};
|
||||
@@ -405,7 +412,7 @@ public:
|
||||
Setting<bool> keepLog{
|
||||
this, true, "keep-build-log",
|
||||
R"(
|
||||
If set to `true` (the default), Nix will write the build log of a
|
||||
If set to `true` (the default), Lix will write the build log of a
|
||||
derivation (i.e. the standard output and error of its builder) to
|
||||
the directory `/nix/var/log/nix/drvs`. The build log can be
|
||||
retrieved using the command `nix-store -l path`.
|
||||
@@ -468,7 +475,7 @@ public:
|
||||
Setting<bool> autoOptimiseStore{
|
||||
this, false, "auto-optimise-store",
|
||||
R"(
|
||||
If set to `true`, Nix automatically detects files in the store
|
||||
If set to `true`, Lix automatically detects files in the store
|
||||
that have identical contents, and replaces them with hard links to
|
||||
a single copy. This saves disk space. If set to `false` (the
|
||||
default), you can still run `nix-store --optimise` to get rid of
|
||||
@@ -519,7 +526,7 @@ public:
|
||||
network).
|
||||
|
||||
Currently, sandboxing only work on Linux and macOS. The use of a
|
||||
sandbox requires that Nix is run as root (so you should use the
|
||||
sandbox requires that Lix is run as root (so you should use the
|
||||
“build users” feature to perform the actual builds under different
|
||||
users than root).
|
||||
|
||||
@@ -545,7 +552,7 @@ public:
|
||||
If the source is in the Nix store, then its closure will be added to
|
||||
the sandbox as well.
|
||||
|
||||
Depending on how Nix was built, the default value for this option
|
||||
Depending on how Lix was built, the default value for this option
|
||||
may be empty or provide `/bin/sh` as a bind-mount of `bash`.
|
||||
)",
|
||||
{"build-chroot-dirs", "build-sandbox-paths"}};
|
||||
@@ -556,7 +563,7 @@ public:
|
||||
Setting<bool> requireDropSupplementaryGroups{this, getuid() == 0, "require-drop-supplementary-groups",
|
||||
R"(
|
||||
Following the principle of least privilege,
|
||||
Nix will attempt to drop supplementary groups when building with sandboxing.
|
||||
Lix will attempt to drop supplementary groups when building with sandboxing.
|
||||
|
||||
However this can fail under some circumstances.
|
||||
For example, if the user lacks the `CAP_SETGID` capability.
|
||||
@@ -642,7 +649,7 @@ public:
|
||||
A whitespace-separated list of public keys.
|
||||
|
||||
At least one of the following condition must be met
|
||||
for Nix to accept copying a store object from another
|
||||
for Lix to accept copying a store object from another
|
||||
Nix store (such as a substituter):
|
||||
|
||||
- the store object has been signed using a key in the trusted keys list
|
||||
@@ -665,14 +672,14 @@ public:
|
||||
this, 60 * 60, "tarball-ttl",
|
||||
R"(
|
||||
The number of seconds a downloaded tarball is considered fresh. If
|
||||
the cached tarball is stale, Nix will check whether it is still up
|
||||
to date using the ETag header. Nix will download a new version if
|
||||
the cached tarball is stale, Lix will check whether it is still up
|
||||
to date using the ETag header. Lix will download a new version if
|
||||
the ETag header is unsupported, or the cached ETag doesn't match.
|
||||
|
||||
Setting the TTL to `0` forces Nix to always check if the tarball is
|
||||
Setting the TTL to `0` forces Lix to always check if the tarball is
|
||||
up to date.
|
||||
|
||||
Nix caches tarballs in `$XDG_CACHE_HOME/nix/tarballs`.
|
||||
Lix caches tarballs in `$XDG_CACHE_HOME/nix/tarballs`.
|
||||
|
||||
Files fetched via `NIX_PATH`, `fetchGit`, `fetchMercurial`,
|
||||
`fetchTarball`, and `fetchurl` respect this TTL.
|
||||
@@ -701,7 +708,7 @@ public:
|
||||
R"(
|
||||
System types of executables that can be run on this machine.
|
||||
|
||||
Nix will only build a given [derivation](@docroot@/language/derivations.md) locally when its `system` attribute equals any of the values specified here or in the [`system` option](#conf-system).
|
||||
Lix will only build a given [derivation](@docroot@/language/derivations.md) locally when its `system` attribute equals any of the values specified here or in the [`system` option](#conf-system).
|
||||
|
||||
Setting this can be useful to build derivations locally on compatible machines:
|
||||
- `i686-linux` executables can be run on `x86_64-linux` machines (set by default)
|
||||
@@ -748,16 +755,16 @@ public:
|
||||
"substituters",
|
||||
R"(
|
||||
A list of [URLs of Nix stores](@docroot@/command-ref/new-cli/nix3-help-stores.md#store-url-format) to be used as substituters, separated by whitespace.
|
||||
A substituter is an additional [store](@docroot@/glossary.md#gloss-store) from which Nix can obtain [store objects](@docroot@/glossary.md#gloss-store-object) instead of building them.
|
||||
A substituter is an additional [store](@docroot@/glossary.md#gloss-store) from which Lix can obtain [store objects](@docroot@/glossary.md#gloss-store-object) instead of building them.
|
||||
|
||||
Substituters are tried based on their priority value, which each substituter can set independently.
|
||||
Lower value means higher priority.
|
||||
The default is `https://cache.nixos.org`, which has a priority of 40.
|
||||
|
||||
At least one of the following conditions must be met for Nix to use a substituter:
|
||||
At least one of the following conditions must be met for Lix to use a substituter:
|
||||
|
||||
- The substituter is in the [`trusted-substituters`](#conf-trusted-substituters) list
|
||||
- The user calling Nix is in the [`trusted-users`](#conf-trusted-users) list
|
||||
- The user calling Lix is in the [`trusted-users`](#conf-trusted-users) list
|
||||
|
||||
In addition, each store path should be trusted as described in [`trusted-public-keys`](#conf-trusted-public-keys)
|
||||
)",
|
||||
@@ -866,13 +873,13 @@ public:
|
||||
this, 0, "download-speed",
|
||||
R"(
|
||||
Specify the maximum transfer rate in kilobytes per second you want
|
||||
Nix to use for downloads.
|
||||
Lix to use for downloads.
|
||||
)"};
|
||||
|
||||
Setting<std::string> netrcFile{
|
||||
this, fmt("%s/%s", nixConfDir, "netrc"), "netrc-file",
|
||||
R"(
|
||||
If set to an absolute path to a `netrc` file, Nix will use the HTTP
|
||||
If set to an absolute path to a `netrc` file, Lix will use the HTTP
|
||||
authentication credentials in this file when trying to download from
|
||||
a remote host through HTTP or HTTPS. Defaults to
|
||||
`$NIX_CONF_DIR/netrc`.
|
||||
@@ -898,7 +905,7 @@ public:
|
||||
this, getDefaultSSLCertFile(), "ssl-cert-file",
|
||||
R"(
|
||||
The path of a file containing CA certificates used to
|
||||
authenticate `https://` downloads. Nix by default will use
|
||||
authenticate `https://` downloads. Lix by default will use
|
||||
the first of the following files that exists:
|
||||
|
||||
1. `/etc/ssl/certs/ca-certificates.crt`
|
||||
@@ -915,7 +922,7 @@ public:
|
||||
Setting<StringSet> ignoredAcls{
|
||||
this, {"security.selinux", "system.nfs4_acl", "security.csm"}, "ignored-acls",
|
||||
R"(
|
||||
A list of ACLs that should be ignored, normally Nix attempts to
|
||||
A list of ACLs that should be ignored, normally Lix attempts to
|
||||
remove all ACLs from files and directories in the Nix store, but
|
||||
some ACLs like `security.selinux` or `system.nfs4_acl` can't be
|
||||
removed even by root. Therefore it's best to just ignore them.
|
||||
@@ -926,7 +933,7 @@ public:
|
||||
this, {}, "hashed-mirrors",
|
||||
R"(
|
||||
A list of web servers used by `builtins.fetchurl` to obtain files by
|
||||
hash. Given a hash type *ht* and a base-16 hash *h*, Nix will try to
|
||||
hash. Given a hash type *ht* and a base-16 hash *h*, Lix will try to
|
||||
download the file from *hashed-mirror*/*ht*/*h*. This allows files to
|
||||
be downloaded even if they have disappeared from their original URI.
|
||||
For example, given an example mirror `http://tarballs.nixos.org/`,
|
||||
@@ -939,7 +946,7 @@ public:
|
||||
}
|
||||
```
|
||||
|
||||
Nix will attempt to download this file from
|
||||
Lix will attempt to download this file from
|
||||
`http://tarballs.nixos.org/sha256/2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae`
|
||||
first. If it is not available there, if will try the original URI.
|
||||
)"};
|
||||
@@ -948,7 +955,7 @@ public:
|
||||
this, 0, "min-free",
|
||||
R"(
|
||||
When free disk space in `/nix/store` drops below `min-free` during a
|
||||
build, Nix performs a garbage-collection until `max-free` bytes are
|
||||
build, Lix performs a garbage-collection until `max-free` bytes are
|
||||
available or there is no more garbage. A value of `0` (the default)
|
||||
disables this feature.
|
||||
)"};
|
||||
@@ -983,7 +990,7 @@ public:
|
||||
itself, they must be DSOs compatible with the instance of Nix
|
||||
running at the time (i.e. compiled against the same headers, not
|
||||
linked to any incompatible libraries). They should not be linked to
|
||||
any Nix libs directly, as those will be available already at load
|
||||
any Lix libs directly, as those will be available already at load
|
||||
time.
|
||||
|
||||
If an entry in the list is a directory, all files in the directory
|
||||
@@ -996,7 +1003,7 @@ public:
|
||||
Setting<bool> allowSymlinkedStore{
|
||||
this, false, "allow-symlinked-store",
|
||||
R"(
|
||||
If set to `true`, Nix will stop complaining if the store directory
|
||||
If set to `true`, Lix will stop complaining if the store directory
|
||||
(typically /nix/store) contains symlink components.
|
||||
|
||||
This risks making some builds "impure" because builders sometimes
|
||||
@@ -1009,13 +1016,13 @@ public:
|
||||
Setting<bool> useXDGBaseDirectories{
|
||||
this, false, "use-xdg-base-directories",
|
||||
R"(
|
||||
If set to `true`, Nix will conform to the [XDG Base Directory Specification] for files in `$HOME`.
|
||||
If set to `true`, Lix will conform to the [XDG Base Directory Specification] for files in `$HOME`.
|
||||
The environment variables used to implement this are documented in the [Environment Variables section](@docroot@/command-ref/env-common.md).
|
||||
|
||||
[XDG Base Directory Specification]: https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
|
||||
|
||||
> **Warning**
|
||||
> This changes the location of some well-known symlinks that Nix creates, which might break tools that rely on the old, non-XDG-conformant locations.
|
||||
> This changes the location of some well-known symlinks that Lix creates, which might break tools that rely on the old, non-XDG-conformant locations.
|
||||
|
||||
In particular, the following locations change:
|
||||
|
||||
@@ -1025,7 +1032,7 @@ public:
|
||||
| `~/.nix-defexpr` | `$XDG_STATE_HOME/nix/defexpr` |
|
||||
| `~/.nix-channels` | `$XDG_STATE_HOME/nix/channels` |
|
||||
|
||||
If you already have Nix installed and are using [profiles](@docroot@/package-management/profiles.md) or [channels](@docroot@/command-ref/nix-channel.md), you should migrate manually when you enable this option.
|
||||
If you already have Lix installed and are using [profiles](@docroot@/package-management/profiles.md) or [channels](@docroot@/command-ref/nix-channel.md), you should migrate manually when you enable this option.
|
||||
If `$XDG_STATE_HOME` is not set, use `$HOME/.local/state/nix` instead of `$XDG_STATE_HOME/nix`.
|
||||
This can be achieved with the following shell commands:
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
* a glibc git checkout via `rg --pcre2 'define __NR_fchmodat2 (?!452)'`.
|
||||
*
|
||||
* The problem is that glibc 2.39 and libseccomp 2.5.5 are needed to
|
||||
* get the syscall number. However, a Nix built against nixpkgs 23.11
|
||||
* get the syscall number. However, a Lix built against nixpkgs 23.11
|
||||
* (glibc 2.38) should still have the issue fixed without depending
|
||||
* on the build environment.
|
||||
*
|
||||
|
||||
@@ -18,12 +18,12 @@ struct LocalFSStoreConfig : virtual StoreConfig
|
||||
const PathSetting stateDir{this,
|
||||
rootDir.get() ? *rootDir.get() + "/nix/var/nix" : settings.nixStateDir,
|
||||
"state",
|
||||
"Directory where Nix will store state."};
|
||||
"Directory where Lix will store state."};
|
||||
|
||||
const PathSetting logDir{this,
|
||||
rootDir.get() ? *rootDir.get() + "/nix/var/log/nix" : settings.nixLogDir,
|
||||
"log",
|
||||
"directory where Nix will store log files."};
|
||||
"directory where Lix will store log files."};
|
||||
|
||||
const PathSetting realStoreDir{this,
|
||||
rootDir.get() ? *rootDir.get() + "/nix/store" : storeDir, "real",
|
||||
|
||||
@@ -313,13 +313,13 @@ LocalStore::LocalStore(const Params & params)
|
||||
throw Error(
|
||||
"Your Nix store has a database in Berkeley DB format,\n"
|
||||
"which is no longer supported. To convert to the new format,\n"
|
||||
"please upgrade Nix to version 0.12 first.");
|
||||
"please use the original Nix version 0.12 first.");
|
||||
|
||||
if (curSchema < 6)
|
||||
throw Error(
|
||||
"Your Nix store has a database in flat file format,\n"
|
||||
"which is no longer supported. To convert to the new format,\n"
|
||||
"please upgrade Nix to version 1.11 first.");
|
||||
"please use the original Nix version 1.11 first.");
|
||||
|
||||
if (!lockFile(globalLock.get(), ltWrite, false)) {
|
||||
printInfo("waiting for exclusive access to the Nix store...");
|
||||
@@ -550,7 +550,12 @@ void LocalStore::openDB(State & state, bool create)
|
||||
if (mode == "wal" ) {
|
||||
/* persist the WAL files when the DB connection is closed.
|
||||
* This allows for read-only connections without any write permissions
|
||||
* on the state directory to succeed on a closed database. */
|
||||
* on the state directory to succeed on a closed database. Setting the
|
||||
* journal_size_limit to 2^40 bytes results in the WAL files getting
|
||||
* truncated to 0 on exit and limits the on disk size of the WAL files
|
||||
* to 2^40 bytes following a checkpoint */
|
||||
if (sqlite3_exec(db, "pragma main.journal_size_limit = 1099511627776;", 0, 0, 0) != SQLITE_OK)
|
||||
SQLiteError::throw_(db, "setting journal_size_limit");
|
||||
int enable = 1;
|
||||
if (sqlite3_file_control(db, NULL, SQLITE_FCNTL_PERSIST_WAL, &enable) != SQLITE_OK)
|
||||
SQLiteError::throw_(db, "setting persistent WAL mode");
|
||||
|
||||
@@ -23,6 +23,9 @@ namespace nix {
|
||||
* 0.7. Version 2 was Nix 0.8 and 0.9. Version 3 is Nix 0.10.
|
||||
* Version 4 is Nix 0.11. Version 5 is Nix 0.12-0.16. Version 6 is
|
||||
* Nix 1.0. Version 7 is Nix 1.3. Version 10 is 2.0.
|
||||
*
|
||||
* Lix started at 2.90, it cannot ever go past version 10 (Nix 2.18),
|
||||
* since doing so will break compatibility with future CppNix versions.
|
||||
*/
|
||||
const int nixSchemaVersion = 10;
|
||||
|
||||
@@ -49,7 +52,7 @@ struct LocalStoreConfig : virtual LocalFSStoreConfig
|
||||
R"(
|
||||
Allow this store to be opened when its [database](@docroot@/glossary.md#gloss-nix-database) is on a read-only filesystem.
|
||||
|
||||
Normally Nix will attempt to open the store database in read-write mode, even for querying (when write access is not needed), causing it to fail if the database is on a read-only filesystem.
|
||||
Normally Lix will attempt to open the store database in read-write mode, even for querying (when write access is not needed), causing it to fail if the database is on a read-only filesystem.
|
||||
|
||||
Enable read-only mode to disable locking and open the SQLite database with the [`immutable` parameter](https://www.sqlite.org/c3ref/open.html) set.
|
||||
|
||||
|
||||
@@ -140,7 +140,7 @@ static Machine parseBuilderLine(const std::string & line)
|
||||
};
|
||||
|
||||
auto parseFloatField = [&](size_t fieldIndex) {
|
||||
const auto result = string2Int<float>(tokens[fieldIndex]);
|
||||
const auto result = string2Float<float>(tokens[fieldIndex]);
|
||||
if (!result) {
|
||||
throw FormatError("bad machine specification: failed to convert column #%lu in a row: '%s' to 'float'", fieldIndex, line);
|
||||
}
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user