Compare commits

..
11 Commits
Author SHA1 Message Date
Raito Bezarius 60f41e991d release: 2.93.1 "Bici Bici"
Release produced with releng/create_release.xsh

Change-Id: I64c89d0fba1e228136e50738e7a61d53306d14e6
2025-06-23 23:41:40 +02:00
Raito Bezarius 741a5ca774 release: release notes for 2.93.1
Release created with releng/create_release.xsh

Change-Id: I2d80bc68b7dd184ccf449de747ac46de6ac8786a
2025-06-23 23:40:50 +02:00
Raito Bezarius 2f75369094 version: 2.93.0 -> 2.93.1
* Announce the deprecation of ca-derivations and various other features
as planned initially.
* Fixes papercuts in 2.93.0 (SSH connections).
* Fixes the curl download bug for non-Nixpkgs users.
* Fixes CVE-2025-46415, CVE-2025-46416, CVE-2025-52991, CVE-2025-52992,
and CVE-2025-52993.

Change-Id: I8f700396a5ac57d2a1832833f83c22645c73697d
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-23 23:40:37 +02:00
Raito Bezariusandeldritch horrors 60d50ea31b libstore/build: automatic clean up of unsuccessfully built scratch outputs
When a build fails, its scratch output paths are not cleaned up.

Until recently, this was deemed not a problem but as part of the effort
to harden the Nix builds and protect these paths against being part of a
staged attack (race conditions, etc.), we automatically cleanup after
failed builds.

Change-Id: I58481b1cc83826298b9d80d37fecf81f117ccb09
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-23 20:49:17 +00:00
eldritch horrors 4f0c59b307 libstore: don't default build-dir to temp-dir
if a build directory is accessible to other users it is possible to
smuggle data in and out of build directories. usually this ins only
a build purity problem, but in combination with other issues it can
be used to break out of a build sandbox. to prevent this we default
to using a subdirectory of nixStateDir (which is more restrictive).

Change-Id: Iacfc9b50534de158618c815f9fb99d7dae1be4d0
2025-06-23 20:48:56 +00:00
eldritch horrorsandPuck Meerburg 46e2bb4ca6 libstore: use pasta for FODs if available
This allows using a userspace program, pasta, to handle comms between
the build sandbox, and the outside world; allowing for full isolation
including the network namespace, closing the "fixed-output derivation
talks to the host over an abstract domain socket" hole for good.

Co-Authored-By: Puck Meerburg <puck@puckipedia.com>
Change-Id: Ifd499b7dbb3784600a6e842fede65fc031ff9f15
2025-06-23 20:48:29 +00:00
eldritch horrorsandRaito Bezarius 201be7976b libutil: add capability support to runProgram2
launching pasta to not run as root will ambient require capabilities.

Change-Id: I1dd2506a1fa3944a9d9062123ef8a74903c597ea
2025-06-23 21:34:35 +02:00
eldritch horrorsandRaito Bezarius 8e4b4c62f5 libutil: add generic redirections runProgram2
explicit stderr redirection makes mergeStderrToStdout unnecessary also.

Change-Id: I63de929e6dc53f6c5ceb2d43c2ce288bfc04d872
2025-06-23 21:34:35 +02:00
eldritch horrorsandRaito Bezarius fe82fcdddc libutil: make RunningProgram more useful
make it moveable, make it killable, and add a stdout fd accessor.

Change-Id: I2387cbe8ac67b899a322cd6c7d306ef9ea7abcd0
2025-06-23 21:34:35 +02:00
Raito Bezarius 3af68dcef0 libutil: ensure that _deletePath does NOT use absolute paths with dirfds
When calling `_deletePath` with a parent file descriptor, `openat` is
made effective by using relative paths to the directory file descriptor.

To avoid the problem, the signature is changed to resist misuse with an
assert in the prologue of the function.

Change-Id: I6b3fc766bad2afe54dc27d47d1df3873e188de96
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-23 21:34:35 +02:00
Raito Bezarius 0001c1aeaf libstore: ensure that passAsFile is created in the original temp dir
This ensures that `passAsFile` data is created inside the expected
temporary build directory by `openat()` from the parent directory file
descriptor.

This avoids a TOCTOU which is part of the attack chain of CVE-????.

Change-Id: Ie5273446c4a19403088d0389ae8e3f473af8879a
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-23 17:25:59 +02:00
63 changed files with 647 additions and 1506 deletions
-6
View File
@@ -70,9 +70,6 @@ detroyejr:
display_name: Jonathan De Troye
github: detroyejr
edef:
github: edef1c
edolstra:
display_name: Eelco Dolstra
github: edolstra
@@ -200,9 +197,6 @@ roberth:
display_name: Robert Hensing
github: roberth
sandydoo:
github: sandydoo
seppel3210:
github: Seppel3210
-159
View File
@@ -1,163 +1,4 @@
# Lix 2.93 "Bici Bici" (2025-05-09)
# Lix 2.93.4 (2026-05-04)
## Fixes
- `build-dir` no longer defaults to `temp-dir` [cl/3453](https://gerrit.lix.systems/c/lix/+/3453)
The directory in which temporary build directories are created no longer defaults
to the value of the `temp-dir` setting to avoid builders making their directories
world-accessible. This behavior has been used to escape the build sandbox and can
cause build impurities even when not used maliciously. We now default to `builds`
in `NIX_STATE_DIR` (which is `/nix/var/nix/b` in the default configuration).
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
- Fix develop shells for derivations with escape codes [fj#991](https://git.lix.systems/lix-project/lix/issues/991) [cl/4154](https://gerrit.lix.systems/c/lix/+/4154) [cl/4155](https://gerrit.lix.systems/c/lix/+/4155)
ASCII control characters (including `\e`, used for ANSI escape codes) in derivation variables are now correctly escaped for `nix develop` and `nix print-dev-env`, instead of erroring.
Many thanks to [Qyriad](https://git.lix.systems/Qyriad) for this.
- Fix nix develop for derivations that rejects dependencies with structured attrs [fj#997](https://git.lix.systems/lix-project/lix/issues/997) [cl/4182](https://gerrit.lix.systems/c/lix/+/4182) [cl/4214](https://gerrit.lix.systems/c/lix/+/4214)
For the sake of concision, we refer to `disallowedReferences` in what follows,
but all output checks were equally fixed:
`{dis,}allowed{References,Requisites}`.
Derivations can define *output checks* to reject unwanted dependencies, such as
interpreters like `bash` or compilers like `gcc`. This can be done in two ways:
* **Legacy style**: `disallowedReferences = [ ... ]` in the environment.
* **Structured attrs**: `outputChecks.<output>.disallowedReferences = [ ... ]`,
typically used in `__json`.
Only the structured form supports derivations with multiple outputs.
`nix develop` internally rewrites derivations to create development shells. It
relied on the legacy `disallowedReferences`, and failed to honor the structured
variant. This led to broken shells in cases where `bashInteractive` was
explicitly disallowed using structured output checks, e.g. `nix develop
nixpkgs#systemd` after the "bash-less NixOS" changes.
This fix teaches `nix develop` to respect structured output checks, restoring
support for such derivations.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- `nix-shell` default shell directory is not `/tmp` anymore for `$NIX_BUILD_TOP` [fj#940](https://git.lix.systems/lix-project/lix/issues/940)
Previously, Lix `nix-shell`s could exit non-zero status when `stdenv`'s `dumpVars` phase failed to write to `$NIX_BUILD_TOP/env-vars`, despite `dumpVars` being intended as a debugging aid.
This happens when `TMPDIR` is not set and defaults therefore to `/tmp`, resulting in a `/tmp/env-vars` global file that every `nix-shell` wants to write.
We fix this issue by reusing a pre-created, unique, and writable location, as the build top directory, avoiding shell exiting from write failures silently.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) for this.
- Fix unsigned overflow leading to out-of-band write in the NAR parser [cl/5537](https://gerrit.lix.systems/c/lix/+/5537)
The NAR parser contained an unsigned integer overflow that could be used by an
attacker to write arbitrary data to an unknown memory location and possibly
achieve code execution. A successful attack on the system-wide Lix daemon
could lead to privilege escalation to root. Any process that involves NAR
serialization could trigger this issue, including (but not limited to)
- local user interaction, whether the users are trusted or untrusted
- malicious substituters sending malformed NARs
- remote builders sending malformed build results
- remote daemons sending malformed inputs when requesting remote builds
Successful attacks using this bug require ASLR weakening of some sort, whether
by architecture constraints (e.g. on 32 bit systems, where little randomization
is possible) or system configuration (e.g. low ASLR entropy when loading
libraries), and millions of attempts. Local attacks can be mounted in less than
an hour. Remote builds typically require a fresh SSH connection for each build
and are thus less susceptible. Only one attempt can be made by substituters for
every build using substituters, they are thus not a likely vector for attacks.
At the time of writing, MITRE has not assigned this a CVE yet.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae), [Raito Bezarius](https://git.lix.systems/raito), [edef](https://github.com/edef1c), and [sandydoo](https://github.com/sandydoo) for this.
# Lix 2.93.3 (2025-07-22)
## Improvements
- `--keep-failed` chowns the build directory to the user that request the build [cl/3678](https://gerrit.lix.systems/c/lix/+/3678)
Running a build with `--keep-failed` now chowns the temporary directory from the
builder user and group to the user that request the build if the build came from
a local user connected to the daemon. This makes inspecting failed derivations a
lot easier. On Linux the build directory made visible to the user will not be in
the same path as it was in the sandbox and continuing builds will usually break.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae) for this.
# Lix 2.93.2 (2025-06-30)
## Fixes
- Revert CVE-2025-52992 failed mitigation [fj#883](https://git.lix.systems/lix-project/lix/issues/883) [fj#887](https://git.lix.systems/lix-project/lix/issues/887) [cl/3444](https://gerrit.lix.systems/c/lix/+/3444) [cl/3528](https://gerrit.lix.systems/c/lix/+/3528)
Following the initial mitigation of **CVE-2025-52992** in `cl/3444`, we
received reports of **unexpected deletion of in-use store paths**.
Upon investigation, we found that the patch did **not correctly cancel all
automatic deleters**, resulting in potentially critical path loss during normal
operation.
Given the severity and time-sensitive nature of the situation ([see incident
report](https://lix.systems/blog/2025-06-27-lix-critical-bug/)), we evaluated
possible options to repair the behavior safely. However, we concluded that a
rushed fix would either
* **Overdelete**, i.e. breaking running systems, or,
* **Underdelete**, effectively **reopening CVE-2025-52992** while leaving
orphaned paths behind.
As **CVE-2025-52992 has no known exploit vector**, and correctness is critical
in the Lix project, we have **fully reverted the previous mitigations**.
The affected patches (`cl/3444`) have been rolled back for the time being.
Moving forward, the Lix team will rework this code path in a **long-term,
correctness-first fix** on the main branch. We will explore backporting it to
stable channels once its safety is assured.
We are deeply sorry for the stability incident and the Lix team remain
available for assisting you in recovering your systems.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
- Fallback to safe temp dir when build-dir is unwritable [fj#876](https://git.lix.systems/lix-project/lix/issues/876) [cl/3501](https://gerrit.lix.systems/c/lix/+/3501)
Non-daemon builds started failing with a permission error after introducing the `build-dir` option:
```
$ nix build --store ~/scratch nixpkgs#hello --rebuild
error: creating directory '/nix/var/nix/builds/nix-build-hello-2.12.2.drv-0': Permission denied
```
This happens because:
1. These builds are not run via the daemon, which owns `/nix/var/nix/builds`.
2. The user lacks permissions for that path.
We considered making `build-dir` a store-level option and defaulting it to `<chroot-root>/nix/var/nix/builds` for chroot stores, but opted instead for a fallback: if the default fails, Nix now creates a safe build directory under `/tmp`.
To avoid CVE-2025-52991, the fallback uses an extra path component between `/tmp` and the build dir.
**Note**: this fallback clutters `/tmp` with build directories that are not cleaned up. To prevent this, explicitly set `build-dir` to a path managed by Lix, even for local workloads.
Many thanks to [Raito Bezarius](https://git.lix.systems/raito) and [eldritch horrors](https://git.lix.systems/pennae) for this.
# Lix 2.93.1 (2025-06-23)
## Breaking Changes
+31 -31
View File
@@ -62,37 +62,38 @@ let
++ autoLayered
++ extraPkgs;
users = {
users =
{
root = {
uid = 0;
shell = "${pkgs.bashInteractive}/bin/bash";
home = "/root";
gid = 0;
groups = [ "root" ];
description = "System administrator";
};
nobody = {
uid = 65534;
shell = "${pkgs.shadow}/bin/nologin";
home = "/var/empty";
gid = 65534;
groups = [ "nobody" ];
description = "Unprivileged account (don't use!)";
};
}
// lib.listToAttrs (
map (n: {
name = "nixbld${toString n}";
value = {
uid = 30000 + n;
gid = 30000;
groups = [ "nixbld" ];
description = "Nix build user ${toString n}";
root = {
uid = 0;
shell = "${pkgs.bashInteractive}/bin/bash";
home = "/root";
gid = 0;
groups = [ "root" ];
description = "System administrator";
};
}) (lib.lists.range 1 32)
);
nobody = {
uid = 65534;
shell = "${pkgs.shadow}/bin/nologin";
home = "/var/empty";
gid = 65534;
groups = [ "nobody" ];
description = "Unprivileged account (don't use!)";
};
}
// lib.listToAttrs (
map (n: {
name = "nixbld${toString n}";
value = {
uid = 30000 + n;
gid = 30000;
groups = [ "nixbld" ];
description = "Nix build user ${toString n}";
};
}) (lib.lists.range 1 32)
);
groups = {
root.gid = 0;
@@ -360,8 +361,7 @@ let
"org.opencontainers.image.version" = pkgs.nix.version;
"org.opencontainers.image.description" =
"Minimal Lix container image, with some batteries included.";
}
// lib.optionalAttrs (lixRevision != null) { "org.opencontainers.image.revision" = lixRevision; };
} // lib.optionalAttrs (lixRevision != null) { "org.opencontainers.image.revision" = lixRevision; };
};
meta = {
Generated
+3 -3
View File
@@ -108,11 +108,11 @@
},
"nixpkgs_2": {
"locked": {
"lastModified": 1757198069,
"narHash": "sha256-m3VUcOD4rTs8J7S+3dOjWMrAjw6RcITC3XYQ98zhEFs=",
"lastModified": 1749522908,
"narHash": "sha256-eWANkhWXFL1MmaxzsZ9bhLCNT8OVs7CC+OXaSDGlA8A=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "0747026fc57ecb9c28901c7f7a2b5dc40e8af43c",
"rev": "e5cb99555c45a13dcc5f1317462238530b0066b7",
"type": "github"
},
"original": {
+10 -52
View File
@@ -175,13 +175,8 @@
{
nixStable = prev.nix;
# Nix 2.18 has been removed from Nixpkgs ≥ 25.05, so we need to reintroduce it ourselves for our tests.
nixVersions = prev.nixVersions // {
nix_2_3 = prev.nixVersions.nix_2_3.overrideAttrs (old: {
meta = old.meta // {
knownVulnerabilities = [ ];
};
});
# Nix 2.18 has been removed from Nixpkgs ≥ 25.05, so we need to reintroduce it ourselves for our tests.
nix_2_18 = nix_2_18.outputs.packages.${currentStdenv.hostPlatform.system}.default;
};
@@ -219,9 +214,6 @@
inherit versionSuffix officialRelease;
stdenv = currentStdenv;
busybox-sandbox-shell = final.busybox-sandbox-shell or final.default-busybox-sandbox-shell;
# See below
lowdown = final.lowdown_3_0;
lowdown-unsandboxed = final.lowdown_3_0.override { enableDarwinSandbox = false; };
};
lix-clang-tidy = final.callPackage ./subprojects/lix-clang-tidy { };
@@ -246,30 +238,6 @@
boehmgc-nix = final.nix.passthru.boehmgc-nix;
# And same thing for our build-release-notes package.
build-release-notes = final.nix.passthru.build-release-notes;
# As soon as Nixpkgs updates to >= 3.0.0, change to lowdown_2_0!
# We don't change the default version in order to not change the hash
# of Nix/Lix from upstream Nixpkgs.
lowdown_3_0 =
assert lib.versionOlder prev.lowdown.version "3.0.0";
prev.lowdown.overrideAttrs (
finalAttrs: prevAttrs: {
version = "3.0.1";
src = final.fetchurl {
url = "https://kristaps.bsd.lv/lowdown/snapshots/lowdown-${finalAttrs.version}.tar.gz";
sha512 = "fe68e1b7ff23f3992398356d7aa9a330dfd7b72e22bea9a91eeef74182b209ecea0c9f3e2b2216e1a07b2358da2b746238ec9cbbdeebdd3551cef14dd2d79f46";
};
# no longer compiles with GNU make
nativeBuildInputs = prevAttrs.nativeBuildInputs ++ [ final.bmake ];
# dylib fixups on darwin are no longer necessary
postInstall = "";
# doesn't work on darwin due to disallowed nested sandboxes
doInstallCheck = prevAttrs.doInstallCheck && !(final.stdenv.hostPlatform.isDarwin);
doCheck = prevAttrs.doCheck && !(final.stdenv.hostPlatform.isDarwin);
}
);
};
in
{
@@ -284,15 +252,6 @@
# Binary package for various platforms.
build = forAllSystems (system: self.packages.${system}.nix);
# Ensure support for lowdown < 3.0 doesn't regress for NixOS 25.11
build-lowdown_2_0.aarch64-linux = lib.genAttrs [ "aarch64-linux" ] (
system:
self.packages.${system}.nix.override {
lowdown = nixpkgsFor.${system}.native.lowdown;
lowdown-unsandboxed = nixpkgsFor.${system}.native.lowdown-unsandboxed;
}
);
devShell = forAllSystems (system: {
default = self.devShells.${system}.default;
clang = self.devShells.${system}.native-clangStdenvPackages;
@@ -439,16 +398,15 @@
in
pkgs.symlinkJoin {
name = "nixpkgs-lib-tests";
paths = [
testWithNix
]
# NOTE: nixpkgs 25.05 is being ... *creative*, and requires this dance to override
# the evaluator used for the test. it will break again in the future, don't worry.
++ lib.optionals pkgs.stdenv.isLinux [
((pkgs.callPackage "${nixpkgs}/ci/eval" { inherit nix; }).attrpathsSuperset {
evalSystem = system;
})
];
paths =
[ testWithNix ]
# NOTE: nixpkgs 25.05 is being ... *creative*, and requires this dance to override
# the evaluator used for the test. it will break again in the future, don't worry.
++ lib.optionals pkgs.stdenv.isLinux [
((pkgs.callPackage "${nixpkgs}/ci/eval" { nixVersions.latest = nix; }).attrpathsSuperset {
evalSystem = system;
})
];
}
);
};
+2 -4
View File
@@ -187,8 +187,7 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
if (packages && fromArgs)
throw UsageError("'-p' and '-E' are mutually exclusive");
AutoDelete tmpDir(createTempDir(myName));
AutoDelete buildTopTmpDir(createTempSubdir(tmpDir, "build-top"));
AutoDelete tmpDir(createTempDir("", myName));
if (outLink.empty())
outLink = (Path) tmpDir + "/result";
@@ -432,8 +431,7 @@ static void main_nix_build(AsyncIoRoot & aio, std::string programName, Strings a
}
// Don't use defaultTempDir() here! We want to preserve the user's TMPDIR for the shell
env["NIX_BUILD_TOP"] = env["TMPDIR"] = env["TEMPDIR"] = env["TMP"] = env["TEMP"] =
getEnvNonEmpty("TMPDIR").value_or(buildTopTmpDir);
env["NIX_BUILD_TOP"] = env["TMPDIR"] = env["TEMPDIR"] = env["TMP"] = env["TEMP"] = getEnvNonEmpty("TMPDIR").value_or("/tmp");
env["NIX_STORE"] = store->config().storeDir;
env["NIX_BUILD_CORES"] = std::to_string(settings.buildCores);
+2 -21
View File
@@ -11,34 +11,15 @@ namespace nix {
std::string renderMarkdownToTerminal(std::string_view markdown)
{
int windowWidth = getWindowSize().second;
size_t lowdown_cols = std::max(windowWidth - 5, 60);
struct lowdown_opts opts{
struct lowdown_opts opts {
.type = LOWDOWN_TERM,
#ifdef LOWDOWN_SEPARATE_TERM_OPTS
.term =
{
.cols = lowdown_cols,
.width = 0,
.hmargin = 0,
.hpadding = 4,
.vmargin = 0,
.centre = 0,
},
// maxdepth needs to be part of the ifdefs to match declaration order
.maxdepth = 20,
#else
.maxdepth = 20,
.cols = lowdown_cols,
.cols = (size_t) std::max(windowWidth - 5, 60),
.hmargin = 0,
.vmargin = 0,
#endif /* LOWDOWN_SEPARATE_TERM_OPTS */
.feat = LOWDOWN_COMMONMARK | LOWDOWN_FENCED | LOWDOWN_DEFLIST | LOWDOWN_TABLES,
#ifdef LOWDOWN_CONSOLIDATED_OFLAGS
.oflags = LOWDOWN_NOLINK,
#else
.oflags = LOWDOWN_TERM_NOLINK,
#endif /* LOWDOWN_CONSOLIDATED_OFLAGS */
};
if (!shouldANSI()) {
opts.oflags |= LOWDOWN_TERM_NOANSI;
+65 -147
View File
@@ -6,164 +6,82 @@
namespace nix {
#if HAVE_TOML11_4
/**
* This is what toml11 < 4.0 did when choosing the subsecond precision.
* TOML 1.0.0 spec doesn't define how sub-millisecond ranges should be handled and calls it
* implementation defined behavior. For a lack of a better choice we stick with what older versions
* of toml11 did [1].
*
* [1]:
* https://github.com/ToruNiina/toml11/blob/dcfe39a783a94e8d52c885e5883a6fbb21529019/toml/datetime.hpp#L282
*/
static size_t normalizeSubsecondPrecision(toml::local_time lt)
void prim_fromTOML(EvalState & state, Value * * args, Value & val)
{
auto millis = lt.millisecond;
auto micros = lt.microsecond;
auto nanos = lt.nanosecond;
if (millis != 0 || micros != 0 || nanos != 0) {
if (micros != 0 || nanos != 0) {
if (nanos != 0) {
return 9;
}
return 6;
}
return 3;
}
return 0;
}
/**
* Normalize date/time formats to serialize to the same strings as versions prior to toml11 4.0.
*
* Several things to consider:
*
* 1. Sub-millisecond range is represented the same way as in toml11 versions prior to 4.0.
* Precision is rounded towards the next multiple of 3 or capped at 9 digits.
* 2. Seconds must be specified. This may become optional in (yet unreleased) TOML 1.1.0, but 1.0.0
* defined local time in terms of RFC3339 [1].
* 3. date-time separator (`t`, `T` or space ` `) is canonicalized to an upper T. This is compliant
* with RFC3339 [1] 5.6: > Applications that generate this format SHOULD use upper case letters.
*
* [1]: https://datatracker.ietf.org/doc/html/rfc3339#section-5.6
*/
static void normalizeDatetimeFormat(toml::value & t)
{
if (t.is_local_datetime()) {
auto & ldt = t.as_local_datetime();
t.as_local_datetime_fmt() = {
.delimiter = toml::datetime_delimiter_kind::upper_T,
// https://datatracker.ietf.org/doc/html/rfc3339#section-5.6
.has_seconds = true, // Mandated by TOML 1.0.0
.subsecond_precision = normalizeSubsecondPrecision(ldt.time),
};
return;
}
if (t.is_offset_datetime()) {
auto & odt = t.as_offset_datetime();
t.as_offset_datetime_fmt() = {
.delimiter = toml::datetime_delimiter_kind::upper_T,
// https://datatracker.ietf.org/doc/html/rfc3339#section-5.6
.has_seconds = true, // Mandated by TOML 1.0.0
.subsecond_precision = normalizeSubsecondPrecision(odt.time),
};
return;
}
if (t.is_local_time()) {
auto & lt = t.as_local_time();
t.as_local_time_fmt() = {
.has_seconds = true, // Mandated by TOML 1.0.0
.subsecond_precision = normalizeSubsecondPrecision(lt),
};
return;
}
}
#endif
void prim_fromTOML(EvalState & state, Value ** args, Value & val)
{
auto toml = state.forceStringNoCtx(
*args[0], noPos, "while evaluating the argument passed to builtins.fromTOML"
);
auto toml = state.forceStringNoCtx(*args[0], noPos, "while evaluating the argument passed to builtins.fromTOML");
std::istringstream tomlStream(std::string{toml});
auto visit = [&](this const auto & self, Value & v, toml::value t) -> void {
switch (t.type()) {
case toml::value_t::table: {
auto table = toml::get<toml::table>(t);
auto attrs = state.ctx.buildBindings(table.size());
std::function<void(Value &, toml::value)> visit;
for (auto & elem : table) {
self(attrs.alloc(elem.first), elem.second);
}
visit = [&](Value & v, toml::value t) {
v.mkAttrs(attrs);
} break;
case toml::value_t::array: {
auto array = toml::get<std::vector<toml::value>>(t);
switch(t.type())
{
case toml::value_t::table:
{
auto table = toml::get<toml::table>(t);
size_t size = 0;
for (auto & i : table) { (void) i; size++; }
auto attrs = state.ctx.buildBindings(size);
for(auto & elem : table)
visit(attrs.alloc(elem.first), elem.second);
v.mkAttrs(attrs);
}
break;;
case toml::value_t::array:
{
auto array = toml::get<std::vector<toml::value>>(t);
size_t size = array.size();
v = state.ctx.mem.newList(size);
for (size_t i = 0; i < size; ++i)
visit(*(v.listElems()[i] = state.ctx.mem.allocValue()), array[i]);
}
break;;
case toml::value_t::boolean:
v.mkBool(toml::get<bool>(t));
break;;
case toml::value_t::integer:
v.mkInt(toml::get<int64_t>(t));
break;;
case toml::value_t::floating:
v.mkFloat(toml::get<NixFloat>(t));
break;;
case toml::value_t::string:
v.mkString(toml::get<std::string>(t));
break;;
case toml::value_t::local_datetime:
case toml::value_t::offset_datetime:
case toml::value_t::local_date:
case toml::value_t::local_time:
{
if (experimentalFeatureSettings.isEnabled(Xp::ParseTomlTimestamps)) {
auto attrs = state.ctx.buildBindings(2);
attrs.alloc("_type").mkString("timestamp");
std::ostringstream s;
s << t;
attrs.alloc("value").mkString(s.str());
v.mkAttrs(attrs);
} else {
// NOLINTNEXTLINE(lix-foreign-exceptions)
throw std::runtime_error("Dates and times are not supported");
}
}
break;;
case toml::value_t::empty:
v.mkNull();
break;;
size_t size = array.size();
v = state.ctx.mem.newList(size);
for (size_t i = 0; i < size; ++i) {
self(*(v.listElems()[i] = state.ctx.mem.allocValue()), array[i]);
}
} break;
case toml::value_t::boolean:
v.mkBool(toml::get<bool>(t));
break;
case toml::value_t::integer:
v.mkInt(toml::get<int64_t>(t));
break;
case toml::value_t::floating:
v.mkFloat(toml::get<NixFloat>(t));
break;
case toml::value_t::string:
v.mkString(toml::get<std::string>(t));
break;
case toml::value_t::local_datetime:
case toml::value_t::offset_datetime:
case toml::value_t::local_date:
case toml::value_t::local_time: {
if (experimentalFeatureSettings.isEnabled(Xp::ParseTomlTimestamps)) {
#if HAVE_TOML11_4
normalizeDatetimeFormat(t);
#endif
auto attrs = state.ctx.buildBindings(2);
attrs.alloc("_type").mkString("timestamp");
std::ostringstream s;
s << t;
attrs.alloc("value").mkString(s.str());
v.mkAttrs(attrs);
} else {
// NOLINTNEXTLINE(lix-foreign-exceptions)
throw std::runtime_error("Dates and times are not supported");
}
} break;
case toml::value_t::empty:
v.mkNull();
break;
}
};
try {
visit(
val,
toml::parse(
tomlStream,
"fromTOML" /* the "filename" */
#if HAVE_TOML11_4
,
toml::spec::v(
1, 0, 0
) // Be explicit that we are parsing TOML 1.0.0 without extensions
#endif
)
);
visit(val, toml::parse(tomlStream, "fromTOML" /* the "filename" */));
} catch (std::exception & e) { // NOLINT(lix-foreign-exceptions) // TODO: toml::syntax_error
state.ctx.errors.make<EvalError>("while parsing TOML: %s", e.what()).debugThrow();
}
-1
View File
@@ -190,7 +190,6 @@ try {
auto * buildIdDir = std::get_if<nar_index::Directory>(&narIndex);
for (auto subdir : { "lib", "debug", ".build-id" }) {
if (!buildIdDir) break;
// get returns nullptr subdir does not exist, and std::get_if propagates it.
buildIdDir = std::get_if<nar_index::Directory>(get(buildIdDir->contents, subdir));
}
+5 -1
View File
@@ -885,7 +885,11 @@ void replaceValidPath(const Path & storePath, const Path & tmpPath)
we're repairing (say) Glibc, we end up with a broken system. */
Path oldPath;
if (pathExists(storePath)) {
oldPath = makeTempSiblingPath(storePath);
do {
oldPath = makeTempPath(storePath, ".old");
// store paths are often directories so we can't just unlink() it
// let's make sure the path doesn't exist before we try to use it
} while (pathExists(oldPath));
movePath(storePath, oldPath);
}
+58 -150
View File
@@ -30,13 +30,11 @@
#include "platform/linux.hh"
#include <cstddef>
#include <dirent.h>
#include <exception>
#include <regex>
#include <queue>
#include <stdexcept>
#include <sys/stat.h>
#include <sys/un.h>
#include <fcntl.h>
#include <termios.h>
@@ -115,11 +113,7 @@ LocalDerivationGoal::~LocalDerivationGoal() noexcept(false)
/* Careful: we should never ever throw an exception from a
destructor. */
try { killChild(); } catch (...) { ignoreExceptionInDestructor(); }
try {
finalizeTmpDir(false, true);
} catch (...) {
ignoreExceptionInDestructor();
}
try { deleteTmpDir(false, true); } catch (...) { ignoreExceptionInDestructor(); }
}
@@ -369,14 +363,13 @@ bool LocalDerivationGoal::cleanupDecideWhetherDiskFull()
if (statvfs(localStore.config().realStoreDir.get().c_str(), &st) == 0 &&
(uint64_t) st.f_bavail * st.f_bsize < required)
diskFull = true;
if (statvfs(tmpDirRoot.c_str(), &st) == 0 && (uint64_t) st.f_bavail * st.f_bsize < required)
{
if (statvfs(tmpDir.c_str(), &st) == 0 &&
(uint64_t) st.f_bavail * st.f_bsize < required)
diskFull = true;
}
}
#endif
finalizeTmpDir(false);
deleteTmpDir(false);
/* Move paths out of the chroot for easier debugging of
build failures. */
@@ -395,15 +388,19 @@ bool LocalDerivationGoal::cleanupDecideWhetherDiskFull()
void LocalDerivationGoal::cleanupPostOutputsRegisteredModeCheck()
{
finalizeTmpDir(true);
deleteTmpDir(true);
}
void LocalDerivationGoal::cleanupPostOutputsRegisteredModeNonCheck()
{
/* Delete unused redirected outputs (when doing hash rewriting). */
for (auto & i : redirectedOutputs)
deletePath(worker.store.Store::toRealPath(i.second));
/* In the past, redirected outputs were manually tracked for deletion.
* Now that we have the scratch outputs cleaner which are a superset of
* redirected outputs, we just fire all uncancelled automatic deleters now.
*
* This should clean up any paths that IS NOT registered in the database.
*/
scratchOutputsCleaner.clear();
/* Delete the chroot (if we were using one). */
autoDelChroot.reset(); /* this runs the destructor */
@@ -490,76 +487,23 @@ try {
});
}
try {
auto buildDir = worker.buildDirOverride.value_or(settings.buildDir.get());
createDirs(settings.buildDir.get());
createDirs(buildDir);
/* Create a temporary directory where the build will take
place. */
tmpDirRoot = createTempSubdir(buildDir, std::nullopt, 0700);
} catch (SysError & e) {
/*
* Fallback to the global tmpdir and create a safe space there
* only if it's a permission error.
*/
if (e.errNo != EACCES) {
throw;
}
auto nixBuildsTmp = createTempDir(fmt("nix-builds-%s", geteuid()), 0700);
warn(
"Failed to use the system-wide build directory '%s', falling back to a temporary "
"directory inside '%s'",
settings.buildDir.get(),
nixBuildsTmp
);
tmpDirRoot = createTempSubdir(nixBuildsTmp, std::nullopt, 0700);
worker.buildDirOverride = nixBuildsTmp;
}
/* Create a temporary directory where the build will take
place. */
tmpDir = createTempDir(
settings.buildDir.get(),
"nix-build-" + std::string(drvPath.name()),
false,
false,
0700
);
/* The TOCTOU between the previous mkdir call and this open call is unavoidable due to
* POSIX semantics.*/
tmpDirRootFd = AutoCloseFD{open(tmpDirRoot.c_str(), O_RDONLY | O_NOFOLLOW | O_DIRECTORY)};
if (!tmpDirRootFd) {
throw SysError("failed to open the build temporary directory descriptor '%1%'", tmpDirRoot);
}
#if __APPLE__
// The Darwin sandbox ensures that builds cannot change the
// permissions of their own build directory. Unsandboxed builds
// disable this, but have no isolation by design anyway. The
// minimal sandbox (applied even when `sandbox = false`, though not
// when `_NIX_TEST_NO_SANDBOX` is set) prevents the creation of
// `set{u,g}id` files regardless.
tmpDir = tmpDirRoot;
tmpDirFd = std::move(tmpDirRootFd);
#else
// place the actual build directory in a subdirectory of tmpDirRoot. if
// we do not do this a build can `chown 777` its build directory and so
// make it accessible to everyone in the system, breaking isolation. we
// also need the intermediate level to be inaccessible to others. build
// processes must be able to at least traverse to the directory though,
// without being able to chmod. this means either mode 0750 or 0710. we
// use 0710 just to be extra safe; if we ever add more directories they
// will not be enumerable to other processes in the builder user group.
tmpDir = tmpDirRoot + "/b";
if (mkdirat(tmpDirRootFd.get(), "b", 0700)) {
throw SysError("failed to create the build temporary directory '%1%'", tmpDir);
}
tmpDirFd = AutoCloseFD{openat(tmpDirRootFd.get(), "b", O_RDONLY | O_NOFOLLOW | O_DIRECTORY)};
tmpDirFd = AutoCloseFD{open(tmpDir.c_str(), O_RDONLY | O_NOFOLLOW | O_DIRECTORY)};
if (!tmpDirFd)
throw SysError("failed to open the build temporary directory descriptor '%1%'", tmpDir);
if (buildUser) {
if (fchown(tmpDirRootFd.get(), -1, buildUser->getGID()) == -1) {
throw SysError("cannot change ownership of '%1%'", tmpDirRoot);
}
if (fchmod(tmpDirRootFd.get(), 0710) == -1) {
throw SysError("cannot change mode of '%1%'", tmpDirRoot);
}
}
#endif
chownToBuilder(tmpDirFd);
for (auto & [outputName, status] : initialOutputs) {
@@ -591,6 +535,10 @@ try {
to use a temporary path */
makeFallbackPath(status.known->path);
scratchOutputs.insert_or_assign(outputName, scratchPath);
/* Schedule this scratch output path for automatic deletion
* if we do not cancel it, e.g. when registering the outputs.
*/
scratchOutputsCleaner.insert_or_assign(outputName, worker.store.printStorePath(scratchPath));
/* Substitute output placeholders with the scratch output paths.
We'll use during the build. */
@@ -613,8 +561,6 @@ try {
std::string h2 { scratchPath.hashPart() };
inputRewrites[h1] = h2;
}
redirectedOutputs.insert_or_assign(std::move(fixedFinalPath), std::move(scratchPath));
}
/* Construct the environment passed to the builder. */
@@ -1541,8 +1487,6 @@ void LocalDerivationGoal::runChild()
if (getEnv("_NIX_TEST_NO_SANDBOX") != "1") {
Strings sandboxArgs;
sandboxArgs.push_back("_NIX_BUILD_TOP");
sandboxArgs.push_back(tmpDir);
sandboxArgs.push_back("_GLOBAL_TMP_DIR");
sandboxArgs.push_back(globalTmpDir);
if (allowLocalNetworking) {
@@ -2091,7 +2035,9 @@ try {
}
/* Don't register anything, since we already have the
previous versions which we're comparing. */
previous versions which we're comparing.
NOTE: this means that the `.check` path will be automatically deleted.
*/
continue;
}
@@ -2115,8 +2061,13 @@ try {
/* If it's a CA path, register it right away. This is necessary if it
isn't statically known so that we can safely unlock the path before
the next iteration */
if (newInfo.ca)
if (newInfo.ca) {
TRY_AWAIT(localStore.registerValidPaths({{newInfo.path, newInfo}}));
/* Cancel automatic deletion of that output if it was a scratch output. */
if (auto cleaner = scratchOutputsCleaner.extract(outputName)) {
cleaner.mapped().cancel();
}
}
infos.emplace(outputName, std::move(newInfo));
}
@@ -2156,6 +2107,13 @@ try {
infos2.insert_or_assign(newInfo.path, newInfo);
}
TRY_AWAIT(localStore.registerValidPaths(infos2));
/* Cancel automatic deletion of that output if it was a scratch output that we just registered. */
for (auto & [outputName, _ ] : infos) {
if (auto cleaner = scratchOutputsCleaner.extract(outputName)) {
cleaner.mapped().cancel();
}
}
}
/* In case of a fixed-output derivation hash mismatch, throw an
@@ -2189,6 +2147,13 @@ try {
builtOutputs.emplace(outputName, thisRealisation);
}
/* NOTE: At this point, all outputs MAY NOT have been registered.
* Therefore, there may remains auto-deleters pending in the cleaner list (`scratchOutputsCleaner`).
*
* They will be finally deleted but we have no way to assert they all have been, e.g.
* `assert(scratchOutputsCleaner.size() == 0)` cannot be written.
*/
co_return builtOutputs;
} catch (...) {
co_return result::current_exception();
@@ -2416,78 +2381,21 @@ try {
co_return result::current_exception();
}
// make `entry` in `parentFd` visible to the given user and group, preserving
// inode modes as much as possible. if the builder sets the mode of any inode
// to not be readable by the owner we keep this; not doing so could interfere
// with error analysis. if the builder used multiple uids or gids we will not
// keep them around and instead collapse them all onto the uid/gid given here
// to not leave around inodes owned by unassigned uids/gids in the system. we
// also clear setuid/setgid/sticky bits just to be safe even though a builder
// should not be able to set them to begin, otherwise we may leave setuid/gid
// executables in the tree even with user/group set to -1/-1. there have been
// enough bugs of this kind in the past to warrant some extra attention here.
static void makeVisible(int parentFd, const char * entry, uid_t user, gid_t group)
void LocalDerivationGoal::deleteTmpDir(bool force, bool duringDestruction)
{
struct stat st;
if (fstatat(parentFd, entry, &st, AT_SYMLINK_NOFOLLOW)) {
throw SysError("fstat(%s)", guessOrInventPathFromFD(parentFd));
}
if (S_ISDIR(st.st_mode)) {
int dirfd = openat(parentFd, entry, O_RDONLY | O_DIRECTORY | O_NOFOLLOW);
if (dirfd < 0) {
throw SysError("openat(%s/%s)", guessOrInventPathFromFD(parentFd), entry);
}
AutoCloseDir dir(fdopendir(dirfd));
if (!dir) {
close(dirfd);
throw SysError("fdopendir(%s/%s)", guessOrInventPathFromFD(parentFd), entry);
}
struct dirent * dirent;
while (errno = 0, dirent = readdir(dir.get())) {
if (strcmp(dirent->d_name, ".") == 0 || strcmp(dirent->d_name, "..") == 0) {
continue;
}
makeVisible(dirfd, dirent->d_name, user, group);
}
}
// ignore permissions errors for symlinks. linux can't chmod them.
// clear special permission bits while we're here, just to be safe
if (fchmodat(parentFd, entry, st.st_mode & 0777, AT_SYMLINK_NOFOLLOW) && !S_ISLNK(st.st_mode)) {
throw SysError("fchmod(%s)", guessOrInventPathFromFD(parentFd));
}
if (user != uid_t(-1) && group != gid_t(-1)
&& fchownat(parentFd, entry, user, group, AT_SYMLINK_NOFOLLOW))
{
throw SysError("fchown(%s)", guessOrInventPathFromFD(parentFd));
}
}
void LocalDerivationGoal::finalizeTmpDir(bool force, bool duringDestruction)
{
if (tmpDirRoot != "") {
if (tmpDir != "") {
/* Don't keep temporary directories for builtins because they
might have privileged stuff (like a copy of netrc). */
if (settings.keepFailed && !force && !drv->isBuiltin()) {
printError("note: keeping build directory '%s'", tmpDirRoot);
try {
// always make visible, but don't always chown. if we run as
// root we may not want to chown things to root:root so much
auto creds = worker.store.associatedCredentials();
makeVisible(
tmpDirFd.get(), ".", creds ? creds->user : -1, creds ? creds->group : -1
);
} catch (SysError & e) {
printError("error making '%s' accessible: %s", tmpDir, e.what());
}
chmod(tmpDirRoot.c_str(), 0755);
printError("note: keeping build directory '%s'", tmpDir);
chmod(tmpDir.c_str(), 0755);
}
else if (duringDestruction)
deletePathUninterruptible(tmpDirRoot);
deletePathUninterruptible(tmpDir);
else
deletePath(tmpDirRoot);
tmpDirRoot = "";
deletePath(tmpDir);
tmpDir = "";
}
}
+17 -9
View File
@@ -29,12 +29,12 @@ struct LocalDerivationGoal : public DerivationGoal
/**
* The temporary directory.
*/
Path tmpDirRoot, tmpDir;
Path tmpDir;
/**
* The temporary directory file descriptor
*/
AutoCloseFD tmpDirRootFd, tmpDirFd;
AutoCloseFD tmpDirFd;
/**
* The path of the temporary directory in the sandbox.
@@ -100,8 +100,6 @@ struct LocalDerivationGoal : public DerivationGoal
* Hash rewriting.
*/
StringMap inputRewrites, outputRewrites;
typedef map<StorePath, StorePath> RedirectedOutputs;
RedirectedOutputs redirectedOutputs;
/**
* The outputs paths used during the build.
@@ -118,6 +116,19 @@ struct LocalDerivationGoal : public DerivationGoal
* self-references.
*/
OutputPathMap scratchOutputs;
/**
* Output paths used during the build are scheduled for
* automatic cleanup unless they have been successfully built.
*
* `registerOutputs` take care of cancelling the cleanups
* and clearing this vector.
*
* `startBuilder` take care of filling this vector
* as `scratchOutputs` gets filled.
*
* This is a map from output names to automatic delete handles.
*/
std::map<std::string, AutoDelete> scratchOutputsCleaner;
/**
* Path registration info from the previous round, if we're
@@ -246,12 +257,9 @@ struct LocalDerivationGoal : public DerivationGoal
void cleanupPostOutputsRegisteredModeNonCheck() override;
/**
* Delete the temporary directory or make it visible to the user requesting
* this build, if a temporary directory was created at all. Temporary files
* of derivations using builtin builders are deleted even for `keep-failed`
* builds as otherwise we may expose secrets (e.g. from the system .netrc).
* Delete the temporary directory, if we have one.
*/
void finalizeTmpDir(bool force, bool duringDestruction = false);
void deleteTmpDir(bool force, bool duringDestruction = false);
/**
* Forcibly kill the child process, if any.
+2 -18
View File
@@ -26,28 +26,12 @@ R""(
; Allow getpwuid.
(allow mach-lookup (global-name "com.apple.system.opendirectoryd.libinfo"))
; Disallow messing with the toplevel build directory.
(deny file-write-owner file-write-flags file-write-xattr file-write-mount
file-write-unmount
(literal (param "_NIX_BUILD_TOP")))
; Nixpkgs does `chmod -R` on `$NIX_BUILD_TOP/$sourceRoot` by default,
; which results in it trying to set the mode of `$NIX_BUILD_TOP` when
; derivations set `sourceRoot = ".";`. Thankfully, the GNU `chmod(1)`
; treats `ENOTSUP` as a nonfatal, nonreported error in this case, and
; continues to descend into the directory tree.
;
; See: <https://gitweb.git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f=src/chmod.c;hb=refs/tags/v9.7#l312>
(deny file-write-mode (with errno ENOTSUP)
(literal (param "_NIX_BUILD_TOP")))
; Access to /tmp and the build directory.
; Access to /tmp.
; The network-outbound/network-inbound ones are for unix domain sockets, which
; we allow access to in TMPDIR (but if we allow them more broadly, you could in
; theory escape the sandbox)
(allow file* process-exec network-outbound network-inbound
(literal "/tmp")
(subpath TMPDIR)
(subpath (param "_NIX_BUILD_TOP")))
(literal "/tmp") (subpath TMPDIR))
; Some packages like to read the system version.
(allow file-read*
-1
View File
@@ -195,7 +195,6 @@ public:
Store & store;
Store & evalStore;
AsyncSemaphore substitutions, localBuilds;
std::optional<Path> buildDirOverride;
private:
kj::TaskSet children;
+1 -1
View File
@@ -96,7 +96,7 @@ void LocalStore::createTempRootsFile()
/* Create the temporary roots file for this process. */
while (true) {
auto tmp = makeTempPath(fnTempRoots);
auto tmp = makeTempPath(fnTempRoots, ".tmp");
AutoCloseFD fd{open(tmp.c_str(), O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC, 0600)};
if (!fd && errno != EEXIST) {
throw SysError("opening lock file '%1%'", tmp);
+3 -2
View File
@@ -63,10 +63,11 @@ protected:
const std::string & mimeType) override
{
auto path2 = binaryCacheDir + "/" + path;
Path tmp = makeTempPath(path2);
static std::atomic<int> counter{0};
Path tmp = fmt("%s.tmp.%d.%d", path2, getpid(), ++counter);
AutoDelete del(tmp, false);
StreamToSourceAdapter source(istream);
writeFileExcl(tmp, source);
writeFile(tmp, source);
renameFile(tmp, path2);
del.cancel();
}
+3 -3
View File
@@ -1629,7 +1629,7 @@ std::pair<Path, AutoCloseFD> LocalStore::createTempDirInStore()
/* There is a slight possibility that `tmpDir' gets deleted by
the GC between createTempDir() and when we acquire a lock on it.
We'll repeat until 'tmpDir' exists and we've locked it. */
tmpDirFn = createTempSubdir(config_.realStoreDir, "tmp");
tmpDirFn = createTempDir(config_.realStoreDir, "tmp");
tmpDirFd = AutoCloseFD{open(tmpDirFn.c_str(), O_RDONLY | O_DIRECTORY)};
if (tmpDirFd.get() < 0) {
continue;
@@ -2043,9 +2043,9 @@ try {
createDirs(dirOf(logPath));
auto tmpFile = makeTempSiblingPath(logPath);
auto tmpFile = fmt("%s.tmp.%d", logPath, getpid());
writeFileExcl(tmpFile, compress("bzip2", log));
writeFile(tmpFile, compress("bzip2", log));
renameFile(tmpFile, logPath);
co_return result::success();
-12
View File
@@ -135,8 +135,6 @@ private:
Sync<GCState> _gcState;
std::optional<AssociatedCredentials> association;
public:
const Path dbDir;
@@ -150,16 +148,6 @@ public:
LocalStoreConfig & config() override { return config_; }
const LocalStoreConfig & config() const override { return config_; }
std::optional<AssociatedCredentials> associatedCredentials() const override
{
return association;
}
void associateWithCredentials(uid_t user, gid_t group)
{
association = {user, group};
}
private:
const PublicKeys & getPublicKeys();
+2 -1
View File
@@ -217,7 +217,8 @@ void LocalStore::optimisePath_(Activity * act, OptimiseStats & stats,
its timestamp back to 0. */
MakeReadOnly makeReadOnly(mustToggle ? dirOfPath : "");
Path tempLink = makeTempPath(config().realStoreDir + "/");
Path tempLink = makeTempPath(config().realStoreDir, "/.tmp-link");
unlink(tempLink.c_str()); // just in case; ignore errors
if (link(linkPath.c_str(), tempLink.c_str()) == -1) {
if (errno == EMLINK) {
+3 -3
View File
@@ -2,12 +2,12 @@
name: build-dir
internalName: buildDir
settingType: PathsSetting<Path>
defaultText: "`«nixStateDir»/b`"
defaultExpr: nixStateDir + "/b"
defaultText: "`«nixStateDir»/builds`"
defaultExpr: nixStateDir + "/builds"
---
The directory on the host, in which derivations' temporary build directories are created.
If not set, Lix will use the `b` subdirectory of its configured state directory.
If not set, Lix will use the `builds` subdirectory of its configured state directory.
Lix will create this directory automatically with suitable permissions if it does not
exist, otherwise its permissions must allow all users to traverse the directory (i.e.
it must have `o+x` set, in unix parlance) for non-sandboxed builds to work correctly.
+1 -1
View File
@@ -22,7 +22,7 @@ SSH::SSH(const std::string & host, const std::optional<uint16_t> port, const std
throw Error("invalid SSH host name '%s'", host);
auto state(state_.lock());
state->tmpDir = std::make_unique<AutoDelete>(createTempDir("nix", 0700));
state->tmpDir = std::make_unique<AutoDelete>(createTempDir("", "nix", true, true, 0700));
}
void SSH::addCommonSSHOpts(Strings & args)
-16
View File
@@ -234,22 +234,6 @@ protected:
Store(const StoreConfig & config);
public:
struct AssociatedCredentials
{
uid_t user;
gid_t group;
};
/**
* Credentials of the context using this store if this store is proxied
* to somewhere else and the peer context is known. Only the daemon can
* set this to values that make any sense, using unix peer credentials.
*/
virtual std::optional<AssociatedCredentials> associatedCredentials() const
{
return {};
}
/**
* Perform any necessary effectful operation to make the store up and
* running
+3 -2
View File
@@ -5,9 +5,10 @@
namespace nix {
Path createTempDir(const std::optional<Path> & prefix, mode_t mode)
Path createTempDir(const Path & tmpRoot, const Path & prefix,
bool includePid, bool useGlobalCounter, mode_t mode)
{
return createTempSubdir(defaultTempDir(), prefix, mode);
return createTempSubdir(tmpRoot.empty() ? defaultTempDir() : tmpRoot, prefix, includePid, useGlobalCounter, mode);
}
std::pair<AutoCloseFD, Path> createTempFile(const Path & prefix)
+2 -1
View File
@@ -8,7 +8,8 @@ namespace nix {
/**
* Create a temporary directory.
*/
Path createTempDir(const std::optional<Path> & prefix = "nix", mode_t mode = 0755);
Path createTempDir(const Path & tmpRoot = "", const Path & prefix = "nix",
bool includePid = true, bool useGlobalCounter = true, mode_t mode = 0755);
/**
* Create a temporary file, returning a file handle and its path.
+9 -15
View File
@@ -367,22 +367,16 @@ struct Parser
buffer.clear(); \
u; \
})
#define READ_STRING_LIMITED(limit) \
({ \
size_t len = FETCH_INT(size_t); \
if (len > (limit)) { \
throw SerialisationError( \
"found malformed string tag. input may be a compressed NAR, which cannot be read " \
"directly" \
); \
} \
co_yield WantBytes{len + (8 - len % 8) % 8}; \
StringSource src(std::string_view(buffer.data(), buffer.size())); \
auto str = readString(src, (limit)); \
buffer.clear(); \
std::move(str); \
#define READ_STRING_LIMITED(limit) \
({ \
size_t len = FETCH_INT(size_t); \
co_yield WantBytes{len + (8 - len % 8) % 8}; \
StringSource src(std::string_view(buffer.data(), buffer.size())); \
auto str = readString(src, (limit)); \
buffer.clear(); \
std::move(str); \
})
#define READ_STRING() READ_STRING_LIMITED(1048576)
#define READ_STRING() READ_STRING_LIMITED(std::numeric_limits<size_t>::max())
#define READ_PADDING(size) \
do { \
if ((size) % 8) { \
+67 -87
View File
@@ -3,8 +3,6 @@
#include <filesystem>
#include <atomic>
#include <random>
#include <ranges>
#include <sys/xattr.h>
#include "lix/libutil/environment-variables.hh"
#include "lix/libutil/file-descriptor.hh"
@@ -361,25 +359,11 @@ Generator<Bytes> readFileSource(const Path & path)
}
static AutoCloseFD openForWrite(const Path & path, mode_t mode)
void writeFile(const Path & path, std::string_view s, mode_t mode)
{
AutoCloseFD fd{open(path.c_str(), O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC, mode)};
if (!fd)
throw SysError("opening file '%1%'", path);
return fd;
}
static AutoCloseFD openForWriteExcl(const Path & path, mode_t mode)
{
AutoCloseFD fd{open(path.c_str(), O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC | O_EXCL, mode)};
if (!fd)
throw SysError("opening file '%1%'", path);
return fd;
}
void writeFile(const Path & path, std::string_view s, mode_t mode, bool allowInterrupts)
{
AutoCloseFD fd = openForWrite(path, mode);
writeFile(fd, s, mode);
@@ -387,17 +371,7 @@ void writeFile(const Path & path, std::string_view s, mode_t mode, bool allowInt
fd.close();
}
void writeFileExcl(const Path & path, std::string_view s, mode_t mode, bool allowInterrupts)
{
AutoCloseFD fd = openForWriteExcl(path, mode);
writeFile(fd, s, mode, allowInterrupts);
// Close explicitly to propagate the exceptions.
fd.close();
}
void writeFile(AutoCloseFD & fd, std::string_view s, mode_t mode, bool allowInterrupts)
void writeFile(AutoCloseFD & fd, std::string_view s, mode_t mode)
{
assert(fd);
try {
@@ -411,7 +385,9 @@ void writeFile(AutoCloseFD & fd, std::string_view s, mode_t mode, bool allowInte
void writeFileAndSync(const Path & path, std::string_view s, mode_t mode)
{
{
AutoCloseFD fd = openForWrite(path, mode);
AutoCloseFD fd{open(path.c_str(), O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC, mode)};
if (!fd)
throw SysError("opening file '%1%'", path);
writeFile(fd, s, mode);
fd.fsync();
@@ -422,6 +398,14 @@ void writeFileAndSync(const Path & path, std::string_view s, mode_t mode)
syncParent(path);
}
static AutoCloseFD openForWrite(const Path & path, mode_t mode)
{
AutoCloseFD fd{open(path.c_str(), O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC, mode)};
if (!fd)
throw SysError("opening file '%1%'", path);
return fd;
}
static void closeForWrite(const Path & path, AutoCloseFD & fd, bool sync)
{
if (sync)
@@ -452,27 +436,8 @@ void writeFile(const Path & path, Source & source, mode_t mode)
closeForWrite(path, fd, false);
}
void writeFileExcl(const Path & path, Source & source, mode_t mode)
{
AutoCloseFD fd = openForWriteExcl(path, mode);
std::vector<char> buf(64 * 1024);
try {
while (true) {
try {
auto n = source.read(buf.data(), buf.size());
writeFull(fd.get(), {buf.data(), n});
} catch (EndOfFile &) { break; }
}
} catch (Error & e) {
e.addTrace({}, "writing file '%1%'", path);
throw;
}
closeForWrite(path, fd, false);
}
kj::Promise<Result<void>> writeFile(const Path & path, AsyncInputStream & source, mode_t mode)
kj::Promise<Result<void>>
writeFile(const Path & path, AsyncInputStream & source, mode_t mode)
try {
AutoCloseFD fd = openForWrite(path, mode);
@@ -677,47 +642,51 @@ void AutoDelete::reset(const Path & p, bool recursive) {
//////////////////////////////////////////////////////////////////////
Path createTempSubdir(const Path & parent, const std::optional<Path> & prefix,
mode_t mode)
static Path tempName(PathView parent, const Path & prefix, bool includePid,
std::atomic<unsigned int> & counter)
{
checkInterrupt();
Path tmpDir = makeTempPath(canonPath(parent, true) + "/", prefix);
if (mkdir(tmpDir.c_str(), mode) == 0) {
auto tmpRoot = canonPath(parent, true);
if (includePid)
return fmt("%1%/%2%-%3%-%4%", tmpRoot, prefix, getpid(), counter++);
else
return fmt("%1%/%2%-%3%", tmpRoot, prefix, counter++);
}
Path createTempSubdir(const Path & parent, const Path & prefix,
bool includePid, bool useGlobalCounter, mode_t mode)
{
static std::atomic<unsigned int> globalCounter = 0;
std::atomic<unsigned int> localCounter = 0;
auto & counter(useGlobalCounter ? globalCounter : localCounter);
while (1) {
checkInterrupt();
Path tmpDir = tempName(parent, prefix, includePid, counter);
if (mkdir(tmpDir.c_str(), mode) == 0) {
#if __FreeBSD__
/* Explicitly set the group of the directory. This is to
work around around problems caused by BSD's group
ownership semantics (directories inherit the group of
the parent). For instance, the group of /tmp on
FreeBSD is "wheel", so all directories created in /tmp
will be owned by "wheel"; but if the user is not in
"wheel", then "tar" will fail to unpack archives that
have the setgid bit set on directories. */
if (chown(tmpDir.c_str(), (uid_t) -1, getegid()) != 0) {
throw SysError("setting group of directory '%1%'", tmpDir);
}
/* Explicitly set the group of the directory. This is to
work around around problems caused by BSD's group
ownership semantics (directories inherit the group of
the parent). For instance, the group of /tmp on
FreeBSD is "wheel", so all directories created in /tmp
will be owned by "wheel"; but if the user is not in
"wheel", then "tar" will fail to unpack archives that
have the setgid bit set on directories. */
if (chown(tmpDir.c_str(), (uid_t) -1, getegid()) != 0)
throw SysError("setting group of directory '%1%'", tmpDir);
#endif
return tmpDir;
}
throw SysError("creating directory '%1%'", tmpDir);
}
Path makeTempPath(const Path & root, const std::optional<Path> & prefix)
{
static thread_local std::random_device generator{};
std::uniform_int_distribution<uint64_t> uniform_dist{};
const uint64_t entropy[2] = {uniform_dist(generator), uniform_dist(generator)};
auto unique = base32Encode(std::as_bytes(std::span(entropy)));
if (prefix) {
return fmt("%s%s-%s", root, *prefix, unique);
} else {
return root + unique;
return tmpDir;
}
if (errno != EEXIST)
throw SysError("creating directory '%1%'", tmpDir);
}
}
Path makeTempSiblingPath(const Path & path)
Path makeTempPath(const Path & root, const Path & suffix)
{
return makeTempPath(fs::path(path).remove_filename());
// start the counter at a random value to minimize issues with preexisting temp paths
static std::atomic_uint_fast32_t counter(std::random_device{}());
return fmt("%1%%2%-%3%-%4%", root, suffix, getpid(), counter.fetch_add(1, std::memory_order_relaxed));
}
void createSymlink(const Path & target, const Path & link)
@@ -728,9 +697,20 @@ void createSymlink(const Path & target, const Path & link)
void replaceSymlink(const Path & target, const Path & link)
{
Path tmp = canonPath(makeTempSiblingPath(link));
createSymlink(target, tmp);
renameFile(tmp, link);
for (unsigned int n = 0; true; n++) {
Path tmp = canonPath(fmt("%s/.%d_%s", dirOf(link), n, baseNameOf(link)));
try {
createSymlink(target, tmp);
} catch (SysError & e) {
if (e.errNo == EEXIST) continue;
throw;
}
renameFile(tmp, link);
break;
}
}
void setWriteTime(const fs::path & p, const struct stat & st)
+10 -22
View File
@@ -189,18 +189,11 @@ Generator<Bytes> readFileSource(const Path & path);
/**
* Write a string to a file.
*/
void writeFile(
const Path & path, std::string_view s, mode_t mode = 0666, bool allowInterrupts = true
);
/** Write a string to an exclusively-opened file. */
void writeFileExcl(
const Path & path, std::string_view s, mode_t mode = 0666, bool allowInterrupts = true
);
void writeFileUninterruptible(const Path & path, std::string_view s, mode_t mode = 0666);
void writeFile(const Path & path, Source & source, mode_t mode = 0666);
void writeFileExcl(const Path & path, Source & source, mode_t mode = 0666);
void writeFile(const Path & path, std::string_view s, mode_t mode = 0666);
void writeFile(AutoCloseFD & fd, std::string_view s, mode_t mode = 0666, bool allowInterrupts = true);
void writeFile(const Path & path, Source & source, mode_t mode = 0666);
void writeFile(AutoCloseFD & fd, std::string_view s, mode_t mode = 0666);
kj::Promise<Result<void>>
writeFile(const Path & path, AsyncInputStream & source, mode_t mode = 0666);
@@ -308,21 +301,16 @@ typedef std::unique_ptr<DIR, DIRDeleter> AutoCloseDir;
/**
* Create a temporary directory in a given parent directory.
*/
Path createTempSubdir(const Path & parent, const std::optional<Path> & prefix = "nix",
mode_t mode = 0755);
Path createTempSubdir(const Path & parent, const Path & prefix = "nix",
bool includePid = true, bool useGlobalCounter = true, mode_t mode = 0755);
/**
* Return temporary path constructed by appending to a root path.
* Return temporary path constructed by appending a suffix to a root path.
*
* The constructed path looks like `<root>[<prefix>-]<unique>`. To create a
* path nested in a directory, provide a root ending with `/`.
* The constructed path looks like `<root><suffix>-<pid>-<unique>`. To create a
* path nested in a directory, provide a suffix starting with `/`.
*/
Path makeTempPath(const Path & root, const std::optional<Path> & prefix = ".tmp");
/**
* Return temporary path in the same directory as a given path.
*/
Path makeTempSiblingPath(const Path & path);
Path makeTempPath(const Path & root, const Path & suffix = ".tmp");
/**
* Used in various places.
+48 -3
View File
@@ -80,6 +80,33 @@ static std::string printHash16(const Hash & hash)
}
// omitted: E O U T
const std::string base32Chars = "0123456789abcdfghijklmnpqrsvwxyz";
static std::string printHash32(const Hash & hash)
{
assert(hash.hashSize);
size_t len = hash.base32Len();
assert(len);
std::string s;
s.reserve(len);
for (int n = (int) len - 1; n >= 0; n--) {
unsigned int b = n * 5;
unsigned int i = b / 8;
unsigned int j = b % 8;
unsigned char c =
(hash.hash[i] >> j)
| (i >= hash.hashSize - 1 ? 0 : hash.hash[i + 1] << (8 - j));
s.push_back(base32Chars[c & 0x1f]);
}
return s;
}
std::string printHash16or32(const Hash & hash)
{
return hash.to_string(hash.type == HashType::MD5 ? Base::Base16 : Base::Base32, false);
@@ -98,7 +125,7 @@ std::string Hash::to_string(Base base, bool includeType) const
s += printHash16(*this);
break;
case Base::Base32:
s += base32EncodeStr(std::string_view(charptr_cast<const char *>(hash), hashSize));
s += printHash32(*this);
break;
case Base::Base64:
case Base::SRI:
@@ -198,8 +225,26 @@ Hash::Hash(std::string_view rest, HashType type, bool isSRI)
}
else if (!isSRI && rest.size() == base32Len()) {
auto d = base32Decode(rest);
memcpy(hash, d.data(), hashSize);
for (unsigned int n = 0; n < rest.size(); ++n) {
char c = rest[rest.size() - n - 1];
size_t digit;
for (digit = 0; digit < base32Chars.size(); ++digit) /* !!! slow */
if (base32Chars[digit] == c) break;
if (digit >= 32)
throw BadHash("invalid base-32 hash '%s'", rest);
unsigned int b = n * 5;
unsigned int i = b / 8;
unsigned int j = b % 8;
hash[i] |= digit << j;
if (i < hashSize - 1) {
hash[i + 1] |= digit >> (8 - j);
} else {
if (digit >> (8 - j))
throw BadHash("invalid base-32 hash '%s'", rest);
}
}
}
else if (isSRI || rest.size() == base64Len()) {
+2
View File
@@ -32,6 +32,8 @@ const int sha512HashSize = 64;
extern std::set<std::string> hashTypes;
extern const std::string base32Chars;
enum class Base : int { Base64, Base32, Base16, SRI };
-2
View File
@@ -306,7 +306,6 @@ libutil = library(
openssl,
nlohmann_json,
kj,
libatomic,
],
include_directories : [ '../..' ],
cpp_pch : cpp_pch,
@@ -344,7 +343,6 @@ liblixutil = declare_dependency(
# lix-base pkg-config externally)
kj,
libarchive,
libatomic,
],
link_with : libutil
)
-1
View File
@@ -1,5 +1,4 @@
#include "lix/libutil/references.hh"
#include "lix/libutil/strings.hh"
#include "lix/libutil/hash.hh"
#include "lix/libutil/logging.hh"
+2 -112
View File
@@ -1,7 +1,6 @@
#include "lix/libutil/strings.hh"
#include "lix/libutil/references.hh"
#include <boost/lexical_cast.hpp>
#include <ranges>
#include <stdint.h>
namespace nix {
@@ -129,41 +128,8 @@ std::string shellEscape(const std::string_view s)
std::string r;
r.reserve(s.size() + 2);
r += "'";
for (auto & i : s) {
if (i == '\'') {
// End the single quote, add a single backslash-escaped single quote,
// then start a single quote again.
// i.e., `I didn't know` becomes `'I didn'\''t know'`.
r += "'\\''";
} else {
r += i;
}
}
r += '\'';
return r;
}
std::string bashEscape(const std::string_view s)
{
std::string r;
r.reserve(s.size() + 2);
r += "'";
for (auto & i : s) {
if (!std::isprint(i)) {
// Close the single quote, start an "ANSI-C Quote" ($'foo'), add `\xXX`,
// close the ANSI-C Quote, and finally start a normal single quote again.
r += fmt("'$'\\x%02x''", static_cast<unsigned int>(static_cast<unsigned char>(i)));
} else if (i == '\'') {
// End the single quote, add a single backslash-escaped single quote,
// then start a single quote again.
// i.e., `I didn't know` becomes `'I didn'\''t know'`.
r += "'\\''";
} else {
r += i;
}
}
for (auto & i : s)
if (i == '\'') r += "'\\''"; else r += i;
r += '\'';
return r;
}
@@ -229,82 +195,6 @@ std::string base64Decode(std::string_view s)
return res;
}
// omitted: E O U T
const std::string base32Chars = "0123456789abcdfghijklmnpqrsvwxyz";
std::string base32EncodeStr(std::string_view s)
{
std::span<std::byte const> sp = std::as_bytes(std::span(s));
return base32Encode(sp);
}
std::string base32Encode(std::span<std::byte const> bytes)
{
// log2(32) == 5.
constexpr int B32_BITS_PER_DIGIT = 5;
// We need to do arithmetic.
auto const s = std::views::transform(bytes, std::to_integer<std::uint32_t>);
if (s.empty()) {
return "";
}
ssize_t len = (s.size() * CHAR_BIT - 1) / B32_BITS_PER_DIGIT + 1;
std::string res;
res.reserve(len);
for (ssize_t const n : std::views::iota(0, len) | std::views::reverse) {
unsigned int b = n * B32_BITS_PER_DIGIT;
unsigned int i = b / CHAR_BIT;
unsigned int j = b % CHAR_BIT;
auto const curChar = s[i];
auto const second = i >= s.size() - 1 ? 0 : s[i + 1] << (CHAR_BIT - j);
auto const c = (curChar >> j) | second;
res.push_back(base32Chars[c & 0x1f]);
}
return res;
}
std::string base32Decode(std::string_view s)
{
if (s.empty()) {
return "";
}
std::string res(((s.size() - 1) * 5) / 8 + 1, 0);
for (unsigned int n = 0; n < s.size(); ++n) {
char c = s[s.size() - n - 1];
size_t digit;
for (digit = 0; digit < base32Chars.size(); ++digit) /* !!! slow */ {
if (base32Chars[digit] == c) {
break;
}
}
if (digit >= 32) {
throw Error("invalid character in base-32 string '%s'", s);
}
unsigned int b = n * 5;
unsigned int i = b / 8;
unsigned int j = b % 8;
res[i] |= digit << j;
if (i < res.size() - 1) {
res[i + 1] |= digit >> (8 - j);
} else {
if (digit >> (8 - j)) {
throw Error("invalid base-32 string '%s'", s);
}
}
}
return res;
}
std::string stripIndentation(std::string_view s)
{
-12
View File
@@ -191,24 +191,12 @@ std::string toLower(const std::string & s);
*/
std::string shellEscape(const std::string_view s);
/**
* Same as shellEscape, but also escapes nonprinting characters using $'ANSI C quotes'.
*/
std::string bashEscape(const std::string_view s);
/**
* Base64 encoding/decoding.
*/
std::string base64Encode(std::string_view s);
std::string base64Decode(std::string_view s);
/**
* Base32 encoding/decoding.
*/
extern const std::string base32Chars;
std::string base32EncodeStr(std::string_view s);
std::string base32Encode(std::span<std::byte const> const s);
std::string base32Decode(std::string_view s);
/**
* Remove common leading whitespace from the lines in the string
-10
View File
@@ -1,17 +1,7 @@
# Cursed, but I don't think there's another way to get this environment variable.
lix_suffix = run_command('bash', '-c', 'echo -n "$VERSION_SUFFIX"', check : true).stdout().strip()
lix_version_parts = meson.project_version().split('.')
lix_major = lix_version_parts[0]
lix_minor = lix_version_parts[1]
lix_patch = lix_version_parts[2].replace(lix_suffix, '')
config_h = configure_file(
configuration : {
'PACKAGE_NAME': '"' + meson.project_name() + '"',
'PACKAGE_VERSION': '"' + meson.project_version() + '"',
'LIX_MAJOR': lix_major,
'LIX_MINOR': lix_minor,
'LIX_PATCH': lix_patch,
'PACKAGE_TARNAME': '"' + meson.project_name() + '"',
'PACKAGE_STRING': '"' + meson.project_name() + ' ' + meson.project_version() + '"',
'HAVE_STRUCT_DIRENT_D_TYPE': 1, # FIXME: actually check this for solaris
+1 -7
View File
@@ -364,17 +364,11 @@ static void daemonLoopImpl(std::optional<TrustedFlag> forceTrustClientOpt)
strncpy(savedArgv[1], processName.c_str(), strlen(savedArgv[1]));
}
auto store = aio.blockOn(openUncachedStore());
if (auto local = dynamic_cast<LocalStore *>(&*store); local && peer.uidKnown && peer.gidKnown) {
local->associateWithCredentials(peer.uid, peer.gid);
}
// Handle the connection.
FdSource from(remote.get());
FdSink to(remote.get());
processConnection(
aio, store, from, to, trusted
aio, aio.blockOn(openUncachedStore()), from, to, trusted
);
exit(0);
+12 -32
View File
@@ -6,7 +6,6 @@
#include "lix/libstore/store-api.hh"
#include "lix/libstore/outputs-spec.hh"
#include "lix/libstore/derivations.hh"
#include "lix/libstore/parsed-derivations.hh"
#include "lix/libutil/async.hh"
#include "lix/libutil/json.hh"
#include "run.hh"
@@ -142,20 +141,20 @@ struct BuildEnvironment
for (auto & [name, value] : vars) {
if (!ignoreVars.count(name)) {
if (auto str = std::get_if<String>(&value)) {
out << fmt("%s=%s\n", name, bashEscape(str->value));
out << fmt("%s=%s\n", name, shellEscape(str->value));
if (str->exported)
out << fmt("export %s\n", name);
}
else if (auto arr = std::get_if<Array>(&value)) {
out << "declare -a " << name << "=(";
for (auto & s : *arr)
out << bashEscape(s) << " ";
out << shellEscape(s) << " ";
out << ")\n";
}
else if (auto arr = std::get_if<Associative>(&value)) {
out << "declare -A " << name << "=(";
for (auto & [n, v] : *arr)
out << "[" << bashEscape(n) << "]=" << bashEscape(v) << " ";
out << "[" << shellEscape(n) << "]=" << shellEscape(v) << " ";
out << ")\n";
}
}
@@ -215,8 +214,6 @@ const static std::string getEnvSh =
static kj::Promise<Result<StorePath>> getDerivationEnvironment(ref<Store> store, ref<Store> evalStore, const StorePath & drvPath)
try {
auto drv = TRY_AWAIT(evalStore->derivationFromPath(drvPath));
ParsedDerivation parsedDrv(drvPath, drv);
JSON updatedStructuredAttrs;
auto builder = baseNameOf(drv.builder);
if (builder != "bash")
@@ -233,20 +230,6 @@ try {
drv.env.erase("disallowedRequisites");
drv.env.erase("name");
/* Remove output checks in structured attrs. */
if (auto structuredAttrs = parsedDrv.getStructuredAttrs()) {
drv.env.erase("__json");
updatedStructuredAttrs = *structuredAttrs;
updatedStructuredAttrs.erase("allowedReferences");
updatedStructuredAttrs.erase("allowedRequisites");
updatedStructuredAttrs.erase("disallowedReferences");
updatedStructuredAttrs.erase("disallowedRequisites");
updatedStructuredAttrs.erase("maxSize");
updatedStructuredAttrs.erase("maxClosureSize");
updatedStructuredAttrs.erase("outputChecks");
drv.env.emplace("__json", updatedStructuredAttrs.dump());
}
/* Rehash and write the derivation. FIXME: would be nice to use
'buildDerivation', but that's privileged. */
drv.name += "-env";
@@ -578,7 +561,7 @@ struct CmdDevelop : Common, MixEnvironment
auto [rcFileFd, rcFilePath] = createTempFile("nix-shell");
AutoDelete tmpDir(createTempDir("nix-develop"), true);
AutoDelete tmpDir(createTempDir("", "nix-develop"), true);
auto script = makeRcScript(*state, store, buildEnvironment, (Path) tmpDir);
@@ -599,24 +582,21 @@ struct CmdDevelop : Common, MixEnvironment
else if (!command.empty()) {
std::vector<std::string> args;
for (auto s : command)
args.push_back(bashEscape(s));
args.push_back(shellEscape(s));
script += fmt("exec %s\n", concatStringsSep(" ", args));
}
else {
script = "[ -n \"$PS1\" ] && [ -e ~/.bashrc ] && source ~/.bashrc;\n" + script;
if (developSettings.bashPrompt != "")
script +=
fmt("[ -n \"$PS1\" ] && PS1=%s;\n",
bashEscape(developSettings.bashPrompt.get()));
script += fmt("[ -n \"$PS1\" ] && PS1=%s;\n",
shellEscape(developSettings.bashPrompt.get()));
if (developSettings.bashPromptPrefix != "")
script +=
fmt("[ -n \"$PS1\" ] && PS1=%s\"$PS1\";\n",
bashEscape(developSettings.bashPromptPrefix.get()));
script += fmt("[ -n \"$PS1\" ] && PS1=%s\"$PS1\";\n",
shellEscape(developSettings.bashPromptPrefix.get()));
if (developSettings.bashPromptSuffix != "")
script +=
fmt("[ -n \"$PS1\" ] && PS1+=%s;\n",
bashEscape(developSettings.bashPromptSuffix.get()));
script += fmt("[ -n \"$PS1\" ] && PS1+=%s;\n",
shellEscape(developSettings.bashPromptSuffix.get()));
}
writeFull(rcFileFd.get(), script);
@@ -718,7 +698,7 @@ struct CmdPrintDevEnv : Common, MixJSON
if (json) {
logger->writeToStdout(buildEnvironment.toJSON());
} else {
AutoDelete tmpDir(createTempDir("nix-dev-env"), true);
AutoDelete tmpDir(createTempDir("", "nix-dev-env"), true);
logger->writeToStdout(makeRcScript(*state, store, buildEnvironment, tmpDir));
}
}
-26
View File
@@ -12,31 +12,6 @@ fi
__vars="$(declare -p)"
__functions="$(declare -F)"
# Literal control characters (ASCII 0-31) aren't valid JSON.
__escapeCtrl() {
local escaped="$1"
# I don't know if NUL bytes are at ALL possible in here,
# but covering them is free.
local i=0
# NOTE: safe input `i` to arithmetic expansion.
while [[ "$i" -le 32 ]]; do
# Convert the decimal ASCII value to its actual string.
local asHex; printf -v asHex "%02x" "$i"
local asStr; printf -v asStr "%b" "\x$asHex"
# Format it to \uXXXX.
# All control characters fit within four hex digits.
local asUni; printf -v asUni '\\u%04x' "$i"
escaped="${escaped//"$asStr"/"$asUni"}"
i="$((i + 1))"
done
printf "%s" "$escaped"
}
__dumpEnv() {
printf '{\n'
@@ -150,7 +125,6 @@ __escapeString() {
__s="${__s//$'\n'/\\n}"
__s="${__s//$'\r'/\\r}"
__s="${__s//$'\t'/\\t}"
__s="$(__escapeCtrl "$__s")"
printf '"%s"' "$__s"
}
-1
View File
@@ -177,7 +177,6 @@ nix = executable(
boehm,
nlohmann_json,
kj,
libatomic,
],
cpp_pch : cpp_pch,
install : true,
-33
View File
@@ -340,16 +340,6 @@ editline = dependency('libeditline', 'editline', version : '>=1.14', required :
lowdown = dependency('lowdown', version : '>=0.9.0', required : true, include_type : 'system')
# TODO(sterni): drop the corresponding #ifdef after NixOS 25.05 is EOL which still distributes lowdown < 1.4.0
if lowdown.version().version_compare('>= 1.4.0')
add_project_arguments('-DLOWDOWN_SEPARATE_TERM_OPTS', language: 'cpp')
endif
# TODO(sterni): drop the corresponding #ifdef after NixOS 25.11 is EOL which still distributes lowdown < 3.0.0
if lowdown.version().version_compare('>= 3.0.0')
add_project_arguments('-DLOWDOWN_CONSOLIDATED_OFLAGS', language: 'cpp')
endif
# HACK(Qyriad): rapidcheck's pkg-config doesn't include the libs lol
# Note: technically we 'check' for rapidcheck twice, for the internal-api-docs handling above,
# but Meson will cache the result of the first one, and the required : arguments are different.
@@ -364,9 +354,6 @@ gtest = [
]
toml11 = dependency('toml11', version : '>=3.7.0', required : true, method : 'cmake', include_type : 'system')
configdata += {
'HAVE_TOML11_4': toml11.version().version_compare('>= 4.0.0').to_int(),
}
pegtl = dependency(
'pegtl',
@@ -383,26 +370,6 @@ if is_freebsd
libprocstat = declare_dependency(link_args : [ '-lprocstat' ])
endif
libatomic_test_program = '''
#include <atomic>
int main() {
std::atomic<uint8_t> w1;
std::atomic<uint16_t> w2;
std::atomic<uint32_t> w4;
std::atomic<uint64_t> w8;
return ++w1 + ++w2 + ++w4 + ++w8;
}
'''
libatomic = cxx.find_library('atomic', required : false)
# Some platforms like 32-Bit PowerPC need libatomic because they're lacking 64-Bit hardware atomic instructions
# and compilers don't handle this automatically (yet).
# See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81358 and https://clang.llvm.org/docs/Toolchain.html#atomics-library
if not cxx.links(libatomic_test_program, name : 'test if simple atomic program links')
libatomic = cxx.find_library('atomic', required : true)
endif
#
# Build-time tools
#
+1 -2
View File
@@ -46,8 +46,7 @@ stdenv.mkDerivation rec {
propagatedBuildInputs = [
openssl
zlib
]
++ lib.optional (stdenv.cc.isClang && stdenv.targetPlatform.isStatic) empty-libgcc_eh;
] ++ lib.optional (stdenv.cc.isClang && stdenv.targetPlatform.isStatic) empty-libgcc_eh;
# FIXME: separate the binaries from the stuff that user systems actually use
# This runs into a terrible UX issue in Lix and I just don't want to debug it
+1 -3
View File
@@ -1,4 +1,2 @@
d @localstatedir@/nix/daemon-socket 0755 root root - -
d @localstatedir@/nix/b 0755 root root 7d -
# TODO: Remove this after (at least) August 2026.
R! @localstatedir@/nix/builds - - - - -
d @localstatedir@/nix/builds 0755 root root 7d -
+85 -86
View File
@@ -135,10 +135,10 @@ let
# This could be the dtrace for macOS, etc, but I have no idea if it is
# packaged or if it works.
dtrace-generator = if withDtrace then systemtap-lix else null;
dtrace-generator = lib.optional withDtrace systemtap-lix;
# This is for sys/sdt.h
dtrace-headers = if withDtrace then libsystemtap else null;
dtrace-headers = lib.optional withDtrace libsystemtap;
aws-sdk-cpp-nix =
if aws-sdk-cpp == null then
@@ -209,8 +209,6 @@ assert (lintInsteadOfBuild -> lix-clang-tidy != null);
stdenv.mkDerivation (finalAttrs: {
inherit pname version;
__structuredAttrs = true;
src = fileset.toSource {
root = ./.;
fileset = fileset.intersection baseFiles (
@@ -231,13 +229,14 @@ stdenv.mkDerivation (finalAttrs: {
);
};
outputs = [
"out"
]
++ lib.optionals (!finalAttrs.dontBuild) [
"dev"
"doc"
];
VERSION_SUFFIX = versionSuffix;
outputs =
[ "out" ]
++ lib.optionals (!finalAttrs.dontBuild) [
"dev"
"doc"
];
dontBuild = lintInsteadOfBuild;
@@ -276,79 +275,81 @@ stdenv.mkDerivation (finalAttrs: {
# We only include CMake so that Meson can locate toml11, which only ships CMake dependency metadata.
dontUseCmakeConfigure = true;
nativeBuildInputs = [
lixPythonForBuild
meson
ninja
cmake
rustc
capnproto-lix
# Required for libstd++ assertions that leaks inside of the final binary.
removeReferencesTo
dtrace-generator
]
++ [
(lib.getBin lowdown-unsandboxed)
mdbook
mdbook-linkcheck
]
++ [
pkg-config
nativeBuildInputs =
[
lixPythonForBuild
meson
ninja
cmake
rustc
capnproto-lix
# Required for libstd++ assertions that leaks inside of the final binary.
removeReferencesTo
dtrace-generator
]
++ [
(lib.getBin lowdown-unsandboxed)
mdbook
mdbook-linkcheck
]
++ [
pkg-config
# Tests
git
mercurial
jq
yq
lsof
]
++ lib.optional hostPlatform.isLinux util-linuxMinimal
++ lib.optional (!officialRelease && buildUnreleasedNotes) build-release-notes
++ lib.optional internalApiDocs doxygen
++ lib.optionals lintInsteadOfBuild [
# required for a wrapped clang-tidy
llvmPackages.clang-tools
# load-bearing order (just as below); the actual stdenv wrapped clang
# needs to precede the unwrapped clang in PATH such that calling `clang`
# can compile things.
stdenv.cc
# required for run-clang-tidy
llvmPackages.clang-unwrapped
];
# Tests
git
mercurial
jq
yq
lsof
]
++ lib.optional hostPlatform.isLinux util-linuxMinimal
++ lib.optional (!officialRelease && buildUnreleasedNotes) build-release-notes
++ lib.optional internalApiDocs doxygen
++ lib.optionals lintInsteadOfBuild [
# required for a wrapped clang-tidy
llvmPackages.clang-tools
# load-bearing order (just as below); the actual stdenv wrapped clang
# needs to precede the unwrapped clang in PATH such that calling `clang`
# can compile things.
stdenv.cc
# required for run-clang-tidy
llvmPackages.clang-unwrapped
];
buildInputs = [
curl
bzip2
xz
brotli
editline-lix
openssl
sqlite
libarchive
boost
lowdown
libsodium
toml11
pegtl
capnproto-lix
dtrace-headers
]
++ lib.optionals hostPlatform.isLinux [
libseccomp
busybox-sandbox-shell
passt-lix
]
++ lib.optionals (
stdenv.hostPlatform.isDarwin && lib.versionOlder stdenv.hostPlatform.darwinSdkVersion "11.0"
) [ apple-sdk_11 ]
++ lib.optional internalApiDocs rapidcheck
++ lib.optional hostPlatform.isx86_64 libcpuid
# There have been issues building these dependencies
++ lib.optional (hostPlatform.canExecute buildPlatform) aws-sdk-cpp-nix
++ lib.optionals (finalAttrs.dontBuild) maybePropagatedInputs
# I am so sorry. This is because checkInputs are required to pass
# configure, but we don't actually want to *run* the checks here.
++ lib.optionals lintInsteadOfBuild finalAttrs.checkInputs;
buildInputs =
[
curl
bzip2
xz
brotli
editline-lix
openssl
sqlite
libarchive
boost
lowdown
libsodium
toml11
pegtl
capnproto-lix
dtrace-headers
]
++ lib.optionals hostPlatform.isLinux [
libseccomp
busybox-sandbox-shell
passt-lix
]
++ lib.optionals (
stdenv.hostPlatform.isDarwin && lib.versionOlder stdenv.hostPlatform.darwinSdkVersion "11.0"
) [ apple-sdk_11 ]
++ lib.optional internalApiDocs rapidcheck
++ lib.optional hostPlatform.isx86_64 libcpuid
# There have been issues building these dependencies
++ lib.optional (hostPlatform.canExecute buildPlatform) aws-sdk-cpp-nix
++ lib.optionals (finalAttrs.dontBuild) maybePropagatedInputs
# I am so sorry. This is because checkInputs are required to pass
# configure, but we don't actually want to *run* the checks here.
++ lib.optionals lintInsteadOfBuild finalAttrs.checkInputs;
checkInputs = [
gtest
@@ -363,9 +364,9 @@ stdenv.mkDerivation (finalAttrs: {
lixPythonForBuild
];
# Needed for Meson to find Boost.
# https://github.com/NixOS/nixpkgs/issues/86131.
env = {
# Needed for Meson to find Boost.
# https://github.com/NixOS/nixpkgs/issues/86131.
BOOST_INCLUDEDIR = "${lib.getDev boost}/include";
BOOST_LIBRARYDIR = "${lib.getLib boost}/lib";
@@ -373,8 +374,6 @@ stdenv.mkDerivation (finalAttrs: {
# Turns out the Nix-generated Cargo dependencies are named the same as they
# would be in a Cargo registry cache.
MESON_PACKAGE_CACHE_DIR = finalAttrs.cargoDeps;
VERSION_SUFFIX = versionSuffix;
};
cargoDeps = rustPlatform.importCargoLock { lockFile = ./Cargo.lock; };
+15 -14
View File
@@ -35,20 +35,21 @@ perl.pkgs.toPerlModule (
ninja
];
buildInputs = [
nix
curl
bzip2
xz
perl
boost
perlPackages.DBI
perlPackages.DBDSQLite
# for kj-async
nix.passthru.capnproto-lix
]
++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium
++ lib.optional stdenv.isDarwin darwin.apple_sdk.frameworks.Security;
buildInputs =
[
nix
curl
bzip2
xz
perl
boost
perlPackages.DBI
perlPackages.DBDSQLite
# for kj-async
nix.passthru.capnproto-lix
]
++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium
++ lib.optional stdenv.isDarwin darwin.apple_sdk.frameworks.Security;
# Nixpkgs' Meson hook likes to set this to "plain".
mesonBuildType = "debugoptimized";
+1 -1
View File
@@ -40,7 +40,7 @@ def setup_creds(env: RelengEnvironment):
key = keys.get_ephemeral_key(env)
$AWS_SECRET_ACCESS_KEY = key.secret_key
$AWS_ACCESS_KEY_ID = key.id
$AWS_DEFAULT_REGION = env.s3_region
$AWS_DEFAULT_REGION = 'garage'
$AWS_ENDPOINT_URL = env.s3_endpoint
+11 -20
View File
@@ -5,12 +5,11 @@ import functools
import subprocess
import dataclasses
S3_HOST = 's3-admin.afnix.fr'
S3_USER = 'lix-releng'
S3_HOST = 's3.lix.systems'
DEFAULT_STORE_URI_BITS = {
'region': 'global',
'endpoint': 's3.afnix.fr',
'region': 'garage',
'endpoint': 's3.lix.systems',
'want-mass-query': 'true',
'write-nar-listing': 'true',
'ls-compression': 'zstd',
@@ -55,9 +54,7 @@ class RelengEnvironment:
git_repo: Callable[[], str]
git_repo_is_gerrit: bool
s3_endpoint: str
s3_region: str
s3_ssh_host: str | None
s3_ssh_user: str | None
docker_targets: list[DockerTarget]
@@ -89,19 +86,17 @@ LOCAL = RelengEnvironment(
git_repo_is_gerrit=False,
docker_targets=[],
s3_endpoint = 'http://localhost:3900',
s3_region = 'garage',
s3_ssh_host = None,
s3_ssh_user = None,
)
STAGING = RelengEnvironment(
name='staging',
colour=functools.partial(sgr, GREEN),
docs_bucket='s3://docs.staging.lix.systems',
cache_bucket='s3://cache.staging.lix.systems',
docs_bucket='s3://staging-docs',
cache_bucket='s3://staging-cache',
cache_store_overlay={'secret-key': 'staging.key'},
releases_bucket='s3://releases.staging.lix.systems',
releases_bucket='s3://staging-releases',
git_repo=lambda: 'ssh://git@git.lix.systems/lix-project/lix-releng-staging',
git_repo_is_gerrit=False,
docker_targets=[
@@ -111,10 +106,8 @@ STAGING = RelengEnvironment(
DockerTarget('ghcr.io/lix-project/lix-releng-staging',
tags=['{version}', '{major}']),
],
s3_endpoint = 'https://s3.afnix.fr',
s3_region = 'garage',
s3_endpoint = 'https://s3.lix.systems',
s3_ssh_host = S3_HOST,
s3_ssh_user = S3_USER,
)
GERRIT_REMOTE_RE = re.compile(r'^ssh://(\w+@)?gerrit.lix.systems:2022/lix$')
@@ -134,13 +127,13 @@ def guess_gerrit_remote():
PROD = RelengEnvironment(
name='production',
colour=functools.partial(sgr, RED),
docs_bucket='s3://docs.lix.systems',
cache_bucket='s3://cache.lix.systems',
docs_bucket='s3://docs',
cache_bucket='s3://cache',
# FIXME: we should decrypt this with age into a tempdir in the future, but
# the issue is how to deal with the recipients file. For now, we should
# just delete it after doing a release.
cache_store_overlay={'secret-key': 'prod.key'},
releases_bucket='s3://releases.lix.systems',
releases_bucket='s3://releases',
git_repo=guess_gerrit_remote,
git_repo_is_gerrit=True,
docker_targets=[
@@ -149,10 +142,8 @@ PROD = RelengEnvironment(
tags=['{version}', '{major}']),
DockerTarget('ghcr.io/lix-project/lix', tags=['{version}', '{major}']),
],
s3_endpoint = 'https://s3.afnix.fr',
s3_region = 'global',
s3_endpoint = 'https://s3.lix.systems',
s3_ssh_host = S3_HOST,
s3_ssh_user = S3_USER,
)
ENVIRONMENTS = {
@@ -1,12 +1,12 @@
# SPDX-FileCopyrightText: 2024 Jade Lovelace
# SPDX-FileCopyrightText: 2026 Yureka Lilian <yureka@cyberchaos.dev>
# SPDX-License-Identifier: MIT
import argparse
import json
import sys
import datetime
import dataclasses
import re
from typing import Any
from typing import Any, Literal, Optional
import requests
import os
import logging
@@ -14,34 +14,27 @@ import logging
log = logging.getLogger(__name__)
log.setLevel(logging.INFO)
fmt = logging.Formatter(
"{asctime} {levelname} {name}: {message}",
datefmt="%b %d %H:%M:%S",
style="{",
)
fmt = logging.Formatter('{asctime} {levelname} {name}: {message}',
datefmt='%b %d %H:%M:%S',
style='{')
if not any(isinstance(h, logging.StreamHandler) for h in log.handlers):
hand = logging.StreamHandler()
hand.setFormatter(fmt)
log.addHandler(hand)
API_BASE = os.environ.get("GARAGE_ADMIN_API_BASE", "http://localhost:3903")
API_KEY = os.environ["GARAGE_ADMIN_TOKEN"]
BUCKET_REGEX_STR = os.environ.get("BUCKET_REGEX", ".*")
BUCKET_REGEX = re.compile(BUCKET_REGEX_STR)
API_BASE = os.environ.get('GARAGE_ADMIN_API_BASE', 'http://localhost:3903')
API_KEY = os.environ['GARAGE_ADMIN_TOKEN']
def api(method, endpoint: str, resp_json=True, **kwargs) -> Any:
log.info("http %s %s", method, endpoint)
if not endpoint.startswith("https"):
log.info('http %s %s', method, endpoint)
if not endpoint.startswith('https'):
endpoint = API_BASE + endpoint
resp = requests.request(
method,
endpoint,
headers={"Authorization": f"Bearer {API_KEY}"},
**kwargs,
)
resp = requests.request(method,
endpoint,
headers={'Authorization': f'Bearer {API_KEY}'},
**kwargs)
resp.raise_for_status()
if resp_json:
return resp.json()
@@ -49,64 +42,97 @@ def api(method, endpoint: str, resp_json=True, **kwargs) -> Any:
return resp
def get_bucket_id(bucket_name: str) -> str:
resp: dict = api(
"GET", "/v2/GetBucketInfo", params={"globalAlias": bucket_name}
)
return resp["id"]
@dataclasses.dataclass
class Key:
name: str
id: str
secret_key: Optional[str] = None
DATEFMT = "%Y%m%d%H%M%S"
@dataclasses.dataclass
class Bucket:
id: str
def keys() -> list[Key]:
data: list[dict] = api('GET', '/v1/key?list')
return [Key(name=k['name'], id=k['id']) for k in data]
def delete_key(key: Key):
api('DELETE', '/v1/key', resp_json=False, params={'id': key.id})
def create_key(name: str) -> Key:
resp: dict = api('POST', '/v1/key', json={'name': name})
return Key(name=resp['name'],
id=resp['accessKeyId'],
secret_key=resp['secretAccessKey'])
AccessType = Literal['read'] | Literal['write'] | Literal['owner']
def get_bucket(bucket_name: str) -> Bucket:
resp: dict = api('GET', '/v1/bucket', params={'globalAlias': bucket_name})
return Bucket(resp['id'])
def grant(bucket: Bucket, access_types: list[AccessType], key: Key):
access_types_dict = {k: True for k in access_types}
api('POST',
'/v1/bucket/allow',
json={
'bucketId': bucket.id,
'accessKeyId': key.id,
'permissions': access_types_dict,
})
KEY_RE = re.compile(r'^.*ephemeral-(\d{14})$')
DATEFMT = '%Y%m%d%H%M%S'
def expired_keys(older_than: datetime.datetime) -> list[Key]:
ret = []
for key in keys():
if m := KEY_RE.match(key.name):
date = datetime.datetime.strptime(m.group(1), DATEFMT)
date = date.astimezone(datetime.UTC)
print(date)
if date < older_than:
ret.append(key)
return ret
def do_new(args):
for b in args.buckets:
if not BUCKET_REGEX.match(b):
print(f"Bucket {b} not in allowed buckeds '{BUCKET_REGEX_STR}'")
exit(1)
bucket_ids = [get_bucket_id(b) for b in args.buckets]
buckets = [get_bucket(b) for b in args.buckets]
key_name = args.name + "-" if args.name else ""
expiration = datetime.datetime.now(tz=datetime.UTC) + datetime.timedelta(
seconds=args.age_secs
)
key_name += "ephemeral-" + expiration.strftime(DATEFMT)
def optional(s: str, whether) -> list[str]:
if whether:
return [s]
else:
return []
key_resp: dict = api(
"POST",
"/v2/CreateKey",
json={
"name": key_name,
"expiration": expiration.isoformat(),
"neverExpires": False,
},
)
access_types: list[AccessType] = optional('read', args.read) + optional(
'write', args.write) + optional('owner', args.owner) # type: ignore
for b in bucket_ids:
api(
"POST",
"/v2/AllowBucketKey",
json={
"accessKeyId": key_resp["accessKeyId"],
"bucketId": b,
"permissions": {
"read": args.read,
"write": args.write,
"owner": args.owner,
},
},
)
key_name = args.name + '-' if args.name else ''
key_name += "ephemeral-" + (
datetime.datetime.now(tz=datetime.UTC) +
datetime.timedelta(seconds=args.age_secs)).strftime(DATEFMT)
print(
json.dumps(
{
"name": key_resp["name"],
"id": key_resp["accessKeyId"],
"secret_key": key_resp["secretAccessKey"],
},
indent=2,
)
)
k = create_key(key_name)
for b in buckets:
grant(b, access_types, k)
print(json.dumps(dataclasses.asdict(k), indent=2))
def do_clean(args):
older_than = datetime.datetime.now(tz=datetime.UTC)
for key in expired_keys(older_than):
delete_key(key)
def main():
@@ -122,27 +148,28 @@ def main():
new = sps.add_parser("new", help="Make an ephemeral key")
new.add_argument("--name", help="Name prefix for the key")
new.add_argument(
"--read", action="store_true", help="Grant read access to buckets"
)
new.add_argument(
"--write", action="store_true", help="Grant write access to buckets"
)
new.add_argument(
"--owner", action="store_true", help="Grant owner access to buckets"
)
new.add_argument(
"--age-secs",
type=int,
required=True,
help="Maximum key lifetime in seconds",
)
new.add_argument("buckets", nargs="*", help="Buckets to grant access to")
new.add_argument("--read",
action="store_true",
help="Grant read access to buckets")
new.add_argument("--write",
action="store_true",
help="Grant write access to buckets")
new.add_argument("--owner",
action="store_true",
help="Grant owner access to buckets")
new.add_argument("--age-secs",
type=int,
required=True,
help="Maximum key lifetime in seconds")
new.add_argument("buckets", nargs='*', help="Buckets to grant access to")
new.set_defaults(cmd=do_new)
clean = sps.add_parser("clean", help="Clean up old keys")
clean.set_defaults(cmd=do_clean)
args = ap.parse_args()
args.cmd(args)
if __name__ == "__main__":
if __name__ == '__main__':
main()
+1 -1
View File
@@ -14,7 +14,7 @@ def get_ephemeral_key(
env.docs_bucket.removeprefix('s3://'),
]
if env.s3_ssh_host is not None:
command = ['ssh', f'{env.s3_ssh_user}@{env.s3_ssh_host}', *command]
command = ['ssh', '-l', 'root', env.s3_ssh_host, *command]
output = subprocess.check_output(command)
d = json.loads(output.decode())
return environment.S3Credentials(name=d['name'],
+1 -1
View File
@@ -80,4 +80,4 @@ out="$(nix-build 2>&1 failing.nix \
[[ "$out" =~ .*"note: keeping build directory".* ]]
build_dir="$(grep "note: keeping build" <<< "$out" | sed -E "s/^(.*)note: keeping build directory '(.*)'(.*)$/\2/")"
[[ "foo" = $(<"$build_dir"/b/bar) ]]
[[ "foo" = $(<"$build_dir"/bar) ]]
-25
View File
@@ -174,28 +174,3 @@ test "$(<<<"$out" grep -E '^error:' | wc -l)" = 3
<<<"$out" grepQuiet -E "error: 2 dependencies of derivation '.*-x4\\.drv' failed to build"
<<<"$out" grepQuiet -vE "hash mismatch in fixed-output derivation '.*-x3\\.drv'"
<<<"$out" grepQuiet -vE "hash mismatch in fixed-output derivation '.*-x2\\.drv'"
# Ensure when if the system build dir is inaccessible, we can still build things
BUILD_DIR=$(mktemp -d)
chmod 0000 "$BUILD_DIR"
nix --build-dir "$BUILD_DIR" build -E 'with import ./config.nix; mkDerivation { name = "test"; buildCommand = "echo rawr > $out"; }' --impure --no-link
# ensure that the build directory parent is not world-accessible
chmod 0755 "$BUILD_DIR"
FIFO="$BUILD_DIR/fifo"
mkfifo "$FIFO"
(
echo > "$FIFO"
trap 'echo > "$FIFO"' EXIT
mode=$(stat -c %a $BUILD_DIR/b/*)
[ "$mode" = "700" -o "$mode" = "710" ]
) &
nix build --build-dir "$BUILD_DIR/b" -E '
with import ./config.nix; mkDerivation {
name = "test";
buildCommand = "cat '"$FIFO"'; cat '"$FIFO"' > $out";
}' \
--extra-sandbox-paths "$FIFO" \
--impure \
--no-link
wait
+6 -4
View File
@@ -37,13 +37,15 @@ if checkBuildTempDirRemoved $TEST_ROOT/log; then false; fi
test_custom_build_dir() {
local customBuildDir="$TEST_ROOT/custom-build-dir"
# Nix does not create the parent directories, and perhaps it shouldn't try to
# decide the permissions of build-dir.
mkdir "$customBuildDir"
nix-build check.nix -A failed --argstr checkBuildId $checkBuildId \
--no-out-link --keep-failed --option build-dir "$TEST_ROOT/custom-build-dir" 2> $TEST_ROOT/log || status=$?
[ "$status" = "100" ]
[[ 1 == "$(count "$customBuildDir/"*)" ]]
local buildDir="$customBuildDir/"*
grep $checkBuildId $buildDir/checkBuildId \
|| grep $checkBuildId $buildDir/b/checkBuildId
[[ 1 == "$(count "$customBuildDir/nix-build-"*)" ]]
local buildDir="$customBuildDir/nix-build-"*
grep $checkBuildId $buildDir/checkBuildId
}
test_custom_build_dir
@@ -55,53 +55,11 @@ builtins.fromTOML ''
odt2 = 1979-05-27T00:32:00-07:00
odt3 = 1979-05-27T00:32:00.999999-07:00
odt4 = 1979-05-27 07:32:00Z
# milliseconds
odt5 = 1979-05-27 07:32:00.1Z
odt6 = 1979-05-27 07:32:00.12Z
odt7 = 1979-05-27 07:32:00.123Z
# microseconds
odt8 = 1979-05-27t07:32:00.1234Z
odt9 = 1979-05-27t07:32:00.12345Z
odt10 = 1979-05-27t07:32:00.123456Z
# nanoseconds
odt11 = 1979-05-27 07:32:00.1234567Z
odt12 = 1979-05-27 07:32:00.12345678Z
odt13 = 1979-05-27 07:32:00.123456789Z
# no more precision after nanoseconds
odt14 = 1979-05-27t07:32:00.1234567891Z
ldt1 = 1979-05-27T07:32:00
# milliseconds
ldt2 = 1979-05-27T07:32:00.1
ldt3 = 1979-05-27T07:32:00.12
ldt4 = 1979-05-27T07:32:00.123
# microseconds
ldt5 = 1979-05-27t00:32:00.1234
ldt6 = 1979-05-27t00:32:00.12345
ldt7 = 1979-05-27t00:32:00.123456
# nanoseconds
ldt8 = 1979-05-27 00:32:00.1234567
ldt9 = 1979-05-27 00:32:00.12345678
ldt10 = 1979-05-27 00:32:00.123456789
# no more precision after nanoseconds
ldt11 = 1979-05-27t00:32:00.1234567891
ldt2 = 1979-05-27T00:32:00.999999
ld1 = 1979-05-27
lt1 = 07:32:00
# milliseconds
lt2 = 00:32:00.1
lt3 = 00:32:00.12
lt4 = 00:32:00.123
# microseconds
lt5 = 00:32:00.1234
lt6 = 00:32:00.12345
lt7 = 00:32:00.123456
# nanoseconds
lt8 = 00:32:00.1234567
lt9 = 00:32:00.12345678
lt10 = 00:32:00.123456789
# no more precision after nanoseconds
lt11 = 00:32:00.1234567891
lt2 = 00:32:00.999999
arr1 = [ 1, 2, 3 ]
arr2 = [ "red", "yellow", "green" ]
@@ -1 +1 @@
{ "1234" = "value"; "127.0.0.1" = "value"; a = { b = { c = { }; }; }; arr1 = [ 1 2 3 ]; arr2 = [ "red" "yellow" "green" ]; arr3 = [ [ 1 2 ] [ 3 4 5 ] ]; arr4 = [ "all" "strings" "are the same" "type" ]; arr5 = [ [ 1 2 ] [ "a" "b" "c" ] ]; arr7 = [ 1 2 3 ]; arr8 = [ 1 2 ]; bare-key = "value"; bare_key = "value"; bin1 = 214; bool1 = true; bool2 = false; "character encoding" = "value"; d = { e = { f = { }; }; }; dog = { "tater.man" = { type = { name = "pug"; }; }; }; flt1 = 1; flt2 = 3.1415; flt3 = -0.01; flt4 = 5e+22; flt5 = 1e+06; flt6 = -0.02; flt7 = 6.626e-34; flt8 = 9.22462e+06; fruit = [ { name = "apple"; physical = { color = "red"; shape = "round"; }; variety = [ { name = "red delicious"; } { name = "granny smith"; } ]; } { name = "banana"; variety = [ { name = "plantain"; } ]; } ]; g = { h = { i = { }; }; }; hex1 = 3735928559; hex2 = 3735928559; hex3 = 3735928559; int1 = 99; int2 = 42; int3 = 0; int4 = -17; int5 = 1000; int6 = 5349221; int7 = 12345; j = { "ʞ" = { l = { }; }; }; key = "value"; key2 = "value"; ld1 = { _type = "timestamp"; value = "1979-05-27"; }; ldt1 = { _type = "timestamp"; value = "1979-05-27T07:32:00"; }; ldt10 = { _type = "timestamp"; value = "1979-05-27T00:32:00.123456789"; }; ldt11 = { _type = "timestamp"; value = "1979-05-27T00:32:00.123456789"; }; ldt2 = { _type = "timestamp"; value = "1979-05-27T07:32:00.100"; }; ldt3 = { _type = "timestamp"; value = "1979-05-27T07:32:00.120"; }; ldt4 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123"; }; ldt5 = { _type = "timestamp"; value = "1979-05-27T00:32:00.123400"; }; ldt6 = { _type = "timestamp"; value = "1979-05-27T00:32:00.123450"; }; ldt7 = { _type = "timestamp"; value = "1979-05-27T00:32:00.123456"; }; ldt8 = { _type = "timestamp"; value = "1979-05-27T00:32:00.123456700"; }; ldt9 = { _type = "timestamp"; value = "1979-05-27T00:32:00.123456780"; }; lt1 = { _type = "timestamp"; value = "07:32:00"; }; lt10 = { _type = "timestamp"; value = "00:32:00.123456789"; }; lt11 = { _type = "timestamp"; value = "00:32:00.123456789"; }; lt2 = { _type = "timestamp"; value = "00:32:00.100"; }; lt3 = { _type = "timestamp"; value = "00:32:00.120"; }; lt4 = { _type = "timestamp"; value = "00:32:00.123"; }; lt5 = { _type = "timestamp"; value = "00:32:00.123400"; }; lt6 = { _type = "timestamp"; value = "00:32:00.123450"; }; lt7 = { _type = "timestamp"; value = "00:32:00.123456"; }; lt8 = { _type = "timestamp"; value = "00:32:00.123456700"; }; lt9 = { _type = "timestamp"; value = "00:32:00.123456780"; }; name = "Orange"; oct1 = 342391; oct2 = 493; odt1 = { _type = "timestamp"; value = "1979-05-27T07:32:00Z"; }; odt10 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123456Z"; }; odt11 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123456700Z"; }; odt12 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123456780Z"; }; odt13 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123456789Z"; }; odt14 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123456789Z"; }; odt2 = { _type = "timestamp"; value = "1979-05-27T00:32:00-07:00"; }; odt3 = { _type = "timestamp"; value = "1979-05-27T00:32:00.999999-07:00"; }; odt4 = { _type = "timestamp"; value = "1979-05-27T07:32:00Z"; }; odt5 = { _type = "timestamp"; value = "1979-05-27T07:32:00.100Z"; }; odt6 = { _type = "timestamp"; value = "1979-05-27T07:32:00.120Z"; }; odt7 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123Z"; }; odt8 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123400Z"; }; odt9 = { _type = "timestamp"; value = "1979-05-27T07:32:00.123450Z"; }; physical = { color = "orange"; shape = "round"; }; products = [ { name = "Hammer"; sku = 738594937; } { } { color = "gray"; name = "Nail"; sku = 284758393; } ]; "quoted \"value\"" = "value"; site = { "google.com" = true; }; str = "I'm a string. \"You can quote me\". Name\tJosé\nLocation\tSF."; table-1 = { key1 = "some string"; key2 = 123; }; table-2 = { key1 = "another string"; key2 = 456; }; x = { y = { z = { w = { animal = { type = { name = "pug"; }; }; name = { first = "Tom"; last = "Preston-Werner"; }; point = { x = 1; y = 2; }; }; }; }; }; "ʎǝʞ" = "value"; }
{ "1234" = "value"; "127.0.0.1" = "value"; a = { b = { c = { }; }; }; arr1 = [ 1 2 3 ]; arr2 = [ "red" "yellow" "green" ]; arr3 = [ [ 1 2 ] [ 3 4 5 ] ]; arr4 = [ "all" "strings" "are the same" "type" ]; arr5 = [ [ 1 2 ] [ "a" "b" "c" ] ]; arr7 = [ 1 2 3 ]; arr8 = [ 1 2 ]; bare-key = "value"; bare_key = "value"; bin1 = 214; bool1 = true; bool2 = false; "character encoding" = "value"; d = { e = { f = { }; }; }; dog = { "tater.man" = { type = { name = "pug"; }; }; }; flt1 = 1; flt2 = 3.1415; flt3 = -0.01; flt4 = 5e+22; flt5 = 1e+06; flt6 = -0.02; flt7 = 6.626e-34; flt8 = 9.22462e+06; fruit = [ { name = "apple"; physical = { color = "red"; shape = "round"; }; variety = [ { name = "red delicious"; } { name = "granny smith"; } ]; } { name = "banana"; variety = [ { name = "plantain"; } ]; } ]; g = { h = { i = { }; }; }; hex1 = 3735928559; hex2 = 3735928559; hex3 = 3735928559; int1 = 99; int2 = 42; int3 = 0; int4 = -17; int5 = 1000; int6 = 5349221; int7 = 12345; j = { "ʞ" = { l = { }; }; }; key = "value"; key2 = "value"; ld1 = { _type = "timestamp"; value = "1979-05-27"; }; ldt1 = { _type = "timestamp"; value = "1979-05-27T07:32:00"; }; ldt2 = { _type = "timestamp"; value = "1979-05-27T00:32:00.999999"; }; lt1 = { _type = "timestamp"; value = "07:32:00"; }; lt2 = { _type = "timestamp"; value = "00:32:00.999999"; }; name = "Orange"; oct1 = 342391; oct2 = 493; odt1 = { _type = "timestamp"; value = "1979-05-27T07:32:00Z"; }; odt2 = { _type = "timestamp"; value = "1979-05-27T00:32:00-07:00"; }; odt3 = { _type = "timestamp"; value = "1979-05-27T00:32:00.999999-07:00"; }; odt4 = { _type = "timestamp"; value = "1979-05-27T07:32:00Z"; }; physical = { color = "orange"; shape = "round"; }; products = [ { name = "Hammer"; sku = 738594937; } { } { color = "gray"; name = "Nail"; sku = 284758393; } ]; "quoted \"value\"" = "value"; site = { "google.com" = true; }; str = "I'm a string. \"You can quote me\". Name\tJosé\nLocation\tSF."; table-1 = { key1 = "some string"; key2 = 123; }; table-2 = { key1 = "another string"; key2 = 456; }; x = { y = { z = { w = { animal = { type = { name = "pug"; }; }; name = { first = "Tom"; last = "Preston-Werner"; }; point = { x = 1; y = 2; }; }; }; }; }; "ʎǝʞ" = "value"; }
@@ -55,53 +55,11 @@ builtins.fromTOML ''
odt2 = 1979-05-27T00:32:00-07:00
odt3 = 1979-05-27T00:32:00.999999-07:00
odt4 = 1979-05-27 07:32:00Z
# milliseconds
odt5 = 1979-05-27 07:32:00.1Z
odt6 = 1979-05-27 07:32:00.12Z
odt7 = 1979-05-27 07:32:00.123Z
# microseconds
odt8 = 1979-05-27t07:32:00.1234Z
odt9 = 1979-05-27t07:32:00.12345Z
odt10 = 1979-05-27t07:32:00.123456Z
# nanoseconds
odt11 = 1979-05-27 07:32:00.1234567Z
odt12 = 1979-05-27 07:32:00.12345678Z
odt13 = 1979-05-27 07:32:00.123456789Z
# no more precision after nanoseconds
odt14 = 1979-05-27t07:32:00.1234567891Z
ldt1 = 1979-05-27T07:32:00
# milliseconds
ldt2 = 1979-05-27T07:32:00.1
ldt3 = 1979-05-27T07:32:00.12
ldt4 = 1979-05-27T07:32:00.123
# microseconds
ldt5 = 1979-05-27t00:32:00.1234
ldt6 = 1979-05-27t00:32:00.12345
ldt7 = 1979-05-27t00:32:00.123456
# nanoseconds
ldt8 = 1979-05-27 00:32:00.1234567
ldt9 = 1979-05-27 00:32:00.12345678
ldt10 = 1979-05-27 00:32:00.123456789
# no more precision after nanoseconds
ldt11 = 1979-05-27t00:32:00.1234567891
ldt2 = 1979-05-27T00:32:00.999999
ld1 = 1979-05-27
lt1 = 07:32:00
# milliseconds
lt2 = 00:32:00.1
lt3 = 00:32:00.12
lt4 = 00:32:00.123
# microseconds
lt5 = 00:32:00.1234
lt6 = 00:32:00.12345
lt7 = 00:32:00.123456
# nanoseconds
lt8 = 00:32:00.1234567
lt9 = 00:32:00.12345678
lt10 = 00:32:00.123456789
# no more precision after nanoseconds
lt11 = 00:32:00.1234567891
lt2 = 00:32:00.999999
arr1 = [ 1, 2, 3 ]
arr2 = [ "red", "yellow", "green" ]
-1
View File
@@ -132,7 +132,6 @@ set -u
[[ ${arr2[1]} = $'\n' ]]
[[ ${arr2[2]} = $'x\ny' ]]
[[ $(fun) = blabla ]]
[[ "$ASCII_ESC" = "$(printf "\e")" ]]
[[ $PATH = $(jq -r .variables.PATH.value $TEST_ROOT/dev-env.json):$path ]]
)
-1
View File
@@ -50,7 +50,6 @@ let pkgs = rec {
VAR_FROM_NIX = "bar";
ASCII_PERCENT = "%";
ASCII_AT = "@";
ASCII_ESC = "";
TEST_inNixShell = if inNixShell then "true" else "false";
inherit stdenv;
outputs = ["dev" "out"];
@@ -1,11 +1,12 @@
nix-repl> 1 + 1
2
nix-repl> :doc builtins.add
Synopsis: builtins.add e1 e2
Return the sum of the numbers e1 and e2.
nix-repl> :doc builtins.head
Synopsis: builtins.head list
Return the first element of a list; abort evaluation if
the argument isnt a list or is an empty list. You can
test whether a list is empty by comparing it with [].
nix-repl> f = a: "" + a
-75
View File
@@ -1,75 +0,0 @@
{ lib, config, pkgs, ... }:
let
failedNormal = config: pkgs.writeText "failed.nix" ''
let utils = builtins.storePath ${config.system.build.extraUtils}; in
derivation {
name = "failed";
system = builtins.currentSystem;
PATH = "''${utils}/bin";
builder = "''${utils}/bin/sh";
args = [ "-c" "mkdir dir; echo test > dir/file" ];
}
'';
failedBuiltin = pkgs.writeText "failed.nix" ''
derivation {
name = "failed";
system = builtins.currentSystem;
builder = "builtin:fetchurl";
url = "http://localhost/foo";
outputHashMode = "flat";
}
'';
in
{
name = "chown-to-user";
nodes = {
machine = { config, lib, pkgs, ... }: {
virtualisation.writableStore = true;
virtualisation.additionalPaths = [ config.system.build.extraUtils ];
users.users.test = {
isNormalUser = true;
group = "test";
};
users.groups.test = {};
nix.nrBuildUsers = 1;
};
};
testScript = { nodes, ... }: ''
import re
machine.wait_for_unit("multi-user.target")
# builds using the daemon chown tempdirs
out = machine.fail("runuser -u test -- nix-build ${failedNormal nodes.machine} --keep-failed 2>&1")
dir = re.search("keeping build directory '(.+?)'", out)
assert dir
assert machine.succeed(f"stat -c %U:%G:%a {dir[1]}").strip() == "root:nixbld:755"
assert machine.succeed(f"stat -c %U:%G:%a {dir[1]}/b").strip() == "test:test:700"
assert machine.succeed(f"stat -c %U:%G:%a {dir[1]}/b/dir").strip() == "test:test:755"
assert machine.succeed(f"stat -c %U:%G:%a {dir[1]}/b/dir/file").strip() == "test:test:644"
# builds not using the daemon do not chown tempdirs
out = machine.fail("NIX_REMOTE=local nix-build ${failedNormal nodes.machine} --keep-failed 2>&1")
dir = re.search("keeping build directory '(.+?)'", out)
assert dir
assert machine.succeed(f"stat -c %U:%G:%a {dir[1]}").strip() == "root:nixbld:755"
assert machine.succeed(f"stat -c %U:%G:%a {dir[1]}/b").strip() == "nixbld1:nixbld:700"
assert machine.succeed(f"stat -c %U:%G:%a {dir[1]}/b/dir").strip() == "nixbld1:nixbld:755"
assert machine.succeed(f"stat -c %U:%G:%a {dir[1]}/b/dir/file").strip() == "nixbld1:nixbld:644"
# builds using builtin builders using the daemon do not keep tempdirs
out = machine.fail("runuser -u test -- nix-build ${failedBuiltin} --keep-failed 2>&1")
dir = re.search("keeping build directory '(.+?)'", out)
assert not dir
# builds using builtin builders not using the daemon do not keep tempdirs
out = machine.fail("NIX_REMOTE=local nix-build ${failedBuiltin} --keep-failed 2>&1")
dir = re.search("keeping build directory '(.+?)'", out)
assert not dir
'';
}
+3 -2
View File
@@ -169,6 +169,9 @@ in
symlinkResolvconf = runNixOSTestFor "x86_64-linux" ./symlink-resolvconf.nix;
# Use this test to test things that cannot easily be tested under chroot Nix stores in functional test suite.
non-chroot-misc = runNixOSTestFor "x86_64-linux" ./non-chroot-misc;
noNewPrivilegesInSandbox = runNixOSTestFor "x86_64-linux" ./no-new-privileges/sandbox.nix;
noNewPrivilegesOutsideSandbox = runNixOSTestFor "x86_64-linux" ./no-new-privileges/no-sandbox.nix;
@@ -180,6 +183,4 @@ in
io_uring = runNixOSTestFor "x86_64-linux" ./io_uring;
fetchurl = runNixOSTestFor "x86_64-linux" ./fetchurl.nix;
chown-to-user = runNixOSTestFor "x86_64-linux" ./chown-to-user.nix;
}
+34
View File
@@ -0,0 +1,34 @@
{ ... }:
# Misc things we want to test inside of a non redirected, non chroot Nix store.
let
nonAutoCleaningFailingDerivationCode = ''
derivation {
name = "scratch-failing";
system = builtins.currentSystem;
builder = "/bin/sh";
args = [ (builtins.toFile "builder.sh" "echo bonjour > $out; echo out: $out; false") ];
}
'';
in
{
name = "non-chroot-sandbox-misc";
nodes.machine = {
};
testScript = { nodes }: ''
import re
start_all()
# You might ask yourself why write such a convoluted thing?
# The condition for fooling Nix into NOT cleaning up the output path are non trivial and unclear.
# This is one of those: create a derivation, mkdir or touch the $out path, communicate it back.
# Even with a sandboxed Lix, you will observe leftovers before 2.93.0. After this version, this test passes.
result = machine.fail("""nix-build --substituters "" -E '${nonAutoCleaningFailingDerivationCode}' 2>&1""")
match = re.search(r'out: (\S+)', result)
assert match is not None, "Did not find Nix store path in the result of the failing build"
outpath = match.group(1).strip()
print(f"Found Nix store path: {outpath}")
machine.fail(f'stat {outpath}')
'';
}
-1
View File
@@ -48,7 +48,6 @@ in
imports = [ test.config.builders.config ];
services.openssh.enable = true;
virtualisation.writableStore = true;
virtualisation.additionalPaths = [ config.system.build.extraUtils ];
nix.settings.sandbox = true;
nix.settings.substituters = lib.mkForce [ ];
};
-23
View File
@@ -509,27 +509,4 @@ INSTANTIATE_TEST_SUITE_P(
concat({header, make_directory({{"DE", make_file(false, "meow")}, {"de", make_file(false, "mrrp")}})})
))
);
TEST_F(NarTest, stringSizeLimit)
{
GeneratorSource source([]() -> Generator<Bytes> {
const char preamble[] =
"\x0d\x00\x00\x00\x00\x00\x00\x00nix-archive-1\x00\x00\x00"
"\x01\x00\x00\x00\x00\x00\x00\x00(\x00\x00\x00\x00\x00\x00\x00"
"\x04\x00\x00\x00\x00\x00\x00\x00type\x00\x00\x00\x00";
co_yield Bytes{preamble, sizeof(preamble) - 1};
// the nar parser keeps all strings in a buffer with the 8 byte length prefix in front.
// sufficiently large strings overflowed caused the buffer size calculation to overflow
// and thus allowed out-of-bounds writes in the daemon and potentially privesc to root.
co_yield Bytes{"\xf7\xff\xff\xff\xff\xff\xff\xff", 8};
// overflow would happen while reading data
while (true) {
co_yield Bytes{"foo-", 4};
}
}());
auto parser = nar::parse(source);
ASSERT_THROW(parser.next(), SerialisationError);
}
}
-58
View File
@@ -364,64 +364,6 @@ namespace nix {
ASSERT_THROW(base64Decode("cXVvZCBlcm_0IGRlbW9uc3RyYW5kdW0="), Error);
}
/* ----------------------------------------------------------------------------
* base32Encode
* --------------------------------------------------------------------------*/
TEST(base32Encode, emptyString) {
ASSERT_EQ(base32EncodeStr(""), "");
}
TEST(base32Encode, encodesAString) {
ASSERT_EQ(base32EncodeStr("quod erat demonstrandum"), "6sxb4drhp4x3kdrpnsrb441s62wk541j6yxbi");
}
TEST(base32Encode, encodeAndDecode) {
auto s = "quod erat demonstrandum";
auto encoded = base32EncodeStr(s);
auto decoded = base32Decode(encoded);
ASSERT_EQ(decoded, s);
}
TEST(base32Encode, encodeAndDecodeNonPrintable) {
std::string s(257, '\0');
std::iota(std::rbegin(s), std::rend(s), 0);
auto encoded = base32EncodeStr(s);
auto decoded = base32Decode(encoded);
EXPECT_EQ(decoded.length(), 257);
ASSERT_EQ(decoded, s);
}
TEST(base32Encode, handleNulChars) {
std::string s = "cat girls say meow even with NULs";
// Just throw a NUL in there somewhere.
s[5] = '\0';
auto encoded = base32EncodeStr(s);
auto decoded = base32Decode(encoded);
EXPECT_EQ(decoded, s);
}
/* ----------------------------------------------------------------------------
* base32Decode
* --------------------------------------------------------------------------*/
TEST(base32Decode, emptyString) {
ASSERT_EQ(base32Decode(""), "");
}
TEST(base32Decode, decodeAString) {
ASSERT_EQ(base32Decode("6sxb4drhp4x3kdrpnsrb441s62wk541j6yxbi"), "quod erat demonstrandum");
}
TEST(base32Decode, decodeThrowsOnInvalidChar) {
ASSERT_THROW(base32Decode("6sxb4drhp4x3kdrpnsrb441s62wk541j6yxbe"), Error);
}
/* ----------------------------------------------------------------------------
* getLine
* --------------------------------------------------------------------------*/
+1 -1
View File
@@ -1,5 +1,5 @@
{
"version": "2.93.4",
"version": "2.93.1",
"official_release": true,
"release_name": "Bici Bici"
}