Compare commits

...
19 Commits
Author SHA1 Message Date
Raito Bezarius c8a447db88 release: 2.94.2 "Açaí na tigela"
Release produced with releng/create_release.xsh

Change-Id: I7cfa601006515be2710bc01111c6adc34ee5c048
2026-05-04 19:02:22 +02:00
Raito Bezarius ec912eda76 release: release notes for 2.94.2
Release created with releng/create_release.xsh

Change-Id: I117f10a79a674d262446555206e4976bb330f491
2026-05-04 19:02:20 +02:00
eldritch horrorsandRaito Bezarius 667b40f1ff libutil: fix nar parser buffer overflow
string data shares a buffer with the binary string length field. size
calculations for string read buffers always include the length field;
sufficiently large length fields can cause these calculations to wrap.
a malicious nar could use this for OOB writes in the daemon (as root).

since we use strings only as tags for archive members and for symlinks
with their OS-dependent length limits we can simply limit string size.
1 MiB should be sufficient for all symlinks, and tags are always tiny.

Change-Id: I89fb05f73c1dbeda45d91244aba4cd526a3d83e1
2026-05-04 19:01:44 +02:00
Raito Bezarius 64599c010c releng/keys: update the way to receive the ephemeral key
I don't understand how `ssh -l lix-releng` is supposed to work if it
doesn't say which host to target.

Change-Id: I791f3f3f49ecd5884c9e86b5d3b617fc139e031f
Signed-off-by: Raito Bezarius <raito@lix.systems>
2026-05-04 19:01:44 +02:00
Raito Bezarius 1953c4ae4b releng/environment: update staging parameters
These parameters are now created on https://s3.afnix.fr.

Change-Id: I96b6fd913429ee46d04c412cb141edd288665ced
Signed-off-by: Raito Bezarius <raito@lix.systems>
2026-05-04 19:01:44 +02:00
YurekaandRaito Bezarius 7e56afa0b1 releng: Adapt for AFNix S3
Change-Id: I29dbd62dcc70595ba3f2ac2a466a5c26a28aea99
(cherry picked from commit 0c63036c7d)
2026-05-04 19:01:44 +02:00
Florian KlinkandRaito Bezarius 5d6134064e libcstore: Fix null deref in writeDebugInfo for non-directory NARs
When index-debug-info is enabled and the store path being copied is a
regular file (not a directory), std::get_if<nar_index::Directory>
returns nullptr since the NAR root is a File variant. The loop then
immediately dereferences buildIdDir->contents on the null pointer,
causing a segfault.

Add a null check at the top of the loop to break early when the NAR
root is not a directory.

Change-Id: I3a6e792b84cc12c837ecaddf4fee889e1bcb6397
(cherry picked from commit 6c7ccc2588)
2026-05-04 16:33:04 +00:00
sterni 1d36c544be libcmd: add support for lowdown >= 3.0.0
lowdown 3.0.0 merged some flags into one to save on bits and did not add
any aliases for backward compatibility.

As with the changes for lowdown >= 1.4, we define a preprocessor flag to
gate the changes on and add a job to CI to ensure that lowdown < 3.0
keeps working (which is used by NixOS 25.11).

The channel version this Lix branch uses doesn't have bmake with support
for all tested platforms, so we can't properly build lowdown 3.0 (which
depends on it) for tests in CI. Given this is a relatively simple
backport tested on other Lix branches, it's probably fine.

Change-Id: I20a3e2fdaa05906f032ff66911c42867557fdd11
(cherry picked from commit e839708839aa132c8694abfdf83409a619f72c5a)
2026-04-10 16:04:52 +02:00
Jade Lovelace 7b210490aa version.json: 2.94.2, if we do backport more stuff
Change-Id: Ide1ce2a59a80a524d62c77681c2ea45e859a77e2
2026-03-13 12:17:56 -07:00
Jade Lovelace fba0fa5767 release: merge release 2.94.1 back to mainline
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh

Change-Id: I2c10392b0b80bff519501c8018ed51aa6a151ca1
2026-03-13 09:39:07 -07:00
Jade Lovelace 533429d89a release: 2.94.1 "Açaí na tigela"
Release produced with releng/create_release.xsh

Change-Id: Ie38032fe8dfebed83f9d24762ce48f73b23149b5
2026-03-13 09:39:06 -07:00
Jade Lovelace c135090468 release: release notes for 2.94.1
Release created with releng/create_release.xsh

Change-Id: I970e8f5422e3705917b04c09d28498fd7b849d27
2026-03-13 09:38:52 -07:00
Jade Lovelace 72f7965679 2.94.1: version
Change-Id: I92eeeebfd6b966ce77833785482db989962f8c9e
2026-03-13 09:38:44 -07:00
Raito Bezarius b7be40c785 libstore/build: fix starvation during substitution
When the destructor of PathSubstitutionGoal is run, this happens in a
sync context and can cause starvation of all ongoing IO w.r.t. to other
substitutions, including our own substitution.

While there's only a decompressor thread per stream, the other side of
the IO runs on the event loop.

In order to fix this, it is sufficient to remove the thread indirection
and inline the async code.

Fixes #1126. Great thanks to horrors' patience.

Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I3eb37bc37d156f0f5528364e568fdaa2ced58011
Signed-off-by: Raito Bezarius <raito@lix.systems>
(cherry picked from commit 505d0669dc)
2026-02-11 15:39:06 +00:00
Qyriad 6350f51458 libutil: include LIX_MAJOR, LIX_MINOR, and LIX_PATCH macros
Backport of I7d8a4648890fce7ff15695876c9b9d3a6a6a6964 to 2.94 branch.

Change-Id: Ia6b8d89007974c8cf873fa1f536ce8416a6a6964
2026-01-15 11:48:26 +00:00
eldritch horrorsandRaito Bezarius c6f3f3a0d3 libexpr: fix app chain extension
during the value rewrite we accidentally broke extension of incomplete
primop application. this only shows up when binding on incomplete call
to a primop to a name, binding an incomplete call to *that* to another
name, and then finally calling the second binding with enough args for
a complete primop application. since this only shows up when calling a
primop with three or more args it took a while to surface. we have few
builtins that match this: foldl', replaceStrings, and substring. these
are not used incompletely in this manner very often, so this lingered.

fixes #1102

Change-Id: I218dffc14ae876efc86a86c7eb6c895e2405201c
2026-01-14 22:09:56 +00:00
Maximilian Bosch de4cfec46a libstore: fix reporting output cycles on drvs with references to other drvs
Closes #1064

The culprit here is that `genGraphString` is only invoked with the
store-paths associated with the outputs of the derivation, so when
filling `dependents`, the `graph_data.find(p)` call would return the end
of the iterator when doing this for references to other store-paths.

As a result, the code wrote information behind the graph data-structure
causing a corruption. For me, this resulted in a SIGSEGV most of the
time and in a few cases in an uncaught `map::at`-exception as reported
by Niklas.

This patch changes two aspects of the original implementation:

* When filling `dependents` in the node-set, use `map.at()` instead of
  `map.find()->second`. The latter doesn't make any sense and was the
  cause of corrupting memory. The `at` would've made it far easier to
  spot this in the first place.

* Filter out store-paths that don't belong to a different output of the
  derivation when creating `outputGraph`. This variable is used on two
  places, `genGraphString` and for topological sorting.

  The latter already filters out store-paths from a different drv, so
  this is happening now when creating the variable in the first place
  such that `genGraphString` never ends up with corrupt data in the
  first place. This is the actual bugfix.

Implemented a regression-test for this case to be sure.

Change-Id: Ie02144d89c32b0a776cb1ece0601d0229315ebc3
(cherry picked from commit 0a5f474a25)
2025-12-08 07:20:51 +01:00
Raito Bezarius 0873bed39d nix/upgrade-nix: disallow daemon connections for the store
Prior to I6a6a6964d2b5ad47ae5ea9eb11af9b6373ce2141 — `sudo nix
upgrade-nix` would perform direct store access.

This ensured a certain number of desireable properties for upgrading the
Lix binary itself.

We re-introduce direct store access for upgrading Lix binaries.

Fixes #1060.

Change-Id: I523c4d3023ed5fe9eff8fde9a266c56a0de47d8c
Signed-off-by: Raito Bezarius <raito@lix.systems>
(cherry picked from commit d2ca1810b1)
2025-12-06 22:20:18 +00:00
Zoe ZuserandRaito Bezarius 5dcb90548f meson: fix libstore.pc
typo of aws-cpp-sdk-transfer as aws-cpp-std-transfer prevents linking
against lix

Change-Id: Id847eab2601698696030d31fcd51288aa5f3d274
(cherry picked from commit 06f987fb0c)
2025-12-05 16:58:28 +00:00
32 changed files with 322 additions and 241 deletions
+6
View File
@@ -73,6 +73,9 @@ detroyejr:
display_name: Jonathan De Troye
github: detroyejr
edef:
github: edef1c
edolstra:
display_name: Eelco Dolstra
github: edolstra
@@ -229,6 +232,9 @@ roberth:
display_name: Robert Hensing
github: roberth
sandydoo:
github: sandydoo
seppel3210:
github: Seppel3210
+34
View File
@@ -1,4 +1,38 @@
# Lix 2.94 "Açaí na tigela" (2025-11-17)
# Lix 2.94.2 (2026-05-04)
## Fixes
- Fix unsigned overflow leading to out-of-band write in the NAR parser [cl/5553](https://gerrit.lix.systems/c/lix/+/5553)
The NAR parser contained an unsigned integer overflow that could be used by an
attacker to write arbitrary data to an unknown memory location and possibly
achieve code execution. A successful attack on the system-wide Lix daemon
could lead to privilege escalation to root. Any process that involves NAR
serialization could trigger this issue, including (but not limited to)
- local user interaction, whether the users are trusted or untrusted
- malicious substituters sending malformed NARs
- remote builders sending malformed build results
- remote daemons sending malformed inputs when requesting remote builds
Successful attacks using this bug require ASLR weakening of some sort, whether
by architecture constraints (e.g. on 32 bit systems, where little randomization
is possible) or system configuration (e.g. low ASLR entropy when loading
libraries), and millions of attempts. Local attacks can be mounted in less than
an hour. Remote builds typically require a fresh SSH connection for each build
and are thus less susceptible. Only one attempt can be made by substituters for
every build using substituters, they are thus not a likely vector for attacks.
At the time of writing, MITRE has not assigned this a CVE yet.
Many thanks to [eldritch horrors](https://git.lix.systems/pennae), [Raito Bezarius](https://git.lix.systems/raito), [edef](https://github.com/edef1c), and [sandydoo](https://github.com/sandydoo) for this.
# Lix 2.94.1 (2026-03-13)
# Lix 2.94.0 (2025-11-17)
+4
View File
@@ -74,7 +74,11 @@ std::string renderMarkdownToTerminal(std::string_view markdown, StandardOutputSt
.vmargin = 0,
#endif /* LOWDOWN_SEPARATE_TERM_OPTS */
.feat = LOWDOWN_COMMONMARK | LOWDOWN_FENCED | LOWDOWN_DEFLIST | LOWDOWN_TABLES,
#ifdef LOWDOWN_CONSOLIDATED_OFLAGS
.oflags = LOWDOWN_NOLINK,
#else
.oflags = LOWDOWN_TERM_NOLINK,
#endif /* LOWDOWN_CONSOLIDATED_OFLAGS */
};
if (!shouldANSI(fileno)) {
opts.oflags |= LOWDOWN_TERM_NOANSI;
+13 -3
View File
@@ -20,11 +20,21 @@ inline Value::Value(app_t, EvalMemory & mem, Value & lhs, Value & rhs)
}
inline Value::Value(app_t, EvalMemory & mem, Value & lhs, std::span<Value> args)
: Value(app_t{}, mem, lhs, args, {})
{
auto app = static_cast<Value::App *>(mem.allocBytes(sizeof(Value::App) + args.size_bytes()));
}
inline Value::Value(
app_t, EvalMemory & mem, const Value & lhs, std::span<Value> baseArgs, std::span<Value> moreArgs
)
{
auto app = static_cast<Value::App *>(
mem.allocBytes(sizeof(Value::App) + baseArgs.size_bytes() + moreArgs.size_bytes())
);
app->_left = lhs;
app->_n = args.size();
std::copy(args.begin(), args.end(), app->_args);
app->_n = baseArgs.size() + moreArgs.size();
std::copy(baseArgs.begin(), baseArgs.end(), app->_args);
std::copy(moreArgs.begin(), moreArgs.end(), app->_args + baseArgs.size());
raw = tag(tApp, app);
}
+8 -1
View File
@@ -1639,7 +1639,14 @@ void EvalState::callFunction(Value & fun, std::span<Value> args, Value & vRes, c
Value vCur(fun);
auto makeAppChain = [&]() { vRes = {NewValueAs::app, ctx.mem, vCur, args}; };
auto makeAppChain = [&]() {
if (vCur.isApp()) {
auto & app = vCur.app();
vRes = {NewValueAs::app, ctx.mem, app.left(), app.args(), args};
} else {
vRes = {NewValueAs::app, ctx.mem, vCur, args};
}
};
const Attr * functor;
+4
View File
@@ -544,6 +544,10 @@ public:
/// lazy and/or partial application of a function.
Value(app_t, EvalMemory & mem, Value & lhs, std::span<Value> args);
/// Constructs a nix language value of type "lambda", which represents a
/// lazy and/or partial application of a function.
Value(app_t, EvalMemory & mem, const Value & lhs, std::span<Value> baseArgs, std::span<Value> moreArgs);
/// Constructs a nix language value of type "external", which is only used
/// by plugins. Do any existing plugins even use this mechanism?
Value(external_t, ExternalValueBase & external)
+1
View File
@@ -225,6 +225,7 @@ try {
auto * buildIdDir = std::get_if<nar_index::Directory>(&narIndex);
for (auto subdir : { "lib", "debug", ".build-id" }) {
if (!buildIdDir) break;
// get returns nullptr subdir does not exist, and std::get_if propagates it.
buildIdDir = std::get_if<nar_index::Directory>(get(buildIdDir->contents, subdir));
}
+8 -1
View File
@@ -139,7 +139,14 @@ Goal::WorkResult DerivationGoal::timedOut(Error && ex)
kj::Promise<Result<Goal::WorkResult>> DerivationGoal::workImpl() noexcept
{
KJ_DEFER({ actLock.reset(); });
// always clear the slot token, no matter what happens. not doing this
// can cause builds to get stuck on exceptions (or other early exits).
// ideally we'd use scoped slot tokens instead of keeping them in some
// goal member variable, but we cannot do this yet for legacy reasons.
KJ_DEFER({
actLock.reset();
slotToken = {};
});
BOOST_OUTCOME_CO_TRY(auto result, co_await (useDerivation ? getDerivation() : haveDerivation()));
result.storePath = drvPath;
+3 -1
View File
@@ -246,7 +246,7 @@ struct DerivationGoal : public Goal
WorkResult timedOut(Error && ex);
kj::Promise<Result<WorkResult>> workImpl() noexcept override;
kj::Promise<Result<WorkResult>> workImpl() noexcept override final;
/**
* Add wanted outputs to an already existing derivation goal.
@@ -307,6 +307,8 @@ struct DerivationGoal : public Goal
virtual void cleanupPostOutputsRegisteredModeNonCheck();
protected:
AsyncSemaphore::Token slotToken;
kj::TimePoint lastChildActivity = kj::minValue;
kj::Promise<Result<WorkResult>> wrapChildHandler(kj::Promise<Result<WorkResult>> handler
-6
View File
@@ -22,12 +22,6 @@ kj::Promise<void> Goal::waitForAWhile()
kj::Promise<Result<Goal::WorkResult>> Goal::work() noexcept
try {
// always clear the slot token, no matter what happens. not doing this
// can cause builds to get stuck on exceptions (or other early exist).
// ideally we'd use scoped slot tokens instead of keeping them in some
// goal member variable, but we cannot do this yet for legacy reasons.
KJ_DEFER({ slotToken = {}; });
BOOST_OUTCOME_CO_TRY(auto result, co_await workImpl());
trace("done");
-3
View File
@@ -82,9 +82,6 @@ struct Goal
*/
std::string name;
protected:
AsyncSemaphore::Token slotToken;
public:
struct [[nodiscard]] WorkResult {
ExitCode exitCode;
+9 -8
View File
@@ -1825,15 +1825,18 @@ try {
);
}
outputGraph[scratchOutputs.at(name)] = StorePathSet{};
std::visit(
overloaded{/* Since we'll use the already installed versions of these, we
can treat them as leaves and ignore any references they
have. */
[&](const AlreadyRegistered &) {
outputGraph[scratchOutputs.at(name)] = StorePathSet{};
},
[&](const AlreadyRegistered &) {},
[&](const PerhapsNeedToRegister & refs) {
outputGraph[scratchOutputs.at(name)] = refs.refs;
for (auto & ref : refs.refs) {
if (inverseOutputMap.find(ref) != inverseOutputMap.end()) {
outputGraph[scratchOutputs.at(name)].insert(ref);
}
}
}
},
*orifu
@@ -1844,10 +1847,8 @@ try {
topoSort(outputsToSort, {[&](const std::string & name) {
StringSet dependencies;
for (auto & path : outputGraph.at(scratchOutputs.at(name))) {
auto outputName = inverseOutputMap.find(path);
if (outputName != inverseOutputMap.end()) {
dependencies.insert(outputName->second);
}
auto outputName = inverseOutputMap.at(path);
dependencies.insert(outputName);
}
return dependencies;
}});
+39 -78
View File
@@ -27,13 +27,6 @@ PathSubstitutionGoal::PathSubstitutionGoal(
maintainExpectedSubstitutions = worker.expectedSubstitutions.addTemporarily(1);
}
PathSubstitutionGoal::~PathSubstitutionGoal()
{
cleanup();
}
Goal::WorkResult PathSubstitutionGoal::done(
ExitCode result,
BuildResult::Status status,
@@ -76,8 +69,6 @@ kj::Promise<Result<Goal::WorkResult>> PathSubstitutionGoal::tryNext() noexcept
try {
trace("trying next substituter");
cleanup();
if (subs.size() == 0) {
/* None left. Terminate this goal and let someone else deal
with it. */
@@ -205,61 +196,36 @@ kj::Promise<Result<Goal::WorkResult>> PathSubstitutionGoal::tryToRun() noexcept
try {
trace("trying to run");
if (!slotToken.valid()) {
slotToken = co_await worker.substitutions.acquire();
}
maintainRunningSubstitutions = worker.runningSubstitutions.addTemporarily(1);
auto pipe = kj::newPromiseAndCrossThreadFulfiller<void>();
outPipe = kj::mv(pipe.fulfiller);
thr = std::async(std::launch::async, [this]() {
AsyncIoRoot aio;
/* Wake up the worker loop when we're done. */
Finally updateStats([this]() { outPipe->fulfill(); });
auto & fetchPath = subPath ? *subPath : storePath;
try {
ReceiveInterrupts receiveInterrupts;
auto act = logger->startActivity(
actSubstitute, Logger::Fields{worker.store.printStorePath(storePath), sub->getUri()}
);
aio.blockOn(copyStorePath(
*sub,
worker.store,
fetchPath,
repair,
sub->config().isTrusted ? NoCheckSigs : CheckSigs,
&act
));
} catch (const EndOfFile &) {
throw EndOfFile(
"NAR for '%s' fetched from '%s' is incomplete",
sub->printStorePath(fetchPath),
sub->getUri()
);
}
});
co_await pipe.promise;
co_return co_await finished();
} catch (...) {
co_return result::current_exception();
}
kj::Promise<Result<Goal::WorkResult>> PathSubstitutionGoal::finished() noexcept
try {
trace("substitute finished");
auto & fetchPath = subPath ? *subPath : storePath;
do {
try {
slotToken = {};
thr.get();
break;
try {
AsyncSemaphore::Token slotToken = co_await worker.substitutions.acquire();
auto act = logger->startActivity(
actSubstitute,
Logger::Fields{worker.store.printStorePath(storePath), sub->getUri()}
);
maintainRunningSubstitutions = worker.runningSubstitutions.addTemporarily(1);
TRY_AWAIT(copyStorePath(
*sub,
worker.store,
fetchPath,
repair,
sub->config().isTrusted ? NoCheckSigs : CheckSigs,
&act
));
break;
} catch (const EndOfFile &) {
throw EndOfFile(
"NAR for '%s' fetched from '%s' is incomplete",
sub->printStorePath(fetchPath),
sub->getUri()
);
}
} catch (std::exception & e) { // NOLINT(lix-foreign-exceptions)
printError("%1%", Uncolored(e.what()));
@@ -271,10 +237,20 @@ try {
substituterFailed = true;
}
}
/* Try the next substitute. */
/* Try the next substitute */
co_return co_await tryNext();
} while (false);
co_return co_await finished();
} catch (...) {
co_return result::current_exception();
}
kj::Promise<Result<Goal::WorkResult>> PathSubstitutionGoal::finished() noexcept
try {
trace("substitute finished");
worker.markContentsGood(storePath);
printMsg(lvlChatty, "substitution of path '%s' succeeded", worker.store.printStorePath(storePath));
@@ -294,19 +270,4 @@ try {
} catch (...) {
co_return result::current_exception();
}
void PathSubstitutionGoal::cleanup()
{
try {
if (thr.valid()) {
// FIXME: signal worker thread to quit.
thr.get();
}
} catch (...) {
ignoreExceptionInDestructor();
}
}
}
-9
View File
@@ -48,11 +48,6 @@ struct PathSubstitutionGoal : public Goal
*/
kj::Own<kj::CrossThreadPromiseFulfiller<void>> outPipe;
/**
* The substituter thread.
*/
std::future<void> thr;
/**
* Whether to try to repair a valid path.
*/
@@ -85,7 +80,6 @@ public:
RepairFlag repair = NoRepair,
std::optional<ContentAddress> ca = std::nullopt
);
~PathSubstitutionGoal();
kj::Promise<Result<WorkResult>> workImpl() noexcept override;
@@ -97,9 +91,6 @@ public:
kj::Promise<Result<WorkResult>> tryToRun() noexcept;
kj::Promise<Result<WorkResult>> finished() noexcept;
/* Called by destructor, can't be overridden */
void cleanup() override final;
JobCategory jobCategory() const override {
return JobCategory::Substitution;
};
+1 -1
View File
@@ -471,6 +471,6 @@ configure_file(
'libdir' : libdir,
'includedir' : includedir,
'PACKAGE_VERSION' : meson.project_version(),
'AWS_SDK_IF_FOUND' : aws_sdk.found() ? 'aws-cpp-sdk-core aws-cpp-sdk-s3 aws-cpp-std-transfer' : '',
'AWS_SDK_IF_FOUND' : aws_sdk.found() ? 'aws-cpp-sdk-core aws-cpp-sdk-s3 aws-cpp-sdk-transfer' : '',
},
)
+1 -1
View File
@@ -265,7 +265,7 @@ static std::map<StorePath, Node> mkGraph(
for (auto & node : graph_data) {
for (auto & ref : node.second.dependencies) {
graph_data.find(ref)->second.dependents.insert(node.first);
graph_data.at(ref).dependents.insert(node.first);
}
}
+1 -1
View File
@@ -384,7 +384,7 @@ struct Parser
buffer.clear(); \
std::move(str); \
})
#define READ_STRING() READ_STRING_LIMITED(std::numeric_limits<size_t>::max())
#define READ_STRING() READ_STRING_LIMITED(1048576)
#define READ_PADDING(size) \
do { \
if ((size) % 8) { \
+10
View File
@@ -1,7 +1,17 @@
# Cursed, but I don't think there's another way to get this environment variable.
lix_suffix = run_command('bash', '-c', 'echo -n "$VERSION_SUFFIX"', check : true).stdout().strip()
lix_version_parts = meson.project_version().split('.')
lix_major = lix_version_parts[0]
lix_minor = lix_version_parts[1]
lix_patch = lix_version_parts[2].replace(lix_suffix, '')
config_h = configure_file(
configuration : {
'PACKAGE_NAME': '"' + meson.project_name() + '"',
'PACKAGE_VERSION': '"' + meson.project_version() + '"',
'LIX_MAJOR': lix_major,
'LIX_MINOR': lix_minor,
'LIX_PATCH': lix_patch,
'PACKAGE_TARNAME': '"' + meson.project_name() + '"',
'PACKAGE_STRING': '"' + meson.project_name() + ' ' + meson.project_version() + '"',
'HAVE_STRUCT_DIRENT_D_TYPE': 1, # FIXME: actually check this for solaris
+13
View File
@@ -49,6 +49,19 @@ struct CmdUpgradeNix : MixDryRun, EvalCommand
});
}
// NOTE(Raito): we override the store creation
// to prevent any store daemon connection.
//
// An upgrade, by nature, requires a direct store access
// to avoid having the daemon die in the middle of changing the binary.
//
// If more commands needs that, we can move it into a mixin. This was deliberately not done
// here.
virtual ref<Store> createStore(AsyncIoRoot & aio) override
{
return aio.blockOn(openStore(settings.storeUri.get(), {}, AllowDaemon::Disallow));
}
/**
* This command is stable before the others
*/
+5
View File
@@ -350,6 +350,11 @@ if lowdown.version().version_compare('>= 1.4.0')
add_project_arguments('-DLOWDOWN_SEPARATE_TERM_OPTS', language: 'cpp')
endif
# TODO(sterni): drop the corresponding #ifdef after NixOS 25.11 is EOL which still distributes lowdown < 3.0.0
if lowdown.version().version_compare('>= 3.0.0')
add_project_arguments('-DLOWDOWN_CONSOLIDATED_OFLAGS', language: 'cpp')
endif
# HACK(Qyriad): rapidcheck's pkg-config doesn't include the libs lol
# Note: technically we 'check' for rapidcheck twice, for the internal-api-docs handling above,
# but Meson will cache the result of the first one, and the required : arguments are different.
+1 -1
View File
@@ -37,7 +37,7 @@ def setup_creds(env: RelengEnvironment):
key = keys.get_ephemeral_key(env)
$AWS_SECRET_ACCESS_KEY = key.secret_key
$AWS_ACCESS_KEY_ID = key.id
$AWS_DEFAULT_REGION = 'garage'
$AWS_DEFAULT_REGION = env.s3_region
$AWS_ENDPOINT_URL = env.s3_endpoint
+20 -11
View File
@@ -5,11 +5,12 @@ import functools
import subprocess
import dataclasses
S3_HOST = 's3.lix.systems'
S3_HOST = 's3-admin.afnix.fr'
S3_USER = 'lix-releng'
DEFAULT_STORE_URI_BITS = {
'region': 'garage',
'endpoint': 's3.lix.systems',
'region': 'global',
'endpoint': 's3.afnix.fr',
'want-mass-query': 'true',
'write-nar-listing': 'true',
'ls-compression': 'zstd',
@@ -54,7 +55,9 @@ class RelengEnvironment:
git_repo: Callable[[], str]
git_repo_is_gerrit: bool
s3_endpoint: str
s3_region: str
s3_ssh_host: str | None
s3_ssh_user: str | None
docker_targets: list[DockerTarget]
@@ -86,17 +89,19 @@ LOCAL = RelengEnvironment(
git_repo_is_gerrit=False,
docker_targets=[],
s3_endpoint = 'http://localhost:3900',
s3_region = 'garage',
s3_ssh_host = None,
s3_ssh_user = None,
)
STAGING = RelengEnvironment(
name='staging',
colour=functools.partial(sgr, GREEN),
docs_bucket='s3://staging-docs',
cache_bucket='s3://staging-cache',
docs_bucket='s3://docs.staging.lix.systems',
cache_bucket='s3://cache.staging.lix.systems',
cache_store_overlay={'secret-key': 'staging.key'},
releases_bucket='s3://staging-releases',
releases_bucket='s3://releases.staging.lix.systems',
git_repo=lambda: 'ssh://git@git.lix.systems/lix-project/lix-releng-staging',
git_repo_is_gerrit=False,
docker_targets=[
@@ -106,8 +111,10 @@ STAGING = RelengEnvironment(
DockerTarget('ghcr.io/lix-project/lix-releng-staging',
tags=['{version}', '{major}']),
],
s3_endpoint = 'https://s3.lix.systems',
s3_endpoint = 'https://s3.afnix.fr',
s3_region = 'garage',
s3_ssh_host = S3_HOST,
s3_ssh_user = S3_USER,
)
GERRIT_REMOTE_RE = re.compile(r'^ssh://(\w+@)?gerrit.lix.systems:2022/lix$')
@@ -127,13 +134,13 @@ def guess_gerrit_remote():
PROD = RelengEnvironment(
name='production',
colour=functools.partial(sgr, RED),
docs_bucket='s3://docs',
cache_bucket='s3://cache',
docs_bucket='s3://docs.lix.systems',
cache_bucket='s3://cache.lix.systems',
# FIXME: we should decrypt this with age into a tempdir in the future, but
# the issue is how to deal with the recipients file. For now, we should
# just delete it after doing a release.
cache_store_overlay={'secret-key': 'prod.key'},
releases_bucket='s3://releases',
releases_bucket='s3://releases.lix.systems',
git_repo=guess_gerrit_remote,
git_repo_is_gerrit=True,
docker_targets=[
@@ -142,8 +149,10 @@ PROD = RelengEnvironment(
tags=['{version}', '{major}']),
DockerTarget('ghcr.io/lix-project/lix', tags=['{version}', '{major}']),
],
s3_endpoint = 'https://s3.lix.systems',
s3_endpoint = 'https://s3.afnix.fr',
s3_region = 'global',
s3_ssh_host = S3_HOST,
s3_ssh_user = S3_USER,
)
ENVIRONMENTS = {
@@ -1,12 +1,12 @@
# SPDX-FileCopyrightText: 2024 Jade Lovelace
# SPDX-FileCopyrightText: 2026 Yureka Lilian <yureka@cyberchaos.dev>
# SPDX-License-Identifier: MIT
import argparse
import json
import sys
import datetime
import dataclasses
import re
from typing import Any, Literal, Optional
from typing import Any
import requests
import os
import logging
@@ -14,27 +14,34 @@ import logging
log = logging.getLogger(__name__)
log.setLevel(logging.INFO)
fmt = logging.Formatter('{asctime} {levelname} {name}: {message}',
datefmt='%b %d %H:%M:%S',
style='{')
fmt = logging.Formatter(
"{asctime} {levelname} {name}: {message}",
datefmt="%b %d %H:%M:%S",
style="{",
)
if not any(isinstance(h, logging.StreamHandler) for h in log.handlers):
hand = logging.StreamHandler()
hand.setFormatter(fmt)
log.addHandler(hand)
API_BASE = os.environ.get('GARAGE_ADMIN_API_BASE', 'http://localhost:3903')
API_KEY = os.environ['GARAGE_ADMIN_TOKEN']
API_BASE = os.environ.get("GARAGE_ADMIN_API_BASE", "http://localhost:3903")
API_KEY = os.environ["GARAGE_ADMIN_TOKEN"]
BUCKET_REGEX_STR = os.environ.get("BUCKET_REGEX", ".*")
BUCKET_REGEX = re.compile(BUCKET_REGEX_STR)
def api(method, endpoint: str, resp_json=True, **kwargs) -> Any:
log.info('http %s %s', method, endpoint)
if not endpoint.startswith('https'):
log.info("http %s %s", method, endpoint)
if not endpoint.startswith("https"):
endpoint = API_BASE + endpoint
resp = requests.request(method,
endpoint,
headers={'Authorization': f'Bearer {API_KEY}'},
**kwargs)
resp = requests.request(
method,
endpoint,
headers={"Authorization": f"Bearer {API_KEY}"},
**kwargs,
)
resp.raise_for_status()
if resp_json:
return resp.json()
@@ -42,97 +49,64 @@ def api(method, endpoint: str, resp_json=True, **kwargs) -> Any:
return resp
@dataclasses.dataclass
class Key:
name: str
id: str
secret_key: Optional[str] = None
def get_bucket_id(bucket_name: str) -> str:
resp: dict = api(
"GET", "/v2/GetBucketInfo", params={"globalAlias": bucket_name}
)
return resp["id"]
@dataclasses.dataclass
class Bucket:
id: str
def keys() -> list[Key]:
data: list[dict] = api('GET', '/v1/key?list')
return [Key(name=k['name'], id=k['id']) for k in data]
def delete_key(key: Key):
api('DELETE', '/v1/key', resp_json=False, params={'id': key.id})
def create_key(name: str) -> Key:
resp: dict = api('POST', '/v1/key', json={'name': name})
return Key(name=resp['name'],
id=resp['accessKeyId'],
secret_key=resp['secretAccessKey'])
AccessType = Literal['read'] | Literal['write'] | Literal['owner']
def get_bucket(bucket_name: str) -> Bucket:
resp: dict = api('GET', '/v1/bucket', params={'globalAlias': bucket_name})
return Bucket(resp['id'])
def grant(bucket: Bucket, access_types: list[AccessType], key: Key):
access_types_dict = {k: True for k in access_types}
api('POST',
'/v1/bucket/allow',
json={
'bucketId': bucket.id,
'accessKeyId': key.id,
'permissions': access_types_dict,
})
KEY_RE = re.compile(r'^.*ephemeral-(\d{14})$')
DATEFMT = '%Y%m%d%H%M%S'
def expired_keys(older_than: datetime.datetime) -> list[Key]:
ret = []
for key in keys():
if m := KEY_RE.match(key.name):
date = datetime.datetime.strptime(m.group(1), DATEFMT)
date = date.astimezone(datetime.UTC)
print(date)
if date < older_than:
ret.append(key)
return ret
DATEFMT = "%Y%m%d%H%M%S"
def do_new(args):
buckets = [get_bucket(b) for b in args.buckets]
for b in args.buckets:
if not BUCKET_REGEX.match(b):
print(f"Bucket {b} not in allowed buckeds '{BUCKET_REGEX_STR}'")
exit(1)
bucket_ids = [get_bucket_id(b) for b in args.buckets]
def optional(s: str, whether) -> list[str]:
if whether:
return [s]
else:
return []
key_name = args.name + "-" if args.name else ""
expiration = datetime.datetime.now(tz=datetime.UTC) + datetime.timedelta(
seconds=args.age_secs
)
key_name += "ephemeral-" + expiration.strftime(DATEFMT)
access_types: list[AccessType] = optional('read', args.read) + optional(
'write', args.write) + optional('owner', args.owner) # type: ignore
key_resp: dict = api(
"POST",
"/v2/CreateKey",
json={
"name": key_name,
"expiration": expiration.isoformat(),
"neverExpires": False,
},
)
key_name = args.name + '-' if args.name else ''
key_name += "ephemeral-" + (
datetime.datetime.now(tz=datetime.UTC) +
datetime.timedelta(seconds=args.age_secs)).strftime(DATEFMT)
for b in bucket_ids:
api(
"POST",
"/v2/AllowBucketKey",
json={
"accessKeyId": key_resp["accessKeyId"],
"bucketId": b,
"permissions": {
"read": args.read,
"write": args.write,
"owner": args.owner,
},
},
)
k = create_key(key_name)
for b in buckets:
grant(b, access_types, k)
print(json.dumps(dataclasses.asdict(k), indent=2))
def do_clean(args):
older_than = datetime.datetime.now(tz=datetime.UTC)
for key in expired_keys(older_than):
delete_key(key)
print(
json.dumps(
{
"name": key_resp["name"],
"id": key_resp["accessKeyId"],
"secret_key": key_resp["secretAccessKey"],
},
indent=2,
)
)
def main():
@@ -148,28 +122,27 @@ def main():
new = sps.add_parser("new", help="Make an ephemeral key")
new.add_argument("--name", help="Name prefix for the key")
new.add_argument("--read",
action="store_true",
help="Grant read access to buckets")
new.add_argument("--write",
action="store_true",
help="Grant write access to buckets")
new.add_argument("--owner",
action="store_true",
help="Grant owner access to buckets")
new.add_argument("--age-secs",
type=int,
required=True,
help="Maximum key lifetime in seconds")
new.add_argument("buckets", nargs='*', help="Buckets to grant access to")
new.add_argument(
"--read", action="store_true", help="Grant read access to buckets"
)
new.add_argument(
"--write", action="store_true", help="Grant write access to buckets"
)
new.add_argument(
"--owner", action="store_true", help="Grant owner access to buckets"
)
new.add_argument(
"--age-secs",
type=int,
required=True,
help="Maximum key lifetime in seconds",
)
new.add_argument("buckets", nargs="*", help="Buckets to grant access to")
new.set_defaults(cmd=do_new)
clean = sps.add_parser("clean", help="Clean up old keys")
clean.set_defaults(cmd=do_clean)
args = ap.parse_args()
args.cmd(args)
if __name__ == '__main__':
if __name__ == "__main__":
main()
+1 -1
View File
@@ -14,7 +14,7 @@ def get_ephemeral_key(
env.docs_bucket.removeprefix('s3://'),
]
if env.s3_ssh_host is not None:
command = ['ssh', '-l', 'root', env.s3_ssh_host, *command]
command = ['ssh', f'{env.s3_ssh_user}@{env.s3_ssh_host}', *command]
output = subprocess.check_output(command)
d = json.loads(output.decode())
return environment.S3Credentials(name=d['name'],
+12
View File
@@ -15,6 +15,18 @@ rec {
'';
};
cycle-with-deps = mkDerivation {
name = "cycle-with-deps";
inherit dep;
outputs = [ "foo" "bar" ];
builder = builtins.toFile "builder.sh" ''
mkdir -p $foo/bin $bar/lib
ln -sf $dep $bar/lib
echo $foo > $bar/txt
echo $bar > $foo/txt
'';
};
as_dependency = mkDerivation {
name = "depends-on-cycle";
inherit cycle;
+11
View File
@@ -16,3 +16,14 @@ error="$(! nix-build check-outputs.nix -A as_dependency 2>&1)"
grepQuiet "cycle detected in build of '.*' in the references of output 'bar' from output 'foo'" <<<"$error"
grepQuiet "error: 1 dependencies of derivation" <<<"$error"
error="$(! nix-build check-outputs.nix -A cycle-with-deps 2>&1)"
grepQuiet "cycle detected in build of '.*' in the references of output 'bar' from output 'foo'" <<<"$error"
if [[ "$(uname -s)" = Linux ]]; then
echo "$error"
<<<"$error" grepQuiet "/store/.*-cycle-with-deps-bar"
<<<"$error" grepQuiet "└───txt: ….*cycle-with-deps-foo.*"
<<<"$error" grepQuiet " →.*/store/.*-cycle-with-deps-foo"
<<<"$error" grepQuiet " └───txt:.*-cycle-with-deps-bar.*"
fi
@@ -0,0 +1 @@
"234"
@@ -0,0 +1,5 @@
let
a = builtins.substring 1;
b = a 3;
in
b "1234567890"
+23
View File
@@ -521,4 +521,27 @@ INSTANTIATE_TEST_SUITE_P(
concat({header, make_directory({{"DE", make_file(false, "meow")}, {"de", make_file(false, "mrrp")}})})
))
);
TEST_F(NarTest, stringSizeLimit)
{
GeneratorSource source([]() -> Generator<Bytes> {
const char preamble[] =
"\x0d\x00\x00\x00\x00\x00\x00\x00nix-archive-1\x00\x00\x00"
"\x01\x00\x00\x00\x00\x00\x00\x00(\x00\x00\x00\x00\x00\x00\x00"
"\x04\x00\x00\x00\x00\x00\x00\x00type\x00\x00\x00\x00";
co_yield Bytes{preamble, sizeof(preamble) - 1};
// the nar parser keeps all strings in a buffer with the 8 byte length prefix in front.
// sufficiently large strings overflowed caused the buffer size calculation to overflow
// and thus allowed out-of-bounds writes in the daemon and potentially privesc to root.
co_yield Bytes{"\xf7\xff\xff\xff\xff\xff\xff\xff", 8};
// overflow would happen while reading data
while (true) {
co_yield Bytes{"foo-", 4};
}
}());
auto parser = nar::parse(source);
ASSERT_THROW(parser.next(), SerialisationError);
}
}
+2 -2
View File
@@ -1,5 +1,5 @@
{
"version": "2.94.0",
"official_release": false,
"version": "2.94.2",
"official_release": true,
"release_name": "Açaí na tigela"
}