Compare commits

...
10 Commits
Author SHA1 Message Date
Jade Lovelace 533429d89a release: 2.94.1 "Açaí na tigela"
Release produced with releng/create_release.xsh

Change-Id: Ie38032fe8dfebed83f9d24762ce48f73b23149b5
2026-03-13 09:39:06 -07:00
Jade Lovelace c135090468 release: release notes for 2.94.1
Release created with releng/create_release.xsh

Change-Id: I970e8f5422e3705917b04c09d28498fd7b849d27
2026-03-13 09:38:52 -07:00
Jade Lovelace 72f7965679 2.94.1: version
Change-Id: I92eeeebfd6b966ce77833785482db989962f8c9e
2026-03-13 09:38:44 -07:00
Raito Bezarius b7be40c785 libstore/build: fix starvation during substitution
When the destructor of PathSubstitutionGoal is run, this happens in a
sync context and can cause starvation of all ongoing IO w.r.t. to other
substitutions, including our own substitution.

While there's only a decompressor thread per stream, the other side of
the IO runs on the event loop.

In order to fix this, it is sufficient to remove the thread indirection
and inline the async code.

Fixes #1126. Great thanks to horrors' patience.

Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: I3eb37bc37d156f0f5528364e568fdaa2ced58011
Signed-off-by: Raito Bezarius <raito@lix.systems>
(cherry picked from commit 505d0669dc)
2026-02-11 15:39:06 +00:00
Qyriad 6350f51458 libutil: include LIX_MAJOR, LIX_MINOR, and LIX_PATCH macros
Backport of I7d8a4648890fce7ff15695876c9b9d3a6a6a6964 to 2.94 branch.

Change-Id: Ia6b8d89007974c8cf873fa1f536ce8416a6a6964
2026-01-15 11:48:26 +00:00
eldritch horrorsandRaito Bezarius c6f3f3a0d3 libexpr: fix app chain extension
during the value rewrite we accidentally broke extension of incomplete
primop application. this only shows up when binding on incomplete call
to a primop to a name, binding an incomplete call to *that* to another
name, and then finally calling the second binding with enough args for
a complete primop application. since this only shows up when calling a
primop with three or more args it took a while to surface. we have few
builtins that match this: foldl', replaceStrings, and substring. these
are not used incompletely in this manner very often, so this lingered.

fixes #1102

Change-Id: I218dffc14ae876efc86a86c7eb6c895e2405201c
2026-01-14 22:09:56 +00:00
Maximilian Bosch de4cfec46a libstore: fix reporting output cycles on drvs with references to other drvs
Closes #1064

The culprit here is that `genGraphString` is only invoked with the
store-paths associated with the outputs of the derivation, so when
filling `dependents`, the `graph_data.find(p)` call would return the end
of the iterator when doing this for references to other store-paths.

As a result, the code wrote information behind the graph data-structure
causing a corruption. For me, this resulted in a SIGSEGV most of the
time and in a few cases in an uncaught `map::at`-exception as reported
by Niklas.

This patch changes two aspects of the original implementation:

* When filling `dependents` in the node-set, use `map.at()` instead of
  `map.find()->second`. The latter doesn't make any sense and was the
  cause of corrupting memory. The `at` would've made it far easier to
  spot this in the first place.

* Filter out store-paths that don't belong to a different output of the
  derivation when creating `outputGraph`. This variable is used on two
  places, `genGraphString` and for topological sorting.

  The latter already filters out store-paths from a different drv, so
  this is happening now when creating the variable in the first place
  such that `genGraphString` never ends up with corrupt data in the
  first place. This is the actual bugfix.

Implemented a regression-test for this case to be sure.

Change-Id: Ie02144d89c32b0a776cb1ece0601d0229315ebc3
(cherry picked from commit 0a5f474a25)
2025-12-08 07:20:51 +01:00
Raito Bezarius 0873bed39d nix/upgrade-nix: disallow daemon connections for the store
Prior to I6a6a6964d2b5ad47ae5ea9eb11af9b6373ce2141 — `sudo nix
upgrade-nix` would perform direct store access.

This ensured a certain number of desireable properties for upgrading the
Lix binary itself.

We re-introduce direct store access for upgrading Lix binaries.

Fixes #1060.

Change-Id: I523c4d3023ed5fe9eff8fde9a266c56a0de47d8c
Signed-off-by: Raito Bezarius <raito@lix.systems>
(cherry picked from commit d2ca1810b1)
2025-12-06 22:20:18 +00:00
Zoe ZuserandRaito Bezarius 5dcb90548f meson: fix libstore.pc
typo of aws-cpp-sdk-transfer as aws-cpp-std-transfer prevents linking
against lix

Change-Id: Id847eab2601698696030d31fcd51288aa5f3d274
(cherry picked from commit 06f987fb0c)
2025-12-05 16:58:28 +00:00
Jade Lovelace 54118efaf6 release: merge release 2.94.0 back to mainline
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh

Change-Id: Ida5959473ec469dd5e5e977ebdc8aab6e694b0de
2025-11-17 11:42:20 -08:00
22 changed files with 142 additions and 113 deletions
+3
View File
@@ -1,4 +1,7 @@
# Lix 2.94 "Açaí na tigela" (2025-11-17)
# Lix 2.94.1 (2026-03-13)
# Lix 2.94.0 (2025-11-17)
+13 -3
View File
@@ -20,11 +20,21 @@ inline Value::Value(app_t, EvalMemory & mem, Value & lhs, Value & rhs)
}
inline Value::Value(app_t, EvalMemory & mem, Value & lhs, std::span<Value> args)
: Value(app_t{}, mem, lhs, args, {})
{
auto app = static_cast<Value::App *>(mem.allocBytes(sizeof(Value::App) + args.size_bytes()));
}
inline Value::Value(
app_t, EvalMemory & mem, const Value & lhs, std::span<Value> baseArgs, std::span<Value> moreArgs
)
{
auto app = static_cast<Value::App *>(
mem.allocBytes(sizeof(Value::App) + baseArgs.size_bytes() + moreArgs.size_bytes())
);
app->_left = lhs;
app->_n = args.size();
std::copy(args.begin(), args.end(), app->_args);
app->_n = baseArgs.size() + moreArgs.size();
std::copy(baseArgs.begin(), baseArgs.end(), app->_args);
std::copy(moreArgs.begin(), moreArgs.end(), app->_args + baseArgs.size());
raw = tag(tApp, app);
}
+8 -1
View File
@@ -1639,7 +1639,14 @@ void EvalState::callFunction(Value & fun, std::span<Value> args, Value & vRes, c
Value vCur(fun);
auto makeAppChain = [&]() { vRes = {NewValueAs::app, ctx.mem, vCur, args}; };
auto makeAppChain = [&]() {
if (vCur.isApp()) {
auto & app = vCur.app();
vRes = {NewValueAs::app, ctx.mem, app.left(), app.args(), args};
} else {
vRes = {NewValueAs::app, ctx.mem, vCur, args};
}
};
const Attr * functor;
+4
View File
@@ -544,6 +544,10 @@ public:
/// lazy and/or partial application of a function.
Value(app_t, EvalMemory & mem, Value & lhs, std::span<Value> args);
/// Constructs a nix language value of type "lambda", which represents a
/// lazy and/or partial application of a function.
Value(app_t, EvalMemory & mem, const Value & lhs, std::span<Value> baseArgs, std::span<Value> moreArgs);
/// Constructs a nix language value of type "external", which is only used
/// by plugins. Do any existing plugins even use this mechanism?
Value(external_t, ExternalValueBase & external)
+8 -1
View File
@@ -139,7 +139,14 @@ Goal::WorkResult DerivationGoal::timedOut(Error && ex)
kj::Promise<Result<Goal::WorkResult>> DerivationGoal::workImpl() noexcept
{
KJ_DEFER({ actLock.reset(); });
// always clear the slot token, no matter what happens. not doing this
// can cause builds to get stuck on exceptions (or other early exits).
// ideally we'd use scoped slot tokens instead of keeping them in some
// goal member variable, but we cannot do this yet for legacy reasons.
KJ_DEFER({
actLock.reset();
slotToken = {};
});
BOOST_OUTCOME_CO_TRY(auto result, co_await (useDerivation ? getDerivation() : haveDerivation()));
result.storePath = drvPath;
+3 -1
View File
@@ -246,7 +246,7 @@ struct DerivationGoal : public Goal
WorkResult timedOut(Error && ex);
kj::Promise<Result<WorkResult>> workImpl() noexcept override;
kj::Promise<Result<WorkResult>> workImpl() noexcept override final;
/**
* Add wanted outputs to an already existing derivation goal.
@@ -307,6 +307,8 @@ struct DerivationGoal : public Goal
virtual void cleanupPostOutputsRegisteredModeNonCheck();
protected:
AsyncSemaphore::Token slotToken;
kj::TimePoint lastChildActivity = kj::minValue;
kj::Promise<Result<WorkResult>> wrapChildHandler(kj::Promise<Result<WorkResult>> handler
-6
View File
@@ -22,12 +22,6 @@ kj::Promise<void> Goal::waitForAWhile()
kj::Promise<Result<Goal::WorkResult>> Goal::work() noexcept
try {
// always clear the slot token, no matter what happens. not doing this
// can cause builds to get stuck on exceptions (or other early exist).
// ideally we'd use scoped slot tokens instead of keeping them in some
// goal member variable, but we cannot do this yet for legacy reasons.
KJ_DEFER({ slotToken = {}; });
BOOST_OUTCOME_CO_TRY(auto result, co_await workImpl());
trace("done");
-3
View File
@@ -82,9 +82,6 @@ struct Goal
*/
std::string name;
protected:
AsyncSemaphore::Token slotToken;
public:
struct [[nodiscard]] WorkResult {
ExitCode exitCode;
+9 -8
View File
@@ -1825,15 +1825,18 @@ try {
);
}
outputGraph[scratchOutputs.at(name)] = StorePathSet{};
std::visit(
overloaded{/* Since we'll use the already installed versions of these, we
can treat them as leaves and ignore any references they
have. */
[&](const AlreadyRegistered &) {
outputGraph[scratchOutputs.at(name)] = StorePathSet{};
},
[&](const AlreadyRegistered &) {},
[&](const PerhapsNeedToRegister & refs) {
outputGraph[scratchOutputs.at(name)] = refs.refs;
for (auto & ref : refs.refs) {
if (inverseOutputMap.find(ref) != inverseOutputMap.end()) {
outputGraph[scratchOutputs.at(name)].insert(ref);
}
}
}
},
*orifu
@@ -1844,10 +1847,8 @@ try {
topoSort(outputsToSort, {[&](const std::string & name) {
StringSet dependencies;
for (auto & path : outputGraph.at(scratchOutputs.at(name))) {
auto outputName = inverseOutputMap.find(path);
if (outputName != inverseOutputMap.end()) {
dependencies.insert(outputName->second);
}
auto outputName = inverseOutputMap.at(path);
dependencies.insert(outputName);
}
return dependencies;
}});
+39 -78
View File
@@ -27,13 +27,6 @@ PathSubstitutionGoal::PathSubstitutionGoal(
maintainExpectedSubstitutions = worker.expectedSubstitutions.addTemporarily(1);
}
PathSubstitutionGoal::~PathSubstitutionGoal()
{
cleanup();
}
Goal::WorkResult PathSubstitutionGoal::done(
ExitCode result,
BuildResult::Status status,
@@ -76,8 +69,6 @@ kj::Promise<Result<Goal::WorkResult>> PathSubstitutionGoal::tryNext() noexcept
try {
trace("trying next substituter");
cleanup();
if (subs.size() == 0) {
/* None left. Terminate this goal and let someone else deal
with it. */
@@ -205,61 +196,36 @@ kj::Promise<Result<Goal::WorkResult>> PathSubstitutionGoal::tryToRun() noexcept
try {
trace("trying to run");
if (!slotToken.valid()) {
slotToken = co_await worker.substitutions.acquire();
}
maintainRunningSubstitutions = worker.runningSubstitutions.addTemporarily(1);
auto pipe = kj::newPromiseAndCrossThreadFulfiller<void>();
outPipe = kj::mv(pipe.fulfiller);
thr = std::async(std::launch::async, [this]() {
AsyncIoRoot aio;
/* Wake up the worker loop when we're done. */
Finally updateStats([this]() { outPipe->fulfill(); });
auto & fetchPath = subPath ? *subPath : storePath;
try {
ReceiveInterrupts receiveInterrupts;
auto act = logger->startActivity(
actSubstitute, Logger::Fields{worker.store.printStorePath(storePath), sub->getUri()}
);
aio.blockOn(copyStorePath(
*sub,
worker.store,
fetchPath,
repair,
sub->config().isTrusted ? NoCheckSigs : CheckSigs,
&act
));
} catch (const EndOfFile &) {
throw EndOfFile(
"NAR for '%s' fetched from '%s' is incomplete",
sub->printStorePath(fetchPath),
sub->getUri()
);
}
});
co_await pipe.promise;
co_return co_await finished();
} catch (...) {
co_return result::current_exception();
}
kj::Promise<Result<Goal::WorkResult>> PathSubstitutionGoal::finished() noexcept
try {
trace("substitute finished");
auto & fetchPath = subPath ? *subPath : storePath;
do {
try {
slotToken = {};
thr.get();
break;
try {
AsyncSemaphore::Token slotToken = co_await worker.substitutions.acquire();
auto act = logger->startActivity(
actSubstitute,
Logger::Fields{worker.store.printStorePath(storePath), sub->getUri()}
);
maintainRunningSubstitutions = worker.runningSubstitutions.addTemporarily(1);
TRY_AWAIT(copyStorePath(
*sub,
worker.store,
fetchPath,
repair,
sub->config().isTrusted ? NoCheckSigs : CheckSigs,
&act
));
break;
} catch (const EndOfFile &) {
throw EndOfFile(
"NAR for '%s' fetched from '%s' is incomplete",
sub->printStorePath(fetchPath),
sub->getUri()
);
}
} catch (std::exception & e) { // NOLINT(lix-foreign-exceptions)
printError("%1%", Uncolored(e.what()));
@@ -271,10 +237,20 @@ try {
substituterFailed = true;
}
}
/* Try the next substitute. */
/* Try the next substitute */
co_return co_await tryNext();
} while (false);
co_return co_await finished();
} catch (...) {
co_return result::current_exception();
}
kj::Promise<Result<Goal::WorkResult>> PathSubstitutionGoal::finished() noexcept
try {
trace("substitute finished");
worker.markContentsGood(storePath);
printMsg(lvlChatty, "substitution of path '%s' succeeded", worker.store.printStorePath(storePath));
@@ -294,19 +270,4 @@ try {
} catch (...) {
co_return result::current_exception();
}
void PathSubstitutionGoal::cleanup()
{
try {
if (thr.valid()) {
// FIXME: signal worker thread to quit.
thr.get();
}
} catch (...) {
ignoreExceptionInDestructor();
}
}
}
-9
View File
@@ -48,11 +48,6 @@ struct PathSubstitutionGoal : public Goal
*/
kj::Own<kj::CrossThreadPromiseFulfiller<void>> outPipe;
/**
* The substituter thread.
*/
std::future<void> thr;
/**
* Whether to try to repair a valid path.
*/
@@ -85,7 +80,6 @@ public:
RepairFlag repair = NoRepair,
std::optional<ContentAddress> ca = std::nullopt
);
~PathSubstitutionGoal();
kj::Promise<Result<WorkResult>> workImpl() noexcept override;
@@ -97,9 +91,6 @@ public:
kj::Promise<Result<WorkResult>> tryToRun() noexcept;
kj::Promise<Result<WorkResult>> finished() noexcept;
/* Called by destructor, can't be overridden */
void cleanup() override final;
JobCategory jobCategory() const override {
return JobCategory::Substitution;
};
+1 -1
View File
@@ -471,6 +471,6 @@ configure_file(
'libdir' : libdir,
'includedir' : includedir,
'PACKAGE_VERSION' : meson.project_version(),
'AWS_SDK_IF_FOUND' : aws_sdk.found() ? 'aws-cpp-sdk-core aws-cpp-sdk-s3 aws-cpp-std-transfer' : '',
'AWS_SDK_IF_FOUND' : aws_sdk.found() ? 'aws-cpp-sdk-core aws-cpp-sdk-s3 aws-cpp-sdk-transfer' : '',
},
)
+1 -1
View File
@@ -265,7 +265,7 @@ static std::map<StorePath, Node> mkGraph(
for (auto & node : graph_data) {
for (auto & ref : node.second.dependencies) {
graph_data.find(ref)->second.dependents.insert(node.first);
graph_data.at(ref).dependents.insert(node.first);
}
}
+10
View File
@@ -1,7 +1,17 @@
# Cursed, but I don't think there's another way to get this environment variable.
lix_suffix = run_command('bash', '-c', 'echo -n "$VERSION_SUFFIX"', check : true).stdout().strip()
lix_version_parts = meson.project_version().split('.')
lix_major = lix_version_parts[0]
lix_minor = lix_version_parts[1]
lix_patch = lix_version_parts[2].replace(lix_suffix, '')
config_h = configure_file(
configuration : {
'PACKAGE_NAME': '"' + meson.project_name() + '"',
'PACKAGE_VERSION': '"' + meson.project_version() + '"',
'LIX_MAJOR': lix_major,
'LIX_MINOR': lix_minor,
'LIX_PATCH': lix_patch,
'PACKAGE_TARNAME': '"' + meson.project_name() + '"',
'PACKAGE_STRING': '"' + meson.project_name() + ' ' + meson.project_version() + '"',
'HAVE_STRUCT_DIRENT_D_TYPE': 1, # FIXME: actually check this for solaris
+13
View File
@@ -49,6 +49,19 @@ struct CmdUpgradeNix : MixDryRun, EvalCommand
});
}
// NOTE(Raito): we override the store creation
// to prevent any store daemon connection.
//
// An upgrade, by nature, requires a direct store access
// to avoid having the daemon die in the middle of changing the binary.
//
// If more commands needs that, we can move it into a mixin. This was deliberately not done
// here.
virtual ref<Store> createStore(AsyncIoRoot & aio) override
{
return aio.blockOn(openStore(settings.storeUri.get(), {}, AllowDaemon::Disallow));
}
/**
* This command is stable before the others
*/
+12
View File
@@ -15,6 +15,18 @@ rec {
'';
};
cycle-with-deps = mkDerivation {
name = "cycle-with-deps";
inherit dep;
outputs = [ "foo" "bar" ];
builder = builtins.toFile "builder.sh" ''
mkdir -p $foo/bin $bar/lib
ln -sf $dep $bar/lib
echo $foo > $bar/txt
echo $bar > $foo/txt
'';
};
as_dependency = mkDerivation {
name = "depends-on-cycle";
inherit cycle;
+11
View File
@@ -16,3 +16,14 @@ error="$(! nix-build check-outputs.nix -A as_dependency 2>&1)"
grepQuiet "cycle detected in build of '.*' in the references of output 'bar' from output 'foo'" <<<"$error"
grepQuiet "error: 1 dependencies of derivation" <<<"$error"
error="$(! nix-build check-outputs.nix -A cycle-with-deps 2>&1)"
grepQuiet "cycle detected in build of '.*' in the references of output 'bar' from output 'foo'" <<<"$error"
if [[ "$(uname -s)" = Linux ]]; then
echo "$error"
<<<"$error" grepQuiet "/store/.*-cycle-with-deps-bar"
<<<"$error" grepQuiet "└───txt: ….*cycle-with-deps-foo.*"
<<<"$error" grepQuiet " →.*/store/.*-cycle-with-deps-foo"
<<<"$error" grepQuiet " └───txt:.*-cycle-with-deps-bar.*"
fi
@@ -0,0 +1 @@
"234"
@@ -0,0 +1,5 @@
let
a = builtins.substring 1;
b = a 3;
in
b "1234567890"
+1 -1
View File
@@ -1,5 +1,5 @@
{
"version": "2.94.0",
"version": "2.94.1",
"official_release": true,
"release_name": "Açaí na tigela"
}