lowdown 1.4.0 changed the lowdown_opts to include a new and separate
lowdown_opts_term which allows for configuring values specific to
-Tterm (which we're using). This version should have been called 2.0.0
according to semver, hence 2.0.0 was released later without any actual
breaking changes to sort of migitate the problem.
We need to support lowdown >= 1.3 && < 1.4 since the ship has sailed for
updating lowdown in NixOS 25.05 as well as lowdown >= 1.4 or we'll be
stuck in Nixpkgs forever. Support for < 1.4 can be dropped as soon as
NixOS 25.05 is EOL, assuming this change lands before NixOS 25.11
branch-off.
We detect the changed API based on the lowdown version from pkg-config
and define LOWDOWN_SEPARATE_TERM_OPTS based on that. The ifdef is named
according to the specific API change that impacts us, so that it's
hopefully a little simpler to maintain going forward. In the new API,
all newly configurable settings use what would have been the (implicit)
default before. Changing some of these values, especially hpadding,
could be interesting in future changes.
Compared to cl/3081, this change makes sure to initialize all new fields
of lowdown_opts_term explicitly.
It seems that, while making -Tterm more configurable, lowdown's word
wrapping behavior changed slightly which broke basic_repl.test. I've
chosen to work around this by using builtins.add as an example which has
a very short documentation string, so wrapping doesn't matter.
Change-Id: Id73be4c0e43d7eb4f56e10a261b4254402698ff8
we need this to generate dependency information, and it'll be the entry
point for custom codegen once we need it. a wrapper also makes it a lot
easier to generate a whole namespace's worth of rpc definitions at once
Change-Id: Iba7a1c92a8a40bede9ed71aa3ab455477ff5e568
This allows using a userspace program, pasta, to handle comms between
the build sandbox, and the outside world; allowing for full isolation
including the network namespace, closing the "fixed-output derivation
talks to the host over an abstract domain socket" hole for good.
Fixes CVE-2025-46416.
Co-Authored-By: Puck Meerburg <puck@puckipedia.com>
Change-Id: Ifd499b7dbb3784600a6e842fede65fc031ff9f15
This is useful for certain error recovery paths (no pun intended) that
does not thread through the original path name.
Change-Id: I2d800740cb4f9912e64c923120d3f977c58ccb7e
Signed-off-by: Raito Bezarius <raito@lix.systems>
This fixes Meson's "Project does not target a minimum version but uses
feature introduced in '1.1': meson.options file" warning.
Silly Meson.
I also added a note in the top-level meson.build to indicate
`meson_version` is specified in more than one place.
Change-Id: I2c04278bb46a562a1c96cd2e5e4d9ce59ce8e125
allow opting in to serialization as integers via a trait type instead,
and add string-list serializers for the feature flag set enumerations.
fixes#738
Change-Id: I2746eb5ef1f15c01b4e681f9ba1615b6c6e64f44
USDT probes are statically defined trace points that have nearly zero
disabled-probe effect, i.e. we can put them in hot paths.
The use case for these is both similar and dissimilar to Rust tracing:
We still need better logging and a better structured rust-tracing
looking thing, but probes allow for quite easy programmable interactive
tracing in production, which we also care a lot about.
This CL comes with a perfunctory trace point in
libstore/file-transfer.cc for reading data out of the curl buffer. This
was mostly thrown in there so that I could see what the buffer sizes of
this were, and maybe be able to instrument the perf of the curl usages
in Lix in the future.
Fixes: https://git.lix.systems/lix-project/lix/issues/727
Change-Id: I0f5d9912d76bf3d6923bf53ebfd9b8d6c6e70aea
This is required for moving n-e-j into tree with good meson support: it
needs to be able to include `<lix/config.h>`.
Change-Id: I21f1cfa37dcdce992812c8b008df503c0fc9dea5
currently this header only contains TRY_AWAIT, and it's public version
LIX_TRY_AWAIT. the former is for use in our cpp files only, the latter
may be used anywhere. we don't expose TRY_AWAIT unconditionally due to
it not being namespaced according to usual rules of macro naming, i.e.
exposing it may litter user macro namespaces unnecessarily. hiding our
internal version is achieved using a new define that's not set through
pkg-config or other means. (we could also add a private header that we
simply do not install, but that gets rather messy rather very quickly)
Change-Id: I372b1a414487dd03cfbfc620b3148a6c6e56ca3f
nixpkgs 24.11 changes how we access xonsh yet again
and updates clang.
Unfortunately, clang 18 produces significantly more
warnings on existing code that is challenging to fix.
Make sure that doesn't error when we're running
`-Werror` builds.
n.b. I had to change the "SSL certificate problem: self-signed
certificate" to the old error prior to the improved libcurl errors,
since what is presumably a difference in which TLS library is used has
cropped up between releases? Either way the curl error buffer is empty.
Seems like we aggressively cannot do anything about this.
Change-Id: If0141a46a8b445a0e7d6f86f939e8c8e03569bf5
clangd broke because it can't look through symlinks. compile_commands
manipulation does not fix it, clangd configuration does not fix it, a
vfs overlay does not fix it, and while a combination of those can fix
it with a bind mount in place that's just too cursed to even consider
clangd bug: https://github.com/llvm/llvm-project/issues/116877
Change-Id: I8e3e8489548eb3a7aa65ac9d12a5ec8abf814aec
It's only used in a couple of tests, and only in such a way that
replacing it with a random command suffices.
I also removed a few pointless uses of the variable.
Fixes: https://git.lix.systems/lix-project/lix/issues/376
Change-Id: I90aedb61d64b02f7c9b007e72f9d614cc1b37a2e
I am tired of bad shell scripts, let me write bad python quickly
instead. It's definitely, $100%, better.
This is not planned as an immediate replacement of the old test suite,
but we::jade would not oppose tests getting ported.
What is here is a mere starting point and there is a lot more
functionality that we need.
Fixes: https://git.lix.systems/lix-project/lix/issues/488
Change-Id: If762efce69030bb667491b263b874c36024bf7b6
I followed @pennae's advice and moved the constructor definition of
`AttrName` from the header file `nixexpr.hh` to `nixexpr.cc`.
Change-Id: I733f56c25635b366b11ba332ccec38dd7444e793
The approach that was taken here was to add default values to the type
definitions rather than specify them whenever they are missing.
Now the only remaining warning is '-Wunused-parameter' which @jade said
is usually counterproductive and that we can just disable it:
https://git.lix.systems/lix-project/lix/issues/456#issuecomment-6617
So this change adds the flags '-Wall', '-Wextra' and
'-Wno-unused-parameter', so that all warnings are enabled except for
'-Wunused-parameter'.
Change-Id: Ic223a964d67ab429e8da804c0721ba5e25d53012
We have manually enabled libstdc++ assertions since cl/797. Meson 1.4.0
(the minimum version we mandate) enables this by default, so we can
remove the enabling compiler flag from the list of project arguments.
With this patch, `-D_GLIBCXX_ASSERTIONS=1` is still present in the
compile command logs when building with both gccStdenv and clangStdenv.
See: https://gerrit.lix.systems/c/lix/+/797
See: https://mesonbuild.com/Release-notes-for-1-4-0.html#ndebug-setting-now-controls-c-stdlib-assertions
Change-Id: I53483fadfe5cbd11ba35544b437d3a9ee8031631
This was already the de facto requirement, we use the method `full_path`
on a file object (introduced in Meson 1.4.0) in the functional test
suite's build.
This version of Meson is in NixOS 24.05, so there should be no
compatibility issues should this make it into a backported release of
Lix.
CC: https://git.lix.systems/lix-project/lix/issues/247
Change-Id: I5c640824807353b6eb4287e7ed09c4e89a4bdde2
Using `configure_file` to copy files has been deprecated since Meson 0.64.0.
The intended replacement is the `fs.copyfile` method.
This removes the following deprecation warning that arises when a minimum
Meson version is specified:
``
Project [...] uses feature deprecated since '0.64.0': copy arg in configure_file. Use fs.copyfile instead
``
Change-Id: I09ffc92e96311ef9ed594343a0a16d51e74b114a
while gcc 12 and older miscompile our generators, gcc 13 and older
outright crash on kj coroutines. (newer gcc versions may fix this)
Change-Id: I19f12c8c147239680eb0fa5a84ef5c7de38c9263
using a proper event loop basis we no longer have to worry about most of
the intricacies of poll(), or platform-dependent replacements for it. we
may even be able to use the event loop and its promise system for all of
our scheduling in the future. we don't do any real async processing yet,
this is just preparation to separate the first such change from the huge
api design difference with the async framework we chose (kj from capnp):
kj::Promise, unlike std::future, doesn't return exceptions unmangled. it
instead wraps any non-kj exception into a kj exception, erasing all type
information and preserving mostly the what() string in the process. this
makes sense in the capnp rpc use case where unrestricted exception types
can't be transferred, and since it moves error handling styles closer to
a world we'd actually like there's no harm in doing it only here for now
Change-Id: I20f888de74d525fb2db36ca30ebba4bcfe9cc838
This took parsing time from 1421s or so to 1060s or so. The reason is
entirely nlohmann. All of the stuff below is just Obliterated because it's
built in the PCH instead:
**** Templates that took longest to instantiate:
219051 ms: nlohmann::basic_json<>::parse<const char *> (276 times, avg 793 ms)
169675 ms: nlohmann::basic_json<>::basic_json (1127 times, avg 150 ms)
129416 ms: nlohmann::detail::parser<nlohmann::basic_json<>, nlohmann::detail::i... (276 times, avg 468 ms)
98155 ms: nlohmann::detail::parser<nlohmann::basic_json<>, nlohmann::detail::i... (276 times, avg 355 ms)
81322 ms: nlohmann::basic_json<>::json_value::json_value (1405 times, avg 57 ms)
53531 ms: nlohmann::detail::json_sax_dom_callback_parser<nlohmann::basic_json<... (276 times, avg 193 ms)
clang-only. This brings the clang build time to not far from *half* of
the gcc build time.
Also, clang does not enjoy so much to miscompile coroutines. Maybe we
should just be clang-only.
Change-Id: Id3135db0094e4560830674090e32e6da2c22fcc6
nixpkgs delivered us the untimely gift of a meson 1.5 upgrade, which
*does* make our lives easier by allowing us to delete wrap generation
code, but it does so at the cost of renaming all rust crates in such a
way that the wrap logic cannot tolerate the new names on the old meson
version 😭.
It also means that support burden for this is going to be atrocious
until we either give in and vendor meson 1.5 or we make a CI target for
it. Neither seems appealing, though the latter is not super absurd for
ensuring we don't break nixpkgs unstable.
This commit causes meson 1.5 to ignore the .wrap files in subprojects/
entirely (since they have the wrong names lol) and instead use
Cargo.lock, so it now hard-depends on our workspace reshuffling
improvement.
It also deletes the hack that we were using to get the sources of Cargo
deps into meson by using a feature that went unnoticed when this code
was originally written: MESON_PACKAGE_CACHE_DIR:
https://github.com/mesonbuild/meson/blob/8a202de6ec763284cbb7160b9d43d5e7e0703f19/mesonbuild/wrap/wrap.py#L490-L502
Change-Id: I7a28f12fc2812c6ed7537b60bc3025c141a05874
This is purely to let Cargo's dependency resolver do stuff for us, we do
not actually intend to build this stuff with Cargo to begin with.
Change-Id: I4c08d55595c7c27b7096375022581e1e34308a87
This is required to make more meson stuff easier/possible, and honestly
it *is* now Lix sources anyhow.
Change-Id: Ia6c38fabce9aa5c53768745ee38c5cf344f5c226
lix-doc is now built with Meson, with lix-doc's dependencies built as
Meson subprojects, either fetched on demand with .wrap files, or fetched
in advance by Nix with importCargoLock. It even builds statically.
Fixes#256.
Co-authored-by: Lunaphied <lunaphied@lunaphied.me>
Co-authored-by: Jade Lovelace <lix@jade.fyi>
Change-Id: I3a4731ff13278e7117e0316bc0d7169e85f5eb0c
The target_machine variable is meant for the target
of cross compilers. We are not a cross compiler, so
instead reuse our host_machine based checks.
Fixes Linux→FreeBSD cross, since Meson can't figure
out `target_machine.kernel()` in that case.
Fixes: https://git.lix.systems/lix-project/lix/issues/469
Change-Id: Ia46a64c8d507c3b08987a1de1eda171ff5e50df4
Seems a little bit Rich that musl does not implement close_range because
they suspect that the system call itself is a bad idea, so they uhhhh
are considering not implementing a wrapper. Let's just fix the problem
at hand by writing our own wrapper.
Change-Id: I1f8e5858e4561d58a5450503d9c4585aded2b216
I don't know why but I was getting a spurious -Werror=switch-enum inside
toml11. It does not make sense why it did not occur before, but it
should be stopped.
This was not done at an earlier stage to better match the legacy make
build system, but we don't use it anyway.
Change-Id: I636f8a71e8a0ba5e0feb80b435ae24c3af995c5d
The principle of this is that you can either externally build it with
Nix (actual implementation will be in a future commit), or it can be
built with meson if the Nix one is not passed in.
The idea I have is that dev shells don't receive the one from Nix to
avoid having to build it, but CI can use the one from Nix and save some
gratuitous rebuilds.
The design of this is that you can run `ninja -C build clang-tidy` and
it will simply correctly clang-tidy the codebase in spite of PCH
bullshit caused by the cc-wrapper.
This is a truly horrendous number of hacks in a ball, caused by bugs in
several pieces of software, and I am not even getting started.
I don't consider this to fix the clang-tidy issue filing, since we still
have a fair number of issues to fix even on the existing minimal
configuration, and I have not yet implemented it in CI. Realistically we
will need to do something like https://github.com/Ericsson/codechecker
to be able to silence warnings without physically touching the code, or
at least *diff* reports between versions.
Also, the run-clang-tidy output design is rather atrocious and must
not be inflicted upon anyone I have respect for, since it buries the
diagnostics in a pile of invocation logs. We would do really well to
integrate with the Gerrit SARIF stuff so we can dump the reports on
people in a user-friendly manner.
Related: https://git.lix.systems/lix-project/lix/issues/147
Change-Id: Ifefe533f3b56874795de231667046b2da6ff2461
What if you could find memory bugs in Lix without really trying very
hard? I've had variously scuffed patches to do this, but this is
blocked on boost coroutines removal at this point tbh.
Change-Id: Id762af076aa06ad51e77a6c17ed10275929ed578
Use libprocstat to find garbage collector roots on FreeBSD.
Tested working on a FreeBSD machine, although there is no CI yet
Change-Id: Id36bac8c3de6cc4de94e2d76e9663dd4b76068a9
*accidentally* overriding a function is almost guaranteed to be an
error. overriding a function without labeling it as such is merely
bad style, but bad style that makes the code harder to understand.
Change-Id: Ic0594f3d1604ab6b3c1a75cb5facc246effe45f0
this will be the basis of non-boost coroutines in lix. anything that is
a boost coroutine *should* be representable with a Generator coroutine,
and many things that are not currently boost coroutines but behave much
like one (such as, notably, serializers) should be as well. this allows
us to greatly simplify many things that look like iteration but aren't.
Change-Id: I2cebcefa0148b631fb30df4c8cfa92167a407e34
we do not have any of these warnings appearing at the moment, but it
seems like a good idea to enable [[nodiscard]] checking anyway. once
we start introducing more functions with must-use conditions we will
need such checking, and the rust stdlib has proven them very useful.
Change-Id: Ibb6b042ae1ec5f527f8dc2809a7816a4c1548ae2
this gives about 20% performance improvements on pure parsing. obviously
it will be less on full eval, but depending on how much parsing is to be
done (e.g. including hackage-packages.nix or not) it's more like 4%-10%.
this has been tested (with thousands of core hours of fuzzing) to ensure
that the ASTs produced by the new parser are exactly the same as the old
one would have produced. error messages will change (sometimes by a lot)
and are not yet perfect, but we would rather leave this as is for later.
test results for running only the parser (excluding the variable binding
code) in a tight loop with inputs and parameters as given are promising:
- 40% faster on lix's package.nix at 10000 iterations
- 1.3% faster on nixpkgs all-packages.nix at 1000 iterations
- equivalent on all of nixpkgs concatenated at 100 iterations
(excluding invalid files, each file surrounded with parens)
more realistic benchmarks are somewhere in between the extremes, parsing
once again getting the largest uplift. other realistic workloads improve
by a few percentage points as well, notably system builds are 4% faster.
Benchmarks summary (from ./bench/summarize.jq bench/bench-*.json)
old/bin/nix --extra-experimental-features 'nix-command flakes' eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
mean: 0.408s ± 0.025s
user: 0.355s | system: 0.033s
median: 0.389s
range: 0.388s ... 0.442s
relative: 1
new/bin/nix --extra-experimental-features 'nix-command flakes' eval -f bench/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
mean: 0.332s ± 0.024s
user: 0.279s | system: 0.033s
median: 0.314s
range: 0.313s ... 0.361s
relative: 0.814
---
old/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
mean: 6.133s ± 0.022s
user: 5.395s | system: 0.437s
median: 6.128s
range: 6.099s ... 6.183s
relative: 1
new/bin/nix --extra-experimental-features 'nix-command flakes' eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
mean: 5.925s ± 0.025s
user: 5.176s | system: 0.456s
median: 5.934s
range: 5.861s ... 5.943s
relative: 0.966
---
GC_INITIAL_HEAP_SIZE=10g old/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
mean: 4.503s ± 0.027s
user: 3.731s | system: 0.547s
median: 4.499s
range: 4.478s ... 4.541s
relative: 1
GC_INITIAL_HEAP_SIZE=10g new/bin/nix eval --extra-experimental-features 'nix-command flakes' --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
mean: 4.285s ± 0.031s
user: 3.504s | system: 0.571s
median: 4.281s
range: 4.221s ... 4.328s
relative: 0.951
---
old/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello
mean: 16.475s ± 0.07s
user: 14.088s | system: 1.572s
median: 16.495s
range: 16.351s ... 16.536s
relative: 1
new/bin/nix --extra-experimental-features 'nix-command flakes' search --no-eval-cache github:nixos/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 hello
mean: 15.973s ± 0.013s
user: 13.558s | system: 1.615s
median: 15.973s
range: 15.946s ... 15.99s
relative: 0.97
---
Change-Id: Ie66ec2d045dec964632c6541e25f8f0797319ee2
This is really just a question of turning off the production sanitizer
configuration so we get nice diagnostics. Not much else to say.
Change-Id: I76bd6d225320056ed95bd89955f00beff2db0d2f
-- message from cl/1418 --
The boehmgc changes are bundled into this commit because doing otherwise
would require an annoying dance of "adding compatibility for < 8.2.6 and
>= 8.2.6" then updating the pin then removing the (now unneeded)
compatibility. It doesn't seem worth the trouble to me given the low
complexity of said changes.
Rebased coroutine-sp-fallback.diff patch taken from https://github.com/NixOS/nixpkgs/pull/317227
-- jade resubmit changes --
This is a resubmission of https://gerrit.lix.systems/c/lix/+/1418, which
was reverted in https://gerrit.lix.systems/c/lix/+/1432 for breaking CI
evaluation without being detected.
I have run `nix flake check -Lv` on this one before submission and it
passes on my machine and crucially without eval errors, so the CI result
should be accurate.
It seems like someone renamed forbiddenDependenciesRegex to
forbiddenDependenciesRegexes in nixpkgs and also changed the type
incompatibly. That's pretty silly, but at least it's just an eval error.
Also, `xonsh` regressed the availability of `xonsh-unwrapped`, but it
was fixed by us in https://github.com/NixOS/nixpkgs/pull/317636, which
is now in our channel, so we update nixpkgs compared to the original
iteration of this to simply get that.
We originally had a regression related to some reorganization of the
nixpkgs lib test suite in which there was broken parameter passing.
This, too, we got quickfixed in nixpkgs, so we don't need any changes
for it: https://github.com/NixOS/nixpkgs/pull/317772
Related: https://gerrit.lix.systems/c/lix/+/1428
Fixes: https://git.lix.systems/lix-project/lix/issues/385
Change-Id: I26d41ea826fec900ebcad0f82a727feb6bcd28f3
This reverts commit 28a079f841.
Reason for revert: This caused a pile of regressions in CI, and does not pass nix flake check. Some number of them are fixed in CL: https://gerrit.lix.systems/c/lix/+/1429 but there's more to be fixed.
We should defer this after 2.90.
Change-Id: Ib839d0fcb08eb52094af2b521145e3c1b4e0556f
The boehmgc changes are bundled into this commit because doing otherwise
would require an annoying dance of "adding compatibility for < 8.2.6 and
>= 8.2.6" then updating the pin then removing the (now unneeded)
compatibility. It doesn't seem worth the trouble to me given the low
complexity of said changes.
Rebased coroutine-sp-fallback.diff patch taken from https://github.com/NixOS/nixpkgs/pull/317227
Change-Id: I8c590e9fe25c0f566d0cfeacb96d8cf50abf12e8
sternenseemann