Silly me forgot to add the dependency responsible for handling
`@pytest.mark.timeout` failsafes to kill a test if it takes too long,
which results in a warning when running f2.
This commit adds the required dependency and hence removes the warning
Change-Id: I281cad05bb0cf50208f72080cb2bfd8cbfb09d0c
Maintains compatibility with mdbook 0.4.x.
Includes comments for what to remove once 0.5.x is the only we care
about. Some other changes technically could be changed at that point,
but currently serve to enable universal support of 0.4.x and 0.5.x
Fixes#1051.
Change-Id: Ic5b405038d180bcd357bbd9e5716879e0c26e5f5
libarchive's xz offers single threaded xz compression which is very slow
and provides ~10-20Mbps compression speed in addition to maxing a core.
In exchange, it achieves optimal compression ratios among all our
compression methods.
Nonetheless, xz prevent the saturation of 1Gbps+ connections and slow
down significantly decompression for end users. As these connections and
faster hardware is becoming prevalent for cache servers and clients, we
offer to default to zstd.
Lix is a "compress once, decompress many times" application. To avoid
incurring a high penalty to end users very sensitive to compress ratio
(very slow Internet connections), we dampen the consequences of
switching to zstd by increasing the default zstd level to 12.
On one example, xz will compress a 4.4GB file to 632MB, zstd on 12 will
compress it to 775MB, that is a ~18 % increase over the optimal xz
compression. zstd took 18 seconds to produce this file.
Increasing to level 14 leads to a 773MB file while taking 37s.
Increasing to level 16 leads to 735MB file while taking 66s.
Finally, xz took 77s, so a 50 % reduction in time taken to compress in
exchange of an increase of 18 % of the compressed size.
This change will reduce issues encountered in #945 but is probably not
the root cause.
References:
- https://discourse.nixos.org/t/switch-cache-nixos-org-to-zstd-to-fix-slow-nixos-updates-nix-downloads/23961
Change-Id: I7beda2bf2c1fed146dcb797b8f85dc290c486ab2
Signed-off-by: Raito Bezarius <raito@lix.systems>
Without https://github.com/NixOS/nixpkgs/pull/434761 evaluation of the
`nixpkgsLibTests` will fail in CI with recent enough Lix, due to reliance on
the TOML integer saturation bug.
Reported-by: Sergei Zimmerman <sergei@zimmerman.foo>
Change-Id: I6a6a6964838009d2c525f67035f84072fdfad988
Previously two cryptography libraries were linked into Lix: OpenSSL used for
hashing and (in usual configurations) indirectly via curl for TLS, and Sodium
used only for handling the Ed25519 path info signatures. The latter is
functionally redundant since OpenSSL supports the same use case as well.
Reimplement the Ed25519 handling using OpenSSL and drop Sodium.
Fixes: https://git.lix.systems/lix-project/lix/issues/969
Change-Id: I6a6a696456b9d3ad7fdc2bf9b0759836a6247a38
currenlty we use the external package `toml`, this just adds an
unnessecary dependency, as python ships its own toml as `tomllib`
Change-Id: Ia63fa7558973e853ada20cbfa21d897d700444f8
This version changes the handling of TOML timestamps, and throws an
error on out‐of‐range integer literals rather than the previous
saturating behaviour, as required by [the TOML v1.0.0 specification]:
> Arbitrary 64-bit signed integers (from −2^63 to 2^63−1) should be
> accepted and handled losslessly. If an integer cannot be represented
> losslessly, an error must be thrown.
[the TOML v1.0.0 specification]: <https://toml.io/en/v1.0.0#integer>
The only known use of this is a questionable Nixpkgs test that I have
proposed [a fix] for.
[a fix]: <https://github.com/NixOS/nixpkgs/pull/433710>
Bumping this ahead of Nixpkgs ensures we can test these cases on
HEAD in advance. I presume that the next Lix major version will be
released after 25.05 goes out of support, so it should be fine to
drop support for the old version of toml11.
The co‐authors of this commit are the contributors to the vendored
package definition from Nixpkgs.
Co-authored-by: Anderson Torres <torres.anderson.85@protonmail.com>
Co-authored-by: Artturin <Artturin@artturin.com>
Co-authored-by: Silvan Mosberger <silvan.mosberger@moduscreate.com>
Change-Id: I6a6a69644a188b6e09eee5c9cf91ddd3c81d24ee
So far, the environment used by `command` was completely leaky and the
one used by `nix` was very leaky despite it trying to be a "hermetic"
environment.
This commit moves the hermaticity to `command` and changes its
implementation to be not leak anything.
To achieve this, the following changes were also nessecary:
- the `files` and `snapshot` fixture now use the folder `test-home`
within the tmp_path directory by default, as the `HOME` environment
variable is set to there. (extraction not possible due to dependencies
of command etc also using this directory)
Fixes: #847, #848
Change-Id: I55f86ee0e1615e73fcf442ee2f28f3b89893bbb4
use a thread pool and allow only buffered sources for performance. we
may want to use this code path unconditionally due to gc interactions
of fibers in circumstances we should not even be able to trigger, but
if that becomes important we will have a useful implementation ready.
Change-Id: Ib4e1531fe920847d8e30a42e8df393ace549f52e
We need capnproto-lix to be provided by callPackage, otherwise it won't
be spliced and the same (built for the cross host platform) for both
buildInputs (correct) and nativeBuildInputs (incorrect). We thus move it
into the overlay.
A similar problem exists for the lowdown build. We thus use callPackage
to override it as well. This is horrible (especially because it means we
need to pass the enableDarwinSandbox package argument through
ourselves), but at least it builds...
Fixes: https://git.lix.systems/lix-project/lix/issues/939
Change-Id: I802152072d852903401ef701f526195aa99475f2
`mesonCheckPhase` of course runs `preCheck` and `postCheck` hooks,
not `preInstallCheck`, so this was not doing the right thing. Really
sorry for breaking CI :(
Fixes: e6da29ad6b
Change-Id: I6a6a69642a242e2c8bfe10ac61d3f6756d666548
This gets us `--print-errorlogs` and `--timeout-multiplier=0` for free,
and also fixes the concatenation logic with `__structuredAttrs`.
Change-Id: I6a6a69643ea7224cb24508505c554143f83ae243
This is a collection of Lix plugins that showcase how to write one for
various usecases.
The first is a mTLS store plugin that enable mTLS cache URIs
(`https+mtls://`).
We enable meson build system support for this plugin but we are not
going to distribute it in the official packaging of Lix, we will
repackage each relevant plugin downstream in Nixpkgs.
These plugins have *NO* guarantee support, they are provided as useful
references and are possibly production-ready if your usecase is simple
enough.
Reference: https://github.com/NixOS/nix/pull/13030 (this change has
resemblances but our APIs are different, the tests harness is mostly
from CppNix).
Change-Id: Ib354271981b35dff6c134b12c4748c3eaf743fcb
Co-authored-by: Jörg Thalheim <joerg@thalheim.io>
Co-authored-by: László Vaskó <1771332+vlaci@users.noreply.github.com>
Signed-off-by: Raito Bezarius <raito@lix.systems>
Originally, libboost_context and dependent dynamic libraries have been copied
into the nix package to remove boost from the closure and consequently reduce
the closure size. Since commit ef0de7c79f we do
not depend on Boost coroutines any more, so these libraries are not needed at
all any more and (somewhat ironically) only increase the package size. Remove
them.
Change-Id: I6a6a6964dc3e0b29dfad8b2b232b428ba3cc653c
The issue requiring these environment variables to be set for Meson to find
Boost [1] has been fixed [2] for quite some time now. Drop them since they are
unnecessary in all supported nixpkgs releases (in fact they have already been
removed in the lix package there).
[1] https://github.com/NixOS/nixpkgs/issues/86131
[2] https://github.com/NixOS/nixpkgs/pull/315998
Change-Id: I6a6a69640a30e917cd7a983b9d846d023b393dcd
we need this to generate dependency information, and it'll be the entry
point for custom codegen once we need it. a wrapper also makes it a lot
easier to generate a whole namespace's worth of rpc definitions at once
Change-Id: Iba7a1c92a8a40bede9ed71aa3ab455477ff5e568
staging-next banned !structuredAttrs && separateDebugInfo && disallowedRequisites
due to weird output interactions. Enable structuredAttrs so we can build again.
Also, fix type confusion that makes stdenv explode (https://github.com/NixOS/nixpkgs/issues/422989).
Co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: Ic0c773394ee79e10d427f27750d59892d6d1f1d1
This allows using a userspace program, pasta, to handle comms between
the build sandbox, and the outside world; allowing for full isolation
including the network namespace, closing the "fixed-output derivation
talks to the host over an abstract domain socket" hole for good.
Fixes CVE-2025-46416.
Co-Authored-By: Puck Meerburg <puck@puckipedia.com>
Change-Id: Ifd499b7dbb3784600a6e842fede65fc031ff9f15
This is useful to reformat only changed hunks of a file via
`clang-format`.
Change-Id: I9aa8526d75fd2301113ee57f3a2e595f3b03504f
Signed-off-by: Raito Bezarius <raito@lix.systems>
This creates a framework similar to the old lang.sh from functional.
Some notable changes:
- instead of having a .flags file, a test.toml can declare flags
- additionally the test.toml can also declare extra files and multiple
runners for the given input file.
- there won't be any old tests hanging around anymore which weren't
deleted properly in the installation
- all files for a single test are defined decleratively and there won't
be any residues
Tests can be placed within the functional2/lang folder
most migrations should be rather clean
Implements: #825
Change-Id: I5f9149903ec5b078008969a4ae77305417c11475
Due to https://git.lix.systems/lix-project/lix/issues/832 , Lix 2.93.0
fails to build on Darwin without overrides. Until the root cause has
been determined and fixed, build without LTO.
Change-Id: I4db5eb294d8f19e5a366b1e19efa5a327b3e2e78
Ruff is used to enforce our code-style for the python parts of the
reposity, similar to clang-tidy for the cpp parts.
This includes a pre-commit hook to format code before it is committed
When "unfixable" - i.e. no autoformatting is available - the commit is
rejected
resolves#812
Change-Id: I6830c2fc29ae86337ec18f2b0e3565fac66c5523
USDT probes are statically defined trace points that have nearly zero
disabled-probe effect, i.e. we can put them in hot paths.
The use case for these is both similar and dissimilar to Rust tracing:
We still need better logging and a better structured rust-tracing
looking thing, but probes allow for quite easy programmable interactive
tracing in production, which we also care a lot about.
This CL comes with a perfunctory trace point in
libstore/file-transfer.cc for reading data out of the curl buffer. This
was mostly thrown in there so that I could see what the buffer sizes of
this were, and maybe be able to instrument the perf of the curl usages
in Lix in the future.
Fixes: https://git.lix.systems/lix-project/lix/issues/727
Change-Id: I0f5d9912d76bf3d6923bf53ebfd9b8d6c6e70aea
The code for serialization Expr nodes back into (pseudo-)Nix has been
removed for being subtly error-prone and tedious to maintain. Instead,
`nix-instantiate --parse` now prints a JSON representation of the AST.
Usage patterns of the --parse flag I've found in the wild:
1. Check if a file is well-formed, i.e. discard output and test exit code
2. Get parser errors from a file, i.e. discard stdout and use stderr
3. Nixfmt uses --parse to test equivalence pre/post format, and that property is (should be?) preserved
None of these should break with the current change
Closes#487
Change-Id: Icdbaad17790f2ad8765fa08e02e6597ee4c7a909
Lix requires a non-antiquated macOS SDK, and 24.11 does not yet have a
non-antiquated one as default.
Fixes: https://git.lix.systems/lix-project/lix/issues/588
Change-Id: Iad19c06d7fefe3a736cdcb39ced185e52dcfcbb8
nixpkgs 24.11 changes how we access xonsh yet again
and updates clang.
Unfortunately, clang 18 produces significantly more
warnings on existing code that is challenging to fix.
Make sure that doesn't error when we're running
`-Werror` builds.
n.b. I had to change the "SSL certificate problem: self-signed
certificate" to the old error prior to the improved libcurl errors,
since what is presumably a difference in which TLS library is used has
cropped up between releases? Either way the curl error buffer is empty.
Seems like we aggressively cannot do anything about this.
Change-Id: If0141a46a8b445a0e7d6f86f939e8c8e03569bf5
clangd broke because it can't look through symlinks. compile_commands
manipulation does not fix it, clangd configuration does not fix it, a
vfs overlay does not fix it, and while a combination of those can fix
it with a bind mount in place that's just too cursed to even consider
clangd bug: https://github.com/llvm/llvm-project/issues/116877
Change-Id: I8e3e8489548eb3a7aa65ac9d12a5ec8abf814aec
Currently, a bunch of documentation is generated by embedding parts of it in
the nix executable, getting it out again by running it, and then postprocessing
the output. This is bad, since it creates a pointless dependency of the
documentation on the executable, and also makes documentation generation
impossible when cross-compiling.
Instead, both the code and the documentation should be generated from data, see
https://git.lix.systems/lix-project/lix/issues/292 . Here we start applying
this approach to the experimental and deprecated features, which are done in
one go since the technical implementation is very similar.
Of course, the actual benefits are not realised yet, since the offending
pattern is used in several more places. These will be fixed later.
Change-Id: I4c802052cc7e865c61119a34b8f1063c4decc9cb
The devShell relied on several packages directly from `pkgs`
or used with a non-splice-aware functions.
These would be built for the host system, making them useless
in a devShell for the build system.
Make sure that all packages are for the build system when needed.
Some other minor changes also required:
* Make devShells use `clangStdenv` because GCC is currently broken
* Disable rr when making a cross stdenv
Change-Id: Iee5f8a1a0c594139a50f2261b203491bd1644866
This is done because the one in nixpkgs has several problems and we
don't want to conflict with them fixing those problems:
- not building shared libs
- not building debuginfo (more critical for us due to
https://git.lix.systems/lix-project/lix/issues/549)
- not setting the correct cmake build type
- not setting the correct cxxflags to build the coroutine library
- not building with clang (required for the coroutine library since gcc
is known to miscompile coroutines *in kj*).
CC: https://git.lix.systems/lix-project/lix/issues/551
Fixes: https://git.lix.systems/lix-project/lix/issues/550
Change-Id: Ia5b78dc1809963fdd1a8203b127a216cb575d751
This workaround was for the stdenv not being set when callPackage'ing
package.nix for some of the stranger CI outputs.
Change-Id: I2acdd6efa721b90dd3cb04358544d25d591ff084
This failed due to https://github.com/NixOS/nixpkgs/pull/346945, which
makes a second lowdown-unsandboxed that works in nix builds, and the
regular lowdown has executables that fail closed when the sandbox setup
fails.
The actual failure here is only visible on nixos-unstable at the moment,
not 24.05, but this commit should fix it up for all versions.
Fixes: https://git.lix.systems/lix-project/lix/issues/547
Change-Id: I50c0ecb59518ef01a7c0181114c1b4c5a7c6b78b
Tom Hubrecht