meson fixed the bugs that prevented using the sanitizer args, but using
such a fixed meson will provide broken flags to the subprojects. *sigh*
Change-Id: I0ab54351b19a3879e9a97daed1621dc5f730d376
single-user builds have a writable home directory in the build sandbox,
and the test suite does not like cargo writing anything into there. the
impact of this seems to be localized to f1 (at least on linux), but not
letting cargo write into the homedir is easy enough to do. nixpkgs also
does something much like this when compiling rustc, so we're not alone.
(it also seems that nixpkgs intended to do this for all rustc users but
broke it at some point in the past. some packages set CARGO_HOME in the
same way we do, possibly because they've had exactly the same problems)
fixes#1251
Change-Id: Ie17a0677ef09bb076ef6295bb590a0895924434e
mimalloc is a compact general purpose allocator from Microsoft. It
consistently outperforms glibc's `malloc()` in allocation-heavy
workloads, such as Lix's evaluator
It's currently only linked in the main `nix` executable, as Boehm's GC
uses its own allocator. Other allocations that *do* go through glibc's
`malloc()` are still much faster, though
Benchmarked on x86_64-linux against Nixpkgs `bd07873`:
| attribute | thunks | lix@`1396012` | mimalloc | uplift |
|-------------------|---------|---------------|----------|--------|
| hello | 206444 | 0.726s | 0.620s | 1.17x |
| chromium | 1177382 | 2.273s | 2.136s | 1.06x |
| firefox-unwrapped | 1394797 | 2.521s | 2.378s | 1.06x |
| texliveFull | 3186440 | 4.992s | 4.672s | 1.07x |
| nixosTests.gnome | 7905808 | 6.115s | 5.723s | 1.07x |
Based-on: https://github.com/NixOS/nix/pull/15596
Co-authored-by: Bernardo Meurer Costa <beme@anthropic.com>
Change-Id: I3ad92eacc075efeaf3d9f7730ada7b29835536a4
25.11 is dead, and so is mdbook 0.4
Also reenable linkcheck which was doing nothing this whole time lmao
Change-Id: I4a8b9c763b881de840d6ef1b1d85bf19941f4386
Since d0190cff6f, the `justfile` uses
`$out` from the environment as an argument to `--prefix` in its `meson`
invocation.
The `$out` variable is... A bit spicy to use since its semantics change
depending on whether `nix develop` or `nix-shell` is used.
In a `nix develop` shell, a directory *relative to PWD* (and not
relative to the Nix expression) is exported. In a `nix-shell`, the
`$out` used comes from the derivation.
The alternative to this change is to make the `justfile` use a variable
name that would not conflict with a `nix-shell` and `nix develop` shell.
Since it wants to use that value for the `outdir` variable, this would
be the name I would pick first.
I chose to instead work around the issue within the `nix-shell` since
this would be breaking the existing interface. As tenuous as it is.
Change-Id: I64cd5fd85345457e14880cb133e9ddafd55a25da
rust-analyzer binaries that don't come from nix (such as the one
shipped with vscode's rust-analyzer extension by default) need the
`RUST_SRC_PATH` env var to be the path to rustc's sources, but this
wasn't set inside the devshell. this change explicitely adds it to
the environment. (trying to add pkgs.rustLibSrc and rustcSrc to
the shell's packages unfortunately does nothing, because they are
just pure source packages, nothing more.)
(the reason this probably didn't come up for other devs is because
1. a lot of lix devs use other editors that probably just use the
binary from the path, and the devshell installs pkgs.rust-analyzer,
which is patched to always have `RUST_SRC_PATH` set, and 2. because
pkgs.vscode-extensions.rust-lang.rust-analyzer is also patched to use
pkgs.rust-analyzer by default instead of the bundled one, so installing
the extension using nix instead of from the marketplace would also hide
this problem.)
Change-Id: Ia27daf2a88756311ac5485ea0af44aae242241a9
This is a second attempt at https://gerrit.lix.systems/c/lix/+/5516
instead of letting meson try to be cargo we'll just have cargo at home.
this requires some contortions to link everything together due to quite
a few meson deficiencies, but at the end we get to pretend that rust is
just c++ painted orange. we'll use cxx to bring real interop back soon.
also fixes#1230
also reverts 10845bfe63
co-authored-by: eldritch horrors <pennae@lix.systems>
Change-Id: Ib1fda843fa80d818705d5a65ec9054216a6a6964
Full warning was:
warning: Dependency of package 'lix-shell-env' uses a nested list in attribute 'nativeBuildInputs'.
This is deprecated as of Nixpkgs release 26.05, and support will be removed in a future nixpkgs release.
Change-Id: I6b6c7425e1b03d3238ea21138cb3f2cf6a6a6964
This input was added in cl/2884, but was unused even then. The
`linuxPackages` within `buildPackages` is used instead.
Change-Id: I71c522ef683aa098eac0b356b22007ba6a6a6964
- launch-builder-linux.cc was missing an include for musl
and used function that are not defined in the launchers
- musl caches pids used for raise, breaking sandbox setup
- the mtls contrib plugin won't build, didn't try fixing,
static builds can't really use plugins reliably anyway.
Change-Id: I5ab1664e45ea977e5bcf05e41d825e6014e62146
mdbook-linkcheck was dropped from nixpkgs and replaced with mdbook-linkcheck2,
which is not entirely a drop-in replacement, but drop-in enough for us.
Change-Id: Iab03efab89ebae70811e0de3ea7cf0a9a059a9e4
`nix.full_path()` is the *built* path, not the *installed* path. in the
sandbox this point to a location that contains *only* `nix`, not any of
the old-cli symlinks. any test that needs them in path would thus fail.
Change-Id: Ia651db8d1151a84f8d1b52e8993f6db61b0fe104
this requires some hacks to make error reports visible, but hey. it's
better than no progress reports at all, and successful runs no longer
spew huge amounts of useless log output onto the developer's terminal
Change-Id: I9e4766b2f825a4ec451e117eb4609db65f328785
this will let us migrate the fork+do_stuff combo of builtin builders we
have today to fork+exec of this new command. we use a subcommand rather
than a simple libexec helper because we would like to avoid linking all
of lix into the simple libexec helpers if possible. this is only hookup
for the builders, none of the buiders are migrated here to ease review.
Change-Id: I9358f1f3fee6ca640c81a7bd865128ae0d0e35a5
Adds an example plugin implementing an mTLS-enabled binary cache store
(https+mtls:// scheme) using client certificates for authentication.
Darwin fix: don't link liblix* into plugins (host resolves symbols at
runtime via dynamic_lookup). Explicitly link curl so it binds to
Nix-store libcurl, not /usr/lib/libcurl. This prevents the plugin's
curl_easy_setopt calls from operating on the wrong libcurl instance.
Test portability: BSD sed -i wrapper, OpenSSL -sha256 for cert signing,
redirect test server output to log file.
Change-Id: I652b987d3ac45e31df50ff4ba1f523294438c2b6
Fixes#838.
xattrs were historically disabled in 2017, since then, NFS v4.2 gained
support for xattrs and almost all filesystems support xattrs. In
addition, moving the build directory to a xattrs-supporting filesystem
is always an option. Programs may exploit xattrs (including ACLs) during
their build process and testing phase, to better serve these programs,
we will remove this limitation.
Lix will use POSIX ACLs for its UNIX domain sockets in the future and be
one of these programs which will run tests making use of xattrs, while
not writing any xattrs in the derivation outputs themselves.
xattrs are still scrubbed from derivation outputs so it is not possible
to obtain an advantage by writing a security-related xattr to a well
chosen file.
Tests were added to test these scenarios on Linux. Darwin is carefully
excluded because of #1008 and #1090, solving that is welcome. In the
meantime, they are marked xfail.
Change-Id: Ia3255eeb8442e83db4f10dcb5a51cbc368a2550d
Signed-off-by: Raito Bezarius <raito@lix.systems>
I don't know how this was missed or no one really complained about it,
but perl-bindings was using CppNix if nix was not overlaid to our own
Lix.
If you overlay Lix manually, the resulting Lix's perl-bindings cannot
really refer to Lix itself, therefore, it's CppNix.
Change-Id: Ie5ce95564cb7fb5444acbea6124619c5121e1576
Signed-off-by: Raito Bezarius <raito@lix.systems>
This makes that the default for all builds... which I'm not sure is the
right option, but it is also very cheap to do so, and adding an entire
other Lix build just for static libraries (not to be confused with
static Lix) would be prohibitively expensive.
Change-Id: Id173c77cfa62886724378ca599d21c8c6a6a6964
cl/4796 introduced a bug where running `just test-functional2` would
hang forever on macOS but `meson test -C build --suite installcheck
functional2` would succeed because of a mismatch between the
`$BUILD_TEST_SHELL` environment variable and the `-Dbuild-test-shell`
build option set in `mesonFlags`.
We can move these definitions to `finalAttrs.buildTestShell` and similar
and use those values in _both_ `mesonFlags` and `env` in order to make
them consistent in all cases.
See: https://gerrit.lix.systems/c/lix/+/4796/comments/50def5b4_fa5671f2
Change-Id: If300bbe46d6269ace29b44156fb4a5196a6a6964
...and add both a static bash and busybox into the sandbox for building:
$ /nix/store/fz43jxs5qfg5vldzk38y2hmrcl58qk18-busybox-1.36.1/bin/sh -c 'declare -A foo'
/nix/store/fz43jxs5qfg5vldzk38y2hmrcl58qk18-busybox-1.36.1/bin/sh: declare: not found
The reason is that busybox's `sh` doesn't know about `declare` which
means it cannot build derivations with `__structuredAttrs = true;`.
Change-Id: Ie3ca431f862fd3e59dc649582704ae739ee5834a
A friend reported that this is rebuilding, which doesn't make any sense
because nixpkgs also has a lix from nightly that surely has the same
dependency and thus it surely should be built on hydra, right?
Turns out no, since they were overriding the requiredSystemFeatures to
remove big-parallel. Silly, but we can match that and get cache hits.
Change-Id: I4b1c8e850c6c38b03881354e978f41a10592ec95
Silly me forgot to add the dependency responsible for handling
`@pytest.mark.timeout` failsafes to kill a test if it takes too long,
which results in a warning when running f2.
This commit adds the required dependency and hence removes the warning
Change-Id: I281cad05bb0cf50208f72080cb2bfd8cbfb09d0c
Maintains compatibility with mdbook 0.4.x.
Includes comments for what to remove once 0.5.x is the only we care
about. Some other changes technically could be changed at that point,
but currently serve to enable universal support of 0.4.x and 0.5.x
Fixes#1051.
Change-Id: Ic5b405038d180bcd357bbd9e5716879e0c26e5f5
libarchive's xz offers single threaded xz compression which is very slow
and provides ~10-20Mbps compression speed in addition to maxing a core.
In exchange, it achieves optimal compression ratios among all our
compression methods.
Nonetheless, xz prevent the saturation of 1Gbps+ connections and slow
down significantly decompression for end users. As these connections and
faster hardware is becoming prevalent for cache servers and clients, we
offer to default to zstd.
Lix is a "compress once, decompress many times" application. To avoid
incurring a high penalty to end users very sensitive to compress ratio
(very slow Internet connections), we dampen the consequences of
switching to zstd by increasing the default zstd level to 12.
On one example, xz will compress a 4.4GB file to 632MB, zstd on 12 will
compress it to 775MB, that is a ~18 % increase over the optimal xz
compression. zstd took 18 seconds to produce this file.
Increasing to level 14 leads to a 773MB file while taking 37s.
Increasing to level 16 leads to 735MB file while taking 66s.
Finally, xz took 77s, so a 50 % reduction in time taken to compress in
exchange of an increase of 18 % of the compressed size.
This change will reduce issues encountered in #945 but is probably not
the root cause.
References:
- https://discourse.nixos.org/t/switch-cache-nixos-org-to-zstd-to-fix-slow-nixos-updates-nix-downloads/23961
Change-Id: I7beda2bf2c1fed146dcb797b8f85dc290c486ab2
Signed-off-by: Raito Bezarius <raito@lix.systems>
Without https://github.com/NixOS/nixpkgs/pull/434761 evaluation of the
`nixpkgsLibTests` will fail in CI with recent enough Lix, due to reliance on
the TOML integer saturation bug.
Reported-by: Sergei Zimmerman <sergei@zimmerman.foo>
Change-Id: I6a6a6964838009d2c525f67035f84072fdfad988
Previously two cryptography libraries were linked into Lix: OpenSSL used for
hashing and (in usual configurations) indirectly via curl for TLS, and Sodium
used only for handling the Ed25519 path info signatures. The latter is
functionally redundant since OpenSSL supports the same use case as well.
Reimplement the Ed25519 handling using OpenSSL and drop Sodium.
Fixes: https://git.lix.systems/lix-project/lix/issues/969
Change-Id: I6a6a696456b9d3ad7fdc2bf9b0759836a6247a38
eldritch horrors