if a substituter is entirely offline and cannot be queries at all we
should not be failing if other substituters are configured. likewise
if a substituter goes offline after querying but before we try using
it we should attempt to fetch that path from some other substituter.
ideally we'd treat all substituters as a single entity instead of as
one store each, then have that single entity take care of fallbacks,
retries, error reporting, etc. that requires larger rewrites though.
fixes#1061
Change-Id: I9d8fc0544ff380bf017256e8fcc82823dc634f10
In Nixpkgs, there are several strings like "\d\.\d" which attempt to be
a regex but are just literally "d.d". The escaping rules are silly and
we should warn our users about that.
Co-authored-by: Commentator2.0 <lix@crystal-cavern.systems>
Change-Id: I779b0757358fc9adc34dc140e1670b83abc93b67
The indentation stripping semantics of strings are fairly bad and have a
few gotchas where they behave unintuitively. But the good news is, that
these cases are easy to catch and can be avoided.
This commit adds a warning in the parser when such strings are detected.
Unfortunately Nixpkgs uses this kind of a lot, so we won't be able to
actually enable this warning for a while to come.
Co-authored-by: Commentator2.0 <lix@crystal-cavern.systems>
Change-Id: I3b3b68c2eee4cd70959d3f4ca643cb6caf3a2217
Unfortunately, the previous code only detected superficial __overrides
and did not catch cases like `rec { __overrides.foo = 2; }`.
Change-Id: Ic38bdef1a6a3bdea91915aef44447f4ecc238259
xattrs are revealing some unfortunate brittleness in real world
derivations that are getting -EINVAL errors while executing their test
code.
The reason for this is that Linux checks [1] UID delegations for xattrs
writes.
In the Lix sandbox, except if you enabled a uid-range feature, you have
exactly 3 UIDs: root, nixbld and nobody.
If your test code makes use of UIDs which have not been delegated, you
will receive an EINVAL on your operation. Test code is not resilient
with respect to the number of available UIDs in their namespace.
To avoid further issues for end users who are running into spurious
derivation build failures, we forbid xattrs again for now.
For more information about the plans, please consult or chime in [2].
Fixes#1105. Reopens#838. Fixes#1103.
[1]: https://elixir.bootlin.com/linux/v6.18.6/source/fs/posix_acl.c#L257
[2]: https://zulip.lix.systems/#narrow/channel/9-Store/topic/disablement.20of.20xattrs/with/5295
Change-Id: I864066b34cd8319d2271fac1b179cb4f950d836e
Signed-off-by: Raito Bezarius <raito@lix.systems>
this will let us configure more than one socket to connect/bind to,
which in turn lets us use posix acls on sockets for access control.
we will also need something like this for the final rpc transition.
Change-Id: I9c39f14906e9bf809055ab5c94bf687745b4f69e
See: cl/4840
When importing Python modules, we include `functional2` in the module
path, like this:
from functional2.testlib.fixtures.env import ManagedEnv
This means that python expects to see a file like
`functional2/testlib/fixtures/env.py`. We run `pytest` from `tests/` in
the `justfile` and have `tests/functional2/__init__.py` so `pytest` in
`meson` is able to find these imports.
However, language servers generally consider the `pyproject.toml` to be
the project root, so (e.g.) `pyright` is unable to follow any of the
`functional2` imports, leading to lots of spurious errors.
In cl/4840 I moved `tests/functional2/pyproject.toml` to
`tests/pyproject.toml`, which worked but was considered aesthetically
unappealing.
This diff is much larger but it's a more elegant solution.
Change-Id: I2983c7b87f88f59a4e3521451a9f5acd6a6a6964
Fixes#838.
xattrs were historically disabled in 2017, since then, NFS v4.2 gained
support for xattrs and almost all filesystems support xattrs. In
addition, moving the build directory to a xattrs-supporting filesystem
is always an option. Programs may exploit xattrs (including ACLs) during
their build process and testing phase, to better serve these programs,
we will remove this limitation.
Lix will use POSIX ACLs for its UNIX domain sockets in the future and be
one of these programs which will run tests making use of xattrs, while
not writing any xattrs in the derivation outputs themselves.
xattrs are still scrubbed from derivation outputs so it is not possible
to obtain an advantage by writing a security-related xattr to a well
chosen file.
Tests were added to test these scenarios on Linux. Darwin is carefully
excluded because of #1008 and #1090, solving that is welcome. In the
meantime, they are marked xfail.
Change-Id: Ia3255eeb8442e83db4f10dcb5a51cbc368a2550d
Signed-off-by: Raito Bezarius <raito@lix.systems>
one would expect the lang tests to be selectable (using `-k`) using
`eval_okay` `eval_fail` etc though this was not the case so far.
This commit renames the functions to reflect the expected names
Change-Id: I4cd340c093d774feeb4d96820d28e49c665b159f
during the value rewrite we accidentally broke extension of incomplete
primop application. this only shows up when binding on incomplete call
to a primop to a name, binding an incomplete call to *that* to another
name, and then finally calling the second binding with enough args for
a complete primop application. since this only shows up when calling a
primop with three or more args it took a while to surface. we have few
builtins that match this: foldl', replaceStrings, and substring. these
are not used incompletely in this manner very often, so this lingered.
fixes#1102
Change-Id: I218dffc14ae876efc86a86c7eb6c895e2405201c
When --check is ran with no output, this is seen as having *SOME*
invalid outputs, actually, ALL of them are invalid here.
Instead of reporting a confusing error, let's consider ALL of outputs
being invalid to be no output at all available and advise the user to
rerun the build without --check.
If there's *some* (>0) invalid outputs, let's print them exactly to give
a chance to the user to remediate this.
Fixes#485.
Change-Id: I00955ef9ea4f129e2c98d68c73b1e981f90278a0
Signed-off-by: Raito Bezarius <raito@lix.systems>
`build_env` is assumed to be a certain trivial environment, e.g. a
singleton search path.
An environment in general is either empty, contain one or more search
paths separated by a colon.
It seems that the intent was to set the `build_shell` first as a
prepended path in the managed environment then extend via the
`build_env` parts.
This fixes a usecase when the `BUILD_TEST_ENV` is non-trivial.
Change-Id: If5b8ab976d867a57ce0b8d29255f64695e30a8b2
Signed-off-by: Raito Bezarius <raito@lix.systems>
Certain evaluation requires to pass impurity arguments, add `flags`
to add more arguments to the instantiation.
Change-Id: I538ca2ff12c07cc66e845b42f81baa87f11b8f0f
Signed-off-by: Raito Bezarius <raito@lix.systems>
Is there a situation where `home` can be None? This hinders the ability
to use it quickly without type gating in the test code itself.
Change-Id: Idd739ddad67e10d6f7a6aad283c0e3b7cd08106a
Signed-off-by: Raito Bezarius <raito@lix.systems>
nlohmann has customization points for exception throws. we can use these
instead and wrap json exceptions at the source instead of playing a game
of whack-a-mole with json errors all over the tree. since nlohmann needs
macros set to achieve this we can no longer precompile its headers *and*
must forbid including it anywhere without the proper defines or ordering
of include directives will break lix with ODR violations, if we see them
at all before we get another json-related bug report. a new lint will be
in charge of ensuring this doesn't happen. we also re-allow direct calls
to nlohmann json parsing since error handling is no longer a problem. we
will keep the wrapper for more convenient error context handling though.
fixes#1092
Change-Id: I54ecc14f5bec5e2177729b41c3703216e76cc6a3
Status quo: We have `force$Type` and `eval$Type`, both which first
produce a value and then do a type check. The type checking logic is not
consistently implemented, with lots of code duplication.
This change does:
- Introduce new `check*` functions which unify the logic (the error
handling unfortunately still needs some duplication for now)
- Make both `force*` and `eval*` use the `check*` function for the
actual type checking
- Inline and dismantle the `eval*` functions for being of little use and
little used. This makes the `ExprOp*::eval` implementations for binary
logic operators more verbose, but IMO that's a good thing: The
implementation now needs to be a lot more explicit about the
short-circuiting semantics, something which was previously hidden behind
the short-circuiting semantics of the C++ language, in a way that could
easily be overlooked and lead to confusion, which is something that
happened to me twice in a year.
- Changes `forceAttrs` and `forceList` to include the context in case
`forceValue` fails (compared to only when the type check fails). This
was done for code consistency, because I could not find any reason why
list and attrs had different semantics here than int, float and bool. So
far the visible change is minimal (see the diff on the err.exp), however
this needs vetting for potential performance regressions.
Change-Id: I33e5c706d46850c9e1126293ee01dab85ba07587
I was pretty confused by this when I saw it, not realizing that `build`
meant `tests/functional2/build` and not the top-level Meson `build`
directory.
Change-Id: I4865f196e5bf029419c6b488e78cbfb46a6a6964
...and add both a static bash and busybox into the sandbox for building:
$ /nix/store/fz43jxs5qfg5vldzk38y2hmrcl58qk18-busybox-1.36.1/bin/sh -c 'declare -A foo'
/nix/store/fz43jxs5qfg5vldzk38y2hmrcl58qk18-busybox-1.36.1/bin/sh: declare: not found
The reason is that busybox's `sh` doesn't know about `declare` which
means it cannot build derivations with `__structuredAttrs = true;`.
Change-Id: Ie3ca431f862fd3e59dc649582704ae739ee5834a
Nearly each tests gets a fresh store (in the test root residing under TMPDIR),
which with default settings means 8 MiB of reserved space to be deleted by the
user in case of space shortages, which of course for a test suite is just
wasted. Stop doing that to considerably reduce the litter (experimentally, from
3.7 GiB to 67 MiB per run). The old functional test suite already did the same.
Change-Id: I912cdda6e796ea37cfee0b8fe0478e976a6a6964
this is only needed on macos, and only for tests that build anything. on
linux it can actually *break* stuff due to unfortunate interactions with
store url parsing, which is required to work for any remote build tests.
Change-Id: Ic6f7e090f15e129fc365d7edc56cdbc1a5686047
apparently this file slipped through the code review (the actual test
does exist, no worries) but this empty file and init py exist, sitting
here empty. This commit drops them as they serve no purpose
see commit 1b7ad3a7ad
Change-Id: I64c4e03cd3ceedf9e430ae014127966bd8e4269d
eldritch horrors