Commit Graph
9918 Commits
Author SHA1 Message Date
eldritch horrorsandRaito Bezarius faa50e15c8 libutil: make RunningProgram more useful
make it moveable, make it killable, and add a stdout fd accessor.

Change-Id: I2387cbe8ac67b899a322cd6c7d306ef9ea7abcd0
2025-06-24 10:50:21 +00:00
Raito Bezarius 18a50a1734 libutil: ensure that _deletePath does NOT use absolute paths with dirfds
When calling `_deletePath` with a parent file descriptor, `openat` is
made effective by using relative paths to the directory file descriptor.

To avoid the problem, the signature is changed to resist misuse with an
assert in the prologue of the function.

Fixes CVE-2025-46415.

Change-Id: I6b3fc766bad2afe54dc27d47d1df3873e188de96
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-24 10:50:21 +00:00
Raito Bezarius d27c247715 libstore: ensure that passAsFile is created in the original temp dir
This ensures that `passAsFile` data is created inside the expected
temporary build directory by `openat()` from the parent directory file
descriptor.

Fixes CVE-2025-52993.

Change-Id: Ie5273446c4a19403088d0389ae8e3f473af8879a
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-24 10:46:21 +00:00
Raito Bezarius bda2f174e4 libutil: writeFile variant for file descriptors
`writeFile` lose its `sync` boolean flag to make things simpler.

A new `writeFileAndSync` function is created and all call sites are
converted to it.

Change-Id: Ib871a5283a9c047db1e4fe48a241506e4aab9192
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-19 19:57:58 +02:00
Raito Bezarius 36327a3b2d libstore: chown to builder variant for file descriptors
We use it immediately for the build temporary directory.

Change-Id: I180193c63a2b98721f5fb8e542c4e39c099bb947
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-19 19:57:58 +02:00
Raito Bezarius d61d1c16e5 libstore: open build directory as a dirfd as well
We now keep around a proper AutoCloseFD around the temporary directory
which we plan to use for openat operations and avoiding the build
directory being swapped out while we are doing something else.

Change-Id: I18d387b0f123ebf2d20c6405cd47ebadc5505f2a
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-19 19:57:58 +02:00
Raito Bezarius 0848a16cce libutil: guess or invent a path from file descriptors
This is useful for certain error recovery paths (no pun intended) that
does not thread through the original path name.

Change-Id: I2d800740cb4f9912e64c923120d3f977c58ccb7e
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-19 19:57:58 +02:00
Puck MeerburgandJade Lovelace c89ceb1669 Fix passing custom CA files into the builtin:fetchurl sandbox
Without this, verifying TLS certificates would fail on macOS, as well
as any system that doesn't have a certificate file at /etc/ssl/certs/ca-certificates.crt,
which includes e.g. Fedora.

(cherry picked from commit 37b22dae04)

Change-Id: Iaa2e0e9db3747645b5482c82e3e0e4e8f229f5f9
2024-09-26 14:44:23 -07:00
Eelco DolstraandJade Lovelace 0f099ae619 [security] builtin:fetchurl: Enable TLS verification
This is better for privacy and to avoid leaking netrc credentials in a
MITM attack, but also the assumption that we check the hash no longer
holds in some cases (in particular for impure derivations).

Partially reverts https://github.com/NixOS/nix/commit/5db358d4d78aea7204a8f22c5bf2a309267ee038.

upstream commits:
(cherry picked from commit c04bc17a5a0fdcb725a11ef6541f94730112e7b6)
(cherry picked from commit f2f47fa725fc87bfb536de171a2ea81f2789c9fb)
(cherry picked from commit 7b39cd631e0d3c3d238015c6f450c59bbc9cbc5b)

lix main:
(cherry picked from commit c1631b0a39)

Upstream-PR: https://github.com/NixOS/nix/pull/11585

Change-Id: Ia973420f6098113da05a594d48394ce1fe41fbb9
2024-09-26 14:44:23 -07:00
Yureka ed51a172c6 libutil: fix conditional for close_range availability
This check is wrong and would cause the close_range() function being called even when it's not available

Change-Id: Ide65b36830e705fe772196c37349873353622761
(cherry picked from commit df49d37b71)
2024-08-20 09:09:57 +02:00
Jade Lovelace b9ed79c99a libutil: deal with Linux systems that do not implement close_range
Seems a little bit Rich that musl does not implement close_range because
they suspect that the system call itself is a bad idea, so they uhhhh
are considering not implementing a wrapper. Let's just fix the problem
at hand by writing our own wrapper.

Change-Id: I1f8e5858e4561d58a5450503d9c4585aded2b216
2024-08-10 16:11:58 -07:00
Jade Lovelace b15d5cc6ee nix: remove explosions if you have a window size less than four
Turns out strings do not like being resized to -4.

This was discovered while messing with the tests to remove unbuffer and
trying stdbuf instead. Turns out that was not the right approach.

This basically rewrites the handling of this case to be much more
correct, and fixes a bug where with small window sizes where it would
ALSO truncate the attr names in addition to the optional descriptions.

Change-Id: Ifd1beeaffdb47cbb5f4a462b183fcb6c0ff6c524
2024-08-10 16:11:58 -07:00
Jade Lovelace 292567e0b0 fix: check if it is a Real terminal, not just if it is a terminal
This will stop printing stuff to dumb terminals that they don't support.

I've overall audited usage of isatty and replaced the ones with intent
to mean "is a Real terminal" with checking for that. I've also caught a
case of carelessly assuming "is a tty" means "should be colour" in
nix-env.

Change-Id: I6d83725d9a2d932ac94ff2294f92c0a1100d23c9
2024-08-10 16:07:21 -07:00
eldritch horrors c7d97802e4 libutil: rename and optimize closeMostFDs
this is only used to close non-stdio files in derivation sandboxes. we
may as well encode that in its name, drop the unnecessary integer set,
and use close_range to deal with the actual closing of files. not only
is this clearer, it also makes sandbox setup on linux fast by 1ms each

Change-Id: Id90e259a49c7bc896189e76bfbbf6ef2c0bcd3b2
2024-08-09 19:59:17 +00:00
eldritch horrors 35a2f28a46 libstore: deprecate the build-hook setting
implementing a build hook is pretty much impossible without either being
a nix, or blindly forwarding the important bits of all build requests to
some kind of nix. we've found no uses of build-hook in the wild, and the
build-hook protocol (apart from being entirely undocumented) is not able
to convey any kind of versioning information between hook and daemon. if
we want to upgrade this infrastructure (which we do), this must not stay

Change-Id: I1ec4976a35adf8105b8ca9240b7984f8b91e147e
2024-08-09 19:30:45 +00:00
jadeandGerrit Code Review 790d1079e1 Merge changes Ib7c80826,I636f8a71,I67669b98 into main
* changes:
  perl: un-autos your conf
  build: declare all the deps as -isystem
  darwin: workaround PROC_PIDLISTFDS on processes with no fds
2024-08-09 19:24:29 +00:00
QyriadandGerrit Code Review 346e340cbf Merge "libexpr: move Value implementations out of eval.cc" into main 2024-08-09 14:25:13 +00:00
eldritch horrors 5d4686bcd5 libutil: allow marking settings as deprecated
this is a bit of a hack, but it's apparently the cleanest way of doing
this in the absence of any kind of priority/provenance information for
values of some given setting. we'll need this to deprecate build-hook.

Change-Id: I03644a9c3f17681c052ecdc610b4f1301266ab9e
2024-08-09 11:33:09 +00:00
eldritch horrors baa4fda340 main: require argv[0]
sure, linux has been providing argv[0] by default for a while now. other
OSes may not be as forthcoming though, and relying on the OS to create a
world in which we can just make assumptions we could test for instead is
unnecessarily lazy. we *could* default argv0, but that's a little silly.

notably we abort instead of returning normally to avoid confusions where
a caller interprets our exit status like a Worker build results bitmask.

Change-Id: Id73f8cd0a630293b789c59a8c4b0c4a2b936b505
2024-08-09 11:33:09 +00:00
eldritch horrors 6491cde997 resolve-system-dependencies: remove entirely
this hasn't been used since 2020, and hasn't been compiled since may.

Change-Id: I865550966630eee6ba18d742ba36f0a90901279d
2024-08-09 11:33:09 +00:00
Qyriad 0787dcf5f6 libexpr: move Value implementations out of eval.cc
Change-Id: I2ce8a9713533888b3d109a56947156eb3a5ab492
2024-08-08 22:01:12 -06:00
jadeandGerrit Code Review 9682ab4f38 Merge changes I6358a393,I2d9f276b,Idd096dc9 into main
* changes:
  clang-tidy: write a lint for charptr_cast
  tree-wide: automated migration to charptr_cast
  clang-tidy: enforce the new rules
2024-08-08 23:09:30 +00:00
jadeandGerrit Code Review 757041c3e7 Merge changes I526cceed,Ia4e2f1fa,I22e66972,I9fbd55a9,Ifca22e44 into main
* changes:
  sqlite: add a Use::fromStrNullable
  util: implement charptr_cast
  tree-wide: fix a pile of lints
  refactor: make HashType and Base enum classes for type safety
  build: integrate clang-tidy into CI
2024-08-08 22:43:10 +00:00
Jade Lovelace 4ed8461cac sqlite: add a Use::fromStrNullable
There were several usages of the raw sqlite primitives along with C
style casts, seemingly because nobody thought to use an optional for
getting a string or NULL.

Let's fix this API given we already *have* a wrapper.

Change-Id: I526cceedc2e356209d8fb62e11b3572282c314e8
2024-08-08 14:53:17 -07:00
Jade Lovelace a85c4ce535 tree-wide: automated migration to charptr_cast
The lint did it :3

Change-Id: I2d9f276b01ebbf14101de4257ea13e44ff6fe0a0
2024-08-08 14:53:17 -07:00
Jade Lovelace a318c96851 util: implement charptr_cast
I don't like having so many reinterpret_cast statements that have to
actually be looked at to determine if they are UB. A huge number of the
reinterpret_cast instances in Lix are actually casting to some pointer
of some character type, which is always valid no matter the source type.

However, it is also worth looking at if it is not casting both *from* a
character type and also *to* a character type, since IMO splatting a
struct into a character array should be a very deliberate action instead
of just being about dealing with bad APIs.

So let's write a template that encapsulates this invariant so we can
not worry about the trivially safe reinterpret_cast invocations.

Change-Id: Ia4e2f1fa0c567123a96604ddadb3bdd7449660a4
2024-08-08 14:53:17 -07:00
Jade Lovelace e34833c025 tree-wide: fix a pile of lints
This:
- Converts a bunch of C style casts into C++ casts.
- Removes some very silly pointer subtraction code (which is no more or
  less busted on i686 than it began)
- Fixes some "technically UB" that never had to be UB in the first
  place.
- Makes finally follow the noexcept status of the inner function. Maybe
  in the future we should ban the function from not being noexcept, but
  that is not today.
- Makes various locally-used exceptions inherit from std::exception.

Change-Id: I22e66972602604989b5e494fd940b93e0e6e9297
2024-08-08 14:53:17 -07:00
Jade Lovelace 370ac940dd refactor: make HashType and Base enum classes for type safety
Change-Id: I9fbd55a9d50464a56fe11cb42a06a206914150d8
2024-08-08 14:53:17 -07:00
Jade Lovelace f3ef0899c7 build: integrate clang-tidy into CI
This still has utterly unacceptably bad output format design that I
would not inflict on anyone I like, but it *does* now exist, and you
*can* find the errors in the log.

Future work would obviously be to fix that and integrate the actual
errors into Gerrit using codechecker or so.

Followup issue: https://git.lix.systems/lix-project/lix/issues/457

Fixes: https://git.lix.systems/lix-project/lix/issues/147
Change-Id: Ifca22e443d357762125f4ad6bc4f568af3a26c62
2024-08-08 14:53:17 -07:00
piegamesandGerrit Code Review e03cd8b3a6 Merge "libexpr: Add experimental pipe operator" into main 2024-08-08 18:15:21 +00:00
eldritch horrors a957219df2 libstore: make Worker::waitForInput private
Change-Id: I71a42acd5a4a9a18b55cf754cdf9896614134398
2024-08-08 12:02:17 +00:00
eldritch horrors ba85e501ce libstore: make Worker status flags private
Change-Id: I16ec8994c6448d70b686a2e4c10f19d4e240750d
2024-08-08 12:02:17 +00:00
eldritch horrors fc987b4123 libstore: remove Goal::addWaitee
Change-Id: I1b00d1a537d84790878cb0e81aaa1cbaa143d62d
2024-08-08 12:02:17 +00:00
eldritch horrors 4c3010a1be libstore: make Worker::wakeUp private
Change-Id: Iffa55272fe6ef4adaf3e9d4d25e5339792c2e460
2024-08-08 12:02:17 +00:00
eldritch horrors 3ecb46e3e7 libstore: make Worker::waitForAWhile private
Change-Id: I0cdcd436ee71124ca992b4f4fe307624a25f11e9
2024-08-08 12:02:17 +00:00
eldritch horrors b33c969519 libstore: make Worker::waitForBuildSlot private
Change-Id: I02a54846cd65622edbd7a1d6c24a623b4a59e5b3
2024-08-08 12:02:17 +00:00
piegames 28ae24f3f7 libexpr: Add experimental pipe operator
The |> operator is a reverse function operator with low binding strength
to replace lib.pipe. Implements RFC 148, see the RFC text for more
details. Closes #438.

Change-Id: I21df66e8014e0d4dd9753dd038560a2b0b7fd805
2024-08-08 11:13:53 +02:00
Max SilingandGerrit Code Review 6fdb47f0b2 Merge "src/libcmd/repl.cc: allow :log /path/to/store.drv" into main 2024-08-07 21:48:01 +00:00
jadeandGerrit Code Review 0800a81a95 Merge "oops: fix warning about catching polymorphic exception" into main 2024-08-07 19:06:54 +00:00
piegames ec7552ff74 libexpr/parser: Test experimental features
Currently, the parser relies on the global experimental feature flags.
In order to properly test conditional language features, we instead need
to pass it around in the parser::State.

This means that the parser cannot cache the result of isEnabled anymore,
which wouldn't necessarily hurt performance if the function didn't
perform a linear search on the list of enabled features on every single
call. While we could simply evaluate once at the start of parsing and
cache the result in the parser state, the more sustainable solution
would be to fix `isEnabled` such that all callers may profit from the
performance improvement.

Change-Id: Ic9b9c5d882b6270e1114988b63e6064d36c25cf2
2024-08-07 13:07:50 +00:00
Goldstein 9adfd9b8ad src/libcmd/repl.cc: allow :log /path/to/store.drv
This adds a second form to the `:log` command: it now can accept a
derivation path in addition to a derivation expression. As derivation
store paths start with `/nix/store`, this is not ambiguous.

Resolves: https://git.lix.systems/lix-project/lix/issues/51
Change-Id: Iebc7b011537e7012fae8faed4024ea1b8fdc81c3
2024-08-07 15:58:44 +03:00
Maximilian BoschandGerrit Code Review 27a63db710 Merge "fix: warn and document when advanced attributes will have no impact due to __structuredAttrs" into main 2024-08-07 10:38:39 +00:00
Jade Lovelace 1437d3df15 darwin: workaround PROC_PIDLISTFDS on processes with no fds
This has been causing various seemingly spurious CI failures as well as
some failures on people running tests on beta builds.

lix> ++(nix-collect-garbage-dry-run.sh:20) nix-store --gc --print-dead
lix> ++(nix-collect-garbage-dry-run.sh:20) wc -l
lix> finding garbage collector roots...
lix> error: Listing pid 87261 file descriptors: Undefined error: 0

There is no real way to write a proper test for this, other than to
start a process like the following:

int main(void) {
    for (int i = 0; i < 1000; ++i) {
        close(i);
    }
    sleep(10000);
}

and then let Lix's gc look at it.

I have a relatively high confidence this *will* fix the problem since I
have manually confirmed the behaviour of the libproc call is
as-unexpected, and it would perfectly explain the observed symptom.

Fixes: https://git.lix.systems/lix-project/lix/issues/446
Change-Id: I67669b98377af17895644b3bafdf42fc33abd076
2024-08-07 02:52:00 -07:00
alois31andGerrit Code Review 780998f4ea Merge "package: improve support for building without BDW-GC" into main 2024-08-07 07:07:28 +00:00
Jade Lovelace d280e4990c oops: fix warning about catching polymorphic exception
This was introduced in I0fc80718eb7e02d84cc4b5d5deec4c0f41116134 and
unnoticed since it only appears in gcc builds.

Change-Id: I1de80ce2a8fab63efdca7ca0de2a302ceb118267
2024-08-06 22:45:19 -07:00
jadeandGerrit Code Review 529eed74c4 Merge changes I0fc80718,Ia182b86f,I355f82cb,I8a9b58fa,Id89f8a1f, ... into main
* changes:
  tree-wide: fix various lint warnings
  flake & doxygen: update tagline
  nix flake metadata: print modified dates for input flakes
  cli: eat terminal codes from stdout also
  Implement forcing CLI colour on, and document it better
  manual: fix a syntax error in redirects.js that made it not do anything
  misc docs/meson tidying
  build: implement clang-tidy using our plugin
2024-08-07 00:50:30 +00:00
Alois Wohlschlager 2c48460850 libstore/linux: precompile and cache the seccomp BPF
The growth of the seccomp filter in 127ee1a101
made its compilation time significant (roughly 10 milliseconds have been
measured on one machine). For this reason, it is now precompiled and cached in
the parent process so that this overhead is not hit for every single build. It
is still not optimal when going through the daemon, because compilation still
happens once per client, but it's better than before and doing it only once for
the entire daemon requires excessive crimes with the current architecture.

Fixes: https://git.lix.systems/lix-project/lix/issues/461
Change-Id: I2277eaaf6bab9bd74bbbfd9861e52392a54b61a3
2024-08-06 19:10:33 +02:00
Alois Wohlschlager 403fa9e2b6 libstore/linux: compile the seccomp BPF explicitly
This is a preparation for precompiling the filter, which is done separately.
The behaviour should be unchanged for now.

Change-Id: I899aa7242962615949208597aca88913feba1cb8
2024-08-06 18:31:40 +02:00
Alois Wohlschlager 741d3b441c libstore: add LocalDerivationGoal setupSyscallFilter hook
The seccomp setup code was a huge chunk of conditionally compiled
platform-specific code. For this reason, it is appropriate to move it to the
platform-specific implementation file. Ideally its setup could be moved a bit
to make it happen at the same place as the Darwin restrictions, but that change
is going to be less mechanical.

Change-Id: I496aa3c4fabf34656aba1e32b0089044ab5b99f8
2024-08-06 18:27:09 +02:00
Jade Lovelace ca9d3e6e00 tree-wide: fix various lint warnings
Change-Id: I0fc80718eb7e02d84cc4b5d5deec4c0f41116134
2024-08-04 20:55:45 -07:00