Commit Graph
17390 Commits
Author SHA1 Message Date
eldritch horrors 8bfd2d4bb1 libfetchers: asyncify downloadFile, downloadTarball
Change-Id: I6f6e5b61667efeda8c575ae1f3bd87ee75fb9f9e
2025-02-08 17:26:19 +00:00
eldritch horrors 60f391ef3d libstore: asyncify LocalFSStore::addPermRoot
Change-Id: I0e19470e7fcc3d563b26c078c1dc831dbc5cf422
2025-02-08 17:26:19 +00:00
eldritch horrors 614a63b49a libstore: asyncify createGeneration
Change-Id: Iae47a0f732b2f0fe54525917c091169f4be2214e
2025-02-08 17:26:19 +00:00
eldritch horrors 1ce46c318c libfetchers: asyncify getRegistries, lookupInRegistries
Change-Id: I5a668ad05440e2d7d574e8a470c269b670282c6f
2025-02-08 17:26:19 +00:00
eldritch horrors 07691dd851 libexpr: asyncify FlakeRef::resolve
Change-Id: I31a1080652f03bdddd63b3115fa0686aa340dc34
2025-02-08 17:26:19 +00:00
eldritch horrors 17b3992e78 libcmd: add an aio root to completeFlakeRef
the other completers already inherit one through EvalState. all of them
eventually need it since completing a flake ref may require network io.

Change-Id: Ie664a129a68457a6f0908d226057d7d7fb610ae4
2025-02-08 17:01:31 +00:00
eldritch horrors 0bd93b5208 libfetchers: asyncify GitArchiveInputScheme::getRevFromRef
Change-Id: Id409ed6c13e71eed3628a8ed10f6fd03060c79b0
2025-02-08 17:01:31 +00:00
eldritch horrors 6e95540610 libstore: asyncify Store::pathInfoToJSON
Change-Id: I2e04c6804444b3ca53486ce6acb0c159e7814938
2025-02-08 17:01:31 +00:00
eldritch horrors 6ffa9f4298 libfetchers: asyncify InputScheme::fetch
Change-Id: I8d4d8d1d4137635ade3e4b05b5d4cd3ecb3390c9
2025-02-08 17:01:31 +00:00
eldritch horrors b60314f32e libstore: asyncify makeContentAddressed
Change-Id: I3cdad058a9f015983db5306271c56a9996aaef28
2025-02-08 13:50:33 +00:00
eldritch horrors 5fd2d24e63 cli: asyncify print{DotGraph,GraphML}
Change-Id: I0fc13a861acee55c53c4ae5f6ba77b4100fd8735
2025-02-08 13:50:33 +00:00
eldritch horrors 9b0cd7a7c9 libstore: asyncify copyPaths
Change-Id: Ie7cd94d8760abd9c7593f73fd281cbba91a5be77
2025-02-08 13:50:33 +00:00
eldritch horrors fb9e3f6cb5 libstore: asyncify copyClosure
Change-Id: I18cc324d80b502ac87dda3fba2166d4e25f53664
2025-02-08 13:50:33 +00:00
eldritch horrors ecfda8abe2 libstore: asyncify openStore
Change-Id: Ia152bfd2014851590328e732f434dff6695304bd
2025-02-08 12:45:15 +00:00
eldritch horrors af507e587f libstore: asyncify getDefaultSubstituters
Change-Id: I078c5575f4941ab0d06ef9bf1e549b789e695afc
2025-02-08 12:45:15 +00:00
eldritch horrors bf1c5cb27d libstore: asyncify Store::querySubstitutablePathInfos
this also calls getDefaultSubstituters.

Change-Id: I9c80d8b8311a712a3e73964445c42d10a5dab7db
2025-02-08 12:45:15 +00:00
eldritch horrors c2f71d5b30 libstore: asyncify Store::querySubstitutablePaths
the LocalStore implementation calls `getDefaultSubstituters`, which
calls `openStore`, which calls `Store::init`, which does network io
in binary caches, which will eventually be properly asyncified curl

Change-Id: If6a7a0294e2c4ba8e35722583be6ccde21ed6585
2025-02-08 12:45:15 +00:00
eldritch horrors 6e12f8922c libstore: add async path locking
just single path locks for now since we don't need more.

Change-Id: I0f1be156e762fbe7355dc1381295a7b276bad720
2025-02-08 12:45:15 +00:00
eldritch horrors ec44c8bd6c libstore: rework path locking infra entirely
we now allow for single locked paths rather than only sets, the lock
holder classes are finally resource-safe, and not locking files from
within constructors means we can (in theory!) make path locks async.

Change-Id: I1e1807299d370c07b15c332d5b2ff77b64456e7d
2025-02-08 13:39:49 +01:00
eldritch horrors 72326c4044 libstore: always delete PathLocks lockfiles
they are not being created with O_EXCL and their existence is not
checked for using the PathLocks api, so we will boldy assume that
*nothing* checks for lock file existence. deleting lockfiles when
we're done with them in all cases makes the code cleaner, and any
failing builds no longer litter their store with stale lockfiles.

Change-Id: Iffb588d29b00e6aca32fcb6776980455238ac2bd
2025-02-08 13:14:48 +01:00
eldritch horrors 80189b3c26 libstore: add async FdLock constructor
the local store doesn't have to block the entire thread waiting for a
lock with such support. we'll still block *some* thread, but there is
currently no good way around that without e.g. a lock manager daemon.

Change-Id: I037fdc749e1b56cf9e0f1ca7ac70274442fb2b8c
2025-02-08 08:54:16 +00:00
eldritch horrors 8f5f33c9d8 libstore: bind FdLock to AutoCloseFD and make resource-safe
Change-Id: I2451c15bd0e7d3a5a537bbb65502fd4a18ae0c22
2025-02-08 08:33:26 +00:00
Justin !andGerrit Code Review 00a1afe022 Merge "libutil: make runProgram2 print debug infos" into main 2025-02-07 15:10:53 +00:00
sethandGerrit Code Review 1a13827425 Merge "libutil/file-descriptor: include sys/syscall.h" into main 2025-02-06 23:20:29 +00:00
Justin !andeldritch horrors c526400bb5 libutil: make runProgram2 print debug infos
Fix: https://git.lix.systems/lix-project/lix/issues/481
Change-Id: I8af50a8ba96039b377774d55371868ba82bd3f1a
2025-02-06 21:52:24 +00:00
Ilya KandGerrit Code Review f3f1a2c328 Merge "flake.nix: hack to fix build on staging-next" into main 2025-02-06 17:55:37 +00:00
eldritch horrors 3c7162e6d0 libstore: split PathLocks::lockPaths into lock, tryLock
Change-Id: I202cfc077b7e468c74ecd8039175cef34855944c
2025-02-06 14:05:34 +00:00
K900 91ef5f629c flake.nix: hack to fix build on staging-next
Hopefully we don't actually ship this.

Change-Id: Idb4c2e8e1b28659f3d8f1314381b229a279d56e1
2025-02-06 10:50:38 +03:00
Lily BallardandGerrit Code Review 3b10c19006 Merge "libstore: make SQLite busy back-off logic portable" into main 2025-02-06 04:45:30 +00:00
Seth Flynn 4e45583d2e libutil/file-descriptor: include sys/syscall.h
For `SYS_close_range`

Change-Id: Ie3e6a94a354b336c3967efc4a73593d35096bfe0
2025-02-05 10:11:15 -05:00
alois31andGerrit Code Review 8553adbb41 Merge "libfetchers/mercurial: handle "evil refs" gracefully" into main 2025-02-04 17:40:32 +00:00
jadeandGerrit Code Review 06d3c59db0 Merge "Fix crash in debugger mode using with" into main 2025-02-04 16:36:50 +00:00
eldritch horrors cda78765fa libstore: asyncify ParsedDerivation::prepareStructuredAttrs
Change-Id: I6c34bb5fc67046a0c25486bf4f2fa5bbaa62eacf
2025-02-04 15:37:44 +00:00
eldritch horrors 6423ee2b7e libstore: asyncify Store::exportReferences
Change-Id: Ia4068d31afc48cf9d459bed3d5d8943be06718ee
2025-02-04 14:11:08 +00:00
eldritch horrors de66d7b2b4 libstore: asyncify Store::verifyStore
Change-Id: I12d5f348aa5a253d409b45db8a6bbb18754450c0
2025-02-04 14:10:16 +00:00
eldritch horrors a400394454 libexpr: asyncify DrvInfo constructor
since constructors can't actually be async we need a static wrapper
function that does the async work. it's fine, rust survives it too.

Change-Id: Ifa13d6f96d5a8b5b7fe215530b6eba24ef921d6d
2025-02-04 14:06:04 +00:00
eldritch horrors 68947e7a65 libstore: asyncify Store::derivationFromPath
Change-Id: Ic6b54642da08f258c113d5076b34b7c13096f540
2025-02-04 14:03:54 +00:00
eldritch horrors 338e0c681d libstore: make Store::queryMissing aio-ready
Change-Id: I398023fc200b9070de7af49ed90af8f1443ba418
2025-02-04 13:28:35 +00:00
eldritch horrors 3ce789acb5 libutil: add async io roots to threadpool threads
this needs a new method because overloading rules would make a pair of
methods taking `std::function`s of different argument types ambiguous.
we could've turned enqueue into a template and made `enqueueWithAio` a
private method too, but the necessary template magic is wasted on this

Change-Id: I93ccfde07126c287ad2a4dc5a07caec6a9d83944
2025-02-04 13:10:21 +00:00
eldritch horrors 4138fc7622 libutil: don't do work on threadpool main thread
this is mandatory for async support in worker threads at this time.

Change-Id: Ie58871bf33f6266aec58f37e10cfbc47f82a3aa9
2025-02-04 13:10:01 +00:00
Lily BallardandGerrit Code Review af1f27cac1 Merge "daemon: get peer pid, gid for socket connection on darwin" into main 2025-02-04 04:50:49 +00:00
Rebecca Turner c263070d37 Fix crash in debugger mode using with
Fixes this crash (#592):

    $ nix repl --debugger
    Lix 2.92.0-dev-pre20241120-66f6dbd debugger
    Type :? for help.
    nix-repl> let x = 4; in __seq x (with x; (x: builtins.break x) 1)
    info: breakpoint reached

    [1]    949722 segmentation fault (core dumped)  nix repl --debugger

Change-Id: I13c72941dc325ff984dcce2a84c01c89b1c552af
2025-02-03 15:22:50 -08:00
eldritch horrors d186064c3d libstore: never return from lockFile without a lock
signals could cause lockFile to return without having locked the file.
the garbage collector didn't check for this, and then hilarity ensued.

Change-Id: If86d33595e8bf5510d2b032139342261dc6e07c9
2025-02-03 22:27:47 +00:00
eldritch horrors 79f9c39e36 libstore: split tryLockFile lockFile
lockFile is currently interruptible by signals like SIGCHLD. which
happen a lot in daemons. now imagine that daemon enabled automatic
garbage collection. observe that the local store does not actually
check whether its lock operations have succeeded ... get the idea?

Change-Id: Ibfd7ee786c4fee3add72d4456a7e95e73e09c73e
2025-02-03 21:56:26 +01:00
eldritch horrors aa87c8aa93 libstore: extract unlockFile from lockFile
propagating the "unlock" lock type through this high-level api is
nonsense. it doesn't make sense to treat locking and unlocking as
similar operations; unlocking *must* not not interruptible by our
checkInterrupt machinery or it will just leave locks lying around
for a potentially very long time. unlock operations should not be
taking long enough to *want* them interrupted anyway. even on nfs
this makes very little sense because nfs waits *uninterruptibly*.

Change-Id: I10d605c8fe6c651bee64466eee1f8e20251d39f4
2025-02-03 21:56:26 +01:00
Lily Ballard 3eea7866f6 daemon: get peer pid, gid for socket connection on darwin
Fixes: https://git.lix.systems/lix-project/lix/issues/640
Change-Id: I2c4dc0e2642379e3f16fc39909dd4da8695d7239
2025-02-02 22:27:13 -08:00
eldritch horrors a51380645f libstore: asyncify Store::substitutePaths
Change-Id: I8cb6822ad9145fbcf0f98f32c121482c3a3db2a8
2025-02-01 21:48:48 +00:00
eldritch horrors 48519448e2 libcmd: asyncify lookupFileArg
Change-Id: Ied47f5f96eeb8324c500335fa8f5388d10bed54c
2025-02-01 21:48:48 +00:00
eldritch horrors 29c9dff56b libexpr: asyncify EvalPaths::{findFile, resolveSearchPathPath}
Change-Id: I4ab0d9343efd86bc459870f38d9db909888c070d
2025-02-01 21:48:48 +00:00
eldritch horrors 2da4824d61 libexpr: asyncify FlakeRef::fetchTree
Change-Id: I8334d8e1a9ea0e555bf2af1eb09dd9653013b28b
2025-02-01 21:48:48 +00:00