eldritch horrors cbf7929277 libstore: use thread pools for remote stderr handling ...

spawning threads is not free. reusing threads is also not free, but a
lot cheaper than spawning now ones. the difference can add up quite a
bit on nixos system evaluation; we've seen 20k threads cycled through
on even modestly sized configurations, and at 20µs per thread just to
launch and tear down (excluding any work done) that's potentially not
insignificant. in our test config this was a 3% eval time difference.

Change-Id: I40f901813e700fb99f5def82da54ca969711961b

2025-03-14 09:39:09 +00:00

..

build

libstore: fix substitution failures on racy builds

2025-03-10 21:18:22 +00:00

builtins

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

file-transfer-settings

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

platform

Avoid lix daemon killing unrelated processes when using sandboxes under Linux

2025-02-18 15:59:59 +00:00

settings

fix: pre-build-hook did not run when useChroot == false

2025-02-17 16:05:06 -08:00

binary-cache-store.cc

libstore: turn Store::state into an async mutex

2025-03-05 18:49:45 +01:00

binary-cache-store.hh

libstore: asyncify Store::queryPathInfo{,Uncached}

2025-03-05 18:49:45 +01:00

build-result.cc

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

build-result.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

builtins.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

ca-specific-schema.sql

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

common-protocol-impl.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

common-protocol.cc

tree-wide: fix a pile of lints

2024-12-05 17:13:37 -08:00

common-protocol.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

content-address.cc

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

content-address.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

crypto.cc

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

crypto.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

daemon.cc

libstore: asyncify Store::queryPathInfo{,Uncached}

2025-03-05 18:49:45 +01:00

daemon.hh

treewide: add async io roots to entry points

2025-01-24 13:48:44 +00:00

derivations.cc

libstore: asyncify Store::read{,Invalid}Derivation

2025-03-05 18:49:45 +01:00

derivations.hh

libstore: asyncify hashDerivationModulo

2025-03-05 18:49:45 +01:00

derived-path-map.cc

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

derived-path-map.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

derived-path.cc

libstore: asyncify Store::query{,Partial}DerivationOutputMap

2025-02-25 02:09:22 +00:00

derived-path.hh

libstore: asyncify a bunch of toJSON methods

2025-02-23 17:18:48 +00:00

downstream-placeholder.cc

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

downstream-placeholder.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

dummy-store.cc

libstore: asyncify Store::queryPathInfo{,Uncached}

2025-03-05 18:49:45 +01:00

dummy-store.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

dummy-store.md

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

export-import.cc

libstore: asyncify Store::queryPathInfo{,Uncached}

2025-03-05 18:49:45 +01:00

filetransfer.cc

libstore: don't use curl decompression support

2025-03-08 20:58:34 +01:00

filetransfer.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

fs-accessor.hh

libstore: asyncify FSAccessor, listNar

2025-03-05 18:49:45 +01:00

gc-store.hh

gc: allow continuing deletion even if some paths are still live

2025-03-11 14:10:32 +01:00

gc.cc

gc: allow continuing deletion even if some paths are still live

2025-03-11 14:10:32 +01:00

globals.cc

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

globals.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

http-binary-cache-store.cc

libstore: asyncify Store::isTrustedClient

2025-03-05 16:42:11 +01:00

http-binary-cache-store.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

http-binary-cache-store.md

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

indirect-root-store.hh

libstore: asyncify IndirectRootStore::addIndirectRoot

2025-03-05 16:42:11 +01:00

legacy-ssh-store.cc

libutil/libstore: asyncify Pool

2025-03-05 23:07:20 +01:00

legacy-ssh-store.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

legacy-ssh-store.md

libstore: allow specifying port for ssh[-ng] stores

2025-02-22 20:37:45 +01:00

length-prefixed-protocol-helper.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

lix-store.pc.in

pkg-config: remove legacy include paths!

2025-01-16 00:12:57 -08:00

local-binary-cache-store.cc

libstore: asyncify Store::queryAllValidPaths

2025-03-05 18:49:45 +01:00

local-binary-cache-store.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

local-binary-cache-store.md

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

local-fs-store.cc

libstore: asyncify Store::isValidPath

2025-03-05 18:49:45 +01:00

local-fs-store.hh

libstore: asyncify Store::narFromPath

2025-03-05 18:49:45 +01:00

local-store.cc

libstore: turn LocalStore::dbPool into a mutex

2025-03-05 23:07:20 +01:00

local-store.hh

libstore: turn LocalStore::dbPool into a mutex

2025-03-05 23:07:20 +01:00

local-store.md

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

lock.cc

libstore: split tryLockFile lockFile

2025-02-03 21:56:26 +01:00

lock.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

log-store.cc

libstore: asyncify LogStore::getBuildLogExact

2025-02-24 15:09:02 +00:00

log-store.hh

libstore: asyncify LogStore::addBuildLog

2025-02-24 15:09:02 +00:00

machines.cc

libstore: asyncify openStore

2025-02-08 12:45:15 +00:00

machines.hh

libstore: asyncify openStore

2025-02-08 12:45:15 +00:00

make-content-addressed.cc

libstore: asyncify Store::queryPathInfo{,Uncached}

2025-03-05 18:49:45 +01:00

make-content-addressed.hh

libstore: asyncify makeContentAddressed

2025-02-08 13:50:33 +00:00

meson.build

build: add meson dep overrides for all internal libs

2025-01-28 12:19:00 -08:00

misc.cc

libstore: asyncify Store::queryPathInfo{,Uncached}

2025-03-05 18:49:45 +01:00

names.cc

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

names.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

nar-accessor.cc

libstore: asyncify FSAccessor, listNar

2025-03-05 18:49:45 +01:00

nar-accessor.hh

libstore: asyncify FSAccessor, listNar

2025-03-05 18:49:45 +01:00

nar-info-disk-cache.cc

libutil: split retrySQLite into sync and async

2025-02-22 17:59:54 +00:00

nar-info-disk-cache.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

nar-info.cc

nar-info: use string views for construction

2025-01-23 11:21:24 +00:00

nar-info.hh

nar-info: use string views for construction

2025-01-23 11:21:24 +00:00

optimise-store.cc

libstore: asyncify Store::isValidPath

2025-03-05 18:49:45 +01:00

outputs-spec.cc

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

outputs-spec.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

parsed-derivations.cc

libstore: asyncify Store::pathInfoToJSON

2025-02-08 17:01:31 +00:00

parsed-derivations.hh

libstore: asyncify ParsedDerivation::prepareStructuredAttrs

2025-02-04 15:37:44 +00:00

path-info.cc

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

path-info.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

path-references.cc

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

path-references.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

path-regex.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

path-with-outputs.cc

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

path-with-outputs.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

path.cc

libstore: don't derive store classes from their configs

2024-11-20 14:59:40 +01:00

path.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

pathlocks.cc

libstore: mark blocking file locks as never async

2025-02-16 14:59:32 +00:00

pathlocks.hh

libstore: mark blocking file locks as never async

2025-02-16 14:59:32 +00:00

platform.cc

libstore: create drv temp roots in Worker

2025-02-12 01:32:01 +00:00

profiles.cc

libstore: mark blocking file locks as never async

2025-02-16 14:59:32 +00:00

profiles.hh

libstore: mark blocking file locks as never async

2025-02-16 14:59:32 +00:00

realisation.cc

libstore: asyncify Store::queryRealisation{,Uncached}

2025-03-05 18:49:45 +01:00

realisation.hh

libstore: asyncify Realisation::closure

2025-02-25 02:09:22 +00:00

remote-fs-accessor.cc

libstore: asyncify Store::isValidPath

2025-03-05 18:49:45 +01:00

remote-fs-accessor.hh

libstore: asyncify FSAccessor, listNar

2025-03-05 18:49:45 +01:00

remote-store-connection.hh

libstore: use thread pools for remote stderr handling

2025-03-14 09:39:09 +00:00

remote-store.cc

libstore: use thread pools for remote stderr handling

2025-03-14 09:39:09 +00:00

remote-store.hh

libstore: use thread pools for remote stderr handling

2025-03-14 09:39:09 +00:00

s3-binary-cache-store.cc

libstore: asyncify Store::queryPathInfo{,Uncached}

2025-03-05 18:49:45 +01:00

s3-binary-cache-store.hh

libstore: don't derive store classes from their configs

2024-11-20 14:59:40 +01:00

s3-binary-cache-store.md

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

s3.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

schema.sql

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

serve-protocol-impl.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

serve-protocol.cc

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

serve-protocol.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

sqlite.cc

libstore: drastically shorten sqlite busy timeout

2025-03-05 17:34:25 +01:00

sqlite.hh

libutil: split retrySQLite into sync and async

2025-02-22 17:59:54 +00:00

ssh-store.cc

libstore: asyncify LogStore::getBuildLogExact

2025-02-24 15:09:02 +00:00

ssh-store.hh

libstore: allow specifying port for ssh[-ng] stores

2025-02-22 20:37:45 +01:00

ssh-store.md

libstore: allow specifying port for ssh[-ng] stores

2025-02-22 20:37:45 +01:00

ssh.cc

libstore: allow specifying port for ssh[-ng] stores

2025-02-22 20:37:45 +01:00

ssh.hh

libstore: allow specifying port for ssh[-ng] stores

2025-02-22 20:37:45 +01:00

store-api.cc

libsture: use processGraphAsync in Store

2025-03-05 23:07:20 +01:00

store-api.hh

libstore: turn Store::state into an async mutex

2025-03-05 18:49:45 +01:00

store-cast.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

temporary-dir.cc

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

temporary-dir.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

uds-remote-store.cc

libutil/libstore: asyncify Pool

2025-03-05 23:07:20 +01:00

uds-remote-store.hh

libstore: asyncify Store::narFromPath

2025-03-05 18:49:45 +01:00

uds-remote-store.md

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

worker-protocol-impl.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

worker-protocol.cc

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00

worker-protocol.hh

fix tooling after include reorganization

2024-11-19 22:55:32 +00:00