guiwf/lix
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

queue-runner: limit parallelism of CPU intensive operations

Browse Source 
My current theory is that running more parallel xz than available CPU
cores is reducing our overall throughput by requiring more scheduling
overhead and more cache thrashing.
This commit is contained in:
Pierre Bourdon
2024-04-11 16:43:01 +02:00
parent a596d6c3c1
commit a51bd392a2
4 changed files with 27 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/hydra-queue-runner/hydra-queue-runner.cc
+1
View File
@@ -85,6 +85,7 @@ State::State(std::optional<std::string> metricsAddrOpt)
: config(std::make_unique<HydraConfig>())
, maxUnsupportedTime(config->getIntOption("max_unsupported_time", 0))
, dbPool(config->getIntOption("max_db_connections", 128))
, localWorkThrottler(config->getIntOption("max_local_worker_threads", std::min(maxSupportedLocalWorkers, std::max(4u, std::thread::hardware_concurrency()) - 2)))
, maxOutputSize(config->getIntOption("max_output_size", 2ULL << 30))
, maxLogSize(config->getIntOption("max_log_size", 64ULL << 20))
, uploadLogsToBinaryCache(config->getBoolOption("upload_logs_to_binary_cache", false))