Commit Graph
16 Commits
Author SHA1 Message Date
eldritch horrors 2f16cf6014 treewide: lint syscalls for c string safety
Change-Id: I3caf476e59dcb7899ac5a3d83dfa3fb7ceaaabf0
2025-10-10 16:13:54 +00:00
1e34c37477 flake/inputs: 24.11-small -> 25.05-small
We upgrade to 25.05 release, which contains the curl commit
https://github.com/curl/curl/commit/5fbd78eb2dc4afbd8884e8eed27147fc3d4318f6
done in
https://github.com/NixOS/nixpkgs/pull/396200#issuecomment-2795944006.

This fixes HTTP transfers generating arbitrary errors and possibly
failing unusually.

Users who are already depending on 25.05-small or a recent unstable
already had the fix.

Special mention to the Linux kernel who gave me the opportunity to get
on a 24 hours bisection side quest to fix the local release engineering
test.

Special thanks to everyone who had to endure me ranting.

Change-Id: I866caf65d5ea103f1fa5eccd57df8031c9eacda0
Co-authored-by: eldritch horrors <pennae@lix.systems>
Co-authored-by: helle <helle@h3l.li>
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-13 18:15:47 +00:00
eldritch horrors 549f385d40 libstore: drastically shorten sqlite busy timeout
this is one of the cases retrySQLite is supposed to handle. with busy
timeouts set at the connection level we may instead wait for up to an
hour to acquire locks before actually falling back to the retrySQLite
loop, blocking an entire thread in the process. this isn't painful in
the current system that uses dedicated threads for everything, but an
async executor actually suffers. since we use sqlite in WAL mode, and
WAL mode still allows reads to happen concurrently with writes, we'll
want to not block an executor on sqlite write locks. (note that while
we do have a mutex around db access that mutex is only held inside of
retrySQLite callbacks, so it will be released where we'd block today)

do note that we don't disable the timeout completely since that could
drastically reduce write performance. multiple daemon processes using
the same database file will naturally conflict, but each write is not
usually that expensive. going to sleep immediately instead of waiting
a bit can greatly impact throughput. waiting for 50ms should be fine.

Change-Id: I2d8e36ed7a0183bbc213490ebde2c79fc6e2f4b2
2025-03-05 17:34:25 +01:00
eldritch horrors 044b4c4500 libutil: split retrySQLite into sync and async
also mark the sync version as NeverAsync. a blocking wait on sqlite
locks in a coroutine may never finish if it's a different coroutine
on the same executor that is holding the lock, not another process.

this propagates to the sqlite core interface, but no further. we'll
assume that caches do not block on a database for very long, and we
can't reasonably propagate never-async-ness out of stores unless we
touch everything we'd touch for the async transition, again, twice.
store code already assumes that it can block for however long it'll
feel like that moment. we keep thread pools around for this reason.

Change-Id: I62f77e1ac333cbe2e4e646dbcb1571463f2cf3fc
2025-02-22 17:59:54 +00:00
Lily Ballard 98ccb179f9 libstore: use BEGIN IMMEDIATE SQLite transactions
Upgrading a read transaction to a write transaction will immediately
fail if another connection/process started a write transaction in the
meantime. For transactions that we expect to perform a write after a
read, starting with `BEGIN IMMEDIATE` gives us a write transaction
immediately instead of beginning with a read transaction and upgrading
it.

Change-Id: I82b8a3843382940a4fa4ffe45f30b365b690ff86
2025-02-17 19:07:59 -08:00
Lily Ballard ed03c3985d libstore: use monotonic clock for SQLITE_BUSY warnings
Using time_t meant a warning could be printed arbitrarily soon depending
on when the second rolled over relative to when the SQLite transaction
was attempted.

Change-Id: I93401aea65f90e37cd450a293c4dd4d64fba7628
2025-02-16 22:48:45 -08:00
eldritch horrors 5b98b75f74 libstore: extend retrySQLite for promise blocks
Change-Id: I91c8aad9e49ba3d470c5d6270b4e23da2501a0b8
2025-02-16 22:20:33 +00:00
Lily Ballard a258f78736 libstore: make SQLite busy back-off logic portable
Use C++ standard library not Unix functions for sleeping and randomness.

Also change the PRNG seed from `clock()` to `std::random_device` because
there's no good reason not to and this avoids two threads seeding their
PRNG the same.

Upstream-PR: https://github.com/NixOS/nix/pull/10399
Change-Id: I8424e7406366ea8e825d3e93dbb261ce60e3a37f
2025-01-31 21:27:48 -08:00
eldritch horrors 8897c1af45 libstore: hide raw sqlite C api
this is, once again, what wrappers are for in the first place.

Change-Id: Ied8cb7384f561a345e457ccc977ba98e50dba207
2025-01-20 23:07:44 +01:00
eldritch horrors 509cc7d348 libstore: make SQLite resource-safe
Change-Id: I59133503b4b10e4ee5dc94b8c1da4ecba5262537
2025-01-20 23:07:44 +01:00
eldritch horrors d5dc8361f5 libstore: create statement object from SQLite instance
it's just cleaner this way, same as with transactions.

Change-Id: Ibe8fe0bb3d5846f7b7b51e3454bd2fde44d4a5bb
2025-01-20 23:07:44 +01:00
eldritch horrors 57f5a61ab4 libstore: make SQLiteStmt resource-safe
these too could be copied with unexpected side-effects.

Change-Id: I5594660833f53fa8c3b3b6fbe762782a580c2ee2
2025-01-20 23:07:44 +01:00
eldritch horrors 9b57d05def libstore: don't use raw sqlite3 calls outside of wrapper
avoiding them that is literally what a wrapper is for.

Change-Id: I057701842f3d1aac465762ef7c675cc730cfe503
2025-01-20 23:07:44 +01:00
eldritch horrors 8596da2fca libstore: make SQLiteTxn resource-safe
it was possible to copy a transaction, with predictably bad effects.

Change-Id: Ibbcfa85a63b45296245223e26b9671c933467194
2025-01-20 23:07:44 +01:00
eldritch horrors 4c2ec80fcb libstore: don't require explicit retrySQLite return type
auto to the rescue!

Change-Id: I56c0fc7ccf2631bb7891a041bf228d32d198d219
2025-01-19 16:40:26 +01:00
eldritch horrorsandjade b0d7a81613 fix tooling after include reorganization
clangd broke because it can't look through symlinks. compile_commands
manipulation does not fix it, clangd configuration does not fix it, a
vfs overlay does not fix it, and while a combination of those can fix
it with a bind mount in place that's just too cursed to even consider

clangd bug: https://github.com/llvm/llvm-project/issues/116877

Change-Id: I8e3e8489548eb3a7aa65ac9d12a5ec8abf814aec
2024-11-19 22:55:32 +00:00