Repository navigation
Add opt-in background refresh before cache expiry - #5
Conversation
| (ser/deserialize result {:handlers (:read handlers)})))) | ||
| {:id id | ||
| :value (ser/deserialize result {:handlers (:read handlers)}) | ||
| :refresh? (pos? refresh?)}))) |
There was a problem hiding this comment.
How does the hit get correctly written now? ("Correctly": cognizant of the extremely tricky synchronization issue previously mentioned.)
There was a problem hiding this comment.
The hit update moved from get! to cached; it still uses the existing writer queue:
get!finishes the SELECT onread-connand returns the entry ID and value.- On that hit,
cachedqueueshit!with the entry ID, before scheduling any refresh. - The writer thread executes it on its separate
write-conn.hit!issues a single UPDATE:SET hits = hits + 1, last_hit = unixepoch() WHERE id = ?.
The synchronization property is that the read transaction is never upgraded to a write transaction. The SELECT has finished before the hit is enqueued; the writer starts with an UPDATE on a different connection. Increments happen in SQL, so concurrent hits do not overwrite each other's counts. This preserves the existing solution to the lock-upgrade problem; moving the enqueue into cached does not change it.
The queue is per cache wrapper, not globally per database file. Different wrappers can still contend for SQLite's writer lock; the configured busy timeout handles that contention without a read-to-write upgrade. Hit bookkeeping remains asynchronous and is not atomic with lookup: eviction can make a queued hit a no-op, and refresh resets the entry's counters.
sqlite-cache was originally conceived for calls that are fast but expensive, such as data API calls charged by the credit and LLM invocations. Avoiding repeated calls reduces cost. When the underlying call is slow, whether or not it is expensive, blocking on expiry also makes callers wait. Callers need a way to proactively keep the cache warm so an acceptable cached value remains available while its replacement is computed.
This change provides that lever through opt-in, read-triggered background refresh before expiry. It warms entries that are being read; it does not schedule refreshes for idle entries.
Reads can now return a cached value while refreshing it in the background before expiry. Set
:refresh-afterto an age in seconds or{:before-max-age seconds}to leave a computation margin before the hard age limit. Omitting it preserves blocking refresh on expiry; existing database files need no migration.Background refreshes and blocking misses use the same per-key computation. Expired callers wait for an in-progress refresh, and the computation remains shared until its asynchronous database write completes. Background computation failures log a warning and leave the old value available until its original expiry; later reads can retry. Refreshes run on Clojure's future pool and start on reads.
Lookup now distinguishes missing entries from cached
nilandfalse. The refresh option is normalized at cache construction and assumes the documented input contract. README examples describe both forms, failure behavior, and the existing persisted expiry settings.Validation:
bb testpasses (68 tests, 438 assertions);bb lintandgit diff --checkpass. Added tests cover both configuration forms, exact refresh/expiry boundaries, concurrent reads, pending writes, failures and retries, falsey values, transformed cache keys, caller bindings, independent keys, and existing persisted entries.Review fixes restore the SQLite concurrency explanation, shorten the read-triggered refresh documentation, and replace timing-based tests with a controlled background job queue and the existing virtual clock. A promise-coordinated real-thread test checks joining a running refresh without sleeps or timed assertions.
Independent review found that separately named caches using the same function and arguments could share an in-flight computation, leaving one persisted entry unrefreshed. The computation key now includes
:func-name. The regression test fails with the original key and passes with the fix.