Skip to content

Add opt-in background refresh before cache expiry - #5

Merged
lvh merged 3 commits into
mainfrom
refresh-after
Oct 17, 2026
Merged

lvh merged 3 commits into
mainfrom
refresh-after

Conversation

@lvh

@lvh lvh commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

sqlite-cache was originally conceived for calls that are fast but expensive, such as data API calls charged by the credit and LLM invocations. Avoiding repeated calls reduces cost. When the underlying call is slow, whether or not it is expensive, blocking on expiry also makes callers wait. Callers need a way to proactively keep the cache warm so an acceptable cached value remains available while its replacement is computed.

This change provides that lever through opt-in, read-triggered background refresh before expiry. It warms entries that are being read; it does not schedule refreshes for idle entries.

Reads can now return a cached value while refreshing it in the background before expiry. Set :refresh-after to an age in seconds or {:before-max-age seconds} to leave a computation margin before the hard age limit. Omitting it preserves blocking refresh on expiry; existing database files need no migration.

Background refreshes and blocking misses use the same per-key computation. Expired callers wait for an in-progress refresh, and the computation remains shared until its asynchronous database write completes. Background computation failures log a warning and leave the old value available until its original expiry; later reads can retry. Refreshes run on Clojure's future pool and start on reads.

Lookup now distinguishes missing entries from cached nil and false. The refresh option is normalized at cache construction and assumes the documented input contract. README examples describe both forms, failure behavior, and the existing persisted expiry settings.

Validation: bb test passes (68 tests, 438 assertions); bb lint and git diff --check pass. Added tests cover both configuration forms, exact refresh/expiry boundaries, concurrent reads, pending writes, failures and retries, falsey values, transformed cache keys, caller bindings, independent keys, and existing persisted entries.

Review fixes restore the SQLite concurrency explanation, shorten the read-triggered refresh documentation, and replace timing-based tests with a controlled background job queue and the existing virtual clock. A promise-coordinated real-thread test checks joining a running refresh without sleeps or timed assertions.

Independent review found that separately named caches using the same function and arguments could share an in-flight computation, leaving one persisted entry unrefreshed. The computation key now includes :func-name. The regression test fails with the original key and passes with the fix.

Comment thread src/com/latacora/sqlite_cache/core.clj
Comment thread test/com/latacora/sqlite_cache/refresh_test.clj Outdated
Comment thread test/com/latacora/sqlite_cache/refresh_test.clj Outdated
Comment thread test/com/latacora/sqlite_cache/refresh_test.clj Outdated
Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread src/com/latacora/sqlite_cache/core.clj Outdated
Comment thread src/com/latacora/sqlite_cache/core.clj
(ser/deserialize result {:handlers (:read handlers)}))))
{:id id
:value (ser/deserialize result {:handlers (:read handlers)})
:refresh? (pos? refresh?)})))

@lvh lvh Oct 15, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does the hit get correctly written now? ("Correctly": cognizant of the extremely tricky synchronization issue previously mentioned.)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hit update moved from get! to cached; it still uses the existing writer queue:

  1. get! finishes the SELECT on read-conn and returns the entry ID and value.
  2. On that hit, cached queues hit! with the entry ID, before scheduling any refresh.
  3. The writer thread executes it on its separate write-conn. hit! issues a single UPDATE: SET hits = hits + 1, last_hit = unixepoch() WHERE id = ?.

The synchronization property is that the read transaction is never upgraded to a write transaction. The SELECT has finished before the hit is enqueued; the writer starts with an UPDATE on a different connection. Increments happen in SQL, so concurrent hits do not overwrite each other's counts. This preserves the existing solution to the lock-upgrade problem; moving the enqueue into cached does not change it.

The queue is per cache wrapper, not globally per database file. Different wrappers can still contend for SQLite's writer lock; the configured busy timeout handles that contention without a read-to-write upgrade. Hit bookkeeping remains asynchronous and is not atomic with lookup: eviction can make a queued hit a no-op, and refresh resets the entry's counters.

@lvh
lvh merged commit 2431087 into main Oct 17, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant