:transaction strategy: uncommitted data disappears after perform_enqueued_jobs on Rails ≥ 7.2 #730

Below is summary of bug I have found in my company's app I work for. It is related to how database cleaner and rails internals handles the connections to database. I used AI to help investigating the problem but I verified (as far as I understand it) and reproduced the problem. Below is the summary of but I prepared a detailed rails app (see its' README) with instruction how to reproduce it https://github.com/radarek/database-cleaner-connection-bug.

Summary

On Rails ≥ 7.2, the :transaction strategy can intermittently cause ActiveRecord::RecordNotFound (and lock-wait timeouts on writes): a row created inside a test becomes invisible after an ActiveJob runs inline (perform_enqueued_jobs).

Reproduction app: https://github.com/radarek/database-cleaner-connection-bug (Rails 8.1, MySQL/trilogy, Minitest). for i in {1..100}; do bin/rails test; done should reproduce the problem at least once

Cause

The :transaction strategy opens its wrapping transaction on a leased connection:

connection = ActiveRecord::Base.lease_connection
connection.begin_transaction(joinable: false)

Running any ActiveJob wraps execution in Rails.application.reloader.wrap { … }. On completion, Rails' AR executor hook (ConnectionPool::ExecutorHooks.complete) sees a connection whose transaction is not joinable and releases it back to the pool — while it still holds the open transaction and all of the test's uncommitted rows.

The next query then leases a connection again. If the pool hands it a different physical connection (e.g. after the background Reaper reclaims a connection and reorders the LIFO free list), that connection is a separate DB session that can't see the uncommitted data → RecordNotFound. This is why it's intermittent and why it only shows up alongside other specs that grow the pool past one connection.

Suggested fix

On Rails ≥ 7.2, pin the connection instead of leasing it:

pool.pin_connection!(...)   # setup
pool.unpin_connection!      # teardown

A pinned connection is immune to the executor hook (checkout always returns the pinned connection), so the swap can't happen. This is exactly what Rails' own transactional fixtures do — and the pinned variant in the repro app passes 100% of runs.

Full mechanism write-up (LIFO free list, orphaned connections, Reaper timing) is in the repo README.