Repository navigation
fix(kvdb): improve power-loss recovery for sector headers and KV allocation - #440
Open
brightness-810 wants to merge 3 commits into
Open
brightness-810 wants to merge 3 commits into
brightness-810 wants to merge 3 commits into
Conversation
掉电发生在扇区更新为 USING、首条 KV 尚未写入之间时, 可能留下扇区头有效但数据区全 FF 的状态。 恢复 PRE_DELETE 记录时,内部查找可能将已分配的空白目标 误判为损坏 KV 并写入 ERR_HDR。随后搬移仍在同一地址提交 WRITE,并将旧副本标记为 DELETED,导致该 key 无法正常查询。 在空间计算和内部 KV 遍历入口检查整个数据区: - 空间计算遇到全 FF 数据区时,保留首个空闲地址和完整容量 - 内部遍历跳过数据区全 FF 的 USING/FULL 扇区 - 非空数据区保持原有处理,不改变存储格式或增加擦除
An interrupted first KV write can leave the trailing-FF boundary inside its status table. Caching this boundary as free space allows recovery to allocate a replacement inside the incomplete header. A subsequent search can mark the residual header ERR_HDR at a byte that also belongs to the replacement's DELETED slot. Committing WRITE does not clear that marker, and deleting the source leaves no readable copy. Advance the initial cached free address past the complete first KV header when the FF boundary falls within its status-table interval. Keep the cache-hit condition, capacity calculation, and on-flash format unchanged, without adding flash reads or writes.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Fix three KVDB issues observed during power-loss recovery:
valid header remains. Precheck headers before modification, preserve
valid-header sectors, and reject fully invalid partitions without
writes or erases.
sector space calculation and internal USING/FULL sector iteration.
Preserve the correct free address and capacity.
header to prevent overlapping KV status tables.
Recovery-policy change: Invalid-header sectors are erased even when
automatic formatting is disabled. Unique data in non-empty affected
sectors may be lost. Earlier repairs are not rolled back if a later
repair fails.
The blank-data guards and allocation-boundary adjustment reuse the
existing FF-boundary helper and do not themselves add erases. Existing
ERR_HDR/DELETED records are not resurrected.
The helper ignores underlying read errors. These changes do not establish
ECC safety or comprehensive GC safety.
Related issue
Closes #439
Closes #441
Closes #442
Test closure
tests/fdb_kvdb_tc.cortests/fdb_tsdb_tc.cUTEST_UNIT_RUN(...)list oftestcase()Validation currently uses a standalone host RAM-flash harness. The cases
have not yet been ported to the upstream utest files or registered with
UTEST_UNIT_RUN(...), so the corresponding items remain unchecked.
This is a behavior-changing bug fix and requires regression tests.
Local evidence:
followed by a failed key lookup. Its complete 32,768-byte output matched
the captured hardware failure image byte for byte.
with KV/sector caches both disabled and enabled: 18 replay executions.
EMPTY-sector control, blank-sector space calculation, internal lookup
over a blank FULL sector, first-KV creation, a non-erased final byte,
a partially programmed header, a later valid KV behind an erased first
slot, and preservation of an already-failed snapshot.
generated payload, write/erase counts, and readability after reopening.
space recalculation and advanced the free address by one KV header.
This supports retaining the space-calculation guard.
Remaining work:
in testcase().
including actual GC triggering, allocation exhaustion, write-granularity
crossing, and initialization I/O failures.
Write granularity covered
FDB_WRITE_GRAN= 1 / 8 / 32 / 64 / 128 / 256Data layout and compatibility
inc/flashdb.hchanged, compatibility impact:docs/plusdocs/zh-cn/are updatedNo changes to persistent status encoding, public structures, APIs, or
configuration macros. No data migration is required.
This fix prevents erroneous status programming into blank data areas.
It does not convert existing ERR_HDR or DELETED records back to WRITE.
Power-loss semantics
fdb_xxx_deinit->fdb_xxx_initreboot simulationThese guards affect paths used by allocation and startup recovery.
Although the guards do not write flash themselves, they prevent
read_kv() from programming ERR_HDR and therefore affect persistent
recovery behavior.
Reboot simulation currently runs in the standalone host harness, not
the upstream utest suite:
sector with a fully erased data area in RAM.
no ERR_HDR programming, and no erases.
no additional writes or erases occur.
The original hardware image was captured after recovery and verification
had failed. RAM reconstruction establishes a sufficient mechanism that
fully reproduces that image; it does not identify the unique instruction
or execution path at which the original power loss occurred.
These results do not replace a post-fix hardware power-cut campaign and
do not establish safety for every interruption window or GC source erase.