Repository navigation
feat(admin)!: page the list of indexing requests - #832
Merged
Merged
Conversation
`get_all_indexing_requests` returned every request in 1 reply, which fails once the list outgrows the 10 MiB a reply may be. It now returns a page (1,000 by default, 5,000 at most) and the id to carry on from, so an older CLI can't read it. `indexings list` follows the pages.
A page is read 1 row past its size, and where the next page starts is settled before rows that fail to convert are dropped. Nothing tested that order, so losing either step would have ended listings early unnoticed. The steps now sit in 1 function with a test that fails without each.
The database test read the fixture's 3 requests, which are stored in id order, so the query would have passed it with no ORDER BY at all. It now also stores an older request last and expects it first, which only sorting by id can give.
The reply's `next` was described as absent on the last page, but it is sent as `null`, and nothing said a page can be short or empty and still have a `next`. A client written from the old wording could restart at the first page, or stop early.
Adding the page type and its helpers split the indexing request's doc comment from the type it describes, so the comment opened the page type's docs and the request had none.
…indexing-requests
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
get_all_indexing_requestsno longer returns every request in 1 reply, which would fail once the list outgrew the 10 MiB a reply may be. It returns a page, 1,000 requests by default and 5,000 at most, with optionalafter,limitandstatusinputs, anddipper-cli indexings listfollows the pages. Against 1,000,000 requests in a local database, a page deep in the list took 36 ms and the CLI listed them all in 29 MiB of memory.Important
Upgrade the CLI together with dipper. A CLI from before this change fails on
indexings listwith a parse error, not a partial list.