Skip to content

chore: optimze columnar vs row-wise storage in search and cache - #147

Merged
vcaesar merged 3 commits into
mainfrom
test-v2
Sep 18, 2026
Merged

vcaesar merged 3 commits into
mainfrom
test-v2

Conversation

@vcaesar

@vcaesar vcaesar commented Sep 17, 2026

Copy link
Copy Markdown
Owner

Please provide Issues links to:

  • Issues: #

Provide test code:

```Go
    
```

Description

...

- Add numeric_column_test.go and 5 others
- Update field.go and 20 others

Generated with Codg
Assisted-by: claude-fable-5-1 via Codg <codg@atomai.cc>
- Update config.go

Generated with Codg
Assisted-by: claude-fable-5-1 via Codg <codg@atomai.cc>
Copilot AI lite review requested due to automatic review settings September 17, 2026 05:41
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A new regression test fails after deletion, and scratch/reset changes can corrupt nested aggregations or retain substantial memory.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds typed numeric column reads and configurable ICE stored-chunk caching, with related benchmarks, compatibility handling, and tests.

Changes:

  • Routes numeric, date, and geo values through typed doc-value visitors.
  • Adds reusable numeric prefix encoding and match scratch buffers.
  • Adds stored-chunk cache configuration and ICE version compatibility.
File summaries
File Description
stats_bench_test.go Adds columnar aggregation benchmarks.
search/source.go Reads typed numeric values.
search/search.go Stores typed values and scratch buffers.
search/aggregations/terms.go Optimizes bucket lookup.
numeric/prefix_coded.go Adds append-based encoding.
numeric_column_test.go Tests numeric-column aggregations.
index/snapshot.go Supports typed document-value visits.
index/config.go Adds cache configuration and plugin registration.
index/config_test.go Tests cache configuration.
go.sum Updates dependency checksums.
go.mod Updates segment dependencies.
field.go Exposes numeric field values.
config.go Exposes cache configuration publicly.
Review details
  • Files reviewed: 12/13 changed files
  • Comments generated: 7
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread numeric_column_test.go
Comment on lines +114 to +115
if got := aggs.Metric("min"); got != -201 { // -100.5 * 2
t.Fatalf("min %v", got)
Comment thread search/search.go Outdated
Comment on lines +163 to +167
dm.termScratch = dm.termScratch[:0]
dm.termBytes = dm.termBytes[:0]
for _, n := range dm.docNumbers {
if n.field != field {
continue
Comment thread search/source.go Outdated
Comment on lines +94 to +98
rv := match.numScratch[:0]
if match.hasDocNumbers(string(f)) {
for _, n := range match.docNumbers {
if n.field == string(f) {
rv = append(rv, numeric.Int64ToFloat64(n.value))
Comment thread search/search.go
Comment on lines +210 to +211
for k, v := range dm.docValues {
dm.docValues[k] = v[:0]
Comment thread stats_bench_test.go Outdated
Comment thread numeric_column_test.go
Comment thread stats_bench_test.go Outdated
- Add numeric_nested_test.go, value_lifetime_test.go
- Update numeric_column_test.go and 4 others

Generated with Codg
Assisted-by: gpt-6-astra via Codg <codg@atomai.cc>
@vcaesar
vcaesar merged commit 1b1c911 into main Sep 18, 2026
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants