Skip to content

Add star and contributor thresholds to "Known Users" inclusion criteria #26803

Closed

Description

@kosiew

Context

Follow up to this @alamb's comment

Problem

The "Known Users" active list in docs/source/user-guide/introduction.md
uses one criterion, written only as an HTML comment:

<!-- "Active" means github repositories that had at least one commit in the last 6 months -->

That criterion alone has two gaps:

  • It is not enforced. As of 2026-10-27 (cutoff 2026-04-27), five listed
    entries already fail it.
  • It says nothing about adoption. A brand-new, single-author repo qualifies
    as soon as it has a commit. KalamDB (0 stars, 1 contributor, created
    2026-06-23) is listed alongside InfluxDB (31.8k stars, ~530 contributors).

Why it matters

The list reads as an endorsement of production-grade DataFusion users.
Stale or very small entries weaken that signal and make it harder to judge
PRs that add links. Without objective thresholds, every addition becomes a
judgement call for reviewers.

Invariant / desired behavior

An entry in the active list must link to a public GitHub repo that meets
all of the following:

  1. At least 1 commit in the last 6 months (existing rule).
  2. At least N stars (proposed: 100).
  3. At least M contributors (proposed: 5).

The criteria are stated in visible doc text, not only in an HTML comment.
Entries without a public repo follow an explicitly documented rule (see open
questions).

Current data (GitHub, 2026-10-27)

Stars come from the GitHub API. Contributors are the count shown in each
repo's GitHub sidebar ("Contributors N"), read on 2026-10-27. The REST
contributors endpoint is not used: it consistently undercounts (for
example, GlareDB 5 vs 19 and InfluxDB 314 vs 530), and with anon=1 it
double-counts people who commit from several emails.

Project Repo Stars Contributors Last commit Active (6 mo)
InfluxDB influxdata/influxdb 31,786 530 2026-10-22 ✅
OpenObserve openobserve/openobserve 22,502 176 2026-10-27 ✅
Cube Store cube-js/cube 21,084 407 2026-10-26 ✅
rerun.io rerun-io/rerun 11,650 206 2026-10-26 ✅
ParadeDB paradedb/paradedb 9,454 164 2026-10-27 ✅
Lance lance-format/lance (moved) 7,216 335 2026-10-27 ✅
GreptimeDB GreptimeTeam/greptimedb 6,767 170 2026-10-27 ✅
Arroyo ArroyoSystems/arroyo 5,068 45 2026-10-26 ✅
Restate restatedev/restate 4,610 57 2026-10-26 ✅
Sail lakehq/sail 3,474 58 2026-10-27 ✅
ROAPI roapi/roapi 3,435 41 2026-03-25 ❌
delta-rs delta-io/delta-rs 3,350 307 2026-10-25 ✅
Vortex vortex-data/vortex 3,284 92 2026-10-26 ✅
LakeSoul lakesoul-io/LakeSoul 3,260 27 2026-10-27 ✅
Spice.ai spiceai/spiceai 3,109 73 2026-10-26 ✅
HoraeDB apache/horaedb 2,823 45 2025-12-11 ❌ (archived)
Parseable parseablehq/parseable 2,564 66 2026-10-26 ✅
Kubeflow Trainer kubeflow/trainer 2,258 316 2026-10-27 ✅
Ballista apache/datafusion-ballista 2,157 379 2026-10-26 ✅
Feldera feldera/feldera 2,139 66 2026-10-27 ✅
Auron apache/auron 1,837 69 2026-10-26 ✅
CnosDB cnosdb/cnosdb 1,754 60 2026-10-23 ✅
Iceberg-rust apache/iceberg-rust 1,433 224 2026-10-27 ✅
ArkFlow arkflow-rs/arkflow 1,315 8 2026-10-25 ✅
Comet apache/datafusion-comet 1,304 164 2026-10-26 ✅
GlareDB GlareDB/glaredb 1,024 19 2025-11-05 ❌
dbt Fusion dbt-labs/dbt-fusion 729 106 2026-06-04 ✅ (barely)
Xorq xorq-labs/xorq 556 22 2026-10-27 ✅
SedonaDB apache/sedona-db 520 44 2026-10-27 ✅
Seafowl (EDB Postgres Lakehouse) splitgraph/seafowl 514 10 2025-01-08 ❌
VegaFusion vega/vegafusion 424 10 2026-03-13 ❌
qv timvw/qv 372 7 2026-09-23 ✅
Kamu kamu-data/kamu-cli 351 15 2026-10-26 ✅
datafusion-dft datafusion-contrib/datafusion-dft 205 8 2026-10-26 ✅
Infino infino-ai/infino 138 15 2026-10-27 ✅
Sleeper gchq/sleeper 107 23 2026-10-26 ✅
KalamDB jamals86/KalamDB 0 1 2026-06-23 ✅

No public GitHub repo (stars/contributors not measurable): datapress,
Funnel, hiop, Hotdata, IceGate, Massive.com, R2 Query Engine, Streamling,
Supermetal, Synnada, Telemetry.

Observations

  • Already fail the 6-month rule: ROAPI, HoraeDB (archived), GlareDB,
    Seafowl (backs the EDB Postgres Lakehouse entry), VegaFusion.
  • About to fail: dbt Fusion (last commit 2026-06-04, about 5 weeks from
    the cutoff).
  • Clear outlier: KalamDB (0 stars, 1 contributor, repo created
    2026-06-23).
  • Stars among active repos are mostly in the hundreds to thousands; the
    lowest are Sleeper (107), Infino (138), datafusion-dft (205).
  • Low contributor counts (<10): ArkFlow (8), datafusion-dft (8), qv (7), all
    active.

Threshold impact

Thresholds Removed from active list (beyond 6-month failures)
≥100 stars, ≥5 contributors (proposed) KalamDB
≥100 stars, ≥10 contributors KalamDB, ArkFlow, datafusion-dft, qv

Stale links found

  • Lance: lancedb/lance → lance-format/lance
  • HoraeDB: apache/incubator-horaedb → apache/horaedb (archived)
  • Restate: links to the org restatedev, not the repo restatedev/restate

Proposed direction

  1. Replace the HTML comment with a short visible paragraph that lists the
    three criteria and the exemption rule for entries without a public repo.
  2. Apply the criteria to the current list: move failing entries to "less
    active projects", and remove or move entries below the thresholds.
  3. Fix the stale links above.
  4. Optional: commit a small script (for example under dev/) that prints
    stars, contributors and last commit for every GitHub link in the section,
    so the list can be re-audited periodically.

Scope

In

  • Inclusion criteria text in docs/source/user-guide/introduction.md.
  • Re-categorizing current "Known Users" entries against the criteria.
  • Fixing the stale repo links listed above.

Out

  • The "Integrations and Extensions" / "Language Bindings" lists, unless
    maintainers want the same criteria there (follow-up).
  • Automated CI enforcement of the thresholds.
  • Judging projects on anything beyond the stated criteria.

Open questions

  • Final values for N (stars) and M (contributors). Proposed: 100 / 5.
  • Entries without a public repo (11 today): exempt them (e.g. as
    "commercial products using DataFusion"), list them separately, or require
    some other evidence of use (a blog post or docs page).
  • Should entries that fail the thresholds go to "less active projects" or be
    removed entirely?

Acceptance criteria

  • introduction.md states the activity, star and contributor criteria
    in visible text.
  • The rule for entries without a public GitHub repo is documented.
  • Every GitHub-linked entry in the active list meets all criteria as of
    the PR date.
  • ROAPI, HoraeDB, GlareDB, Seafowl/EDB Postgres Lakehouse and
    VegaFusion are no longer in the active list (unless they regain
    activity before merge).
  • Lance, HoraeDB and Restate links point to the current repos.

Activity

  1. alamb commented on Oct 27, 2026

    @alamb
    Contributor

    That criterion alone has two gaps:
    It is not enforced. As of 2026-10-27 (cutoff 2026-04-27), five listed
    entries already fail it.

    I agree it would be good to update the list

    Final values for N (stars) and M (contributors). Proposed: 100 / 5.

    This sounds good to me

    Should entries that fail the thresholds go to "less active projects" or be
    removed entirely?

    I vote "less active projects"

    Entries without a public repo (11 today): exempt them (e.g. as
    "commercial products using DataFusion"), list them separately, or require
    some other evidence of use (a blog post or docs page).

    I would personally prefer some evidence of use ("blog post or docs page")

    Thanks you @kosiew -- this looks good

  2. added a commit that references this issue on Nov 3, 2026
    05296c7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions