Skip to content

grist(update-choice-options): API envelope shape-trusted — get_widget_options/api die as raw Key/Type/Attribute/JSONDecode tracebacks on wrong-shape 200s, bypassing the #6588 exit-1 contract #6867

Open

Description

@dezren39

Summary

pm-stack/grist/update-choice-options.py guards the inner stored
widgetOptions shape (#6588, check_widget_options), but the Grist API
envelope around it in get_widget_options — and the JSON decode in
api() — are still shape-trusted. A wrong-shape 200 response dies as a
raw KeyError / TypeError / AttributeError / JSONDecodeError
traceback, even though the module's own contract (docstring, exit
codes) says an unexpected shape is an exit-1 failure reported on
stderr, and run() converts exactly the two named error classes
(TimeoutError, WidgetOptionsError, lines 221–226) to that exit.

Evidence (verified this run, laptop @ 65beb42, probe ~/git/fleet/probe-grist-envelope, real file verbatim, stubbed api transport)

get_widget_options (update-choice-options.py:136–144):

for col in data["columns"]:  # :138
    if col["id"] == column:  # :139
        raw = col["fields"].get("widgetOptions") or ""  # :140
        parsed = json.loads(raw)  # :144 — only JSONDecodeError is wrapped

Controls pass (empty widgetOptions → {}; valid JSON string →
{'choices': ['id']}). Every wrong shape below is a raw error:

payload result
{} (no columns) KeyError: 'columns'
[] (top-level list) TypeError: list indices must be integers
{"error": "not found"} KeyError: 'columns'
{"columns": "abc"} TypeError: string indices must be integers
{"columns": ["Area"]} TypeError: string indices must be integers
{"columns": [{"id": "Area"}]} (no fields) KeyError: 'fields'
fields: null AttributeError: 'NoneType' object has no attribute 'get'
fields: [] AttributeError: 'list' object has no attribute 'get'
widgetOptions already a dict (not a JSON string) TypeError: the JSON object must be str, bytes or bytearray, not dict
widgetOptions: 123 TypeError: ... not int
{"columns": null} TypeError: 'NoneType' object is not iterable

End-to-end: run([...]) with a {} response propagates
KeyError: 'columns' instead of returning 1 — the CLI traceback names
no endpoint, column, or field.

Second door, same class: api() (:133) returns
json.loads(raw) if raw and raw != "null" else None with no decode
guard — a non-JSON 200 body (e.g. an HTML proxy/error page) raises a
raw JSONDecodeError: Expecting value: line 1 column 1 (char 0)
(probe2 in the same probe dir), also uncaught by run().

Why these shapes are realistic: the columns endpoint is a server the
tool does not control — other Grist versions, an error object returned
with 200 by a proxy, or a column whose fields/widgetOptions were
written by another tool (the exact premise #6588 already accepted for
the inner value). Both the initial read and the post-PATCH read-back
verification go through get_widget_options, so a wrong-shape
read-back also crashes after a successful PATCH, reporting a
traceback for a change that actually landed.

Fix shape

Validate the envelope at the same boundary #6588 used for the inner
value: in get_widget_options, require data to be an object with a
columns list of objects each carrying an id and an object
fields; require a present widgetOptions to be a string before
json.loads. Raise WidgetOptionsError (or a sibling named error
that run() already converts) with messages naming the endpoint
payload part that was wrong. In api(), wrap the json.loads decode
failure the same way, naming method + path. No coercion, no PATCH on a
shape failure (the existing pre-PATCH ordering already guarantees
this for the first read).

Acceptance criteria

  • Each wrong-shape payload in the table above produces a named,
    human-readable error on stderr and exit 1 via run() — no
    traceback, no raw KeyError/TypeError/AttributeError.
  • A non-JSON 200 body from api() is reported as a named API
    failure (method + path), exit 1, not a raw JSONDecodeError.
  • Controls unchanged: empty widgetOptions → {}; valid stored
    options merge and read-back verification behave exactly as today.
  • A column genuinely absent from a well-shaped columns list
    still exits via the existing "column not found" path.
  • New cases land in pm-stack/grist/test_update_choice_options.py
    (stubbed transport, no live Grist), covering at least: missing
    columns, non-list columns, entry without fields,
    non-string widgetOptions, non-JSON response body.

Activity

  1. added
    kind/bugSomething isn't working (prefixed form of the classic bug label)
    task/small-fixQueue packet kind: bounded fix with reproduction and targeted tests
    source/creatorFiled by the new-issue-creator job
    on Nov 7, 2026
  2. developing-today commented on Nov 7, 2026

    @developing-today
    Contributor
  3. added
    priority/normalOrdinary priority (the default)
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 7, 2026
  4. dezren39 commented on Nov 7, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · filed

    • Agent: new-issue-creator (new-issue-creator-code-b)
    • Session / ID: cron new-issue-creator-code-b, run 2026-11-07 04:50 CST
    • Found via: filed this run
    • Task: pm-stack/grist/update-choice-options.py guards the inner stored widgetOptions shape (grist(update-choice-options): a non-list choices value is silently shredded or crashes the tool — a string choice becomes one-character choices and the read-back verification passes the corruption #6588) but not the API envelope: get_widget_options (lines 138-144) indexes data["columns"] / col["id"] / col["fields"] unguarded and api() (line 133) decodes JSON unguarded, so wrong-shape 200s die as raw Key/Type/Attribute/JSONDecode tracebacks and run() — which converts only TimeoutError/WidgetOptionsError to the documented exit 1 — propagates them. Verified with the real file verbatim on the laptop @65beb42c1a (probe ~/git/fleet/probe-grist-envelope, stubbed transport): controls OK; 11 wrong shapes all raw errors; run() propagated KeyError instead of returning 1.
    • Expectation: effort/s small-fix — validate the envelope at the get_widget_options boundary (object with a columns list of {id, fields-object} entries, widgetOptions a string before json.loads) and wrap api()'s decode failure, raising the named error run() already converts, with messages naming the wrong part; new cases in test_update_choice_options.py; queued for the repo worker.
    • Links: packet creator-b-2026-11-07-0450/packet-6867-small-fix.json (goal hidden_files)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    app/gristpm-stack/gristkind/bugSomething isn't working (prefixed form of the classic bug label)priority/normalOrdinary priority (the default)queue/queuedHas a queued task in the Supabase agent queuesource/creatorFiled by the new-issue-creator jobstatus/needs-triageNot yet triaged; needs a first looktask/small-fixQueue packet kind: bounded fix with reproduction and targeted tests

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions