Skip to content

[Draft] Size limits: dimension guard, 64-bit counts and offsets, device-side creation, vecmat - #42

Draft
AdvaitTahilyani wants to merge 6 commits into
scale/24-index-data-movementfrom
scale/draft-size-limits
Draft

AdvaitTahilyani wants to merge 6 commits into
scale/24-index-data-movementfrom
scale/draft-size-limits

Conversation

@AdvaitTahilyani

Copy link
Copy Markdown

Draft: the review-fix round and the merge with #24 have passed CPU tests but have not been validated on GPU.

  • A dimension over 2^31−1 raises a clear error, including in reshape.
  • Element counts, offsets and strides are 64-bit, so a single tile can hold more than 2^32 elements; tested on 4 A100s at 8.6e9 elements.
  • meshgrid, full and linspace build on the device.
  • Row-stepped matvec reads in place.
  • vec @ mat is a transposed GEMV, so there's no cached transpose.
  • einsum ij,i->j no longer builds an n² temporary.

meshgrid on 1 A100 at N=8000: 1.89 s / 1.4 GB on the client → 4 ms.

🤖 Generated with Claude Code

…creation, vecmat

- Frontend: reject dimensions over 2**31-1 with a clear error
- Backend: 64-bit element indices for chare tiles over 2**31 elements
- Creation: meshgrid, full and linspace without client-side grids
- matvec: row-stepped matrix views in place; vec @ mat as a transposed MATMUL
- einsum: vector-matrix contraction over the leading axis without an n^2 temporary
- Tests: test_limits (dimension guard, device creation, einsum/vecmat, stepped matvec, >2**31 tiles)
- Frontend: guard reshape targets over 2**31-1; clamp single-element slice steps
- Backend: matvec review fixes
- Indexing: unit step for slices of at most one element next to index arrays

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@adityapb adityapb left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: not ready (CPU-only validation per the description, plus one fix).

This is a static review. Line numbers are at this PR's head.

These are correct:

  • linspace follows NumPy's formula, including array endpoints, num=1, endpoint and float32.
  • meshgrid matches NumPy for xy, ij and sparse.
  • vecmat and row-stepped matvec: the message counts agree across sides.
  • einsum ij,i->j.
  1. [minor] The new step clamp overflows 32-bit region sizes for dimensions over 2^30.
    • Cause: _parse_key (charmnumeric.py:505) and compose (:379) set step = stop - start when the step exceeds the extent. ArrayRegion::size then computes (stop - start + step - 1) / step in int (array_region.hpp:110).
    • Example: a = cnp.zeros(2**31 - 1, np.int8); a[5::2**40].get(). The count overflows to 0 or a negative value, so you get a wrong or empty result, or a hung get. Before the clamp, this input failed cleanly.
    • Fix: normalise single-element slices to [start, start+1) with step 1, as indexing._items does, and/or compute sizes in int64.
    • The new test uses an 8×5 array, so it can't hit this.
  2. Nit: the 3-D MATMUL path ignores the new transpose flag (executor_matmul.cpp:166). It's unreachable from the frontend today, but an abort would stop a future caller silently getting A·x instead of x·A.
  3. Nit: the meshgrid docstring (charmnumeric.py:3040) still says device inputs must come from arange or linspace.

Rebase: this is 3 PRs behind (#40, #43, #44). The merge is clean, and I found no semantic conflicts.

AdvaitTahilyani and others added 2 commits November 9, 2026 22:34
…on counts

- _parse_key and ArrayRegion.compose turn a slice that selects at most one
  element (step >= stop - start) into [start, start + 1) with step 1,
  instead of clamping the step to the extent: a[5::2**40] on a 2**31-1
  array no longer overflows 32-bit region sizes.
- ArrayRegion::size(d) and CrossSetRegionMap's count compute in int64.
- MATMUL: the 3-D path aborts on the transpose flag (x @ A) instead of
  computing A @ x.
- meshgrid docstring: any input works.
- test_limits: a 'long' section on a (2**31 - 1,) int8 array with steps
  over 2**30, composed views and stepped writes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… overflow

The new (2**31 - 1,) test hung or crashed in places that compute past the
last element in int:
- (c + 1) * tile for the last chare's end (2**31 with 2**29 tiles): a
  tile_end() helper in ArrayDecomp, the shape, transfer, matmul and
  matmatmul executors; num_chares and the solver's tile counts in int64;
  balanced_tile's rounded-up candidate and pow2 doubling no longer wrap
  (a (2**31 - 1,) array got a negative tile at P > 1).
- view_block / the shape executor's view bounds, the lattice-aligned
  covered stop in subtract(), decompose()'s chare range, the matvec
  partial-result send ends: int64; map() and align_fragments() clamp
  stops past the last element to INT_MAX.
- Strided odometers (set_region fill, region copies) advanced idx past
  2**31 with steps over 2**30 and wrote out of bounds; they now test
  stop - idx > step before adding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AdvaitTahilyani

Copy link
Copy Markdown
Author
  1. [minor] The step clamp overflows 32-bit region sizes for dimensions over 2^30: fixed in 64079a1.
    • _parse_key and ArrayRegion.compose now turn a slice that selects at most one element (step >= stop - start) into [start, start + 1) with step 1.
    • ArrayRegion::size(d) and the CrossSetRegionMap count now compute in int64.
    • New test_limits section long, on by default: a (2**31 - 1,) int8 array, reads with steps of 2**40, 2**30 + 1, 2**31 - 2 and 2**31 - 1, composed views, stepped writes, and a sum.
    • The normalisation was also fuzzed against Python range slicing: 20k slices and composes, 0 mismatches.
    • The new test also found pre-existing int overflows near 2^31 on this head, all fixed in a49f89d:
      • sum over a (2e9,) array hung, because the last chare's end (c + 1) * tile reached 2^31. A tile_end() helper now covers ArrayDecomp and the shape, transfer, matmul and matmatmul executors.
      • At P > 1, balanced_tile gave a (2**31 - 1,) array a negative tile, so writes were lost and reads returned zeros.
      • Reads with steps over the tile size returned zeros. decompose()'s chare range, subtract()'s covered stop, view_block, and the stops in map() and align_fragments() overflowed.
      • A stepped write with a step over 2^30 segfaulted: the strided odometers pushed the index past 2^31.
  2. Nit, the 3-D MATMUL path ignores the transpose flag: fixed in 64079a1. It now calls CkAbort when the flag is set.
  3. Nit, stale meshgrid docstring: fixed in 64079a1.

Not rebased.

Validated:

  • nc and GPU builds of this head.
  • CPU regress: orig 15/15, ports 25/25 and ops 35/35 at both +p1 and +p2.
  • test_limits long also passes with CNUM_ODF=1 and at +p3.
  • GPU validation is submitted but has not run yet. Jobs 23703722 (1 A100) and 23703723 (4 A100s) run the ops suite and LIMITS_BIG=1 (long and big sections, CNUM_ODF=1).

Ready for review: not yet. It is ready once the A100 runs above pass; a49f89d is a wider overflow fix than the review asked for and should get a look of its own.

AdvaitTahilyani and others added 2 commits November 10, 2026 16:19
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AdvaitTahilyani

Copy link
Copy Markdown
Author

GPU validation (4 A100s) hit its time limit at 25/37 op tests. The failures include test_limits, test_index_scale, test_indexing and test_jit_params; I haven't separated real failures from CCS connection errors under load yet. Still not ready; needs a clean rerun on the merged head.

@AdvaitTahilyani

Copy link
Copy Markdown
Author

Inconclusive GPU run (job 23703723): not a bug in this PR. No code change.

GPU validation of 02d7494 (binaries and JIT/CUDA caches in node-local /tmp, one draft per server set):

  • 1 GPU (A40, +p1): orig 15/15, ports 25/25, ops 36/36. test_limits with LIMITS_BIG=1 CNUM_ODF=1 (long + big, a 2**31-element int8 2-D array) passes.
  • 4 GPUs (A40, srun +p4, one GPU per process): orig 15/15, ports 25/25, ops 36/36. LIMITS_BIG=1 long + big passes.

Ready for review: yes from my side. a49f89d (the wider overflow fix) still deserves its own look, as noted before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants