Repository navigation
[Draft] Size limits: dimension guard, 64-bit counts and offsets, device-side creation, vecmat - #42
AdvaitTahilyani wants to merge 6 commits into
Conversation
…creation, vecmat - Frontend: reject dimensions over 2**31-1 with a clear error - Backend: 64-bit element indices for chare tiles over 2**31 elements - Creation: meshgrid, full and linspace without client-side grids - matvec: row-stepped matrix views in place; vec @ mat as a transposed MATMUL - einsum: vector-matrix contraction over the leading axis without an n^2 temporary - Tests: test_limits (dimension guard, device creation, einsum/vecmat, stepped matvec, >2**31 tiles) - Frontend: guard reshape targets over 2**31-1; clamp single-element slice steps - Backend: matvec review fixes - Indexing: unit step for slices of at most one element next to index arrays Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
adityapb
left a comment
There was a problem hiding this comment.
Verdict: not ready (CPU-only validation per the description, plus one fix).
This is a static review. Line numbers are at this PR's head.
These are correct:
- linspace follows NumPy's formula, including array endpoints,
num=1,endpointand float32. - meshgrid matches NumPy for xy, ij and sparse.
- vecmat and row-stepped matvec: the message counts agree across sides.
- einsum
ij,i->j.
- [minor] The new step clamp overflows 32-bit region sizes for dimensions over 2^30.
- Cause:
_parse_key(charmnumeric.py:505) andcompose(:379) setstep = stop - startwhen the step exceeds the extent.ArrayRegion::sizethen computes(stop - start + step - 1) / stepinint(array_region.hpp:110). - Example:
a = cnp.zeros(2**31 - 1, np.int8); a[5::2**40].get(). The count overflows to 0 or a negative value, so you get a wrong or empty result, or a hung get. Before the clamp, this input failed cleanly. - Fix: normalise single-element slices to
[start, start+1)with step 1, asindexing._itemsdoes, and/or compute sizes in int64. - The new test uses an 8×5 array, so it can't hit this.
- Cause:
- Nit: the 3-D MATMUL path ignores the new transpose flag (
executor_matmul.cpp:166). It's unreachable from the frontend today, but an abort would stop a future caller silently gettingA·xinstead ofx·A. - Nit: the meshgrid docstring (
charmnumeric.py:3040) still says device inputs must come from arange or linspace.
Rebase: this is 3 PRs behind (#40, #43, #44). The merge is clean, and I found no semantic conflicts.
…on counts - _parse_key and ArrayRegion.compose turn a slice that selects at most one element (step >= stop - start) into [start, start + 1) with step 1, instead of clamping the step to the extent: a[5::2**40] on a 2**31-1 array no longer overflows 32-bit region sizes. - ArrayRegion::size(d) and CrossSetRegionMap's count compute in int64. - MATMUL: the 3-D path aborts on the transpose flag (x @ A) instead of computing A @ x. - meshgrid docstring: any input works. - test_limits: a 'long' section on a (2**31 - 1,) int8 array with steps over 2**30, composed views and stepped writes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… overflow The new (2**31 - 1,) test hung or crashed in places that compute past the last element in int: - (c + 1) * tile for the last chare's end (2**31 with 2**29 tiles): a tile_end() helper in ArrayDecomp, the shape, transfer, matmul and matmatmul executors; num_chares and the solver's tile counts in int64; balanced_tile's rounded-up candidate and pow2 doubling no longer wrap (a (2**31 - 1,) array got a negative tile at P > 1). - view_block / the shape executor's view bounds, the lattice-aligned covered stop in subtract(), decompose()'s chare range, the matvec partial-result send ends: int64; map() and align_fragments() clamp stops past the last element to INT_MAX. - Strided odometers (set_region fill, region copies) advanced idx past 2**31 with steps over 2**30 and wrote out of bounds; they now test stop - idx > step before adding. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Not rebased. Validated:
Ready for review: not yet. It is ready once the A100 runs above pass; a49f89d is a wider overflow fix than the review asked for and should get a look of its own. |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
GPU validation (4 A100s) hit its time limit at 25/37 op tests. The failures include |
|
Inconclusive GPU run (job 23703723): not a bug in this PR. No code change.
GPU validation of 02d7494 (binaries and JIT/CUDA caches in node-local /tmp, one draft per server set):
Ready for review: yes from my side. a49f89d (the wider overflow fix) still deserves its own look, as noted before. |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Draft: the review-fix round and the merge with #24 have passed CPU tests but have not been validated on GPU.
vec @ matis a transposed GEMV, so there's no cached transpose.ij,i->jno longer builds an n² temporary.meshgrid on 1 A100 at N=8000: 1.89 s / 1.4 GB on the client → 4 ms.
🤖 Generated with Claude Code