Repository navigation
Distribution.from_shape is producing localarrays with size 0 #442
Description
With 4 engines:
In [1]: from distarray.dist import Context, Distribution
In [2]: context = Context()
In [3]: distribution = Distribution.from_shape(context, (9, 8), dist=('b', 'n'))
In [4]: distribution.localshapes()
Out[4]: [(3, 8), (3, 8), (3, 8), (0, 8)]This appears to be from this line.
The size of this axis of the grid_shape is set to comm_size, which doesn't divide into the size of the corresponding axis of the DistArray.
Activity
cowlicks commentedon Jun 16, 2014
From some experimenting if we call the processor grid size for an axis
pand the distarray size in an axisdthen the following must be satisfiedWhere
n = p // dsod - p < pn < d.From experimenting, it looks like for a given
pthere arep - 2cases where the above relations are satisfied and we would end up with an empty localarray.It seems like the best solution would be to check if the above conditions are satified in
make_grid_shape, if not reducepby1and try again.kwmsmith commentedon Jul 7, 2014
Should be partially addressed by PR #502.