Repository navigation
RFC: add data type inspection utilities to the array API specification #425
Description
Activity
Also related ( #152 )
Edit: If we look into type naming (briefly discussed), this discussion around typing naming in Zarr may be of interest ( zarr-developers/zarr-specs#131 )
Like
can_cast()orresult_type(), could these utils take both dtypes and arrays? I'd personally want these utils for dtype objects themselves, but definitely my own use cases are not quite aligned with most array consumers.Reacted by Leo FangUpdate: I've updated the OP as follows based on feedback here and in the last array API consortium meeting.
- Functions now accept both arrays and dtypes.
- Function names include a
_dtypesuffix (as suggested during the consortium meeting) - Function names begin with a
has_prefix. This helps avoid conflicts with existing APIs (e.g., PyTorch) and matches how one might describe an array (e.g., has shape X, has data type Y, etc). - Included both real-valued and generic float APIs to match specification data type categories.
- Included a generic real dtype API to match specification data type categories.
Reacted by Matthew BarberReacted by Matthew BarberSome more prior art:
- TensorFlow has methods like
is_boolonDTypeobjects: https://www.tensorflow.org/api_docs/python/tf/dtypes/DType - JAX doesn't have anything other than a subset of numpy APIs (
result_type,can_cast,promote_dtypes): https://jax.readthedocs.io/en/latest/_modules/jax/_src/dtypes.html - NumPy issue on the too many ways of comparing: ENH: add a canonical way to determine if dtype is integer, floating point or complex numpy/numpy#17325
As this specification has favored functional APIs
Given that dtype objects are immutable and have no state, this should also work for JAX et al. Not saying that that's my preference (I'm not yet sure), but this RFC proposes a lot of functions ...
3 underscores in a name like
has_real_float_dtypeis also not ideal.- TensorFlow has methods like
Given that dtype objects are immutable and have no state, this should also work for JAX et al. Not saying that that's my preference (I'm not yet sure), but this RFC proposes a lot of functions ...
Whether methods or functions, surface area would be the same. The list can obviously be culled; however, I do think there is some advantage to matching the categories as used in the spec, especially for providing consistent APIs for input argument validation.
3 underscores in a name like has_real_float_dtype is also not ideal.
The number of underscores is not super important, IMO. Instead, we're probably concerned about number of characters. Originally, I left out the
_dtypesuffix, which would reduce the function name length; however, consortium members voiced desire for such a suffix in the array API meeting.I don't have a strong opinion here; although, the current naming convention is arguably more literate.
Silly question, why not do:
if array.dtype in <set_of_dtypes>: ...
and require implementations to provide some predefined sets, such as "set of all supported integer dtypes" or "set of all supported floating point dtypes"?
and require implementations to provide some predefined sets, such as "set of all supported integer dtypes" or "set of all supported floating point dtypes"?
That does seem more appealing indeed; it's what can already be done today and it reads fairly well. I think I prefer that over both the
has_*functions in this proposal and the numpyissubdtypedesign.I don't like
issubdtype. For NumPy, I could imagineisinstance(arr.dtype, InexactDType)(or similar). So that way the API here would beisinstance(arr.dtype, some_object). The problem is that I am not sure if anisinstanceAPI would work for everyone.For
arr.dtype in set_of_dtypesthere aretwothree things to keep in mind:- The
set_of_dtypeswill be different for each library, becausebfloat16,float16, and others do not exist for implementers. Implementers can extend the API after all. - For NumPy, users may extend the API reasonably soon. For example adding
bfloat16or a multi-precision float object. - Sets might be tricky right now NumPy in either case (although that could likely be made sure to work). There are arbitrary number of possible instances for dtypes, although they should compare equal with a limited set, that set is confusingly large (byte-order matters).
I do think neither of these is particularly problematic. But, I would say that this would not be a
set, but rather an opaque object that supports theinoperator.Reacted by Ralf Gommers and Neil Girdhar- The
A minor pro of dtype sets is that it could be a way for a library to communicate what dtypes they support—thinking of PyTorch and it only supporting
uint8unsigned integers. Useful here and there, like telling Hypothesis to not try generatinguint{16/32/64}.Reacted by Athan@leofang point out that this is blocking for adding
realandconj(and I imagineimagtoo), it'd be great to finalize this. The majority of folks who have weighed in seem to prefer aset/collection type of approach. So here's a suggested API for that, in line with @seberg's last sentence above.- There must be objects
integer_dtypes,floating_point_dtypes, andcomplex_dtypes, - The syntax
dtype in xxx_dtypesmust yield a boolean value with the expected result (to be detailed out more in the spec), xxx_dtypesmust contain all the expected dtypes that are part of the standard, and may contain additional dtypes of the same kind- The objects may be of any kind, e.g. a
setor a custom class instance.
Other thoughts:
- No object for boolean is needed, because
boolalready supports__eq__, soarray.dtype == boolis enough. - Also no separate signed/unsigned integer objects, because that's a bit much for the API / less needed. This is mostly a convenient way to spell
array.dtype in (dtype1, dtype2, ...)anyway. - The one name where there's not a single obvious choice is
floating_point_dtypes. It could also befloat_dtypes,floating_dtypes, orreal_dtypesfor example. - Not specifying the type of these objects is on purpose, to make it easy to for example have an API that adds user-defined dtypes in.
- That means that for static typing we need another
Protocol. Not completely ideal, but imho better than restricting implementation choices for libraries (see point 3 in @seberg's comment above about whysetis tricky for NumPy).
- That means that for static typing we need another
One alternative with a similar API surface is to add 3 functions with the same functionality instead. Those functions could be 3 of the ones in the issue description here (e.g.,
has_integer_dtype,has_floating_point_dtype,has_complex_dtype). Considerations:- Pro: it's better for static typing,
- Con: it introduces an asymmetry between supported and unsupported sets - we need the
dtype in xxxanyway when the predefined objects aren't the right ones.
I think the con is more important than the pro here. But I'd say either choice is pretty reasonable here.
- There must be objects
Just to make sure @leofang, both flavors are fine for accelerators, right? When the spec says something should return a
bool, that's not a problem - only Python control flow likeif _expr_yielding_a_boolis. So a function is not preferred from that perspective. Or maybe there's a significant amount of extra implementation complexity for thedtype in xxx_dtypesversion?Reacted by Leo FangMy preference would be to match more closely the spec on this. Namely, have the following objects:
- numeric_dtypes:
int8...64,uint8...64,float32/64,complex64/128 - real_dtypes:
int8...64,uint8...64,float32/64 - float_dtypes:
float32/64,complex64/128 - real_float_dtypes:
float32/64 - complex_float_dtypes:
complex64/128 - integer_dtypes:
int8...64,uint8...64
This would mean 6 objects, which would, as it stands now, cover almost the entirety of the spec. As these are relatively trivial to implement and expose, I don't see this as imposing an undue burden on array libraries.
However, if only
integer,float, andcomplex, their repetition in order to generate composite groups matching the spec in userland and library implementations would be mildly annoying and would possibly just lead array libraries to implement the composite groups anyway.E.g., suppose we want to validate an array for a function which supports all numeric dtypes. With just
integer,float, andcomplexcollections, I'd need to dodef foo( x: array ): dt = x.dtype if dt in integer_dtypes or dt in float_dtypes or dt in complex_dtypes: ...
Given the opacity of what's intended in the conditional, one might be tempted to write a small helper function transforming the check to something more literate. And given the ubiquity of composite dtype categories in the spec, I'd argue we should just include the composite groups in the spec directly so that array library clients don't need to reimplement these groups from library to library.
- numeric_dtypes:
E.g., suppose we want to validate an array for a function which supports all numeric dtypes. With just
integer,float, andcomplexcollections, I'd need to doThis is a good point. Although in general this isn't done for library code, even if the library provided string/object/etc. dtypes. It is difficult to pick the right sets here.
My preference would be to match more closely the spec on this. Namely, have the following objects:
I don't think that will work, the names don't map to current practice and are not intuitive enoug.
float_dtypesin particular is bad. See torch.is_floating_point and for numpy:>>> x = np.ones(2, dtype=np.float64) >>> x2 = np.ones(2, dtype=np.complex128) >>> np.issubdtype(x.dtype, np.floating) True >>> np.issubdtype(x2.dtype, np.floating) False
Understood. We're not starting from a blank slate. Although, presumably, at least for Torch, the need for
is_complexandis_floating_pointwould no longer exist, opening up a path to eventual deprecation.For NumPy, well, 🤷♂️.
The notion of what is considered a "floating-point" dtype arose previously in the consortium. Then, it was decided that under the umbrella of floating-point are both real and complex. Hence, the OP.
Unfortunately, however, I don't have, atm, a more intuitive name for "real + complex floating-point dtypes", but I don't think this negates the general desirability of composite groups.
6 remaining items
Good point @jbrockmendel. I never noticed it in
numpy, but did a quick check and yes these checks are expensive (still fast though):>>> import numpy as np >>> real_dtypes = {np.float16, np.float32, np.float64, np.longdouble} >>> %timeit np.float64 in real_dtypes 48.7 ns ± 0.211 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each) >>> %timeit np.issubdtype(np.float64, np.floating) 257 ns ± 2.3 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)
The main thing that won't work I believe is dtype subclasses.
To circle back to this, this is true only for user-defined dtypes. If those exist (which may be unique to
numpy), it's perhaps okay to then require registering them somehow so they get added toxxx_dtypes.Just to be sure, custom dtypes (working for each backend) won't be ever added to the standard, right? It would be great to be able to implement the logic for a custom dtype once and have it working everywhere, but probably that would be difficult to standardize.
I saw that for units, for example, you recommended to wrap the API backends instead in https://discuss.scientific-python.org/t/advice-and-guidance-about-array-api-for-a-units-package/.
I think it's safe to say that custom dtype support won't be added. Most libraries don't have it, and for NumPy it's still a work-in-progress to define a good API with all the functionality downstream library authors may need.
That said, it would be nice that standard-compliant code like
x.dtype in real_dtypesin libraries like SciPy and scikit-learn will work for those NumPy users that do end up creating their own dtype. I think it will, as long as NumPy has an API that allows those users to extendreal_dtypeswith their new dtype.Reacted by Leo FangLet me make a few points for why I am leaning against the set approach, although it is still not quite clear cut:
- The non-set approach is similar to
isinstance. I am not sure the set approach has a clear inspiration e.g. in typing? (The notation of using set operations has, but checking within?) - The set approach just feels a bit too smart to me. :)
- If
floating_dtypesis just a set/tuple (NumPy cannot do that, I think), it is not clear thatarr in floating_dtypeswould raise an error rather than always returningFalse(it must bearr.dtype in floating_dtypes). - In NumPy, I can see things like
is_of_dtype/has_dtype(1., np.floating)making sense. Where1.is actually just a Python float. Allowing to generalize "dtype checking" to objects that may not have a.dtypeattribute.
Yes, this would be to have better support of scalars, which is something that ideally are not supposed to exist here. Would this be useful e.g. for pandas, @jbrockmendel (since I always wonder if pandas has more need of scalars than an array API)? - In NumPy it would be nice to use
np.floatingfor this, but that is also the scalar type, which may lead to a bit strange overloading. If we have to functions (has_dtypeandis_dtype) that becomes unproblematic. (An error could point to the other where appropriate.)
In the end, I am not certain yet that the set approach works well for NumPy proper. Of course that is not actually a blocker for this API since there can be differences.
Reacted by Oleksandr Pavlyk- The non-set approach is similar to
Would this be useful e.g. for pandas, @jbrockmendel (since I always wonder if pandas has more need of scalars than an array API)?
IIUC, I don't think it's likely pandas would change our current usage
Thanks, @seberg, for the nice thoughts. Just wanna add a quick note.
- If
floating_dtypesis just a set/tuple (NumPy cannot do that, I think), it is not clear thatarr in floating_dtypeswould raise an error rather than always returningFalse(it must bearr.dtype in floating_dtypes).
This is very nice point. It seems
floating_dtypescannot be a plain set/tuple, but at least a subclass of them with a custom__contains__first checking the type of the object before delegating to theincheck of the parent class.- In NumPy, I can see things like
is_of_dtype/has_dtype(1., np.floating)making sense. Where1.is actually just a Python float. Allowing to generalize "dtype checking" to objects that may not have a.dtypeattribute.
Also a very good point. Since we include the Python types in the type lattice, I think it is legitimate to do the said check even if we don't plan to support scalars.
- If
So it looks like we're (a) leaning towards the single-function version, and (b) only have it accept either a dtype or an array (avoiding the union of both).
For (b), most of the time the thing to check is an array. However, dtype checking is also needed, and getting a dtype from an array is trivial while an array from a dtype is not. If the input was an array,
has_dtypeis a logical name. If it's a dtype, I thinkis_dtypeis better. That is also a name that AFAIK isn't used anywhere.So we'd be looking at some flavor of:
def is_dtype(x: dtype, kind: Union[str, dtype, tuple[Union[str, dtype], ...]]) -> bool: """ >>> is_dtype(x, 'integer') >>> is_dtype(x, 'real') >>> is_dtype(x, 'complex') >>> is_dtype(x, ('real, 'complex')) # avoid both 'floating' and 'inexact', those are not good names >>> is_dtype(x, 'numeric') # shorthand for ('integer', 'real', 'complex') >>> is_dtype(x, 'signed integer') >>> is_dtype(x, 'unsigned integer') >>> is_dtype(x, float32) >>> is_dtype(x, (float64, complex128)) """
or
def is_dtype(x: dtype, kind: str) -> bool: """ >>> is_dtype(x, 'integer') >>> is_dtype(x, 'real') >>> is_dtype(x, 'complex') >>> is_dtype(x, 'numeric') >>> is_dtype(x, 'signed integer') >>> is_dtype(x, 'unsigned integer') """
or something in between (e.g,
kind: str | dtype]).Looking at the
np.issubdtypeusage in SciPy, there's a roughly equal mix between checking against a set of dtypes (e.g.,np.issubdtype(dtype, np.complexfloating)) and checking against a single dtype (e.g.,np.issubdtype(dtype, np.int32)). Both seem kinda useful. A combination (tuple of sets/dtypes) is probably not necessary.So perhaps this is the way to go: ?
def is_dtype(x: dtype, kind: str | dtype) -> bool:
We had another look at this yesterday. We want to go for a flavor of the function-based implementation here; there was no clear preference for which of the above was preferred. So let's try a vote - use emoji's on this comment:
- 👍🏼 if you prefer
is_dtype(x: dtype, kind: str) - 🎉 if you prefer
is_dtype(x: dtype, kind: str | dtype) - 🚀 if you prefer
is_dtype(x: dtype, kind: str | dtype | tuple[Union[str, dtype], ...])
Reacted by Sebastian Berg and Athan- 👍🏼 if you prefer
I know I'm very late to this discussion, but as the array API is now implemented in NumPy, I've been exploring what it would do to my code.
A couple problems with
is_dtypeover ordinary Python types (proposed by seberg) is that:-
is_dtypeuses strings, which can be error prone. If you have a typo, it may not be caught until you run your program and run into the offending code. Yes, you can annotateis_dtypewithLiteral, but if the type codes are passed from other functions (asstr), then the validation won't happen. It feels more ergonomic to me to have objects rather than magic strings.One of the things I love about the Array API is the constrained interface that feels way less bug prone. It's not a huge burden to have to import a special object instead of using a string, and it prevents mistakes and allows type-checkers to find errors. It's the same reason people generally prefer enumeration objects over strings.
-
The various kinds cannot be checked by type checkers. Right now, it's possible to annotate an array as
numpy.typing.NDArray[np.floating[Any]]. I do this for various numpy array types, and this catches many bugs thanks to numpy's excellent implementation of type annotations. If you don't provide base classes, then how are you supposed to have these annotations?If I were to vote, I would have voted for:
is_dtype(x: dtype, kind: dtype | tuple[dtype, ...])
which may as well have been written as simply
issubclass(x.type, kind).
I personally prefer seberg's proposal to use ordinary Python
issubclasswith a tree of Python types. Any thoughts on this? Withis_dtype, how can I accomplish the above type annotations?-
It's the same reason people generally prefer enumeration objects over strings.
I think this isn't really true? At least, I can't think of many APIs where enums are common, while I can think of lots of libraries that use string args for keywords.
Enums have a major design flaw - namespace cluttering. Imho that is far more important, also for ergonomics, than static type checking.
The array API standard doesn't have many strings, but if NumPy had enums instead of strings or True/False/None keywords everywhere, that would be hundreds of extra objects.
With
is_dtype, how can I accomplish the above type annotations?I think we still have a more fundamental issue to solve: how to annotate arrays themselves. This should be done using a
ProtocolI believe, see gh-229.The same will apply to other objects. Giving that we have to be concerned about usage by consuming libraries and end users in an array-library-agnostic way, where it's effectively impossible for objects to have a class relationship, this is nontrivial to design. We haven't spent a whole lot of time on that aspect yet - and we should do that.
The
array[dtype]is one level more complex. And it's not just dtype, there's also device, dimensionality, etc. Even in NumPy this is still very much a work in progress. It's probably best to split that off into a new issue - I don't think dtypes having a class hierarchy or not is the primary issue here.Reacted by Leo FangI think this isn't really true? At least, I can't think of many APIs where enums are common, while I can think of lots of libraries that use string args for keywords.
In fairness, Python didn't have enums until Python 3.4, and after that there has been talk about updating old APIs to use them.
Enums have a major design flaw - namespace cluttering. Imho that is far more important, also for ergonomics, than static type checking.
I do love the Array API's compact namespace. I understand being very judicious about what gets into the namespace. I agree that if every method got its own enumerations, then the namespace might become overwhelming.
Perhaps the ABCs (number, integer, inexact, signedinteger, unsignedinteger, floating, and complexfloating) could be tucked into
xp.abc? Then, the root namespace would only have one extra symbol (abc) and one fewer symbol (is_dtype). It also mirrors Python's ABCs in collections and numbers. What do you think?The array API standard doesn't have many strings, but if NumPy had enums instead of strings or True/False/None keywords everywhere, that would be hundreds of extra objects.
RIght, I'm not suggesting that.
I think we still have a more fundamental issue to solve: how to annotate arrays themselves. This should be done using a Protocol I believe, see #229.
Yeah, I'm looking forward to this!
I don't think dtypes having a class hierarchy or not is the primary issue here.
I understand, but if they don't, then it is impossible (as far as I can tell) to maintain the type checking of array dtypes, which already works in numpy.
Edit: I just realized, but isn't the
is_dtypeapproach more complex for users? You have to use it like this, right?def f(x: xp.Array): # x can be from any Array API library. yp = array_api_of(x) # IIC, you have to get the right array API library to answer the question since x's dtype may not be known to numpy.array_api. if yp.is_dtype(x.dtype, 'integer'):
versus
import numpy.array_api as xp def f(x: xp.Array): # x can be from any Array API library. if issubclass(x.dtype.type, xp.abc.integer): # All Array API implementers can inherit appropriately.
Perhaps the ABCs (number, integer, inexact, signedinteger, unsignedinteger, floating, and complexfloating) could be tucked into
xp.abc? Then, the root namespace would only have one extra symbol (abc) and one fewer symbol (is_dtype). It also mirrors Python's ABCs in collections and numbers. What do you think?Maybe, not sure .... I have to think about the typing aspect, it's nontrivial. The
collectionsABCs are useful indeed. Thenumbersones terrible and I believe even Guido is on record saying they were a mistake.Reacted by Neil GirdharEdit: I just realized, but isn't the
is_dtypeapproach more complex for users? You have to use it like this, right?That's how you should use any function in the whole namespace (your
import numpy.array_api as xpalternative is non-portable), so I think they're equivalent. Theis_dtype(x.dtype, 'integer')line is shorter and simpler I'd say.
This RFC proposes adding data type inspection utilities to the array API specification.
Overview
Currently, the array API specification requires that conforming implementations provide a specified set of data type objects (see https://data-apis.org/array-api/2021.12/API_specification/data_types.html) and casting functions (see https://data-apis.org/array-api/2021.12/API_specification/data_type_functions.html).
However, the specification does not include APIs for array data type inspection (e.g., an API for determining whether an array has a complex number data type or a floating-point data type, etc).
Prior Art
NumPy and its derivatives have
dtypeobjects with extensive properties, including akindproperty, which returns a character code indicating the general "kind" of data. For example, for relevant dtypes in the specification, NumPy uses the following character codes:b: booleani: signed integeru: unsigned integerf: floating-point (real-valued)c: complex floating-pointThis availability of the
kindproperty is useful when wanting to branch based on input array data types (e.g., applying summation algorithms).In PyTorch,
dtypeobjects haveis_complexandis_floating_pointproperties for checking a data type "kind".Additionally, PyTorch offers functional APIs
is_complexandis_floating_pointproviding equivalent behavior.Proposal
Given the proposal for adding complex number support to the specification (see #373 and #418), a greater need arises for the specification to require conforming implementations to provide standardized ways for data type inspection.
For example, conforming implementations will need to branch in
abs(x)depending on whetherxis real-valued or complex-valued. Similarly, in downstream user code, we can expect that users will inevitably encounter situations where they need to branch based on input array data types (e.g., when choosing summation algorithms).As this specification has favored functional APIs, this RFC follows suit and proposes adding the following APIs to the specification:
has_complex_float_dtype(x: Union[array, dtype]) -> bool
Returns a
boolindicating whether an input array has a complex number data type (e.g.,complex64orcomplex128).has_real_float_dtype(x: Union[array, dtype]) -> bool
Returns a
boolindicating whether an input array has a (real-valued) floating-point number data type (e.g.,float32orfloat64).has_float_dtype(x: Union[array, dtype]) -> bool
Returns a
boolindicating whether an input array has a complex or real-valued floating-point number data type (e.g.,float32,float64,complex64, orcomplex128).has_unsigned_int_dtype(x: Union[array, dtype]) -> bool
Returns a
boolindicating whether an input array has an unsigned integer data type (e.g.,uint8,uint16,uint32,uint64).has_signed_int_dtype(x: Union[array, dtype]) -> bool
Returns a
boolindicating whether an input array has a signed integer data type (e.g.,int8,int16,int32,int64).has_int_dtype(x: Union[array, dtype]) -> bool
Returns a
boolindicating whether an input array has an integer (signed or unsigned) data type.has_real_dtype(x: Union[array, dtype]) -> bool
Returns a
boolindicating whether an input array has a real-valued (integer or floating-point) data type.has_bool_dtype(x: Union[array, dtype]) -> bool
Returns a
boolindicating whether an input array has a boolean data type.The above APIs cover the list of data types currently described in the specification, are sufficiently specific to cover most use cases, and can be composed to address most anticipated data type set combinations.