Repository navigation
Python TileUDT and rf_type.Tile improvements - #146
Conversation
Signed-off-by: Jason T. Brown <jason@astraea.earth>
| def my_udf(t): | ||
| import numpy as np | ||
| a = np.log1p(t.cells) | ||
| return Tile(a, CellType.from_numpy_dtype(a.dtype)) |
There was a problem hiding this comment.
This kind of pattern is showing up more and more. See also our Tile.add, Tile.sub etc.
There was a problem hiding this comment.
My hope is that we can make udfs a true path of last resort... what can we do to get closer to feature complete with columnar functions?
There was a problem hiding this comment.
Indeed. i think the familiar model here is that one could iterate over values in a numpy array but it is uncommon in practice because the built in functions are more convenient and known to be faster. I am glad the user gets some rope. But we also have to make this point clear in docs.
|
Marked WIP going to add a few more goodies. |
…ality correctly handles no_data; add __matmul__ operator Signed-off-by: Jason T. Brown <jason@astraea.earth>
| def __eq__(self, other): | ||
| if type(other) is type(self): | ||
| return self.cell_type == other.cell_type and np.array_equal(self.cells, other.cells) | ||
| return self.cell_type == other.cell_type and np.ma.allequal(self.cells, other.cells) |
There was a problem hiding this comment.
turns out np.ma.allequal behaves the same as np.array_equal for regular (not masked) ndarray's.
Signed-off-by: Jason T. Brown <jason@astraea.earth>
| ) | ||
|
|
||
| df_result = df.select( | ||
| (rf_dimensions(df.tile).cols.cast('int') * rf_dimensions(df.tile).rows.cast('int') - |
There was a problem hiding this comment.
Do we need an rf_size function?
| @udf(TileUDT()) | ||
| def my_udf(t): | ||
| import numpy as np | ||
| return Tile(np.log1p(t.cells)) |
No description provided.