Repository navigation
[Format] Is Buffer lengths buffer REQUIRED in C data format of BinaryView #43989
Description
Describe the bug, including details regarding any error messages, version, and platform.
C Data interface section on BinaryView mentions that this array type requires an additional buffer to be produced when sending the array. However, it doesn't mention whether that addtional array always has to be present even if there's no data buffers. This could happen for example if your array only contains short strings.
For reference the implementation assumes there are at least 3 buffers which I assume is view, nulls and buffer lengths
arrow/cpp/src/arrow/c/bridge.cc
Line 1741 in 089ad78
Component(s)
Format
Activity
mapleFU commentedon Sep 6, 2024
94fc124
I think it's mentioned in the
docs/source/format/CDataInterface.rst?robert3005 commentedon Sep 6, 2024
Ok, so the correct interpretation of the docs I had linked https://arrow.apache.org/docs/format/CDataInterface.html#binary-view-arrays is that there MUST be a length buffer even if there are no data buffers?
mapleFU commentedon Sep 6, 2024
cc @pitrou @bkietz @felipecrv
bkietz commentedon Sep 6, 2024
When I wrote it, it was my intent that the lengths buffer always be present even if there were no data buffers. In that case, there would be exactly three buffers: validity, views, and lengths. However if there are no data buffers the lengths buffer would be empty, present only to give the array correct layout.
robert3005 commentedon Sep 6, 2024
When I started looking it made sense that it must be present. Thanks, I might make a small docs pr but this answers my question