You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
A file scan with Limit 0 loses the file's columns, and the CSVOld scan throws #8602
The Limit on a file-scan source is meant to bound how many rows the operator outputs. It also bounds the sample sourceSchema() reads to infer the columns, so a Limit of 0 leaves the inference nothing to look at and the operator reports a schema that does not describe the file.
The three readers fail differently on the same file. CSV File Scan and JSONL File Scan return a schema with no attributes at all. CSVOld File Scan throws ArrayIndexOutOfBoundsException, because it takes the column names from the header row and the types from the sample, then asks the empty type array for the first header's type.
Zero is a value the property editor accepts, and it is the natural way to ask for no rows. The columns of a file do not depend on how many of its rows the user wants, so a Limit of 0 should give an operator with the file's schema and an empty output, not one with no columns or one that throws.
How to reproduce?
Point a CSVOld File Scan at a CSV with a header and a couple of data rows, set Limit to 0, and run. The execution fails as the executor is constructed, on the operator's own schema inference, before a row is read. The same two steps on a CSV File Scan or a JSONL File Scan instead give downstream operators an operator with no columns.
Set Limit to 1 on any of the three and the schema is correct again, which is what tells the two concerns apart: CSVOldScanSourceOpDesc.sourceSchema reads its inference sample from slice(startOffset, startOffset + limit), and CSVScanSourceOpDesc and JSONLScanSourceOpDesc bound theirs the same way.
java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
at org.apache.texera.amber.operator.source.scan.csvOld.CSVOldScanSourceOpDesc.$anonfun$sourceSchema$8(CSVOldScanSourceOpDesc.scala:204)
at org.apache.texera.amber.operator.source.scan.csvOld.CSVOldScanSourceOpDesc.sourceSchema(CSVOldScanSourceOpDesc.scala:201)
at org.apache.texera.amber.operator.source.scan.csvOld.CSVOldScanSourceOpExec.<init>(CSVOldScanSourceOpExec.scala:38)
There are two ways to fix this. Stop bounding the inference sample by Limit, which settles all three readers but changes the inferred types of workflows that already set a small one; or only stop CSVOld from indexing an empty type array, which changes nothing that runs today but leaves CSV and JSONL with a column-less schema. I would take the first, since a file's columns do not depend on how many rows the user wants, and can write either.
Heads up before you spend time on this: the fix is already in #8341, which closes it. It takes the first option above, the inference sample no longer reads through Limit, in the CSV, parallel CSV, CSVOld and JSONL scans.
Activity
kz930 commentedon Sep 18, 2026
There are two ways to fix this. Stop bounding the inference sample by Limit, which settles all three readers but changes the inferred types of workflows that already set a small one; or only stop CSVOld from indexing an empty type array, which changes nothing that runs today but leaves CSV and JSONL with a column-less schema. I would take the first, since a file's columns do not depend on how many rows the user wants, and can write either.
VuMartin commentedon Sep 19, 2026
/take
fix(operator): keep a file's columns when its scan window asks for no…
kz930 commentedon Sep 19, 2026
Heads up before you spend time on this: the fix is already in #8341, which closes it. It takes the first option above, the inference sample no longer reads through Limit, in the CSV, parallel CSV, CSVOld and JSONL scans.
VuMartin commentedon Sep 19, 2026
I’ll untake since #8341 already addresses #8602. Thanks for the heads up!
VuMartin commentedon Sep 19, 2026
/untake