What happened?
The Limit on a file-scan source is meant to bound how many rows the operator outputs. It also bounds the sample sourceSchema() reads to infer the columns, so a Limit of 0 leaves the inference nothing to look at and the operator reports a schema that does not describe the file.
The three readers fail differently on the same file. CSV File Scan and JSONL File Scan return a schema with no attributes at all. CSVOld File Scan throws ArrayIndexOutOfBoundsException, because it takes the column names from the header row and the types from the sample, then asks the empty type array for the first header's type.
Zero is a value the property editor accepts, and it is the natural way to ask for no rows. The columns of a file do not depend on how many of its rows the user wants, so a Limit of 0 should give an operator with the file's schema and an empty output, not one with no columns or one that throws.
How to reproduce?
Point a CSVOld File Scan at a CSV with a header and a couple of data rows, set Limit to 0, and run. The execution fails as the executor is constructed, on the operator's own schema inference, before a row is read. The same two steps on a CSV File Scan or a JSONL File Scan instead give downstream operators an operator with no columns.
Set Limit to 1 on any of the three and the schema is correct again, which is what tells the two concerns apart: CSVOldScanSourceOpDesc.sourceSchema reads its inference sample from slice(startOffset, startOffset + limit), and CSVScanSourceOpDesc and JSONLScanSourceOpDesc bound theirs the same way.
Version/Branch
1.4.0-incubating-SNAPSHOT (main)
Commit Hash (Optional)
e7d1676
What browsers are you seeing the problem on?
No response
Relevant log output
java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
at org.apache.texera.amber.operator.source.scan.csvOld.CSVOldScanSourceOpDesc.$anonfun$sourceSchema$8(CSVOldScanSourceOpDesc.scala:204)
at org.apache.texera.amber.operator.source.scan.csvOld.CSVOldScanSourceOpDesc.sourceSchema(CSVOldScanSourceOpDesc.scala:201)
at org.apache.texera.amber.operator.source.scan.csvOld.CSVOldScanSourceOpExec.<init>(CSVOldScanSourceOpExec.scala:38)
What happened?
The Limit on a file-scan source is meant to bound how many rows the operator outputs. It also bounds the sample
sourceSchema()reads to infer the columns, so a Limit of 0 leaves the inference nothing to look at and the operator reports a schema that does not describe the file.The three readers fail differently on the same file. CSV File Scan and JSONL File Scan return a schema with no attributes at all. CSVOld File Scan throws
ArrayIndexOutOfBoundsException, because it takes the column names from the header row and the types from the sample, then asks the empty type array for the first header's type.Zero is a value the property editor accepts, and it is the natural way to ask for no rows. The columns of a file do not depend on how many of its rows the user wants, so a Limit of 0 should give an operator with the file's schema and an empty output, not one with no columns or one that throws.
How to reproduce?
Point a CSVOld File Scan at a CSV with a header and a couple of data rows, set Limit to 0, and run. The execution fails as the executor is constructed, on the operator's own schema inference, before a row is read. The same two steps on a CSV File Scan or a JSONL File Scan instead give downstream operators an operator with no columns.
Set Limit to 1 on any of the three and the schema is correct again, which is what tells the two concerns apart:
CSVOldScanSourceOpDesc.sourceSchemareads its inference sample fromslice(startOffset, startOffset + limit), andCSVScanSourceOpDescandJSONLScanSourceOpDescbound theirs the same way.Version/Branch
1.4.0-incubating-SNAPSHOT (main)
Commit Hash (Optional)
e7d1676
What browsers are you seeing the problem on?
No response
Relevant log output