[RFC] Add CSV extension (ext/csv)

[damek24]

Summary

This draft PR proposes adding a new CSV extension (ext/csv) to PHP core.

The extension is based on Gina Peter Banyard's girgias/csv project (BSD-3-Clause) and extends it with file and stream handling.

Features

• RFC 4180-compatible CSV parsing and serialization
• Locale-independent parsing
• Multibyte delimiters, enclosures, and EOL sequences
• Strict and lax collection parsing
• Lazy iteration over CSV files
• Streaming CSV output
• No proprietary escape character

Implementation

• Six functions and one class in the Csv\ namespace
• 78 PHPT tests
• Six upstream issues identified and fixed during the port

RFC

https://wiki.php.net/rfc/csv_extension

Status: Draft - this PR is not intended for merging until the RFC has been discussed and accepted.

Attribution

Based on girgias/csv by Gina Peter Banyard and contributors, licensed under BSD-3-Clause.

Parts of the implementation were developed with LLM assistance. I have reviewed the code and take full responsibility for the contribution.

Feedback on API design, implementation details, and portability is welcome.

[ArtUkrainskiy]

I built the branch at 38e463a and went through csv.c with a few probes; two things I'd consider blockers, then smaller ones.

Use-after-free with a generator. collection_to_buffer()/collection_to_file() take the row from get_current_data() and iterate Z_ARRVAL_P(fields) without an addref. A Stringable element whose __toString() advances the generator frees the row mid-iteration:

class S { function __toString(): string { $GLOBALS['g']->next(); return "s"; } }
function rows() { yield [new S, 'b']; yield ['c', 'd']; }
$g = rows();
Csv\collection_to_buffer($g);

valgrind (USE_ZEND_ALLOC=0): invalid read in hashtable_to_rfc4180_string (csv.c:135), block freed by zend_array_destroy in ZEND_YIELD. Engine foreach copies the value before the body; the same is needed here. This one is inherited from girgias/csv 0.6.0.

Quadratic rescan in createFromFile(). php_csv_lazy_collection_stream_next() rescans from the start of the row after every 8 KiB read. One quoted field: 1 MB 0.09 s, 2 MB 0.35 s, 4 MB 1.4 s, 8 MB 5.4 s, against 0.01 s through buffer_to_collection() on the same 4 MB; an unterminated " at the start of a large file does not finish. The scanner state (position, in-enclosure) should survive across reads.

Smaller:

• row_to_array("a,b\r\nc,d") returns ["a", "b"] and drops the rest silently (csv.c:401 goto eol, position never checked by the caller).
• A collection element that is a reference is a TypeError "must be an array" (no ZVAL_DEREF at csv.c:520/620); $rows = [&$r] works with foreach but not here.
• Iteration state lives on the collection, not the iterator: foreach ($c as $a) foreach ($c as $b) yields 3 pairs instead of 9, and two getIterator() objects interleave.
• Stream open goes through php_stream_open_wrapper_ex(..., 0, NULL, NULL) — no REPORT_ERRORS, no default context — so ENOENT/EACCES are reported as a bare "Failed to open", and stream_context_set_default() has no effect.
• Detaching the stream from its resource (res->type = -1, zend_list_delete()) to make it private: a collection still alive at shutdown never calls a user wrapper's stream_close() (fopen() does), and compress.zlib:// leaks the descriptor, as the comment in free_obj says. PHP_STREAM_FLAG_NO_FCLOSE plus closing in free_obj is the existing pattern (spl_directory.c) and has neither problem.
• "a"b → ["ab"] and an unterminated enclosure are accepted silently while a"b is a ValueError; worth deciding which way the parser is strict, and the stream scanner must match.

Probes and numbers: https://github.com/ArtUkrainskiy/php-src-bench/tree/main/reports/csv-pr-24199

[damek24]

Thanks a lot for the thorough review and the reproducer scripts — they were extremely helpful!

I've reproduced and addressed all the reported issues in 121c02f, with dedicated regression tests for each. Full results, benchmarks, reproducer scripts, and ASan logs are available here:

https://github.com/damek24/php-csv-rfc/tree/main/reports/csv-pr-24199-fixes

Correctness and memory safety

Use-after-free: Confirmed under ASan before the fix (same trace: read at csv.c:135, freed by zend_array_destroy during ZEND_YIELD). The ArrayIterator variant, where __toString() replaces the current row, and collection_to_file() were also affected. The row is now kept alive using ZVAL_COPY() during formatting, following the same ownership principle as foreach.

Quadratic rescanning: The stream scanner now preserves its position and enclosure state across reads, resuming from the last point where parsing decisions were final. Only potential token matches at the read boundary need reconsideration.

This eliminates the quadratic rescanning behavior:

• 8 MB quoted field: 4.15 s → 0.008 s (~519× faster).
• Unterminated enclosure: raises ValueError at EOF without quadratic rescanning.
• Differential fuzzing with 1–20 byte stream reads, multibyte and self-overlapping dialects: no mismatches against buffer_to_collection_lax().

Other fixes

• Single-row validation: row_to_array() now throws ValueError when given multiple rows.
• References: Row references are handled using ZVAL_DEREF().
• Iteration: Iterator state is now stored on the iterator itself. Nested iteration produces all 9 pairs for createFromBuffer(). Concurrent iteration over a single file stream raises an Error, since the stream cannot maintain two independent positions.
• Stream errors: Opening uses REPORT_ERRORS and the default stream context. Wrapper errors are propagated similarly to SplFileObject.
• Stream lifecycle: Streams remain registered using PHP_STREAM_FLAG_NO_FCLOSE, following the pattern in spl_directory.c. One difference from your suggestion: cleanup happens in dtor_obj rather than free_obj, since destructors still run while the executor is alive. This ensures user-defined stream wrappers receive stream_close() during shutdown. If object destructors are skipped, the resource list handles cleanup.
• Strict parsing: The parser and stream scanner now consistently reject malformed CSV, including trailing data after a closing enclosure, unterminated enclosures, and bare CR/LF outside enclosed fields when the configured EOL is CRLF.

The strictness changes also exposed an ambiguity with self-overlapping multibyte enclosures (aa, --, xyx). Closing enclosures are now recognized based on the following delimiter, EOL, or end of input. Previously, the xyx case failed 236 out of 5,000 randomized round-trips; it now passes all 5,000.

Performance

I also optimized the parser's hot path using a per-dialect byte lookup table, bulk scanning, and direct string creation from contiguous input slices instead of byte-by-byte appending.

This improved buffer parsing performance by approximately 3.5×.

On my i9-12900K, using your bench.php and comparing against a local build with #24207 applied:

File reading (ns/row)	ext/csv	fgetcsv() + #24207
Plain	363	402
Quoted	446	586
Wide	1,539	1,742

The writers remain unchanged and are approximately 10–20% slower than fputcsv() in these benchmarks.

The complete benchmark methodology and before/after measurements are included in the linked report.

Thanks again for putting together such a detailed and reproducible review, and nice work on #24207!

[ArtUkrainskiy]

Re-ran everything on 121c02f: the use-after-free is gone under valgrind on both reproducers, the 32 MB quoted field reads in 0.06 s and the unterminated one raises in 0.01 s, and every smaller item behaves as you describe. The new parser passes a round-trip fuzz (4,000 random rows with NUL, CR/LF, \xC3 and token bytes in the fields, through collection_to_buffer → buffer_to_collection / createFromBuffer / createFromFile / row_to_array) with zero mismatches in the seven non-overlapping dialects, is valgrind-clean on it, and handles multibyte tokens across the 8 KiB read boundary. On my machine createFromFile() now reads plain rows at 468 ns against 500 for fgetcsv() with #24207, so the performance point from my first comment no longer stands.

One thing remains, and it predates the port (girgias/csv 0.6.0 behaves the same): dialects whose tokens overlap with field content don't round-trip, because the writer doesn't enclose a field when the token appears across the field/delimiter boundary:

Csv\row_to_array(Csv\array_to_row(['-', 'z'], '--', 'aa', "\r\n"), '--', 'aa', "\r\n");
// "---z\r\n" -> ["", "-z"]
Csv\row_to_array(Csv\array_to_row(['x', 'x'], 'xy', 'xyx', "\n"), 'xy', 'xyx', "\n");
// "xxyx\n" -> ValueError: Enclosure sequence is used in a non escaped field

In the fuzz, xy/xyx fails 365 of 560 cases and --/aa 43 of 549 (the first was a misparse before 121c02f, now a ValueError). Two ways out: reject dialects where a token is a substring of another or overlaps itself, or have the writer enclose a field whenever field + delimiter (or delimiter + field) contains a token across the boundary. Since multibyte tokens are one of the RFC's listed features, I'd rather see which one the RFC commits to.

Fuzz and the minimal-case search: https://github.com/ArtUkrainskiy/php-src-bench/tree/main/reports/csv-pr-24199