Skip to content

FileSplitterTask produces wrong files: first line lost, line breaks doubled, trailing empty file #205

Description

@njoubert-cleverage

Description

FileSplitterTask does not split the source file correctly:

  • the first line of the source file is lost,
  • every line break is doubled (each line is followed by an extra empty line),
  • an extra empty file is emitted at the end of the iteration.

The produced files contain neither max_lines lines of the input nor the original content.

Reproduction

var/data/file_splitter.txt contains a\nb\nc\nd\ne\n:

clever_age_process:
    configurations:
        demo.file_splitter_content:
            tasks:
                split:
                    service: '@CleverAge\ProcessBundle\Task\File\FileSplitterTask'
                    options:
                        file_path: '%kernel.project_dir%/var/data/file_splitter.txt'
                        max_lines: 2
                    outputs: [ read_chunk ]
                read_chunk:
                    service: '@CleverAge\ProcessBundle\Task\TransformerTask'
                    options:
                        transformers:
                            callback#1:
                                callback: 'file_get_contents'
                            callback#2:
                                callback: 'json_encode'
                    outputs: [ dump ]
                dump:
                    service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'
$ printf 'a\nb\nc\nd\ne\n' > var/data/file_splitter.txt
$ bin/console cleverage:process:execute demo.file_splitter_content
Starting process 'demo.file_splitter_content'...
""b\n\nc\n\n""
""d\n\ne\n\n""
""""
Process 'demo.file_splitter_content' executed successfully

Expected: "a\nb\n", "c\nd\n", "e\n". Tested on main (c1a2804), PHP 8.5, Symfony 7.4.

Cause

Proposed fix

  • Open the source without SplFileObject flags and without counting and rewinding, and read one line ahead to know whether another chunk remains.
  • Write exactly max_lines lines per file (the last one fewer), stripping only the trailing \r\n/\n of each line before writeLine(): no line is lost, content is preserved, line breaks are not doubled.
  • next() stops as soon as no line is left, so no empty file is emitted; an empty source file is skipped and produces no output.
  • Update the reference documentation, add unit tests.

Requirements

  • Documentation updates
    • Reference
    • Cookbooks
    • Changelog
  • Unit tests

Breaking changes

None: this fixes the content of the produced files (the task was introduced in v5.0, #174). Line breaks in produced files are written as PHP_EOL, and an empty source file now produces no output instead of an empty file.

Activity

  1. added a commit that references this issue on Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions