Skip to content

Exploring an additional binary record reader approach for Cobrix [micro file with header] #883

Open

Description

@yassinessadi

Hello Cobrix team,

First, thank you for your work on Cobrix. It is a very interesting project, especially for processing COBOL and mainframe data with Apache Spark.

I have been experimenting with a small independent Scala project called EcoPass, which explores binary record reading and record-header interpretation for mainframe files.

Why I started EcoPass

While exploring mainframe file formats and Cobrix's implementation, I wanted to better understand how record headers, record lengths, record statuses, and record boundaries could be handled at a lower level.

This led me to build a small Scala-based reader.

Current implementation

The initial implementation includes:

  • Reading binary files sequentially.
  • Supporting configurable 2-byte and 4-byte record headers.
  • Extracting record status and record length.
  • Handling record alignment and padding.
  • Returning record payloads as raw bytes.
  • Unit tests using sample binary files.
  • A JVM interface that can be accessed from PySpark.

The project is still experimental and is not intended to replace Cobrix.

What I would like to discuss

I would appreciate your feedback on a few questions:

  1. Could this type of low-level record-header reader be useful as an additional component in Cobrix?
  2. Are there existing Cobrix components that already handle these record formats or similar functionality?
  3. Are there opportunities to improve record reading, streaming, or record-boundary handling?
  4. Would you be interested in reviewing the implementation and discussing whether any ideas could be contributed to Cobrix?

My intention is to learn from the Cobrix team's experience and, if there is a useful opportunity, contribute improvements to the existing project rather than duplicate existing functionality.

Project

I would be happy to provide additional examples, benchmarks, or implementation details.

Thank you for your time and for maintaining Cobrix.

Best regards,
Yassine Essadi

Activity

  1. changed the title [-]Exploring an additional binary record reader approach for Cobrix[/-] [+]Exploring an additional binary record reader approach for Cobrix [micro file with header][/+] on Nov 14, 2026
  2. yruslan commented on Nov 16, 2026

    @yruslan
    Collaborator

    Hi @yassinessadi , thank you for the post. I'm going to review the repository, questions and proposals and get back soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions