Skip to content

Rdw header missing for variable size ebcdic file #772

Open

Description

@sobhaw

I do not have rdw headers ,when it is single copybook it works with variable size true .but with multiple copybooks in used copybooks as well as combined all 4 into single copybook using redefines. Both r giving arrayindexouf of bounds exception as it might not be getting record length correctly for ebcdic file. It is working fine for txt file. I have each record starting with 01ABC1 , 01ABC2, 01ABC3, 01ABC4 which is the record id for 1st , 2nd and so on . How do I tackle this ?

Activity

  1. yruslan commented on Aug 15, 2025

    @yruslan
    Collaborator

    Hi @sobhaw , please share options you pass to 'spark-cobol', and the copybook, if possible. Array index out of bounds error should not be happening in any circumstance.

  2. sobhaw commented on Aug 19, 2025

    @sobhaw
    Author

    I used the options like variable_size_occurs schema retention keep original and removed the ascii option and tried to read a binary file the copy had last part as array so once it can't determine the length after reading the first record it gives this issue

  3. sobhaw commented on Aug 19, 2025

    @sobhaw
    Author

    Not closed i closed it

  4. sobhaw commented on Aug 19, 2025

    @sobhaw
    Author

    I do not have rdw headers ,when it is single copybook it works with variable size true .but with multiple copybooks in used copybooks as well as combined all 4 into single copybook using redefines. Both r giving arrayindexouf of bounds exception as it might not be getting record length correctly for ebcdic file. It is working fine for txt file. I have each record starting with 01ABC1 , 01ABC2, 01ABC3, 01ABC4 which is the record id for 1st , 2nd and so on . How do I tackle this ?

  5. reopened this on Aug 19, 2025
  6. sobhaw commented on Aug 19, 2025

    @sobhaw
    Author

    I used the options like variable_size_occurs schema retention keep original and removed the ascii option and tried to read a binary file the copy had last part as array so once it can't determine the length after reading the first record it gives this issue

  7. yruslan commented on Aug 19, 2025

    @yruslan
    Collaborator

    There are multiple ways to read variable record length file depending on the way to determine the length of each record. From your description looks like record lengths can be derived from record types. If so, you can use the mapping option:

    .option("record_format", "F")
    .option("record_length_field", "FIELD_STR")
    .option("record_length_map", """{"SEG1":100,"SEG2":200}""") 
    

    where FIELD_STR is the field that determines the record type. When it equals to SEG1 the record length is 100, and so on.

    Here are details: https://github.com/AbsaOSS/cobrix?tab=readme-ov-file#variable-length-records-support

  8. sobhaw commented on Aug 19, 2025

    @sobhaw
    Author

    Ok thx for the solution let me try that. I suppose 100 and 200 will change according to the copybook

  9. sobhaw commented on Aug 20, 2025

    @sobhaw
    Author

    I tried with 01ABC for record field length it didn't accept a string unsupported function ABC expr syntax error

  10. sobhaw commented on Aug 20, 2025

    @sobhaw
    Author

    I tried giving the field name also from the copybook

  11. yruslan commented on Sep 5, 2025

    @yruslan
    Collaborator

    Please, send all options you are passing to spark-cobol.

  12. sobhaw commented on Sep 19, 2025

    @sobhaw
    Author

    I cannot send anything outside of my org. So just typing it.
    file_start _offset 274, file_end _offset 303
    Copybooks , copybooks paths
    Variablesize true also checked with recordformat f
    Generate record id true
    After ur suggestion put a recordlength map 01ABC1 etc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions