Skip to content

Review Current Usage and Customer Entity Classes to ensure they match their ESPI Schema XSD files #28

New issue
New issue
Closed
2 of 2 issues completed
Closed
Review Current Usage and Customer Entity Classes to ensure they match their ESPI Schema XSD files#28
2 of 2 issues completed
#102 (+4)

Description

@dfcoffin
dfcoffin
opened on Oct 4, 2025

ESPI Usage Definition Issues

The following Usage Entity files extend IdentifiedObject, but extend Object in the ESPI Usage schema:

Entity Current Definition Schema Definition
AggregatedNodeRefEntity IdentifiedObject Object
BatchListEntity IdentifiedObject BatchListType
IntervalReadingEntity IdentifiedObject Object
LineItemEntity IdentifiedObject Object
PnodeRefEntity IdentifiedObject Object
ReadingQualityEntity IdentifiedObject Object
RetailCustomerEntity IdentifiedObject Object (Not Defined by espi.xsd)
ServiceDeliveryPointEntity IdentifiedObject Object

The above-listed Entities are not ESPI Resources and therefore should not be wrapped in Atom Protocol statements but should be emitted as elements of an existing ESPI Resource. Consequently, they should be associated collections within their related ESPI Resources.

The BatchListEntity is a standalone XML message and not related to any ESPI Resource.

Reverting the above not to extend IdentifiedObject will affect the following directories and classes:

Activity

  1. dfcoffin
    added a sub-issue on Oct 4, 2025
  2. dfcoffin
    added a sub-issue on Dec 18, 2025

  3. dfcoffin commented on Dec 26, 2025

    @dfcoffin
    dfcoffin
    on Dec 26, 2025
    ContributorAuthor

    Multi-Phase NAESB ESPI Schema Compliance Plan

    Overview

    This plan reviews domain entities in /domain/usage, /domain/customer, and /domain/common to ensure compliance with NAESB ESPI 4.0 schema (espi.xsd and customer.xsd) element sequences.

    Excluded Entities (already completed):

    • ApplicationInformationEntity
    • AuthorizationEntity

    Processing Order:

    1. Usage domain entities (Phases 1-16)
    2. Customer domain entities (Phases 17-26)

    Branch Strategy:

    • Each phase uses a separate feature branch for single entity and all associated files
    • Branch naming: feature/schema-compliance-phase-{number}-{entity-name}
    • Commit, push, and create PR for each phase when completed
    • Merge PR before starting next phase

    Entity Ordering Principles:

    • Dependency order (parents before children)
    • Progress from simple to complex entities
    • Field order in entities must match XSD element sequence exactly

    Usage Domain Entities (Phases 1-16)

    Phase 1: TimeConfiguration (LocalTimeParameters)

    Branch: feature/schema-compliance-phase-1-time-configuration

    Dependencies: None (standalone entity)
    Referenced By: UsagePoint, Customer, ServiceLocation (via bidirectional Atom rel='related' links)

    Tasks:

    1. Entity Updates (TimeConfigurationEntity.java):

      • Review field order against espi.xsd TimeConfiguration element sequence
      • Verify JPA annotations match schema constraints
      • Ensure JAXB annotations align with XML element names and order
      • Update field JavaDoc to reference XSD documentation

    2. DTO Updates (TimeConfigurationDto.java):

      • Match DTO field order to espi.xsd TimeConfiguration element sequence
      • Verify JAXB annotations for XML marshalling
      • Add XSD constraint validation annotations

    3. MapStruct Mapper Updates (TimeConfigurationMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping
      • Ensure field order preservation during mapping

    4. Repository Updates (TimeConfigurationRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (TimeConfigurationService.java, TimeConfigurationServiceImpl.java):

      • Review service methods for schema compliance
      • Update any field access order to match schema sequence

    6. Flyway Migration Updates:

      • Update db/migration/V1__Create_Base_Tables.sql and db/migration/V3__Create_additiional_Base_Tables.sql
      • Verify column order matches espi.xsd element sequence
      • Update vendor-specific scripts if needed:
        • db/vendor/h2/V2__H2_Specific_Tables.sql
        • db/vendor/mysql/V2__MySQL_Specific_Tables.sql
        • db/vendor/postgres/V2__PostgreSQL_Specific_Tables.sql

    7. Testing:

      • Update unit tests for TimeConfiguration entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against espi.xsd using XSD schema validation

    8. Commit, Push, PR:

      • Stage all changes: git add .
      • Commit: git commit -m "Phase 1: TimeConfiguration schema compliance"
      • Push: git push origin feature/schema-compliance-phase-1-time-configuration
      • Create PR and merge when approved

    Phase 2: ReadingType

    Branch: feature/schema-compliance-phase-2-reading-type

    Dependencies: None (standalone entity with embedded RationalNumber and ReadingInterharmonic)
    Referenced By: MeterReading (via Atom rel='related' links), IntervalBlock (indirectly via MeterReading)

    Tasks:

    1. Entity Updates (ReadingTypeEntity.java):

      • Review field order against espi.xsd ReadingType element sequence
      • Verify embedded RationalNumber and ReadingInterharmonic field order
      • Verify JPA @AttributeOverrides match schema constraints
      • Ensure JAXB annotations align with XML element names and order

    2. DTO Updates (ReadingTypeDto.java):

      • Match DTO field order to espi.xsd ReadingType element sequence
      • Verify embedded RationalNumber and ReadingInterharmonic DTOs
      • Add XSD constraint validation annotations

    3. MapStruct Mapper Updates (ReadingTypeMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping
      • Handle embedded object mappings

    4. Repository Updates (ReadingTypeRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (ReadingTypeService.java, ReadingTypeServiceImpl.java):

      • Review service methods for schema compliance
      • Update any field access order to match schema sequence

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match espi.xsd element sequence
      • Verify embedded object column order

    7. Testing:

      • Update unit tests for ReadingType entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against espi.xsd

    8. Commit, Push, PR

    Phase 3: IntervalReading

    Branch: feature/schema-compliance-phase-3-interval-reading

    Dependencies: None (embedded DateTimeInterval, owned by IntervalBlock)
    Referenced By: IntervalBlock (via collection, parent owns child), ReadingQuality (via Atom rel='related' links)

    Tasks:

    1. Entity Updates (IntervalReadingEntity.java):

      • Review field order against espi.xsd IntervalReading element sequence
      • Verify embedded DateTimeInterval (timePeriod) field order
      • Verify JPA annotations match schema constraints

    2. DTO Updates (IntervalReadingDto.java):

      • Match DTO field order to espi.xsd IntervalReading element sequence
      • Verify embedded DateTimeInterval DTO

    3. MapStruct Mapper Updates (IntervalReadingMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping
      • Handle embedded DateTimeInterval mapping

    4. Repository Updates (IntervalReadingRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (IntervalReadingService.java, IntervalReadingServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match espi.xsd element sequence

    7. Testing:

      • Update unit tests for IntervalReading entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against espi.xsd

    8. Commit, Push, PR

    Phase 4: ReadingQuality

    Branch: feature/schema-compliance-phase-4-reading-quality

    Dependencies: IntervalReading (via Atom rel='related' links)
    Referenced By: None

    Tasks:

    1. Entity Updates (ReadingQualityEntity.java):

      • Review field order against espi.xsd ReadingQuality element sequence
      • Verify relationship to IntervalReading

    2. DTO Updates (ReadingQualityDto.java):

      • Match DTO field order to espi.xsd ReadingQuality element sequence

    3. MapStruct Mapper Updates (ReadingQualityMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping

    4. Repository Updates (ReadingQualityRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (ReadingQualityService.java, ReadingQualityServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match espi.xsd element sequence

    7. Testing:

      • Update unit tests for ReadingQuality entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against espi.xsd

    8. Commit, Push, PR

    Phase 5: IntervalBlock

    Branch: feature/schema-compliance-phase-5-interval-block

    Dependencies: IntervalReading (contains collection), ReadingQuality (indirectly via IntervalReading)
    Referenced By: MeterReading (via bidirectional Atom rel='related' links)

    Tasks:

    1. Entity Updates (IntervalBlockEntity.java):

      • Review field order against espi.xsd IntervalBlock element sequence
      • Verify embedded DateTimeInterval (interval) field order
      • Verify IntervalReading collection mapping and order
      • Verify @onetomany cascade and orphanRemoval settings

    2. DTO Updates (IntervalBlockDto.java):

      • Match DTO field order to espi.xsd IntervalBlock element sequence
      • Verify IntervalReading collection DTO

    3. MapStruct Mapper Updates (IntervalBlockMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping
      • Handle IntervalReading collection mapping

    4. Repository Updates (IntervalBlockRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (IntervalBlockService.java, IntervalBlockServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match espi.xsd element sequence

    7. Testing:

      • Update unit tests for IntervalBlock entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against espi.xsd

    8. Commit, Push, PR

    Phase 6: MeterReading

    Branch: feature/schema-compliance-phase-6-meter-reading

    Dependencies: ReadingType (via Atom rel='related' links), IntervalBlock (via bidirectional Atom rel='related' links)
    Referenced By: UsagePoint (via bidirectional Atom rel='related' links)

    Note: MeterReading has NO child elements, ONLY Atom links to IntervalBlock and ReadingType

    Tasks:

    1. Entity Updates (MeterReadingEntity.java):

      • Review field order against espi.xsd MeterReading element sequence
      • Verify @manytoone relationship to UsagePoint
      • Verify @OnetoOne relationship to ReadingType (via Atom links, NOT embedded)
      • Verify relationship to IntervalBlock collection (via Atom links, NOT embedded)
      • Ensure NO embedded child elements exist

    2. DTO Updates (MeterReadingDto.java):

      • Match DTO field order to espi.xsd MeterReading element sequence
      • Verify ReadingType reference (Atom link, not embedded)
      • Verify IntervalBlock references (Atom links, not embedded collection)

    3. MapStruct Mapper Updates (MeterReadingMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping
      • Handle ReadingType and IntervalBlock reference mappings (NOT embedded objects)

    4. Repository Updates (MeterReadingRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (MeterReadingService.java, MeterReadingServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match espi.xsd element sequence

    7. Testing:

      • Update unit tests for MeterReading entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate that XML output contains ONLY Atom links, NO embedded elements
      • Validate generated XML against espi.xsd

    8. Commit, Push, PR

    Phase 7: ServiceDeliveryPoint

    Branch: feature/schema-compliance-phase-7-service-delivery-point

    Dependencies: None (standalone ESPI resource)
    Referenced By: UsagePoint (via @OnetoOne)

    Tasks:

    1. Entity Updates (ServiceDeliveryPointEntity.java):

      • Review field order against espi.xsd ServiceDeliveryPoint element sequence
      • Verify JPA annotations match schema constraints

    2. DTO Updates (ServiceDeliveryPointDto.java):

      • Match DTO field order to espi.xsd ServiceDeliveryPoint element sequence

    3. MapStruct Mapper Updates (ServiceDeliveryPointMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping

    4. Repository Updates (ServiceDeliveryPointRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (ServiceDeliveryPointService.java, ServiceDeliveryPointServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match espi.xsd element sequence

    7. Testing:

      • Update unit tests for ServiceDeliveryPoint entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against espi.xsd

    8. Commit, Push, PR

    Phase 8: PnodeRef

    Branch: feature/schema-compliance-phase-8-pnode-ref

    Dependencies: None (embedded in UsagePoint)
    Referenced By: UsagePoint (via collection)

    Tasks:

    1. Entity Updates (PnodeRefEntity.java):

      • Review field order against espi.xsd PnodeRef element sequence

    2. DTO Updates (PnodeRefDto.java):

      • Match DTO field order to espi.xsd PnodeRef element sequence

    3. MapStruct Mapper Updates (PnodeRefMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping

    4. Repository Updates (PnodeRefRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (PnodeRefService.java, PnodeRefServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match espi.xsd element sequence

    7. Testing:

      • Update unit tests for PnodeRef entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against espi.xsd

    8. Commit, Push, PR

    Phase 9: AggregatedNodeRef

    Branch: feature/schema-compliance-phase-9-aggregated-node-ref

    Dependencies: None (embedded in UsagePoint)
    Referenced By: UsagePoint (via collection)

    Tasks:

    1. Entity Updates (AggregatedNodeRefEntity.java):

      • Review field order against espi.xsd AggregatedNodeRef element sequence

    2. DTO Updates (AggregatedNodeRefDto.java):

      • Match DTO field order to espi.xsd AggregatedNodeRef element sequence

    3. MapStruct Mapper Updates (AggregatedNodeRefMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping

    4. Repository Updates (AggregatedNodeRefRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (AggregatedNodeRefService.java, AggregatedNodeRefServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match espi.xsd element sequence

    7. Testing:

      • Update unit tests for AggregatedNodeRef entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against espi.xsd

    8. Commit, Push, PR

    Phase 10: LineItem

    Branch: feature/schema-compliance-phase-10-line-item

    Dependencies: None (embedded child of UsageSummary)
    Referenced By: UsageSummary (via costAdditionalDetailLastPeriod embedded collection)

    Tasks:

    1. Entity Updates (LineItemEntity.java):

      • Review field order against espi.xsd LineItem element sequence
      • Verify embedded relationship to UsageSummary

    2. DTO Updates (LineItemDto.java):

      • Match DTO field order to espi.xsd LineItem element sequence

    3. MapStruct Mapper Updates (LineItemMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping

    4. Repository Updates (LineItemRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (LineItemService.java, LineItemServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match espi.xsd element sequence

    7. Testing:

      • Update unit tests for LineItem entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against espi.xsd

    8. Commit, Push, PR

    Phase 11: UsageSummary

    Branch: feature/schema-compliance-phase-11-usage-summary

    Dependencies: LineItem (embedded child via costAdditionalDetailLastPeriod)
    Referenced By: UsagePoint (via bidirectional Atom rel='related' links)

    Tasks:

    1. Entity Updates (UsageSummaryEntity.java):

      • Review field order against espi.xsd UsageSummary element sequence
      • Verify embedded DateTimeInterval fields (billingPeriod, etc.)
      • Verify embedded SummaryMeasurement fields
      • Verify LineItem collection (costAdditionalDetailLastPeriod)

    2. DTO Updates (UsageSummaryDto.java):

      • Match DTO field order to espi.xsd UsageSummary element sequence
      • Verify LineItem collection DTO

    3. MapStruct Mapper Updates (UsageSummaryMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping
      • Handle LineItem collection mapping

    4. Repository Updates (UsageSummaryRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (UsageSummaryService.java, UsageSummaryServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match espi.xsd element sequence

    7. Testing:

      • Update unit tests for UsageSummary entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against espi.xsd

    8. Commit, Push, PR

    Phase 12: ElectricPowerQualitySummary

    Branch: feature/schema-compliance-phase-12-electric-power-quality-summary

    Dependencies: None (standalone summary entity)
    Referenced By: UsagePoint (via bidirectional Atom rel='related' links)

    Tasks:

    1. Entity Updates (ElectricPowerQualitySummaryEntity.java):

      • Review field order against espi.xsd ElectricPowerQualitySummary element sequence
      • Verify embedded DateTimeInterval (summaryInterval)
      • Verify embedded SummaryMeasurement fields

    2. DTO Updates (ElectricPowerQualitySummaryDto.java):

      • Match DTO field order to espi.xsd ElectricPowerQualitySummary element sequence

    3. MapStruct Mapper Updates (ElectricPowerQualitySummaryMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping

    4. Repository Updates (ElectricPowerQualitySummaryRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (ElectricPowerQualitySummaryService.java, ElectricPowerQualitySummaryServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match espi.xsd element sequence

    7. Testing:

      • Update unit tests for ElectricPowerQualitySummary entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against espi.xsd

    8. Commit, Push, PR

    Phase 13: RetailCustomer

    Branch: feature/schema-compliance-phase-13-retail-customer

    Dependencies: None (standalone entity)
    Referenced By: UsagePoint (via @manytoone), Subscription (via @manytoone)

    Tasks:

    1. Entity Updates (RetailCustomerEntity.java):

      • Review field order against espi.xsd RetailCustomer element sequence
      • Verify relationships to UsagePoint collection
      • Verify relationships to Subscription collection

    2. DTO Updates (RetailCustomerDto.java):

      • Match DTO field order to espi.xsd RetailCustomer element sequence

    3. MapStruct Mapper Updates (RetailCustomerMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping

    4. Repository Updates (RetailCustomerRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (RetailCustomerService.java, RetailCustomerServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match espi.xsd element sequence

    7. Testing:

      • Update unit tests for RetailCustomer entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against espi.xsd

    8. Commit, Push, PR

    Phase 14: Subscription

    Branch: feature/schema-compliance-phase-14-subscription

    Dependencies: RetailCustomer (via @manytoone)
    Referenced By: UsagePoint (via @manytomany and @OnetoOne)

    Tasks:

    1. Entity Updates (SubscriptionEntity.java):

      • Review field order against espi.xsd Subscription element sequence
      • Verify relationship to RetailCustomer
      • Verify @manytomany relationship to UsagePoint collection
      • Verify @OnetoOne relationship to single UsagePoint

    2. DTO Updates (SubscriptionDto.java):

      • Match DTO field order to espi.xsd Subscription element sequence

    3. MapStruct Mapper Updates (SubscriptionMapper.java):

      • Update Entity-to-DTO conversion mapping
      • There is no DTO-to-Entity conversion mapping required

    4. Repository Updates (SubscriptionRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests
      • Replace @query statements that can use Data JPA named queries

    5. Service Updates (SubscriptionService.java, SubscriptionServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match espi.xsd element sequence

    7. Testing:

      • Update unit tests for Subscription entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against espi.xsd

    8. Commit, Push, PR

    Phase 15: BatchList

    Branch: feature/schema-compliance-phase-15-batch-list

    Dependencies: None (standalone entity)
    Referenced By: None

    Tasks:

    1. Entity Updates (BatchListEntity.java):

      • Review field order against espi.xsd BatchList element sequence

    2. DTO Updates (BatchListDto.java):

      • Match DTO field order to espi.xsd BatchList element sequence

    3. MapStruct Mapper Updates (BatchListMapper.java):

      • Update Entity-to-DTO conversion mapping
      • No DTO-to-Entity conversion mapping is required

    4. Repository Updates (BatchListRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests
      • Replace any @query statements that are redundant for Data JPA named queries

    5. Service Updates (BatchListService.java, BatchListServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match espi.xsd element sequence

    7. Testing:

      • Update unit tests for BatchList entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against espi.xsd

    8. Commit, Push, PR

    Phase 16: UsagePoint

    Branch: feature/schema-compliance-phase-16-usage-point

    Dependencies: ServiceDeliveryPoint, PnodeRef, AggregatedNodeRef, TimeConfiguration (via Atom links), MeterReading (via Atom links), UsageSummary (via Atom links), ElectricPowerQualitySummary (via Atom links), RetailCustomer (via @manytoone), Subscription (via @manytomany and @OnetoOne)
    Referenced By: ServiceLocation (cross-stream href reference from customer.xsd PII stream)

    Tasks:

    1. Entity Updates (UsagePointEntity.java):

      • Review field order against espi.xsd UsagePoint element sequence
      • Verify embedded SummaryMeasurement fields (estimatedLoad, nominalServiceVoltage, ratedCurrent, ratedPower)
      • Verify @OnetoOne relationship to ServiceDeliveryPoint
      • Verify @manytoone relationship to RetailCustomer
      • Verify @manytoone relationship to TimeConfiguration (via Atom links)
      • Verify @onetomany relationships: MeterReading, UsageSummary, ElectricPowerQualitySummary, PnodeRef, AggregatedNodeRef collections
      • Verify @manytomany relationship to Subscription collection
      • Verify @OnetoOne relationship to single Subscription

    2. DTO Updates (UsagePointDto.java):

      • Match DTO field order to espi.xsd UsagePoint element sequence

    3. MapStruct Mapper Updates (UsagePointMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping
      • Handle all embedded and related entity mappings

    4. Repository Updates (UsagePointRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests
      • Remove any @query statements that can be performed using Spring Data JPA named queries

    5. Service Updates (UsagePointService.java, UsagePointServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match espi.xsd element sequence

    7. Testing:

      • Update unit tests for UsagePoint entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against espi.xsd

    8. Commit, Push, PR

    Customer Domain Entities (Phases 17-26)

    Phase 17: ProgramDateIdMappings

    Branch: feature/schema-compliance-phase-17-program-date-id-mappings

    Dependencies: None (standalone entity)
    Referenced By: CustomerAgreement (via bidirectional Atom rel='related' links)

    Tasks:

    1. Entity Updates (ProgramDateIdMappingsEntity.java):

      • Review field order against customer.xsd ProgramDateIdMappings element sequence
      • Verify JPA annotations match schema constraints
      • Verify JAXB annotations align with XML element names and order

    2. DTO Updates (ProgramDateIdMappingsDto.java):

      • Match DTO field order to customer.xsd ProgramDateIdMappings element sequence
      • Verify JAXB annotations for XML marshalling

    3. MapStruct Mapper Updates (ProgramDateIdMappingsMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping

    4. Repository Updates (ProgramDateIdMappingsRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests
      • Remove any @query statements that can be performed using Spring Data JPA named queries

    5. Service Updates (ProgramDateIdMappingsService.java, ProgramDateIdMappingsServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match the customer.xsd element sequence

    7. Testing:

      • Review all Customer Tests and use it as a template for developing tests
      • Update unit tests for ProgramDateIdMappings entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against customer.xsd
      • Run all tests, including integration tests, to ensure CI/CD passes

    8. Commit, Push, PR

    Phase 18: CustomerAccount

    Branch: feature/schema-compliance-phase-18-customer-account

    Dependencies: None (standalone entity)
    Referenced By: CustomerAgreement (via bidirectional Atom rel='related' links), Statement (via bidirectional Atom rel='related' links), Customer (via bidirectional Atom rel='related' links)

    Tasks:

    1. Entity Updates (CustomerAccountEntity.java):

      • Review field order against customer.xsd CustomerAccount element sequence
      • Verify embedded AccountNotification field order

    2. DTO Updates (CustomerAccountDto.java):

      • Match DTO field order to customer.xsd CustomerAccount element sequence

    3. MapStruct Mapper Updates (CustomerAccountMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping

    4. Repository Updates (CustomerAccountRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (CustomerAccountService.java, CustomerAccountServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match customer.xsd element sequence

    7. Testing:

      • Update unit tests for CustomerAccount entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against customer.xsd

    8. Commit, Push, PR

    Phase 19: Statement

    Branch: feature/schema-compliance-phase-19-statement

    Dependencies: None (standalone entity)
    Referenced By: Customer (via bidirectional Atom rel='related' links), CustomerAccount (via bidirectional Atom rel='related' links)

    Tasks:

    1. Entity Updates (StatementEntity.java):

      • Review field order against customer.xsd Statement element sequence
      • Verify embedded DateTimeInterval (billingPeriod) field order
      • Verify embedded SummaryMeasurement fields

    2. DTO Updates (StatementDto.java):

      • Match DTO field order to customer.xsd Statement element sequence

    3. MapStruct Mapper Updates (StatementMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping

    4. Repository Updates (StatementRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (StatementService.java, StatementServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match customer.xsd element sequence

    7. Testing:

      • Review all Customer Tests and use it as a template for developing tests
      • Update unit tests for Statement entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against customer.xsd
      • Run all tests, including integration tests, to ensure CI/CD passes

    8. Commit, Push, PR

    Phase 20: Customer

    Branch: feature/schema-compliance-phase-20-customer

    Dependencies: TimeConfiguration (via bidirectional Atom rel='related' links), Statement (via bidirectional Atom rel='related' links), CustomerAccount (via bidirectional Atom rel='related' links)
    Referenced By: None

    Tasks:

    1. Entity Updates (CustomerEntity.java):

      • Review field order against customer.xsd Customer element sequence
      • Verify embedded Organisation fields (organisation, residence)
      • Verify embedded Status, Priority fields
      • Verify relationships to TimeConfiguration, Statement, CustomerAccount

    2. DTO Updates (CustomerDto.java):

      • Match DTO field order to customer.xsd Customer element sequence

    3. MapStruct Mapper Updates (CustomerMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping

    4. Repository Updates (CustomerRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (CustomerService.java, CustomerServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match customer.xsd element sequence

    7. Testing:

      • Review all Customer Tests and use it as a template for developing tests
      • Update unit tests for Customer entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against customer.xsd

    8. Commit, Push, PR

    Phase 21: ServiceSupplier

    Branch: feature/schema-compliance-phase-21-service-supplier

    Dependencies: None (standalone entity)
    Referenced By: CustomerAgreement (via bidirectional Atom rel='related' links)

    Tasks:

    1. Entity Updates (ServiceSupplierEntity.java):

      • Review field order against customer.xsd ServiceSupplier element sequence
      • Verify embedded Organisation fields

    2. DTO Updates (ServiceSupplierDto.java):

      • Match DTO field order to customer.xsd ServiceSupplier element sequence

    3. MapStruct Mapper Updates (ServiceSupplierMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping

    4. Repository Updates (ServiceSupplierRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (ServiceSupplierService.java, ServiceSupplierServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match customer.xsd element sequence

    7. Testing:

      • Review all Customer Tests and use it as a template for developing tests
      • Update unit tests for ServiceSupplier entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against customer.xsd
      • Run all tests, including integration tests, to ensure CI/CD passes

    8. Commit, Push, PR

    Phase 22: Asset

    Branch: feature/schema-compliance-phase-22-asset

    Dependencies: None (base class for EndDevice)
    Referenced By: EndDevice (subclass)

    Tasks:

    1. Entity Updates (AssetEntity.java):

      • Review field order against customer.xsd Asset element sequence
      • Verify embedded Status field order
      • Verify JPA @Inheritance and @DiscriminatorColumn annotations

    2. DTO Updates (AssetDto.java):

      • Match DTO field order to customer.xsd Asset element sequence

    3. MapStruct Mapper Updates (AssetMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping

    4. Repository Updates (AssetRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (AssetService.java, AssetServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match customer.xsd element sequence

    7. Testing:

      • Review all Customer Tests and use it as a template for developing tests
      • Update unit tests for Asset entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against customer.xsd
      • Run all tests, including integration tests, to ensure CI/CD passes

    8. Commit, Push, PR

    Phase 23: ServiceLocation

    Branch: feature/schema-compliance-phase-23-service-location

    Dependencies: TimeConfiguration (via bidirectional Atom rel='related' links), UsagePoint (cross-stream reference via href URL from espi.xsd), EndDevice (via bidirectional Atom rel='related' links)
    Referenced By: CustomerAgreement (via bidirectional Atom rel='related' links), EndDevice (via bidirectional Atom rel='related' links)

    Note: ServiceLocation exists in customer.xsd PII data stream, UsagePoint exists in espi.xsd non-PII data stream. ServiceLocation stores UsagePoint's rel="self" href URL directly, NOT via Atom link element.

    Tasks:

    1. Entity Updates (ServiceLocationEntity.java):

      • Review field order against customer.xsd ServiceLocation element sequence
      • Verify embedded StreetAddress fields (mainAddress, secondaryAddress)
      • Verify embedded ElectronicAddress field
      • Verify embedded Status field
      • Verify relationship to TimeConfiguration (via Atom links)
      • Verify UsagePoint href URL storage (String field, NOT Atom link)
      • Verify relationship to EndDevice (via Atom links)

    2. DTO Updates (ServiceLocationDto.java):

      • Match DTO field order to customer.xsd ServiceLocation element sequence
      • Verify UsagePoint href URL field

    3. MapStruct Mapper Updates (ServiceLocationMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping

    4. Repository Updates (ServiceLocationRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (ServiceLocationService.java, ServiceLocationServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match customer.xsd element sequence

    7. Testing:

      • Review all Customer Tests and use it as a template for developing tests
      • Update unit tests for ServiceLocation entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate that UsagePoint reference is href URL string, NOT Atom link
      • Validate generated XML against customer.xsd
      • Run all tests, including integration tests, to ensure CI/CD passes

    8. Commit, Push, PR

    Phase 24: CustomerAgreement

    Branch: feature/schema-compliance-phase-24-customer-agreement

    Dependencies: CustomerAccount (via bidirectional Atom rel='related' links), ServiceLocation (via bidirectional Atom rel='related' links), ServiceSupplier (via bidirectional Atom rel='related' links), ProgramDateIdMappings (via bidirectional Atom rel='related' links)
    Referenced By: ServiceSupplier (via bidirectional Atom rel='related' links), ServiceLocation (via bidirectional Atom rel='related' links), ProgramDateIdMappings (via bidirectional Atom rel='related' links)

    Tasks:

    1. Entity Updates (CustomerAgreementEntity.java):

      • Review field order against customer.xsd CustomerAgreement element sequence
      • Verify embedded DateTimeInterval (validityInterval) field order
      • Verify relationships to CustomerAccount, ServiceLocation, ServiceSupplier, ProgramDateIdMappings

    2. DTO Updates (CustomerAgreementDto.java):

      • Match DTO field order to customer.xsd CustomerAgreement element sequence

    3. MapStruct Mapper Updates (CustomerAgreementMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping

    4. Repository Updates (CustomerAgreementRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (CustomerAgreementService.java, CustomerAgreementServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match customer.xsd element sequence

    7. Testing:

      • Review all Customer Tests and use it as a template for developing tests
      • Update unit tests for CustomerAgreement entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against customer.xsd
      • Run all tests, including integration tests, to ensure CI/CD passes

    8. Commit, Push, PR

    Phase 25: EndDevice

    Branch: feature/schema-compliance-phase-25-end-device

    Dependencies: Asset (base class), ServiceLocation (via bidirectional Atom rel='related' links)
    Referenced By: Meter (subclass), ServiceLocation (via bidirectional Atom rel='related' links)

    Note: EndDevice and ServiceLocation have bidirectional Atom rel='related' links. Since ServiceLocation is processed in Phase 23, the EndDevice → ServiceLocation link will be added in this phase.

    Tasks:

    1. Entity Updates (EndDeviceEntity.java):

      • Review field order against customer.xsd EndDevice element sequence
      • Verify inheritance from AssetEntity
      • Verify embedded Status, Organisation.ElectronicAddress fields
      • Verify relationship to ServiceLocation (via Atom links)

    2. DTO Updates (EndDeviceDto.java):

      • Match DTO field order to customer.xsd EndDevice element sequence

    3. MapStruct Mapper Updates (EndDeviceMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping

    4. Repository Updates (EndDeviceRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (EndDeviceService.java, EndDeviceServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match customer.xsd element sequence

    7. Testing:

      • Review all Customer Tests and use it as a template for developing tests
      • Update unit tests for EndDevice entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate bidirectional Atom link with ServiceLocation
      • Validate generated XML against customer.xsd
      • Run all tests, including integration tests, to ensure CI/CD passes

    8. Commit, Push, PR

    Phase 26: Meter

    Branch: feature/schema-compliance-phase-26-meter

    Dependencies: EndDevice (base class), ServiceLocation (inherited from EndDevice via Atom links)
    Referenced By: None

    Tasks:

    1. Entity Updates (MeterEntity.java):

      • Review field order against customer.xsd Meter element sequence
      • Verify inheritance from EndDeviceEntity
      • Verify Meter-specific fields (formNumber, intervalLength)

    2. DTO Updates (MeterDto.java):

      • Match DTO field order to customer.xsd Meter element sequence

    3. MapStruct Mapper Updates (MeterMapper.java):

      • Update Entity-to-DTO conversion mapping
      • Update DTO-to-Entity conversion mapping

    4. Repository Updates (MeterRepository.java):

      • Keep ONLY index field queries
      • Remove all non-index queries not required for tests

    5. Service Updates (MeterService.java, MeterServiceImpl.java):

      • Review service methods for schema compliance

    6. Flyway Migration Updates:

      • Update original Flyway scripts to match customer.xsd element sequence

    7. Testing:

      • Review all Customer Tests and use it as a template for developing tests
      • Update unit tests for Meter entity, service, repository
      • Add/update integration tests using TestContainers
      • Add XML marshalling/unmarshalling tests to verify schema compliance
      • Validate generated XML against customer.xsd
      • Run all tests, including integration tests, to ensure CI/CD passes

    8. Commit, Push, PR

    Summary

    This 26-phase plan ensures comprehensive schema compliance review for all NAESB ESPI 4.0 domain entities and associated classes. Each phase focuses on a single entity and includes:

    1. Entity field order verification against XSD schema
    2. DTO field order verification and JAXB annotations
    3. Bidirectional MapStruct mapper updates (Entity-to-DTO and DTO-to-Entity)
    4. Repository query simplification (index fields only)
    5. Service method schema compliance review
    6. Flyway migration script updates (original scripts, no new scripts)
    7. Comprehensive testing including XML marshalling/unmarshalling and XSD validation
    8. Git workflow (branch, commit, push, PR merge)

    Processing Order:

    • Usage domain first (Phases 1-16)
    • Customer domain second (Phases 17-26)
    • Dependency order (parents before children)
    • Simple to complex entities within each dependency level

    Key Notes:

    • MeterReading has NO child elements, ONLY Atom links
    • ServiceLocation (customer.xsd) references UsagePoint (espi.xsd) via href URL string, NOT Atom link (cross-stream reference)
    • UsageSummary contains LineItem collection (costAdditionalDetailLastPeriod)
    • EndDevice and ServiceLocation have bidirectional Atom rel='related' links
    • CustomerAgreement has bidirectional Atom links with ServiceSupplier, ServiceLocation, and ProgramDateIdMappings
    • Statement has bidirectional Atom links with both Customer and CustomerAccount
    • Customer has bidirectional Atom links with TimeConfiguration, Statement, and CustomerAccount

    Git Branch Naming Pattern:
    feature/schema-compliance-phase-{number}-{entity-name}

    Test Requirements:

    • Unit tests for entity, service, repository
    • Integration tests using TestContainers (MySQL, PostgreSQL, H2)
    • XML marshalling/unmarshalling tests for ALL phases
    • XSD schema validation (espi.xsd for usage domain, customer.xsd for customer domain)
  4. dfcoffin
    added 3 commits that reference this issue on Dec 26, 2025

    Add multi-phase ESPI schema compliance plan and reference diagrams

    b06f468

    Add multi-phase ESPI schema compliance plan and reference diagrams (#49

    8cbe19c

    docs: Add DTO pattern guide and update schema compliance plan

    b3b63ec
  5. dfcoffin
    added 3 commits that reference this issue on Jan 7, 2026

    refactor: remove 11 incorrect related_links tables (Issue #28 Phase A)

    5313c62

    refactor: ESPI 4.0 Schema Compliance - Phase 0: Fix Object-based enti…

    1628ade

    refactor: ESPI 4.0 Schema Compliance - Phase 0: Fix Object-based enti…

    701d407
  6. dfcoffin
    closed this as completedin #66on Jan 7, 2026

  7. dfcoffin commented on Jan 7, 2026

    @dfcoffin
    dfcoffin
    on Jan 7, 2026
    ContributorAuthor

    Only Phase 1 and 2 have been completed

  8. dfcoffin
    reopened this on Jan 7, 2026
  9. dfcoffin
    added a commit that references this issue on Jan 8, 2026

    refactor: Phase 2 - ReadingType ESPI 4.0 schema compliance (field order)

    8ecad71

  10. dfcoffin commented on Jan 8, 2026

    @dfcoffin
    dfcoffin
    on Jan 8, 2026
    ContributorAuthor

    Phase 2 Complete: ReadingType Schema Compliance ✅

    Completed: ReadingType field ordering to match ESPI 4.0 XSD specification (espi.xsd:382-481)

    Changes Merged (PR #67):

    • Reordered ReadingTypeEntity fields 13-18 to match XSD sequence: timeAttribute → tou → uom → cpp → interharmonic → measuringPeriod → argument
    • Updated ReadingTypeDto @XmlType propOrder and record component order
    • Removed description from ReadingType propOrder (belongs to IdentifiedObject/Atom wrapper)
    • Updated V1__Create_Base_Tables.sql to match XSD column order
    • All 550 tests passing ✅

    Progress: Phase 2 of 26 completed

    Next: Phase 3 - IntervalReading schema compliance

    Part of ESPI 4.0 Schema Compliance - Multi-Phase Plan

  11. dfcoffin commented on Jan 8, 2026

    @dfcoffin
    dfcoffin
    on Jan 8, 2026
    ContributorAuthor

    Phase 3 Completed ✅

    IntervalReadingEntity has been updated to comply with ESPI 4.0 schema (espi.xsd:1016).

    Changes Merged in PR #68

    IntervalReadingEntity:

    • ✅ Changed from extending IdentifiedObject to plain JPA entity (Object pattern)
    • ✅ Changed ID from UUID to Long (48+ bits) with @GeneratedValue(strategy = GenerationType.IDENTITY)
    • ✅ Removed all Atom metadata fields (description, created, updated, published, links)
    • ✅ Reordered fields to match XSD sequence: cost → readingQualities → timePeriod → value → consumptionTier → tou → cpp

    Database Migrations:

    • ✅ Updated interval_readings table to use BIGINT AUTO_INCREMENT (MySQL/H2) or BIGSERIAL (PostgreSQL)
    • ✅ Moved table creation to vendor-specific V2 migration files
    • ✅ Updated reading_qualities.interval_reading_id FK from CHAR(36) to BIGINT

    Test Results:

    • ✅ All 550 tests passing locally
    • ✅ CI/CD pipeline passed

    Updated Status

    Entity Status PR/Commit
    AggregatedNodeRefEntity ❌ Not started -
    BatchListEntity ❌ Not started -
    IntervalReadingEntity ✅ Complete PR #68
    LineItemEntity ❌ Not started -
    PnodeRefEntity ❌ Not started -
    ReadingQualityEntity ❌ Not started -
    RetailCustomerEntity ❌ Not started -
    ServiceDeliveryPointEntity ✅ Complete PR #66 (Phase 0)

    Next Steps

    Remaining entities to update:

    1. ReadingQualityEntity (extends Object per espi.xsd:1062)
    2. PnodeRefEntity (extends Object)
    3. AggregatedNodeRefEntity (extends Object)
    4. LineItemEntity (extends Object)
    5. RetailCustomerEntity (not defined by ESPI)
    6. BatchListEntity (BatchListType)

    Related: Phase 3 tracked in multi-phase compliance plan
    Commit: b9127e2

  12. 48 remaining items

  13. dfcoffin commented on Jan 29, 2026

    @dfcoffin
    dfcoffin
    on Jan 29, 2026
    ContributorAuthor

    Phase 25: EndDevice - Complete ✅

    Phase 25 implementation is complete with full ESPI 4.0 XSD compliance for EndDevice entity.

    Pull Request

    Phase 25 Deliverables

    1. EndDevice Implementation

    • ✅ EndDeviceEntity - Matches customer.xsd lines 539-881 (12 Asset + 4 EndDevice fields)
    • ✅ EndDeviceDto - ESPI 4.0 compliant DTO (removed Atom fields, added 16 XSD fields)
    • ✅ EndDeviceMapper - Entity-to-DTO mapping with LifecycleDateMapper and AcceptanceTestMapper
    • ✅ EndDeviceRepository - JPA repository for database operations
    • ✅ EndDeviceService + EndDeviceServiceImpl - Service layer implementation

    2. XSD Compliance Verification

    EndDevice 16 Fields (customer.xsd):

    • Asset fields (12): aliasName, initialCondition, initialLossOfLife, lifecycleDate, serialNumber, type, utcNumber, lotNumber, electronicAddress, acceptanceTest, status, category
    • EndDevice fields (4): amrSystem, installCode, isPan, timeZoneOffset

    All fields properly mapped to:

    • Entity @column annotations
    • DTO @xmlelement annotations with proper namespace
    • MapStruct mapper methods
    • Database columns in Flyway migration

    3. Test Coverage

    Unit Tests (18 tests):

    • EndDeviceDtoTest.java - XML marshalling validation (5 tests)
    • EndDeviceRepositoryTest.java - Repository operations (13 tests)

    Integration Tests (16 tests):

    • EndDeviceMySQLIntegrationTest.java (8 tests)
    • EndDevicePostgreSQLIntegrationTest.java (8 tests)

    4. Integration Tests Backfill

    Created 8 integration test files for Phases 18, 23, 24, 25:

    • ✅ CustomerAccountMySQLIntegrationTest.java (Phase 18)
    • ✅ CustomerAccountPostgreSQLIntegrationTest.java (Phase 18)
    • ✅ ServiceLocationMySQLIntegrationTest.java (Phase 23)
    • ✅ ServiceLocationPostgreSQLIntegrationTest.java (Phase 23)
    • ✅ CustomerAgreementMySQLIntegrationTest.java (Phase 24)
    • ✅ CustomerAgreementPostgreSQLIntegrationTest.java (Phase 24)
    • ✅ EndDeviceMySQLIntegrationTest.java (Phase 25)
    • ✅ EndDevicePostgreSQLIntegrationTest.java (Phase 25)

    ESPI 4.0 DTO Cleanup (Critical Achievement)

    All DTOs Now Strictly XSD-Compliant

    Cleaned 13 DTOs across customer and usage domains:

    Customer Domain (4 DTOs):

    • CustomerDto
    • CustomerAccountDto
    • CustomerAgreementDto
    • ServiceLocationDto

    Usage Domain (9 DTOs):

    • UsagePointDto
    • MeterReadingDto
    • IntervalBlockDto
    • ReadingTypeDto
    • TimeConfigurationDto
    • UsageSummaryDto
    • ElectricPowerQualitySummaryDto
    • ApplicationInformationDto
    • AuthorizationDto

    Changes Applied

    • ❌ Removed id field (Long) from all DTOs
    • ❌ Removed uuid field (String) from all DTOs
    • ✅ Resource DTOs now contain ONLY XSD-defined fields
    • ✅ AtomEntryDto handles all Atom protocol metadata (id, published, updated, links)
    • ✅ MapStruct: @Mapping(target = "id", ignore = true) in toEntity methods only
    • ✅ Service layer maps: entity.id ↔ AtomEntryDto.id

    Bug Fixes

    1. NotificationMethodKind Enum Alignment

    Issue: Java enum values didn't match customer.xsd specification

    Before:

    EMAIL, PHONE, SMS, LETTER, IN_PERSON, OTHER

    After (matches XSD lines 1961-1996):

    CALL, EMAIL, LETTER, OTHER, IVR

    2. JAXB OffsetDateTime Marshalling

    Issue: OffsetDateTime fields serializing as empty XML elements

    Fix: Added @XmlJavaTypeAdapter(OffsetDateTimeAdapter.class) to:

    • LifecycleDateDto: manufacturedDate, installationDate
    • AcceptanceTestDto: dateTime

    Result: Proper ISO-8601 timestamp serialization

    3. Test Constructor Fixes

    Fixed 46 constructor calls across 16 test files to match cleaned DTO signatures.

    Test Results

    Unit Tests: 654 tests, 0 failures, 0 errors
Integration Tests: 99 tests, 0 failures, 0 errors
BUILD SUCCESS

    Database Migration

    • ✅ Verified end_devices table in V3__Create_additiional_Base_Tables.sql
    • ✅ All 16 EndDevice XSD fields properly mapped to database columns
    • ✅ Status, LifecycleDate, AcceptanceTest embedded objects configured
    • ✅ ElectronicAddress collection table created

    ESPI 4.0 Compliance Status

    ✅ EndDevice - Fully compliant with customer.xsd lines 539-881
    ✅ All Customer DTOs - Strictly XSD-compliant (no Atom metadata)
    ✅ All Usage DTOs - Strictly XSD-compliant (no Atom metadata)
    ✅ AtomEntryDto Pattern - Properly separates Atom protocol from ESPI resources
    ✅ MapStruct Mappings - Correct ignore patterns for id fields
    ✅ Service Layer - Proper entity.id ↔ AtomEntryDto.id mapping

    Next Steps

    This phase completes EndDevice implementation. The issue remains open for:

    1. Additional entity reviews as needed
    2. Tracking remaining XSD compliance work across other entities
    3. Future customer domain entity implementations

    Files Changed: 60 files
    Lines Added: 4,310 insertions
    Lines Removed: 406 deletions
    New Files: 16

    🤖 Generated with Claude Code

  14. springframeworkguru
    added a commit that references this issue on Jan 29, 2026

    feat: ESPI 4.0 Schema Compliance - Phase 24: CustomerAgreement Comple…

    c1fa7bd
  15. dfcoffin
    added a commit that references this issue on Jan 31, 2026

    feat: ESPI 4.0 Schema Compliance - Phase 21: ServiceSupplier Complete…

    7c54297

  16. dfcoffin commented on Jan 31, 2026

    @dfcoffin
    dfcoffin
    on Jan 31, 2026
    ContributorAuthor

    Phase 21: ServiceSupplier Complete Implementation ✅

    Pull Request: #96
    Branch: feature/schema-compliance-phase-21-service-supplier
    Status: Ready for review

    Overview

    Phase 21 implements full ESPI 4.0 schema compliance for ServiceSupplier and refactors all Organisation phone number handling from the legacy PhoneNumberEntity polymorphic table pattern to embedded TelephoneNumber fields. This comprehensive update affects ServiceSupplier, Customer, CustomerAccount, ServiceLocation, and all related infrastructure.

    Key Achievements

    1. New DTOs (JAXB/XML Marshalling)

    Created three standalone DTOs with proper JAXB annotations:

    • ✅ ServiceSupplierDto - 4 fields (organisation, kind, issuerIdentificationNumber, effectiveDate)
    • ✅ OrganisationDto - 6 fields with embedded phone1/phone2
    • ✅ TelephoneNumberDto - 8 fields matching customer.xsd specification

    All DTOs follow the XSD-compliant pattern: NO IdentifiedObject fields (id, uuid) - these are handled by AtomEntryDto.

    2. New MapStruct Mappers

    Implemented three mappers for entity ↔ DTO conversion:

    • ✅ ServiceSupplierMapper - Maps 4 ServiceSupplier-specific fields
    • ✅ OrganisationMapper - Maps Organisation with embedded phone1/phone2
    • ✅ TelephoneNumberMapper - Maps all 8 TelephoneNumber fields

    3. SupplierKind Enum Fix (Critical)

    Issue: Enum values didn't match customer.xsd specification (lines 2231-2271)

    Before (WRONG):

    RETAIL, GENERATION, TRANSMISSION, DISTRIBUTION, OTHER  // 5 values, wrong names

    After (CORRECT - matches XSD):

    UTILITY, RETAILER, OTHER, LSE, MDMA, MSP  // 6 values, correct sequence

    Why Critical: ESPI standard uses ordinal values (0-5) for serialization. Sequence must match XSD exactly for compatibility with ESPI Sandbox and CMD Certification Platform.

    4. Phone Number Architecture Refactoring

    From: PhoneNumberEntity polymorphic table with discriminator column
    To: Embedded TelephoneNumber fields (phone1, phone2)

    Benefits:

    • ✅ XSD-compliant structure (matches customer.xsd lines 1089-1125)
    • ✅ Type-safe (no string discriminators)
    • ✅ Better performance (no JOINs, single table queries)
    • ✅ Standard JPA embedded pattern
    • ✅ Simpler queries and mappings

    Entities Updated:

    • Organisation - Added phone1/phone2 embedded fields
    • ServiceSupplierEntity - 16 phone @AttributeOverride annotations
    • CustomerEntity - 16 phone @AttributeOverride annotations
    • CustomerAccountEntity - 16 phone @AttributeOverride annotations
    • ServiceLocationEntity - Already had embedded phones, sizes optimized

    5. TelephoneNumber Column Size Optimization

    Issue: MySQL row size limit (65,535 bytes) exceeded with VARCHAR(256) columns

    Solution: Optimized column sizes based on actual data requirements:

    countryCode: VARCHAR(10)           // Was: VARCHAR(256)
areaCode: VARCHAR(10)              // Was: VARCHAR(256)
cityCode: VARCHAR(10)              // Was: VARCHAR(256)
localNumber: VARCHAR(30)           // Was: VARCHAR(256)
ext: VARCHAR(20)                   // Was: VARCHAR(256)
dialOut: VARCHAR(10)               // Was: VARCHAR(256)
internationalPrefix: VARCHAR(10)   // Was: VARCHAR(256)
ituPhone: VARCHAR(50)              // Was: VARCHAR(256)

    Impact: Reduced row size by ~1,648 bytes per embedded Organisation (2 phones × 8 fields × 103 bytes saved per field)

    6. Legacy Code Cleanup

    Deleted Files (No Longer Needed):

    • ❌ OrganisationRole.java - Unused wrapper with no additional fields
    • ❌ PhoneNumberEntity.java - Replaced with embedded TelephoneNumber
    • ❌ PhoneNumberService.java - Service for managing PhoneNumberEntity relationships

    Mapper Cleanup:

    • Removed extends BaseMapperUtils from CustomerMapper (wasn't using any methods)
    • Removed extends BaseMapperUtils from ServiceSupplierMapper (wasn't using any methods)
    • Removed BaseMapperUtils.class from ElectricPowerQualitySummaryMapper uses clause

    DTO Cleanup:

    • Removed legacy usagePointId field from ElectricPowerQualitySummaryDto (@XmlTransient field serving no purpose)

    Issue Resolved: Spring bean conflict - NoUniqueBeanDefinitionException when multiple mappers extended BaseMapperUtils

    7. Database Migration Updates

    File: V3__Create_additiional_Base_Tables.sql

    Changes:

    • ✅ Added 16 phone columns to service_suppliers table
    • ✅ Added 16 phone columns to customers table
    • ✅ Added 16 phone columns to customer_accounts table
    • ✅ Optimized phone column sizes in service_locations table
    • ❌ Removed phone_numbers table (no longer needed)

    Column Naming Pattern:

    -- ServiceSupplier
supplier_phone1_country_code, supplier_phone1_area_code, ...
supplier_phone2_country_code, supplier_phone2_area_code, ...

-- Customer
customer_phone1_country_code, customer_phone1_area_code, ...
customer_phone2_country_code, customer_phone2_area_code, ...

-- CustomerAccount
contact_phone1_country_code, contact_phone1_area_code, ...
contact_phone2_country_code, contact_phone2_area_code, ...

    8. Mapper Updates

    CustomerMapper.java:

    • Removed @Mapping(target = "phoneNumbers", ignore = true)
    • Deleted extractPhoneByType() method (30+ lines)
    • Updated mapOrganisationToDto() to use direct phone1/phone2 mapping
    • Added telephoneNumberToDto() helper method
    • Removed PhoneNumberEntity imports and logic

    ServiceLocationMapper.java:

    • Added TelephoneNumberMapper.class to uses clause
    • Removed 30+ lines of custom mapTelephone() methods
    • Now leverages TelephoneNumberMapper for centralized mapping logic

    9. DtoExportService Updates

    File: DtoExportServiceImpl.java

    Changes:

    • Added ServiceSupplierMapper injection
    • Registered ServiceSupplierDto.class in CustomerExportService JAXBContext
    • Uncommented @XmlElement for ServiceSupplier in CustomerAtomEntryDto.java

    Result: ServiceSupplier can now be properly marshalled to XML with correct namespace and element name

    New Tests (24 Tests Added)

    ServiceSupplierDtoTest (8 tests)

    • ✅ shouldMarshalServiceSupplierWithAllFields()
    • ✅ shouldMarshalServiceSupplierWithMinimalFields()
    • ✅ shouldUnmarshalServiceSupplierXml()
    • ✅ shouldHandleNullFields()
    • ✅ shouldVerifyXmlNamespaceAndElements()
    • ✅ shouldVerifySupplierKindEnumValues()
    • ✅ shouldVerifyServiceSupplierFieldOrder()
    • ✅ shouldMarshalWithEmbeddedOrganisation()

    OrganisationDtoTest (8 tests)

    • ✅ shouldMarshalOrganisationWithAllFields()
    • ✅ shouldMarshalOrganisationWithMinimalFields()
    • ✅ shouldUnmarshalOrganisationXml()
    • ✅ shouldHandleNullFields()
    • ✅ shouldVerifyXmlNamespaceAndElements()
    • ✅ shouldVerifyOrganisationFieldOrder()
    • ✅ shouldMarshalWithEmbeddedPhoneNumbers()
    • ✅ shouldMarshalWithEmbeddedAddresses()

    TelephoneNumberDtoTest (8 tests)

    • ✅ shouldMarshalTelephoneNumberWithAllFields()
    • ✅ shouldMarshalTelephoneNumberWithMinimalFields()
    • ✅ shouldUnmarshalTelephoneNumberXml()
    • ✅ shouldHandleNullFields()
    • ✅ shouldVerifyXmlNamespaceAndElements()
    • ✅ shouldVerifyTelephoneNumberFieldOrder()
    • ✅ shouldFormatInternationalNumber()
    • ✅ shouldFormatNorthAmericanNumber()

    Test Results

    Unit Tests: 760 tests, 0 failures, 0 errors
Integration Tests:
  - PostgreSQL TestContainer: 2/2 ✓
  - H2 In-Memory: 3/3 ✓
  - MySQL TestContainer: 2/2 ✓
BUILD SUCCESS

    Test Count Evolution:

    • Before Phase 21: 736 tests
    • After Phase 21: 760 tests
    • Added: 24 new DTO tests

    Breaking Changes

    ⚠️ Database Schema Changes:

    • phone_numbers table removed
    • 16 phone columns added to: service_suppliers, customers, customer_accounts tables
    • Phone column sizes reduced in: service_locations table

    ⚠️ Code Changes:

    • PhoneNumberEntity, OrganisationRole, PhoneNumberService deleted
    • SupplierKind enum values changed to match XSD specification
    • Any code referencing deleted classes needs updates

    ⚠️ Migration Required:
    Existing deployments must run Flyway migrations. Data in phone_numbers table will be lost if not manually migrated before deployment.

    Files Changed

    Total: 34 files changed

    • Insertions: 2,214 lines
    • Deletions: 865 lines
    • Net Change: +1,349 lines

    Created (6 files):

    • PHASE_21_SERVICE_SUPPLIER_IMPLEMENTATION_PLAN.md
    • ServiceSupplierDto.java
    • OrganisationDto.java
    • TelephoneNumberDto.java
    • ServiceSupplierMapper.java
    • TelephoneNumberMapper.java
    • OrganisationDtoTest.java
    • ServiceSupplierDtoTest.java
    • TelephoneNumberDtoTest.java

    Deleted (3 files):

    • OrganisationRole.java
    • PhoneNumberEntity.java
    • PhoneNumberService.java

    Modified (25 files):

    • Entity files: 5 (TelephoneNumber, CustomerAccountEntity, CustomerEntity, Organisation, ServiceSupplierEntity)
    • DTO files: 5 (CustomerAccountDto, CustomerDto, ServiceLocationDto, ElectricPowerQualitySummaryDto, CustomerAtomEntryDto)
    • Mapper files: 4 (CustomerMapper, OrganisationMapper, ServiceLocationMapper, ElectricPowerQualitySummaryMapper)
    • Service files: 1 (DtoExportServiceImpl)
    • Migration files: 1 (V3__Create_additiional_Base_Tables.sql)
    • Test files: 4 (CustomerAccountDtoTest, CustomerDtoMarshallingTest, CustomerDtoTest, ServiceLocationDtoTest, ServiceLocationRepositoryTest)
    • Enum files: 1 (SupplierKind)

    ESPI 4.0 Compliance Status

    ✅ ServiceSupplier - Fully compliant with customer.xsd lines 1159-1186
    ✅ Organisation - Fully compliant with customer.xsd lines 1089-1125
    ✅ TelephoneNumber - Fully compliant with customer.xsd lines 1428-1478
    ✅ SupplierKind - Enum matches XSD lines 2231-2271 (correct sequence for ordinal serialization)
    ✅ ServiceSupplierDto - Strictly XSD-compliant (no Atom metadata)
    ✅ OrganisationDto - Strictly XSD-compliant with embedded phone1/phone2
    ✅ TelephoneNumberDto - All 8 XSD fields implemented
    ✅ AtomEntryDto Pattern - Properly separates Atom protocol from ESPI resources
    ✅ MapStruct Mappings - Correct ignore patterns, no BaseMapperUtils conflicts
    ✅ Embedded Pattern - XSD-compliant, type-safe, performant

    Next Steps

    Phase 21 is complete. This issue remains open for:

    1. Additional customer domain entity implementations (CustomerAgreement with pending service)
    2. Future schema compliance reviews
    3. Tracking remaining multi-phase ESPI 4.0 work

    Upcoming Phases:

    • Phase 22+: Additional customer domain entities as needed
    • Continued XSD compliance verification across all domains

    🤖 Generated with Claude Code

  17. dfcoffin
    added 6 commits that reference this issue on Jan 31, 2026

    feat: ESPI 4.0 Schema Compliance - Phase 21: ServiceSupplier Complete…

    410f4b5

    docs: Update Phase 17 plan to include EndDevice and Meter in Phase A0

    d3b82c2

    feat: Issue #97 - Standardize relatedLinks Infrastructure with Compos…

    fe77497

    feat: Phase 17 (Part 1) - ProgramDateIdMapping Core Structure

    9639490

    feat: ESPI 4.0 Schema Compliance - Phase 17: ProgramDateIdMappings C-…

    6c5f2e8

    feat: ESPI 4.0 Schema Compliance - Phase 17: ProgramDateIdMappings Co…

    18a8e8b

  18. dfcoffin commented on Feb 2, 2026

    @dfcoffin
    dfcoffin
    on Feb 2, 2026
    ContributorAuthor

    Phase 17: ProgramDateIdMappings - ✅ COMPLETE

    PR: #100 (Merged)
    Merge Commit: 18a8e8bb
    Status: All tasks completed and merged into main

    Completed Tasks:

    • ✅ Entity Updates: ProgramDateIdMappingsEntity with embedded ProgramDateIdMapping
    • ✅ DTO Updates: ProgramDateIdMappingsDto and ProgramDateIdMappingDto following AtomEntryDto pattern
    • ✅ MapStruct Mapper Updates: ProgramDateIdMappingsMapper and ProgramDateIdMappingMapper
    • ✅ Repository Updates: ProgramDateIdMappingsRepository with ID-based queries only
    • ✅ Service Updates: ProgramDateIdMappingsService and ProgramDateIdMappingsServiceImpl
    • ✅ Flyway Migration Updates: V3 migration updated for ProgramDateIdMappings table
    • ✅ Testing: Comprehensive test suite created:
      • Unit tests: ProgramDateIdMappingsRepositoryTest (11 tests)
      • MySQL integration tests: ProgramDateIdMappingsMySQLIntegrationTest
      • PostgreSQL integration tests: ProgramDateIdMappingsPostgreSQLIntegrationTest
      • All tests passing (787/787)
    • ✅ Commit, Push, PR: Completed and merged

    Test Results:

    Tests run: 787, Failures: 0, Errors: 0, Skipped: 0
BUILD SUCCESS

    Implementation Notes:

    • ProgramDateKind enum created with 4 values per customer.xsd
    • ProgramDateIdMapping embedded object for complex type mapping
    • All field types and sequences verified against customer.xsd lines 1997-2030
    • UUID Version 5 compliance maintained
    • AssertJ chained assertions used throughout tests

    Next Phase: Phase 18 (CustomerAccount) - Ready to begin

  19. dfcoffin
    added a commit that references this issue on Feb 2, 2026

    feat: ESPI 4.0 Schema Compliance - Phase 19: Statement Complete Imple…

    dd72192
  20. dfcoffin
    closed this as completedin #102on Feb 3, 2026

  21. dfcoffin commented on Feb 3, 2026

    @dfcoffin
    dfcoffin
    on Feb 3, 2026
    ContributorAuthor

    ✅ Phase 19: Statement - COMPLETE

    PR #102 Merged: #102

    Implementation Summary

    Schema Compliance

    • StatementRefEntity converted to @embeddable (extends Object, not IdentifiedObject)
    • statement_refs as @ElementCollection table (no id column)
    • StatementDto/StatementRefDto XSD-compliant (customer.xsd lines 285-307, 373-393)

    Entity Enhancements

    • Added 4 @manytoone relationships (Customer, CustomerAccount, CustomerAgreement, UsageSummary)
    • Removed non-schema fields (statement_date, billing_period_start, billing_period_duration)
    • @AssociationOverride for statement_related_links (bidirectional Atom protocol)

    Repository & Service

    • Removed 7 non-ID query methods
    • Added 4 ID-based relationship queries
    • Simplified to CRUD + ID operations only

    Testing

    • 781 tests passing (added 15 new tests)
    • Integration tests for MySQL + PostgreSQL
    • Cross-platform compatible (Windows/Mac/Linux)

    All Phases Complete! 🎉

    This completes the full ESPI 4.0 Schema Compliance effort for customer.xsd entities.

    🤖 Generated with Claude Code

  22. springframeworkguru
    added a commit that references this issue on Feb 15, 2026

    feat: ESPI 4.0 Schema Compliance - Phase 19: Statement Complete Imple…

    4441d92
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions