Nested dictionary/list objects in context are shared across context splits

Summary

A shallow copy is performed when a context needs to be split into several parts. This works fine if the context is just storing primitive values like integers and strings. This does not work if there are nested structures within the context.

Implementation

https://www.delftstack.com/howto/python/python-copy-dictionary/#deep-copy-with-copy-module-in-python

In theory this sounds like a problem. Pending a test that can prove this bug exists in the current implementation of the pipeline logic.

Proposed test for how to verify this bug:

Create a standalone pipeline.
Create a step that outputs a list of two DTO objects with a sample field. Set the output mode to split.
Create a step that sets the sample field of each object to the current value of a counter that measures how many times that step has run.
Create a verification step that confirms that the values of the sample fields are 1 and 2.

#6 Fixed objects being shared across contexts

#6 Updated CHANGELOG.md

#6 Linter fixes

Navigation Menu

Repository navigation

Nested dictionary/list objects in context are shared across context splits #6

Summary

Implementation

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Nested dictionary/list objects in context are shared across context splits #6

Description

Summary

Implementation

Activity

ARMmaster17 commented on Mar 27, 2022

ARMmaster17 commented on Apr 6, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions