Skip to content

5821 Add interface for bundle workflows - #5822

Merged
Nic-Ma merged 65 commits into
Project-MONAI:devfrom
Nic-Ma:5821-bundle-interface
Mar 15, 2023
Merged

Nic-Ma merged 65 commits into
Project-MONAI:devfrom
Nic-Ma:5821-bundle-interface

Conversation

@Nic-Ma

@Nic-Ma Nic-Ma commented Jan 6, 2023 •

Copy link
Copy Markdown
Contributor

Part of #5821 .

Description

The main idea of this PR is to solve 3 pain points of the current bundle training or inference workflow:

  1. Training or inference is defined in a "rigid" training or evaluation list, so 3rd party package can't flexibly initialize and run the workflows separately, see: Define initialize, run, finalize sections in bundle config model-zoo#294. This PR splits it into initialize, run and finalize, and still compatible with current existing bundles.
  2. 3rd party packages need to access the bundle config directly with specifed keys based on ConfigParser, but some developers of bundles don't want to write JSON or YAML configs. This PR implemented the interface to support both config-based and python-based workflows.
  3. MONAI Label, MONAI deploy, MONAI-FL require many fixed properties of a training or inference workflow, but we don't have definition for them in the code, just described in the developer guide doc of model-zoo. This PR implemented the TrainProperties and inferenceProperties interfaces and check tool.

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

Nic-Ma and others added 29 commits February 1, 2021 19:15
Signed-off-by: Nic Ma <nma@nvidia.com>
Merge branch 'Project-MONAI:main' into main
Signed-off-by: Nic Ma <nma@nvidia.com>
Signed-off-by: Nic Ma <nma@nvidia.com>
Signed-off-by: Nic Ma <nma@nvidia.com>
Signed-off-by: Nic Ma <nma@nvidia.com>
Signed-off-by: Nic Ma <nma@nvidia.com>
Signed-off-by: Nic Ma <nma@nvidia.com>
Signed-off-by: Nic Ma <nma@nvidia.com>
Signed-off-by: Nic Ma <nma@nvidia.com>
@wyli wyli added this to the Bundle class [P1 v1.2] milestone Feb 6, 2023
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Nic-Ma marked this pull request as ready for review March 10, 2023 15:37
@Nic-Ma Nic-Ma changed the title [WIP] 5821 Add interface for bundle workflows 5821 Add interface for bundle workflows Mar 10, 2023
@Nic-Ma

Nic-Ma commented Mar 10, 2023

Copy link
Copy Markdown
Contributor Author

Hi @ericspod @wyli ,

I added all the missing pieces, could you please help review it again?

Thanks in advance.

@Nic-Ma

Nic-Ma commented Mar 10, 2023

Copy link
Copy Markdown
Contributor Author

/black

@Nic-Ma

Nic-Ma commented Mar 13, 2023

Copy link
Copy Markdown
Contributor Author

/black

@Nic-Ma

Nic-Ma commented Mar 13, 2023

Copy link
Copy Markdown
Contributor Author

/build

Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma

Nic-Ma commented Mar 13, 2023

Copy link
Copy Markdown
Contributor Author

/build

@ericspod ericspod left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm good with this now, thanks!

@wyli wyli left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, I have some minor comments

Comment thread monai/bundle/scripts.py
Comment thread monai/bundle/workflows.py Outdated
Comment thread monai/bundle/workflows.py
Comment thread monai/bundle/workflows.py Outdated
Comment thread monai/bundle/workflows.py Outdated
Comment thread monai/bundle/workflows.py Outdated
Comment thread monai/utils/enums.py Outdated
Comment thread monai/bundle/properties.py Outdated
Comment thread monai/bundle/workflows.py Outdated
Nic-Ma added 5 commits March 15, 2023 14:11
Signed-off-by: Nic Ma <nma@nvidia.com>
Signed-off-by: Nic Ma <nma@nvidia.com>
Signed-off-by: Nic Ma <nma@nvidia.com>
Signed-off-by: Nic Ma <nma@nvidia.com>
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma

Nic-Ma commented Mar 15, 2023

Copy link
Copy Markdown
Contributor Author

/black

@Nic-Ma

Nic-Ma commented Mar 15, 2023

Copy link
Copy Markdown
Contributor Author

/build

Comment thread monai/bundle/workflows.py
Nic-Ma added 2 commits March 15, 2023 17:29
Signed-off-by: Nic Ma <nma@nvidia.com>
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma

Nic-Ma commented Mar 15, 2023

Copy link
Copy Markdown
Contributor Author

/black

@Nic-Ma

Nic-Ma commented Mar 15, 2023

Copy link
Copy Markdown
Contributor Author

/build

@Nic-Ma
Nic-Ma enabled auto-merge (squash) March 15, 2023 09:39
@Nic-Ma
Nic-Ma merged commit c696773 into Project-MONAI:dev Mar 15, 2023
@Nic-Ma Nic-Ma self-assigned this Mar 20, 2023
@yiheng-wang-nv

Copy link
Copy Markdown
Contributor

spleen_ct_segmentation in model zoo has updated to use this new interface (https://github.com/Project-MONAI/model-zoo/tree/dev/models/spleen_ct_segmentation)
cc @holgerroth

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants