Repository navigation
Support Mixup and Cutmix transformation #2872
Description
HI, the more strong transformation "RandCoarseDropout" has been released in the latest version.
It's possible to support Mixup or Cutmix in MonAI?
{{ message }}
HI, the more strong transformation "RandCoarseDropout" has been released in the latest version.
It's possible to support Mixup or Cutmix in MonAI?
Activity
Nic-Ma commentedon Aug 31, 2021
Hi @JiYuanFeng ,
Thanks for your feedback.
Seems like a new transform feature request, I marked for feature discussion.
@ericspod @wyli Do you guys think it's a common approach in medical imaging augmentation?
Thanks in advance.
Thanks.
wyli commentedon Sep 1, 2021
for the mixup, I can find this paper as a reference https://openreview.net/pdf?id=rkBBChjiG. not sure if we have an example/renference for cutmix.
sandylaker commentedon Sep 1, 2021
@wyli For CutMix, you can refer to this paper. These two augmentations mix one image with another randomly selected one, which can be selected from the same batch or the whole dataset. Thus, the API of these two augmentations are slightly different from the other ones.
wyli commentedon Sep 2, 2021
Thanks, I think these are good augmentations to have. @Nic-Ma these could be in v0.8?
Nic-Ma commentedon Sep 2, 2021
Sounds good to me.
I can try to implement in v0.8.
Thanks.
Nic-Ma commentedon Sep 9, 2021
Hi @wyli ,
Currently, all the MONAI transforms work with a single image, so I think maybe this feature is not a transform, It should be a layer before the network to mix up a batch data?
Thanks.
ericspod commentedon Sep 10, 2021
My thoughts on mixup that I wanted to play with at some point was as a special sort of DataLoader that would combine images from a batch to make a smaller batch with some controls for how the mixing would be done. Alternatively if we wanted to do it as a transform we would have to first create a dataset of image pairs that are to be mixed (this wouldn't involve copying the original images though so should be lightweight), a normal transform sequence, and a final transform that does the mixing of the two images to produce one. This can get us around the issue of transform sequences being one-to-one for the most part, they operate on individual items in a batch rather than being able to operate on a whole batch.
masadcv commentedon Oct 2, 2021
@Nic-Ma Just to add to discussion here, there is an existing implementation of this for medical images, perhaps for paper @wyli linked (https://openreview.net/pdf?id=rkBBChjiG), in NiftNet at: https://github.com/NifTK/NiftyNet/blob/935bf4334cd00fa9f9d50f6a95ddcbfdde4031e0/niftynet/application/segmentation_application.py#L325
Nic-Ma commentedon Oct 2, 2021
Hi @masadcv ,
Thanks for your sharing.
Let me investigate it and get back ASAP.
Thanks.
JohnMasoner commentedon Jun 2, 2023
Hi @Nic-Ma ,
It seems that Mixup and Cutmix are still unsupported.
Is there any consideration to include this data transformation method?
Because I've been working on related content lately, if consider adding these transformations. I will submit pr.
Thanks.
Nic-Ma commentedon Jun 2, 2023
Hi @JohnMasoner ,
Thanks for your interest here.
If @ericspod and @wyli don't have any other comments, welcome to contribute a PR.
JohnMasoner commentedon Jun 2, 2023
Thanks for your reply,
I readed @ericspod comments, i will contribute a PR about CutOut( Maybe like 'RandCoarseDropout', also similar to Mixup and Cutmix)
ericspod commentedon Jun 2, 2023
Hi @JohnMasoner, thanks for looking into this. I'd definitely like to see some contributions along the lines of mixup/cutmix, something new you're working on would be great as well as reference implementations of what's been published already. I had suggested implementing it as a dataloader of some sort but it's possible to be instead a special collate function, or some other implementation entirely, so feel free to propose your own solution.
2872 implementation of mixup, cutmix and cutout (#7198)
2872 implementation of mixup, cutmix and cutout (Project-MONAI#7198)
2872 implementation of mixup, cutmix and cutout (Project-MONAI#7198)