This repository was archived by the owner on Aug 28, 2025. It is now read-only.
Repository navigation
This repository was archived by the owner on Aug 28, 2025. It is now read-only.
Finetune Transformers Models with PyTorch Lightning: documentation error? #139
Description
When calculating the total steps, shouldn't we use number of batches * epoch size ? In this case, it would be self.total_steps = (len(train_loader.dataset) // tb_size) * ab_size instead of self.total_steps = (len(train_loader.dataset) // tb_size) // ab_size.
Please fix me if anywhere is wrong.
Activity
rohitgr7 commentedon Feb 4, 2022
I guess it should be
batches * num_epochsbut why would it be* ab_size?yfeng24816 commentedon Feb 4, 2022
Is
ab_sizesomething likenum_epochs? It becomesself.trainer.max_epochswhenaccumulate_grad_batchesis 1.rohitgr7 commentedon Feb 4, 2022
okay. yes... I didn't see max_epochs there. should be something like
yfeng24816 commentedon Feb 5, 2022
So do you also think there is an error in the documentation too? I am not sure by my own.
Borda commentedon Jul 24, 2024
@stancld mind have a look, please? 🐿️
stancld commentedon Jul 25, 2024
@Borda Yes, I will have a look on Friday (tmrw). Can you assign the issue to me, please? :]
stancld commentedon Jul 26, 2024
It looks like no such example is present in tutorial anymore 🤔
Borda commentedon Jul 26, 2024
https://github.com/Lightning-AI/tutorials/tree/main/lightning_examples/text-transformers
stancld commentedon Jul 26, 2024
Oh sorry, I have forked
tutorialsrepo, but it'spytorchone :Dstancld commentedon Jul 27, 2024
@Borda I checked the notebook and it looks like the calculation of the total number of training steps is now the responsibility of a Lightning Trainer. The reported error is, therefore, no more relevant for this example, and I believe the issue can be closed/marked as done.