Hello, I am running an example summarization training task taken from here (official HuggingFace example) on a multi-GPU machine, using the following versions: torch==1.11.0+cu113 and transformers==4.20.1. The only difference is that instead of using google/mt5-small as model I am using facebook/b…

I have the same issue, do you resolve it?

Not yet unfortunately :frowning: Code seems to run fine but the warnings are still there and I’m not sure what should be done about them

Hello, I find that this problem may be caused by trainer. When I used script run_glue.py, I got the UserWarning. But when I used script run_glue_no_trainer.py, everything is ok.

Thanks for pointing this out,. Unfortunately I am not using their training script but a custom one. What I would like to understand is why there is this issue and what underlining code part is causing it. Knowing that run_glue.py raises the warning but run_glue_no_trainer.py does not, may be helpfu…

HuggingFace summarization training example notebook raises two warnings when run on multi-GPUs

🤗Transformers

brando August 17, 2022, 3:38pm 6

did this help you? Using Transformers with DistributedDataParallel — any examples?

Topic		Replies	Views
Not able to scale Trainer code to single node multi GPU 🤗Transformers	0	1159	September 14, 2023
RuntimeError: arguments are located on different GPUs 🤗Transformers	2	1895	October 24, 2020
How to run an end to end example of distributed data parallel with hugging face's trainer api (ideally on a single node multiple gpus)? Intermediate	17	18545	September 6, 2023
Get UserWarning: "Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector" when run official example code run_mlm.py 🤗Transformers	1	2243	November 17, 2022
Trainer warning with the new version 🤗Transformers	2	6517	January 2, 2025

HuggingFace summarization training example notebook raises two warnings when run on multi-GPUs

Related topics