Hi, I’ve been able to train a multi-label Bert classifier using a custom Dataset object and the Trainer API from Transformers. The Dataset contains two columns: text and label. After tokenizing, I have all the needed columns for training. For multi-label classification I also set model.config.prob…

@zbeloki Defining the labels as one-hot worked for me, the other thing I did was cast it as a Sequence of ClassLabel! Make sure your one-hot encoded labels is float datatype. If you are using pandas dataframes, here’s a short snippet! label_enum = {k:j for j, k in enumerate(df['label_categories'…

Thanks @dhruvmetha . Defining labels as “one”-hot vectors ([1, 1, 0, 0, 1]) also works for me, but I’d like to avoid it and define them as a Sequence(feature=ClassLabel(…)). I can’t manage to train the model that way. When I train a basic single-label multi-class model with labels as ClassLabel it …

[image] Fine-Tune for MultiClass or MultiLabel-MultiClass Models sylvain beat me to it slight_smile i was curious how the problem_type really worked so cooked up a quick 'n dirty notebook which shows how the type conversion can be made: transformers_multilabel-text-classi…

That notebook was helpful! It does basically the same as I do, encoding the labels as multi-hot vectors. So I guess that is the way to go. Thank you!

Hello, do you know any example notebooks to train using custom dataset for multi-label classification?

[image] zbeloki: encoding the labels as multi-hot vectors Hi @zbeloki , I read your posts in this thread with attention as I have the same issue: How to use a Dataset with input text as string and labels as Sequence(feature=ClassLabel(…)) like GoEmotions-simplified to train a multilabel cla…

HI @pierreguillou did you find a way to avoid converting the label vectors to one-hot encoded vectors?

No. I did not find another solution :frowning:

Dataset label format for multi-label text classification

🤗Datasets

trasformernewbe February 9, 2023, 6:35pm 10

What about the solution provided here: Multi-class Using Dataset - #2 by lhoestq

Topic		Replies	Views
Dataset for multilabel classification 🤗Transformers	1	313	January 20, 2025
Multi-class Using Dataset 🤗Datasets	5	3806	February 10, 2023
Creating a Sequence of ClassLabel for multi-label and multi-class problems 🤗Datasets	5	882	March 26, 2024
How to build a multi-label & multi-class dataset correctly? Beginners	4	961	April 18, 2025
Data format for BertForSequenceClassification with num_labels > 2 Beginners	5	4999	August 2, 2021

Dataset label format for multi-label text classification

Related topics