Hi, I create a small dataset to mimic the format (oai) recommended by STFTrainer doc page: from datasets import load_dataset from trl import SFTTrainer from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig, TrainingArguments from peft import LoraConfig import torch, sys …

Did you get this solved? I am also getting the same problem.

Any update on this? 2000 sample is not enough? autotrain.trainers.common:wrapper:116 - Error occurred while packing the dataset. Make sure that your dataset has enough samples to at least yield one packed sequence.

I had the same problem. In my case, passing the ‘train’ subset of the dataset to the SFTTrainer’s train_dataset solved the issue.

I had the same problem on my private dataset. In my case, dropping NaN values solved my the issue. import pandas as pd from datasets import Dataset def create_conversation(sample): return { "messages": [ {"role": "system", "content": system_message.format(schema=sample["context"])}, …

[image] celsowm: packing=True, Just remove the packing=True argument from the SFTTrainer

Hi, where can I do this in Autotrain? I am using the ‘train’ subset anyway.

Error using SFTTrainer: Make sure that your dataset has enough samples to at least yield one packed sequence

Beginners

sc4r3cr0w May 9, 2024, 12:57pm 7

Just remove the packing=True argument from the SFTTrainer

Model responses are random ignoring my dataset

Topic		Replies	Views
Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' 🤗Transformers	2	911	July 24, 2024
SFTTrainer work but without result Beginners	1	137	October 13, 2025
Fine tuning LLM for text classification -- error with SFTTrainer Intermediate	2	1459	June 3, 2025
Can only automatically infer lengths for datasets whose items are dictionaries with an 'input_ids' key 🤗Transformers	0	653	May 30, 2024
Packing issue, SFTTrainer 🤗Transformers	0	368	November 10, 2023

Error using SFTTrainer: Make sure that your dataset has enough samples to at least yield one packed sequence

Related topics