lightrft.datasets.srm_dataset

class lightrft.datasets.srm_dataset.RankDatasetAL(*args: Any, **kwargs: Any)

Bases: Dataset

Preference ranking dataset used for audio-language scalar reward model (RM) training.

RMRankDatasetAL dataset supports multiple audio-language data sources through pluggable Data Handlers and support training reward model for text-to-audio task.

Parameters:

• dataset_paths (List[str]) – List of dataset file paths or directories, in the format source:path where the handler is determined by the source keyword such as audio-alpaca or omnirewardbench-t2a.

• processor (transformers.AutoProcessor) – Multimodal processor used for tokenization and audio processing.

• tokenizer (transformers.AutoTokenizer) – Tokenizer used for text tokenization.

• strategy (Any) – Optional data loading strategy.

• max_length (int) – Maximum sequence length for tokenization/truncation.

• config (Dict[str, Any]) – Additional configuration options. Supported keys include task_instruction for the evaluation instruction and system_prompt_template for a template containing a {prompt} placeholder.

Example:

dataset = RankDatasetAL([
    'audio-alpaca:/path/to/file.parquet'
], processor=proc, tokenizer=tok, max_length=4096)

collate_fn(batch: List[Tuple]) → Tuple | None

Collate a batch of items into a single batch for model processing.

Parameters:

batch (List[Tuple]) – A list of items returned by __getitem__

Returns:

A tuple containing batched inputs for both samples in the pair and extras.

Return type:

Optional[Tuple]

Example:

batch = dataset.collate_fn([dataset[i] for i in range(4)])

class lightrft.datasets.srm_dataset.RankDatasetVL(*args: Any, **kwargs: Any)

Bases: Dataset

Preference ranking dataset used for vision-language scalar reward model (RM) training.

RMRankDatasetVL dataset supports multiple data sources through pluggable Data Handlers and covers both understanding tasks (image-to-text, video-to-text) and generation tasks (text-to-image, text-to-video).

Each example contains two inputs to be compared. Labels use “A”, “B”, and “C” to indicate which input is better or if they tie. For example, label “A” means input0 is preferred over input1; “C” means a tie.

Parameters:

• dataset_paths (List[str]) – List of dataset file paths or directories, in the format source:path where the handler is determined by the source keyword such as hpdv3, rapidata, or omnirewardbench.

• processor (transformers.AutoProcessor) – Multimodal processor used for tokenization and visual processing.

• tokenizer (transformers.AutoTokenizer) – Tokenizer used for text tokenization.

• strategy (Any) – Optional data loading strategy.

• max_length (int) – Maximum sequence length for tokenization/truncation.

• config (Dict[str, Any]) – Additional configuration options. Supported keys include task_instruction for the evaluation instruction and system_prompt_template for a template containing a {prompt} placeholder.

Example:

dataset = RankDatasetVL([
    'hpdv3:/data/hpdv3/train.json'
], processor=proc, tokenizer=tok, max_length=4096)

collate_fn(batch: List[Tuple]) → Tuple | None

Collate a batch of items into a single batch for model processing.

Parameters:

batch (List[Tuple]) – A list of items returned by __getitem__

Returns:

A tuple containing batched inputs for both samples in the pair and extras.

Return type:

Optional[Tuple]

Example:

batch = dataset.collate_fn([dataset[i] for i in range(4)])