lightrft.datasets.sft_dataset_vl

class lightrft.datasets.sft_dataset_vl.SFTDatasetVL(*args: Any, **kwargs: Any)

Bases: Dataset

Dataset for SFT model

collate_fn(item_list)

Collate function to batch vision-language samples with padding.

Parameters:

item_list (list) – List of tuples (prompt_ids_len, input_id, attention_mask, pixel_value, image_grid_thw, info).

Returns:

Batched tensors (prompt_ids_lens, input_ids, attention_masks, pixel_values, image_grid_thws, infos).

Return type:

tuple

packing_collate_fn(item_list)

Collate function for packing multiple vision-language samples into a single sequence.

Parameters:

item_list (list) – List of tuples (prompt_ids_len, input_id, _, pixel_value, image_grid_thw, info).

Returns:

Packed tensors (packed_input_ids, packed_attention_masks, prompt_ids_lens, pixel_values, image_grid_thws, infos).

Return type:

tuple

process_data(data)

Process a single vision-language data sample for SFT training.

Parameters:

data (dict) – Raw data sample dictionary.

Returns:

Processed data with prompt, response, images, and metadata.

Return type:

dict

lightrft.datasets.sft_dataset_vl.preprocess_data(data, input_template=None, input_key='input', output_key=None, images_key='images', apply_chat_template=None)

Preprocess vision-language data sample into prompt, response, and images.

Parameters:

• data (dict) – Raw data sample dictionary.

• input_template (Optional[str]) – Optional template string to format the input.

• input_key (str) – Key for input field in data.

• output_key (Optional[str]) – Key for output field in data (None for pretrain mode).

• images_key (str) – Key for images field in data.

• apply_chat_template (Optional[Callable]) – Optional chat template function.

Returns:

Tuple of (prompt, response, images).

Return type:

Tuple[Optional[str], Optional[str], Any]