lightrft.datasets.rapidata

class lightrft.datasets.rapidata.RapidataI2VHandler

Bases: RapidataT2VHandler

Data Handler for Rapidata image-to-video human preferences dataset.

Support datasets:

• Rapidata/image-2-video-human-preferences-seedance-1-pro

Dataset Repo: https://huggingface.co/Rapidata/datasets

get_media_info(item: Dict[str, Any]) → Dict[str, Dict[str, str]]

Extract media info (paths) for the two videos and bytes for initial image.

Parameters:

item (Dict[str, Any]) – A data item from load_data

Returns:

Dict containing local paths for videos and bytes for ‘init_image’

Return type:

Dict[str, Dict[str, Any]]

Example:

info = handler.get_media_info(item)

parse_item(item: Dict[str, Any], media_content: Dict[str, Any], config: Dict[str, Any]) → Tuple[List[Dict], List[Dict], Dict]

Parse a single Rapidata-I2V data item into message pairs for ranking.

Parameters:

• item (Dict[str, Any]) – The raw data item

• media_content (Dict[str, Any]) – Loaded visual content (videos and images)

• config (Dict[str, Any]) – Configuration for task instructions, max_pixels, and fps

Returns:

A tuple of (messages0, messages1, metadata)

Return type:

Tuple[List[Dict], List[Dict], Dict]

Example:

msg0, msg1, other = handler.parse_item(item, media_content, config)

task_type = 'image-to-video'

class lightrft.datasets.rapidata.RapidataI2VPairHandler

Bases: RapidataI2VHandler

Data Handler for Rapidata image-to-video human preferences dataset in pairwise format.

parse_item(item: Dict[str, Any], media_content: Dict[str, Any], config: Dict[str, Any]) → Tuple[List[Dict], Dict]

Parse an image-to-video data item into pairwise messages and metadata.

Parameters:

• item (Dict[str, Any]) – The raw data item

• media_content (Dict[str, Any]) – Loaded visual content

• config (Dict[str, Any]) – Configuration for task instructions, max_pixels, and fps

Returns:

A tuple of (messages, metadata)

Return type:

Tuple[List[Dict], Dict]

Example:

messages, other = handler.parse_item(item, media_content, config)

task_type = 'image-to-video'

class lightrft.datasets.rapidata.RapidataT2VHandler

Bases: BaseDataHandler

Data Handler for Rapidata text-to-video human preferences dataset.

Support datasets:

• Rapidata/text-2-video-human-preferences-pika2.2

• Rapidata/text-2-image-human-preferences-veo3

• Rapidata/text-2-video-human-preferences-wan2.1

This dataset contains pairs of videos (video1, video2) generated from a prompt. It includes weighted scores for Preference, Coherence, and Alignment.

• ‘A’ means video1 (messages0) is preferred.

• ‘B’ means video2 (messages1) is preferred.

• ‘C’ means they are equal or tied.

Dataset Repo: https://huggingface.co/Rapidata/datasets

get_media_info(item: Dict[str, Any]) → Dict[str, Dict[str, str]]

Extract path info for the two videos.

load_data(path: str) → List[Dict[str, Any]]

Loads data from parquet file.

Parameters:

path (str) – Path to the parquet file

Returns:

List of samples with ‘data_root’ attached

Return type:

List[Dict[str, Any]]

Example:

handler = RapidataT2VHandler()
data = handler.load_data("path/to/Rapidata/data.parquet")

parse_item(item: Dict[str, Any], media_content: Dict[str, Any], config: Dict[str, Any]) → Tuple[List[Dict], List[Dict], Dict]

Parse a single Rapidata-T2V data item into message pairs for ranking.

Parameters:

• item (Dict[str, Any]) – The raw data item

• media_content (Dict[str, Any]) – Loaded visual content

• config (Dict[str, Any]) – Configuration for task instructions, max_pixels, and fps

Returns:

A tuple of (messages0, messages1, metadata)

Return type:

Tuple[List[Dict], List[Dict], Dict]

Example:

msg0, msg1, other = handler.parse_item(item, media_content, config)

task_type = 'text-to-video'

class lightrft.datasets.rapidata.RapidataT2VPairHandler

Bases: RapidataT2VHandler

Data Handler for Rapidata text-to-video human preferences dataset in pairwise format.

parse_item(item: Dict[str, Any], media_content: Dict[str, Any], config: Dict[str, Any]) → Tuple[List[Dict], Dict]

Parse a text-to-video data item into pairwise messages and metadata.

Parameters:

• item (Dict[str, Any]) – The raw data item

• media_content (Dict[str, Any]) – Loaded visual content

• config (Dict[str, Any]) – Configuration for task instructions, max_pixels, and fps

Returns:

A tuple of (messages, metadata)

Return type:

Tuple[List[Dict], Dict]

Example:

messages, other = handler.parse_item(item, media_content, config)