lightrft.strategy.strategy

Strategy factory module for lightrft.

This module provides functionality to create appropriate distributed training strategies based on configuration arguments. It supports three types of strategies:

1. DeepSpeed - For training with the DeepSpeed library

2. FSDP V2 - For training with PyTorch’s Fully Sharded Data Parallel (newer version)

The module automatically selects the appropriate strategy based on the provided arguments.

get_strategy

lightrft.strategy.strategy.get_strategy(args: Any) → FSDPV2Strategy | DeepspeedStrategy

Create and return the appropriate training strategy based on configuration arguments.

This function examines the provided arguments and instantiates either a DeepSpeed, or FSDP V2 strategy with the appropriate parameters.

Parameters:

args (Any (usually argparse.Namespace)) – Configuration arguments containing strategy selection flags and parameters

Returns:

An instantiated strategy object

Return type:

Union[FSDPV2Strategy, DeepspeedStrategy]

Example:

>>> parser = ArgumentParser()
>>> parser.add_argument("--fsdp", action="store_true")
>>> parser.add_argument("--seed", type=int, default=42)
>>> args = parser.parse_args()
>>> strategy = get_strategy(args)