data

CarlaBenchmarkCollector

class core.data.carla_benchmark_collector.CarlaBenchmarkCollector(cfg: Dict, env: BaseEnvManager, policy: Any)

Collector to collect Carla benchmark data with envs. It uses several environments in EnvManager to collect data. It will automatically get params to reset environments. For every suite provided by user, collector will find all available reset params from benchmark files and store them in a list. When collecting data, the collector will collect each suite in average and store the index of each suite, to make sure each reset param is collected once and only once. The collected data are stored in a trajectory list, with observations, actions and reset param of the episode.

Note:

Env manager must run WITHOUT auto reset.

Arguments:

• cfg (Dict): Config dict.

• env (BaseEnvManager): Env manager used to collect data.

• policy (Any): Policy used to collect data. Must have forward method.

Interfaces:

reset, collect, close

Properties:

• env (BaseEnvManager): Env manager with several environments used to sample data.

• policy (Any): Policy instance to interact with envs.

close() → None

Close collector and env manager if not closed.

collect(n_episode: int, policy_kwargs: Dict | None = None) → List

Collect data from policy and env manager. It will collect each benchmark suite in average according to ‘n_episode’.

Arguments:

• n_episode (int): Num of episodes to collect.

• policy_kwargs (Dict, optional): Additional arguments in policy forward. Defaults to None.

Returns:

List: List of collected data. Each elem stores an episode trajectory.

reset(suite: List | str | None = None) → None

Reset collector and policies. Clear data cache storing data trajectories. If ‘suite’ is provided in arguments, the collector will change its collected suites and generate reset params again.

Arguments:

• suite (Union[List, str], optional): Collected suites after reset. Defaults to None.

BenchmarkDatasetSaver

class core.data.benchmark_dataset_saver.BenchmarkDatasetSaver(save_dir: str, obs_cfg: Dict, post_process_fn: Callable | None = None, lmdb_obs: List | None = ['lidar', 'birdview'])

Benchmark dataset saver in DI-drive. It can save dataset in standard benchmark dataset form defined in DI-drive. User can pass a post-process function to specialize ‘sensor_data’ and ‘others’ saved in dataset.

Arguments:

• save_dir (str): Dataset folder path.

• obs_cfg (Dict): Observation config dict in simulator.

• post_process_fn (Callable, optional): Post-process function defined by user. Defaults to None.

• lmdb_obs (List, optional): Observation types that saved as lmdb rather than image, default to [‘lidar’, ‘bev’]

Interfaces:

make_dataset_path, save_episodes_data, make_index

make_dataset_path(dataset_metainfo: Dict = {}) → None

Make dataset folder and write dataset meta infomation into a json file.

Arguments:

• dataset_metainfo (Dict): the metainfo of datasets

make_index(command_index: int = 11) → None

Make an index txt file to save all the command of each frame in dataset.

Arguments:

• command_index (int, optional): The index of command in ‘measurements.lmdb’. Defaults to 11.

save_episodes_data(episodes_data: List, start_episode: int = 0) → None

Save data from several episodes sampled from collector, with ‘env_param’ and ‘data’ key saved in each episode.

Arguments:

• episode_count (int): Start count of episode to save.

• episodes_data (List): Saved data of episodes.