worker.collector.sample_serial_collector

sample_serial_collector

Please Reference ding/worker/collector/sample_serial_collector.py for usage

SampleSerialCollector

class ding.worker.collector.sample_serial_collector.SampleSerialCollector(cfg: EasyDict, env: BaseEnvManager = None, policy: namedtuple = None, tb_logger: SummaryWriter = None, exp_name: str | None = 'default_experiment', instance_name: str | None = 'collector')

Overview:

Sample collector(n_sample), a sample is one training sample for updating model, it is usually like <s, a, s’, r, d>(one transition) while is a trajectory with many transitions, which is often used in RNN-model.

Interfaces:

__init__, reset, reset_env, reset_policy, collect, close

Property:

envstep

__del__() → None

Overview:

Execute the close command and close the collector. __del__ is automatically called to destroy the collector instance when the collector finishes its work

__init__(cfg: EasyDict, env: BaseEnvManager = None, policy: namedtuple = None, tb_logger: SummaryWriter = None, exp_name: str | None = 'default_experiment', instance_name: str | None = 'collector') → None

Overview:

Initialization method.

Arguments:

• cfg (EasyDict): Config dict

• env (BaseEnvManager): the subclass of vectorized env_manager(BaseEnvManager)

• policy (namedtuple): the api namedtuple of collect_mode policy

• tb_logger (SummaryWriter): tensorboard handle

_output_log(train_iter: int) → None

Overview:

Print the output log information. You can refer to the docs of Best Practice to understand the training generated logs and tensorboards.

Arguments:

• train_iter (int): the number of training iteration.

_reset_stat(env_id: int) → None

Overview:

Reset the collector’s state. Including reset the traj_buffer, obs_pool, policy_output_pool and env_info. Reset these states according to env_id. You can refer to base_serial_collector to get more messages.

Arguments:

• env_id (int): the id where we need to reset the collector’s state

close() → None

Overview:

Close the collector. If end_flag is False, close the environment, flush the tb_logger and close the tb_logger.

collect(n_sample: int | None = None, train_iter: int = 0, drop_extra: bool = True, random_collect: bool = False, record_random_collect: bool = True, policy_kwargs: dict | None = None, level_seeds: List | None = None) → List[Any]

Overview:

Collect n_sample data with policy_kwargs, which is already trained train_iter iterations.

Arguments:

• n_sample (int): The number of collecting data sample.

• train_iter (int): The number of training iteration when calling collect method.

• drop_extra (bool): Whether to drop extra return_data more than n_sample.

• record_random_collect (bool) :Whether to output logs of random collect.

• policy_kwargs (dict): The keyword args for policy forward.

• level_seeds (dict): Used in PLR, represents the seed of the environment that generate the data

Returns:

• return_data (List): A list containing training samples.

property envstep: int

Overview:

Print the total envstep count.

Return:

• envstep (int): The total envstep count.

reset(_policy: namedtuple | None = None, _env: BaseEnvManager | None = None) → None

Overview:

Reset the environment and policy. If _env is None, reset the old environment. If _env is not None, replace the old environment in the collector with the new passed in environment and launch. If _policy is None, reset the old policy. If _policy is not None, replace the old policy in the collector with the new passed in policy.

Arguments:

• policy (Optional[namedtuple]): the api namedtuple of collect_mode policy

• env (Optional[BaseEnvManager]): instance of the subclass of vectorized env_manager(BaseEnvManager)

reset_env(_env: BaseEnvManager | None = None) → None

Overview:

Reset the environment. If _env is None, reset the old environment. If _env is not None, replace the old environment in the collector with the new passed in environment and launch.

Arguments:

• env (Optional[BaseEnvManager]): instance of the subclass of vectorized env_manager(BaseEnvManager)

reset_policy(_policy: namedtuple | None = None) → None

Overview:

Reset the policy. If _policy is None, reset the old policy. If _policy is not None, replace the old policy in the collector with the new passed in policy.

Arguments:

• policy (Optional[namedtuple]): the api namedtuple of collect_mode policy