Trial Schedulers

Stop unpromising trials early with ASHA, HyperBand, PBT, and PB2.

A scheduler decides which trials to keep running, which to stop, and which to copy and perturb.

ASHA (recommended default)

Asynchronous Successive Halving Algorithm. The most common choice. Stops bad trials early without requiring synchronization.

from ray.tune.schedulers import ASHAScheduler

ASHAScheduler(
    time_attr="training_iteration",
    max_t=100,
    grace_period=1,
    reduction_factor=2,
)

Field	Effect
time_attr	What units max_t and grace_period measure (training_iteration, time_total_s, etc.).
max_t	Maximum units a trial can run.
grace_period	Minimum units before a trial can be stopped.
reduction_factor	At each rung, keep 1/factor of trials.

HyperBand

Synchronous version of ASHA. Useful when you want strict bracket boundaries.

from ray.tune.schedulers import HyperBandScheduler

HyperBandScheduler(time_attr="training_iteration", max_t=100)

Population-Based Training (PBT)

Treats hyperparameters as evolvable. Mid-training, low-performing trials clone the weights of high-performing ones and perturb their hyperparameters.

from ray.tune.schedulers import PopulationBasedTraining

PopulationBasedTraining(
    time_attr="training_iteration",
    metric="loss",
    mode="min",
    perturbation_interval=10,
    hyperparam_mutations={
        "lr": tune.loguniform(1e-5, 1e-1),
        "momentum": [0.85, 0.9, 0.95],
    },
)

PBT requires that your training function support resuming from a checkpoint.

PB2

Bayesian variant of PBT. Uses a Gaussian process to choose perturbations rather than mutating randomly.

from ray.tune.schedulers.pb2 import PB2

PB2(time_attr="training_iteration", perturbation_interval=10, hyperparam_bounds={"lr": [1e-5, 1e-1]})

FIFO (default)

No early stopping. Every trial runs to completion.

Combining schedulers and search algorithms

Scheduler	Compatible search algorithms
ASHA / HyperBand	Random, Optuna, Ax, BayesOpt, HyperOpt, Nevergrad
HyperBandForBOHB	TuneBOHB only
PBT / PB2	Use with random search
FIFO	Anything

Next steps

Stoppers

Stop the entire experiment based on a condition.

Trial checkpoints

Save and restore trial state for PBT.