Search Space API

Define hyperparameter distributions for Ray Tune trials.

A search space is a dict mapping each hyperparameter name to a distribution. Tune samples one value per trial.

Continuous distributions

tune.uniform(lower, upper)
tune.quniform(lower, upper, q)        # quantized to multiples of q
tune.loguniform(1e-5, 1e-1)           # log-uniform
tune.qloguniform(lower, upper, q)
tune.randn(mean, sd)
tune.qrandn(mean, sd, q)

Integer distributions

tune.randint(lower, upper)
tune.qrandint(lower, upper, q)
tune.lograndint(lower, upper)
tune.qlograndint(lower, upper, q)

Categorical

tune.choice(["adam", "sgd", "rmsprop"])
tune.grid_search([0.0, 0.1, 0.2])     # exhaustive grid

Conditional spaces

Use tune.sample_from for parameters that depend on other parameters.

search_space = {
    "model": tune.choice(["small", "large"]),
    "lr": tune.sample_from(
        lambda spec: 1e-3 if spec.config["model"] == "small" else 1e-4
    ),
}

Mixed grid and random

search_space = {
    "optimizer": tune.grid_search(["adam", "sgd"]),
    "lr": tune.loguniform(1e-5, 1e-1),
}

# num_samples=10 → 20 trials total (10 random samples × 2 grid points)

Reproducibility

Set tune.TuneConfig(seed=...) to make sampling deterministic.

Next steps

Search algorithms

Pair a search space with the right search algorithm.

Schedulers

Stop unpromising trials early.