Ray on VMs

Provision a Ray cluster on cloud VMs (AWS, GCP, Azure) or on-premises with the Ray cluster launcher.

The cluster launcher provisions VMs, installs Ray, and configures autoscaling on AWS, GCP, Azure, and on-premises.

Workflow

1. Write a cluster.yaml

   Declare the provider, head and worker node types, and any setup commands.

2. Bring up the cluster

   ray up cluster.yaml provisions VMs, installs Ray, and starts the head and initial workers.

3. Submit work

   ray submit cluster.yaml my_script.py, or use the Jobs API.

4. Tear it down

   ray down cluster.yaml terminates all VMs.

Pick your provider

AWS

EC2 with the AWS provider.

GCP

GCE with the GCP provider.

Azure

VMs with the Azure provider.

On-premises

Static node lists.

Sample config

cluster_name: my-ray
provider:
  type: aws
  region: us-west-2
auth:
  ssh_user: ubuntu
max_workers: 8
available_node_types:
  head:
    node_config:
      InstanceType: m5.xlarge
      ImageId: ami-...
    resources: {}
  worker:
    min_workers: 1
    max_workers: 8
    node_config:
      InstanceType: c5.4xlarge
    resources: {}
head_node_type: head
setup_commands:
  - pip install -U "ray[default]==2.43.0"

Next steps

Configuring autoscaling

Tune the autoscaler.

Best practices

Run large VM clusters reliably.