RayCluster Configuration

Configure pod templates, init containers, sidecars, and runtime environments for KubeRay.

RayCluster.spec accepts standard Kubernetes pod template overrides. Anything you’d put in a Pod spec (volumes, init containers, env vars, sidecars, security contexts) you can put here.

Resource requests and limits

template:
  spec:
    containers:
      - name: ray-worker
        image: rayproject/ray:2.43.0
        resources:
          requests: { cpu: "4", memory: "8Gi" }
          limits:   { cpu: "4", memory: "8Gi" }

Set requests == limits to avoid CPU throttling and OOM kills under contention.

Init containers

template:
  spec:
    initContainers:
      - name: download-model
        image: amazon/aws-cli
        command: ["aws", "s3", "cp", "s3://bucket/model.bin", "/models/"]
        volumeMounts:
          - { name: models, mountPath: /models }
    volumes:
      - name: models
        emptyDir: {}

Custom Ray start params

rayStartParams:
  num-cpus: "8"
  num-gpus: "1"
  resources: '"{\"high_memory\": 8}"'
  block: "true"

These map to ray start ... flags on the head and worker pods.

Runtime environments

For RayJob and RayService, set a per-job runtime env:

runtimeEnvYAML: |
  pip:
    - torch==2.1.0
  working_dir: "s3://bucket/code.zip"

Custom Docker images

Build your own image with extra dependencies baked in:

FROM rayproject/ray:2.43.0
RUN pip install transformers vllm

Reference it in your spec:

containers:
  - name: ray-worker
    image: ghcr.io/myorg/ray-app:2.43.0

Service mesh and sidecars

Add Istio, Linkerd, or other sidecars via standard pod annotations or sidecar containers. KubeRay’s services use headless DNS that’s compatible with most service meshes.

Next steps

Autoscaling

Configure cluster-level scaling.

Storage

Mount data and checkpoints.