Spawning Subprocesses

Run subprocesses inside Ray tasks and actors, and ensure they’re cleaned up correctly.

Workers can spawn subprocesses just like any Python program. Ray adds two requirements: the subprocess must inherit the worker’s resources, and Ray must be able to reap it when the worker exits.

Spawn a subprocess

Standard subprocess calls work without modification.

import subprocess
import ray

@ray.remote
def run_shell(cmd: str) -> str:
    return subprocess.check_output(cmd, shell=True, text=True)

Long-running subprocesses

For subprocesses that outlive a single method call (a server, a worker pool), wrap them in an actor and clean up in __del__ or an explicit shutdown method.

import signal
import subprocess

@ray.remote
class Server:
    def __init__(self):
        self.proc = subprocess.Popen(["my-server", "--port", "8080"])

    def shutdown(self):
        self.proc.send_signal(signal.SIGTERM)
        self.proc.wait(timeout=10)

    def __del__(self):
        try:
            self.shutdown()
        except Exception:
            pass

Resource isolation

Subprocesses spawned by a Ray worker share its resource budget. If your task requests num_cpus=4 and spawns a subprocess that uses 8 CPUs, Ray won’t enforce the limit — the subprocess can over-subscribe the host.

For strict isolation, use container runtime environments or cgroup-based scheduling on Kubernetes.

Killing orphaned processes

Ray sends SIGTERM to the worker when it shuts down or restarts. The worker is responsible for terminating its subprocesses; otherwise they may linger as zombies.

Register an atexit hook or use a try/finally block to clean up subprocesses when a task or actor exits.

Next steps

Fault tolerance

Recovery semantics when workers crash.

Configure

System-level configuration knobs.