# Ray > Ray is an open-source unified framework for scaling AI and Python applications. ## Docs - [API Reference Overview](https://ray-preview.mintlify.app/api-reference/introduction.md): Ray's HTTP and Python APIs across Ray Core, Ray Data, Ray Train, Ray Tune, Ray Serve, and Ray Job Submission. - [Get cluster version](https://ray-preview.mintlify.app/api-reference/ray-jobs/cluster/get-cluster-version.md): Return the Ray version and dashboard agent version. - [Get job details](https://ray-preview.mintlify.app/api-reference/ray-jobs/jobs/get-job-details.md) - [Get job logs](https://ray-preview.mintlify.app/api-reference/ray-jobs/jobs/get-job-logs.md): Return the captured stdout/stderr for the job entrypoint. - [List jobs](https://ray-preview.mintlify.app/api-reference/ray-jobs/jobs/list-jobs.md): Return all jobs submitted to the cluster. - [Stop a job](https://ray-preview.mintlify.app/api-reference/ray-jobs/jobs/stop-a-job.md): Request the cluster to stop a running job. The job transitions through STOPPING and ends in STOPPED. - [Stream job logs](https://ray-preview.mintlify.app/api-reference/ray-jobs/jobs/stream-job-logs.md): Open a WebSocket-like stream of new log lines as they're produced. - [Submit a job](https://ray-preview.mintlify.app/api-reference/ray-jobs/jobs/submit-a-job.md): Submit a new Ray job. The cluster runs the entrypoint command in the runtime environment provided. - [Apply Serve config](https://ray-preview.mintlify.app/api-reference/ray-serve/applications/apply-serve-config.md): Push a new Serve config. The controller performs a rolling update if the config differs from the active one. - [Delete all Serve applications](https://ray-preview.mintlify.app/api-reference/ray-serve/applications/delete-all-serve-applications.md): Tear down every running Serve application on the cluster. - [Get Serve status](https://ray-preview.mintlify.app/api-reference/ray-serve/applications/get-serve-status.md): Return the active Serve configuration plus per-application and per-deployment status. - [List active routes](https://ray-preview.mintlify.app/api-reference/ray-serve/health/list-active-routes.md): Return the route map of every healthy ingress deployment. Used as a readiness probe. - [Liveness probe](https://ray-preview.mintlify.app/api-reference/ray-serve/health/liveness-probe.md): Returns 200 OK when the proxy is alive. - [Cluster CLI](https://ray-preview.mintlify.app/cluster/cli.md): Lifecycle commands for Ray clusters: ray start, ray stop, ray up, ray status, and the Jobs API. - [Configure the Dashboard](https://ray-preview.mintlify.app/cluster/configure-manage-dashboard.md): Configure, secure, and access the Ray dashboard. - [Cluster FAQ](https://ray-preview.mintlify.app/cluster/faq.md): Common questions about running Ray on Kubernetes, cloud VMs, and on-premises hardware. - [Ray Clusters](https://ray-preview.mintlify.app/cluster/getting-started.md): Deploy Ray on Kubernetes, cloud VMs, or on-premises hardware to run distributed workloads at any scale. - [Ray Cluster Key Concepts](https://ray-preview.mintlify.app/cluster/key-concepts.md): Head node, worker node, GCS, raylet, dashboard, and the autoscaler — the building blocks of a Ray cluster. - [Autoscaling on Kubernetes](https://ray-preview.mintlify.app/cluster/kubernetes/autoscaling.md): Configure the in-tree Ray autoscaler to add and remove workers on KubeRay. - [RayCluster Configuration](https://ray-preview.mintlify.app/cluster/kubernetes/configuration.md): Configure pod templates, init containers, sidecars, and runtime environments for KubeRay. - [Get Started With KubeRay](https://ray-preview.mintlify.app/cluster/kubernetes/getting-started.md): Install the KubeRay operator and create your first RayCluster. - [GPU Workloads](https://ray-preview.mintlify.app/cluster/kubernetes/gpu.md): Run GPU workloads on KubeRay with the right node pools, drivers, and resource requests. - [Ray on Kubernetes](https://ray-preview.mintlify.app/cluster/kubernetes/index.md): Deploy Ray on Kubernetes with the KubeRay operator and the RayCluster, RayJob, and RayService CRDs. - [Observability on Kubernetes](https://ray-preview.mintlify.app/cluster/kubernetes/observability.md): Set up Prometheus, Grafana, and log aggregation for KubeRay. - [RayCluster Quickstart](https://ray-preview.mintlify.app/cluster/kubernetes/raycluster-quickstart.md): Create a long-running Ray cluster on Kubernetes with the RayCluster CRD. - [RayJob Quickstart](https://ray-preview.mintlify.app/cluster/kubernetes/rayjob-quickstart.md): Run a Ray job on Kubernetes with the RayJob CRD. - [RayService Quickstart](https://ray-preview.mintlify.app/cluster/kubernetes/rayservice-quickstart.md): Deploy a Ray Serve application on Kubernetes with the RayService CRD. - [Storage on Kubernetes](https://ray-preview.mintlify.app/cluster/kubernetes/storage.md): Attach volumes, mount object storage, and configure shared filesystems for KubeRay. - [TPU Workloads](https://ray-preview.mintlify.app/cluster/kubernetes/tpu.md): Run Ray on Google Cloud TPUs with KubeRay and GKE. - [Troubleshooting KubeRay](https://ray-preview.mintlify.app/cluster/kubernetes/troubleshooting.md): Diagnose common KubeRay issues: pods stuck pending, RayService stuck deploying, autoscaler not scaling. - [Kubernetes User Guides](https://ray-preview.mintlify.app/cluster/kubernetes/user-guides.md): Operational guides for running Ray on Kubernetes. - [Cluster Metrics](https://ray-preview.mintlify.app/cluster/metrics.md): Scrape Ray metrics with Prometheus and visualize with Grafana. - [Usage Statistics](https://ray-preview.mintlify.app/cluster/usage-stats.md): Ray's anonymous usage stats reporting and how to opt out. - [Configuring Autoscaling on VMs](https://ray-preview.mintlify.app/cluster/vms/configuring-autoscaling.md): Tune the Ray autoscaler on VM clusters. - [Get Started With Ray on VMs](https://ray-preview.mintlify.app/cluster/vms/getting-started.md): Install the cluster launcher, write your first cluster.yaml, and bring up a Ray cluster. - [Ray on VMs](https://ray-preview.mintlify.app/cluster/vms/index.md): Provision a Ray cluster on cloud VMs (AWS, GCP, Azure) or on-premises with the Ray cluster launcher. - [Best Practices for Large Clusters](https://ray-preview.mintlify.app/cluster/vms/large-cluster-best-practices.md): Operational guidance for Ray clusters with hundreds of nodes. - [Launching on AWS](https://ray-preview.mintlify.app/cluster/vms/launching-aws.md): Provision a Ray cluster on AWS EC2 with the cluster launcher. - [Launching on Azure](https://ray-preview.mintlify.app/cluster/vms/launching-azure.md): Provision a Ray cluster on Microsoft Azure with the cluster launcher. - [Launching on GCP](https://ray-preview.mintlify.app/cluster/vms/launching-gcp.md): Provision a Ray cluster on Google Cloud with the cluster launcher. - [On-Premises Clusters](https://ray-preview.mintlify.app/cluster/vms/launching-on-premises.md): Configure the Ray cluster launcher to manage a static set of on-premises hosts. - [Logging on VM Clusters](https://ray-preview.mintlify.app/cluster/vms/logging.md): Where Ray writes logs and how to aggregate them across nodes. - [Batch Inference](https://ray-preview.mintlify.app/data/batch-inference.md): Run model predictions over a large dataset with Ray Data, mixing CPU and GPU stages and scaling across the cluster. - [Ray Data Internals](https://ray-preview.mintlify.app/data/data-internals.md): How Ray Data's streaming executor schedules operators, manages memory, and recovers from failures. - [Inspecting Data](https://ray-preview.mintlify.app/data/inspecting-data.md): Examine a Ray Dataset's schema, sample rows, statistics, and execution plan. - [Iterating Over Data](https://ray-preview.mintlify.app/data/iterating-over-data.md): Consume Ray Datasets in training and inference loops with iter_batches, iter_torch_batches, and iter_rows. - [Ray Data Key Concepts](https://ray-preview.mintlify.app/data/key-concepts.md): The core abstractions in Ray Data: datasets, blocks, operators, and streaming execution. - [Loading Data](https://ray-preview.mintlify.app/data/loading-data.md): Read data into Ray Data from files, databases, in-memory objects, and Hugging Face datasets. - [Ray Data Overview](https://ray-preview.mintlify.app/data/overview.md): Ray Data is a scalable data processing library for ML workloads, built for the last mile of training and inference pipelines. - [Performance Tips for Ray Data](https://ray-preview.mintlify.app/data/performance-tips.md): Diagnose and resolve common Ray Data performance issues, from throughput tuning to memory pressure. - [Preprocessors](https://ray-preview.mintlify.app/data/preprocessors.md): Reusable, fittable transformations for Ray Datasets that integrate with Ray Train. - [Ray Data Quickstart](https://ray-preview.mintlify.app/data/quickstart.md): Build your first Ray Data pipeline: load a dataset, transform it, and consume the results. - [Saving Data](https://ray-preview.mintlify.app/data/saving-data.md): Write Ray Datasets to Parquet, CSV, JSON, images, and other formats. - [Shuffling Data](https://ray-preview.mintlify.app/data/shuffling-data.md): Choose between local shuffle, block-order randomization, and full distributed shuffle in Ray Data. - [Transforming Data](https://ray-preview.mintlify.app/data/transforming-data.md): Apply user-defined functions, filters, flat maps, and aggregations to a Ray Dataset. - [Working With LLMs](https://ray-preview.mintlify.app/data/working-with-llms.md): Run batch inference on large language models with Ray Data and vLLM. - [Working With Tensors](https://ray-preview.mintlify.app/data/working-with-tensors.md): Use tensor columns in Ray Data to handle multi-dimensional arrays alongside tabular data. - [Welcome](https://ray-preview.mintlify.app/index.md): Ray is an open-source unified framework for scaling AI and Python applications. - [LLM Batch Inference](https://ray-preview.mintlify.app/llm/batch-inference.md): Run batched LLM inference at scale with Ray Data. - [LLM Configuration](https://ray-preview.mintlify.app/llm/configuration.md): Tune engine arguments, tensor parallelism, quantization, and resource requests for Ray LLM. - [Multi-LoRA](https://ray-preview.mintlify.app/llm/multi-lora.md): Hot-swap LoRA adapters with Ray Serve and a single shared base model. - [Ray LLM Overview](https://ray-preview.mintlify.app/llm/overview.md): Train, fine-tune, and serve large language models with Ray's purpose-built LLM APIs. - [Ray LLM Quickstart](https://ray-preview.mintlify.app/llm/quickstart.md): Run a small LLM with Ray Serve and call it from an OpenAI-compatible client. - [Serving LLMs](https://ray-preview.mintlify.app/llm/serving.md): Deploy LLMs as autoscaling, OpenAI-compatible HTTP endpoints with Ray Serve. - [Ray Actors](https://ray-preview.mintlify.app/ray-core/actors.md): Use Ray actors to build long-lived, stateful workers that persist across method calls. - [Configuring Ray](https://ray-preview.mintlify.app/ray-core/configure.md): Tune cluster, runtime, and node-level configuration for Ray Core. - [Cross-Language Programming](https://ray-preview.mintlify.app/ray-core/cross-language.md): Use Ray to call between Python, Java, and C++ workloads in the same cluster. - [Fault Tolerance in Ray Core](https://ray-preview.mintlify.app/ray-core/fault-tolerance.md): Understand how Ray handles task failures, actor crashes, node failures, and the guarantees it provides for distributed applications. - [Handling Dependencies](https://ray-preview.mintlify.app/ray-core/handling-dependencies.md): Use runtime environments to specify per-task, per-actor, and per-job Python and system dependencies. - [Ray Core Key Concepts](https://ray-preview.mintlify.app/ray-core/key-concepts.md): The foundational building blocks of Ray: tasks, actors, objects, placement groups, environments, and namespaces. - [Ray Objects](https://ray-preview.mintlify.app/ray-core/objects.md): Share Python objects efficiently across tasks and actors using Ray's distributed object store. - [Ray Patterns and Anti-Patterns](https://ray-preview.mintlify.app/ray-core/patterns.md): Battle-tested patterns for structuring Ray applications, plus common pitfalls to avoid. - [Scheduling and Placement Groups](https://ray-preview.mintlify.app/ray-core/scheduling.md): Control where Ray tasks and actors run with resource requests, scheduling strategies, and placement groups. - [Ray Tasks](https://ray-preview.mintlify.app/ray-core/tasks.md): Use Ray tasks to parallelize Python functions across CPUs, GPUs, and nodes. - [Troubleshooting Ray Core](https://ray-preview.mintlify.app/ray-core/troubleshooting.md): Diagnose and fix common Ray Core issues, from worker crashes to object spilling. - [Spawning Subprocesses](https://ray-preview.mintlify.app/ray-core/user-spawn-processes.md): Run subprocesses inside Ray tasks and actors, and ensure they're cleaned up correctly. - [Ray Core Walkthrough](https://ray-preview.mintlify.app/ray-core/walkthrough.md): A guided tour of Ray Core's primary primitives: tasks, actors, and the distributed object store. - [Get Started With Observability](https://ray-preview.mintlify.app/ray-observability/getting-started.md): Set up the dashboard, Prometheus, and Grafana for a Ray cluster. - [Observability Key Concepts](https://ray-preview.mintlify.app/ray-observability/key-concepts.md): Sources of observability data in Ray: metrics, logs, profiles, traces, and the State API. - [Logging](https://ray-preview.mintlify.app/ray-observability/logging.md): Configure, route, and aggregate Ray logs. - [Metrics](https://ray-preview.mintlify.app/ray-observability/metrics.md): Scrape Ray's Prometheus metrics and visualize them with Grafana. - [Observability Overview](https://ray-preview.mintlify.app/ray-observability/overview.md): Inspect, debug, and monitor Ray applications with the dashboard, metrics, logs, profiles, and traces. - [Profiling](https://ray-preview.mintlify.app/ray-observability/profiling.md): Use py-spy, memray, and Ray timeline to profile Ray applications. - [Ray Dashboard](https://ray-preview.mintlify.app/ray-observability/ray-dashboard.md): Tour the Ray dashboard's tabs: cluster, jobs, actors, tasks, logs, metrics, and library-specific views. - [State API](https://ray-preview.mintlify.app/ray-observability/state-api.md): Programmatic access to the live state of a Ray cluster: nodes, jobs, actors, tasks, objects. - [Distributed Tracing](https://ray-preview.mintlify.app/ray-observability/tracing.md): Trace requests through Ray Serve, Ray Data, and your own application code with OpenTelemetry. - [Ray Examples](https://ray-preview.mintlify.app/ray-overview/examples.md): Curated end-to-end examples across Ray Core, Ray Data, Ray Train, Ray Tune, Ray Serve, and RLlib. - [Getting Started With Ray](https://ray-preview.mintlify.app/ray-overview/getting-started.md): Run your first Ray tasks, actors, and pipelines and learn how the core API scales Python from a laptop to a cluster. - [Welcome to Ray](https://ray-preview.mintlify.app/ray-overview/index.md): Ray is an open-source unified framework for scaling AI and Python applications, providing the compute layer for parallel processing without distributed systems expertise. - [Installing Ray](https://ray-preview.mintlify.app/ray-overview/installation.md): Install Ray on Linux, macOS, or Windows using pip, conda, Docker, or build from source. - [Ray Use Cases](https://ray-preview.mintlify.app/ray-overview/use-cases.md): Common AI and Python workloads that organizations build with Ray, from batch inference to reinforcement learning. - [Community](https://ray-preview.mintlify.app/ray-references/community.md): Connect with the Ray community on Slack, the discussion forum, GitHub, and at meetups. - [Contributing to Ray](https://ray-preview.mintlify.app/ray-references/contributing.md): Contribute code, documentation, and bug reports to the Ray project. - [Glossary](https://ray-preview.mintlify.app/ray-references/glossary.md): Definitions of terms used throughout Ray's documentation. - [Project Governance](https://ray-preview.mintlify.app/ray-references/governance.md): How the Ray project is governed and how technical decisions get made. - [Security](https://ray-preview.mintlify.app/ray-references/security.md): Ray's security model, threat assumptions, and how to report vulnerabilities. - [RLlib Algorithms](https://ray-preview.mintlify.app/rllib/algorithms.md): Choose between PPO, DQN, SAC, IMPALA, APPO, BC, MARWIL, CQL, and other algorithms RLlib ships with. - [Checkpoints](https://ray-preview.mintlify.app/rllib/checkpoints.md): Save, load, and restore RLlib algorithm state. - [Environments](https://ray-preview.mintlify.app/rllib/environments.md): Wrap Gymnasium, PettingZoo, and custom environments for RLlib. - [Get Started With RLlib](https://ray-preview.mintlify.app/rllib/getting-started.md): Train your first RL agent with RLlib in a few lines of Python. - [RLlib Key Concepts](https://ray-preview.mintlify.app/rllib/key-concepts.md): Algorithms, RL modules, learners, env runners, replay buffers, and the new API stack. - [Learner](https://ray-preview.mintlify.app/rllib/learner.md): RLlib's Learner consumes batches and updates weights. Configure it for multi-GPU and distributed training. - [Offline RL](https://ray-preview.mintlify.app/rllib/offline-rl.md): Train RLlib agents on logged trajectories with BC, MARWIL, and CQL. - [Ray RLlib Overview](https://ray-preview.mintlify.app/rllib/overview.md): RLlib is an open-source reinforcement learning library built on Ray for production-grade RL training and serving. - [Replay Buffers](https://ray-preview.mintlify.app/rllib/replay-buffers.md): Configure replay buffers for off-policy algorithms in RLlib. - [RL Modules](https://ray-preview.mintlify.app/rllib/rl-modules.md): Define custom policy networks with RLlib's RLModule API. - [Training Loop](https://ray-preview.mintlify.app/rllib/training.md): Inside an RLlib training iteration: rollouts, batches, learner updates, and metrics. - [Autoscaling](https://ray-preview.mintlify.app/serve/autoscaling.md): Scale Ray Serve deployments up and down based on traffic with built-in autoscaling. - [Configure a Deployment](https://ray-preview.mintlify.app/serve/configure-deployment.md): All options for the @serve.deployment decorator: replicas, resources, autoscaling, batching, and more. - [Develop and Deploy](https://ray-preview.mintlify.app/serve/develop-and-deploy.md): Move a Ray Serve application from local development to production with serve.run, serve deploy, and YAML configuration. - [Get Started With Ray Serve](https://ray-preview.mintlify.app/serve/getting-started.md): Define a Ray Serve deployment, run it locally, and call it via HTTP. - [gRPC Guide](https://ray-preview.mintlify.app/serve/grpc-guide.md): Serve gRPC endpoints from Ray Serve deployments. - [HTTP Guide](https://ray-preview.mintlify.app/serve/http-guide.md): Build HTTP APIs with Ray Serve: routes, request parsing, FastAPI, streaming, and middleware. - [Ray Serve Key Concepts](https://ray-preview.mintlify.app/serve/key-concepts.md): Deployments, replicas, applications, and the Serve controller. - [Serving LLMs](https://ray-preview.mintlify.app/serve/llm-serving.md): Serve large language models with Ray Serve, vLLM, and TensorRT-LLM. - [Model Composition](https://ray-preview.mintlify.app/serve/model-composition.md): Compose Ray Serve deployments into pipelines, ensembles, and dynamic dispatch graphs. - [Monitoring Ray Serve](https://ray-preview.mintlify.app/serve/monitoring.md): Metrics, logs, and dashboards for Ray Serve. - [Multi-Application Deployments](https://ray-preview.mintlify.app/serve/multi-app.md): Run multiple Ray Serve applications on the same cluster, isolated by route prefix. - [Ray Serve Overview](https://ray-preview.mintlify.app/serve/overview.md): Ray Serve is a scalable model-serving library for building online inference APIs in Python. - [Production Guide](https://ray-preview.mintlify.app/serve/production-guide.md): Run Ray Serve in production: deployment, observability, capacity planning, and failure handling. - [Checkpointing](https://ray-preview.mintlify.app/train/checkpointing.md): Save, resume, and manage training checkpoints with Ray Train. - [Data Loading for Ray Train](https://ray-preview.mintlify.app/train/data-loading.md): Feed Ray Datasets, PyTorch DataLoaders, and TensorFlow datasets into distributed training jobs. - [Distributed PyTorch](https://ray-preview.mintlify.app/train/distributed-pytorch.md): DDP, FSDP, and tensor parallelism with Ray Train and PyTorch. - [Distributed TensorFlow](https://ray-preview.mintlify.app/train/distributed-tensorflow.md): Use Ray Train with TensorFlow's MultiWorkerMirroredStrategy. - [Fault Tolerance in Ray Train](https://ray-preview.mintlify.app/train/fault-tolerance.md): How Ray Train handles worker, node, and storage failures, and how to configure recovery. - [Get Started With PyTorch](https://ray-preview.mintlify.app/train/getting-started-pytorch.md): Distributed PyTorch training with Ray Train: take a single-GPU script and scale it to many workers. - [Get Started With PyTorch Lightning](https://ray-preview.mintlify.app/train/getting-started-pytorch-lightning.md): Distributed Lightning training with Ray Train: scale your existing LightningModule with a few lines of configuration. - [Get Started With Hugging Face Transformers](https://ray-preview.mintlify.app/train/getting-started-transformers.md): Distributed fine-tuning of Hugging Face Transformers and Accelerate models with Ray Train. - [Ray Train Key Concepts](https://ray-preview.mintlify.app/train/key-concepts.md): The building blocks of Ray Train: trainers, workers, scaling configs, run configs, and checkpoints. - [Monitoring and Logging](https://ray-preview.mintlify.app/train/monitoring.md): Track Ray Train runs with the dashboard, MLflow, Weights & Biases, and TensorBoard. - [Ray Train Overview](https://ray-preview.mintlify.app/train/overview.md): Ray Train is a scalable distributed training library that integrates with PyTorch, Lightning, Transformers, JAX, TensorFlow, and XGBoost. - [Run Config](https://ray-preview.mintlify.app/train/run-config.md): Configure storage, naming, callbacks, checkpointing, and failure handling for a Ray Train run. - [Scaling Config](https://ray-preview.mintlify.app/train/scaling-config.md): Configure the number of workers, resource allocation, and GPU usage for a Ray Train job. - [Analyzing Results](https://ray-preview.mintlify.app/tune/analyzing-results.md): Inspect, compare, and export Ray Tune experiment results. - [Distributed Hyperparameter Tuning](https://ray-preview.mintlify.app/tune/distributed.md): Run Ray Tune trials across many nodes, with distributed-training trials and shared storage. - [Get Started With Ray Tune](https://ray-preview.mintlify.app/tune/getting-started.md): Run your first hyperparameter tuning job with Ray Tune. - [Ray Tune Key Concepts](https://ray-preview.mintlify.app/tune/key-concepts.md): Trials, search spaces, search algorithms, schedulers, and the Tuner API. - [Ray Tune Overview](https://ray-preview.mintlify.app/tune/overview.md): Ray Tune is a scalable hyperparameter tuning library for any ML framework, with state-of-the-art search algorithms and early-stopping schedulers. - [Trial Schedulers](https://ray-preview.mintlify.app/tune/schedulers.md): Stop unpromising trials early with ASHA, HyperBand, PBT, and PB2. - [Search Algorithms](https://ray-preview.mintlify.app/tune/search-algorithms.md): Choose the right search algorithm for your hyperparameter space: random, grid, Optuna, Ax, BayesOpt, BOHB, HyperOpt, Nevergrad. - [Search Space API](https://ray-preview.mintlify.app/tune/search-space.md): Define hyperparameter distributions for Ray Tune trials. - [Stoppers](https://ray-preview.mintlify.app/tune/stoppers.md): Stop a Ray Tune experiment when a condition is met. - [Trial Checkpoints](https://ray-preview.mintlify.app/tune/trial-checkpoints.md): Save and restore Ray Tune trial state for resumption, PBT, and best-trial recovery. - [Troubleshooting Ray Tune](https://ray-preview.mintlify.app/tune/troubleshooting.md): Diagnose common Ray Tune issues: hangs, OOMs, lost trials, and slow searches. ## OpenAPI Specs - [ray-serve](https://ray-preview.mintlify.app/api-reference/openapi/ray-serve.json) - [ray-jobs](https://ray-preview.mintlify.app/api-reference/openapi/ray-jobs.json) ## Optional - [GitHub](https://github.com/ray-project/ray) - [Slack](https://www.ray.io/community#slack) - [Discuss Forum](https://discuss.ray.io)