How NVIDIA Mission Control 2.5 Brings Autonomous Recovery to Kubernetes
According to NVIDIA Developer. NVIDIA has extended its autonomous recovery engine to Kubernetes-based workloads managed by NVIDIA Run:ai in Mission Control 2.5.
- • Mission Control 2.5 extends autonomous recovery to Kubernetes workloads managed by NVIDIA Run:ai.
- • The system uses a three-layer control loop consisting of the scheduler, job recovery, and hardware recovery.
- • Hardware faults trigger parallel tracks where workloads restart while faulty nodes undergo diagnostics.
Extending Autonomous Recovery
NVIDIA announced Mission Control 2.5, extending its autonomous recovery engine to Kubernetes-based workloads managed using NVIDIA Run:ai. Previously, this recovery engine was limited to high-performance computing workloads leveraging Slurm, helping move AI factory operations toward self-healing infrastructure.
Three-Layer Control Loop
The autonomous recovery engine operates through an integrated three-layer framework: the Kubernetes scheduler places workloads, the autonomous job recovery service classifies failures using job context, and the autonomous hardware recovery service isolates and diagnoses faulty infrastructure.
- 1. developer.nvidia.comOpen source