HPC Infrastructure Engineer - GPU Clusters
About ElevenLabs
ElevenLabs is an AI research and product company transforming how we interact with technology. We launched in January 2023 with the first human-like AI voice model. Today, we serve millions of users and thousands of businesses - from fast-growing startups to large enterprises like Deutsche Telekom and Meta. Our investors are some of the world's most prominent, including Andreessen Horowitz, ICONIQ Growth and Sequoia. We've raised $781M in funding and our last valuation was $22B - multiples of 11, always.
We have expanded from voice into three main platforms:
- ElevenAgents enables businesses to deliver seamless and intelligent customer experiences, with the integrations, testing, monitoring, and reliability necessary to deploy voice and chat agents at scale.
- ElevenCreative empowers creators and marketers to generate and edit speech, music, image, and video across 70+ languages.
- ElevenAPI gives developers access to our leading AI audio foundational models.
Everything we do is the result of the creativity and commitment of our team - builders doing the best work of their lives. We are researchers, engineers, and operators. IOI medalists and ex-founders. If you want to work hard and create lasting positive impact, we want to hear from you.
How we work
High-velocity
Impact not job titles
AI first
Excellence everywhere
Global team
What we offer
Innovative culture
Growth paths
Learning & development
Social travel
Annual company offsite
Co-working
About the role
What you’ll be doing
- Operate and improve our GPU fleet end to end: provisioning, scheduling, monitoring, upgrades, capacity planning
- Build automation that keeps the fleet healthy without human intervention - node health checks, automated draining and remediation, burn-in pipelines for new capacity
- Own the stack beneath the training code: OS images, NVIDIA drivers, CUDA, container runtimes, NCCL, high-speed networking (InfiniBand/RoCE)
- Run and tune job scheduling (Slurm or similar) so researchers get compute fairly and fast
- Build and maintain high-performance storage for datasets and checkpoints
- Hunt down performance problems: stragglers, degraded links, thermal issues, flaky GPUs - and fix the class of problem, not just the instance
- Evaluate rented GPU capacity: benchmark it, validate it, hold providers to their SLAs
- Hands-on hardware work when it's needed: racking, cabling, diagnostics, coordinating with datacenter staff and vendors
- Keep clusters secure by default: access control, network isolation, secrets
Requirements
- Have run large-scale Linux server or GPU environments in production and enjoy both building and operating
- Know the NVIDIA stack well - drivers, CUDA, NCCL, DCGM - or have deep systems experience and learn hardware stacks fast
- Are comfortable with bare-metal environments, server hardware, and high-speed networking
- Write solid automation in Python and/or Bash, with IaC tools like Ansible or Terraform
- Are happy digging into noisy data (metrics, logs, PromQL) to find what's actually wrong
- Like owning real scope end to end and being the person others rely on
- Don't consider any task above or beneath you - datacenter trips included
Nice to have
- Experience supporting ML training workloads from the infra side (distributed training failure modes, checkpointing patterns)
- Experience evaluating and working with GPU cloud providers
- Parallel filesystems (WEKA, VAST, etc) or large-scale object storage
- BMC/IPMI/Redfish automation, PXE provisioning at scale
- Power and cooling awareness for dense GPU deployments
Location
Skills
Who can apply
The employer didn't state any visa, work authorization, citizenship or clearance requirements in this posting. Confirm with the employer before applying.
Read automatically from the employer's posting text. Always confirm with the employer — requirements can change after a job is published.