Software Engineer, Inference (AI Data Engineering)

• Develop highly reliable, high-throughput inference systems that serve the best AI models internally across SpaceX

• Architect and implement scalable distributed infrastructure for model serving, including load balancing, auto-scaling, batch scheduling, global KV cache, and continuous batching

• Optimize latency and throughput of model inference under real production workloads, including low-level GPU kernel work, quantization, speculative decoding, and other acceleration techniques

• Build reliable, high-concurrency serving systems with 100% uptime, low tail latency, and excellent observability

• Own end-to-end components such as request routing, SDK development, rate limiting, and efficient scaling for internal SpaceX AI inference platforms

• Benchmark, fine-tune, and accelerate inference engines (e.g., SGLang, vLLM, TensorRT-LLM)

• Develop custom tools for tracing, replaying, and resolving issues across the full stack - from orchestration down to GPU kernels

• Create robust CI/CD infrastructure for seamless endpoint deployment, image publishing, and inference engine updates

• Collaborate across SpaceXAI teams to integrate inference capabilities into broader systems and workflows

• Bachelor's degree in computer science, engineering, math, or scientific discipline; OR 2+ years of professional experience building software in lieu of a degree

• Experience in designing, implementing, and maintaining reliable and horizontally scalable distributed systems

• 1+ years of experience in full stack development or backend development with production systems

• 1+ years of experience with Rust or C++

• Experience with LLM inference engines and serving frameworks (e.g., SGLang, vLLM, Triton, TensorRT-LLM)

• Deep low-level systems programming and optimizations: GPU kernels, code generation, batching, caching, parallelism, quantization, and speculative decoding

• Experience with large-scale, high-concurrency production serving systems

• Knowledge of service observability and reliability best practices

• Experience operating commonly used databases such as PostgreSQL, ClickHouse, or MongoDB

• Experience designing or building with agent SDKs and agent orchestration frameworks

• Experience with Docker, Kubernetes, and containerized applications

• Expert knowledge of gRPC (unary, response streaming, bi-directional streaming, REST mapping)

• Programming experience in Python, Go, or similar languages

• Experience with version control, continuous integration, continuous delivery, build systems, and monitoring

• Expertise in profiling and improving application performance

• You may be asked to work extended hours/weekends dependent on launch cadence and platform demands

• This role requires you to be onsite in Palo Alto. Remote and/or hybrid work will not be considered

• To conform to U.S. Government export regulations, applicant must be a (i) U.S. citizen or national, (ii) U.S. lawful, permanent resident (aka green card holder), (iii) Refugee under 8 U.S.C. § 1157, or (iv) Asylee under 8 U.S.C. § 1158, or be eligible to obtain the required authorizations from the U.S. Department of State. Learn more about the ITAR here .

SpaceX is an Equal Opportunity Employer; employment with SpaceX is governed on the basis of merit, competence and qualifications and will not be influenced in any manner by race, color, religion, gender, national origin/ethnicity, veteran status, disability status, age, sexual orientation, gender identity, marital status, mental or physical disability or any other legally protected status.

Applicants wishing to view a copy of SpaceX's Affirmative Action Plan for veterans and individuals with disabilities, or applicants requiring reasonable accommodation to the application/interview process should reach out to EEOCompliance@spacex.com .