Head of Quality Engineering
Radai
San Francisco - OnsitePosted 2 days agoDiscoveredMatch locked
On-siteFullTime5+ years expEngineering
About Rad AI
At Rad AI, we're on a mission to transform healthcare with artificial intelligence. Founded by a radiologist, our AI-driven solutions are revolutionizing radiology - saving time, reducing burnout, and improving patient care. With one of the largest proprietary radiology report datasets in the world, our AI has helped uncover hundreds of new cancer diagnoses and reduced error rates in tens of millions of radiology reports by nearly 50%.
Rad AI has secured over $140M in funding, including a recently oversubscribed Series C ($68M round) led by Transformation Capital, bringing our valuation to $528M. Our investors include Khosla Ventures, World Innovation Lab, Gradient Ventures, Cone Health Ventures, and others - all backing our mission to empower physicians with cutting-edge AI.
Our latest advancements in generative AI are used by thousands of radiologists daily, supporting more than one-third of radiology groups and healthcare systems and nearly 50% of all medical imaging in the U.S. at partners including Cone Health, Jefferson Einstein Health, Geisinger, Guthrie Healthcare System, and Henry Ford Health.
Recognized as one of the most promising healthcare AI companies by CB Insights and AuntMinnie , and ranked by Deloitte as the 19th fastest-growing company in North America, we are building AI-powered solutions that make a real impact. Most recently, Rad AI was named to CNBC's Disruptor 50 list, highlighting the innovation and momentum behind our mission.
If you're ready to shape the future of healthcare, we'd love to have you on our team!
Why We Need You
Radiologists use our products every day to produce the reports that decide what care a patient gets next. A defect here isn't an inconvenience; it's a blank report after signing, a missing accession, or a formatting error in a clinical impression. That's not a context where "good enough" ships.
We already have a strong, embedded quality team across Reporting, Impressions, Continuity. and it's stronger than our current quality outcomes show. Some areas already run PR-level checks, developer-written end-to-end tests, AI-assisted test creation, and automated regression triage. The gap is consistency: practices and maturity vary by product area, and the quality customers experience doesn't yet match the work happening inside.
Our principle is simple:
Engineering owns quality; Quality owns the quality system.
You'll lead that system, spreading what already works rather than rebuilding it. You'll make release risk explicit, bring real radiologist workflows and customer environments into how we test, and make quality visible from the team level up to the executive team. The team you build will define how clinicians trust AI in their daily work.
You won't start from zero. You'll inherit a three-phase quality roadmap already underway, an initial quality baseline, and a revamped weekly incident review, and you'll own where they go next.
Here's What You'll Be Doing
You'll lead Rad AI's Quality organization, reporting to the CPO, where you'll:
•
Lead one Quality team while keeping it embedded.
Run a shared operating cadence, priorities, standards, and roadmap across quality engineers who stay embedded in their product teams. Start from what already works and spread it, standardizing where it matters and allowing local differences where they help.
•
Own the quality system, not every test.
Provide strategy, risk assessment, tooling, environments, coaching, and visibility so developers own the quality of what they build. Standardize test case management, suite organization, and CI integration across product areas.
•
Bring quality into planning.
Make test strategy and risk part of planning with Product, Engineering, Design, and Clinical, so every change knows what to test, at which level, and why. Keep coverage efficient by deduplicating across unit, API, and end-to-end tests.
•
Make release readiness risk-based.
Establish one release lifecycle across product areas: a pre-release risk assessment, radiologist persona test cases, a common minimum smoke-test baseline and explicit testing expectations for hotfixes. The risk assessment drives the test matrix: which product areas, integrations, environments, and customer workflows each change touches. Evolve go/no-go process from per-release into an org-wide definition of "ready to ship" that also accounts for customer's own quality control processes while preserving flexibility for Rad AI to roll out change management.
•
Close the gap between green tests and the customer experience.
Define radiologist personas and representative workflows with Product and Clinical, and give the team direct exposure to how radiologists actually work through shadowing, customer recordings, and product telemetry. Bring customer-matched environments (Windows workstations, dictation systems, PACS/RIS integrations, site-specific configurations) into how we test, and decide where exploratory and manual testing add more value than automation.
•
Keep the test signal trustworthy.
Own test stability end to end: failure triage, a clear policy for quarantining and fixing flaky tests, and stable test environments built with our platform and infrastructure teams. Rationalize duplicated and low-value coverage.
•
Make quality visible.
Define and report quality metrics consistently across product areas: customer-found vs. internally found defects, escaped defects by product area, change failure rate, test stability, and time to detect. Set quality OKRs tied to company goals, publish a company-level quality dashboard, and run quarterly quality reviews with Engineering and Product leadership, so leadership can always answer: what is the state of quality, where are the biggest risks, and are we getting better?
•
Turn incidents and escaped defects into prevention.
Partner with incident review owners, Support, and Customer Success so corrective actions land as tests, observability, or fixes, and incident and customer-defect learnings update regression coverage and risk profiles. Hold a quality bar for root cause analyses, and make sure Quality learns from customers without becoming a Tier 2 support desk.
•
Define quality for AI systems.
Establish how we evaluate non-deterministic outputs, such as speech recognition and generated impressions, alongside our ML and product teams.
•
Embed clinical risk and compliance into quality.
Partner with Clinical, Security, and Compliance so severity reflects patient impact and quality practices support HIPAA, auditability, and regulatory expectations for clinical AI.
•
Codify and share how we ensure quality.
Document our standards, build a Quality community of practice across product teams, run Quality Week, office hours, and demos, and close the feedback loop from Quality back to the engineers who build.
•
Build and develop the team.
Hire, coach, and grow quality engineers, own the Test Automation Engineer career framework, manage quality tooling, vendors, and test infrastructure cost-effectively, and push the limits of AI to raise both the efficiency and the reach of quality work.
Who We're Looking For
- 10+ years in software quality and test engineering, including 5+ years leading quality teams, with a track record of building a quality function and staffing it cost-effectively as the company grows.
- Experience taking over a capable but inconsistent quality organization and raising the bar across teams by spreading what works, not rebuilding it.
- A strong hands-on foundation in test automation (end-to-end, API, and desktop or Electron UI automation in CI/CD pipelines, plus test environments), and you can still go deep on test architecture and flaky-test root causes.
- Experience shifting quality left: engineers own quality, the quality team builds the system, and quality never becomes the gate everyone waits on.
- A data-driven approach: you've defined quality metrics such as escaped defects, change failure rate, and test stability, set OKRs against them, and moved them.
- Experience with risk-based testing for integration-heavy products that run across many customer configurations and environments.
- The ability to build a long-term quality roadmap and broker decisions with engineering leaders on architecture and testability.
- Excellent communication across technical and non-technical audiences, with a track record of influencing Engineering, Product, Support, and executives without formal authority.
- Expert hiring and people management for distributed teams across time zones: hiring, onboarding, career frameworks, and performance management.
- Experience managing quality tooling and vendor relationships, such as test management platforms, outsourced QA, or test environment providers.
- Asks great questions, start with the smallest change that adds value, and use AI tools to make quality work faster and broader.
Nice to Haves
- Experience in healthtech or a regulated environment (HIPAA, PHI, SOC 2), and ideally with medical device software practices (SaMD, IEC 62304, ISO 14971 risk management).
- Familiar with radiology workflows and systems: PACS, RIS, EHR, dictation platforms such as PowerScribe or Fluency, HL7/FHIR, FHIRcast, or DICOM.
- Evaluated ML or AI systems in production, such as speech recognition accuracy or generated-text quality, where outputs aren't deterministic.
- Owned performance, load, or reliability testing, and partnered closely with incident management.
- Led a multi-team quality organization, including managers or Staff-level engineers.
- Built or scaled a quality function at a high-growth startup.
Join our world-class team as we build and deploy AI solutions that empower physicians and transform patient care - making a meaningful impact on millions of lives. Driven by our mission, we prioritize transparency, inclusion, and close collaboration, bringing together exceptional people to revolutionize healthcare. If you're passionate about driving innovation and delivering impactful healthcare solutions, we'd love to hear from you! To learn more about what it's like to work at Rad AI, visit https://www.radai.com/life-at-rad-ai and be sure to follow us on LinkedIn to stay up to date!
Location Details:
For roles listed as San Francisco - Onsite :
- This role will be based in our San Francisco office and we expect employees to work onsite four days per week. The remaining time may be worked remotely or onsite, depending on team and business needs.
For roles listed as United States - Remote :
- This role is open to candidates located anywhere in the United States.
For roles listed as San Francisco - Onsite + United States - Remote :
- We will prioritize candidates who can work onsite four days per week in San Francisco, while also considering remote candidates located anywhere in the United States.
For US-Based Full-Time Roles, Rad AI offers a variety of benefits, including
- Comprehensive Medical, Dental, Vision & Life insurance
- HSA (with employer match), FSA, & DCFSA
- 401(k)
- 11 Paid Company Holidays
- Flexible PTO policy
- Annual company-wide offsite
- Periodic team offsites
- Annual equipment stipend
- For roles based outside the US, your recruiter can share more details
At Rad AI, we value diversity and provide equal employment opportunities (EEO) to all employees and applicants without regard to race, color, religion, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. We will consider for employment qualified applicants with criminal histories in a manner consistent with the requirements of the San Francisco Fair Chance Ordinance.
Please be vigilant regarding job scams. We advise all candidates to apply directly through our official careers page. Our recruiters will use email addresses with the domain @ radai.com or no-reply@ashbyhq.com.
Not included in the source posting: what you'll do.
Skills
artificial-intelligencegoci-cdgenerative-aimachine-learningui
Who can apply
The employer didn't state any visa, work authorization, citizenship or clearance requirements in this posting. Confirm with the employer before applying.
Read automatically from the employer's posting text. Always confirm with the employer — requirements can change after a job is published.