Senior QA Automation Engineer

Arbital Health is a rapidly growing healthcare technology and actuarial leader that centralizes, measures, and adjudicates value-based care contracts at scale.  We enable payers and providers to design, measure, and execute value-based agreements with greater transparency, efficiency, and financial predictability. We invest in hiring high potential and humble individuals who thrive in fast-paced environments and can rapidly grow their responsibilities as we continue to accelerate our growth. We were co-founded by Brian Overstreet and Travis May (founder & former CEO of LiveRamp and Datavant, the two biggest data companies of the last 20 years), and are backed by Transformation Capital, Valtruis and other leading investors. In our first 2 years, Arbital Health has established itself as a trusted partner for over 40 payers, providers, and other stakeholders looking to navigate the complexities of risk-based contracting. We’re hiring a Senior QA Automation Engineer to lead and elevate quality across our SaaS platform and its expanding suite of AI-driven capabilities. In this role, you will architect and implement scalable test frameworks, drive our automation strategy, and ensure robust, reliable releases. You’ll work closely with product, engineering, and data teams to uphold the highest standards—especially important given our work with regulated data, including healthcare and medical claims. This role will be expected to work from our SF office on Mondays and Wednesdays. This role might be for you if:

You thrive in ambiguity, value collaboration, and want to be part of a culture that supports rapid innovation without compromising quality.

You have experience managing manual and automated testing needs in a fast paced startup.

You’ve tested AI features in production and know how to validate retrieval quality, hallucination risks, and prompt sensitivity

You have proven success in creating comprehensive and maintainable automated tests for a fast moving product.

You have the experience and intuition to see around corners, identify quality risks, and implement preemptive solutions through automation.

You have built a scalable, reliable QA automation framework integrated into the development pipeline for a SaaS platform.

What you’ll do:

Lead the design, development, and execution of comprehensive test plans and QA strategy for our growing healthcare data-centric SaaS Platform

Build and execute comprehensive testing frameworks for AI/ML systems, including LLMs, RAG, and agentic workflows

Apply modern AI testing techniques including adversarial testing, red-teaming, and safety and robustness evaluation

Write, maintain, and scale automated test code for the full stack, APIs, and data validation using modern testing frameworks.

Collaborate with product managers, actuaries, UX designers, and other engineers to design and implement automated UI tests, API tests, Performance monitoring tests to validate new features and detect regressions in existing features.

Own the QA automation lifecycle and continuously enhance the automation infrastructure.

Select and implement automated testing frameworks and tools that are the best fit for Arbital’s platform and team.

Manage and monitor both automated and manual test execution to support continuous delivery.

Ensure compliance and robustness when handling regulated data such as healthcare or financial information.

Create detailed and easy to understand bug reports.

Provide technical mentorship and leadership within the QA team, especially in a dynamic startup environment.

What you bring:

8+ years of experience in QA Automation, with at least 2 years in a senior or lead role.

Experience leading QA Automation initiatives for a SaaS platform.

Strong programming skills and hands-on experience writing automation code (JavaScript/TypeScript, Python, or similar).

Proven experience using automation frameworks for UI testing, such as Playwright, Selenium, Cypress, or similar.

Hands-on experience in testing AI/ML systems, particularly LLM-based applications, RAG pipelines, or agentic workflows

Solid understanding of CI/CD pipelines, version control (Git), and agile development methodologies.

Working knowledge of build and release tools such as GitHub Actions or Jenkins

Excellent communication, collaboration, analytical, and problem-solving skills.

Plus's:

Experience with performance and security testing

Experience with healthcare data (e.g., HIPAA, PHI) or other regulated environments. Understanding of AI failure modes, including hallucinations, prompt sensitivity, data leakage, and non-deterministic behavior

Experience with Data pipelines and frameworks for data testing, such as Great Expectations.

Contributions to open-source testing tools or QA automation libraries

Interest or past experience managing/leading other QA engineers