AI QA Engineer (R-18856)

Shape the Future with Dun & Bradstreet At Dun & Bradstreet, we believe data has the power to create a better tomorrow. As a global leader in business decisioning data and analytics, we help companies worldwide grow, manage risk, and innovate. For over 180 years, businesses have trusted us to turn uncertainty into opportunity. We’re a diverse, global team that values creativity, collaboration, and bold ideas. Are you ready to make an impact and help shape what’s next? Join us! Explore opportunities at dnb.com/careers. We are looking for a highly skilled AI Tool/Agent Testing Engineer to evaluate, validate, and ensure the reliability of AI agents, AI automation tools, and agentic workflows used across our analytics platform. This role blends test engineering, GenAI understanding, Python programming, and agent development lifecycle knowledge. You will work closely with Data Science, AI Engineering, and Platform teams to ensure that AI agents behave predictably, safely, and in alignment with business and compliance requirements Key Responsibilities:

Design, develop, and execute test plans and test cases to validate the functionality, performance, and reliability of AI/ML systems

Collaborate with the development team to understand model requirements and identify potential areas of weakness

Perform data validation to ensure the quality, integrity, and representativeness of the datasets used for training and testing

Test for and identify potential biases in AI models to ensure fairness and ethical compliance

Analyze and report on model performance using key metrics (e.g., accuracy, precision, recall, F1-score)

Ensure accuracy, consistency, tool-call reliability, trace quality, and guardrail adherence

Assess regression, functionality, performance, safety, and hallucination risks

Document and track defects and work with developers to facilitate their resolution

Assist in the development and maintenance of automated testing frameworks for AI applications

Conduct exploratory testing to discover edge cases and unexpected behaviors in our AI systems

Stay current with the latest advancements in AI testing methodologies and tools

Produce test plans, scenario libraries, coverage reports, and defect logs

Deliver insights to Data Science & Engineering teams to improve reliability

Key Skills:

Bachelor's degree in computer science, Engineering, Statistics, or a related technical field

Solid understanding of software testing principles and the software development lifecycle (SDLC)

Basic programming proficiency, preferably in Python, for writing test scripts and analyzing data

A foundational understanding of machine learning concepts (e.g., supervised/unsupervised learning, classification, regression)

Strong analytical and problem-solving skills with an exceptional eye for detail

Excellent communication and collaboration skills, with the ability to articulate technical issues clearly

Prior internship or project experience in software testing or a data-related field

Familiarity with machine learning libraries and frameworks such as TensorFlow, PyTorch, or Scikit-learn

Experience with testing tools and frameworks like PyTest, Selenium, or Postma

Knowledge of SQL for querying and validating data

Familiarity with version control systems like Git

A genuine passion for artificial intelligence and a desire to learn and grow in the field