Senior AI QA Analyst (R-18849)

Shape the Future with Dun & Bradstreet At Dun & Bradstreet, we believe data has the power to create a better tomorrow. As a global leader in business decisioning data and analytics, we help companies worldwide grow, manage risk, and innovate. For over 180 years, businesses have trusted us to turn uncertainty into opportunity. We’re a diverse, global team that values creativity, collaboration, and bold ideas. Are you ready to make an impact and help shape what’s next? Join us! Explore opportunities at dnb.com/careers. We are looking for a highly skilled AI Tool/Agent Testing Engineer to evaluate, validate, and ensure the reliability of AI agents, AI automation tools, and agentic workflows used across our analytics platform. This role blends test engineering, GenAI understanding, Python/PySpark proficiency, and agent development lifecycle knowledge. You will work closely with Data Science, AI Engineering, and Platform teams to ensure that AI agents behave predictably, safely, and in alignment with business and compliance requirements Key Responsibilities:

Build agentic workflows using Lang Chain/Lang Graph and similar frameworks

Develop autonomous agents for data validation, reporting, document processing, and domain workflows

Deploy scalable, resilient agent pipelines with monitoring and evaluation

Develop GenAI applications using models like GPT, Gemini, and LLaMA

Implement RAG, vector search, prompt orchestration, and model evaluation

Partner with data scientists to productionize POCs

Build distributed data pipelines (Python, PySpark)

Develop APIs, SDKs, and integration layers for AI-powered applications

Optimize systems for performance and scalability across cloud/hybrid environments

Contribute to CI/CD workflows for AI models—deployment, testing, monitoring

Implement governance, guardrails, and reusable GenAI frameworks

Work with analytics, product, and engineering teams to define and deliver AI solutions

Participate in architecture reviews and iterative development cycles

Support knowledge sharing and internal GenAI capability building

Key Skills:

6-8 years of experience in AI/ML engineering, data science, or software engineering, with at least 2 years focused on GenAI

Strong programming expertise in Python, distributed computing using PySpark, and API development

Hands on experience with LLM frameworks (Lang Chain, Lang Graph, Transformers, OpenAI/Vertex/Bedrock SDKs)

Experience developing AI agents, retrieval pipelines, tool calling structures, or autonomous task orchestration

Solid understanding of GenAI concepts: prompting, embeddings, RAG, evaluation metrics, hallucination identification, model selection, fine tuning, context engineering

Experience with cloud platforms (Azure/AWS/GCP), containerization (Docker), and CI/CD pipelines for ML/AI

Strong problem solving, system design thinking, and ability to translate business needs into scalable AI solutions

Excellent verbal, written communication and presentation skills