Director, Global AI Monitoring and Testing

The Global Data Office
With data being the fuel that drives our future - our strategies, policies, and business successes around data will define our future growth prospects. Unlocking the value available through the innovative use of data on behalf of consumers, businesses, and communities is key to our future. With our ongoing commitment to Visa’s Data Values and the responsible use of data, we at Visa have a bold vision to continue to grow and accelerate our data-related businesses and capabilities.

Led by Visa’s first Chief Data Officer, the Global Data Office coordinates high-impact, complex, company-wide projects related to data business strategy, critical investments, policy development, and marketplace execution. Working closely with Visa’s Chief Privacy Officer, the Technology organization, the wider data community, and the full set of Visa’s global lines of business, the Global Data Office is evolving Visa’s data-related work with momentum and enthusiasm. Driven by its commitment to trusted use of data, Visa is seeking enthusiastic leaders who are change-agents and passionate about the future of data around the world.

 JOB DESCRIPTION

AI Governance, part of the newly established AI & Data Innovation Department is unleashing new opportunities at Visa by developing and driving the business strategy for AI / Gen AI and data innovation. The AI & Data Innovation Department is driving the development of novel AI / GenAI / data science technologies, leading data product development to create value for clients and partners and driving innovative application of AI / data science to solve business problems among other Innovation activities. The AI & Data Innovation Department is part of the broader Global Data Office (GDO) led by Visa’s Chief Data Officer.

The AI Governance Team is accountable for creating and owning Visa’s entire AI Governance policy and processes. The AI Governance team is a critical catalyst for data innovation, maximizing the impact from AI and Gen AI technologies and capabilities in a way that aligns with our Data and AI principles.

The Director of Global AI Monitoring & Testing will report into the Global Head of AI Governance and will be a key member of the AI Governance team. This role will develop, build, own and update Visa’s monitoring and testing framework, policies, and processes in close collaboration with cross-functional stakeholders. The Director will work hand in hand with Technology and Risk partners to roadmap and support the build of AI model tests (e.g., bias, fairness, etc.) as well as implement ongoing monitoring and testing.

This individual will effectively evaluate and measure adherence to policies, standards, frameworks and identify, assess, and communicate areas of strength and opportunity to strengthen our AI capabilities and control environment. The individual will provide oversight to ensure practices meets internal standards, applicable regulatory requirements and apply industry best practices. This role will have visibility with senior leaders. The AI Governance team will work closely with Visa’s functions in Privacy, Legal, Risk, Data Use, Technology, Product Delivery, Cybersecurity among others.

• ESSENTIAL FUNCTIONS
  • Work together with AI sponsors, developers, and cross-functional stakeholders to help maximize value creation through AI in the right way, balancing innovation, regulatory guidance, and consumer risks.
  • Use knowledge of Visa’s business strategy, AI Governance, Global Data Policy, and Global Privacy Program requirements to inform risk prioritized test plans.
  • Develop testing strategies and work plans, informed by interviews with first, second, and third lines of defense.
  • Partner closely with Technology and Model Risk Management to gather technical requirements and execute scalable AI systems monitoring and testing.
  • Execute testing engagements to test, monitor, and validate controls are working as intended.
  • Independently assess AI practices against policies, frameworks, standards, and guidelines.
  • For issues identified in testing, provide recommendations for control enhancements and follow-up as necessary to ensure proper resolution.
  • Prepare management reports and related test results dashboards both historical and to assist management in the early identification of risks and trends.
  • Ensure the implementation of processes and testing in a scalable way through technology solutions for risk management and process automation.
  • Track and understand the fast-moving AI monitoring and testing landscape and propose changes to monitoring & testing as well as overall AI Governance as needed.
  • Provide ad hoc support for other AI Governance workstreams as needed or special projects.

This is a hybrid position. Hybrid employees can alternate time between both remote and office. Employees in hybrid roles are expected to work from the office 2-3 set days a week (determined by leadership/site), with a general guidepost of being in the office 50% or more of the time based on business needs.

If you are a self-starter, detail-oriented problem solver, change agent, and global cross-functional collaborator who loves to work with complex data technologies, this will be a perfect role for you. Ideal candidates would have experience in AI policies as well as monitoring and testing.

Basic Qualifications: 

• 10+ years of relevant work experience with a Bachelor’s Degree or at least 7 years of work experience with an Advanced degree (e.g. Masters, MBA, JD, MD) or 4 years of work experience with a PhD, OR 13+ years of relevant work experience.
• 6+ years of work experience in high-profile/high-impact project management
• Experience formulating technical requirements and working with technology and engineering to build platforms and systems.
• Excellent data analysis skills and demonstrated experience in testing, validation, and quality assurance
• Strong written and verbal communication and presentation skills (ability to translate complex information into comprehensive and concise test results and recommendations)
• Advanced knowledge of Microsoft Suite of Office Tools
• Advanced knowledge of AI / Generative AI / Data Science
• Extensive project management skills with the use of project management software
• Curious and lifelong learner who keeps up to date on emerging trends and technology
• Successful track record of operating cross-functionally, ideally in a federated enterprise design
• Proven ability to work under deadlines and manage multiple projects with evolving requirements.
• Team player possessing a strong drive to add value to the business

Preferred Qualifications: 

• 12 or more years of work experience with a Bachelor’s Degree or 8-10 years of experience with an Advanced Degree (e.g. Masters, MBA, JD, MD) or 6+ years of work experience with a PhD
• Education or work experience in data science or engineering
• Knowledge and understanding of risk management practice (e.g., experience in identifying, assessing, treating, and monitoring risks) with ability to turn academic concepts into business practical concepts
• Experience working to uplift and mature monitoring and testing capabilities, policies definition, controls and frameworks
• Audit and/or compliance experience, preferably with big 4 or other large financial institution, consulting firm or government agency
• Knowledge of industry frameworks and best practices
• Sector experience in payment processing/payment services, financial services (within a regulated environment), other regulated industries, or technology/financial technology
• CPA, ACAMs, CRCM CISA, or CIA

Strategic & Functional Excellence: 

Self-starter, detail, and process oriented

• Self-starter who can responsibly advance work and is comfortable making sound, well-informed judgement calls for progress, without needless delays.
• Experienced at designing and building monitoring and testing programs that balance innovation with governance.
• High attention to detail and quality with a deep commitment to accountability.

Collaborator

• Ability to successfully lead complex projects and processes with multiple stakeholders in an unstructured and sometimes ambiguous environment.
• Translate highly complex technology, technical concepts, and issues into language understandable by cross-functional stakeholders.
• Proven ability to influence and build credibility with business and technical leaders across a federated organization, despite sometimes competing priorities.

Proactive and effective communicator

• Strong leadership and interpersonal skills to develop and sustain relationships across levels, role types, and geographies.
• Exceptionally skilled at keeping complex issues clear, simple, and user friendly.
• Experienced at presenting information to senior levels, with the ability to facilitate group discussions and debate and a proven ability to influence and communicate effectively both virtually and in person.

AI / Data Science

• Need broad technical knowledge and depth on AI / ML / Data Science to liaise with technical teams to execute on responsibilities listed above effectively.

Work Hours: Varies upon the needs of the department.

Travel Requirements: This position requires travel 5-10% of the time.

Mental/Physical Requirements: This position will be performed in an office setting.  The position will require the incumbent to sit and stand at a desk, communicate in person and by telephone, frequently operate standard office equipment, such as telephones and computers.

Visa is an EEO Employer.  Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status.  Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.

Visa will consider for employment qualified applicants with criminal histories in a manner consistent with applicable local law, including the requirements of Article 49 of the San Francisco Police Code.

U.S. APPLICANTS ONLY: The estimated salary range for a new hire into this position is 179,200.00 to 233,000.00 USD per year, which may include potential sales incentive payments (if applicable). Salary may vary depending on job-related factors which may include knowledge, skills, experience, and location. In addition, this position may be eligible for bonus and equity. Visa has a comprehensive benefits package for which this position may be eligible that includes Medical, Dental, Vision, 401 (k), FSA/HSA, Life Insurance, Paid Time Off, and Wellness Program.