AI Tech Lead, San Francisco Bay Area

Role Overview

We're seeking an experienced AI Technical Lead to spearhead our AI initiatives within DataHub, focusing on intelligent metadata management and shaping our AI infrastructure strategy. This role combines hands-on technical leadership in implementing AI-powered features with strategic thinking about how enterprises deploy and manage AI systems at scale. You'll work at the intersection of data catalog systems and modern AI infrastructure, helping organizations navigate the complexities of enterprise AI deployment while ensuring robust governance and efficiency.

Key Responsibilities

AI Features & Implementation

• Lead the technical implementation of AI-powered features in DataHub, including automated data classification, PII detection, and sensitive data identification
• Architect and implement scalable ML pipelines for continuous learning and model updates
• Design and implement systems for model monitoring, validation, and performance tracking
• Guide the team in implementing privacy-preserving ML techniques and ensuring compliance with data protection standards

AI Infrastructure Strategy

• Shape the metadata framework needed to support enterprise AI systems, including model cards, lineage tracking, and deployment metadata
• Define standards for capturing and managing AI-related metadata, including training data versioning, model provenance, and deployment configurations
• Design systems to track and manage AI assets across the development lifecycle
• Develop best practices for AI observability and governance in enterprise settings

Technical Leadership

• Lead architectural decisions for AI systems integration within DataHub
• Mentor team members on ML engineering best practices and AI system design
• Collaborate with product management to define AI feature roadmap
• Work with customers to understand their AI infrastructure needs and challenges

Required Qualifications

• 8+ years of software engineering experience, with at least 4 years focused on ML/AI systems
• Strong experience with modern ML frameworks (PyTorch, TensorFlow) and MLOps tools
• Deep understanding of LLM deployment, fine-tuning, and operational considerations
• Experience with AI governance, including model monitoring, bias detection, and fairness metrics
• Strong background in data privacy and security, particularly in AI contexts
• Experience with enterprise AI deployment and infrastructure management
• Proficiency in Python and modern AI development tools
• Understanding of vector databases, embedding systems, and semantic search
• Experience with distributed systems and scalable architecture

Preferred Qualifications

• Experience working with DataHub is a huge plus!
• Experience building AI-powered features in enterprise SaaS products
• Background in data catalog or metadata management systems
• Familiarity with AI governance frameworks and standards
• Experience with AI infrastructure cost optimization
• Knowledge of regulatory requirements around AI systems
• Track record of building production ML systems

Essential Knowledge Areas

Deep understanding of enterprise AI infrastructure components

• Model serving platforms
• Vector databases
• Training infrastructure
• Feature stores
• Model monitoring systems
• AI governance tools

Familiarity with key considerations for enterprise AI deployment

• Cost optimization strategies
• Security requirements
• Compliance considerations
• Performance monitoring
• Resource management
• Model versioning and rollback strategies

If you're passionate about technology, enjoy working with customers, and want to be part of a fast-growing company changing the industry, we want to hear from you!

This is a hybrid role with the expectation the employee will travel to the office a few times a week during the first few months of employment, and will continue to come to the office in Palo Alto on a regular basis.