Drive the end-to-end lifecycle of data annotation programs while ensuring quality at scale and managing workflows among teams and vendors.
Data Science at TRACTIAN
The Data Science team at TRACTIAN focuses on extracting valuable insights from vast amounts of industrial data. Using advanced statistical methods, algorithms, and data visualization techniques, this team transforms raw data into actionable intelligence that drives decision-making across engineering, product development, and operational strategies. The team constantly works on optimizing prediction models, identifying trends, and providing data-driven solutions that directly enhance the company’s operational efficiency and the quality of its products.
What you'll do
We are looking for a strategic Data Operations Lead to own the end-to-end lifecycle of our data annotation programs. You will not just manage tasks; you will build the "Ground Truth" engine that powers our AI. This role is a hybrid of Project Management and Quality Assurance, requiring you to orchestrate workflows between internal teams, external vendors, and automated tools. You will be responsible for defining the strategy, ensuring quality at scale, and delivering datasets that meet strict Service Level Agreements (SLAs).
Responsibilities
End-to-End Ownership: Define program objectives, timelines, and deliverables for multiple data labeling projects simultaneously.
Workflow Design: Create scalable SOPs (Standard Operating Procedures) and guidelines. You will decide when to use a "Consensus" model (multiple labelers per item) versus a "Single-Pass" model based on cost/quality trade-offs.
Risk Mitigation: Proactively identify bottlenecks (e.g., ambiguity in guidelines, tool downtime) and implement "Risk Mitigation Strategies" before they impact model training schedules.
Crowd/Team Oversight: Recruit, train, and manage a distributed team of annotators. Monitor "Throughput" (items/hour) and "Efficiency" to ensure productivity targets are met.
Vendor Relations: Act as the primary interface for external data vendors. Negotiate timelines, track budget utilization, and hold vendors accountable to accuracy SLAs (e.g., 98% quality on Gold Sets).
Performance Coaching: Implement data-driven feedback loops. If an annotator's quality drops, you will analyze their errors and provide targeted retraining materials.
Gold Set Management: Maintain a "Gold Set" (master answer key) to blindly test annotators.
Metric Analysis: Track and report on key quality metrics: Inter-Annotator Agreement (IAA), Accuracy, and Precision/Recall of the human labels.
Root Cause Analysis: When model performance dips, you will investigate the training data to determine if the issue stems from "Labeler Bias," "Guideline Drift," or "Edge Case Ambiguity."
Platform Operations: Help set up the UI/UX and configurations for the internal platforms used for labeling and annotation.
Reporting: Generate weekly executive dashboards using Excel/Google Sheets (Pivot Tables, VLOOKUP) to visualize "Spend vs. Output" and "Quality Trends" for stakeholders.
Requirements
Experience: 3+ years in Data Operations, Program Management, or QA for Machine Learning/AI.
Technical Literacy: Familiarity with the AI lifecycle (Training vs. Validation vs. Test sets).
Operational Rigor: Experience writing technical documentation/guidelines that leave no room for interpretation.
Data Skills: Advanced proficiency in Excel/Google Sheets. (SQL experience is a strong plus).
Please mention you found this job on AI Jobs. It helps us get more startups to hire on our site. Thanks and good luck!
Ace your job interview
Understand the required skills and qualifications, anticipate the questions you may be asked, and study well-prepared answers using our sample responses.
