Senior/Staff Applied Scientist
TLDR
Collaborate closely with product data science and engineering to enhance A/B testing frameworks, directly impacting decision-making and product optimization.
- Collaborate with product data science and engineering teams to identify techniques, tooling and process improvements in online A/B experimentation to assist rigorous decision making across all relevant product domains.
- Develop and maintain our A/B experimentation platform based on stakeholder feedback
- Write code or identify vendors to deploy these techniques to production in a scalable manner that’s easy to use by engineering, product data science, product management and design teams
- Conduct end to end evaluation of various hero use cases like document/URL uploads, including evaluation set generation, coming up with evaluation criteria and methods to interpret the results.
- Break down end to end evaluations into more granular evaluation of various tasks & skills including but not limited to content summarization/analysis/generation, multi-step reasoning & strategizing, tool selection & use, coding & system design.
- Design, develop and own best practices, tools and processes across various evaluation problems, e.g. query intent classification, standardizing the use of best statistical principles to handle LLM stochasticity, industry benchmarking.
- You have 5+ years of experience as a Masters degree holder, 3+ as a PhD degree holders (Masters/PhD degree in Statistics, Mathematics or Computer Science, or another quantitative field)
- You’re strong in statistics and/or machine learning. You have experience in applying these skills into tangible improvements in products, internal tools, and processes in a pragmatic way that puts business urgencies first.
- You are very proficient in Python, e.g. proficient enough to maintain an internal source-controlled library used by dozens of others.
- You are concise and precise in written and verbal communication. Technical documentation is your strong suit.
- You are proficient in SQL and the modern data stack (e.g. source-controlled dbt pipelines for ETL/ELT).
- You are strong at defining good product KPIs/guardrail metrics, dashboarding and analysis of raw data to derive strategic insights.
- You have experience in B2B SaaS.
- You have experience working on ranking, developing, and maintaining A/B experimentation platforms and/or ML measurement problems.
- You are passionate about using AI to improve the productivity of data teams as well as non-data professionals trying to derive more value of their company’s data.
- This role is hybrid (3-4 days a week in one of our SF Bay Area offices)
AI-First Mindset at Glean:
At Glean, AI fluency is core to how we work and we're committed to ensuring every new hire feels confident integrating AI into their everyday work. As part of the interview process, you'll complete a brief AI-focused exercise or discussion so we can understand how you think about, design, and use AI to drive impact in your role. Feel free to reference any tools, platforms, or workflows you use today — prior Glean experience isn't required.
Benefits
Education Stipend
an annual education and wellness stipends to support your growth and wellbeing.
Health Insurance
Medical, Vision, and Dental coverage
Home Office Stipend
When you join, you'll receive a home office improvement stipend.
Paid Time Off
generous time-off policy
Remote-Friendly
This role is hybrid (3-4 days a week in one of our SF Bay Area offices)
Glean is a Work AI platform designed to help organizations optimize their operations through intelligent search and AI-driven capabilities. By offering a scalable and secure infrastructure, Glean empowers businesses across various industries to harness the full potential of AI while maintaining control and customization.
- Founded
- Founded 2019
- Employees
- 51-200 employees
- Total raised
- $160M raised