Sr. Applied AI Scientist

About Us HighLevel is an AI powered, all-in-one white-label sales & marketing platform that empowers agencies, entrepreneurs, and businesses to elevate their digital presence and drive growth. We are proud to support a global and growing community of over 2 million businesses, comprised of agencies, consultants, and businesses of all sizes and industries. HighLevel empowers users  with all the tools needed to capture, nurture, and close new leads into repeat customers. As of mid 2025, HighLevel processes over 4 billion API hits and handles more than 2.5 billion message events every day. Our platform manages over 470 terabytes of data distributed across five databases, operates with a network of over 250 microservices, and supports over 1 million hostnames. Our People With over 1,500 team members across 15+ countries, we operate in a global, remote-first environment. We are building more than software; we are building a global community rooted in creativity, collaboration, and impact. We take pride in cultivating a culture where innovation thrives, ideas are celebrated, and people come first, no matter where they call home. Our Impact As of mid 2025, our platform powers over 1.5 billion messages, helps generate over 200 million leads, and facilitates over 20 million conversations for the more than 2 million businesses we serve each month. Behind those numbers are real people growing their companies, connecting with customers, and making their mark - and we get to help make that happen. Why Join HighLevel?  - Collaborative Culture: A passionate and driven team. - Ownership and Responsibility: It’s never somebody else’s job. - User-Centric: Focused on adding value for our users. - Agility: Fast delivery using lean principles, going to market in weeks instead of quarters. - Innovation: Good ideas always get tested. - Team Care: We take care of our team so they can take care of our users. - Continuous Improvement: Embracing constant and iterative improvement. - Learning Opportunity: Learn how to scale a B2B SaaS startup and build impactful products. About the Team The Workflows Module is the key component of HighLevel’s automation engine, enabling agencies and businesses to create powerful, automated workflows that enhance efficiency, save time, and improve outcomes. By integrating with multiple product areas within the HighLevel platform, the Workflows Module empowers users to streamline processes, nurture leads, and manage tasks seamlessly. As part of the team, you’ll work on improving existing capabilities within workflows, conceptualizing new workflow features, and delivering solutions that drive adoption, activation, and business growth. The team is committed to optimizing user experience, increasing efficiency, and helping customers get the most value out of HighLevel. About the Role We are seeking a Data Scientist to drive data-driven decision making for our Generative AI products. In this role, you will transform thousands of daily AI interaction traces into actionable insights, build evaluation frameworks for AI quality assurance, and leverage billions of workflow executions to identify high-potential use cases. You'll work closely with product and engineering teams to turn complex data into strategic recommendations that shape our AI roadmap. Responsibilities: - Data Preparation - Prepare and maintain datasets for AI product evaluation, create annotation pipelines, build ground truth datasets, and support fine-tuning efforts with curated training examples - AI Model Evaluation - Test agent performance across model versions and prompt variations, build automated evaluation pipelines, and generate periodic reports that inform model selection and prompt engineering - Product Intelligence - Analyze user request patterns to identify high-demand use cases and capability gaps, cluster and classify large volumes of traces to find common themes and edge cases - User Analytics - Identify behaviors differentiating power users from casual users, analyze which user segments or industries drive highest engagement, and build statistical models to understand success vs failure drivers - Workflow Analytics - Optimize the first-time user funnel, correlate workflow usage patterns with customer retention, and identify power user behaviors across all workflows - Cost Optimization - Track quality metrics over time and alert on degradation, analyze token usage to identify cheaper model opportunities, and monitor latency across the agent pipeline (RAG, embeddings, generation) - Experimentation - Design and analyze statistically rigorous A/B tests for prompts and features, test new agent capabilities, and measure feature impact on user behavior and success rates Requirements: - 5+ years of experience in data science, analytics, or related role with demonstrated ownership of end-to-end analytics projects - Strong Python and SQL skills for working with large datasets at scale - Deep expertise in statistical analysis, experimental design, and A/B testing methodologies - Experience building data pipelines, automation scripts, and evaluation frameworks - Hands-on experience with LLM applications, prompt engineering, or AI products - Exceptional attention to detail with ability to identify edge cases and patterns in complex data - Strong communication skills - able to explain complex findings simply to stakeholders - Bachelor's degree or equivalent in Computer Science, Data Science, Statistics, Engineering, or related field Nice to Have: - Familiarity with AI observability tools (Langfuse, LangSmith, Weights & Biases) or ML monitoring platforms - Experience with RAG pipelines, embedding models, and LLM evaluation frameworks - B2B SaaS or product analytics background - Track record of working with workflow automation or marketing automation platforms Why Should You Join Us? At HighLevel, we foster an exciting and dynamic work environment driven by a passionate team. We believe in a collective responsibility where no task is considered someone else's job. Our unwavering focus is on providing value to our users, and we achieve this by delivering solutions swiftly through lean principles, allowing us to bring products to market in a matter of weeks rather than quarters. Every good idea is put to the test, ensuring that we maintain a high standard of innovation. We prioritise the well-being of our team, recognizing that by taking care of them, they can better serve our users. We embrace the concept of continuous and iterative improvement, understanding that progress is an ongoing journey. We are also a well funded & profitable company.  Join us at High Level, and you will have the opportunity to learn the intricacies of scaling a B2B SaaS startup and develop impactful products that cater to the needs of our customers.