Staff Engineer - Workflows Engine

About HighLevel: HighLevel is an AI powered, all-in-one white-label sales & marketing platform that empowers agencies, entrepreneurs, and businesses to elevate their digital presence and drive growth. We are proud to support a global and growing community of over 2 million businesses, comprised of agencies, consultants, and businesses of all sizes and industries. HighLevel empowers users  with all the tools needed to capture, nurture, and close new leads into repeat customers. As of mid 2025, HighLevel processes over 4 billion API hits and handles more than 2.5 billion message events every day. Our platform manages over 470 terabytes of data distributed across five databases, operates with a network of over 250 micro-services, and supports over 1 million hostnames. Our People With over 1,500 team members across 15+ countries, we operate in a global, remote-first environment. We are building more than software; we are building a global community rooted in creativity, collaboration, and impact. We take pride in cultivating a culture where innovation thrives, ideas are celebrated, and people come first, no matter where they call home. Our Impact As of mid 2025, our platform powers over 1.5 billion messages, helps generate over 200 million leads, and facilitates over 20 million conversations for the more than 2 million businesses we serve each month. Behind those numbers are real people growing their companies, connecting with customers, and making their mark - and we get to help make that happen. Role Summary: We’re hiring a Staff Engineer – Workflows (Engine) to own and evolve the orchestration layer that powers automation at scale within HighLevel. You’ll rebuild the engine that drives 80M+ workflow enrolments per day and nearly 1 Billion action executions per day at peak load — with reliability, performance, and clarity at its core. You’ll operate as a hands-on individual contributor, designing and writing core engine logic in Go and Node.js, while defining contracts, reliability guarantees, and scaling patterns. This is a Staff Architect track role — deep technical ownership, not management. You’ll have decision authority on engine architecture and are expected to set the gold standard for how distributed systems are built at HighLevel’s automation team. Team & Project Overview: You’ll join the Workflows (Engine) team — a small, autonomous group of top-tier engineers within the Workflows organization. The team’s mission is to build the next-generation orchestration engine that powers all automations at HighLevel: reliable, testable, and ready for future expansion. This engine handles concurrency, ordering, retries, and scaling under unpredictable load. Your rewrite will make the system more robust, easier to extend with new actions and context types, and dramatically simpler to maintain. Ultimately, your work will enable faster feature delivery across the entire automation platform. Day to day, you’ll work closely with the Engineering Manager (Workflows) and a Lead Engineer from Workflow Core — operating as the founding engineer for the new Engine org. Responsibilities:

Re-architecture: Rebuild the Workflow Engine from Node.js to Go, creating a modular, high-performance foundation for billions of executions

Core abstractions: Design orchestration, state, retries, and execution guarantees with clear contracts and isolation boundaries

Performance model: Optimise for throughput-first execution while maintaining strict ordering within each workflow execution context

APIs & contracts: Define interfaces and schemas between Engine, Triggers, and Actions. Ensure consistent, reliable, and versioned communication

Reliability & observability: Partner with SRE to instrument metrics (latency, throughput, failure rate) and build replay and diagnostics tooling

Operational ownership: Own the engine’s runtime — incidents, RCA, and prevention. Deliver measurable reliability improvements (<1% failures/day)

Migration & rollout: Drive dual-run migration with progressive rollout and auto-rollback safety

Engineering culture: Set the technical benchmark for clarity, testability, and performance within Workflows and beyond

Requirements:

10+ years of backend engineering experience with deep hands-on work in distributed systems, job schedulers, or orchestration engines

Advanced proficiency in Go (preferred) and Node.js, with experience writing low-latency, high-throughput microservices

Strong understanding of testability and isolation principles; you design systems that are easy to test, reason about, and extend

Production-grade database experience (MongoDB, Firestore, or equivalent) with sound data modelling

Cloud experience (GCP, AWS, or Azure), especially event-driven services like Pub/Sub, SQS, or Cloud Tasks

Proven record of measurable performance wins — reduced p95/p99 latency, improved throughput, or increased reliability

Strong fundamentals in concurrency, idempotency, ordering guarantees, and fault tolerance

Pragmatic engineering mindset — simplicity and clarity over abstraction for abstraction’s sake

Strong applied understanding of design patterns and system architecture principles — able to model orchestration, state, and retries using proven, scalable patterns

Good to have:

Experience with automation systems (e.g., n8n, Zapier-like architectures) and the ability to generalize such models

Observability experience (Grafana, Prometheus, OpenTelemetry, Kibana)

Strong debugging and performance profiling skills

Open-source contributions or technical writing in distributed systems

Experience helping define engineering best practices or building early-stage teams