Our client:
WebPros, the largest web hosting software and automation company, manages 900,000+ servers, 85 million domains, and 33 million users. WebPros unites top providers in web hosting, billing automation, infrastructure, server management, and online marketing software. Currently, their lineup includes cPanel, Plesk, SolusVM, WHMCS, XOVI NOW, Sitejet, 360 Monitoring, and koality, with ongoing additions.
About the Project:
WebInsights is a domain and website data analysis platform. We perform high-performance web crawling, analyze markets, and deliver analytical reports for businesses.
Technology Stack:
-
Backend: Go
-
Data: Amazon Redshift, Google BigQuery, AWS S3
-
Infrastructure: Kafka, Dragonfly, Docker
-
DevOps: AWS (ECS Fargate/EC2), Terraform, Jenkins
-
Monitoring: Prometheus, Grafana
What You'll Be Doing:
- Design and optimize high-load crawling and data processing services
- Build data processing and transformation pipelines
- Optimize queries and workflows for analytical data warehouses (Redshift, BigQuery)
- Set up inter-service communication via Kafka
- Participate in architectural decisions and code reviews
- Configure monitoring and alerting (Prometheus, Grafana)
Ideal Candidate Experience & Expertise:
- 4+ years of backend development in Go (expert level)
- Experience with high-load data processing systems
- Strong understanding of Go concurrency: goroutines, channels, sync primitives
- Experience with performance optimization and profiling
Data & Distributed Systems:
- Hands-on experience with columnar databases (Redshift, BigQuery) or similar
- Understanding of event-driven architecture and message brokers (Kafka)
- Experience working with large data volumes and batch processing
- Understanding of trade-offs between consistency and availability
Technology Stack:
- Dragonfly/Redis: caching, rate limiting
- Docker & Terraform: container orchestration, IaC
- AWS: ECS, EC2, S3, IAM, VPC
DevOps & Infrastructure:
- CI/CD pipelines (Jenkins)
- Infrastructure as Code (Terraform)
- Experience with containerization and deployment to AWS ECS
Monitoring & Debugging:
- Prometheus + Grafana: metrics, dashboards, alerting
- Go application profiling: pprof, flame graphs
- Production incident analysis and resolution
Soft Skills:
- Ability to explain complex technical concepts clearly
- Systems thinking and proactive problem-solving
- Balance between code quality and pragmatism
- Experience working in cross-functional teams
Nice to Have:
- Experience building web crawlers and parsers
- Deep knowledge of HTTP protocol
- Experience with rate limiting and anti-bot systems
- Experience optimizing SQL queries in analytical databases
What We Offer:
- Work on a product with real business impact
- Influence on architectural decisions and technology choices
- Work with large-scale data and high-load systems
- Culture of code quality and engineering excellence
- Flexible work arrangements and remote work options
- Professional growth and mentorship
