Data Engineer

Let's start with the role

At Kaizen, our commitment to the unparalleled value of data drives every decision we make. Expanding from our robust Lake House architecture, we envision a transformative shift towards a Data Mesh architecture. This approach emphasizes treating data as a product with well-defined contracts, distributed ownership, and domain-oriented design.

As we aim to further enhance our data ecosystem, we’re focusing keenly on data democratization, governance and observability. The ideal scenario? To have every piece of data perfectly accessible, monitored and governed.

Our data tribe is well-prepared to lead this transition, championing the design and expansion of next-generation Data Products. By joining our crew, you’ll be part of an initiative that positions Kaizen at the forefront of data-driven innovation.

Our Mission

Our Lake House infrastructure has been a foundational pillar, seamlessly supporting a diverse cohort from BI, Data Science and ML teams to our innovative product squads. As we navigate towards the next frontier characterized by Data Products and insights from a Data Mesh paradigm, we anticipate transformative outcomes. Central to our mission, we aim to enhance the robustness of our data pipelines, the agility of our data products and the precision of our solutions, all while upholding the highest standards of data governance.

In this new chapter, we are assembling cross-functional domain-aligned data teams, focused on delivering end-to-end data pipelines and scalable data products that simplify domain complexity for analytics and product teams. We are seeking data engineers proficient in DataOps, with a software engineering background and a product-oriented mindset. These engineers will play a pivotal role in developing efficient, scalable and high-quality data pipelines and products, leveraging tools and services provided by our data platform team to empower analytics and product innovation.

Our Tech Stack includes Spark, Python (or Scala), Databricks, Delta, Apache Airflow, Gitlab, Grafana and more

• Develop scalable, efficient and high-quality end-to-end data pipelines and data products, abstracting domain complexity for consumption by analytics and product teams.
• Collaborate closely with analytics engineers within your team to produce end-to-end results and smooth flow.
• Utilize orchestration, observability, data quality, data governance and CI/CD tools and services provided by our data platform team.
• Deploy and manage data products in alignment with the DataOps philosophy to ensure smooth and efficient operations.
• Engage in knowledge exchange with other data engineers in the data engineering chapter to stay at the forefront of high-quality data product development.
• Advocate for a documentation-centric culture, where technical documentation lives along with our products, ensuring they are user-friendly and maintainable.

What you’ll bring

• Spark Proficiency: Demonstrated expertise in Apache Spark, capable of writing and optimizing effective code for data processing tasks.
• Data Pipeline Expertise: Hands-on experience using data pipeline orchestration frameworks, ensuring seamless data flow and integration across different sources and platforms.
• End-to-End Data Solutions: Ability to design, plan, and deliver comprehensive data pipelines and products, from sourcing and ingestion to processing and serving.
• Data Quality Experience: Proficiency in testing the quality of datasets and integrating data observability into the ecosystem.
• Software Engineering: Software engineering knowledge, with skills in designing, writing, and deploying production-grade code that supports our data-first vision.
• Data Governance: Awareness of the importance of data governance, focusing on ensuring data assets are protected, accessible and compliant with regulatory standards.
• Technology Evaluation: A curious mindset, eager to understand, evaluate and potentially integrate new technologies to enhance our data-driven approach.