Data Engineer

WHAT IS THE POSTION

As a Data Engineer, you will play a key role on our Technology team, a collaborative group dedicated to solving Edmentum’s big challenges. Your work will encompass development and operations for our data lake, the core architecture powering Data Analytics and Research. You’ll be directly involved in the design, development, and deployment of scalable data infrastructure, pipelines, and storage solutions. Using cutting-edge cloud tools like AWS and Databricks, you’ll ingest and process both streaming and batch data within a purpose-built architecture. If you’re passionate about data, join us to bring Edmentum’s strategic vision to life through efficient, impactful data pipelines.

In this role, you’ll create and manage data pipelines to ingest, transform, and maintain large-scale datasets from diverse sources, both internal and external. Working with billions of interactions from millions of students and educators across complex, multi-layered customer organizations, you’ll develop high-performance systems optimized for data retrieval and in-line aggregation, supporting Edmentum’s rich, interactive product experiences. As you grow within this role, you’ll gain deep domain expertise within Edmentum’s ecosystem, developing insights into the nuanced relationships between data sources and optimizing them for usability across various teams and solutions.

The Data Engineer will contribute to shaping the future of education by building scalable data infrastructure that powers Edmentum’s innovative learning solutions, reaching educators and learners worldwide. As part of a high-performing team, you’ll work in a fast-paced, collaborative environment where continuous learning, knowledge-sharing, and innovation are core values. We celebrate diversity of thought and maintain an openness to new ideas as we support educational success for students everywhere.

WHAT YOU WILL DO

• Design, develop, and implement scalable data pipelines to collect, connect, centralize, and curate data from diverse internal and external sources, ensuring seamless data flow throughout the ecosystem.
• Collaborate with a high-performing development team to build best-in-class data pipelines, contributing to key data initiatives that drive the data lake infrastructure in AWS.
• Create purpose-built data architecture solutions for data users, such as data marts, third-party integrations, and dimensional models, optimizing for efficient, flexible data consumption across the organization.
• Develop and maintain APIs, microservices, and related libraries that support core capabilities within our technology ecosystem, enabling targeted functionality and robust data interactions.
• Engage closely with data-focused colleagues in Application Development, Business Intelligence, Data Science, and Research to align data solutions with broader organizational goals and drive impactful data initiatives.

WHAT IS REQUIRED

Education and Experience

• Bachelor’s degree in Computer Science, Information Management Systems, or related field; or equivalent work experience.
• Programming Skills: 3+ years with Python and SQL, with 2+ years specifically using Spark (PySpark DataFrames).
• Data Engineering Experience:
  • 2+ years with relational databases, such as SQL Server or Postgres.
  • 1+ years with non-relational data lakes or warehouses (e.g., BigQuery, Snowflake, Databricks Delta Lake, or Redshift).
• Data Processing Tools:
  • 1+ years with batch ETL tools (Databricks ETL preferred).
  • 1+ years with streaming data tools (e.g., Kinesis, AWS Lambda).
  • 1+ years managing complex data workloads with job administration and scheduling tools, such as Apache Airflow.

Technical Skills

• Software Engineering: Experience with Gitflow in multi-environment team deployments.
• Cloud Expertise: Proficiency with AWS data tools (preferred) including EMR, S3, EC2, Kinesis, and ECS.
• Databricks Tools: Familiarity with the Databricks platform, including Delta Lake and Unity Catalog.
• Proficiency in an additional programming language (e.g., C#, Scala, Java, Node.js).
• Foundational Knowledge: Understanding of Data Lake architecture patterns, cloud infrastructure, operations, and the software development lifecycle.

Soft Skills and Competencies

• Problem-Solving Skills: Strong critical-thinking abilities, capable of developing elegant solutions to complex data challenges.
• Communication Skills: Able to effectively articulate key considerations of complex technical topics, both verbally and in writing.
• Business Insight: Ability to understand business problems deeply and design data solutions that effectively address them.
• Ownership and Accountability: Demonstrated pride of ownership, ensuring high-quality solutions throughout their lifecycle.
• Team Collaboration and Ethics: Commitment to inclusion, integrity, cross-functional teamwork, and alignment with Edmentum’s mission and values.

Continuous Learning

• Maintain awareness of industry trends, market developments, and competitive landscape.

WHY JOIN EDMENTUM

• Competitive compensation package and best in class Total Rewards offerings.
• Opportunity to lead and shape the revenue generation strategy of a dynamic company.
• Collaborative and inclusive Remote First work environment
• Company culture that values innovation, growth, and impact.
• Commitment to employee development and career advancement.