Middle Data Engineer

We are looking for a Middle Data Engineer to join our team. If you are passionate about technology and constantly seeking to improve your skill set, let’s meet and discuss!

Work at Exadel - Who We Are

We don’t just follow trends—we help define them. For 25+ years, Exadel has transformed global enterprises. Now, we’re leading the charge in AI-driven solutions that scale with impact. And it’s our people who make it happen—driven, collaborative, and always learning.

About Our Customer

The leading provider of vehicle lifecycle solutions, with headquarters in Chicago, enables the companies that build, insure, and replace vehicles to power the next generation of transportation. Its platform delivers advanced mobile, artificial intelligence, and car technologies. It connects a network of 350+ insurance companies, 24,000+ repair facilities, hundreds of parts suppliers, and dozens of third-party data and service providers. The customer's collective solutions enhance productivity and help clients deliver better experiences for end consumers.

Project Tech Stack

Python, Spark, Hive, Spark, Kafka, AWS (Amazon EMR, S3, Presto), Airflow, SQL, Unix, Hadoop 

Requirements

• Skills in programming with Python, Spark
• Proficiency in open-source big data tools (Hive, Spark, Kafka)
• Background with AWS eco-system (Amazon EMR, S3, Presto), Airflow for scheduling and monitoring of big data ETL pipelines, SQL for data profiling and data validation
• Knowledge of Unix commands and scripting
• Experience with Hadoop fundamentals and architecture (HDFS, map-reduce, job performance)

English level

Upper-Intermediate+

Responsibilities

• Build end-to-end data pipelines from sources to fully curate and enhance data sets
• Locate and analyze source data: create data flows to extract, profile, and store ingested data; define and build data cleansing and imputation; map to a common data model; transform to satisfy business rules and statistical computations, and validate data content
• Produce software data building blocks, data models, and data flows for varying client demands, such as dimensional data, data feeds, dashboard reporting, and data science research and exploration
• Assist with producing automated software tests of data flow components and automate business-specific tests for data content quality
• Help automate orchestration and error handling for use by production operation teams