Data Engineer

AI overview

Design and maintain scalable data pipelines using AWS, ensuring high performance and data integrity while collaborating with data analysts and stakeholders.

Job Summary:

We are looking for a skilled Data Engineer with strong hands-on experience in AWS to design, build, and maintain scalable data pipelines and cloud-based data platforms. The ideal candidate will have expertise in modern data warehousing, ETL/ELT development, and distributed data processing while ensuring data quality, performance, and security.

Key Responsibilities:

· Design, develop, and maintain scalable data pipelines using AWS services.

· Build and optimize ETL/ELT workflows for structured and unstructured data.

· Implement data lakes and data warehouses on AWS.

· Work with large datasets to ensure high performance, reliability, and data integrity.

· Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements.

· Perform data modeling for analytical and operational use cases.

· Ensure data quality, governance, and security best practices.

· Monitor and troubleshoot data workflows and production issues.

· Support CI/CD and automation for data platform deployments.

Required Skills & Experience:

· Strong hands-on experience with AWS services such as S3, Glue, Redshift, Lambda, EMR, Athena, and RDS.

· Proficiency in Python and SQL for data processing and analysis.

· Experience in building and optimizing ETL/ELT pipelines.

· Solid understanding of data warehousing and data lake architecture.

· Experience with Apache Spark / PySpark.

· Knowledge of workflow orchestration tools (Airflow or similar).

· Familiarity with streaming frameworks (Kinesis/Kafka) is a plus.

· Experience with data modeling and performance tuning.

· Understanding of DevOps, CI/CD, and infrastructure as code (Terraform/CloudFormation).

· Experience working in Agile/Scrum environments.

Good to Have:

· Experience with Snowflake on AWS.

· Exposure to real-time data processing.

· AWS certification (e.g., AWS Certified Data Analytics / Solutions Architect).

Soft Skills:

· Strong problem-solving and analytical skills.

· Excellent communication and stakeholder management.

· Ability to work in a fast-paced, collaborative environment.