Lead Cloud Data Engineer

TLDR

Design and lead scalable cloud-based data solutions, combining hands-on engineering with leadership responsibilities while guiding a team of data engineers.

Job Title: Lead Cloud Data Engineer
Locations: Fort Mill, SC or New York City, NY (Hybrid/Onsite)
Type: Full Time
Job Summary
We are looking for a highly experienced Data Engineer – AWS Tech Lead to design and lead scalable cloud-based data solutions. This role will combine hands-on engineering with leadership responsibilities, guiding a team of data engineers while collaborating with architects, product owners, and business stakeholders. The ideal candidate will have deep expertise in AWS data services, ETL/ELT pipelines, and big data frameworks, with proven experience in leading technical delivery.
 
Key Responsibilities
  • Lead the design, development, and deployment of large-scale data pipelines and solutions on AWS.
  • Provide technical leadership and mentorship to a team of data engineers, ensuring best practices in coding, architecture, and performance optimization.
  • Architect data lake and data warehouse solutions leveraging AWS Glue, Redshift, S3, Athena, EMR, and Lambda.
  • Drive development of ETL/ELT processes using PySpark, Python, and SQL.
  • Implement and enforce data governance, quality checks, and security standards across the platform.
  • Collaborate with business analysts, data scientists, and product teams to translate business requirements into scalable solutions.
  • Optimize cost and performance of AWS-based data workloads.
  • Troubleshoot production issues and ensure high availability of data pipelines.
  • Participate in sprint planning, backlog refinement, and Agile ceremonies as a technical lead.
Required Qualifications
  • 13+ years of experience in Data Engineering, with at least 4+ years in a technical leadership role.
  • Strong expertise in AWS data services: Glue, S3, Redshift, Athena, Lambda, Step Functions, EMR.
  • Proficiency in PySpark, Python, and SQL for large-scale data processing.
  • Solid experience in data modeling (star, snowflake, dimensional modeling) and data lake/warehouse architectures.
  • Proven track record in leading and mentoring data engineering teams.
  • Strong understanding of Agile methodologies and CI/CD practices for data pipelines.
 
Preferred Qualifications
  • Experience in BFSI / Wealth Management domain.
  • AWS Certification (Data Analytics Specialty, Solutions Architect, or Big Data).
  • Familiarity with containerization (Docker, Kubernetes) and orchestration frameworks.
 
Education
  • Bachelor’s degree in Computer Science, Information Technology, Engineering, or related field.
  • Master’s degree preferred.

Qode is a technology-driven platform that transforms how recruiters and candidates connect by leveraging data and automation. Our solutions streamline the hiring process through machine learning, creating private talent pools and automating workflows, ultimately enhancing the quality of candidate evaluation and decision-making. With our no-code tools, we empower organizations to develop tailored recruitment strategies without needing extensive technical skills.

View all jobs
Ace your job interview

Understand the required skills and qualifications, anticipate the questions you may be asked, and study well-prepared answers using our sample responses.

Cloud Data Engineer Q&A's
Report this job
Apply for this job