Position: Lead Data Engineer
Location: Gurgaon (Hybrid), Bangalore (Hybrid)
Accepted Notice Period: Immediate – 30 days
Years of Experience: 6 – 8 Years
Opportunity Type: Full time
**Note: This position is with one of our hiring partners
Skills Required
- Extensive experience with Hadoop ecosystem technologies such as Hadoop, Hive, and Pyspark/Spark
- Proficiency in programming languages like Python, Java, and Unix scripting for data engineering tasks
- Strong understanding of data engineering principles, best practices, and design patterns
- Hands-on experience in designing, building, and optimizing large-scale data pipelines and ETL processes
- Ability to lead and mentor a team of data engineers, providing technical guidance and expertise
- Knowledge of cloud platforms like AWS/GCP/Azure is highly desirable
- Excellent problem-solving skills and ability to troubleshoot complex data issues
- Strong communication and collaboration skills, with the ability to work effectively in cross-functional teams
Responsibilities
- Lead the design, development, and implementation of data solutions using Hadoop, Hive, and Spark
- Architect and build scalable and reliable data pipelines to support various data processing and analytics requirements
- Provide technical leadership and mentorship to the data engineering team, ensuring best practices and standards are followed
- Collaborate with stakeholders to gather and analyze requirements, and translate them into technical specifications
- Optimize and tune data workflows for performance, scalability, and efficiency
- Ensure data quality, integrity, and security across all data processes and systems
- Stay updated with the latest trends and technologies in the field of data engineering, and drive continuous improvement initiatives
- Drive innovation and contribute to the development of new data products and services
About Pesto Tech:
Pesto is where software developers go to build their career path for the next 5 years. We don't just offer jobs; we provide unparalleled opportunities for your growth and success in the dynamic landscape of Tech Jobs.
