Data Engineer- Python, Spark, SQL - 6months Contract

Role: Data Engineer

Experience: 7-10years

Location: Mumbai Preferred, Open to PAN India

Job Summary:

Skills:

• Experience with programing in Python, Spark and SQL
• Prior experience in AWS services (Such as AWS Lambda, Glue, Step function, Cloud Formation, CDK)
• Knowledge of building bespoke ETL solutions
• Data modelling, and T-SQL for managing business data and reporting
• Capable of technical deep-dives into code and architecture
• Ability to design, build and manage data pipelines for data structures encompassing data transformation, data models, schemas, metadata and workload management
• Experience in working with data science teams in refining and optimizing data science and machine learning models and algorithms.
• Effective communication skills

Responsibilities:

• Build data pipelines: Architecting, creating and maintaining data pipelines and ETL processes in AWS
• Support and Transition: Support and optimize our current desktop data tool set and Excel analysis pipeline to a transformative Cloud based highly scalable architecture.
• Work in an agile environment: within a collaborative agile cross-functional product team using Scrum and Kanban
• Collaborate across departments: Work in close relationship with data science teams and with business (economists/data) analysts in refining their data requirements for various initiatives and data consumption requirements
• Educate and train: Required to train colleagues such as data scientists, analysts, and stakeholders in data pipelining and preparation techniques, which make it easier for them to integrate and consume the data they need for their own use cases
• Participate in ensuring compliance and governance during data use: To ensure that the data users and consumers use the data provisioned to them responsibly through data governance and compliance initiatives.
• Work within, and encourages a Devops culture and Continuous Delivery process