In this role, you will work with our Business Analytics & Insights (BAI) team, collaborating with cross-functional teams to deliver high-quality data solutions in areas such as Supply Chain, Finance, Operations, Customer Experience, HR, Risk Management, and Global IT.
Responsibilities:
- Lead the technical planning for data migration, including data ingestion, transformation, storage, and access control within Azure Data Factory and Azure Data Lake.
- Design and implement scalable, efficient data pipelines to ensure smooth data movement from multiple sources using Azure Databricks.
- Develop reusable frameworks for the ingestion of large datasets.
- Ensure data quality and integrity by implementing robust validation and cleansing mechanisms throughout the data pipeline.
- Work with event-based/streaming technologies to ingest and process data in real-time.
- Provide technical support to the team, resolving challenges during the migration and post-migration phases.
- Stay current with the latest advancements in cloud computing, data engineering, and analytics technologies; recommend best practices and industry standards for data lake solutions.
- 4+ years of IT experience.
- Minimum of 4 years working with Azure Databricks.
- Proficiency in Data Modeling and Source System Analysis.
- Strong knowledge of PySpark and SQL.
- Experience with Azure components: Data Factory, Data Lake, SQL Data Warehouse (DW), and Azure SQL.
- Experience with Python for data engineering purposes.
- Ability to conduct data profiling, cataloging, and mapping for technical design and construction of data flows.
- Familiarity with data visualization/exploration tools.
- Technical leadership skills with the ability to guide and mentor teams.
- Strong problem-solving capabilities, able to translate business requirements into data science solutions.
- Excellent communication skills, with the ability to explain complex concepts to both technical and non-technical stakeholders.
- Strong team player with excellent interpersonal and collaboration skills.
- Ability to deliver high-quality results within specified timelines.
