The Data Engineer is responsible for managing, operating, and supporting cloud-based data platforms. This role involves close collaboration with customers and internal teams to ensure stable, secure, and efficient operations of the cloud data ecosystem.
The engineer will work across multiple technologies, including Azure Data Lake Storage, Databricks, Azure Data Factory, Azure Synapse Analytics, Snowflake, Azure Event Hub, Tidal, Pyramid, Collibra, Power BI, Tableau, MicroStrategy (MSTR), and Alteryx, ensuring reliable data ingestion, processing, analytics, and reporting.
Key Responsibilities (Level 2 Operational Support)
Resolve data pipeline issues and perform proactive monitoring for sensitive and critical batch processes.
Conduct Root Cause Analysis (RCA) and retrospectives for incidents and operational issues.
Create, update, and maintain operational runbooks and manage entitlements as required.
Implement configuration changes to batch processes and underlying cloud infrastructure.
Manually control and manage pipeline executions during scheduled maintenance windows.
Perform on-demand operational changes based on instructions provided by the Data Engineering (L3) team.
Handle user service requests; for non-runbook-based requests, coordinate effectively with the L3 team.
Participate in Disaster Recovery (DR) testing activities (twice a year).
Support development and production data synchronization activities.
Represent the team in weekly change management board meetings.
Knowledge, Skills, and Abilities
Strong hands-on experience with:
Azure Data Lake Storage
Azure Databricks
Azure Data Factory
Azure Synapse Analytics
Snowflake
Azure Event Hub
Tidal Scheduler
Pyramid, Collibra
Power BI, Tableau, MicroStrategy (MSTR), Alteryx
Ability to read, write, and troubleshoot SQL queries and stored procedures.
Proven experience in configuring, managing, and troubleshooting cloud data platforms.
Strong analytical and problem-solving skills.
Excellent written and verbal communication skills, with the ability to clearly explain technical concepts to non-technical stakeholders.
Ability to work effectively in small teams while promoting collaboration and inclusiveness.
Job Complexity
Requires extensive problem-solving capabilities, including independent investigation of issues and identification of root causes.
Ability to research and utilize multiple information sources, including Microsoft and vendor documentation, to resolve complex issues.
Must be capable of prioritizing multiple incidents and tasks while managing competing deadlines.
Experience and Education
Bachelor’s degree in Computer Science or a related field.
3–6 years of hands-on experience in configuring and managing Azure-based data analytics solutions.
Prior experience working in Azure cloud environments is strongly preferred.
Physical Demands and Work Conditions
General office work environment; no special physical demands required.
Schedule flexibility required, including regular weekend and holiday coverage to support 24×7 operations.
Occasional travel may be required (less than 10%).
