At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making. As a Senior Data Engineer, you will hold a crucial position within our dynamic team, actively contributing to thrilling projects that reshape data analytics for our clients, providing them with a competitive advantage in their respective industries.
Responsibilities
- 6+ Years of relevant industry experience in Data Engineering working with large scale data driven systems.
- Experience designing data schemas and fine-tuning SQL queries around large, complex data sets. (Expert)
- Experience with AWS
- Design, build and maintain data infrastructure that powers both batch and realtime processing of billions of records a day.
- Improve the data quality and reliability of data pipelines through monitoring, validation and failure detection.
- Design, build and maintain a central data cataloging system to ease integration and discovery of datasets.
- Develop data pipelines that provide fast, optimized, and robust end-to-end solutions.
- Automate manual processes and create a platform in favor of self-service data consumption.
- Deploy and configure components to production environments.
- Participate in an on-call schedule to provide emergency incident support.
- Mentor and train teammates on design and operation of data platforms.
- Stay current with industry trends, making recommendations as needed to help the company excel.
