🌟 Accomplished Tech Visionary:
Embark on an exciting journey into the realm of software development with 3Pillar! We extend an invitation for you to join our team and gear up for a thrilling adventure. At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making. As a Senior Data Engineer, you will hold a crucial position within our dynamic team, actively contributing to thrilling projects that reshape data analytics for our clients, providing them with a competitive advantage in their respective industries.
If your passion for data analytics solutions that make a real-world impact, consider this your pass to the captivating world of Data Science and Engineering! 🔮🌐
Key Responsibilities:
- Understanding the business requirements.
- Writing code and implementing the proposed solutions.
- Creating data pipelines, versioning and change management.
- Manage the complexity inherent in data pipelines.
- Develop ETL/ELT processes to help extract and manipulate data from multiple sources.
-
Design, build and maintain batch or data pipelines in production.
-
Automate data workflows such as data ingestion, aggregation, and ETL processing.
-
Logging and instrumentation of pipelines and services.
- Ensure data accuracy, integrity, privacy, security, and compliance through quality control procedures.
Minimum Qualifications:
- Demonstrated expertise with a minimum of 5+ years of experience as data engineer or similar role.
-
Advanced SQL skills and experience with relational databases and database design.
- Experience working with cloud Data Warehouse solutions (e.g., Snowflake, DataBricks, Redshift, BigQuery, Azure Synapse, etc.).
-
Python skills with hands-on experience
- Experience with Big Data technologies like Spark, Map Reduce, Hadoop, Hive etc.
- Proficiencient in data pipeline and workflow management tools e.g. Airflow
- Experience with data visualization tools like PowerBI, Tableau, AWS QuickSight etc.
- Experience with the AWS/Azure/GCP data engineering services
- Knowledge of AWS services viz., DMS, S3, Lambda, EMR, GLUE ETL, Athena, RDS, Redshift, EC2, IAM (or associate Azure services)
- Very good exposure of working on Data Lakes & Data Warehouses solutions
- Excellent problem-solving, communication, and organizational skills.
- Proven ability to workindependently and with a team.
- Ability to guide other data engineers.
What is it like working for 3Pillar Global?