A Senior Data Engineer - Datalake is a problem solver who analyzes models, designs, creates, modifies, and supports a set of complex systems, processes, or operations that enable optimal business capabilities.
KEY DUTIES/RESPONSIBILITIES
- Creates Application and System Design Documents and Develops Applications / Reports / Systems / Enterprise Solutions
- Estimates component/application system level and Enterprise solution work efforts
- Creates RFI/RFP requests / responses for vendor product evaluations
- Designs, Develops, and Implements Complex Business Rules
- Responsible for fulfilling end-user requests. Provides on call support as required. Periodically provides guidance to and assists in training less experienced individuals. Routinely operates under deadlines and may be subject to extreme workloads
- Delivers personal tasks on time and leads the delivery of tasks for natural or cross-functional workgroup
- Participates on initiatives with deliverables and meets quality standards on time
- Leads cross-functional initiatives with deliverables and meets quality standards on time
EDUCATION
- Bachelor’s degree or equivalent work experience
EXPERIENCE
- 5+ years of experience in Data Engineering or ETL Development role
- Strong experience with PySpark, Python for building solid data pipelines
- Strong experience with Iceberg, Hive, S3, Trino
- Hands-on experience with Hadoop ecosystems, relational databases and SQL queries
- Hands-on experience with Apache Ranger, Rancher/Kubernetes preferred
- Hands-on experience with Talend, Red Point or other ETL technologies an advantage
- Experience with Agile Software Development methodologies
- Experience with GitLab, CI/CD process and ServiceNow etc..
KNOWLEDGE/SKILLS/ABILITIES
- Solid programming skills in object-oriented/functional scripting languages like Python, PySpark for building data pipelines with experience in testing, logging to ensure quality of code and data observability (Required)
- Experience in distributed systems and parallel data processing using big data tools such as Spark, PySpark, Hadoop, Kafka, Hive. (Required)
- Proficiency in querying with relational databases (Required)
- Strong knowledge of Linux/Unix-based computer systems (Required)
- Strong experience with Iceberg, Hive, S3, Trino (Required)
- Experience in building Data Processing pipelines using ETL tools like Talend, SSIS etc.. (Required)
- Hands-on experience with Apache Ranger, Rancher/Kubernetes
- Understanding of Machine Learning models and algorithms interfacing with Data Science team
- Proficiency in data visualization tools to showcase insights using Tableau, matplotlib etc..
- Nice to have AWS cloud experiences in Redshift, Lambda, Sage Maker, Glue etc..
- Experience with building Rest API
- Excellent data analytical, conceptual, and problem-solving skills
- Excellent communication skills to promote cross-team collaboration
Base Salary: $110,000 - $130,000
The ultimate compensation offered for the position will depend upon several factors such as skill level, cost of living, experience, and responsibilities.
Vericast offers a generous total rewards benefits package that includes medical, dental and vision coverage, 401K with company match and generous PTO allowance. A wide variety of additional benefits like life insurance, employee assistance and pet insurance are also available, not to mention smart and friendly coworkers!
At Vericast, we don’t just accept differences - we celebrate them, we support them, and we thrive on them for the benefit of our employees, our clients, and our community. As an Equal Opportunity employer, Vericast considers applicants for all positions without regard to race, color, creed, religion, national origin or ancestry, sex, sexual orientation, gender identity, age, disability, genetic information, veteran status, or any other classifications protected by law. Applicants who have disabilities may request that accommodations be made in order to complete the selection process by contacting our Talent Acquisition team at [email protected]. EEO is the law. To review your rights under Equal Employment Opportunity please visit: www.dol.gov/ofccp/regs/compliance/posters/pdf/eeopost.pdf.
#LI-KK1 #LI-REMOTE
