The Data Engineer is responsible for developing, maintaining, and optimizing data pipelines and infrastructure to ensure the availability, accuracy, and reliability of data for various business functions. The incumbent will play a critical role in creating the foundation for data-driven decision-making.
Principal Duties and Responsibilities:
- Design, build, and maintain efficient and scalable data pipelines to process structured and unstructured data from multiple sources.
- Work closely with the data architect to design and maintain a scalable and efficient data architecture that supports the organization's data needs.
- Assist in the development of data models and schemas to support analytical and reporting requirements.
- Integrate data from diverse sources, including databases, APIs, and streaming data, to provide a unified view of our data.
- Collaborate with software engineers, analysts, and other stakeholders to understand data requirements and implement data solutions that meet business needs.
- Ensure data quality, integrity, and accuracy by implementing data validation, monitoring, and testing processes.
- Optimize data storage, processing, and retrieval mechanisms to improve performance and reduce costs.
- Develop and maintain documentation related to data engineering processes, governance, best practices, systems, and tools.
- Identify opportunities for data-related process improvements and work with cross-functional teams to implement them.
- Stay updated with emerging trends and technologies in data engineering and adopt best practices to continuously improve the data infrastructure.
Minimum Knowledge, Skills, and Abilities:
- A minimum of four years of experience with data engineering, database development or related experience is required.
- Experience working in both AWS and Azure cloud environments, including Databricks.
- Familiarity with big data technologies such as Spark, Apache, and Hadoop.
- Knowledge of database design, data modeling, and data warehousing concepts.
- Proficiency in SQL and experience working with relational and non-relational databases.
- Familiarity with cloud databases, such as CosmosDB and AWS, and cloud data warehouses and lakes, such as Snowflake.
- Experience with data integration tools and ETL processes.
- Proficiency in programming languages such as Python, Java, or Scala.
- Strong analytical and problem-solving skills with the ability to identify and address complex data-related issues.
- Attention to detail and a commitment to ensuring data quality and accuracy.
Our comprehensive benefits package includes all traditional offerings such as:
- Health Insurance options, Dental Insurance options and Vision Insurance
- Employee Life Insurance/AD&D and Dependent Life Insurance options
- Short-term & Long-term Disability
- Health Savings Account with potential employer match
- Flexible Medical and Dependent Care Account
- Accident Insurance
- Critical Illness Insurance
- Employee Assistance Program
- Legal/Identify Theft Insurance options
- Long Term Care Insurance
- Pet Insurance
- 401 (k) Retirement Plan with match up to 5%, plus profit sharing & discretionary contributions (subject to vesting)
- 5 weeks of Paid Time Off (PTO)
- 10 paid holidays
We also offer other benefits to help foster a healthy, balance lifestyle such as:
- Flextime schedules
- Paid Parental Leave
- Student loan paydown and refinancing assistance
- Educational assistance for job related courses, seminars, certifications or degrees
- One paid day every year to volunteer for your non-profit of choice
- On-site fitness center (Maine & Virginia) or fitness reimbursement
- Subsidized parking
- Sit-Stand desks & daily stretch breaks
MEMIC is committed to a policy of nondiscrimination and equal opportunity for all employees and qualified applicants without regard to race, color, religious creed, national origin, ancestry, age, disability, genetics, gender identity, veteran's status, sexual orientation, or any other characteristic protected by law.
#IND2022
