Company Summary
Irth Solutions is a software product company building cutting-edge technology platforms that continuously set industry benchmarks. With a strong product culture, collaborative environment, and high growth potential, Irth offers an exciting opportunity to work on modern, enterprise-scale data platforms.
Irth is currently building a modern, multi-cloud, enterprise-grade data estate using Databricks to centralize data across AWS, Azure, and GCP. We are hiring a Data Engineer to support the implementation and operationalization of this unified data platform.
Job Summary
Irth is developing a unified Databricks-based Lakehouse platform to consolidate and govern data across multiple cloud environments and products.
As a Data Engineer, you will play a hands-on role in implementing scalable data pipelines, applying governance and quality standards, and supporting the enterprise data architecture. You will work closely with the Senior Data Architect to build reliable, secure, and high-performance data pipelines using Databricks, Spark, and cloud-native technologies.
This role is ideal for mid-level engineers looking to expand their expertise in Databricks, Spark, Delta Lake, and modern lakehouse architecture.
Requirements
Data Pipeline Development
· Build and maintain ingestion pipelines across AWS, Azure, and GCP.
· Implement batch and streaming pipelines using Databricks, Spark/PySpark, SQL, Delta Live Tables, and Lakeflow.
· Apply medallion architecture patterns for structured data transformation.
· Implement CDC, SCD Type 1/2, schema evolution, and data validation.
Platform & Storage Implementation
· Manage Delta Lake storage, tables, partitions, and performance optimization (OPTIMIZE, Z-ORDER, VACUUM).
· Support metadata management, lineage tracking, and cataloging using Unity Catalog.
· Assist with multi-cloud integrations between cloud storage platforms and Databricks.
Governance, Quality & Compliance
· Implement data quality validation, profiling, and monitoring aligned with governance standards.
· Apply security controls including RBAC, encryption, and data classification.
· Support metadata and lineage implementation using Unity Catalog, Purview, or similar tools.
Orchestration, Automation & Operations
· Build and manage workflows using Databricks Workflows, Delta Live Tables, Azure Data Factory, or similar tools.
· Support CI/CD pipelines, code versioning, and deployment automation.
· Assist in troubleshooting, pipeline recovery, and performance tuning.
Collaboration & Documentation
· Work closely with the Senior Data Architect to implement architecture designs.
· Participate in design reviews and technical discussions.
· Document pipelines, data models, transformation logic, and operational procedures.
Skills & Experience
Required:
· 3–5 years of experience in data engineering, ETL, or cloud data platforms.
· Experience with Databricks, Spark, PySpark, or distributed data processing.
· Strong SQL and structured data transformation skills.
· Experience with at least one cloud platform (Azure preferred; AWS/GCP acceptable).
· Knowledge of data modeling, schema evolution, pipeline troubleshooting, and data quality.
· Understanding of security practices including RBAC, encryption, and credential management.
Preferred:
· Experience with Delta Lake, medallion architecture, and lakehouse design.
· Familiarity with Unity Catalog, Purview, Glue Catalog, or similar tools.
· Experience with orchestration tools (ADF, Airflow, Databricks Workflows).
· Experience with Git, CI/CD, and DevOps practices.
· Knowledge of Power BI, geospatial data, or AI/ML data preparation.
· Relevant cloud or Databricks certifications (DP-203, Data Engineer Associate, etc.).
EDUCATION
· Bachelor’s or master’s degree in computer science, Software Engineering, or a related field, or equivalent professional experience.
Benefits
WHAT IS IN IT FOR YOU
· Being an integral part of a dynamic, growing company that is well respected in its industry.
· Competitive pay based on experience.
Above our heads and below our feet is a spiderweb of cables, pipelines, electric lines, sewer systems, telecommunications networks and more. From the assets themselves to the people that manage and maintain them, everything and everyone must work together seamlessly to ensure reliability, resiliency, safety and ongoing compliance with local and federal mandates. For decades, Irth Solutions has worked with global leaders such as Shell, ExxonMobil, T-Mobile, Verizon and more to provide industry-leading technology to help enhance the reliability and resiliency of their critical network infrastructure. We know our work is mission-critical and we are committed to our goal of responding to customer needs and continuing to innovate to deliver impactful solutions for the future. But we can’t do it alone. We need dedicated, hard-working professionals that foster a collaborative, fun and supportive environment that ensures respect and a safe place to allow ideas to flourish. At our core, we value transparency, honesty and integrity. Our team thrives on diversity and those who take responsibility, ownership and make a commitment to teamwork.At Irth Solutions, teamwork means supporting one another’s goals and celebrating collective victories. Individually, we can get our jobs done — but together, we can all achieve bold visions. Are you interested in joining Irth Solutions? Start by applying below.Irth Solutions is an equal-opportunity employer. All applicants will be considered for employment without attention to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status.
Please mention you found this job on AI Jobs. It helps us get more startups to hire on our site. Thanks and good luck!
Understand the required skills and qualifications, anticipate the questions you may be asked, and study well-prepared answers using our sample responses.
Data Engineer Q&A's