Data Engineer I

About DAT

DAT is an award-winning employer of choice and a next-generation SaaS technology company that has been at the leading edge of innovation in transportation supply chain logistics for 45 years. ders optimistically share future possibilities to inspire and motivate others toward their full potential. We expect our employees to find ways to embrace positive change, be curious and challenge the status quo, and provide solutions to unmet problems. Joining DAT means joining a culture focused on fostering development, building genuine connections, recognizing each other’s strengths and sharing in successes.

We continue to transform the industry year over year, by deploying a suite of software solutions to millions of customers every day - customers who depend on DAT for the most relevant data and most accurate insights to help them make smarter business decisions and run their companies more profitably. We operate the largest marketplace of its kind in North America, with 400 million freights posted in 2022, and a database of $150 billion of annual global shipment market transaction data. Our headquarters are in Denver, CO, with additional offices in Missouri, Oregon, and Bangalore, India. For additional information, see www.DAT.com/company.

The Opportunity

Key Technologies

• Data Warehouse: Snowflake
• Transformation: dbt (data build tool)
• Orchestration: Apache Airflow
• Programming: Python, SQL

Core Responsibilities

Data Pipeline Development & Engineering

• ELT Process Implementation: Assist senior engineers in designing, building, testing, and maintaining cloud-native ELT (Extract, Load, Transform) data pipelines, ensuring data is reliably loaded into Snowflake.
• Transformation with dbt: Develop and maintain data models using dbt (data build tool) for data cleaning, aggregation, and transformation and utilize the Snowflake data warehouse.
• SQL Development: Write, optimize, and review complex SQL queries for data manipulation, transformation, and performance tuning on Snowflake.
• Python Scripting: Utilize Python to build custom data extraction scripts, implement monitoring tools, and contribute to general automation efforts.

Workflow Orchestration and Automation

• Airflow DAGs: Learn to author, schedule, and monitor data workflows defined as Directed Acyclic Graphs (DAGs) in Apache Airflow.
• Pipeline Scheduling: Integrate and orchestrate dbt runs and other pipeline tasks within Airflow to manage dependencies and execution timing.
• Automation: Focus on automating repetitive tasks across the data lifecycle, reducing manual effort and improving pipeline efficiency.

Data Quality, Testing, and Monitoring

• Data Quality: Implement data validation and testing frameworks using features of dbt (e.g., uniqueness, non-null checks) to ensure high data quality and accuracy within Snowflake data marts.
• Troubleshooting: Monitor data pipeline health, troubleshoot failed Airflow tasks and dbt runs, and quickly resolve data flow issues.
• Documentation: Maintain clear and current technical documentation for data models, Airflow DAGs, and pipeline logic.

Qualifications & Experience

• Experience: 1-3 years of professional experience in a Data Engineering, Analytics Engineering, or similar technical role (including relevant internship experience).
• Education: Bachelor's Degree in Computer Science, Information Technology, Engineering, or a related quantitative field.
• Technical Proficiency:
• Required: Strong proficiency in Python and advanced knowledge of SQL.
• Hands-on experience with a cloud data warehouse, preferably Snowflake.
• Familiarity with data transformation concepts and tools, specifically dbt.
• Basic experience creating or running jobs/workflows using an orchestration tool like Apache Airflow.
• Soft Skills: Strong problem-solving abilities, excellent attention to detail, and a proactive, collaborative approach to teamwork.

DAT embraces the value of a diverse workforce, and believes it is a core strength of our company that we encourage those values in every DAT employee, at every level of our organization, regardless of tenure or rank. We provide equal employment opportunities (EEO) to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, disability, genetic information, marital status, amnesty, or status as a covered veteran in accordance with applicable federal, state, and local laws.

Equal Opportunity Employer/Protected Veterans/Individuals with Disabilities 

The contractor will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. However, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by the employer, or (c) consistent with the contractor’s legal duty to furnish information. 41 CFR 60-1.35(c)

#LI-SC1