Data Engineer

Anticipated Contract End Date/Length: November 30, 2026
Work set up: Hybrid, 60% office and 40% home (must be eligible for BPSS)

Our client in the Information Technology and Services industry is looking for a Data Engineer to design, implement, and maintain scalable data pipelines that ingest and process OpenShift telemetry including metrics, logs, and traces. This role focuses on streaming telemetry via Kafka, engineering resilient consumer services for transformation and enrichment, and integrating processed data into Splunk for advanced visualization, dashboards, alerting, and analytics. The position plays a key role in advancing observability maturity toward proactive insights while ensuring governance, data quality, lineage, and compliance across hybrid and multi-cluster environments.

What you will do:

• Design, implement, and maintain data pipelines to ingest and process OpenShift telemetry at scale.
• Stream OpenShift telemetry via Kafka using producers, topics, and schemas and build resilient consumer services for transformation and enrichment.
• Engineer data models and routing for multi-tenant observability and ensure lineage, quality, and SLAs across the stream layer.
• Integrate processed telemetry into Splunk for visualization, dashboards, alerting, and analytics to achieve Observability Level 4.
• Implement schema management using Avro or Protobuf, including governance and versioning for telemetry events.
• Build automated validation, replay, and backfill mechanisms to ensure data reliability and recovery.
• Instrument services with OpenTelemetry and standardize tracing, metrics, and structured logging across platforms.
• Use LLMs to enhance observability capabilities such as query assistance, anomaly summarization, and runbook generation.
• Collaborate with platform, SRE, and application teams to integrate telemetry, alerts, and SLOs.
• Ensure security, compliance, and best practices for data pipelines and observability platforms.
• Document data flows, schemas, dashboards, and operational runbooks.

• Hands-on experience building streaming data pipelines with Kafka including producers, consumers, schema registry, Kafka Connect, KSQL, or KStream.
• Proficiency with OpenShift or Kubernetes telemetry using OpenTelemetry, Prometheus, and CLI tooling.
• Experience integrating telemetry into Splunk using HEC, UF, sourcetypes, and CIM and building dashboards and alerting.
• Strong data engineering skills in Python or similar languages for ETL, ELT, enrichment, and validation.
• Knowledge of event schemas using Avro, Protobuf, or JSON and understanding of backward and forward compatibility.
• Familiarity with observability standards and practices and ability to drive toward Level 4 maturity with proactive monitoring and automated insights.
• Understanding of hybrid cloud and multi-cluster telemetry patterns.
• Knowledge of security and compliance for data pipelines including secret management, RBAC, and encryption in transit and at rest.
• Strong problem-solving skills and ability to work in a collaborative team environment.
• Strong communication and documentation skills.

All your information will be kept confidential according to EEO guidelines.

Candidates must be legally authorized to live and work in the country where the position is based, without requiring employer sponsorship.

HelloKindred is committed to fair, transparent, and inclusive hiring practices. We assess candidates based on skills, experience, and role-related requirements.

We appreciate your interest in this opportunity. While we review every application carefully, only candidates selected for an interview will be contacted.

HelloKindred is an equal opportunity employer. We welcome applicants of all backgrounds and do not discriminate on the basis of race, colour, religion, sex, gender identity or expression, sexual orientation, age, national origin, disability, veteran status, or any other protected characteristic under applicable law.