Qode
Site Reliability Architect
TLDR
This role entails designing advanced monitoring, alerting, and observability solutions for high-stakes financial service platforms using AI and AIOps methodologies.
Site Reliability Architect (SRE Architect) – Unified Observability & AIOps
Location- Austin, TX
Role Summary
We are seeking a Senior SRE with strong expertise in Unified Observability, proactive detection, AIOps, and GenAI-driven operations to support complex, distributed financial services platforms. The role requires hands-on experience designing SLI/SLO-driven monitoring, dynamic thresholds, intelligent alerting, and AI/ML-based anomaly detection across multi-stream architectures.
Key Responsibilities
Observability & Reliability Engineering
- Design and implement unified observability dashboards across metrics, logs, traces, events, and topology
- Define and manage SLIs, SLOs, and error budgets aligned to business outcomes
- Build actionable dashboards for operations, engineering, and leadership
- Implement alerting strategies using static and dynamic thresholds
Proactive Detection & AIOps
- Leverage AI/ML/AIOps to detect anomalies, predict incidents, and reduce MTTR
- Transition monitoring from reactive alerts to proactive insights
- Implement noise reduction, alert correlation, and root cause analysis
- Apply baseline modeling, seasonality detection, and anomaly scoring
Distributed Systems & Dependency Analysis
- Monitor and troubleshoot multi-service architectures involving:
- Microservices
- Downstream APIs
- Kafka / streaming platforms
- Cloud infrastructure (Terraform, IaC)
- Identify whether issues originate from:
- Upstream/downstream dependencies
- Streaming platform
- Infrastructure
- Application code
Tooling & Platforms
- Deep hands-on experience with Dynatrace (mandatory)
- Experience with:
- OpenTelemetry
- Prometheus / Grafana
- ELK / EFK
- Cloud-native monitoring (AWS/Azure/GCP)
- Strong JSON-based telemetry manipulation and enrichment
GenAI & LLM Enablement
- Apply GenAI / LLMs for:
- Incident summarization
- Root cause explanation
- Runbook recommendations
- Auto-remediation suggestions
- Collaborate with platform teams to operationalize GenAI safely
Required Skills & Experience
✅ 15+ years in SRE / Production Engineering
✅ Strong Unified Observability background (not infra-only)
✅ Hands-on Dynatrace experience (metrics, traces, logs, Davis AI)
✅ SLI/SLO engineering experience in production systems
✅ Experience implementing dynamic thresholds and anomaly detection
✅ Knowledge of AI/ML concepts applied to Ops (AIOps)
✅ Distributed systems troubleshooting expertise
✅ Experience with Kafka or streaming data platforms
Differentiators (Highly Valued)
- Experience in financial services or regulated environments
- Proven reduction of alert noise and MTTR using AIOps
- GenAI / LLM integration into operations workflows
Interview Question Bank (Mapped to LPL Expectations)
1. Dashboards, SLAs, and Reliability Targets
Purpose: Identify true SREs vs dashboard builders
- How do you design dashboards differently for engineers vs leadership?
- Explain how SLIs and SLOs differ from SLAs. Which do you operationalize?
- How do you map SLOs to alerting without creating noise?
- What KPIs would you track for a critical trading or advisor-facing platform?
Red Flag: Talks only about CPU, memory, uptime
2. Alerting Strategy & Threshold Design
Purpose: Assess signal-to-noise maturity
- How do you decide when to use static vs dynamic thresholds?
- Explain how you prevent alert storms during high traffic or seasonal spikes.
- What makes an alert actionable?
- How do you design alerts for early symptom detection?
Follow-up
- What happens after an alert fires? Walk me through the lifecycle.
3. Dynamic Thresholds & Anomaly Detection
Purpose: Validate AIOps fundamentals
- How do dynamic thresholds work under the hood?
- How do you account for baseline drift and seasonality?
- What risks do dynamic thresholds introduce?
- How would you tune sensitivity to avoid false positives?
Expected Concepts ✅ Baselines
✅ ML models
✅ Adaptive learning
✅ Time-series analysis
4. Multiplexing (Metrics, Signals, Streams)
Purpose: Test system observability depth
- What is multiplexing in observability?
- How do multiple telemetry signals strengthen diagnosis?
- Provide an example where one signal was misleading.
- How do you correlate metrics, traces, logs, and events?
5. JSON Tooling & Proactive Detection
Purpose: Ensure hands-on operational telemetry skills
- How have you used JSON-based event payloads to enrich observability?
- How do you normalize data across heterogeneous sources?
- How do structured logs improve proactive detection?
- How do you extract signals from high-volume telemetry?
6. Proactive vs Reactive Detection
Purpose: Directly aligned to LPL concern
- Give an example where you predicted an incident before customer impact.
- What indicators help you identify impending failures?
- How do you measure the success of proactive detection?
7. Multi-Service Failure Diagnosis (Critical Question)
Purpose: Core differentiator at LPL
Scenario Question
A user-facing issue is reported. The architecture includes:
- Frontend
- Backend microservices
- Downstream APIs
- Kafka streams
- Terraform-managed infrastructure
Ask:
- How do you determine if the issue is:
- Application-related?
- Kafka or streaming lag?
- Downstream API latency?
- Infrastructure drift via Terraform?
Expected Approach ✅ Dependency mapping
✅ Golden signals
✅ Trace correlation
✅ Change analysis
8. Dynatrace (Mandatory)
Purpose: Address explicit gap in feedback
- What Dynatrace features have you used most?
- How does Davis AI determine root cause?
- How do you implement service-level baselining in Dynatrace?
- How do you reduce alert noise using Dynatrace?
Red Flag: “I’ve mostly used dashboards”
9. AI/ML & AIOps Fundamentals
Purpose: Ensure non-theoretical knowledge
- What ML techniques are commonly used in AIOps?
- How do supervised vs unsupervised models differ in Ops?
- Where does AI fail in observability?
- How do you validate AI-based decisions?
10. GenAI & LLM Use Cases for SRE
Purpose: Explicit LPL requirement
- Where do you see GenAI adding value in SRE?
- Have you used LLMs for incident response?
- How would you integrate GenAI without introducing risk?
- What data would you restrict from LLM exposure?
Expected Use Cases ✅ Incident summarization
✅ RCA explanation
✅ Runbook suggestions
✅ MTTR reduction
Qode is a technology-driven platform that transforms how recruiters and candidates connect by leveraging data and automation. Our solutions streamline the hiring process through machine learning, creating private talent pools and automating workflows, ultimately enhancing the quality of candidate evaluation and decision-making. With our no-code tools, we empower organizations to develop tailored recruitment strategies without needing extensive technical skills.
- Industry
- Internet Software & Services
Architect