Opportunity brief
Overview: Reliability and Operational Excellence
The Site Reliability Engineer (SRE) is responsible for the ultimate stability, performance, and scalability of our entire integrated supply chain e-commerce platform. You will apply software engineering principles to operations, ensuring the high availability and resilience of the customer-facing e-commerce storefront, internal SaaS tools (WMS, OMS), and specialized AI agent services.
Internship Details
Duration: 3 months
Start Date: Immediate
Location: Remote
Stipend: None initially. Based on your first-quarter performance, you may be offered a paid full-time opportunity, or even be absorbed directly by the client as an FTE.
Key Responsibilities & Core Projects
You will be the champion of uptime, performance, and automated operations for systems handling the critical MES → WMS → OMS flow.
- Availability & SLO Management: Define, implement, and track Service Level Objectives (SLOs) and Service Level Indicators (SLIs) for core business processes and all application layers. Manage the platform's overall Service Level Agreement (SLA).
- Observability & Alerting: Architect, maintain, and optimize the comprehensive observability stack (Prometheus, Grafana, Loki, Tempo, OpenTelemetry). Develop high-fidelity alerting and ensure distributed tracing across the NestJS modular monolith and associated data stores (PostgreSQL, Redis).
- Incident Response & Review: Own the incident response workflow, ensuring rapid triage, mitigation, and root cause analysis. Conduct thorough post-incident reviews to drive continuous improvement and eliminate recurring toil.
- Scalability & Capacity Planning: Optimize auto-scaling policies for all services running on Docker containers. Conduct capacity planning based on business projections, especially for peak e-commerce and manufacturing load.
- Disaster Recovery (DR): Design, implement, and regularly test Disaster Recovery procedures, including backup and restoration workflows for PostgreSQL 15 using tools like pgBackRest.
- Automation: Eliminate operational toil through automation, managing infrastructure-as-code (Terraform) and CI/CD pipelines (Makefile).
Required Technologies & Tools
Candidates must possess deep experience in cloud operations, observability, and infrastructure automation:
- Observability Stack: Prometheus, Grafana, Loki, Tempo, OpenTelemetry (mandatory).
- Infrastructure & Platform: Terraform, Docker, Traefik, Oracle Cloud Free VMs (or equivalent public cloud).
- Data & Resilience: PostgreSQL (Deep knowledge), Redis, pgBackRest.
- Automation: Strong scripting skills (Python/Bash) and experience with CI/CD tools and Makefile.
- Methodology: Expert knowledge of SRE principles, toil reduction, and error budgeting.
AI Agent Focus
You will be responsible for the operational reliability of the emerging AI layer.
- Agent Reliability: Implement specialized monitoring and logging for the AI agent services, ensuring LLM integrations and multi-agent systems (built with frameworks like LangChain) meet defined performance and availability SLOs.
- Resource Optimization: Efficiently manage resource allocation for computationally intensive AI workloads to maintain platform stability and cost-efficiency.
Success Metrics & Career Path
Performance will be measured by:
- Uptime/Availability: Achieving defined SLAs/SLOs across the platform.
- MTTR: Reduction in Mean Time To Recover from production incidents.
- Toil Reduction: Measured percentage reduction in manual, repetitive operational tasks through automation.
Mentorship Structure: Reports to the Head of Technology/CTO, working collaboratively with DevSecOps and development teams to ensure software is designed for reliability. Originally posted on Himalayas