Design and validate realistic broken Linux and containerized infrastructure environments for AI systems. Build multi-service failure scenarios using Docker, Kubernetes, databases, and message queues; troubleshoot complex production-like incidents; create reference solutions, setup scripts, automated verifiers, and testing guidance. Investigate networking, storage, permissions, replication, and recovery issues while ensuring scenarios are reproducible, technically accurate, and challenging.
This role is for one of Weekday’s clients
Min Experience: 2+ years
Engagement Type: Ongoing, Hourly Contract
Work Mode: Fully Remote
Working Hours: 5–6 hours per day
Hourly Compensation: USD 20–25 per hour
We are looking for an experienced DevOps / SRE Test Bench Engineer to design, develop, and validate challenging infrastructure troubleshooting tasks for AI systems. In this role, you will create realistic, intentionally broken Linux and containerized environments that require deep technical investigation, systematic debugging, and recovery.
You will also develop reference solutions and automated verification scripts to ensure that each issue is genuinely resolved. The ideal candidate has strong hands-on infrastructure experience, a methodical troubleshooting approach, and the ability to build complex, realistic scenarios that test practical engineering skills.
RequirementsKey Responsibilities
- Design realistic, multi-service failure scenarios that require extensive investigation, root-cause analysis, and recovery.
- Build and configure test environments using Linux, Docker, Kubernetes, databases, message queues, and service dependencies.
- Develop broken infrastructure setups that simulate real-world production incidents and operational challenges.
- Write detailed reference solutions, troubleshooting guidance, and reliable automated verifiers with clearly defined success criteria.
- Validate task quality to ensure scenarios are fair, technically accurate, reproducible, and not solvable through a fixed runbook.
- Troubleshoot complex issues involving networking, storage, permissions, service configurations, replication, and system recovery.
- Develop setup scripts, automation utilities, and verification tests using Python or Bash.
- Accurately log working hours, communicate progress, and actively participate in the project’s WhatsApp community.
- Maintain the confidentiality of all project-related tasks, environments, and technical materials.
- Hands-on experience in DevOps, Site Reliability Engineering (SRE), platform engineering, infrastructure engineering, or a related technical role.
- Strong Linux administration and command-line skills, including SSH, shell scripting, process management, log analysis, file permissions, networking, and system services.
- Strong hands-on expertise in Docker and Kubernetes, including diagnosing container failures, broken deployments, service connectivity issues, and orchestration problems.
- Experience with databases and infrastructure components such as PostgreSQL, Redis, message queues, storage systems, replication, and recovery mechanisms.
- Proven ability to troubleshoot production-like environments involving multiple interdependent services.
- Familiarity with CI/CD pipelines, Infrastructure as Code (IaC), and cloud infrastructure.
- Ability to investigate ambiguous failures independently, identify root causes, and implement effective solutions without relying on predefined runbooks.
- A strong testing mindset, including experience designing automated checks, validation scripts, and deterministic verifiers.
- Proficiency in Python or Bash for environment setup, troubleshooting automation, and test development.
- Experience with cloud platforms such as AWS, Google Cloud Platform (GCP), or Microsoft Azure.
- Familiarity with Nginx, systemd, Logical Volume Manager (LVM), and Linux security hardening.
- Understanding of distributed systems, service dependencies, and fault recovery.
- Experience working with AI coding agents or AI-assisted software engineering tools.
- Complete the sign-up form to express your interest.
- Shortlisted candidates may be invited to complete a brief technical screening.
- Selected candidates will receive onboarding instructions and access to the project’s WhatsApp community.
- Compensation is hourly, based on approved and accurately logged working hours.
- All project tasks, technical materials, and related information must remain confidential and must not be shared outside the project.
Similar Jobs
.jpeg)
Artificial Intelligence • Hardware • Information Technology • Machine Learning
Monitor quality deviations, investigate manufacturing defects, perform root cause analysis, manage lot containment, and coordinate corrective actions. The role also monitors equipment and process performance, supports qualification and troubleshooting, drives yield and quality improvements, and uses Excel, Power BI, AI tools, dashboards, and statistical methods to identify trends and risks in semiconductor manufacturing.
Top Skills:
Automated Vision InspectionChatgptFmeaGenerative AiMachine LearningMicrosoft CopilotExcelMicrosoft Power BiStatistical Analysis
Artificial Intelligence • Consumer Web • Digital Media • Information Technology • Social Impact • Software
Lead the AI Quality engineering team while remaining hands-on building evaluation infrastructure, observability tooling, diagnostic systems, datasets, annotation workflows, and CI/CD pipelines for production AI agents. Run experiments across prompts, models, routing, caching, and agent harnesses to improve quality, latency, and cost. Partner with AI Core engineering and manage technical direction and team priorities.
Top Skills:
Ai AgentsBraintrustCi/CdLangsmithLlmsPythonRuby On Rails
Aerospace • Artificial Intelligence • Cloud • Machine Learning • Software • Cybersecurity • Defense
Conduct third-party due diligence and compliance screenings, manage escalations, monitor screening-system effectiveness, and drive compliance data analytics. Develop employee training and communications while partnering cross-functionally to ensure adherence to laws, regulations, and company policies. The role promotes ethical conduct and strengthens Honeywell Aerospace’s integrity and compliance culture.
What you need to know about the Mumbai Tech Scene
From haggling for the best price at Chor Bazaar to the bustle of Crawford Market, the energy of Mumbai's traditional markets is a key part of the city's charm. And while these markets will always have their place, the city also boasts a thriving e-commerce scene, ranking among the largest in the region. Driven by online sales in everything from snacks to licensed sports merchandise to children's apparel, the local industry is worth billions, with companies actively recruiting to meet the demands of continued growth.
