Lead LLMOps initiatives to scale and govern generative AI: define AI quality standards, manage prompt and model versioning, coordinate safety/red team testing, implement observability for latency/cost/quality using tools like Langfuse, manage Jira workflows, develop playbooks, and lead AI incident response and root-cause analysis.
Build your future with Sovos.
At Sovos, we're transforming tax compliance through innovative technology and AI-powered solutions used by customers worldwide. We're looking for an LLM Ops Lead with 2-3 years of experience to help scale, govern, and operationalize Generative AI across the business while ensuring quality, safety, observability, and continuous improvement.
What You'll Do- Own AI quality standards, evaluation frameworks, and release gates across Ask Sovi products
- Lead prompt and model versioning, regression detection, and AI governance initiatives
- Coordinate Safety, Guardrails, and Red Team testing programs
- Build observability practices across latency, cost, quality, and safety metrics using tools such as Langfuse
- Manage Jira epics, ticket triage, and cross-functional coordination across Product and Engineering teams
- Develop LLMOps playbooks, best practices, and operational standards
- Lead AI incident response, troubleshooting, and root-cause analysis
- 2-3 years of experience in Applied AI, LLMOps, AI Quality Engineering, AI Operations, Test Automation, or related fields
- Hands-on experience with Langfuse, LangSmith, or similar AI observability and evaluation tools
- Strong understanding of AI quality, evaluation pipelines, prompt versioning, and AI governance
- Experience with AI safety, guardrails, content moderation, audit logging, and release management
- Strong stakeholder management and communication skills
- Ability to support teams operating across UK business hours
- Applied AI Engineers from AI-native or AI-first organizations with hands-on Langfuse/LangSmith experience
- Product-minded Engineers who combine strong technical skills with customer and business impact awareness
- QA/Test Automation Engineers who have transitioned into AI evaluation, quality, or LLM testing
- AI Quality Analysts with expertise in hallucination detection, model drift monitoring, LLM-as-a-Judge methodologies, and AI quality governance
- Work on cutting-edge Generative AI products and platforms
- Collaborate with global AI, Product, and Engineering leaders
- Bi-Weekly Meeting Free Days
- Mentoring and career development programs
- Globally recognized training opportunities
- Comprehensive health and wellness benefits
- Flexible and collaborative work environment
CCAT & EPP Assessment → Automated Video Interview → Hiring Manager Interview → Final Interview
#Hiring #LLMOps #AIOps #GenerativeAI #AIEngineering #LangSmith #Langfuse #SovosCareers
Similar Jobs
Cloud • Information Technology • Security • Software • Cybersecurity
Manage the post-sale customer lifecycle from onboarding through renewals, driving adoption, retention, upsells, and contract growth. Serve as a trusted advisor, lead business reviews, negotiate renewals, maintain forecasts, and identify commercial opportunities through data analysis. Collaborate with Sales, Product, Engineering, Support, and Marketing to resolve customer needs and execute success plans. Use generative AI, analytics, and workflow automation to improve customer outcomes and operational efficiency.
Top Skills:
Ai AnalyticsCloudflareComputer NetworkingCRMGenerative AiLarge Language Models (Llms)Network SecuritySaaSWorkflow Automation
Cloud • Information Technology • Security • Software • Cybersecurity
Lead the design, implementation, and operation of globally distributed MCP Server Portal services. Own complex features across architecture, deployment, observability, and production support; design authentication, authorization, OAuth, identity, and credential-management flows; improve reliability and scalability; investigate incidents; evolve systems safely as protocols change; and mentor engineers while partnering across product, security, and engineering teams.
Top Skills:
APIsC++Cloudflare Developer PlatformDistributed TracingGoHTTPJavaScriptJwtMetricsModel Context ProtocolOauth 2.0Openid ConnectPkceRustServerless Edge ComputingStructured LoggingTypescriptZero Trust
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
Participate in Mastercard’s 18-month Launch Graduate Program within Product Management. Develop business cases, create scalable customer-centric products, measure product and business metrics, run experiments, prioritize roadmap features, support product releases, and help resolve escalated customer operational issues.
What you need to know about the Mumbai Tech Scene
From haggling for the best price at Chor Bazaar to the bustle of Crawford Market, the energy of Mumbai's traditional markets is a key part of the city's charm. And while these markets will always have their place, the city also boasts a thriving e-commerce scene, ranking among the largest in the region. Driven by online sales in everything from snacks to licensed sports merchandise to children's apparel, the local industry is worth billions, with companies actively recruiting to meet the demands of continued growth.
