Lead LLMOps initiatives to scale and govern generative AI: define AI quality standards, manage prompt and model versioning, coordinate safety/red team testing, implement observability for latency/cost/quality using tools like Langfuse, manage Jira workflows, develop playbooks, and lead AI incident response and root-cause analysis.
Build your future with Sovos.
At Sovos, we're transforming tax compliance through innovative technology and AI-powered solutions used by customers worldwide. We're looking for an LLM Ops Lead with 2-3 years of experience to help scale, govern, and operationalize Generative AI across the business while ensuring quality, safety, observability, and continuous improvement.
What You'll Do- Own AI quality standards, evaluation frameworks, and release gates across Ask Sovi products
- Lead prompt and model versioning, regression detection, and AI governance initiatives
- Coordinate Safety, Guardrails, and Red Team testing programs
- Build observability practices across latency, cost, quality, and safety metrics using tools such as Langfuse
- Manage Jira epics, ticket triage, and cross-functional coordination across Product and Engineering teams
- Develop LLMOps playbooks, best practices, and operational standards
- Lead AI incident response, troubleshooting, and root-cause analysis
- 2-3 years of experience in Applied AI, LLMOps, AI Quality Engineering, AI Operations, Test Automation, or related fields
- Hands-on experience with Langfuse, LangSmith, or similar AI observability and evaluation tools
- Strong understanding of AI quality, evaluation pipelines, prompt versioning, and AI governance
- Experience with AI safety, guardrails, content moderation, audit logging, and release management
- Strong stakeholder management and communication skills
- Ability to support teams operating across UK business hours
- Applied AI Engineers from AI-native or AI-first organizations with hands-on Langfuse/LangSmith experience
- Product-minded Engineers who combine strong technical skills with customer and business impact awareness
- QA/Test Automation Engineers who have transitioned into AI evaluation, quality, or LLM testing
- AI Quality Analysts with expertise in hallucination detection, model drift monitoring, LLM-as-a-Judge methodologies, and AI quality governance
- Work on cutting-edge Generative AI products and platforms
- Collaborate with global AI, Product, and Engineering leaders
- Bi-Weekly Meeting Free Days
- Mentoring and career development programs
- Globally recognized training opportunities
- Comprehensive health and wellness benefits
- Flexible and collaborative work environment
CCAT & EPP Assessment → Automated Video Interview → Hiring Manager Interview → Final Interview
#Hiring #LLMOps #AIOps #GenerativeAI #AIEngineering #LangSmith #Langfuse #SovosCareers
Similar Jobs
Enterprise Web • Hardware • Internet of Things • Software
Embed with manufacturing customers to design, build, and deploy Tulip solutions. Lead OT/IT integrations (PLCs, MES/ERP, quality systems), troubleshoot production issues, and deliver scalable, production-ready applications. Advise on architecture, document solutions, and represent customer needs to Product and Engineering.
Top Skills:
Cloud-NativeEmbedded AiErpMachine Data IntegrationsMesNo-Code PlatformPlcsQuality SystemsTulip Frontline Operations Platform
Cloud • Information Technology • Security • Software • Cybersecurity
Lead and automate Cloudflare's Deal Desk by architecting AI-driven approval workflows, structuring complex enterprise deals, advising Sales on pricing/terms, tuning AI agents, and partnering cross-functionally to reduce friction and improve deal velocity and long-term value.
Top Skills:
Agentic WorkflowsAutomated Pricing ModelsGenerative AiLlmsPredictive AnalyticsPrompt Engineering
Cloud • Information Technology • Security • Software • Cybersecurity
Support EMEA PR operations: manage editorial calendars, coordinate agencies, pitch and amplify technical content, handle media relations and inbound requests, organize briefings and events, support spokespeople and regional speaking/awards initiatives, and execute day-to-day PR campaigns from concept to completion.
Top Skills:
Ai ToolsCloudflare
What you need to know about the Mumbai Tech Scene
From haggling for the best price at Chor Bazaar to the bustle of Crawford Market, the energy of Mumbai's traditional markets is a key part of the city's charm. And while these markets will always have their place, the city also boasts a thriving e-commerce scene, ranking among the largest in the region. Driven by online sales in everything from snacks to licensed sports merchandise to children's apparel, the local industry is worth billions, with companies actively recruiting to meet the demands of continued growth.
