Create rigorous mathematics coding problems, verified Python solutions, unit tests, and discriminative evaluation cases for frontier AI models. Perform quality control through the Central Task Platform, refine tasks based on feedback, and optimize datasets for LLM evaluation. The role requires advanced mathematics expertise, scientific programming, deterministic outputs, mathematical research or writing experience, and familiarity with AI-generated code assessment.
Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.
Role Overview
We are seeking a mathematics-focused professional to support an advanced AI research initiative developing high-quality STEM coding datasets. In this role, you will create rigorous mathematical problems, implement verified Python solutions, and develop evaluation tests designed to assess the capabilities of frontier AI models. The position combines mathematical problem-solving, scientific programming, and AI quality evaluation, with a strong emphasis on accuracy, determinism, and first-submission quality.
CONTRACT: Contractor assignment, 4 weeks
COMMITMENT: Hourly, up to 40h/week
LOCATIONS: Remote - Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Indonesia, Pakistan, Turkey, Vietnam
PROCESS: Interest Check Form, CV and Google Scholar review
Key Responsibilities
- Author mathematical coding tasks containing one main problem and at least 3 logically connected sub-problems that progressively build toward the main solution.
- Implement verified golden solutions in Python with complete unit test coverage.
- Design discriminative test cases that distinguish correct from incorrect AI-generated outputs.
- Perform quality control validation using the Central Task Platform (CTP), including Tier 1 structure checks and Tier 2 quality rubrics.
- Refine problem specifications, solutions, and test cases based on QC feedback.
- Optimize tasks to meet Pass@K evaluation criteria across multiple LLM judges, including GPT, Gemini, and Nemotron.
- Ensure mathematical correctness, well-posedness, determinism, and compliance with project quality standards.
- Maintain high first-submission quality and minimize rework, targeting consistent L1 approval.
- Participate in review sessions, feedback meetings, and project standups during required overlap hours.
Requirements
- Master's degree or PhD in Mathematics.
- Strong Python programming skills with experience in scientific computing.
- Experience using scientific Python libraries such as NumPy, SciPy, SymPy, or relevant domain-specific tools.
- Ability to formulate rigorous mathematical problems with clear constraints and expected outputs.
- Experience implementing scientific or mathematical solutions with unit tests and deterministic outputs.
- Prior experience in AI data annotation, scientific research, or scientific writing.
- Familiarity with LLM evaluation frameworks, coding benchmarks, or AI-generated code assessment.
- Published research or academic project experience in mathematics or another STEM discipline.
Similar Jobs
Agency • Gaming • Marketing Tech • Mobile • Analytics
Manages packaging supply chain operations across assigned markets, including category strategy execution, strategic sourcing, supplier relationship management, negotiations, contracts, supplier performance, supply assurance, risk mitigation, cost optimization, and supply planning. Partners with product development, quality, commercial, procurement, and suppliers to ensure continuity, quality, service, innovation, and financial performance. Requires strong commercial, analytical, stakeholder management, and procurement expertise.
Top Skills:
MS OfficeProcurement SystemsSourcing Systems
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Promotes hematology pharmaceutical products to healthcare professionals, manages a geographic territory, builds relationships with key opinion leaders, and achieves sales targets. Responsibilities include presenting clinical data, executing promotional strategies, monitoring competitors and market trends, maintaining compliant customer records, preparing reports, and using omnichannel and digital sales tools. The role requires hematology experience, medical representative experience, scientific knowledge, strong selling and presentation skills, and a pharmacy or related scientific degree.
Top Skills:
ExcelMicrosoft OutlookMicrosoft PowerpointMicrosoft WordPfizer SoftwareSales Analytics Platforms
Cloud • Information Technology • Internet of Things • Machine Learning • Software • Cybersecurity • Infrastructure as a Service (IaaS)
Designs and supports IP core and transport network solutions for customer projects. Responsibilities include requirements analysis, high-level architecture, integration planning, troubleshooting, testing, acceptance, migration, technical documentation, stakeholder coordination, customer workshops, and operational handover. The role also supports proposals and solution estimates while contributing to network automation, knowledge sharing, and Ericsson solution reuse.
Top Skills:
BgpCcdCcnaCcnpCiscoCloud InfrastructureCnisEricsson Ip NetworkingEricsson PlatformsEvolved Ip NetworkHuaweiIp NetworkingIp/MplsIs-IsJncipJuniperL2/L3 VpnsNetwork AutomationNetwork ResiliencyNetwork SecurityNetwork SynchronizationNfviNokiaObservabilityOspfQosRoutingScriptingSwitchingTelecommunicationsTelemetryTraffic Engineering
What you need to know about the Mumbai Tech Scene
From haggling for the best price at Chor Bazaar to the bustle of Crawford Market, the energy of Mumbai's traditional markets is a key part of the city's charm. And while these markets will always have their place, the city also boasts a thriving e-commerce scene, ranking among the largest in the region. Driven by online sales in everything from snacks to licensed sports merchandise to children's apparel, the local industry is worth billions, with companies actively recruiting to meet the demands of continued growth.
