Toloka

Training data for AI agents and LLMs·toloka.ai

Toloka provides expertly curated training and evaluation data for AI agents, LLMs, VLMs, and coding models, combining a managed human expert network with technology-driven quality control. Services include agent trajectory and RL-environment data, LLM/VLM fine-tuning datasets, AI safety red-teaming, creative AI data, and Physical AI/egocentric robotics data collection. The company also offers Toloka Arena, a benchmarking product to evaluate and rank AI models against hidden test sets.

What it's for

Training and evaluating AI agents across tool-use workflowsGenerating RL environments and MCP replicas for agent testingAI safety red-teaming and injection vulnerability testingCollecting SFT, preference, and reasoning-chain data for LLMs and VLMsProducing coding training and evaluation dataHuman and model-based evaluation of AI model outputsCreative AI data collection across text, image, video, and audioPhysical AI data collection (egocentric/exocentric video, robotics)Benchmarking model performance via Toloka Arena

Features 18

Agent trajectory demonstrations and step-by-step evaluations
Automated quality control methods (50+)
Computer-use testbeds
Custom evaluation frameworks (e.g., pairwise comparison)
Domain-specific demonstrations and preference data
Egocentric and exocentric Physical AI data capture
Expert-captured workflows from real teams and tooling
Full repository structures and rapid prototyping data
Hidden, private benchmark test sets (Toloka Arena)
Multi-format creative content collection (text, image, video, audio)
Open-source HomER egocentric robotics dataset
Platform-level antifraud methods (60+)
Production-ready code generation examples
Professional annotation and quality filtering
Reinforcement learning tasks with built-in verification
Safety red-teaming for injection vulnerabilities and policy compliance
Step-by-step reasoning chains for complex problem-solving
Virtual environments and RL-gyms with MCP replicas

At a glance

DeploymentCloud

PlatformsWeb

ForAI teams, enterprises, frontier AI labs, big tech companies

CompanyToloka AI BV · 2014

Complianceself-reported

SOC 2GDPRHIPAAISO 27701CCPAISO 27001

Resources

Docs

Pricing

All pricing is quote-based; the pricing page lists four evaluation/data tiers (Closed automatic evaluation, Human-in-the-loop, Off-the-shelf datasets, Bespoke RL-gyms) but no numeric prices are shown — buyers must contact sales for a quote.

Closed automatic evaluationQuote

Pricing not publicly listed; requires contacting sales.

Human-in-the-loopQuote

Pricing not publicly listed; requires contacting sales.

Off-the-shelf datasetsQuote

Pricing not publicly listed; licensing of pre-built RL gyms, via catalog/contact.

Bespoke RL-gymsQuote

Custom/enterprise offering; pricing not publicly listed, requires contacting sales.