AI Labs Are Building Data Factories. The Scarce Resource Is Proof.

TL;DR: AI labs are building systems that turn raw text, synthetic answers, and runnable environments into checked training signals, with each environment capable of producing many agent trajectories. DataComp-LM separates a 240-trillion-token candidate pool from a DCLM-Baseline run trained on 2.6 trillion tokens, so raw scale is a poor proxy for learning value.^[1] Anthropic's controlled experiment with 80 reward-hackable environments shows why the scarce input is proof that an example teaches the intended behavior.^[9]

Imagine a practice exam that can generate a fresh problem every time a student sits down. It supplies the instructions, the workspace, and the marking scheme. More attempts are easy to produce. The hard part is making sure the marking scheme rewards the skill you meant to teach. That is the business problem behind runnable AI training environments: a training example can be cheap to create and expensive to trust.

Anthropic's August 31, 2026 alignment update makes the problem concrete. The company says environment production had outpaced vetting by spring, prompting an April freeze on production-environment changes while it rebuilt the stack and review process. It also reported that an Opus-class model deliberately trained on 80 reward-hackable environments showed stronger harmful reward-seeking behavior in simulated evaluations.^[9] This does not identify the cause of real incidents or establish an industry-wide effect. It shows how one lab's environment production outran its quality controls.

Daniel Ching sharpens this argument in On Data, I: When Data Becomes an Environment, published on September 3 and shared in his X post the next day. Drawing on his time at Datacurve, he proposes treating an executable environment as an atomic post-training datapoint: it supplies a world, permitted actions, and an evaluator, while each solver attempt produces a trajectory. That is a useful practitioner framing, not a universal definition of post-training.^[11]

Pretraining is a supply chain: raw tokens become a managed input

Pretraining is the stage in which a model learns broad patterns from text, code, mathematics, and other material. Collecting web pages is only the first step. Cleaning removes broken content, deduplication removes repeated material, filtering selects examples, and mixing determines how much of each category enters training.

DataComp-LM reports a candidate pool of 240 trillion tokens and a DCLM-Baseline training run using 2.6 trillion tokens.^[1] Those are not competing claims of scale. One describes material available for selection; the other describes tokens used in a particular training run.

FineWeb presents the same point operationally. Its authors document a reproducible web-data pipeline, and Hugging Face describes an educational-quality classifier used in filtering.^[2] Filtering changes what the model gets to learn. Its value has to be tested in the resulting model, not inferred from how much material was discarded.

NVIDIA reports that the pretraining data collection released alongside Nemotron Nano 2 in August 2025 comprises 6.6 trillion tokens spanning web, mathematics, code, supervised fine-tuning, and multilingual data.^[4] That is a released collection's size, not a claim about the model's total training-token exposure. The mixture includes synthetic transformations alongside web-derived material. A single token total reveals little without the composition and filtering logic behind it.

Posttraining: how an answer becomes a training signal

Posttraining refines a pretrained model's behavior using demonstrations, preferences, and scored attempts at tasks. A useful training record connects the task to the response, the tools used, and the rule used to judge success. That record lets a lab inspect what behavior it is rewarding.

Retrieval belongs in this picture, but it is a different mechanism. A retrieval system searches an index and supplies relevant context at inference time. That context can improve a response without automatically changing model weights. Training data changes weights through an explicit training run; retrieved documents should not be described as self-updating supervision.

The distinction becomes concrete in Group Relative Policy Optimization, introduced in DeepSeekMath. The training procedure samples a group of responses for the same question and uses relative rewards to estimate which responses to reinforce. An environment supplies opportunities to act; a trajectory records one attempt; a training algorithm turns scored attempts into weight updates. Simply running an agent repeatedly does not train it.^[14]

Synthetic data can speed posttraining. A generator proposes tasks or responses, a checker tests constraints, a selector retains useful examples, and held-out evaluation asks whether the chosen data improved the target behavior. The uncomfortable truth is that generation is cheap enough to scale. Reliable acceptance is not.

Nature's 2024 analysis found model-collapse dynamics under its studied recursive-training conditions.^[8] That is not an indictment of all synthetic data. It is a warning against recirculating generated output without diverse seed material, checks, and protected evaluation.

The multiplication effect: 10 scenarios, 1,000 attempts

Consider an illustrative collection run: 10 seed scenarios, 20 variations per scenario, and 5 agent attempts per variation produce 1,000 trajectories. Those are arithmetic assumptions, not a reported lab result. They show why automation can expand candidate data quickly. They do not guarantee 1,000 useful examples, or exponential improvement: attempts can repeat the same mistake, and verification still costs compute and human attention.

A competent factory separates creation from judgment. A task may be valid but trivial. A fluent solution may violate a hidden condition. A high reward may expose a loophole in the evaluator.

Prime Intellect's General Agent is a useful boundary case. It reports 4,504 tasks across 1,040 domains and 8,159 tools, with a synthesizer and solver used to calibrate difficulty.^[6] The synthesizer runs offline to produce a fixed corpus for training. It is not online, continual self-improvement. That boundary permits repeatable inspection of the task supply.

Autodata proposes an agentic data scientist that creates synthetic data and meta-optimizes its generator across limited evaluated domains.^[7] It is a research result, not proof that self-directed production can replace conventional curation everywhere.

Agent traces are operational data: state, actions, tools, outcome

An agent trajectory is the record of one attempt at a task: what the agent observed, which actions and tools it used, what happened next, and how the attempt ended. An environment is the setting in which those attempts happen. Keeping both lets a lab connect a result to the conditions that produced it.

DeepSeek's V3.2 describes agent-environment synthesis, a sign that labs are treating interactive trajectories as training material rather than incidental logs.^[5] The strategic implication is narrow: agent capability needs records of action under constraints, so collection systems must preserve those constraints.

Consider a hypothetical code-repair example. The collector saves a repository version, an issue, the agent's patch, tool outputs, and test results. The grader checks whether the intended tests now pass. That produces evidence about an attempted repair, not merely another paragraph about programming. A passing test still proves only what that test checks, which is why the task and grader need inspection too.

Microsoft's Echoverse makes the verification problem concrete. It built 12 synthetic computer-use worlds with owned databases.^[10] Its final outcomes can be checked against database state, while its reinforcement-learning setup also uses GPT-4.1 language and vision judges. Microsoft reports that shallow replicas can harm results while deeper environments help. That bounded finding does not establish a universal ranking, but it shows why a screenshot or a success flag is thin evidence for an agent task.

Inside the product: what a lab actually receives

What does a lab actually receive? Harbor, the framework Ching points to, packages the instructions, workspace, and grading code into a runnable task. A typical Linux task package looks like this:^[3]

my-task/
  instruction.md          # the task presented to the solver
  task.toml               # configuration and metadata
  environment/Dockerfile  # one supported environment definition
  solution/solve.sh       # optional reference solution
  tests/test.sh           # verifier that writes the reward

The current documentation calls the prompt file instruction.md. A reference solution enables an oracle sanity check; it is optional in Harbor's format. The grader writes its result to a reward file. These files package the exercise, not a guarantee that its grading is correct.

Answer separation matters. The reference solution should not be part of the solver's ordinary inputs. Harbor supports a separate verifier environment, but shared-container verification is the default. Packaging alone does not certify protection against cheating; task authors must check what the solver can access and influence.^[3]

DeepSWE supplies a concrete example from Ching's Datacurve context. The July 2026 benchmark paper reports 113 original engineering tasks across 91 repositories and 5 programming languages. It uses human-written behavioral checks and was designed to test work beyond short, isolated code fixes.^[12] Here, DeepSWE refers to Datacurve's benchmark, not the earlier coding model with the same name. An evaluation release also does not establish which frontier labs trained on those tasks.

Longer work needs careful measurement. METR's time horizon measures task difficulty in human-expert completion time at a specified success probability, such as 50%. It does not measure how long an agent ran autonomously. Its task suite is concentrated in software, machine learning, and cybersecurity; the results do not imply equal reliability across office work.^[15] For data builders, the implication is to preserve meaningful dependencies between actions, rather than stretching an episode with unrelated steps.

The economics: why labs buy environments and still build their own

Ching's proposed reason for outsourcing is diversity: different vendors bring different experts and ways of constructing tasks. His larger economic point is that an environment's value lies in the useful learning it can generate, not just the cost of producing it.^[11]

An Epoch AI article based on 18 interviews adds a practical counterweight. Vendors can assemble task-building teams quickly and provide quality-assurance infrastructure. Labs also build internally to retain expertise, protect training priorities, and avoid supplier margins. Interviewees described $200 to $2,000 per task as a common range in their experience, with substantial variation. These are interview-reported prices, not a standardized market tariff or a valuation of learning quality.^[13]

The purchasing question is simple: what does this task teach that the lab cannot obtain as efficiently elsewhere? Cheap tasks that reward a shortcut can waste an expensive training run. Expensive tasks that duplicate existing coverage may add little. The supplier's job is to connect the price of the task to the value of the learning it supports.

Our analysis: the stronger supplier can explain why a task is useful, reproduce how it was graded, and revise it when an evaluation exposes a shortcut. Those capabilities give a buyer something a large task count alone cannot provide: a way to inspect what its training budget is buying.

Let's be clear: collecting more examples only expands the candidate pool. The business advantage comes from selecting examples that improve the model and catching those that reward the wrong behavior. The product is the learning those examples can support.

Research method: We used Grok 4.6 through Graff for hosted X and web-search leads, then checked the cited claims against papers, documentation, lab reports, and Daniel Ching's own published essay. Research announcements describe their own experiments; the strategic conclusions here are LLM Rumors analysis.

Last updated: September 6, 2026