Many evaluations end after a single answer. Real work does not. It involves changing inputs, partial failures, and decisions that must remain coherent hours later.
What long tasks expose
Long tasks reveal whether a system can preserve state, notice contradictions, and choose when to ask for help. These capabilities rarely appear in isolated prompts.
Design the environment
A useful benchmark includes tools, constraints, and a meaningful definition of done. The task should resemble the work people actually care about.
