The Hidden Workforce Behind AI: Labor Ethics and Rater Wellbeing in Human Evaluation

We have become fluent in the language of AI ethics. We debate bias in datasets, hallucination rates in production, the concentration of power in foundation models. What we rarely examine is the supply chain that begins before the model trains — the long chain of human attention and human judgment that makes machine evaluation possible at all.

The Labor Nobody Names

Human Evaluation for AI sits at the foundation of almost every major AI system in use today. Preference data from comparative rating sessions shaped the behavior of conversational AI. Quality judgments from annotation campaigns defined what “helpful” and “harmful” mean inside these systems. Even automated evaluation frameworks like SWE Bench rely on human judgment to curate the test cases and validate the evaluation logic itself. And the Rubric Design choices that govern what raters are asked to consider — which dimensions matter, which edge cases count — arrive to workers as finished parameters in a task specification, shaped entirely by people who will never perform the work.

The humans doing this labor are almost entirely absent from how we discuss AI development. Their wages don’t appear in model cards. Their working conditions don’t feature in capability reports. The questions that bear most directly on their wellbeing — what content they are exposed to, for how many hours, with what psychological support — rarely surface in the research that celebrates the alignment advances their work made possible.

Workers hired by a subcontractor to help train OpenAI’s ChatGPT earned between $1.32 and $2 per hour depending on seniority, according to reporting by TIME (January 2023). Research analyzing Amazon Mechanical Turk found a median effective hourly wage of $2.83 for paid tasks — and just $1.77 when the unpaid time spent searching and reading task descriptions is included (CHI 2018 paper). Only 4% of MTurk workers earned at or above the US federal minimum wage. The Equidem “Scroll. Click. Suffer.” report documented more than 60 incidents of serious psychological harm — including PTSD, depression, panic attacks, and suicidal ideation — among 76 workers surveyed across Colombia, Ghana, and Kenya.

These figures have been published. They sit in remarkable tension with an industry that positions itself as building tools for human benefit.

The Case for the Current Architecture

The efficiency argument deserves to be stated in its strongest form, because the people who make it are not all acting in bad faith.

Crowd-sourced annotation allows AI labs to gather preference data at a scale and variety that no in-house team could match. It provides flexible income to workers in economies where the alternatives are frequently worse — less stable, less accessible, or requiring physical presence that annotation work does not. The researchers who design these annotation campaigns often believe genuinely in the value of what they are asking people to do, even if they rarely meet the people doing it. And there is an empirical point underneath the efficiency claim: Cohen's Kappa scores improve when diverse rater pools are combined with well-designed rubrics, because scale enables the kind of demographic diversity that captures genuinely different perspectives on the same content.

Oxford Fairwork’s analysis of platform labor documents this complexity directly: none of the fifteen annotation platforms it studied scored above the bare minimum on fair work standards — and yet workers continue to rely on them, often because the available alternatives are less reliable or more precarious, not because the platforms are good (Oxford Fairwork).

The defense of the current system, at its most honest, is not simply greed or institutional indifference. It is a genuine belief that scale serves impact, that globally distributed annotation labor is more efficient than local alternatives, and that efficiency, ultimately, serves the users who benefit from better AI.

The Assumption Hidden in Scale

There is a premise inside that argument that almost never gets named.

The efficiency framing treats the aggregate as the relevant moral unit — aggregate model quality, aggregate benefit to users, aggregate value created. The costs are distributed downward: to the workers processing content, to the governments of countries where a $2 hourly wage represents a labor market advantage rather than a scandal. The assumption, implicit but structurally embedded, is that the people who are closest to the harm are also the people whose preferences about that harm matter least.

Consider what this looks like from the inside. Platforms used for AI data work paid as little as $1–$2 per hour, with some tasks priced under a cent per completion (Brookings). Some workers processed up to 700 sexually explicit or violent texts per day. In March 2024, Kenyan labelers working through Remotasks were suddenly locked out of the platform without receiving wages they had already earned — a payment cut-off that received almost no attention in the mainstream AI press (Brookings). The ELO Rating systems used in platforms like Chatbot Arena aggregate thousands of preference judgments into leaderboard scores that drive product decisions worth enormous sums — while the workers who generated many of those judgments operate under NDAs that prevent them from describing their own experience publicly.

Equidem’s research found that 75 out of 105 Colombian workers and 68 out of 110 Kenyan workers declined to participate in their study, citing fear of NDA violations. These are people earning under $2 per hour, classified in most cases as independent contractors rather than employees, with no union representation. The legal architecture of their relationship with the platforms they work for is designed — whether intentionally or not — to make accountability structurally difficult to assign.

That is not efficiency. That is the externalization of moral cost.

What Supply Chains Already Taught Us

We have been here before.

The garment industry, conflict mineral supply chains, the agricultural labor systems that underpin food markets in wealthy countries — each followed the same structural logic. A complex chain of intermediaries distributed profit upward and concentrated risk downward, with legal architecture at every node that made accountability difficult to trace. The finished product was celebrated. The conditions of its production were not asked about, because asking would have been inconvenient, and the chain was long enough that inconvenience could masquerade as ignorance.

The lesson of those industries was not that efficiency is wrong. It was that a supply chain can be efficient and unethical simultaneously, and that the ethical failure only becomes visible when someone traces the chain backward — from the product to the person who made it possible.

For AI development, that trace begins with a question that almost no benchmark paper asks: who actually sat with Label Studio open on their screen for eight hours, reading content that most researchers would find difficult to encounter once? What was the task specification? What happened when they developed trauma responses? An ILO analysis of AI data worker conditions explicitly identified PTSD, depression, and reduced capacity for empathy among annotation workers, and named the labor arrangement as a form of invisible wage subsidy for AI systems whose market value accrues entirely upstream (ILO).

ILO Convention 193, adopted on June 12, 2026 by a vote of 406 to 8 — with the United States voting against — covers platform workers including those performing AI data labeling and content moderation. It is the first international instrument of its kind to address their conditions explicitly — adopted, though as of this writing not yet ratified by individual countries. The fact that it passed by that margin, with the US among the eight opposing votes, is itself a data point about which actors in the world regard this as a labor rights question and which prefer it to remain an efficiency question.

The Alignment Gap We Don’t Call a Crisis

Thesis: The ethical legitimacy of AI alignment depends on whether the humans doing the aligning are treated as ends in themselves — not as undifferentiated inputs into a pipeline whose output we celebrate while its labor conditions remain invisible.

There is a structural wrinkle here that deepens the concern. LLM as a Judge frameworks are increasingly offered as the solution to the problem of annotation scale: replace human raters with AI raters, because AI is cheaper and faster. This is framed as a technical advance. But if we adopt automated evaluation partly to avoid the moral complexity of employing humans in difficult conditions, we have resolved an ethical problem by making it invisible rather than by addressing it.

Research on AI red-team evaluator wellbeing found that adversarial engagement creates psychological harms that parallel those of conflict photographers and mental health professionals — harms “uniquely tied to adversarial strategies,” the researchers note (arXiv 2504.20910). These harms do not disappear when the task routes through an LLM. They transfer — to the humans who generate the adversarial training data, and to those who review the cases the automated judge fails. The moral weight relocates. It does not evaporate.

The alignment research community has sophisticated conversations about reward hacking, preference data quality, and the difficulty of specifying what “good” means for an AI system. The labor conditions of the people who operationalize that specification — who translate abstract values into labeled examples, in annotation environments they did not design — belong inside that conversation. They do not currently appear there in any systematic way.

The Questions That Follow

The resistance to this framing usually arrives at the same place: yes, but what would you actually do differently? Structural critique without structural remedy, the argument goes, is rhetorical rather than useful.

This seems like a deflection more than a refutation. The garment industry question — what would you actually do differently? — eventually produced auditing standards, supply chain disclosure requirements, and in some countries, legal frameworks that assigned liability upward through the chain. None of these solutions appeared immediately, and none were complete. They appeared because the question was asked persistently enough that avoiding it became more expensive than engaging with it.

The equivalent moves in AI are not mysterious: wage transparency in annotation supply chains; psychological support and exposure limits for workers processing harmful content; worker input into annotation guideline design rather than receiving guidelines as fixed parameters; employee classification rather than contractor classification for workers performing ongoing work. These are not radical demands. They are the same demands that moved other industries from their equivalent of the current moment.

What is lacking is not imagination. What is lacking is the institutional pressure that makes accountability a cost rather than an option.

Where This Argument Is Weakest

The most honest challenge here is empirical. I do not know whether every worker in this labor market would prefer that it not exist. The Brookings analysis found workers who regarded annotation work as a meaningful entry point into a digital economy that would otherwise be inaccessible to them. That preference — imperfect arrangement over no arrangement — is a real fact that ethical analysis has to accommodate, and not by dismissing it.

The argument is also weakest when it implies a single, unified exploitative intent. Most researchers who design annotation campaigns are thinking about data quality, not supply chain ethics. The problem is not malice. It is that when you work at scale on a technical problem, the ethical conditions of the labor that enables it can become invisible by default. Default invisibility is its own kind of ethical failure — but it is a different failure from intentional extraction, and conflating the two makes the argument easier to dismiss.

And finally: if the transition toward automated evaluation reduces the human annotation workforce over time, the question of what labor reform would accomplish becomes more complex. Better conditions matter regardless of whether the workforce contracts. But the urgency of the question shifts.

The Question That Remains

The AI systems we use daily were shaped, in part, by people whose names appear in no acknowledgment section and whose working conditions few product teams have examined. The question that follows from that fact is not technical. Who carries the moral weight of a supply chain that has been made deliberately difficult to see — and what does it mean to build “beneficial” AI on a foundation we have chosen not to look at?