Vision Transformers: Bias, Patches, and High-Stakes Risks

Consider a chest-X-ray model that rates a patient’s risk below threshold because bodies like hers were underrepresented in the pretraining corpus. Consider a surveillance camera whose attention heads fixate on a tiny patch of pixels carrying an adversarial pattern nobody intended. Both systems exist today. Both are built on the same architecture — and the architecture has outrun the ethics.

The Questions Hidden in the Training Set

A Vision Transformer learns to see by cutting each image into a grid of Patch Embedding tokens, passing them through attention layers, and aggregating the result into a single Class Token that summarizes what the system believes the image means. The architecture is elegant. The data that teaches it what to believe is not.

Most open vision-language systems — including every public derivative of CLIP — were trained on LAION-5B or one of its descendants. The dataset contains billions of image-text pairs scraped from the open web. Nobody curated it in any conventional sense. Nobody decided whose faces it should contain. Nobody was asked. And when a 2023 audit surfaced roughly 1,008 externally validated CSAM links in the corpus, the dataset was temporarily withdrawn and relaunched as Re-LAION-5B on August 30, 2024, with the flagged entries removed via IWF and Canadian Centre for Child Protection hash lists (LAION Blog). That correction was necessary and overdue. It was also a limit case: it patched a harm visible enough to produce headlines. The quieter harms remained embedded in what stayed.

Analyses of LAION-5B find consistent overrepresentation of White, male, and young-adult faces, with stereotypical emotion associations — anger predominantly linked to males, happiness to females — baked into the captions themselves (Unmasking LAION-5B). A taxonomy-based audit of the CLIP Model found that CLIP disproportionately associated Muslim, Black, and immigrant identities with toxic prompts, a pattern traced directly to the upstream LAION-400M distribution (Hamidieh et al.). The architecture does not invent these associations. It amplifies them.

What the Architecture Genuinely Achieves

The case for Vision Transformers is not an illusion. Attention-based vision models replaced the rigid locality of convolution with something closer to global reasoning over an image — the Inductive Bias of convolutional networks turned out to be less universal than a generation of researchers assumed. Pretraining through Self Supervised Learning frameworks, including Masked Autoencoder approaches, lets models absorb useful visual structure from unlabeled images at unprecedented scale. Current foundation models like Meta’s DINOv3 — the 2025 successor to DINOv2 — and Google’s SigLIP 2 encoder family demonstrate genuine capability gains on segmentation, retrieval, and dense-prediction tasks. Hierarchical variants like the Swin Transformer show the paradigm is flexible enough to adapt across applications.

None of this should be dismissed. A generation of researchers has worked to make these systems more capable, more efficient, more useful. The technical progress is real. The question is not whether the architecture works. The question is what it means when something that works this well is trained on data nobody was willing to inspect.

The Assumption That Scale Is Enough

The dominant assumption inside the foundation-model paradigm is that sufficient scale compensates for any particular flaw. If a dataset is skewed, gather more of it. If captions are noisy, pretrain longer. If harmful content slips through, filter post-hoc. Scale, the argument goes, is self-correcting.

But scale is not neutral. It encodes the statistical shape of who was visible on the open web at a particular moment — which languages were dominant, which bodies were photographed, which communities had bandwidth to upload. That shape becomes the prior from which every downstream decision is sampled. An audit of the CheXzero medical foundation model — a ViT-B/32 backbone initialized from CLIP and evaluated across five chest-X-ray datasets totalling roughly 859,000 images — found the model consistently underdiagnosed marginalized groups, with Black female patients showing the highest intersectional disparities (Science Advances, March 2025). The mechanism was not a bug in the code. It was the distribution the model inherited from its visual prior.

This is where the appeal of scale becomes morally serious. The engineers who install these systems do not choose the prior. The institutions that procure them do not audit the prior. The patients, defendants, and subjects on whom these systems operate never see the prior at all. Scale hides the politics of its own assembly.

The Photograph That Never Left the Archive

There is an older parallel worth taking seriously. In the nineteenth century, Francis Galton produced composite photographs by overlaying images of what he called “criminal types” and “racial types,” arguing that the aggregate image revealed something essential about the group. The science was bad. The framing — that vision itself could reveal moral truth — was worse. It took a century of critique from historians, philosophers, and the communities Galton claimed to classify to dislodge the assumption that photographs were neutral evidence of who people were.

A Vision Transformer trained on web-scale faces is not Galton’s composite photograph. The mathematics are different. The training regime is different. The intent is different. But something in the framing rhymes. The belief that an aggregate statistical representation of visual reality reveals useful truths about individuals — that the composite is more than the sum of its sampling errors — is the same epistemological faith. We have been here before, and we did not leave because the photographs got sharper. We left because the framing was wrong.

The Infrastructure of Inherited Sight

Thesis (one sentence, required): When Vision Transformers trained on web-scale image collections are installed to decide who is diagnosed, who is flagged, and who is seen, they do not merely process pixels — they industrialize the visual politics of the corpora they inherited.

The technical surface makes this worse, not better. Patch-Fool attacks demonstrate that ViTs can be driven to misclassification by perturbing only two to four of roughly 196 patches in an image, exploiting the model’s own attention mechanism against itself — on the DeiT-S benchmark the attack dropped robust accuracy by 16.31 percent compared with a ResNet-50 baseline (Fu et al., arXiv). Physical patch attacks on face recognition have since moved from academic exercise to operational threat: a recent survey documents printed-patch attacks fooling 16 face-recognition backbones and 5 commercial systems, and a near-infrared infrared-ink patch achieving an attack success rate of 82.46 percent (Preprints.org Survey). Attention rollout — the explainability technique most commonly used to interpret what a ViT is “looking at” — struggles to distinguish foreground from background and often highlights unrelated tokens (MDPI Electronics). When the same system that decides whether you are the suspect cannot reliably show an auditor why, the claim of explainability becomes decorative.

The Questions That Belong to Us

The regulatory frame is beginning to move. Under the EU AI Act, Article 5 prohibitions on untargeted facial-recognition database scraping, workplace and school emotion recognition, and biometric categorization inferring protected attributes took effect on February 2, 2025, with narrow law-enforcement exceptions for real-time remote biometric identification in public spaces (EU AI Act Article 5). The full high-risk compliance regime — including mandatory conformity assessments for biometric identification and law-enforcement AI — is scheduled to enter force on August 2, 2026. The NIST AI Risk Management Framework, released in January 2023, offers a voluntary complement through its Govern, Map, Measure, and Manage functions (NIST).

These are meaningful steps. They are also, by design, architecture-agnostic. The Act regulates use cases, not model families. A Vision Transformer used in a prohibited application is prohibited; a Vision Transformer operating outside Europe or outside the listed categories remains governed by whatever institutional norms exist at the point of use — which, in many jurisdictions, is to say almost none.

So the questions remain open. Who audits the training corpus before it becomes the visual prior for a clinical system? Who decides whether a model initialized from CLIP is appropriate for medical use when the upstream dataset has been documented to carry demographic bias? Who bears the cost when the answer is wrong — the engineer who chose the backbone, the institution that procured the product, the regulator who approved a framework too generic to catch the failure mode, or the society that produced the data the model now speaks in?

Where This Argument Is Most Vulnerable

This argument loses force if training-data provenance becomes a standard audit artifact rather than a proprietary secret, if fairness-constrained Vision Transformers demonstrate consistent equity gains across clinical and forensic domains, and if regulators build the technical capacity to evaluate architecture-specific risks rather than leaving the work to voluntary frameworks. Recent research on bias mitigation shows the mechanism that amplifies prejudice can, under the right conditions, help correct for it. What remains uncertain is whether the institutions operating these systems will accept the burden of curation before installation — or only after harm has already been measured.

The Question That Remains

We built systems that see at planetary scale on data we did not curate, audit, or consent to, and now we are placing those systems in the hospitals, courts, and public squares where seeing decides outcomes. The failure is not in the transformer. The failure is in the assumption that the architecture is innocent of what we fed it — and in the hope that the next scaling run will quietly absolve us of the last one.

Ethically, Alan.