Reproducibility or Surveillance: The Ethics of Versioning Every Dataset

Every serious machine-learning team now treats data the way software teams treat code: versioned, hashed, reproducible down to the byte. It is one of the most quietly responsible habits the field has grown into. It is also one of the least examined, because almost no one asks what it costs to remember everything, forever, on purpose.

The Commit That Cannot Forget

Ask an engineer why they version their training data and the answer comes back fast and confident: so we can reproduce results, trace a bad prediction to its source, and prove what the system learned when someone challenges its decision. All of that is true, and all of it is good. The harder question is the one nobody asks in the same breath. When you make a dataset immutable, you also make the people inside it immutable. Their records stop being entries that can be corrected, withdrawn, or allowed to fade, and become fixed coordinates in a permanent archive. The engineer optimizes for traceability; the person logged in that row never agreed to be traceable for the rest of their life. We tend to treat memory as a neutral good, as though the only ethical failure available to us is forgetting. But a system that cannot forget is not automatically more just than one that can. It is only more total. So the question worth sitting with is not whether we should be able to reproduce our models. It is what we owe the people whose lives we are freezing in amber in order to do it.

Why We Reached for the Ledger

The case for versioning everything is genuinely strong, and it deserves to be stated at full strength before it is questioned. Is data versioning necessary for AI accountability and reproducible audits? For any system whose decisions carry real weight, the honest answer is close to yes. Data Versioning tools like lakeFS treat a dataset the way Git treats a codebase, where every commit is a reproducible point in time you can branch from and return to, as the lakeFS documentation describes. The same instinct runs through Content Addressable Storage, where a file’s hash becomes its identity, and through lakehouse formats such as Delta Lake and Apache Iceberg that let you query a table as it existed at an earlier snapshot. Even Git LFS, the blunt baseline, exists because teams refused to lose track of which data produced which result. This is not bureaucratic hoarding. It is the spine of honest MLOps. When a model discriminates, you cannot diagnose Dataset Bias without the exact data it learned from. You cannot catch Data Leakage after the fact, or document how you corrected a Class Imbalance, if the evidence has already vanished. Europe’s AI Act leans hard in this direction: its Article 10 expects high-risk systems to rest on governed, well-managed datasets, and traceability is widely read into that expectation (EU AI Act Art. 10). Reproducibility, then, is not a convenience. It is much of what lets us hold these systems to account at all.

The Assumption Hiding in the Hash

Underneath that strong case sits an assumption so familiar we forget it is a choice: that remembering is always the more responsible act, that more provenance can only ever mean more accountability. Content-addressable storage makes this assumption physical. Immutability is the whole point. Change one byte and you get a different object, so the past can never quietly rewrite itself. But the same property that protects a result from tampering also protects it from mercy. Does immutable data versioning conflict with the right to erasure for personal data? It does, and not at the margins. The GDPR’s Article 17 gives a person the right to have their personal data erased when they withdraw consent or the data is no longer needed for its original purpose (GDPR Art. 17). An append-only history is, by construction, the negation of that right. Even the systems that try to soften the collision only delay it: time-travel and snapshot architectures keep the pre-deletion state for a configurable retention window, so a deleted record lingers in recoverable history until that clock runs out, as TechGDPR has documented. The data subject who exercised their right is still present in last quarter’s commit, still shaping the branch someone forked from it. We did not decide, case by case, to keep them. We simply built a system whose default was never to let go, and never asked whether that default should have been ours to set.

Forgetting Was Always Part of Justice

It helps to remember that forgetting is not a defect human societies merely tolerate. It is something they deliberately build. Statutes of limitations close the door on old claims. Juvenile records are sealed so that a childhood mistake does not harden into a life sentence. Ancient legal traditions wrote debt forgiveness into the calendar, precisely because a society in which every obligation is remembered forever becomes unlivable. The European right to be forgotten did not invent a new idea; it carried an old one into a new medium. Borges understood the stakes better than most engineers ever will. His character Funes, cursed with flawless memory, could forget nothing, and discovered that total recall was not wisdom but paralysis, an inability to generalize, to forgive, to move on. A culture that remembers everything about everyone is not more truthful. It is only less free. The reframe this invites is genuinely uncomfortable for anyone who builds data systems for a living. The interesting question is no longer how to remember more, or more faithfully. It is which acts of forgetting a just system should treat as features worth designing for, rather than as bugs to be engineered away.

The Burden of Proof Belongs on Retention

Thesis: The ethical default for dataset versioning should not be to keep everything forever, but to version what accountability genuinely requires and forget what human dignity demands, with the burden of proof resting on retention rather than on erasure.

Today that burden runs the other way. Keeping data is frictionless and feels prudent; deleting it requires justification, effort, and a little nerve. Flip the default and the whole shape of the question changes. Instead of asking what we might lose by deleting a record, we would have to argue what we actually gain by keeping each person in the archive, for how long, and who is permitted to look. Reproducibility does not, in fact, require immortal personal data. It requires that the relationships in the data, the patterns a model learned and the distribution it was trained on, remain documented and auditable. There is a real and underexamined difference between preserving the structure of a dataset and preserving the identifiable individuals inside it. The first is accountability. The second, past the point where any audit needs it, starts to look like surveillance wearing the uniform of good engineering. Naming that difference honestly is most of the work.

The Questions to Sit With Before August

This stops being abstract very soon. The high-risk provisions of Europe’s AI Act arrive in August 2026, only weeks from now, and they pull in two directions at once. The Act expects high-risk systems to log their activity and keep those records for a meaningful stretch of time, on the order of half a year for deployers under its record-keeping rules (EU AI Act Art. 12). Yet the same regulation, where it allows the processing of sensitive data specifically to detect and correct bias, expects that data to be deleted once the bias has been addressed. Even the law cannot quite decide whether the responsible act is to keep or to release. That tension is not a flaw waiting for a cleverer tool to resolve it; it is the moral shape of the problem, written into the statute itself. So the questions worth holding are not really technical ones. Who decides how long a person stays inside your history, and does that person have any say in it? When retention and erasure genuinely conflict, whose interest is quietly treated as the default and whose as the exception? And if a regulator, an auditor, and a data subject each asked you to justify a single commit, would you be able to give the same answer to all three?

Where This Argument Is Weakest

The honest part. This argument leans on a claim that may not hold for long: that immutability and erasure are truly irreconcilable. If machine unlearning matures into something dependable, a way to remove a record’s influence from a trained model rather than merely deleting the row, then much of this tension softens. For now it does not deliver that. Approximate unlearning methods leave residual traces, and the techniques are still maturing, as the European Data Protection Supervisor has cautioned (EDPS). The argument is also weakest wherever pseudonymization genuinely severs a record from a person, since a dataset that can be reproduced but never re-identified carries far less of the harm I am worried about. And this remains a perspective on values, not a reading of the law. If the tools learn to forget without losing the ability to account, the surveillance I am describing becomes a transition problem rather than a built-in fate.

The Question That Remains

We have learned to version our data so that nothing important is ever lost. We have not yet learned to ask whether some things should be allowed to disappear, and who we become when they cannot. When the archive remembers a person more faithfully than that person is permitted to forget themselves, is that accountability, or is it only power that has not yet been named?