The Presumption of Impartiality: When Archivers Choose a Side

In the ethics of web archiving, one principle is held as sacrosanct: impartiality. The ideal archiver is a neutral observer, a dispassionate collector who preserves the digital record without bias, like a camera automatically snapping a picture of everything that passes before its lens. This is the received wisdom we cling to, a comforting notion that the archives we build are objective mirrors of the web. But this is a fallacy, and a dangerous one. The act of archiving is not, and can never be, impartial. It is, from its very inception, an act of selection, and therefore, an act of power.

Consider the starting point: the URL. Before a single byte is saved, a choice is made. An archiving institution, be it a national library or a community project, decides which websites are worthy of preservation. This curation is not a technical necessity; it is a value judgment. It answers the question: what is important enough to remember? The selection criteria, often hidden in internal policy documents, are the first and most profound expression of bias. They decide that the website of a major political party is archival-worthy, but perhaps not the sprawling, vibrant forum of a niche online community. We save the cathedral, but we let the bustling, messy marketplace of ideas crumble into the digital ether.

This bias extends to the very architecture of the web crawler. A crawler follows links. It is a tool that maps a network, and in doing so, it privileges the well-connected. Sites that are heavily linked to—the hubs of the network—are captured deeply and frequently. The obscure blog, the personal portfolio, the digital art project tucked away in a forgotten corner of the web, these are the leaves on the tree, often missed. The archive, by its technical design, naturally reinforces existing power structures. It is far better at preserving the roar of the mainstream than the whisper from the margins.

Furthermore, the practical constraints of preservation create their own hierarchy of importance. Storage is not infinite. Bandwidth is not free. Faced with these limitations, archivists must make choices about frequency and depth. A news website might be captured every hour, while a site documenting local history is captured once a year. This temporal bias creates a record where the frantic churn of the 24-hour news cycle is preserved in high definition, while slower, more deliberate cultural shifts are documented with blurry, intermittent snapshots. We are building an archive with a hyperactivity disorder, over-documenting the immediate while neglecting the gradual.

This is not a critique of archivists, who perform heroic work under difficult circumstances. It is a critique of the myth we tell ourselves about their work. By pretending our archives are impartial, we grant their contents an unearned authority. We risk treating the archived record as the complete and objective truth, when it is, in reality, a curated collection shaped by a series of conscious and unconscious choices.

The path forward is not to abandon the goal of fairness, but to abandon the pretense of neutrality. We need archives that are transparent about their biases. We need collection policies that are public and debated. We need to actively seek out the digital ephemera that crawlers naturally miss. An honest archive doesn't claim to be a perfect mirror. It is more like a library assembled by a thoughtful, fallible, but intentional curator. It acknowledges its gaps, explains its priorities, and invites us to understand not just the data it holds, but the story of why it was chosen to be held at all. Only then can we read the archive with the critical eye it requires.

Notes & further reading

A few pages I came back to while writing this: