The Ghost in the Spreadsheet: When Open Data Erases Its Human Source

We celebrate open data for its clarity, its neat rows and columns promising objective truth. A dataset is clean, machine-readable, and free from the mess of human bias—or so we tell ourselves. But in our rush to liberate information, we often perform a quiet act of erasure: we sever the data from the people who created it.

Consider a simple, powerful example: a municipal database of building permits. Released as a CSV file, it’s a treasure trove for urban researchers and civic watchdogs. It tells you the date, the address, the type of work, the estimated cost. But it doesn’t tell you about the young couple arguing with a contractor over the price of a new deck, their voices hushed so the neighbors won’t hear. It doesn’t capture the weary sigh of a city clerk, processing the fiftieth form of the day, their handwriting becoming a little sloppier with each entry. The data is a shadow; the human drama that cast it is lost.

This isn’t a failure of the open data movement, but a inherent characteristic of its most common format. The process of making information 'open' necessitates standardization. Stories, emotions, and the sheer friction of lived experience are the first things to be filtered out. We are left with a brilliant, useful, but ultimately hollow record. It answers the 'what' with stunning efficiency, but it is utterly silent on the 'why' and the 'who'.

The Archival Silence of the Machine-Readable

This creates a peculiar kind of archival silence. Traditional archives often preserve the human trace: a coffee stain on a letter, a marginal note, the hesitant script of a novice. These imperfections are data points in themselves, telling a richer story. The machine-readable dataset, in its quest for perfection, sanitizes this history. It presents the world not as it was lived, but as it was bureaucratically codified.

The danger is that we begin to mistake the clean spreadsheet for the complete reality. We analyze the correlations and trends and believe we understand the event itself. We see a spike in permits for a certain neighborhood and call it 'gentrification,' a clinical term that masks a thousand individual stories of displacement, ambition, anxiety, and hope.

The call, then, is not to abandon open data, but to consciously supplement it. True preservation means archiving not just the final, tidy output, but also the messy human inputs wherever possible. It means linking that permit database to the scanned, handwritten application forms, or the audio recordings of public planning meetings. It means acknowledging that data is not born pristine; it is a product of human labor and human context. The most honest archive is one that preserves not just the ghost, but the body that made it.

Notes & further reading

A few pages I came back to while writing this: