The Ghost in the Spreadsheet: On the Tyranny of Structured Data
We are living in the golden age of open data, a time where governments and institutions proudly unveil vast troves of public records. The promise is seductive: raw, structured information, neatly arranged in CSV files and JSON blobs, ready for analysts and algorithms to parse. This is progress, we’re told. This is transparency. But in our rush to celebrate the spreadsheet, we have quietly accepted a dangerous premise: that for data to be truly ‘open,’ it must first be structured.
This received wisdom is a form of tyranny. It dictates that the messy, nuanced, and profoundly human context of information must be sacrificed at the altar of machine readability. A PDF of meeting minutes, scanned and OCR’d, is treated as a failure—a mere stepping stone to the ‘real’ data contained within its tables. But in that conversion, what is lost? The handwritten margin note from a citizen, the passionate inflection in a transcribed speech, the very layout of a document that signals its importance. The structure becomes a straitjacket, forcing the rich tapestry of public discourse into predefined columns that cannot hold its meaning.
The Illusion of Neutral Formatting
The most pernicious aspect of this dogma is the illusion of neutrality. A CSV file feels objective. It is rows and columns, numbers and text. But the choice of which data to include, which to exclude, and how to categorize it is an act of profound interpretation. It is a argument framed as a fact. By the time a dataset is published, the debate has already been curated, the boundaries of discussion pre-defined by an anonymous administrator’s schema. The structure, far from being a neutral vessel, is a powerful editorial voice that speaks in the language of cold, hard facts, making its biases all but invisible.
This obsession with structure also creates a new kind of gatekeeper. The technical ability to wrangle a massive SQL dump becomes a prerequisite for engaging with one’s own civic records. The journalist, the historian, the curious neighbor—those without data science skills are left with a ‘simplified’ dashboard or a pre-packaged visualization, a second-hand account of the primary source. True openness should lower barriers, not replace old, physical ones with new, technical ones.
We must push back against the idea that unstructured data is a problem to be solved. It is often the solution. It is the evidence of how we actually communicate, reason, and disagree. The future of open data shouldn’t be a choice between a scanned PDF and a sterile spreadsheet. It should be an and. It must involve the preservation of raw, un-massaged documents alongside their structured extracts, allowing the full, chaotic, human record to remain intact for those who know that truth often resides not in the cell of a table, but in the texture of the page.
Notes & further reading
A few pages I came back to while writing this: