The Ghost in the Spreadsheet: On the Untold Lives of Public Data Fields

We talk about open data as if it were born pristine and pure, a perfect digital clone of a real-world process. A city releases a dataset of building permits, and we see rows and columns. We see dates, addresses, permit types. We see a tidy, logical system. But what we rarely see is the ghost in the spreadsheet: the human story behind each field.

Consider a simple column header: ‘APPLICATION_STATUS’. To a database, it’s a string of text, a categorical variable. To the person entering the data, it’s a story condensed into a dropdown menu. ‘Approved’ might mean a family’s dream addition is finally greenlit after months of anxious waiting. ‘Pending’ could represent a city planner’s overstuffed inbox, a file physically sitting under two others on a dusty desk. ‘Rejected’ might hide a tale of a misunderstood zoning code, a frustrating conversation, a dream temporarily halted. The data field is the end of that story, the period on the sentence. The human context—the sigh of relief, the grumble of frustration, the bureaucratic confusion—is the text that was erased before the final draft.

This erasure isn’t malicious. It’s a necessity of scale. Governments cannot record the emotional weight of every transaction. But in our rush to analyze, to aggregate, to map, we risk accepting the field as the whole truth. We build dashboards that track ‘efficiency’ based on the time between ‘SUBMISSION_DATE’ and ‘DECISION_DATE’, blind to the human realities that dictate that timeline—a holiday, a staff illness, a complex case that required more attention.

Digital preservation often focuses on saving the bits themselves, ensuring that ‘APPLICATION_STATUS’ remains readable for centuries. But a far more difficult, perhaps impossible, task is preserving the meaning behind the field. The institutional knowledge of why that dropdown menu has five specific options and not six. The memory of the software update that changed the field name from ‘STATUS’ to ‘APPLICATION_STATUS’ for clarity, inadvertently breaking a decade’s worth of automated scripts.

This is the ghost we must learn to listen for. When we use public data, we are not handling inert facts. We are handling the fossilized remains of countless human decisions, compromises, and moments. The data is not the reality; it is the shadow reality casts. To truly understand it, we must acknowledge the invisible hands that shaped it, the stories compressed into every cell, and the quiet hum of human experience that lies just beneath the surface of the spreadsheet.

Notes & further reading

A few pages I came back to while writing this: