The Summer Harvest: On Gathering Public Data Before the Rot Sets In
There is a particular urgency to a high summer afternoon. The air is thick, the light is golden, and the produce at the market is at its peak of ripeness. It’s a moment you can’t freeze, only savor. This seasonal tension—the race to gather the bounty before it spoils—is one I’ve come to recognize in my own work. For those of us who care about public data, this is also our harvest season. Not a harvest of corn or tomatoes, but of information, and the window for gathering it is often just as fleeting.
I’m not talking about the grand, federally mandated open data portals, though they have their place. I mean the small, local, and profoundly human records: the planning board minutes uploaded as a Word doc to a small town’s website; the unique dataset a passionate city official built in a Google Sheet to track park maintenance; the PDF of public comments on a new bike lane. These are the ripe fruits of civic engagement, often created by individuals who are focused on utility, not perpetuity. They are posted to serve an immediate need, with little thought for their survival beyond the next website redesign or software update.
Like a farmer who knows a storm is coming, the web archivist and data scraper feels this pressure instinctively. We see a well-structured, readable dataset on a municipal site and know it is vulnerable. The official who created it might retire. The grant funding the project might expire. The new CMS might not play nice with old file formats. The rot of obsolescence and link decay sets in quietly, often completely unnoticed by the public the data was meant to serve.
The Imperative of the Season
So we harvest. We write scripts to gently pull down these spreadsheets. We configure crawlers to capture the context of the web pages that host them. We do it not because the data is needed today, but because we know that a historian, a journalist, or a curious citizen might need it in five or ten years. We are putting up preserves for the long civic winter. This work is a quiet, continuous act of faith in the future’s right to interrogate the past.
This seasonal metaphor shifts the framing of digital preservation from a dry, technical process to something more fundamentally human. It is an act of tending. It requires us to know the landscape, to recognize what is ripe and valuable, and to act before the opportunity is lost to the silent frost of a server migration. The next time you download a clean, clear set of local records, consider the summer urgency behind its preservation. Someone, somewhere, looked at that data and saw not just numbers in cells, but a perfect, ripe fruit, and they rushed to gather it in before the rot could set in.
Notes & further reading
A few pages I came back to while writing this: