The Autumn Harvest: On the Ephemeral Nature of Open Civic Data

There is a particular slant of light in autumn, as Dickinson noted, that speaks of the year’s turning. It’s a light that illuminates change, that makes the transient beauty of a crimson leaf seem all the more precious because you know it will soon be gone. This seasonal shift has me thinking about another kind of ephemera: the live data feeds from our city governments—the real-time bus locations, the constantly updating energy usage maps, the active construction permits. We treat this data as a perennial spring, a constant, gushing fountain of now. But what happens when the first frost comes?

Unlike a scanned, PDFed, and archived public record—a static snapshot committed to a server—this live data is a performance. It exists only for the moment it is called. A bus’s GPS pings its location; an API serves that point on a map; a user sees it. Then, a second later, it is gone, replaced by a new point. We harvest this data for immediate use, for planning our commutes or monitoring power grids, but we rarely store the sheaves. We are content to watch the wind move through the field without ever considering that the same configuration of stalks will never occur again.

The Granary of the Now

This presents a unique and often overlooked challenge for digital preservationists and open data advocates. Our tools and philosophies are often geared toward preserving the finished document, the final report, the scanned image. We are excellent librarians of the definitive. But we are poor naturalists of the continuous. The lived experience of a city, as told through its real-time data, vanishes into the past with every passing millisecond, largely unrecorded.

How do we archive a moment? Why would we even want to? The value isn’t in any single data point of a bus’s location, but in the patterns that emerge over time. The chronicle of a transit system’s delays on a rainy Tuesday, the ebb and flow of energy demand across a heatwave, the shifting patterns of public life during a festival—these are the rich stories told by the aggregate of countless ephemeral data points. Without a conscious effort to capture and preserve this flow, we are writing a history of our cities with only the chapter headings and losing all the prose.

The autumn reminds us that nothing is permanent, that cycles require both growth and decay. It asks us to consider what we gather before the winter. As advocates for open knowledge, perhaps our focus should broaden from just ensuring the data door is left unlocked. We must also consider building the granaries. We need to champion initiatives that don’t just publish live data feeds but also automatically archive their own output, creating historical mirrors of the city’s pulse. This isn’t about hoarding, but about stewardship—recognizing that the ‘now’ is the future’s history, and that the most fleeting data often tells the most human story.

Notes & further reading

A few pages I came back to while writing this: