The Static Seed and the Sprawling Forest: On Cataloging a Public Domain Library
There is a quiet tension in the heart of open data, a philosophical rift that becomes sharply apparent when you try to archive something as seemingly straightforward as a public domain library. Two methods, two opposing ideals, present themselves. One is the static seed, a perfect, bounded snapshot. The other is the sprawling forest, a living, breathing index of what is already out there. Recently, I attempted to preserve a local historical society’s digital pamphlet collection and found myself caught between these two worlds.
The static seed approach is beguiling in its neatness. You identify a corpus—say, 500 digitized pamphlets from the 19th century hosted on a single server. You use a tool to scrape the site, downloading every PDF, JPG, and HTML index page. The result is a tidy package, a WARC file or a folder on a hard drive, that contains the entire collection as it existed on a specific Tuesday afternoon. It is a seed vault, a sealed unit of knowledge, protected from the rot of bit decay and the whims of webmasters. The integrity of the original collection is preserved, its internal logic intact.
Contrast this with the method of the sprawling forest. Instead of replicating the society’s bespoke website, you might instead catalog each pamphlet by its URL and submit those URLs to a massive, general-purpose web archive like the Internet Archive. The collection, as a singular entity, dissolves. Its components are absorbed into the vast, global archive, accessible only by searching for them individually. The original curation is lost, but each item gains a new kind of immortality, backed by the robust, distributed infrastructure of a major preservation project.
This is the core of the dilemma. The static seed values the curator’s intent. It preserves the context: how the pamphlets were arranged, the introductory texts written by the archivists, the specific sequence that told a story. It is a digital preservation of a physical act of collection. The sprawling forest values maximum accessibility and redundancy. It cares less about the narrative of the collection and more about the survival of the individual items. It bets on the power of search and the resilience of a network to ensure that a curious student, fifty years from now, can still find the pamphlet on the town’s centennial celebration, even if they can no longer see how it related to the one about the founding of the grange hall.
My own project stalled for weeks as I wrestled with this choice. Downloading the entire site felt like creating a beautiful, fossilized diorama—accurate, but isolated. Submitting the links felt like scattering the seeds to the wind, trusting a system I couldn't control to nurture them. In the end, I did both. The seed sits on my server, a perfect artifact of a moment. The forest, through the Internet Archive, now holds the pamphlets within its endless branches. The first approach preserves a specific memory; the second ensures the information survives. In the ledger of open data, perhaps the most honest entry is the one that acknowledges the need for both the preserved specimen and the wild, untamed copy.
Notes & further reading
A few pages I came back to while writing this: