The Apothecary's Chest: On Provenance and the Data Chain of Custody
In the hushed back rooms of old apothecaries, remedies were stored in drawers and jars, each bearing a label not just of contents, but of origin. The label might name the merchant who supplied the bark, the ship that carried it, the region where the tree grew. This wasn't mere record-keeping for inventory; it was a matter of life and death. The efficacy, and safety, of a treatment depended on this unbroken chain—the provenance. A jar of 'cinchona bark' was worthless, even dangerous, if you couldn't trace its journey from a specific hillside in Peru to your shelf. In our work with public records and digital preservation, we've been curiously lax in applying this ancient, vital standard.
We are excellent at collecting and publishing data, often celebrating its 'openness' as an end in itself. Yet, we frequently present it like a pile of unlabeled herbs on a market table. A dataset is released: a CSV file of city contract awards, a JSON dump of environmental readings, a WARC file of a crawled website. The 'what' is clear. The 'who, how, when, and why' behind its journey to us is often a ghost story, hinted at but never fully told. Who collected the raw figures? On what authority and with what methodology? What transformations were applied—was a column filtered, a value averaged, a format migrated? Each of these steps is a drawer in the apothecary's chest, and losing the label for any one of them compromises the whole.
The Silent Transformations
The peril is in the silent transformations. A historian in 2124 examining a 2024 web archive might see a perfectly rendered page, but have no record that it was reconstructed from a broken JavaScript framework by an emulation service, subtly altering its behavior. A policy analyst using a 'cleaned' public spending dataset will not see the subjective decisions that labeled certain entries as 'outliers' and removed them. We have focused on preserving the specimen, to borrow from the biologist, but often discarded the notes scribbled in the field notebook about the weather, the soil, and the collector's uncertainty.
Web archiving and digital preservation need an apothecary's ledger. This goes beyond basic metadata schema. It requires a cultural shift to document the process as sacredly as the product. Every migration, every normalization script, every decision to exclude a 'rotted' link should be recorded and linked, creating an auditable trail. This chain of custody isn't bureaucratic bloat; it's the antidote to digital doubt. It allows future users to judge not just the data, but its journey, to understand its strengths and its latent frailties.
The apothecary knew that trust in the remedy was built upon trust in the chain. For data to be truly authoritative—to be a foundation for policy, for justice, for history—it must carry its provenance with it, not as a footnote, but as its core constitution. Our digital chests are full, but we must inscribe each drawer. Otherwise, we are handing future generations potent but unmarked vials, asking them to trust contents whose origin story is already forgotten.
Notes & further reading
A few pages I came back to while writing this: