The Museum's Method: On Applying Provenance to Public Datasets
In the hushed halls of an art museum, the most critical information isn't the painter's name or the date on the placard. It's the story scrawled in the registrar's ledger: the unbroken chain of ownership, the journey from the artist's studio through various collections to its present home. This record of origin and custody, known as provenance, is the bedrock of authenticity and value. It’s a concept that feels centuries removed from our world of API endpoints and CSV dumps, yet it holds a powerful, untapped lesson for the stewards of public data.
We often treat open datasets as static, finished products. A file is released, we ingest it, and we trust its contents. But data, much like a painting, has a life. It is created, modified, aggregated, corrected, and versioned. Without its own form of provenance, a dataset can become a ghost—a thing without a clear history, its authenticity and reliability impossible to verify. How do we know if the figures from a 2018 economic survey are the ones originally published, or if they were silently amended in 2021? The difference matters, and our current methods of tracking it are often woefully inadequate.
Curating the Chain of Custody
What if we borrowed the museum's meticulous practice? Applying data provenance would mean treating every transformation as a documented event in the object's life. It would require systems that don’t just store the final data point, but also cryptographically sign the who, what, when, and why of every change. This isn't merely version control; it's a cultural shift towards treating data as a curated artifact with a biography.
This approach would fundamentally alter how we trust public information. A citizen examining civic spending records could trace a specific line item back through its various iterations, seeing which office corrected a typo or updated a figure after an audit. Researchers could know with certainty that the climate data they’re using to model future trends is precisely the same dataset referenced in a seminal paper from a decade ago, not a subtly altered successor. The data’s value would be intrinsically linked to the verifiability of its history.
Adopting this mindset from museology wouldn't be simple. It demands new tools and a commitment to transparency that makes many institutions uncomfortable. But the reward is a foundation of trust built not on blind faith, but on a verifiable, unbroken chain of evidence. In the end, the goal is the same for both the curator and the data archivist: to preserve not just the object itself, but the truth of its story.
Notes & further reading
A few pages I came back to while writing this: