The Cathedral and the Bazaar, Revisited: Open Data's Tale of Two Cities

Two decades ago, Eric S. Raymond’s famous essay used the metaphor of the cathedral and the bazaar to describe competing modes of software development. The cathedral was built carefully, silently, by a small group of master artisans behind walls. The bazaar was a bustling, chaotic, public affair, where contributions came from anyone and everyone. It’s a framework that maps surprisingly well, and perhaps even more literally, onto the contemporary landscape of open data and public records.

On one hand, we have the digital cathedral. This is the approach of the grand, centralized portal: Data.gov, the European Union Open Data Portal, or a major city’s official data repository. These are monuments to order and authority. They are built with significant resources, governed by strict metadata schemas, and presented with a clean, professional interface. The goal is comprehensive, vetted, and reliable. A researcher looking for national census data or a developer building an application on transit schedules would rightly seek sanctuary here. The cathedral offers the comfort of legitimacy; when you download a dataset from an official source, you trust its provenance.

But this trust comes with constraints. Cathedrals have opening hours, rules of entry, and a pace of their own. Publishing data through a major portal is often a slow, bureaucratic process. Datasets can become outdated, trapped in approval workflows, or siloed by the very structures meant to organize them. The architecture, while impressive, can be inflexible, slow to adapt to new types of data or emerging public needs.

This is where the bazaar flourishes. The data bazaar is the ecosystem of unofficial, crowdsourced, and guerrilla archives. It’s the Google Sheet shared among activists tracking local air quality, the GitHub repository where a journalist scrapes and cleans procurement data after a failed FOIA request, the community-moderated wiki that documents changes to a popular API that the company itself no longer properly archives. The bazaar is messy, decentralized, and driven by immediate, often passionate, need. Its greatest strength is its agility; it responds to crises and curiosities in real-time, unburdened by institutional inertia.

Of course, the bazaar has its own perils. Provenance can be murky, accuracy can be uneven, and longevity is never guaranteed. The Google Sheet can vanish if its sole maintainer loses interest. The GitHub repository might fork into competing versions. The bazaar lives on the passion of its participants, a resource that can be as ephemeral as the data it seeks to preserve.

The most resilient approach to open data isn't a choice between the two, but an acknowledgment of their symbiotic relationship. The cathedral provides the authoritative foundation, the canonical record we can all refer back to. The bazaar acts as a rapid-response team and an innovation lab, testing new ideas, highlighting gaps in the official record, and creating value from data long before it’s deemed "official." A wise civic body will not just build a better cathedral, but will also listen to the noise of the bazaar right outside its walls, incorporating its best discoveries and learning from its urgent demands. In the end, a healthy information society needs both the enduring stone of the cathedral and the vibrant, chaotic life of the bazaar.

Notes & further reading

A few pages I came back to while writing this: