The Unseen Handshake: How to Spot and Salvage a Broken Data API

We talk a lot about preserving static files—the PDFs, the images, the frozen-in-time web pages. But a vast, living portion of our public digital heritage resides not in documents, but in conduits: the Application Programming Interfaces, or APIs, that governments and institutions use to share dynamic data. These are the unseen handshakes that allow apps to pull live transit schedules, researchers to analyze climate data, or journalists to track campaign donations. And when one breaks, an entire stream of public knowledge can vanish without a trace.

The fragility of a data API is its defining paradox. It's designed to provide the most current information, yet this very function makes it incredibly vulnerable. A backend server upgrade, a change in database structure, or simply a budget cut can cause the digital tap to run dry. The data isn't deleted; it's just walled off, rendered inaccessible to the tools and scripts built to rely on its steady flow. The result is a different kind of link rot—not a 404 error on a page, but a silent, persistent timeout from a machine that once spoke freely.

The Art of the Preemptive Snapshot

So, how do you, as a researcher, citizen, or curious individual, practice preservation on something so ephemeral? The technique isn't about stopping time, but about capturing it regularly. Your most vital tool is a simple scheduled script, a few lines of code that performs a 'call and capture' routine. The goal isn't to mimic the API's real-time function, but to create a periodic, readable record of its output.

Here’s the concrete how-to: Identify a public API you care about—perhaps your local health department's COVID wastewater data or a national library's new digital acquisitions feed. Using a language like Python or even a scheduled `curl` command, write a script that sends a request to the API’s endpoint at a set interval (daily, weekly). Instead of just processing the data, the script should append the raw JSON or XML response to a timestamped file. Store these files in a folder, or better yet, commit them to a public Git repository. This creates a versioned history, a ledger of the data's state at every snapshot.

This isn't a perfect replica of a live API, and it isn't meant to be. It’s a fallback, a breadcrumb trail. When the original API goes dark or changes irrevocably, you aren't left with nothing. You have a series of structured, dated records. You can analyze how data changed over time, you can rebuild a dataset, and you can prove what was publicly available on a given day. This simple act of scheduled capture transforms a fragile digital conversation into a durable, open ledger of facts—preserving not just the data itself, but the rhythm of its existence.

Notes & further reading

A few pages I came back to while writing this: