The Forgotten Bridge: Salvaging Data from a Discontinued API

We talk a lot about archiving web pages, but the web’s skeleton is built on something more dynamic and fragile: application programming interfaces, or APIs. These are the unseen channels that let apps and services whisper data to each other. When a public data portal, a community map, or a research tool shuts down its API, it doesn’t always vanish with a 404 error. It often goes quietly, leaving behind a unique preservation challenge. The data isn’t presented on a page; it’s locked behind a door for which the key has been thrown away. Today, we’re going to walk through a technique for making one last, documented call to a dying API before the lights go out for good.

This isn’t about bulk scraping. It’s about creating a precise, reproducible snapshot of a single data structure in its native format. The core tool is `curl`, the humble command-line workhorse, paired with a simple text editor. Your goal is to construct one perfect API request and capture its raw response exactly as the service delivers it. Start by finding the API’s official documentation, if it still exists. Look for what’s called an “endpoint”—a specific URL that returns data—and its required parameters. Often, services require an API key, a unique identifier you include in your request. If you have one, this is your last chance to use it.

Crafting the Final Request

Open a terminal. Let’s say the endpoint is `https://api.publicdata.example/v2/water_samples` and it requires a parameter like `?zipcode=90210`. A basic `curl` command would be `curl 'https://api.publicdata.example/v2/water_samples?zipcode=90210'`. But we need to be thorough. Use the `-H` flag to include headers, especially `Authorization: Bearer YOUR_API_KEY` if needed. Crucially, add the `-i` flag to include the HTTP response headers in your output. These headers contain metadata like the date of the response, the content type, and rate limits—context that will be priceless to a future researcher.

The most important step is directing this output to a file. The full command might look like: `curl -i -H 'Authorization: Bearer YOUR_KEY' 'https://api.publicdata.example/v2/water_samples?zipcode=90210' > api_final_capture_20231027.txt`. Run it. You’ve now made your call. But you’re not done. Open the saved file and, at the very top, before the `curl` output, write a plaintext note. Document the exact date and time, the source of the endpoint documentation, the purpose of the data, and the full command you used. This human-readable preface turns a data dump into a preserved specimen.

This single text file is your bridge. It contains the request, the server’s metadata, and the raw JSON or XML payload. It’s not a user-friendly interface, but it’s the purest form of the data as it existed. Store it alongside a copy of the official API documentation (saved as a PDF or HTML file) in your archive. This technique is a digital act of witnessing. It acknowledges that the living, queryable system is gone, but by capturing its response in transit, you preserve a functional artifact. You save not just the data points, but the shape of the conversation that was once possible.

Notes & further reading

A few pages I came back to while writing this: