Besides the scraping of millions of Delicious users, a small subset of archive team has formed URL team, dedicated to pulling down the content of URL shorteners. URL shorteners may be one of the worst ideas, one of the most backward ideas, to come out of the last five years. In very recent times, per-site shorteners, where a website registers a smaller version of its hostname and provides a single small link for a more complicated piece of content within it.. those are fine. But these general-purpose URL shorteners, with their shady or fragile setups and utter dependence upon them, well. If we lose TinyURL or bit.ly, millions of weblogs, essays, and non-archived tweets lose their meaning. Instantly. To someone in the future, it’ll be like everyone from a certain era of history, say ten years of the 18th century, started speaking in a one-time pad of cryptographic pass phrases. We’re doing our best to stop it. Some of the shorteners have been helpful, others have been hostile. A number have died. We’re going to release torrents on a regular basis of these spreadsheets, these code breaking spreadsheets, and we hope others do too.
Jason Scott: