[This was originally posted on August 13, 2011, but had to be "reconstructed" following the crash of a MySQL server at my Web hosting service.]
About this time last summer, I wrote three back-to-back posts about the URL-shortening service bit.ly, now routinely re-directed — how ironic! — to bitly.com and self-referenced as simply “bitly.” The posts were entitled “Soooo big!” Counting on bit.ly and were focused on estimating when bitly might run out of unique URLs:
● Part I calculated that the available 62 characters (26 upper- and 26 lower-case letters, plus 10 digits), taken six-at-a-time with repetition allowed, yielded a pool of 62^6 = 56,800,235,584 unique hash strings available for assignment in URLs.
● Part II presented data from compete.com on unique visitors per month (the only data then available) and laid out the analytics and assumptions from which I calculated that with the upward growth in traffic which bitly was then experiencing, it would be 278 months (a bit more than 23 years) from initiation of the service in July 2008 until exhaustion — roughly 2030. In keeping with the casual nature of all this, I used a simple polynomial model, even though the likely long-term pattern might have been better simulated with a sigmoid.
● Part III offered some suggestions for postponing this day of reckoning. For example, bitly could adopt 7-character strings, although that would implicitly reduce the size of a Tweet even further; or issue strings in other character sets — Greek, Cyrillic, Hebrew, math symbols, even Dingbats; or see if Estonia would be amenable to hosting hu.ee, dew.ee, and lou.ee; or just wait until nearer 2030 for the inevitable introduction of the iLobe implant with content pushed over VerizoNeural.
During the subsequent year, I didn’t think any more about the URL-exhaustion problem, although I used bitly a couple of dozen times to create links for my own tweets or emails. Today I decided to look again at bitly’s unique-visitors data on compete.com, which shows end-of-month counts spanning the past year, currently June 2010 through June 2011. That turned out to be a perfect dovetail with the previous data for May 2009 through May 2010. This fortuitous timing also produced a big surprise, however — traffic has been generally downhill since May 2010, which was bitly’s only month in excess of 10 million unique visitors.
Here is the expanded version of the graph; for comparison, the original is in Part II or separately here.
So what to make of this? There are four modest upticks in traffic during the past year, but the overall trend is definitely down. There was also a downturn evident in the original, from November to December 2009. It’s tempting to think that this early modest change was related to introduction of URL shorteners by Facebook and Google in mid-December. Subsequently, I found reports of, for example, Facebook blocking bitly posts in July 2010 and December 2010. All such reports were related to spam and malware problems at the original sites to which bitly was redirecting users, not anti-competititve practices. My interest is not in detailed reasons for bitly’s traffic, however, so I made no attempt to find one-to-one stories to explain each downturn (June, October, and December 2010, March and June 2011).
My takeaway is that there are fundamental factors outside of anything I originally considered and that they may (or may not) have already mitigated the URL-exhaustion problem. The good news for bitly’s business model is that the bitly Pro for Enterprise white-label service seems to have taken off. I use it on The New York Times at least once a week to send a story link via the bitly-powered site nyti.ms, and there are hundreds, perhaps thousands, of other corporate users. The only thing that was not clear to me is whether the 6-character paths or hash strings are held in common by bitly and bitly Pro for Enterprise. If the latter has a completely separate pool, then the apparent drop in traffic is actually a diversion to other bitlyverses, and the 2030 reckoning will recede. But a simple test with Bruce Feiler’s story, ‘Our Plugged-in Summer’, from yesterday, strongly suggests that it is all just one big pot, as you can try for yourself with these respective links: nyti.ms and bit.ly. Not surprisingly, the same string, “pfUOj2,” appended to bitly’s j.mp domain also takes you to Seiler’s story.
These identical results imply substantial “leakage” of traffic in the past year. In fact, a straight-line projection with the last 14 data points shows a drop of more than 200,000 unique visitors per month over that period, for a net drop of 2-3 million and almost 5 million if you take the extremes! It’s hard to see where that is being made up. The bitly.com domain has only picked up 500,000 visitors, most of that in the past month; and the big-name Enterprise sites that I checked on compete.com for June 2011 are smaller: yhoo.it (~300,000), nyti.ms (~150,000), amzn.to (~80,000), n.pr (~31,000), and pep.si (~12,000). Those six together will account for a little more than 1 million of the shortfall. Perhaps there are enough other Enterprise sites to balance the other millions; or perhaps the compete.com data are just too imprecise for this kind of analysis; or perhaps we’re all just Tweeting far fewer shortened URLs, though I doubt it. But any way you count it, it appears that the reckoning of greater concern is not the exhaustion of bitly’s pool of unique URLs, but rather of bitly’s longevity, with that extrapolated line zeroing-out in less than three years.
UPDATED August 15, 2011: Even if my words aren’t read, my face is red….
Imagine my surprise this morning when I used twitter.com to announce this post (instead of bitly.com, as I generally do), only to see that the shortened URL was linked from t.co instead of bit.ly. My mental hiatus for the past year (see above) was not only from thinking about exhaustion of bitly’s pool of unique hash strings. I also stopped paying attention to what Twitter was doing, in particular introducing its own shortener!!! Excuse me while I post something at duh.me.
It didn’t take long to discover what everyone else interested in social media already knew — Twitter had introduced its own link service in June 2010, precisely coincident with the downturn over which I spilled so much ink above:
● from the Twitter blog: Links and Twitter: Length Shouldn’t Matter
● from Fast Company: Twitter Introduces Bit.ly-Baiting URL Shortener, T.co
● from Mashable: Twitter to Change Links and How They Are Displayed
So back to the spreadsheet to wrap up the numerical loose ends.
Above is a smoothed graph of the past 14 months of unique-visitors data (again from compete.com) for bit.ly, bitly.com, t.co, and the sum of all three. The missing millions are accounted for, or close enough.
