Kobayashi Maru - Dealing with downtime

One of the worst things that can happen to an online business is downtime. It doesn't matter if the servers crashed, the network connection "broke", or the data center was compromised; your users can't access your service which is bad.

Theory vs. Practice
In theory, there is no difference between theory and practice - in practice, there is.

Ideally, a website should never suffer any downtime. In reality, downtime can't be avoided and it will happen whether planned or unplanned. Unlike brick and motor structures which are designed to withstand storms and earthquakes, online businesses are very fragile with many single points of failure such as DNS, loss of power, failed hardware, the easily cut T-1 or OC-3 cable, etc.

Implementing a contingency plan isn't an easy task and your plans will only work well when systems fail as you expected them to fail. You not only have to figure out what you're going to do under certain circumstances, but you have to rehearse it on a regular basis - this is what the military refers to as "exercise training". Not exercising such as pull-ups or push-up, but rather wargaming the scenarios which are most likely to occur and then actually executing the planned response.

Contingency Planning At Adjix
Adjix is a small company with limited resources, so we've implemented a simple solution to keep links running if our servers go down. We do this by relying on Amazon's web servers.

Every time our users create a shortened Adjix or ad.vu link, we implement it as a meta-refresh web page instead of the industry's more common HTML redirect (sometimes referred to as a 301 or 302 redirect). Should our servers go down, we can make a quick DNS change which takes about five minutes to propagate throughout the Internet. We believe broken links are a bad thing and although we may not be able to capture detailed link click data when this happens, our links will continue to work. We've tested this plan, at Adjix, a few times without skipping a beat - of course that's no guarantee that it will work perfectly in a future crisis, but it does lower risk while increasing confidence.

Serving up shorten URLs in this manner is not the industry norm, but we believe it gives us the security, should bad things happen, that we can continue to keep our links working. No plan is perfect - it's foreseeable that both our servers and Amazon's servers could go down at the same time – but we believe it's a solid plan commensurate with our budget.

Data Migration

This week, we are in the process of migrating your link click data to CSV format. This process will continue for a while since there are many millions of hits to migrate.

You'll know that your data has been migrated when you see a CSV file, next to your link, on your link stats page. We've migrated link click data up to April 15. Link clicks received after April 15 will show up where they always have: under your hits detail page.

The DiggBar Solution

Digg has received a lot of attention since launching their DiggBar a couple weeks ago. Sites like Engadget block the DiggBar which seems fair - engadget.com is their real estate and blocking it is trivial requiring a simple line of Javascript.

Of course, Digg could retaliate and block any story submitted that links to engadget.com (or, more simply, any link that leads to 207.200.75.1 or 207.200.95.1). But, I could never see this happening nor would it be productive.

However, I can envision a couple solutions that would allow the DiggBar to coexist with sites like Engadget.

1. Simple Solution
One issue that people have with the DiggBar is that users cannot see the domain name of where the content is hosted. To solve this problem, Engadget could set up a DNS alias such that digg.engadget.com points to the Digg URL.

This is a technique we allow people to use at Adjix. For example, the following two links point to the same resource located on ps-enable.com's servers:
http://adjix.com/35zs
http://partner.ps-enable.com/35zs

Note, in the ps-enable.com link, that the URL in the address field of your web browser corresponds the actual domain of the content. Now imagine replacing the ad, at the top of the page, with the DiggBar.

When using this solution, Digg could simply redirect http://digg.com/1234 to http://digg.engadget.com/1234. Although this isn't a perfect solution it does give the reader an idea of which website has published the content.

2. Partnership Solution
A more elegant solution would be if websites partnered with Digg. Digg could provide their partners with a JavaScript snippet to drop on their website much like Google AdSense.

In this solution, a Digg URL, such as http://digg.com/1234 would redirect to the partner's website which would display the DiggBar "organically". In other words, the DiggBar would be served up by the site where the content is hosted.

This partnership solution is win-win since the content provider gets the traffic from Digg and the DiggBar is displayed at the top of the page making it easier for people to digg stories.

Adjix Links on Twitter

If you've been posting Adjix links to Twitter, you'll notice something interesting. Within a minute or two of posting a link ten or twenty bots will click on your link to see where it leads. This is most noticeable with Adjix links that don't contain any ads since they are simple redirects.

Due to Twitter's popularity and API every time a link is posted to Twitter it can register dozens of clicks from these third party bots which automatically index every tweet. These are usually easy to spot, when reviewing your Adjix link stats, either due to their host name containing something obvious like crawl or bot or due to the fact that they usually don't have a referrer.

So, even though it seems that your links, posted to Twitter, receive a dozen clicks off the bat - it's not always people who are clicking on them.