Is There a DNS Doctor in the House?

An apple a day may keep the doctor away, but what if the apple itself is sick?

Starting a little before 6 am EST on Wednesday, users across Apple’s various internal applications services such as iTunes, iBooks, the App store, and several others experienced growing frustration as repeated attempts to access these services were met with the same maddening error message.

photo courtesy of iMore

And this was no mere blip on the radar, with exasperation continuing to mount as the outages persisted throughout the day. Finally, Apple released a statement to their users and the media apologizing for the errors, and blamed the problem on an internal DNS server.

“We apologize to our customers experiencing problems with iTunes and other services this morning,” the company said. “The cause was an internal DNS error at Apple. We’re working to make all of the services available to customers as soon as possible, and we thank everyone for their patience.”

A situation like this highlights the many obstacles that all companies – especially those serving a global user base – face when trying to deliver excellent web performance. DNS is but one of many different systems that have to be constantly tested and monitored to ensure that such failures happen as rarely as possible. And since no one is perfect and failures WILL happen at least once in a while, the need for troubleshooting protocol and backup systems becomes abundantly clear.

The cost of an extended outage such as this – going on 10 hours at the time of this publication – for a major eCommerce retailer like Apple can be staggering (UPDATE: all services have been restored as of 5:10 pm EST). Based off their latest quarterly earnings report, iTunes generates roughly $1.2 million per hour, meaning that a 10-hour outage would cost the company $12 million. While the spotty nature of this outage and its geographic variances makes it impossible to calculate an exact dollar amount, we can safely assume that the extended length of time and the range of services that have been affected mean that Apple will lose at least several million dollars when all is said and done. That’s a lot of cash riding on one DNS server.

Additionally, the timing of this failure is less than ideal, with the company having made major waves over the past two weeks. First came the announcement that it would be replacing AT&T in the Dow Jones Industrial Average, followed by its big launch of the Apple Watch and HBO Now streaming service.

The company is surely too prevalent and too trusted for this incident alone to have any effect on its long-term financial health, but it should still serve as a cautionary tale: no matter how big you are, you’re not immune to web performance failures, and must take every necessary precaution to mitigate their impact.

The post Is There a DNS Doctor in the House? appeared first on Catchpoint's Blog.