BOSTON — An outage at a little-known firm that speeds up access to websites knocked a lot of top internet destinations offline on Tuesday, disrupting business and leisure for untold millions globally. The problem was quickly resolved. The company, Fastly, blamed a configuration error in its technology.
But the incident — Fastly’s traffic dropped 75% for about an hour just as the East Coast was beginning to stir — raises questions about how vulnerable the global internet is to more serious disruption.
Why Fastly’s technical problems led to so many outages
San Francisco-based Fastly isn’t a household name, but its “edge server” computing technology is used by many of the world’s most popular websites, such as The New York Times, Shopify, the Guardian, Ticketmaster, Pinterest, Etsy, Wayfair and Stripe. The British government is among its clients.
The company provides what’s called a content delivery network — an arrangement that allows customer websites to store data such as images and videos on various mirror servers across 26 countries so that the data is closer to users, and thus shows up faster. Many of Fastly’s customers are news sites that use its technology to update their websites with breaking news. Buzzfeed, for example, used Fastly to cut the time its users took to reach the site by half. Fastly had $290.9 million in revenues last year.
Alternate providers aren’t good alternative
Customers rely on Fastly and its rivals to host and protect their website data from denial-of-service attacks and disruption from spikes in traffic. Had this outage been more serious, customers could have moved to competitors such as Cloudflare or Akamai. But that’s not simple; many businesses would have had to scramble and might have suffered losses.
“You can’t switch quickly to another service unless you had it set up ahead of time,” said Doug Madory, an internet infrastructure expert with the traffic-measuring company Kentik. “If Fastly were down for a day, that would be pretty bad.”
Even if they do have an alternative provider, engineering a smooth switchover from one to another is not for the faint of heart, said Ben April, chief technical officer of Farsight Security.
Madory and other experts said Fastly and its competitors spend heavily and devote major engineering resources to reducing the possibilities of such outages and ensuring they can recover as quickly as Fastly did on Tuesday.
Such outages are not new — but not at all common. “There may be years between when a company has an outage like this,” Madory added. “I think we are going to have these very rare but probably impactful short outages for the foreseeable future.”
Other vulnerabilities
Like the content distribution network world, cloud computing — when computing services are entrusted to a remote provider — is dominated by just a few major players led by Amazon Web Services, Google and Microsoft. Amazon, the biggest cloud provider, periodically has brief outages, which are a big deal for customers.
“And if it became a major outage of, say, more than six, eight hours — but days — it could put companies out of business,” said Josh Chessman, an analyst with the tech market researcher Gartner Inc.
The question is: What could cause such a serious outage that might destroy customer data? A major cyberattack is one possibility. Another is fire or catastrophic natural disaster. These businesses, after all, are based in datacenters. In March, a fire at a datacenter in Strasbourg, France, owned by a major cloud computing firm knocked out service to millions of websites.
Setting security standards
“I don’t know that we need regulation,” Chessman said. Suppose Congress proposed to mandate additional cloud providers to increase competition. “How do you do that?” he asked.
Of course, the federal government can set new standards for security at companies that control vast data resources online. It’s already beginning to tighten up cybersecurity requirements for critical infrastructure in the energy sector following last month’s cyberattack on the Colonial Pipeline, he said.
In a regulatory filing last year, Fastly said it had been subject to “cyber-attacks from third parties — including parties who we believe are sponsored by government actors.” Those attacks “have strained our network” and could harm it in the future, it said.
Businesses and consumers, meanwhile, should be thinking seriously about how much they should rely on the cloud for their most valuable data. “If there’s an outage, what’s the impact on our business?” Chessman asked. Perhaps it makes sense not to rely on a cloud-based service for your company’s email if you’d go bankrupt without it during a two-week outage.
But running your own email and backup services is complicated and costly — one reason companies turned to the cloud in the first place.
About the Author
