The platform on Monday, Sept. 5, “experienced the loss of its Sacramento (SMF) datacenter due to extreme weather,” an internal message to Twitter engineers sent Friday said in part, according to a story published Monday by CNN.
Sept. 5 marked the beginning of the most intense stretch within one of the biggest heat waves in Northern California history: downtown Sacramento reached 113 degrees, followed by a 116-degree high that made last Tuesday the hottest day in the city’s recorded history.
Data centers are warehouse-type sites with large volumes of computers, servers and other equipment, much of which must be closely temperature-controlled else risk overheating.
The internal memo, which CNN reported was sent by vice president of engineering Carrie Fernandez, said the Sacramento outage left Twitter in a “non-redundant” state.
This means that if one of two other sites were to experience an outage, said in the memo to be located in the Atlanta and Portland areas, Twitter “may not be able to serve traffic to all Twitter’s users.”
Twitter in a statement to CNN said there “have been no disruptions impacting the ability for people to access and use Twitter at this time.”
The former head of security at Twitter, Peiter Zatko, in a whistleblower complaint last month to multiple federal government agencies including the Securities and Exchange Commission, warned of Twitter’s “insufficient data center redundancy,” CNN reported.
Zatko’s complaint noted that in the period of 2020 to 2021, Twitter had data centers in Sacramento and Atlanta, with plans as of 2020 to add a new data center, according to recent reporting from the Washington Post, which obtained a copy of the complaint.
Zatko’s whistleblower complaint cites a February 2020 post by a blog called Data Center Dynamics, which said Twitter leases space at data center in Sacramento operated by a subsidiary of Japanese telecom giant NTT.
The exact location of Twitter’s Sacramento data center has not been publicly disclosed. Tech companies typically keep that information guarded for security reasons; likewise, data centers that lease out space tend not to reveal their clients.
NTT has three data centers in Sacramento’s Natomas area. The newest of the three is a 180,000-square-foot wholesale data center, according to the company’s website.
The Sacramento Bee could not immediately confirm whether NTT’s Sacramento location leases out to Twitter, or whether the Natomas data centers were impacted by last week’s record-breaking heat. A company spokesperson could not be immediately reached for comment.
But Twitter’s data ties with NTT America dates back to at least 2009, according to media reports as well as archived Twitter blog posts from the time, the latter of which said the social media giant would work with NTT to maintain its data center “footprint.”
Twitter around 2010 announced plans to build its own dedicated data center in Salt Lake City, parting ways at least in part with NTT, but the project failed for numerous reasons, according to a 2011 story by Reuters.
Twitter instead moved its data center, formerly in Santa Clara, into a Sacramento facility owned by a company called RagingWire, Reuters reported at the time. NTT acquired a majority stake of RagingWire in 2014 and 100% of the company in 2018, according to a company news release.
NTT on its website describes Sacramento as an optimal data center location because of its proximity to the Bay Area and Silicon Valley while offering less risk of earthquake damage.
Zatko is due to give whistleblower testimony Tuesday regarding Twitter security to the U.S. Senate Judiciary Committee.
© 2022 The Sacramento Bee. Distributed by Tribune Content Agency, LLC.
