Rethinking the Worst Case

How disaster recovery is changing in the wake of Katrina and Rita.

To understand how Hurricane Katrina is changing the way Corporate America thinks about disaster recovery, look no further than the computer-services company in New Orleans that located its data center to the third floor of its building to preclude any possible risk of flood damage. “I salute them for planning,” says Hank Chase, director of homeland-security programs for management consulting firm Smart and Associates LLP, in Devon, Pennsylvania. “They knew they were in an area subject to flooding.” Unfortunately, he says, no one imagined the building could be inundated by floodwater 23 feet deep, which is what Katrina delivered when it broke through the city’s levees in August. The company’s data center was destroyed. “They had no backup systems, no redundant infrastructure whatsoever,” observes Chase. “The last I heard, they’re down completely — out of business temporarily.”

Of all the lessons learned from Hurricane Katrina, the most important is this: companies must rethink the definition of “worst case.” “Based on public information and the information we’re getting as a company, there were not many effective worst-case recovery and reconstitution plans in place when Katrina hit,” says Chase. “A lot of plans focused on reconstituting or rebuilding at an existing location, but there were not many that called for companies to shift to a redundant infrastructure so that they would lose a minimum amount of time and revenue.”

There are, to be sure, companies that got it right. Days before the storm hit, $285.2 billion Wal-Mart Stores Inc. had dozens of managers and support personnel camped out in the company’s emergency command center in Bentonville, Arkansas. At the same time, Wal-Mart warehouses began sending truckloads of emergency supplies, from generators to dry ice, to staging areas where they would be available to stores that needed them. (Ultimately, 173 Wal-Mart facilities were affected. By late September, all but 12 had been reopened.) Meanwhile, privately held vacuum maker Oreck Corp. relocated its headquarters from New Orleans to an IBM Corp. disaster recovery center in Dallas, and, thanks to a backup computer system in Colorado, didn’t miss a beat in issuing paychecks for its 1,200 employees. Hancock Bank in Gulfport, Mississippi, was also on the prepared list. Although it saw its 15-story headquarters ravaged by a 30-foot wall of water, it didn’t lose critical data: in the days before Katrina, the company shipped its records to Chicago on 200 backup tapes.

Still, there were many employers whose plans fell tragically short — or that didn’t have plans at all. Many were small businesses for which relocation wasn’t an option. Others were the hospitals and nursing homes that lost patients when their emergency generators didn’t work. Even companies that had recovery plans had a rough go of it. Gilsbar Inc., a $35 million insurance services firm in Covington, Louisiana, missed the eye of the hurricane by about 30 miles but still saw its hometown hammered by heavy winds. About 10 percent of its 350 employees lost their homes, and some lost family. Although Gilsbar suffered only cosmetic damage, the company was left without electricity, telephone service, or Internet access, and was forced to retreat to a SunGard backup facility in Chicago to continue serving customers.

Discuss

Your email address will not be published. Required fields are marked *