American Dreaming - 1990's / Part XIV

Another effort of note that I took on during my tenure with this managed care company was to embark on developing a disaster recovery program for the operating units that I was responsible for.

Because of the bombing attack on the World Trade Center in the early 1990’s a renewed focus was prevalent in a lot of organizations (both large and small) to focus on developing a plan/process and procedures to deal with catastrophic events that might disrupt normal corporate operations in any capacity.

I enlisted the aide of a local consultancy to work with my staff to compile all the relevant forms, and resources that we would need as an organization to keep key functions operating while any time of relocation or rebuilding effort took place. The effort involved interviews with administration and human resource staff to accumulate information on all the various functions in those realms that would be required to be restored quickly. We also went to each of the clinical areas and garnered application and data needs that those departments would need and a minimum time to restore for those various IT functions.

After six months of documentation work and doing some investigation on potential “warm” sites (locations where we could have equipment that could be utilized in a restoration effort but, not running) and “hot” sites (locations that would essentially be duplicate sites to our production facilities and ready to be utilized in a moment’s notice) along with the costing and time frames to outfit such resources.

We presented our findings to senior management in late November of 1995. The reception we received was respectful but, the price tags were seen as cost prohibitive. There was no appetite to take on such a project at that point in time.

There are rare instances in life where you throw your support behind an effort (or cause) only to see it go for naught followed by circumstances or events that conspire to actually prove out the worth and correctness of your position. For the lack of a better term, I’ll dub this the “neenerneenerneener” moments or an “I told ya so”…

Well, this was one of those occurrences.

Less than six weeks after our proposal to senior management on developing a disaster recovery plan, the river spoke. The primary office of this region was in a building that bordered closely the Hudson River. During the month of December 1995, we had an uncharacteristically harsh number of snow storms. During the second week in January 1996 there was a sudden change in climate amounting to a mid-winter thaw. As a result of this thaw, the river flooded. I got a phone call on Saturday morning as I was traveling down to pick up my son for our customary weekend visit. I was told that the building had seen a little of bit of flooding.

I asked “How much is a little bit of flooding?”

“The basement is under water and the water is anticipated to continue to rise for some time.”

“The whole basement area?”
“Yeah.”

“Okay I’ll be there in less than a couple of hours but, that’s the best I can do, I’m in transit right now.”

I then endeavored to reach some members of my staff but, being a sunny Saturday and still relatively early, I was only able to leave voice messages.

By the time I got there, I encountered a vision I’ll never forget. Here was a seven story building, a former shirt factory with a large smoke stack construct on one side and it looked like the Hudson River had widened to the point that it was literally flowing through the base area of the entire building.

Our data center was on the second floor of this building. Our offices consisted of the second, fourth and fifth floors. We had 24/7 clinical coverage that was dependent on the second floor of the building being habitable.

When I arrived, I conferred with our facility staff (who were on site) and asked for current status. I was told that if the water rose another six inches, that our local utility provider would be cutting electrical feeds into the building as a precaution.

My first thoughts were that we were going to have to shut down the data center ahead of any effort to cut electricity to the building. The second thought was where were we going to shift our clinical referral line coverage.