Hospital tech executives are managing more digital health applications at a time when their systems face multiple threats to business continuity and patient safety.
Amid greater threats to cybersecurity and an increasing number of weather-related events ranging from storms to excessive temperatures, health systems have had to map out extensive strategies to minimize data server downtimes. The average cost of data center downtime is $7,900 per minute, according to a study from research firm Ponemon Institute.
While restoring data servers has always been a tough job, the interconnectivity of systems has made it even more challenging, said Aaron Miri, chief digital and information officer at Jacksonville, Florida-based Baptist Health.
“In the old days, you had a mainframe running and it was a self-contained ecosystem,” Miri said. “Nowadays, systems rely on each other, and everything is connected like a new car. If something breaks in a new car, everything else in the car breaks too because it’s all digital. It’s the same with health IT systems. The complexity of the system is through the roof.”
Miri said Baptist Health dodged disruptions from Hurricane Idalia in late August, but he knows some interruptions are unavoidable for the coastal health system.
“You're seeing cities spend billions of dollars in putting up sea walls,” Miri said. “Hospitals are trying to go green … but their doors are going to be flooded if the sea level rises. You have all these different variables. And then on top of the weather events, you have the always-evolving cyber threats.”
Read more: Increased cyberattacks lead to steep consequences for organizations
At Baptist, the system has a secondary data center that can restore 42 tier one software programs like its electronic health record and revenue cycle systems, out of 712 distinct enterprise applications, in four hours. If both data centers are down, Miri said Baptist has locally hosted devices that provide clinicians with a static snapshot of the patient.
At Cleveland Clinic, disaster recovery strategy is paramount, as the health system has scaled far beyond northeast Ohio, with locations in Florida, Las Vegas, Canada and Abu Dhabi. Ian Willis, the system's enterprise architect, leads a team that developed a strategy to restore critical applications in two hours of a declared event and more than 200 patient care and business critical applications within eight hours.
The strategy relies on a facility in Ashburn, Virginia—known as the data center capital of the world. Located 300 miles from Cleveland, the facility, along with a cloud service provider, helps put the health system's data in a separate, isolated environment where critical applications are replicated and can act as a restoration point if the original servers fail. The separate environment allows for a 300% faster restoration than the health system's previous strategy, Willis said.
“The cost of being down for a few days would be more than what it costs us to maintain this for a year,” Willis said.
Turning up the heat at Utah Health
In late August, University of Utah Health found itself with a five-hour outage when summer heat caused problems with its data center’s humidification system. Chief Technology Officer Jim Livingston said as his team was testing a new motor that had gone out in a separate incident, humidity infiltrated the data center and caused the shutdown.
University of Utah Health isn’t the only system that has seen a heat-related outage. In 2022, data centers at two London hospitals shut down because of a heat wave. The outage cost the U.K.’s National Health Service around $1.7 million, according to a NHS review.
As healthcare has become more dependent on its tech systems, Utah is looking at how to modernize its approach and use more cloud-based services, he said.
“We have a cloud center of excellence to start mapping out and prioritizing our move to the cloud,” Livingston said. “We’re an Epic organization and that's really our target right now, getting that migrated to the cloud.”
While cloud-based systems are ideal in disaster recovery strategies, tech leaders say some on-premises data centers will remain necessary in healthcare. At Baptist, 60% of its data is on the cloud and 40% is on on-premises physical servers. Local servers are important if a weather event wipes out power to an entire region, he said.
“You have to have local survivability,” Miri said. “If a large Category 5 hurricane wipes out power to an entire region, hospitals are required to have 72 hours of fuel reserves that can run generators. In that scenario, if you don’t have local IT systems to answer your phones or make copies of data, you’re going to be crushed.”
More than hurricanes, wildfires or any weather event, Willis said ransomware keeps him up at night the most. Nearly 25% of cyberattacks in 2022 targeted the healthcare industry, according to data cited in a July report from cybersecurity company Trustwave. Since May, 88 provider organizations have disclosed breaches relating to Progress Software's MOVEit program, according to data from security firm Emsisoft released last week.
The rising threat of cybersecurity makes it more important for health systems to have a plan for when this threat reaches them, Willis said.
“We practice our disaster recovery plan, so it becomes second nature to us,” Willis said. “So, if and when something happens, it’s very natural for us to recover.”
