Making checklists work: South Carolina's statewide experiment

CAMDEN, S.C.—Surgical team members shuffle in and out of the operating room preparing for the second procedure of the day at KershawHealth, a 120-bed hospital in a town of about 7,000 in north central South Carolina.

With a man on the operating table for a laparoscopic gallbladder removal, an anesthesiologist checks levels on a monitor. A nurse sets up bandages and other supplies on a tray. Dr. Edward Gill, the surgeon, takes a final look at markings designating the surgical area on the patient's stomach. Some make small talk about the risk of flooding because of heavy rains as Christmas approaches. “All right, let's get started,” Gill announces. It's 7:35 a.m.

But then everyone stops. No scalpel is lifted. All eyes turn to a poster labeled in jumbo print, “KershawHealth Safe Surgery Checklist,” on the wall at the foot of the surgical table.

The team goes over each item on the list aloud, and each member participates. The registered nurse confirms the patient's name and date of birth, that X-ray equipment is in the room, and that the man is allergic to a common pain medicine. The anesthesiologist states the type of drug being administered and that there are no current issues with the airway. The surgical technician, circulating nurse and others also weigh in.

Finally, Gill again confirms the type of surgery, says it should last about an hour, and notes that the patient has hypertension, a chronic health problem that could affect the procedure's outcome.

“Does anyone have anything to add? Speak up,” Gill says as he looks around the room. When no one speaks, the operation begins at 7:37 am. The process lasted just under two minutes.

The pause for the checklist is intended to help the team avoid terrible mistakes like operating on the wrong person or body part. It also facilitates communication among clinicians, who otherwise say surprisingly little to one another before, during or after a procedure. When used effectively, proponents say checklists improve efficiency in the OR, an area considered by some safety leaders to be among the most chaotic places in a hospital.

Surgical checklists, an approach drawn from commercial aviation and other high-risk industries, gained popularity when the World Health Organization promoted them in 2007 under the leadership of surgeon and author Dr. Atul Gawande. He further popularized them in an influential book, The Checklist Manifesto, published in 2009.

But they have yet to become widely or systematically adopted. As a result, there's not much data on their effectiveness, which in turn complicates the sales pitch to persuade organizations to invest the time and resources required to make them work.

South Carolina is now the testing ground for a much more focused approach. In 2013, the South Carolina Hospital Association, working with Gawande and the Harvard University School of Public Health, launched a structured initiative to get every hospital in the state to regularly use a pre-surgical safety checklist process.

The leaders of the project estimated that doing so could save 500 patient lives a year by averting medical mistakes. Harvard researchers gathered administrative data and data from the state's death registry to track mortality outcomes in places that have adopted checklists. Initial findings have been submitted for peer review and may be published this spring.

South Carolina was chosen for Harvard's project because of the state's previous investment in an all-payer administrative data set. In addition, the state launched a multiyear project with the Joint Commission in 2013 to improve the overall quality and safety of patient care.

Hospitals and health systems struggle to create the culture required to become high-reliability organizations, Joint Commission CEO Dr. Mark Chassin said in a recent interview, using a term that describes organizations that are engaged in complex and high-risk activities but still manage to avoid catastrophic events. “Healthcare can get to that state where the operation of the organization is so good that zero harm is a byproduct of the way they do their work.”

But checklists fail when they're tossed into an environment that doesn't fully support the effort or when the checklist approach isn't tailored to match the organization's needs and culture, Gawande said.

“It takes leadership support at the top and enthusiasts on the frontline,” he said. “You need both, because enthusiasm dies on the vine without a system behind you.”

Indeed, the South Carolina Hospital Association found that acceptance of the checklist process varied from hospital to hospital, but those that had the most success had committed to following all of the steps needed to become high-reliability organizations—they offered leadership support, financial resources and cultivated staff members who were dedicated to the enterprise. They also allowed staff to customize the process.

“One size does not fit all,” said Lorri Gibbons, vice president for quality improvement and patient safety at the association. To that end, the group brought in engineers with expertise in process improvement to visit every hospital, observe its procedures, and make recommendations to help each facility tailor the tool to meet its needs.

Ashley Childers, an industrial engineer from Clemson University, has driven over 19,000 miles to help more than 50 South Carolina hospitals with their checklist processes. Many hospitals she visited, including KershawHealth, thought their surgical units were already safe, even though surgeons could start a procedure without uttering a word to the team, and staffers were afraid to speak up when they knew something was wrong. “That happened even if they already had a checklist in place,” she said.

Jamie Thompson, the hospital's chief certified registered nurse anesthetist, said the OR teams were going through the checklist process by rote without paying close attention to what people were saying.

There was very little pre-surgery communication, agreed Rosa Canty, a charge nurse. “We'd just go in the room and start making an incision.”

The surgeons were one of the biggest hurdles. “ 'This is going to slow me down' ” was the main complaint, said Dr. Benjamin Blackmon, an anesthesiologist who championed the facility's surgical checklist program. “People saw it as just another time intrusion.”

KershawHealth updated its checklist in 2014, and its approach quickly gained statewide recognition. The hospital's custom pre-surgery checklist has more than 25 boxes. While that seems like a lot, the team is able to go through them quickly. The hospital's process also includes a required post-surgery “debrief.” Again, all members of the surgical team have a speaking role. This time, specimen labels are read out loud to confirm their accuracy, and staff has a chance to speak up if a piece of equipment did not work properly.

The surgical profession has only begun to appreciate the potential benefits of the instrument, according to the American College of Surgeons, which in 2013 joined a campaign to encourage the use of surgical checklists.

“We're trying to change a culture that has deep roots,” said Dr. Bill Berry, chief medical officer for Ariadne Labs, a not-for-profit healthcare solutions group that operates in partnership with Brigham and Women's Hospital and the Harvard School of Public Health. Gawande is the lab's executive director.

“I may win some people over,” Berry said. But others, he said, will gripe, saying, “ 'I've been practicing for 40 years and never used this ... I don't need no stinking checklist.' ”

That pervasive attitude explains why surgical checklists have produced mixed results so far.

A study published in the New England Journal of Medicine in 2014 found that surgical safety checklists used in more than 100 hospitals in Ontario, Canada, did not significantly reduce complications or deaths. The Ontario Ministry of Health and Long-Term Care had required all hospitals in the province to incorporate checklists by July 2010.

A retrospective, longitudinal study published in JAMA Surgery in 2015 examined the surgical outcomes among 64,891 patients in 29 Michigan hospitals from 2006 through 2010. Researchers looked at whether implementation of a checklist-based quality improvement program called Keystone Surgery was associated with improved 30-day mortality rates, fewer surgical-site infections and reduced rates of other complications. They found no association.

“The story of surgical quality improvement has become a saga of high hopes followed by dashed expectations,” Dr. David Urbach, of the University Health Network in Toronto, wrote in an editorial accompanying the JAMA Surgery study. It was “one more disappointment to this boulevard of broken dreams.”

To change the culture so that surgeons and everyone else were on board, KershawHealth re-evaluated the items on the original checklist first promoted by the WHO and inspired by Gawande's work.

They noticed, for example, that the checklist included tasks that their hospital performed in the holding area instead of the OR, so some surgeons didn't think the checklist items were relevant, or they would try to memorize the parts they needed. Blackmon, the “physician champion” at KershawHealth, also used what he calls old-fashioned peer pressure to get colleagues to buy in to the program.

“Nobody wants to be seen as the outlier,” Blackmon said. “Surgeons are competitive.”

The facility made the full checklist process mandatory for every surgical patient, instead of optional as before. It implemented the hospital association's recommendation that every staffer in the OR say something during the pre-surgery pause and post-surgery debrief, from the surgeon to the surgical assistant.

The KershawHealth staff also added a final box to the checklist, which asks: “What could have been done to make this case safer or more efficient?” In addition, the hospital posted checklist reminders in prominent places. A checklist hangs in each OR, and its jumbo font is large enough for everyone to clearly see. Reminders about timeouts and debriefs are posted at the entrance and exit of each surgical suite.

The facility now has nearly 100% compliance, which is tracked through monthly assessments of how often the checklist process was skipped.

Some South Carolina hospitals have had less success, acknowledged Childers, the industrial engineer coaching the state's hospitals on checklist implementation. One of the biggest challenges is a lack of data that proves using them makes a difference in patient outcomes.

“It's going to take a huge number of opportunities before we see any kind of statistical change,” Childers said. “People ask where the data is, but the feeling comes first.”

And until Ariadne Labs has findings to share from the data collected in South Carolina, Berry argues that the day-to-day benefit of better communication among team members and the promise of a more efficient OR (and, with it, the ability to book more procedures) should be compelling selling points for clinicians.

“In the end it's worth it because they end up taking better care of their patients,” Berry said. “And I think we're going to be able to show that in data over time.”