Survey dissatisfaction

Most physicians have long been critical of patient satisfaction surveys. They question whether the surveys truly measure quality and whether it's fair to use them to determine physician bonuses.

They will finally get their wish under the new physician payment system under the Medicare Access & CHIP Reauthorization Act. The 90% of physicians expected to use the standard incentive program (dubbed MIPS for Merit-based Incentive Payment System) will have the option of excluding patient satisfaction ratings from the formula for determining rewards or penalties to their pay.

The CMS escape clause came in response to the general hostility most physicians have toward the surveys. In a survey for Modern Healthcare conducted by New York City-based SERMO, an online physicians' social networking website with over 600,000 members around the world, only 9% of 2,100 respondents thought patient satisfaction surveys accurately reflect the quality of care.

“There's no clarity at present on the issue of how strongly clinical measures of quality and patient satisfaction are related,” said Timothy Hoff, a medical sociologist and professor of management at Northeastern University in Boston. The healthcare industry generally oversimplifies patient experience, he said, making it difficult to link it clearly to other health outcomes.

Yet a number of healthcare systems that have experimented with public posting of patient satisfaction survey results and comments have seen their physicians strive to improve their relationships with patients, which can mean anything from improving their bedside manner to being more attentive to basic needs.

In December 2012, the University of Utah Hospitals & Clinics in Salt Lake City became the first hospital to post patient survey results online. “It actually resulted in a big turnaround for our institution,” said Dr. Thomas Miller, chief medical officer at the system.

But even Miller is opposed to linking physician compensation to survey results, which the CMS' new proposed policy will give his system and its employed physicians the option of doing. “I would advise against it,” he said. “Our success is an indication that you don't need to because people tend to look out for their own self-interest and self-image.

“Once your reputation is out there, it's yours to manage,” he said. “You don't know that you need to compensate people for that.”

Critics of patient satisfaction ratings have focused most of their ire on the Consumer Assessment of Healthcare Providers and System (CAHPS) measurement tool, which is used by the CMS for both physicians and hospitals and is considered state of the art in measuring patient satisfaction. They say the questions in the patient experience sections are crude proxies for deeper or more complex aspects of care.

Hoff pointed to the example of a provider giving a patient an unnecessary antibiotic simply because it will earn him or her a better rating because the patient came away from the encounter with a prescription. “Is that necessarily the right clinical thing to do in that situation? Maybe not,” Hoff said. “High levels of patient satisfaction don't always mean that good quality is being delivered and also it means sometimes that there's overutilization being committed.”

The debate has raged across the pages of medical journals and the trade press that caters to various physician specialties. For instance, Dr. William Sullivan, a Chicago-area emergency physician and medical attorney who is the former medical director of St. Mary's Hospital in Streator, Ill., published a series of articles in Emergency Physicians Monthly starting in 2010 that took aim at the superficiality of surveys administered by South Bend, Ind.-based Press Ganey, the nation's largest patient satisfaction survey firm.

He said the difficulties in getting patients to answer surveys meant satisfaction scores would sometimes be based on only five or six responses out of more than 1,000 emergency department visits. “The scores were all over the map. One month we'd be in the 4th percentile; the next in the 99th,” he said.

Press Ganey defended itself in the magazine, although the company declined requests to be interviewed for this article. “Many survey organizations, Press Ganey included, will issue reports with as few as seven returned surveys despite having a stated absolute minimum sample size,” wrote Donald Malott, vice president for healthcare analytics and survey methodology, and Brad Fulton, senior research associate. “This is because it is the ED'S data and it has a right to see it.”

Dr. Jay Kaplan, vice chairman of emergency services for the Ochsner Health System in New Orleans and president of the American College of Emergency Physicians, representing more than 35,000 physicians, agreed that a major problem with interpreting patient experience surveys is poor survey response rates. The small number of respondents made it virtually impossible to develop valid performance improvement strategies.

David Levin, CEO of Chapel Hill, N.C.-based Bivarus, a cloud-based analytics software company that measures patient experience, said statistical significance of surveys is a nagging problem that frustrates physicians. His firm is among the many vendors pushing CMS to use validated electronic data capture techniques to capture more CAHPS survey results.

“The only mechanism for us to collect data is through paper or telephone,” he said. “Those are antiquated data collection modes,” he said.

Despite the drawbacks of surveys, Joshua Halverson, principal at ECG Management Consultants in Dallas, who is involved in developing incentives in physician compensation plans, said MACRA's move away from fee-for-service to reimbursement that rewards quality, performance and measures other than productivity will increase the emphasis on patient experience surveys.

“Satisfaction scores should be taken in context and should be balanced with other measures of performance,” he said.

MACRA reforms do allow for voluntary inclusion of patient satisfaction scores in the MIPS track of its quality incentive program. The Alternative Payment Models track, which is reserved for physicians taking financial risk for the overall cost of care, does not include incentive payments for process measures like patient satisfaction.

Laura Wooster, vice president of public policy of the Chicago-based American Osteopathic Association, representing many small and rural medical practices in the MIPS quality category, said CAHPS is no longer required for any groups, but groups with two or more clinicians can voluntarily participate for bonus points. “The quality component of MIPS does include some patient satisfaction/experience survey components, but as of now (under the proposed rules), they are voluntary,” she said.

Wooster said the CMS requires clinicians to report on six measures out of over 300. CAHPS would count as one such measure. Physicians with good scores will be able to use that measure and earn bonus points for their practice, Wooster said.

But even so, it would only affect at most 20% of the overall quality component score, which determines half of the overall MIPS composite score. And that overall score is applied to only 4% or under of Part B compensation for 2019. “While (including solid patient satisfaction scores) is a bump, it's not a significant one,” Wooster said.

Outside Medicare, the use of the survey results in helping determine physician compensation packages has been growing in recent years. “Most progressive organizations have tied patient satisfaction surveys to physician compensation,” said Meryl Luallin, co-founder of San Diego-based SullivanLuallin Group, a pioneer in patient satisfaction surveys. “Many groups have some sort of a bonus pool that's contributed to and a measure of patient satisfaction is an element of the bonus.”

And the number of hospital systems or large practices that are posting those scores is growing. Grand Rapids, Mich.-based Spectrum Health, which has a medical group of more than 1,200 providers, gives bonuses to physicians based on how they fare on patient experience surveys, but it's generally well under 5% of total compensation.

“If you're a physician making $250,000 a year and we say to you, 'We'd like you to work on your patient experience and we're going to put $1,000 or $2,000 at risk on that.' You're likely to not be as invested,” said James Bonner, director of patient experience at Spectrum.

Still he's a big believer in public posting. Creating a balanced scorecard of experience, quality, safety, and outcomes will be essential to drive change, Bonner said.

Trade groups for physicians and physician practices have been leery of tying compensation to patient satisfaction scores. Kenneth Hertz, principal and compensation expert at Englewood, Colo.-based Medical Group Management Association, representing smaller group practices, said physician buy-in is needed before it will work.

The SERMO poll found 73% of physician respondents did not consider it fair to use patient satisfaction surveys in part to determine physician pay.

In June, the board of the American College of Emergency Physicians reiterated its opposition to tying experience surveys to compensation. “Due to the difficulty in segregating whether patient-experience-of-care scores (are) a result of physician performance or due to demands and restrictions of the current healthcare system or other factors out of the control of the physician, patient experience of care methods that have not been validated should not be used for purposes such as credentialing, contract renewal and incentive bonus programs,” the policy stated.

But firms operating in developing new technologies for collecting patient responses say the move to tying compensation to satisfaction scores is inevitable. “Patient experience is an important component of an overall patient-centered system,” Levin of Bivarus said. “Continuous measurement is the new normal.”

Howard Wolinsky is a freelance writer based in Flossmoor, Ill.