The current flood of initiatives relating to quality, value, outcomes and performance monitoring of health services can be confusing and overwhelming. These efforts have evolved from the threat of healthcare reform and the dramatically increased presence of managed care.
Reform initiatives have focused our attention on the relative value of various medical interventions. Managed care has increased the demand for performance data. Since the belief is "you can't manage what you can't measure," efforts to develop medical information systems that track financial and clinical data simultaneously have intensified.
Methods missing.While the demand for performance data has sent many providers, purchasers and payers scrambling for information, the methods for the use of these data to assess and improve outcomes are noticeably absent. Healthcare organizations are being asked by purchasers, payers and regulatory agencies to develop "report cards" demonstrating the level of quality service they provide. These report cards are conceptualized as an overall quality scorecard, akin to Consumer Reports. It's suggested that such reporting mechanisms will make healthcare providers accountable and provide consumers with comparative data to use in making better healthcare decisions.
At first blush, such a focus on demonstrable quality of care would seem desirable and of strategic importance in a competitive environment. However, when considered from a methodological perspective, there are several significant problems. These must be addressed before rushing into consumer-directed performance monitoring.
Two assumptions are consistent among all the report card initiatives:
Measures associated with the report cards are based on well-developed constructs reflecting some dimensions of the quality of care.
Healthcare providers will find value in these data and take action to change the way they deliver care, thus improving the quality of care.
Unfortunately, there is little data to suggest that either of those assumptions is true. In fact, the constructs underlying report cards are, in most cases, loosely defined. More importantly, many healthcare providers view these data as suspect and have little sense of what to do with report cards once they're developed and the data is fed back to them. Thus, report cards become a tool for the marketing department.
Equally troubling is the possibility the consumer will respond to the report cards, flawed or not, creating even greater problems.
Those clamoring for report cards and other summary outcomes measures have been of little help. Whether there is such a mandate from the public for performance data is debatable, but certainly healthcare payers and purchasers are either using or developing their own measures to assess cost and quality using a variety of data sources.
Adding to the complexity is the fact that many of those espousing the need for performance measures have become vendors of databases and measurement systems. So they're viewed by many as having reduced their ability to act as independent evaluators.
In violation of the second assumption, the measures used to develop report cards often are not directly associated with the process they're measuring. Satisfaction levels and readmission rates are two examples. There are a variety of reasons for a particular grade on a report card. Many of these reasons, case mix for example, have nothing to do with the process of care.
In most cases, the quality model addresses structure, process and outcomes for assessing care. Structure would be the resources used to deliver care, such as buildings, credentialing and procedures. Processes are the actions taken during the provision of care, such as diagnosis, use of medications and procedures applied. And outcomes are the results of care, such as patient satisfaction and quality of life.
The underlying assumption of this model is a causal relationship between the processes of care and outcomes. The implication is that if you find outcomes that are less than expected, "working back in the processes" will identify where improvements can be made. This assumed relationship in the development of report cards is still unproven.
Since report cards are a summary of these measures, it's likely they are of limited value and may have little, if any, relationship to the quality of care.
Direction needed.Before any mandate for report cards is entertained, a variety of problems need to be solved. First, a clear set of methods must be developed to guide providers in understanding and responding to performance data in a way that actually improves the quality and outcomes of care. We call these approaches "road maps."
Second, the proposed paradigms that underlie report cards are theories that must be rigorously tested in a manner that is applied to most aspects of medicine. In other words, they need to meet criteria of generalizability, predictability and repeatability.
Third, assessment models of healthcare delivery that are well-validated and comprehensible to the final users must be developed.
Currently, the only road map offered is the use of total quality management techniques, which have many definitions of quality and performance. But in whatever variation TQM is presented, its focus is on improving the current processes in an organization. What has become apparent is that the re-engineering of an organization is often required before TQM methods can be used successfully. That is, the fundamental operations of an organization may need to change. Questions such as "Should we be doing this at all?" and "What is it that we do here?" need to be posed to the leadership of an organization before the question "What do we change?" can be addressed.
Methods that bring together re-engineering and TQM need to be developed to bring about the changes most organizations need to make. When coupled with performance measures, they will provide the components of a road map.
Developing these maps requires a detailed set of methods based on an investigation of the components of care related to measures involved in the report cards. Only those who generate and use the data can make such an effort successful.
How or to what extent performance, outcomes or process data will provide sign posts for road maps is yet to be determined. Yet, the fact is that databases and measures are being marketed and used by a variety of healthcare professionals and others to demonstrate, evaluate and change the amount and quality of services provided. The implication is that the marketplace will make decisions on these data and take action whether the data are understood or not.
Providers and those purchasing healthcare are left to their own interpretations of the implications of those decisions. Some have argued that irrespective of the quality of the data, report cards should be used because the data are available. Such a position is irresponsible.
Danger ahead?Report cards without well-defined road maps are of little use and actually may be detrimental to quality of care. This situation is becoming more complicated with the inception of the "information superhighway." Demanding report cards without providing road maps is like turning onto a highway without knowing your destination and how to get there. Such a scenario certainly would cost time and money and result in considerable frustration. In some circumstances it could even be dangerous.
It would seem that until report cards have demonstrated an ability to provide a value-added component to those providing and purchasing healthcare, they remain more a product of vendor marketing and less an instrument of improvement.
