Background
Initial efforts within the health care reform movement in the United States largely focused on reducing cost. This became the preeminent issue as health care expenditures continued to increase on an annual basis, reaching 14% of gross domestic product in 1994. Although some of these expenses can be attributed to improve- ments in diagnostic and therapeutic regimens, particularly in those areas highly influenced by new technologies, concern has also been expressed about the quality of the product being provided to patients.
The publication of the Institute of Medicine’s (IOM) Committee on Quality of Care report,“To Err Is Human: Building a Safer Health System,”
shifted the focus of attention from cost alone to medical errors within the U.S. health care system and the costs associated with these errors. This has led to an erosion of patient trust in the medical care system. It has been estimated that medical errors are responsible for between 44,000 and 98,000 deaths annually, becoming the eighth leading cause of death in the U.S., ahead of other causes such as breast cancer and motor vehicle accidents. Over 7000 of these deaths were attributed to medication errors alone, with the total cost to society of these errors estimated to be $11637.6 billion a year.
Much of the initial research describing the problem of medical errors was performed in the 1990s and supported by the Agency for Health- care Research and Quality (AHRQ).
The IOM report also led to the establishment of the Quality Interagency Coordination Task Force (QuIC), which was charged with coordi- nating the quality improvement activities in U.S.
federal health care programs. These groups have been responsible for the establishment of standardized guidelines and protocols based on clinical trials, which may decrease variability in the patient care processes while improving care and reducing costs. The vast majority of the errors identified in these studies were systems related and not attributable to negligence or misconduct. In fact, up to 75% of the medical errors and 54% of the surgical errors were found to be preventable.
The U.S. Veterans Administration (VA) hospi- tal system had already begun to look at these quality and outcomes issues through the estab- lishment of the National VA Surgical Quality Improvement Program (NSQIP), which has been responsible for assessing specific surgical out- comes throughout the U.S. VA hospital system, while using risk adjusted data to assess surgical morbidity and mortality and providing specific institutional feedback (Khuri et al., 1998). In fact, this program was so successful that in the late 1990s it was expanded to include three large academic medical centers at the University of Kentucky, the University of Michigan, and Emory University in Atlanta. Currently this program is being expanded to include multiple academic medical institutions across the U.S.
In addition, large companies, which are pur- chasers of health care for their employees, have
Outcome Measures in Vascular Surgery
Christopher J. Kwolek and Alun H. Davies
149
begun to take an interest in improving the quality and potentially decreasing the cost of health care as demonstrated by the activities of the Leap Frog Group (www.leapfroggroup.org).
The stated purpose of this group of Fortune 500 companies is to mobilize employer purchasing power to trigger breakthroughs in the safety and overall value of health care to American consumers. Using the results of these and other studies aimed at evaluating medical outcomes, the Leap Frog Group has already identified three areas to decrease errors and improve the quality in U.S.-based health care systems. Health care providers who adopt these recommendations will be rewarded with preferential use and other incentives by this group of health care purchasers. The current recommendations are listed in Table 14.1. It is estimated that imple- mentation of this program would save 60,000 lives with a monetary savings of $3.8 billion a year.
Although these recommendations are sup- ported by reviews of outcome studies in the current medical literature, certain limitations do exist. For example, volume-based outcome studies demonstrate that on average, high- volume centers have better patient outcomes for certain high-risk procedures such as repair of abdominal aortic aneurysm (AAA) or carotid endarterectomy (CEA). However, some high- volume centers may have average or poor out- comes, whereas some smaller centers with low volumes may have excellent outcomes. It may be more important to measure the clinical process
for a health care system to identify the reasons and processes that allow one facility to have excellent outcomes so that these can be utilized by other facilities to achieve the same excellent results. This concept was used by the Northern New England Cardiovascular Disease Study Group to improve the outcomes of patients undergoing coronary artery bypass grafting in northern New England (O’Connor et al., 1996).
Outcomes Measures
It has become important that we evaluate the relative value of different treatment regimens to include individual and societal costs and poten- tial risks and benefits. Individual patient per- spectives on quality have also become an important part of the evaluation process.
There are several reasons for physicians to participate in this process. Ideally, our partici- pation should lead to an overall improvement in the quality of care that we deliver. In addition, participation in these programs will soon be necessary to qualify for reimbursement from many insurers and purchasers of health care such as the U.S. government and the Leap Frog Group. Finally, as physicians we have a societal responsibility to maximize the good and mini- mize adverse outcomes in health care, while best utilizing the limited resources that exist.
If physicians choose not to participate in this process, then others will make these difficult decisions for us.
These changes have led to the development of “extended outcome assessment” rather than just the traditional physician-oriented out- comes measures that we are used to. The con- cept of a value compass has been proposed to describe the interplay between traditional medical outcomes, patient satisfaction, func- tional assessment, and cost/utility outcomes (McDaniel et al., 2000).
Clinical Status
The most traditional group of medical out- comes measures utilized by vascular surgeons usually describes clinical status. These measures are physician oriented and include measure- ments of morbidity and mortality, graft patency, limb salvage, complications, and laboratory testing such as the ankle–brachial index (ABI) Table 14.1. Current recommendations of the Leap Frog Group
Computerized physician order entry: mandated use to minimize medication errors
Intensive care unit (ICU) physician staffing: use of critical care certified physicians to provide exclusive care of ICU patients
Evidence-based hospital referral: preferential referral of patients undergoing five high-risk surgical procedures to “high volume centers”
Procedure Annual volume
requirement Coronary artery bypass grafting 500/year Percutaneous coronary angioplasty 400/year Carotid endarterectomy 100/year Abdominal aortic aneurysm repair 30/year
Esophagectomy 6/year
and duplex ultrasound results. Many of these outcomes are important measures of technical success and are easily evaluable by statistical methods such as life table analysis. However, even the long-term results within this category such as patient survival after aneurysm repair, stroke-free survival after carotid endarterec- tomy, or amputation-free survival after lower extremity bypass grafting do not necessarily correlate with patients’ overall well-being, day- to-day functioning, or their perceived quality of life. Physicians are also comfortable with these outcomes since the Society for Vascular Surgery (SVS) and the International Society for Cardio- vascular Surgery (ISCVS) have published guide- lines for measuring these types of outcomes for commonly performed arterial and venous vas- cular procedures.
Patient Satisfaction
Patient-oriented outcomes tend to be less fre- quently measured and not as well defined. None of these outcomes are included in the SVS- ISCVS reporting standards. Patient satisfaction with the health care process is one area being more closely evaluated. Although customer sat- isfaction surveys have long been used in the world of business, they are now being applied to the area of health care. Two areas being fre- quently evaluated include patient satisfaction with the physician–patient relationship and sat- isfaction with the health care delivery process.
This may include such issues as timeliness and access to care, provider and staff communica- tion, the physical environment where care is delivered, and courtesy and respect shown to the patient. Managed care plans in the U.S. are now required to assess themselves using stan- dardized patient satisfaction surveys such as the Health Plan Employer Data and Information Set (HEDIS), which was developed by the National Committee for Quality Assurance (NCQA).
Many employers and purchasers of health care are now utilizing this information when decid- ing on which health care providers to include in their panel of providers.
Functional Status
Functional status is another of the patient- oriented outcomes that is being more com- monly described in the vascular literature. These
health assessment instruments are designed to quantify how illnesses and treatments affect different aspects of patient functioning in every- day life. This will allow physicians to evaluate not only the presence or absence of a leg or the patency of a graft, but how patients are func- tioning after a specific intervention. These meas- ures are also helpful when evaluating patients’
expectations before and after specific interven- tions.A list of commonly used measures of func- tional status is included in Table 14.2. There are two types of assessment tools. The specific instrument focuses on a specific disease or client group, and changes in this are more likely to detect subtle changes in quality of life, whereas generic tools give a broader summary of health- related quality of life, hence enabling compar- isons with patients suffering from other disease processes (Table 14.3).
The most widely used generic functional health assessment instrument in the United States is the Medical Outcomes Short Form 36 (SF-36). This survey evaluates patient function in physical and social roles, limitations due to health or emotional problems, patient percep- tions of general health, mental health, bodily pain, and vitality. In addition, a question con- cerning change in health status compared to 1 year previously is included in the survey. This survey combines aspects of a Quality of Well- Being Scale and a Functional Status Question- naire. The reason that this survey has become so useful is that large numbers of patients have been evaluated using this instrument, thus allowing researchers to compare results to the general population or a patient subgroup with specific characteristics. These results may be
Table 14.2. Examples of measures of functional status General status: quality of life
SF-36 Euro-QOL
Nottingham Health Profile Functional Status Questionnaire Quality of Well-Being Scale Sickness Impact Profile
Disease/symptom specific: quality of life Walking Impairment Questionnaire: for patients
with lower extremity arterial occlusive disease Charing Cross Claudication Questionnaire Charing Cross Venous Ulcer Questionnaire VascQuol
helpful in predicting long-term functional out- comes in patients. Thus patients scoring above a certain level in the area of bodily pain may have a decreased chance of successfully return- ing to work, and patients with decreased overall scores in the areas of physical and social func- tion and health may have an increased 5-year mortality. However, the abbreviated form of the SF-36, the SF-12, is growing in popularity.
In addition to general evaluations of func- tional status, disease-specific instruments have also been developed. One of the most widely used surveys for the evaluation of patients with lower extremity arterial occlusive disease is the Walking Impairment Questionnaire (WIQ);
however, there are newer and more specific questionnaires becoming available (Chong et al., 2002). Because peripheral vascular disease has a detrimental effect on quality of life even in patients without the most severe forms of limb-threatening ischemia, these instruments may be very useful in evaluating the benefit to patients undergoing treatment for claudication.
These surveys may be even more important for evaluating patients undergoing prophylactic interventions for the management of asympto- matic disease.
Some authors have recommended that a com- bination of two different surveys such as the SF-36 and the EuroQol be used to evaluate the quality of life outcomes in patients with and without ischemic complications who are under- going infrainguinal bypass grafting (Tangelder et al., 1999). Tangelder et al. found that the com- bination of the SF-36 and the EuroQol provided useful information concerning the patients’
quality of life after lower extremity bypass graft- ing. Interestingly, patients’ functional outcomes were similar for those with asymptomatic graft occlusions and patent grafts, although the
lowest outcomes were found in patients who underwent amputation after failed attempts at secondary revascularization. These results confirm clinical findings that are often well known in clinical practice but that are not shown by primary or secondary patency rates or limb salvage.
Additional concerns have been expressed about the potential for patient bias with self- reporting of health status. However, patient reports of functional health status appear to have good face value validity. Thus patients who suffer from severe strokes report worse func- tioning in the areas of physical and general health than patients with mild strokes. Similarly, patients with venous ulcers report impaired social interaction, domestic activity, and emo- tional status with improved functioning after healing of their venous ulcers (Smith et al., 2000). Also, the benefits of varicose vein surgery can be justified in terms of improvements in quality of life scores.
Cost/Utility
Cost outcomes are another important area of recent interest. Increasing pressures are being exerted by the government and insurers to optimize the quality of the health care while minimizing expenditures. True costs to the indi- vidual and society must be calculated both for the acute illness and over the long term.
Practice guidelines can often be established based on the results from existing controlled clinical trials. These guidelines can then be used on a regional or national basis to evaluate physi- cians and health care organizations. Using the same processes of continuous quality improve- ment (CQI) found in major industrial manufac- Table 14.3. Advantages and disadvantages of quality of life instruments
Instrument Advantages Disadvantages
Generic Single instrument May not focus adequately on main problem
Detects different aspects of health May lack responsiveness
Enables comparison between different conditions Does not take account of values attributed to levels Can be used in cost analysis of quality of life
Specific Focus on primary are of interest Not comprehensive More relevant to clinicians and clinical condition May miss side effects
May be more responsive Cannot compare across conditions
turing plants, these guidelines should reduce variation in the processes of care, thus improv- ing quality while decreasing cost. These guide- lines are designed to evaluate processes, with the ultimate philosophy of continuously impro- ving the process to ensure the best possible outcome.
This process is in contradistinction to the more traditional concept of quality analysis (QA), where minimum standards or thresholds are set to ensure a good outcome and only out- liers are evaluated rigorously. The pitfall of this approach is that it encourages organizations to be just “good enough” and meet the minimum standards set rather than aim for being the best possible. In addition, it singles out only those groups or individuals doing a poor job rather than rewarding groups that are doing a good job and trying to reproduce those results in other areas.
The value that patients and society place on certain outcomes and levels of functioning can also play a role in evaluating the cost- effectiveness of certain procedures. The cost- effectiveness of a certain intervention can be expressed as the net benefit to a population by using statistical probabilities of certain out- comes occurring along with specific costs asso- ciated with various outcomes as documented from the medical literature. One example of this is the use of the Markov decision model to evaluate the cost-effectiveness of performing carotid endarterectomy in asymptomatic pa- tients with >60% stenosis (Cronenwett et al., 1997). The measurement units in this study are in quality-adjusted life years (QALYs) defined as the fraction of a year in perfect health that the patient believes to be equivalent in value to a year in the health state in question.
In this study, Cronenwett et al. demonstrated that from a societal standpoint, carotid endar- terectomy appeared to be cost-effective for the young asymptomatic patient with standard risk factors in the hands of a surgeon with a 2.3% 30-day perioperative stroke and death rate.
However, the procedure was not found to be cost-effective for patients older than 79 years of age, those with a high perioperative stroke risk, or those with a particularly low stroke risk with medical management. In this study it is impor- tant to note that cost-effectiveness is compared to cost/QALY for other medical interventions, and that a cost of over $100,000/additional
QALY was defined as not cost-effective. Inter- estingly, one could extend the evaluation even further to include the individual’s risk aversion and risk taking behavior when defining the cost-effectiveness for that person. This is es- sentially what occurs every time we obtain informed consent from a person who is about to undergo a high-risk procedure.
Conclusion
The reasons for physicians to utilize extended outcomes assessment include the following:
(1) to achieve a better understanding of the effectiveness of our interventions; (2) to pro- vide health care consumers, both patients and insurers, with information that will allow them to make better informed decisions; and (3) to develop public health standards that will allow us provide the most cost-effective care (McDaniel et al., 2000).
In addition, the Accreditation Council for Graduate Medical Education (ACGME) in the United States has mandated that all residency training programs evaluate as part of their core competency requirements six areas, all of which involve some component involving outcomes measures: (1) practice-based learning, (2) systems-based practice, (3) medical knowledge, (4) patient care, (5) interpersonal and com- munication skills, and (6) professionalism (www.acgme.org/outcomes).
Finally, the incorporation of outcomes meas- ures are now being studied by the task force on competence of the American Board of Medical Specialties to be used in the process of recer- tification.In fact,recommendations have already been published stating that outcomes measures should be included as part of the requirement for recertification in vascular surgery by the American Board of Surgery (Hertzer, 2001).
References
Chong PF, Garratt AM, Golledge J, Greenhalgh RM, Davies AH. (2002) J Vasc Surg 36:764–71; discussion 863–4.
Cronenwett JL, Birkmeyer JD, Nackman GB, et al. (1997) J Vasc Surg 25:298–309; discussion 310–1.
Hertzer NR. (2001) J Vasc Surg 34:371–3.
Khuri SF, Daley J, Henderson W, et al. (1998) Ann Surg 228:491–507.
McDaniel MD, Nehler MR, Santilli SM, et al. (2000) J Vasc Surg 32:1239–50.
O’Connor GT, Plume SK, Olmstead EM, et al. (1996) JAMA 275:841–6.
Smith JJ, Guest MG, Greenhalgh RM, Davies AH. (2000) J Vasc Surg 31:642–9.
Tangelder MJ, McDonnel J, Van Busschbach JJ, et al. (1999) J Vasc Surg 29:913–9.
