3 Decision Analytic Techniques

21

3

Decision Analytic Techniques

Anirban Basu and Amy G. Lehman

clinical trials, as well as incorporate patients’

preferences to inform the fi nal decision. There are several published reviews and books that are entirely devoted to this topic. Interested readers are pointed to these references.

This chapter is intended to provide a succinct description of the fundamental theory and methods of decision analysis to clinicians, and to illustrate a stylized example of building a DAM based on a clinical choice problem in thoracic surgery.

3.1. Theoretical Foundations

The goal of a DAM is simple – it compares two or more treatment decisions or strategies for dealing with the same problem, and applies a set of rules to identify the preferred decision that would produce the maximum benefi t or the minimum loss. Expected utility theory is the most com- monly used rule to calculate the fi nal outcome in terms of benefi ts (or loss). Under this theory, the fi nal outcome resulting from a treatment deci- sion is given by

B

p b

= ∑

⋅

^(3.1)

where payoff values (b

) are associated with each level (l = 1,2, . . . , L) of success (or failure). The expected outcome is then calculated by summing over all levels, the product of the probability of a specifi c level of success (p

) with its correspond- ing payoffs. Expected utility theory suggests that the preferred treatment decision is the one with the maximum expected benefi t (or minimum expected loss).

Accumulation of new and more reliable informa- tion has been monumental over the last decades, mediated via unprecedented growth in biomedi- cal and associated social sciences research.

This research has certainly played a key role in the tre- mendous improvement of health throughout the world. It has also complicated decision making, both at the individual and at the policy level, by presenting clinicians with an increasing number of medical technologies and strategies for the management of a given medical situation. A fun- damental concern in clinical decision making is how to synthesize information about the effect of a medical intervention on patients with specifi c characteristics. Furthermore, an additional criti- cal step involves integrating population level evi- dence about outcomes with patient-level values for these outcomes in order to produce individualized care. Thus, individualized decisions for patients demand a systematic approach to sort through the evidence and to incorporate patients’ and their family members’ values and preferences into the decision-making processes, so as to com bine the much-acclaimed evidence-based medicine

with the pragmatism of shared decision making.

In this context, the decision analytic model

(DAM) has proven to be a successful tool that

can provide a systematic approach for decision

makers.

These models enable the clinicians to

apply a systematic, quantitative approach called

decision analysis and thereby assess the relative

value of one or more options or strategies to

approach a particular medical problem. Further-

more, the approach allows them to pool evidence

from a variety of settings, including randomized

Although almost all DAM perform under the auspices of this simple theoretical intuition, any DAM that attempts to model a practical clinical situation can become complicated very quickly.

Each level of success at the end depends on a sequence of chance outcomes and various inter- mediate decisions that, in turn, may depend on further chances and decisions. Therefore, under- standing, defi ning, and structuring the decision process is essential for clinical decision analysis.

In this pursuit, a fundamental decision tool is a decision tree that helps the clinician to systemati- cally display the temporal and logical structure of the decision problem and to thus carry out the analysis. In order to facilitate explanation of each part of this process in more detail, we begin with a stylized example of clinical decision making in thoracic surgery.

3.2. A Motivating Example

Let us begin with a common problem in thoracic surgery – the solitary pulmonary nodule – that can illustrate the basic principles of decision analysis. Suppose we have a patient referred to a thoracic surgery clinic. She is a 65-year-old woman who presents with an incidentally found solitary pulmonary nodule discovered on chest X ray taken in an emergency room after a minor motor vehicle collision. She has a 35 pack-year smoking history, but quit smoking 6 years ago.

She has well-controlled hypertension, no heart disease, and no diabetes. She has no previous chest X ray with which to compare to this new abnormal one. The patient underwent a com- puted tomography (CT) scan scheduled by her primary care doctor, and this scan shows an 8- mm peripheral nodule with no enlarged lymph nodes. Let us further specify that due to the size of her lesion, positron emission tomography (PET) scan is nondiagnostic, and that due to her particular anatomy, she is not a candidate for a video-assisted thorascopic resection (VATS) biopsy. How shall we advise this patient? We are now faced with a choice in recommendations:

watchful waiting, or perform open diagnostic thoracotomy. There are several considerations that inform our decision that have been discussed in detail elsewhere

; namely, we must consider

the pretest probability of cancer, the risk of surgi- cal complications, and whether the appearance of the nodule on CT suggest benign or malignant disease. Finally, what are the patient’s prefer- ences given the possibility of adverse outcomes of open surgery? A DAM can help in the systematic integration of all this information in order for the surgeon to make an individualized and informed decision. A simple, stylized decision tree to address this clinical decision is illustrated in Figure 3.1. We now discuss in detail each part that goes into making up this decision tree and how one would arrive at the fi nal result.

3.3. Elements of the Decision Analytic Approach

3.3.1. Identify and Bound the Decision Problem

The fi rst step in decision analysis is to under- stand the decision problem at hand and the par- ticular issues associated with making that decision. To do this, one must defi ne the set of alternative actions under consideration and the primary outcome measure based on which deci- sion will be made, determine the perspective and the time frame of the analysis, and consider several factors, such as the clinical characteris- tics of the patient, which may infl uence the primary outcome measure. These considerations can be broadly classifi ed into the following:

1. Defi ne the set of alternative actions. Deci- sion analysis always presumes that there is more than one action for the same decision problem. If this were not the case, there would be no decision to make. Note that one of these alternative actions may, and often does, include the option to do nothing. In our stylized example, the alternatives are watchful waiting versus thoracotomy.

2. Perspective. The most common perspective

taken in clinical decision making is that of the

patient, whose welfare is the fundamental

outcome for the decision at stake. This is also the

perspective we take in our example. Chapter 4

(this book) explores the implications of the

patient perspective in greater detail. Neverthe-

less, it is easy to foresee that a clinical decision

based on a patient’s perspective will be integrally

tied with the preferences of that individual patient. Comprehending an individual patient’s preferences and incorporating them in the deci- sion process is essential for optimal decision making. However, in certain acute conditions, where the patient’s preferences are unknown, alternative preferences of family members or sometimes those of clinicians may be used as proxies. More broadly, depending on the types of questions asked, perspectives of different stake- holders – for example, the hospital, the health insurance company, and even society at large – may become relevant in the analyses.

For example, from the hospital’s perspective, reduc- ing inpatient mortality may be more important than a patient’s potentially diminished quality of life due to side effects of treatments. Outcomes such as costs and cost effectiveness may be more relevant to health insurance companies than to individual patients.

3. Clinical conditions and demographics of patient(s). Several risk factors affect patient out- comes and therefore are important to consider when choosing between alternative therapies. A patient’s clinical condition constitutes the funda- mental source of information for appropriate medical care. For example, surgeons require clini- cal information on the stage, and often grade, of a cancer before performing a surgical resection.

Often, information on comorbidities, such as genetic susceptibilities to malignancy, becomes critical for prescribing appropriate treatment.

Demographics also play a key role in determining outcomes. For example, pretest probability of cancer would depend on a patient’s gender, smoking history, age, and possibly many other factors.

4. Time frame of analysis. The time frame of analysis should refl ect the time over which the consequences of the clinician’s choices have the potential to infl uence the patient’s survival and

Dead

(1%) Dead

Payoff (QOL) 0 0.5*q1

0.5*q1 0.5*(q1+q2) 0.5*(q1+q1) Lobectomy

Lobectomy

(3% + 12%) Stage I

Stage II (54%)

Stage I

Stage II (54%) Cancer

No Cancer (37%)

Cancer

Watchful Waiting

No Cancer (37%) No Comorbidities (Th) Comorbidities (Th)

with Comorbidities (Lb) (25%) with no Comorbidities (Lb)

Same as Comorbidities (Th) branch

(Outcomes depend on only comorbidities due to Lobectomy) Thoracotomy

Alive

Alive

Dead

0.5*q1

0.5*q3

q3 0.5*q3

q3 0.5*q3

q3 q1 0.5*(q1+q2) 0.5*(q1+q1) (3% + 12%)

with Comorbidities (Lb) (25%) with no Comorbidities (Lb) Alive

Dead (18%)

Alive Dead (36%)

Alive Dead (12%)

Alive Dead (12%) Alive (19%)

FIGURE 3.1. A decision tree to model the clinical decision between thoracotomy and watchful waiting.

quality of life. In some acute conditions, the time frame may be the time until the patient is sent home from inpatient care. In chronic conditions, the time frame may extend up to the patient’s remaining life expectancy. In the clinical choice problem that we illustrate, the ideal time frame should be the lifetime of the patient because both the disease process as well as the potential complications of surgery infl uence the patient’s quality of life over her entire lifetime. However, because our analysis is a mere illustration of the concepts and not a substantive analysis, we will use a time frame of 1 year.

5. Primary outcome of interest. The optimal clinical decision may vary depending on what type of benefi ts the patients and/or the clinicians want to maximize. Identifying the primary outcome on which the fi nal decision is made is perhaps the most crucial issue in decision analy- sis. Is the primary concern the survival of the patient over the next few months, or is the overall quality of life for the patient over his/her remain- ing life expectancy the most relevant measure to dwell on? Answering this question often involves incorporating the patient’s preference and per- spective, as well as determining the overall goal of medical treatment. It also uniquely determines what type of analysis the clinician is interested in.

The types of analyses can be broadly classifi ed into four categories based on the types of out- comes being evaluated: (1) clinical outcomes; (2) patient’s values about the clinical outcomes; (3) costs, and (4) both costs and outcomes.

Outcomes Analysis: In such analysis, neither costs nor patient preferences are used to choose the optimal decision. Instead, the focus is entirely on one of the clinical outcomes (e.g., survival or length of hospital stay) that are used as the primary outcome of interest. The optimal treat- ment is selected based on the most benefi cial clinical outcome (e.g., lowest mortality or short- est length of hospital stay).

Utility Analysis: In these analyses, costs are not incorporated in the decision-making process;

however, a patient’s preferences are included, in conjunction with the relevant clinical outcomes.

Most clinical decisions have an effect not only on life and death, but also on the quality of life of patients, mediated through a variety of health states. In utility analyses, the value of any par-

ticular health state to the patient, popularly known as utility or quality of life (QOL) weight, is measured using either a time-tradeoff or stan- dard gamble method.

The utility for any health state is constructed to lie between 0 (representing death) and 1 (representing perfect health). In time-tradeoff methods, patients are asked to trade-off a longer time in a particular health state for a shorter time in perfect health.

In standard gamble methods, patients are asked to choose between living with a particular health state and a gamble between perfect health and death. The utility for the health state under con- sideration is obtained at the point of indifference between the choices in either method. These util- ities, multiplied with the duration of time that the person is in that health state under a specifi c decision choice, form the quality-adjusted life- years (QALYs) corresponding to that decision.

The decision producing the maximum QALYs is chosen to be the optimal one. The advantage of utility analyses over outcomes analyses is that a variety of outcomes that infl uence the patient’s overall quality of life can be summarized using one generic measure such as the QALYs, and therefore the effects of a decision on multiple out- comes can be simultaneously determined and compared to other decisions.

We use QALYs as the primary outcome of inter- est in our analysis.

Cost Analysis: In these analyses, only the costs of alternative treatments form the primary outcome of interest and the least costly treatment is recognized as the optimal choice.

Such analy- ses are carried out when the clinical outcomes of the alternative treatments are not a contentious issue – a situation that is becoming increasingly less common in clinical practice.

Cost-effectiveness Analysis: Cost-effectiveness

analysis (CEA) compare both the resources used

(costs) and the health benefi ts achieved (e.g.,

QALYs or simply life years) among alternative

treatments, making these trade-offs explicit to

both the clinician and to the patient so that

together they can make the optimal decision for

the patient.

The practice of cost-effectiveness

analysis when comparing two interventions, for

example, a new treatment versus standard care,

can be summarized as follows: the fi rst step is to

calculate the mean costs incurred and the mean

benefi ts produced by each intervention; next, an incremental cost-effectiveness ratio (ICER) is formed by dividing the difference in the mean costs over the difference in the mean benefi ts between the new and standard interventions. The ICER represents the additional costs required by the new intervention in producing one extra unit of benefi t over that produced by the standard intervention. The ICER is then compared with the threshold value that represents the maximum a decision maker is willing to pay for an addi- tional unit of benefi t.

If the ICER is lower than this threshold value then the new intervention is deemed to be cost effective.

Because, in most cases, a substantial portion of the costs of health care is often borne by health insurance and not the patient, several interesting normative issues arise when an attempt is made to compare both costs and benefi ts simultane- ously. A widely debated question revolves around whose perspective is most appropriate to consider when the burden of costs is distributed amongst patients, healthcare providers, and third-party payers. Furthermore, obtaining a threshold value that represents the maximum willingness-to-pay for an additional unit of benefi t is diffi cult to ascertain at the individual patient level. However, in order to preserve some notion of fairness, one can uniformly apply a societal threshold to all patients. Such discussions are beyond the scope of this chapter but interested readers are encour- aged to explore this important literature.

6. Other considerations. Several other consid- erations may infl uence treatment choices. They include information on how much weight patients place on outcomes that will arise in the future compared to those at present time (this is popu- larly summarized by the concept of the discount rate),

whether consideration of the effects of patients outcomes on the family members are important,

and how patient demographic char- acteristics, health insurance status, and out-of- pocket payments infl uence the primary outcome of interest.

3.3.2. Structure the Decision Problem over Time

Although choosing between alternative actions is the primary goal of a decision analysis, often the

decision problem will involve a temporal sequence of choices that inevitably infl uence the fi nal choice of action. Moreover, these choices may themselves depend on certain chance outcomes that may or may not be controlled by previous decisions. Therefore, the second step in decision analysis is to identify the components of the deci- sion problem. To do this, one defi nes a structure for the temporal and logical sequence of choices, chances, and outcomes, as well as their interac- tions, which would lead to the fi nal outcome of interest as defi ned in the previous step. A deci- sion tree helps to structure this temporal decision problem in a systematic format. An excellent primer for building decision trees is given by Detsky and colleagues.

Figure 3.1 illustrates the decision tree for our simple example on the choice between proceed- ing to open thoracotomy and watchful waiting.

Each point in the decision tree that leads to multiple outcomes or decisions is called a node.

There are two types of nodes: (1) a decision node is indicated by a square box, and (2) a chance node is indicated by a circle. A decision node represents the alternative choices that are avail- able to the clinician. A chance node represents the alternative outcomes and patient’s responses that are possible. Each chance node is associa- ted with a probability with which a specifi c outcome is realized, thereby accounting for the inherent uncertainties in such processes. The rightmost column of the decision tree illustrates the fi nal payoffs or outcomes associated with each possible branch of the decision tree. Each branch can be visualized as a level of success given the initial choice of treatment, and is defi ned by the sequence of events starting with the initial treatment choice and leading up to the fi nal outcome.

In our example, we assume that the clinician makes a decision aiming to maximize the patient’s QALYs over the next 1-year period after consider- ing the potential risks and benefi ts of each choice.

If the patient undergoes open thoracotomy, the

benefi ts will include defi nitive diagnosis (cancer

or no cancer), as well as pathological staging and

a potential curative resection (lobectomy) with or

without adjuvant therapy if the nodule is indeed

malignant. The risks include possible morbidity

and mortality from both thoracotomy and

lobectomy (if needed). These outcomes generally depend on a number of issues, including the experience of the surgeon and the patient’s other medical comorbidities, as well as the risk of development of other chronic problems such as a post-thoracotomy chronic pain syndrome that would affect the patient’s quality of life.

If the patient undergoes watchful waiting, she avoids the comorbidities associated with thora- cotomy. However, the patient is then subjected to a higher mortality risk if the nodule is indeed malignant; she may also suffer from the anxiety of not knowing whether she has cancer, and therefore a reduced quality of life. Please note that the time frame of 1 year does not permit the increased risk of mortality associated with a cancer diagnosis and the choice of watchful waiting to become fully realized in the real world.

We have arbitrarily chosen this time frame to simplify our tree, and facilitate the model. There- fore, the reader should be cautioned to not use this model for actual clinical decision making, as it is not clinically accurate.

The non-cancer yearly mortality risk applies irrespective of the choice. The associate payoffs include the quality of life weights of the patient.

Death from thoracotomy happens at the begin- ning of the period and so is assigned a QOL value of 0. Death from lobectomy or natural death is assumed to occur at the middle of the year; so half year of life is weighted by the patient’s QOL during that time. Under the thoracotomy arm, if the patient stays alive for the year, her QOL depends on the presence or absence of comor- bidities. Under watchful waiting, death from cancer or natural death is also assumed to occur at the middle of the year. If the patient stays alive, then her quality of life is determined by her level of anxiety about cancer.

Note again, that the model in Figure 3.1 is illus- trative and uses a very simplifi ed and stylized version of an actual decision-making process.

Several additional factors, such as the sensitivity and specifi city of detecting cancer through tho- racotomy, differences between clinical staging, and the true pathological stage and recurrence of cancer post-lobectomy, to name a few, must be considered for developing a comprehensive model that can appropriately represent this clinical situation.

3.3.3. Characterize the Information to Fill in the Structure

Once the clinician has identifi ed the sequence of choices to be made and the sequence of chance nodes and their associated outcomes determined by these choices, the next step is to obtain infor- mation on these chances and outcomes. Choices at each step of the decision process can be made based on this information. This is the critical part of the decision model because the quality of information that goes into these chance nodes entirely determines the credibility of the decision model and the decision it generates.

Bayes’ formula provides the key theoretical insight for determining what types of informa- tion are required.

The classic example for application of the Bayes’ formula lies in the inter- pretation of results from a diagnostic test. For example, given that a diagnostic test correctly detects a clinical problem 90% of the time (speci- fi city – a measure of prior belief ) and correctly detects the absence of a clinical problem 85% of the time (sensitivity – also a measure of prior belief ), what is the probability that the patient has the problem (posterior belief about success in diagnosis) given a positive or negative test result (evidence)? This question is readily answered using the Bayes’ formula. We now provide a more intuitive discussion on the Bayes’ formula.

A clinician is often interested in knowing about the probability of success (at any chance node) associated with a treatment decision for a patient, based on the current evidence on success rates with that decision. However, the clinician can only observe, from published research and his or her own experience, the likelihood of evidence given some underlying prior belief about success rates. Bayes’ formula, as shown below, helps the clinician to go from the latter quantity to the former one.

Pr

Pr Pr

Pr Success Evidence

Evidence Success Success Evidence

( )

= ( ) × ( )

( ) ^(3.2)

where Pr(x|y) represents probability of x condi-

tional on or given y. Here, P

(Evidence|Success)

indicates the likelihood (or probability) of

observing the data that clinicians observe in

practice or in clinical trials given a specifi c

hypothesis about the prior belief on success rates.

Often, there will be uncertainty regarding the true success rates and there may be more than one prior belief. However, once new evidence (e.g., observed data in clinical trials) is revealed, the likelihood of that evidence under a variety of prior beliefs becomes known. Consequently, one can feel more confi dent to restrict the beliefs on success rates to those ranges that correspond to the highest likelihood for the evidence. These new updated beliefs then form the evidence- based posterior beliefs based on which that particular chance node in the model can be informed.

Note that although this exposition of Bayes’

formula suggests a prospective process of updat- ing beliefs and information about specifi c param- eters in the model, it also can be readily applied to generate a current estimate of success rates that seem most likely given all the evidence that has accumulated to date. Thus, the role of evi- dence is important because more evidence would tend to strengthen the posterior beliefs and dispel a larger part of uncertainty associated with prior beliefs. Incorporating all relevant evidence into the model is accomplished by a detailed search on the published and possibly unpublished research literature that point to evidence for the specifi c parameters in the model. Posterior esti- mates of parameters are generally weighted by the quality of evidence as defi ned by sample size,

study design, and use of robust analytical methods.

Information gathering for decision analysis almost always uses one or more of the following:

literature review, meta-analysis, primary data collection, and consultation with experts.

The relevant information for our model and their sources are outlined in Table 3.1. Because we are using a stylized example, we will use point estimates for our model parameters that reason- ably lie within the widely disparate ranges that are reported in the literature concerning morbid- ity and mortality from thoracic procedures. In a more formal treatment of this problem, one has to pay considerable attention to pooled and meta- analyzed available evidence so as to obtain esti- mates that more closely refl ect true values and also properly account for uncertainties from multiple sources. We discuss these sorts of issues further in section 3.5.

Attention should also be paid to the timeliness of information. For example, estimates given in Table 3.1 do not account for the fact that pub- lished literature shows a consistent trend over time toward lower morbidity and mortality from thoracic procedures, probably resulting from a combination of improved technique, improved anesthesia, and improved performance by dedi- cated thoracic surgeons, as well as the vast improvements in adjuvant chemotherapy therapy now available to patients.

TABLE 3.1. Information on parameters for the choice between thoracotomy and watchful waiting.

Description Value References

Pr(death due to thoracotomy) 1% 24

Pr(cancer|patient characteristics) 37% 25

Pr(stage I|cancer, patient characteristics) 54% 26

Pr(death|lobectomy, stage I cancer)^a 3% + 12% 27

Pr(co-morbidities|lobectomy, no death) 25% 28

Pr(death|lobectomy, stage II cancer)^a 5% + 12% 27

Pr(comorbidities|thoracotomy, no death) 19% 29

Pr(death|watchful waiting, stage I cancer) 18% 30, 31 Pr(death|watchful waiting, stage II cancer) 36% 30, 31 Pr(death|watchful waiting, no cancer, patient characteristics) 12% 32, 33, 34 QOL(alive with no comorbidity & no cancer) 1 Standard QOL(comorbidities due to lobectomy) = q2 = 0.9*q1 Assumed

QOL(comorbidities due to thoracotomy) = q1 0.90 Assumed

QOL(alive with the anxiety of knowing that cancer might be present) = q3 0.80 Internal data

QOL(death) 0 Standard

aWe model this probability as inclusive of the probability of natural death given no cancer, i.e., Pr(death|lobectomy, stage I cancer) = Pr(death due to lobectomy) + Pr(death|watchful waiting, no cancer, patient characteristics).

3.3.4. Apply Decision Analysis

The fi nal step is to synthesize the information gathered and the choices made in the process of decision making. This will allow us to obtain estimates of the primary outcome of interest that can be directly compared, in order to choose among alternative actions. This is accomplished by working backwards, that is, from right to left, on our decision tree for which we have fi lled in probability values for nodes. At decision nodes, we roll back (alternatively, the term fold back is also used

) along the best choices – in effect, the choices that maximize gain or mini- mize harm. At chance nodes, we average out along all branches, yielding an expected value, such as QALYs, expected number of days of hos- pital stay, or expected risk of postoperative sur- vival, etc.

Consider the expected QALYs of lobectomy for the patient after she survives thoracotomy with comorbidities and is detected with stage I cancer and undergoes lobectomy. Let the q1 and q2 be utility weights for comorbidities due to thora- cotomy and lobectomy, respectively. As men- tioned previously, we also assume that death occurs on average at the middle of the year. The levels of outcomes possible are (1) death from lobectomy (payoff = 0.5*q1); (2) survive with comorbidities from lobectomy (payoff = 0.5*q1 + 0.5*q2); and (3) survive without comorbidities from lobectomy (payoff = q1). The overall expected payoff is then calculated using the formula in Equation 3.1, by multiplying the corresponding probability of each level of outcome (Table 3.1) with it corresponding payoffs. This process is repeated for each possible level of outcome under thoracotomy and under watchful waiting.

Based on the parameter estimates in Table 3.1, thoracotomy produces an expected QALY of 0.899 while watchful waiting produces an ex - pected QALY of 0.731. This is our baseline result, which reveals that thorocotomy may be the optimal decision for this patient.

3.3.5. Sensitivity Analysis

Clinical decision making using a DAM is usually followed by an assessment of the strength of the

fi nal decision that can be accomplished by quan- titatively assessing the sensitivity of the fi nal decision to the structural assumptions of the models and feasible alternative information sets.

This is done by substituting a range of values for those parameters that are believed to be the most variable in practice. If the conclusions of the model are robust over a range of values for each node, then one can feel very comfortable with the decision that the model suggests. If the decision changes as values are varied, then one must be pay closer attention to those par- ameters and try to get a better sense of the par- ameters for the case at hand. This exercise will produce a threshold value where two deci- sions stand at equipoise. In this latter case, the strength of the data used to generate the parameter estimates becomes exceedingly important.

In our case, we vary the QOL of the patient anxiety and that of the comorbidities of thora- cotomy to see how our baseline result change.

Figure 3.2 shows the phase diagram for this sensitivity analysis. In the diagram, the y-axis represents different levels of QOL for anxiety, while the x-axis represents different levels of QOL for thoracotomy-related comorbidities. The area in the graph identifi es the regions where, conditional of the respective QOL weights, either thoracotomy or watchful waiting is more benefi - cial. As evident from Figure 3.2, a patient with high levels of anxiety [i.e., low QOL(Anxiety)]

would benefi t from thoracotomy. A patient with low anxiety [i.e., high QOL(Anxiety)] but low QOL(complications of thoracotomy) would benefi t from watchful waiting. Such phase dia- grams can help clinicians identify where their patient lie in this graph so that they can make an informed decision about treatment choices.

Phase diagrams, as in Figure 3.2, can be pro-

duced based on several other parameters in the

model. However, multiple phase diagrams can

easily make the decision as complicated as it

was to begin with, and hence lose the utility of

DAMs. Such scenarios, where there are consider-

able uncertainties and heterogeneity in several

parameters in the model, can be addressed using

probabilistic analysis, which we discuss in

section 3.5.

3.4. Clinical Decision Analysis in Thoracic Surgery

Heretofore, DAMs have been used to address a number of issues and/or problems in thoracic surgery. Many of these issues and controversies will be more thoroughly explored in the context of this book. Some examples of models that have been published in the past include those that evaluate different surgical techniques,

whether or not to proceed to more invasive and/or aggres- sive treatments given a common problem in tho- racic surgery,

and appropriate management of certain types of metastases,

to name a few. The perspectives that have been used include: the patient’s perspective (as we have, and is most common in clinical DAMs); the treating hospi- tal’s perspective; and the government’s perspec- tive, via Medicare and Medicaid programs. Many DAMs use clinical conditions of patients that have been somewhat simplifi ed, although this may change as older, sicker patients undergo tho- racic procedures, and new data are generated about their outcomes. Time frames tend to be dominated by traditional markers of success or failure in surgery; that is, 30-day morbidity and mortality or 1- to 5-year survival. There are centers,

however, that have published long-term data. Such information will become increasing relevant and important as outcomes of interest

expand to include measures like QALYs and cost- effectiveness ratios that usually require long time frames for appropriate conclusions to be drawn.

These longer term time horizons are more com- monly found when a DAM incorporates data from sources like the Surveillance, Epidemiol- ogy, and End Results database or the Veterans Administration Hospitals extensive database, where patients are followed for many years.

Several different primary outcomes of interest have been measured, including mortality, par- ticular morbidities for certain operations, and length of hospital stay, as well as QOL, cost effec- tiveness, and patient satisfaction.

The chief problem with many of the published DAMs revolves around the fact that almost all point estimates are generated from a few retro- spective studies or blinded, prospective, random- ized trials, mostly comprised of a handful of patients. The robustness of these estimates can be greatly improved by meta-analyzing the data across studies and properly accounting for the uncertainties arising out of different sources.

For example, one DAM compares three choices concerning optimal management for patients undergoing esophagectomy for carcinoma of the esophagus

and bases all probabilities on a single, randomized, prospective trial that compared the use of pyloroplasty with non-use.

However, in the 72 patients enrolled, there was no statistically signifi cant difference demonstrated between

baseline QALY result with respect to QOL weights for patient’s anxiety and of thoracotomy-related comorbidities. The shaded and the blank areas in the graph identify the values of QOL weights for which thoracotomy or watchful waiting produces the maximum expected QALY, respectively.

populations. In addition, there was little explora- tion of the potential negative side effects of both procedure non-use and use from the patient’s per- spective; that is, which might decrease quality of life more: delayed gastric emptying or dumping syndrome? Though the model is very insightful, it is also quite hypothetical – just as our example is in this chapter. Therefore, to overcome these sorts of limitations, one can employ more advanced techniques that have been developed by practitioners of decision analysis.

3.5. Advanced Issues in Decision Analytic Techniques

The discussions below are an attempt to provide a general understanding of the advanced issues in modeling and analysis. It does not, in any way, serve to provide a comprehensive review of these topics.

More importantly, we recommend that clinicians interested in taking advantage of these advanced techniques consult with professionals who are well versed with the nuances of these methods.

3.5.1. Markov Models

Although decision trees provide intuitive repre- sentation of the disease process and the conse- quences of choice over a short time period, they can get extremely complicated when extended over long periods of time. To address this situa- tion, one can use a Markov process that is a mod- eling technique based on matrix algebra. The

fundamental idea behind these models is that, instead of considering health state transitions over a short period of time, as in decision trees, a Markov process is concerned with transitions during a series of short time intervals or cycles.

For example, consider the watchful waiting arm of the decision tree in Figure 3.1. One can revise the model to include a bi-annual follow-up for detecting whether the lesion is growing or not. The probability that the clinician would fi nd an advanced lesion in the next follow-up will be based on the natural progression of cancer over time. In order to model the progression of cancer, the cancer can be delineated into mutually exclu- sive health states based on its stage and grade. An example of such a model, borrowed from our col- league David Meltzer’s work in prostate cancer, is shown in Figure 3.3, also known as a bubble diagram. Each health state, defi ned by a combi- nation of stage and grade, is represented by a bubble. The fi gure represents progression of cancer by stage and grade leading up to cancer- related death. The patient begins in one of these bubbles and in every cycle, with some transition probabilities, either stays in that bubble or moves to another bubble representing an advanced health state. The sum of all transition probabili- ties in a cycle must add up to one because, by defi nition, a patient has to be in one of the mutually exclusive health states. The transition probabilities can vary with time (or number of cycles) and also with patient characteristics such as age of diagnosis. The transition probabilities determine the progression of the disease over

FIGURE 3.3. Structure of a Markov model for cancer progression

Stage I Low Grade

Stage I Mod. Grade

Stage I High Grade

Stage II Low Grade

Stage II Mod. Grade

Stage II High Grade

Metastatic Low Grade

Metastatic Mod. Grade

Death from Cancer

Metastatic High Grade

time, and therefore are critical in evaluating the costs and outcomes of any intervention at a particular point in time. We refer interested readers to some of the applications of Markovian processes in decision models available in the literature.

3.5.2. Probabilistic Analysis

Uncertainty remains an integral part of any analysis. Although a part of this uncertainty is refl ected in the probability estimates used in decision models (e.g., such as those in Table 3.1), they do not refl ect the overall uncertainty for an outcome. For example, if we say that Mrs. X has a 40% chance of having cancer, we make a state- ment about a population comprised of millions of Mrs. X clones in which 40% of them will have cancer and 60% will not. When the clinician is faced with one Mrs. X from that population, her true cancer status is not known with certainty.

But by incorporating the point estimate of 40%, the clinician can characterize the uncertainty that she might fall in the part of the population who has cancer. This type of model, as the one we illustrate above, is called a deterministic model as one uses predetermined point estimates for probabilities to refl ect the uncertainty.

However, we seldom know how many patients in that population truly have cancer; instead, we rely on a random sample from the population to determine how many patients in that sample have cancer. If we fi nd 40% of patients in the sample have cancer, it gives us a reasonable estimate for the population, but not the exact estimate. That is, 35% of patients in the population may truly have cancer, while in the random sample we have chosen, 40% show up with cancer. The bigger the sample size, the closer will be the sample esti- mate to the population estimate. Hence, there remains a degree of uncertainty about the true value of the population parameter because we almost always infer based on a fraction of that population.

The traditional way to deal with uncertainty is to a perform sensitivity analysis by varying the parameter of interest across a range of values and observing the changes in outcomes associated with it, as we have shown in our stylized example.

However, there are several limitations to this

method. One-way or two-way sensitivity analysis may severely undermine the effect of uncertainty in a multiparameter model. Multiple-way sensi- tivity analysis can easily become extremely complicated as to disallow straightforward inter- pretation. Even in the absence of complicated analysis, traditional sensitivity analysis can iden- tify the optimal decision if only the true values of the parameters are known, as, for example, in the case of patient preferences that can be directly measured. This poses problems for the clinicians who may often fi nd it hard to use sensitivity anal- ysis results in clinical decision making due to lack of guidance and uncertainty about the true value of many clinical parameters.

Probabilistic analyses help to characterize these uncertainties in the model parameters and to arrive at the fi nal decision by simultaneously averaging out the uncertainties in multiple parameters. In these models, instead of specifi c point estimates of different parameters in the model, one specifi es distributions for these input parameters where these distributions might center on the point estimates. Using Monte Carlo techniques, a patient is then propagated through the model several times (iterations), each time with random values of a parameter drawn from its respective distribution. The costs and out- comes are then averaged over all iterations and compared across treatment options.

A good example of a probabilistic analysis can be found Andrew Brigg’s work on gastroesopha- geal refl ux disease.

3.5.3. Bayesian Meta-analysis

Bayesian meta-analysis is a sophisticated hierar- chical modeling approach that utilizes the concept of Bayes’ theorem discussed earlier. It summarizes and integrates the fi ndings of research studies in a particular area. One can obtain the posterior distribution of Pr(Success|

Evidence), that is the current distribution of

belief about success rates given all the past evi-

dence, and directly use it as an input in a proba-

bilistic analysis. Such an approach provides a

combined analysis of studies that indicate the

overall strength of evidence for a success while

properly accounting for multiple sources of

uncertainty.

For example, suppose there are four studies with varying sample sizes indicating that a woman with a smoking history similar to that in our example has a pretest probability of cancer of 60%, 45%, 30%, and 75%, respectively. Perhaps we also know, from even bigger studies, that women are 1.5 times more likely than men to have a lung cancer unrelated to their smoking, while a person with a signifi cant smoking history is 9 times more likely to have cancer than a never- smoker. A Bayesian meta-analysis can combine this prior information on the effects of gender and smoking history with the data from the four samples to produce the posterior distribution of the pretest probability of cancer for this female patient given all relevant evidence.

With the advent of the Society of Thoracic Sur- geons General Thoracic Surgery Database, these methods will be most relevant for the fi eld of thoracic surgery in order to synthesize informa- tion arising out of a large number of prospec- tively collected data.

Several books on Bayesian estimation and meta-analysis have been written and several examples of Bayesian meta-analysis can be found in the literature. Interested readers are encour- aged to explore Peter Congdon’s book on this topic.

Of interest to the readers of this book may be the work by Tweedie and colleagues, who perform a Bayesian meta-analysis of the pub- lished literature to determine the association between incidence of lung cancer in female never smokers and exposure to environmental tobacco smoke.

Bayesian meta-analysis can be most conveniently implemented using freely available software called the WinBUGS (http://www.mrc- bsu.cam.ac.uk/bugs).

3.5.4. Value of Information Analyses

As we discussed earlier, probabilistic models often can provide a more effi cient way to address uncertainty in decision models. It provides the clinician with a sense of the strength of evidence for an optimal decision after accounting for mul- tiple sources of uncertainty. It also provides the basis for conducting value of information analy- sis on specifi c parameters. In these analyses, uncertainties in parameter estimates that trans-

late into uncertainties surrounding the outcome of interest can be used to establish the value of acquiring additional information by conducting further research. Information is valuable because it reduces the expected costs of uncertainty sur- rounding a clinical decision. The expected costs of uncertainty are determined by the probability that a treatment decision based on existing infor- mation will be wrong and by the consequences (or costs) if the wrong decision is made. The expected costs of uncertainty can also be interpreted as the expected value of perfect information (EVPI), because perfect information (an infi nite sample) can eliminate the possibility of making a wrong decision. It is also the maximum a decision maker should be willing to pay for additional evidence to inform this decision in the future. Such analy- ses can identify research priorities by focusing on those parameters where more precise estimates would be most valuable.

Additionally, an analogous type of calculation may be done on heterogeneous parameters such as quality of life weights and patient’s preferences and attitudes towards treatment. Knowing this information can help clinicians make individual- ized decisions by incorporating patients’ values and preferences into the process of making treatment decisions. Concretely, clinicians might pursue this approach using a variety of decision aids that are designed to facilitate transmission of information on patients’ values and prefer- ences to physicians. These decision aids are often expensive in terms of program development and implementation as well as in time costs for patients and physicians.

Therefore, in order to decide how to best allocate limited resources towards decision aids and similar approaches to implementing individualized care, it is impor- tant to have information on the potential social value of such endeavors and which dimensions of patient preference are most valuable to elicit.

This is accomplished by the Expected Value of

Individualized Care (EVIC) analysis.

EVIC rep-

resents the expected costs of ignorance of patient-

level heterogeneity. It is the potential value of

research, as compared to optimal population

level decision making, to elicit information on

heterogeneous parameters so that individualized

information about each patient can be conveyed

to the physician.

3.6. Conclusions

Decision analytic modeling is a systematic approach to integrating information about a variety of different aspects of clinical decision making. It helps the clinician to make an objec- tive decision by incorporating both the popula- tion-level evidence on outcomes and also the individual-level preferences in the decision- making process. It is worthwhile to issue a reminder at this point that the sole purpose of DAM is to provide information in a systematic way so as to facilitate decision making based on best and comprehensive evidence. Decision analytic modeling, although viewed by some clinician and researchers as prescriptive, only prescribes the consideration of a decision to be optimal and should not be viewed as the fi nal prescription for the patient. Decision analytic modeling in no way serves to make a decision for the clinician, who must utilize his/her own exper- tise to interpret the evidence placed in front him/

her and determine the relevance of this evidence in the context of an individual patient’s case.

The tension inherent in any DAM lies between the desire to create a model that is conceptually rich – to more accurately refl ect the complexity of real-world decision making – that is also some- what data poor, versus creating a more simple model which utilizes few, highly reliable esti- mates. However, if real-world decision making does demand a richer model, we believe that it is more appropriate to develop such a model, even if subjective decisions and estimates inform some parts of this model. We say this because often in real-world clinical practice, such subjective deci- sions and estimates do play a part in clinical deci- sion making. Decision analytic modeling helps to translate these subjective estimates into a formal statement about uncertainty and help clinicians understand the implications of these uncertain- ties in the decision process. It also helps clini- cians to identify priorities in research areas and patient preferences that are most critical for patient outcomes.

Most DAMs in thoracic surgery have focused on information from clinical data. Such informa- tion is certainly crucial and forms the backbone of a DAM. However, any attempt to translate effi - cacy information available from clinical trials

and other studies into effectiveness for a given patient requires a careful examination of patient preferences and behavior. Therefore, creating models that accurately refl ect the choices at hand requires a fusion of several types of information.

This information can be crudely divided into two categories – information that is derived from bio- logical science, and that derived from social science. As we learn more about the genetic behavior and cell biology of tumors, as well as genetic information about individual patients, clinical decision making will most certainly be impacted. But this sort of information must be contextualized within the study of actual human behaviors – information that is captured by fi elds as diverse as behavioral and classical economics to statistics to political science to epidemiology.

Consequently, it is imperative that surgeons and practitioners in these other fi elds collaborate if DAMs are to function as nuanced, effective tools to improve the clinical outcomes, and indeed the lives, of patients.

Acknowledgments. We are grateful to Caleb G.

Alexander, William Dale, and Mark Ferguson at the University of Chicago for helpful comments and suggestions.

References

1. Meltzer D. Can medical cost-effectiveness analysis identify the value of research? In: Murphy KM, Topel RH, eds. Measuring the Gains from Medical Research: An Economic Approach. Chicago: The University of Chicago Press; 2003:206–247.

2. Sackett DL, Rosenberg WM, Gray JA. Evidence based medicine: what it is, what it isn’t. BMJ 1996;312:71–72.

3. Coutler A. Partnerships with patients: the pros and cons of shared clinical decision-making. J Health Serv Res 1997;2:112–121.

4. Pauker SG, Kassirer JP. Decision analysis. N Engl J Med 1987;316:250–257.

5. Weinstein MC, Fineberg HV. Clinical Decision Making. Philadelphia: Saunders; 1980.

6. Petitti DB. Meta-analysis, Decision Analysis and Cost-Effectiveness Analysis. New York: Oxford University Press; 1994.

7. Detsky AS, Naglie G, Krahn MD, et al. Primer on medical decision analysis: getting started. Med Decis Making 1997;17:123–125.

8. Eddy D. Clinical Decision Making: From Theory to Practice. Boston: Jones & Bartlett; 1996.

9. Ost D, Fein A. Management strategies for the soli- tary pulmonary nodule. Curr Opin Pulm Med 2004;10:272–278.

10. Gold MR, Siegel JE, Russell LB, Weinstein MC.

Cost-effectiveness in Health and Medicine. New York: Oxford University Press; 1996.

11. Drummond M, Stoddard G, Torrence G. Methods of Economic Evaluation of Health Care Pro- grammes. Oxford: Oxford University Press; 1987:6–

26.

12. Torrence GW. Utility approach to measuring health-related quality of life. J Chron Dis 1987;40:

593–600.

13. Torrence GW. Measurement of health state utili- ties for economic appraisal: a review. J Health Econ 1986;5:1–30.

14. Lowson KV, Drummond MF, Bishop JM. Costing new services: long-term domiciliary oxygen therapy.

Lancet 1981; ii:1146–1149.

15. Weinstein MC, Stason WB. Foundation of cost- effectiveness analysis for health and medical practices. N Eng J Med 1977;296:716–721.

16. Garber A, Phelps C. Economic foundations of cost- effectiveness analysis. J Health Econ 1997;16:1–32.

17. Weinstein M, Zeckhauser R. Critical ratios and effi cient allocation. J Pubic Econ 1973;2:147–158.

18. Basu A, Meltzer D. Implications of spillover effects within the family for medical cost-effectiveness analysis. J Health Econ 2005;24:751–773.

19. Alexander GC, Casalino LP, Meltzer DO. Patient- physician communication about out-of-pocket costs. JAMA 2003;290:953–958.

20. Alexander GC, Casalino LP, Meltzer DO. Physi- cian strategies to reduce patients’ out-of –pocket prescription costs. Arch Intern Med 2005;165:633–

636.

21. Detsky AS, Naglie G, Krahn MD, et al. Primer on medical decision analysis: building a tree. Med Decis Making 1997;17:126–135.

22. Bayes T. An essay towards solving a problem in the doctrine of chances. Philos Trans R Soc 1763;330–

418. Taken from: Biometrika 1958;45:293–315.

23. Pratt JW. Bayesian interpretation of standard inference statements (with discussions). J R Stat Soc B 1965;27:169–203.

24. Dominioni L, Imperatore A, Rovera F, et al. Stage I non-small cell lung carcinoma, analysis of survival and implications for screening. Cancer 2000;89:2334–2344.

25. Swensen SJ, Silverstein MD, Ilstrup DM, et al. The probability of malignancy in solitary pulmonary nodules. Application to small radiologically inde-

terminate nodules. Arch Intern Med 1997;157:849–

855.

26. Wisnivesky JP, Yankelevitz D, Henschke CI. Stage of lung cancer in relation to its size. Chest 2005;127:1136–1139.

27. Ginsberg RJ, Hill LD, Eagan RT, et al. Modern thirty-day operative mortality for surgical resec- tions in lung cancer. J Thorac Cardiovasc Surg 1983;86:654–658.

28. Stéphan F, Boucheseiche S, Hollande J, et al. Pul- monary complications following lung resection: a comprehensive analysis of incidence and possible risk factors. Chest 2000;118:1263–1270.

29. Barrera R, Shi W, Amar D, et al. Smoking and timing of cessation: impact on pulmonary compli- cations after thorocatomy. Chest 2005;127:1977–

1983.

30. Chadha AS, Ganti AK, Sohi JS, et al. Survival in untreated early stage non-small cell lung cancer.

Anticancer Res 2005;25:3517–3520.

31. McGarry RC, Song G, des Rosiers P, et al. Observa- tion-only management of early stage, medically inoperable lung cancer: poor outcome. Chest 2002;121:1155–1158.

32. U.S. Department of Health and Human Services.

The Health Consequences of Smoking: A Report of the Surgeon General. Washington, DC: U.S.

Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Offi ce on Smoking and Health; 2004.

33. Arias E. United States Life Tables, 2002. National Vital Statistics Reports. Hyattsville, MD: National Center for Health Statistics; 2004:53.

34. English DR, Holman CDJ, Milne E, et al. The Quantifi cation of Drug Caused Morbidity and Mortality in Australia. Canberra: Commonwealth Department of Human Services and Health;

1995.

35. Ferguson MK, Lehman AG. Sleeve lobectomy or pneumonectomy: optimal management strategy using decision analysis techniques. Ann Thorac Surg 2003;76:1782–1788.

36. Morimoto T, Tsuguya F, Koyama H, et al. Optimal strategy for the fi rst episode of primary spontane- ous pneumothorax in young men. J Gen Intern Med 2002;17:193–202.

37. Falcoz PE, Binquet C, Clement F, et al. Manage- ment of the second episode of spontaneous pneu- mothorax: a decision analysis. Ann Thorac Surg 2003;76:1843–1848.

38. Porter G, Cantor S, Walsh G, et al. Cost- effectiveness of pulmonary resection and systemic

chemotherapy in the management of metastatic soft tissue sarcoma: a combined analysis from the University of Texas M.D. Anderson and Memorial Sloan-Kettering Cancer Centers. J Thorac Cardio- vasc Surg 2004;127:1366–1372.

39. Martini N, Rusch V, Bains, MS et al. Factors infl u- encing ten-year survival in resected stages I to III a non-small cell lung cancer. J Thorac Cardiovasc Surg 1999;117:32–36.

40. Olak J, Detsky A. Surgical decision analysis:

esophagectomy/esophagogastrectomy with or with - out drainage? Ann Thorac Surg 1992;53:493–497.

41. Cheung HC, Siu KF, Wong J. Is pyloroplasty neces- sary in esophageal replacement by stomach? A prospective, randomized controlled trial. Surgery 1987;102:19–24.

42. Goldie SJ, Kuntz KM, Weinstein MC, et al. Cost- effectiveness of screening for anal squamous intraepithelial lesions and anal cancer in human immunodefi ciency virus-negative homosexual and bisexual men. Am J Med 2000;108:634–641.

43. Goldie SJ, Weinstein MC, Kuntz KM, et al. The costs, clinical benefi ts, and cost-effectiveness of screening for cervical cancer in HIV-infected women. Ann Intern Med 1999;130:97–107.

44. Goldie SJ, Kuhn L, Denny L, et al. Policy analysis of cervical cancer screening strategies in low- resource settings: clinical benefi ts and cost- effectiveness. JAMA 2001;285:3107–3115.

45. Zeliadt SB, Etzioni RD, Penson DF, et al. Lifetime implications and cost-effectiveness of using fi nas- teride to prevent prostate cancer. Am J Med 2005;118:850–857.

46. Manser R, Dalton A, Carter R, et al. Cost- effectiveness analysis of screening for lung cancer

with low dose spiral CT (computed tomography) in the Australian setting. Lung Cancer 2005;48:171–185.

47. Briggs AH, Goeree R, Blackhouse G, et al. Proba- bilistic analysis of cost-effectiveness models:

choosing between treatment strategies for gastro- esophageal refl ux disease. Med Decis Making 2002;22:290–308.

48. Congdon P. Bayesian Statistical Modeling. New York: Wiley; 2001.

49. Tweedie RL, Scott DJ, Biggerstaff BJ, et al. Bayes- ian meta-analysis, with application to studies of ETS and lung cancer. Lung Cancer 1996;14(suppl 1):S171–S194.

50. Meltzer D. Addressing uncertainty in medical cost-effectiveness analysis: implications of ex - pected utility maximization for methods to perform sensitivity analysis and the use of cost- effectiveness analysis to set priorities for medical research. J Health Econ 2001;20:109–129.

51. Claxton K. The irrelevance of inference: a decision making approach to the stochastic evaluation of health care technologies. J Health Econ 1999;18:

341–364.

52. Claxon K, Neumann PJ, Araki S, et al. Bayesian value of information analysis: An application to a policy model of Alzheimer’s disease. Int J Tech Assoc Health Care 2001;17:38–55.

53. Kennedy AD, Sculpher MJ, Coulter A, et al. Effects of decision aids for menorrhagia on treatment choices, health outcomes, and costs: a randomized controlled trial. JAMA 2002;288:2701–2708.

54. Basu A. Value of information on preference heterogeneity and individualized care. In: Three Essays in Cost-Effectiveness Analysis [master’s thesis]. Chicago: University of Chicago; 2004.