11 Clinical Research Center-Based Investigations and Evidence-Based Medicine

155

11

From: Contemporary Endocrinology: Evidence-Based Endocrinology Edited by: V. M. Montori © Mayo Foundation for Medical Education and Research

Clinical Research

Center-Based Investigations and Evidence-Based Medicine

William L. Isley, ^MD

C

ONTENTS

I

NTRODUCTION

T

YPES OF

C

LINICAL

I

NVESTIGATORS

S

TRENGTHS AND

W

EAKNESSES OF

C

LASSICAL

C

LINICAL

I

NVESTIGATION

S

TRENGTHS AND

W

EAKNESSES OF

R

ANDOMIZED

C

LINICAL

T

RIALS

T

HE

R

OLE OF

C

LASSICAL

C

LINICAL

I

NVESTIGATION IN THE

21

ST

C

ENTURY

R

EFERENCES

INTRODUCTION

The last decade of the 20th century saw the emergence of a new paradigm in clini- cal decision making with the appearance of evidence-based medicine (EBM) (1). The ultimate in the hierarchy of evidence in EBM is the randomized controlled clinical trial (RCT). Unfortunately, the N-of-1 RCT, ideal for evaluating the effectiveness of inter- ventions in the individual patient, is rarely feasible or if feasible, is rarely conducted.

RCTs have become common place in the past 35 yr (beginning with the hypertension trials in the 1970s). Pragmatic drug RCTs enroll hundreds to thousands of participants in the community, outside the ivory towers of traditional medical academia. Their pur- pose is to assess the efficacy, and sometimes the safety and cost effectiveness, of exper- imental interventions in real patients cared for in real clinical settings.

The rise of the RCT in the hierarchy of evidence has meant the decline in priority of other forms of evidence to support clinical decision making, namely the clinical investigation.

Until the biomedical revolution inaugurated after World War II, unsystematic clinical

observations and oligo-subject minimally systemized studies were the basis of much of

clinical decision making, with the few observation masters passing on their conclusions

in an authoritarian fashion as exemplified by the incomparable Sir William Osler. Some

aspects of this paradigm still survive today, particularly in the surgical specialties and in the concept of “opinion leaders” used (and abused) by the pharmaceutical industry on a regular basis. The expansion of the National Institutes of Health (NIH) in the late 1940s saw the development of the modern biomedical research enterprise, with the clinical investigator being the hero of this paradigm of knowledge acquisition and application.

For those schooled in the 1970s, the burgeoning biomedical fields were to provide the basic knowledge of molecular biology, physiology, pathophysiology, and applied phar- macology in order to tackle disease in a far more sophisticated manner than the purported

“ignorance” of our forebears. The triumph of Brown and Goldstein in the discovery of the low-density lipoprotein (LDL) receptor (2) and the subsequent modern assault on ather- osclerosis from a metabolic standpoint exemplifies such exploits. The physician of the 1980s was to be an applied physiologist and pharmacologist par excellence.

To support this goal, highly trained researchers conducted standard metabolic stud- ies of a highly selected and small group of research subjects. These studies involve tightly controlled experiments and intensively monitored observations conducted in a clinical research center (CRC). Whereas many CRC studies are done to marshal physi- ological and pathophysiological evidence for a therapeutic intervention’s putative ben- efit while awaiting the outcome of RCTs, they cannot provide the ultimate “proof of the pudding” in most contexts. For example, in the field of endocrinology and metabolism, there are numerous CRC studies showing a positive benefit of insulin sensitizers (thiazo- lidinediones) on cardiovascular risk factors (3), but there is a lack of RCTs showing that the effect of these agents on surrogate outcomes translate into improved cardiovascular health. Thus, those seeking to use evidence to support clinical decisions may deem clinical investigations peripheral or, sadly, irrelevant to their activity.

TYPES OF CLINICAL INVESTIGATORS

The “chemically pure” clinical investigator (CI) is often a full-time medical school faculty member who spends significant time in scientific pursuit. CIs generally have had substantial training in the art and science of detailed physiological studies in humans and practice their craft in the CRC or similar environment. CIs formulate hypotheses in their field of interest and secure funding, mostly from noncommercial sources, to per- form the experiments to answer the questions of interest. CIs are solely responsible for reporting their findings to the scientific community.

The modern for-profit drug development enterprise has so changed the concepts of clinical investigation (4) that when asked to identify a clinical investigator today, physi- cians are likely to identify the local head of a “study center” (usually with an impres- sive sounding name) doing phase 2 and 3 drug development studies. This enterprise is often run by a clinician with little to no training in clinical investigations or lacking any particular expertise in the disease being studied, or a specialist far removed in time and training from traditional clinical investigation. Membership in learned societies such as the American Society for Clinical Investigation and the American Federation for Clinical Research are irrelevant to this undertaking.

Paramount concerns of drug development investigators are usually economic rather

than scientific. The goal of research is to achieve Food and Drug Administration (FDA)

approval of a drug or device, a new indication for an approved intervention, or to

develop evidence of equivalence of the latest me-too drug. Funding of this investigator

is based on the ability to recruit subjects and fill out case report forms, rather than on a track record of scientific achievement. Often these physicians take up drug develop- ment work because it is more lucrative that the delivery of clinical care. Their papers reporting results (if they are fortunate enough to be granted such status by the studies’

sponsors) are often written or heavily censored by the pharmaceutical company or device maker sponsor. In light of this reality, and to differentiate these physicians from classical CIs, I have chosen to label such physicians “practice-based investigators” (PI).

Certainly there is a moderating position between the pure CI and the for-profit PI.

This position is usually occupied by physicians who have clinical research training, with clinical appointments at medical schools, and who derive most or all of their income from the practice of medicine. These clinical researchers may conduct clinical care research, derive questions from their clinical practice, obtain funding from com- mercial interests as well as from nonprofit funding agencies (investigator-initiated stud- ies), conduct studies or trials that are explanatory (highly selected subjects in tightly controlled environments) or pragmatic (usual patients in usual clinical practice), and publish the results without participation of the funding source in peer-reviewed journals.

Apart from the level of evidence they generate, the differentiation in the spectrum from CI to PI is appropriately made based on the flow of intellectual and financial capital. The CI formulates a scientific question and then obtains funding, often from noncommercial sources, to carry out clinical investigations necessary to answer that question in a CRC or similar environment. The flow of ideas is from the investigator to a funding organization. On the other extreme, the PI takes a question that is generated by industry and signs a contract to help answer that question in the context of the prac- tice setting or nonacademic study center. The flow of ideas is from the sponsor to the investigator. The intermediate position, may have at times features that more closely resemble the CI (generation of ideas, independent conduct and report of findings, con- duct of tightly controlled experiments) or the PI (management of large pragmatic trials, funded by for profit interests). All types of investigators are vital elements in the modern medical enterprise. In theory, the work of all is equally legitimate and should be of equal quality.

STRENGTHS AND WEAKNESSES OF CLASSICAL CLINICAL INVESTIGATION

With the change in the hierarchy of evidence for the practice of medicine, has the modern “classical” CI fallen on hard times? Has EBM devalued their trade to the point that it is largely irrelevant? The following discussion will seek to provide an answer to the latter question while assessing the strengths and weaknesses of the RCT and “clas- sic” clinical investigation. However, prior to that exercise, we will briefly trace the sources of information that contribute to the greater body of medical knowledge.

In the perusal of the modern medical library, one will find works in epidemiology,

biochemistry, cell biology, genetics, animal and human physiology, and pathophysiol-

ogy, pharmacology, and standard medical texts and journals chronicling descriptions

of diseases and their treatment (observations and RCTs). From the standpoint of the

physician, the stream of knowledge pathway can be construed as in Fig. 1. Whereas

basic science investigation (at the molecular and cellular level and animal studies)

serves to inform ones basic body of knowledge, including the formulation of new

hypotheses and the refining of existing ones, it is only investigations in humans, the experimental animal of relevance, that should lead to actual changes in the practice of medicine. In a broad sense, the body of knowledge formulated by the many contribu- tors previously mentioned and achieved through induction, allows one to reason deduc- tively as to what interventions may be reasonable.

Epistemologically speaking, the nature of science, particularly in the fields of biol- ogy and medicine, is always changing and incomplete. Paradigms and hypotheses are continually being refined. Often the CI is forced to conclude that his or her investiga- tions have resulted in a quantitative increase in the body of knowledge that is unknown as opposed to an absolute reduction in what is unknown. Unfortunately, the novice in science (which is often true in the PI environment) fails to appreciate this tension.

As previously described, the CI applies his trade in a carefully controlled environ- ment (in-patient or out-patient CRC or similar facility); highly skilled personnel carry out classic clinical investigations that are unparalleled in their ability to define physi- ology and pathophysiology. The subjects are homogeneous and presumably healthy (for the study of normal physiology), or with a well-defined illness (for the study of pathophysiology), usually without concomitant significant disease or intake of medica- tions that may potentially confound the findings of the studies. The knowledge acquired allows the investigator to formulate basic paradigms of bodily function and dysfunction.

Thus the reasoning process is fundamentally inductive.

Furthermore, the investigator may alter the physiological system (with a hormone or

drug) in a tightly controlled environment to precisely define the effects of the pertur-

bation. Both common clinically monitored parameters (such as standard chemistry tests

and commonly measured hormones) and analytes of pure research interest (such as the

hormones leptin and ghrelin) can be assessed. Furthermore, highly invasive studies

(catheterization of multiple vessels, muscle, or nerve biopsies) may be carried out in

these willing and compensated subjects. What is deemed reasonable and acceptable to

Fig. 1. The development of disease paradigms and the testing of potential therapies in the develop- ment of evidence-based medicine.

the watchful eyes of the institutional review board (IRB, usually local to the institution in the setting of the clinical investigation) can be accomplished. Similar studies might often be viewed extreme and unnecessary in the context of real world care and the study environment that approximates the real world, the pragmatic RCT.

Some shortcomings of the work of the CI as evidence for the practice of medicine are apparent. In experimental design, the CI may not use random allocation to the inter- vention or control conditions and blinding increasing the likelihood of bias. The rela- tive short-term nature of the observations and interventions may be inadequate to extrapolate to years of disease and therapy. For a drug intervention where cumulative exposure may result in side effects, typical clinical investigation protocols are unlikely to realize the downside potential of the intervention in question. In a sense, CI knowl- edge thus acquired represents the ideal. Unfortunately, this ideal does not apply to most patients. Patients have one or more diseases, take one or multiple medications, do not consume a carefully controlled diet, and do not have highly regimented lifestyles.

Deductions about physiological models are often isolated to healthy young men to the exclusion of older subjects and women in their reproductive years. Furthermore, the CRC environment often caters to the “professional subject” who may be unemployed or have unusual access to the study center (medical center employee or spouse of employee). These sources of selection bias may also limit the generalized conclusions reached from CRC data.

There are notable examples of findings in clinical investigation that did not confirm in clinical trials. Consider, for instance, the induction of idiosyncratic liver failure and death with the insulin sensitizer troglitazone in clinical trials and clinical practice occur- ring in the context of improvement of numerous physiological surrogates in short term studies. This example points out the inability of typical small-scale physiological stud- ies to assess all aspects of potential drug toxicity (or efficacy). Thus, only weak infer- ences result from studies of physiological endpoints, surrogates for “hard” clinical outcomes (5). The clear superiority of the RCT to show real world evidence for bene- fit and/or harm is striking.

From these foundational paradigms, the scientist and clinician can reason deduc- tively on how a patient’s bodily function should behave in a particular situation (such as with the administration of a drug). The CRC environment allows testing of hypothe- ses developed with lesser levels of evidence (observational studies, animal studies, in vitro studies). Hypotheses generated from these lines of evidence can be further con- firmed, modified, or rejected by the findings from CRC studies. Hypotheses confirmed or generated by the CI are then ultimately tested in an RCT. The CI develops and “fine tunes” the science; the investigators in RCTs (including PIs), test whether the applica- tion of the current understanding of the science has any real world value.

A now classic example of discrepancy between the results of clinical investigations

refers to the disparity between the many physiological studies documenting beneficial

effects of estrogen therapy on lipids and vasculature (and the accompanying compelling

epidemiological studies showing improved cardiovascular health in postmenopausal

women treated with estrogens) (6), and the recently completed HERS (7), and WHI (8)

studies, showing no cardiovascular benefit, and perhaps cardiovascular harm. Given the

large RCT findings, the informed clinician is left no choice but to conclude that the best

evidence shows that for the average postmenopausal woman oral estrogen therapy is

not cardioprotective. However, what should the practitioner conclude about the posi- tive physiological studies? Assuming adequate clinical investigation rigor, we must conclude that the results of some of the surrogate physiological parameters were indeed in the direction of cardiovascular benefit. The disparity may be owing to a dif- ference in effect between women without occult cardiovascular disease (protective or neutral) and women with occult cardiovascular disease (harmful). Or known poten- tially harmful effects (raising C-reactive protein or triglycerides) or unknown harmful effects may overwhelm known potentially beneficial effects (improved endothelial function, LDL cholesterol, and high-density lipoprotein [HDL] cholesterol).

Large RCTs must trump the knowledge from the CI environment because the RCT has addressed the question of relevance in real patients receiving health care (with their genetic and lifestyle diversity and varied concomitant disease and medications).

Given the known findings of postmenopausal estrogen replacement therapy (ERT) and hormone replacement therapy (HRT), it may be reasonable for the clinician given the hierarchy of evidence to give ERT or HRT to women early after menopause for symp- tomatic relief, but not for long-term cardiovascular prevention, because the women studied in the large RCTs were older women, not women taking hormones soon after menopause. And this may be so until results from an ongoing large RCT of ERT on women shortly after menopause ultimately tests this hypothesis. Given the uncertain- ties surrounding this issue, risk attitudes—and other values and preferences of the patients—will have decisively important role in the decision (a reminder that accord- ing to EBM, the evidence alone never completely informs the decision and requires expert consideration of the circumstances and preferences of the patients).

STRENGTHS AND WEAKNESSES OF RANDOMIZED CLINICAL TRIALS

The RCT is essentially a test based on deductions from scientific principles enun-

ciated as the result of classic clinical investigation and other foundational work. The

obvious strengths of the RCT relate to the ability to conceal the allocation of partici-

pants to the intervention or control and to produce two groups with the same progno-

sis (thanks to the random allocation of patients with equivalent distribution of known

and unknown prognostic factors) such that differences in prognosis at the end of the

trial can only be explained by differences in treatment effect. When researchers organize

large pragmatic RCTs the large numbers of subjects enrolled allow for more precise

findings with broader applicability. However, this strength of numbers and economy

of scale may be its Achilles’ heel as well. With heterogeneity of the study population

comes confounding factors from concomitant diseases and medications. As a RCT

becomes less like classic clinical investigation (from explanatory to pragmatic), it

becomes more difficult to isolate the effect of the treatment from the other aspects of

delivering the intervention. Thus, some mechanistic inferences become weaker while

the applicability of the findings becomes more secure. As the population becomes less

uniform, the ability to realistically isolate the effect of a single intervention in that pop-

ulation declines. Dropouts may be substantial with a real-world population rather than

the more “professional” study population often enrolled in CI studies. Although the

statistician may be able to preserve the protection from bias that randomization affords

with an intention-to-treat analysis, this protection is inadequate in the face of large loss to follow-up.*

Montori and colleagues have recently reviewed common problems in RCT papers (9). Such problems include faulty comparators (inflating treatment effect by comparing vs placebo or substandard treatment), composite endpoints (to inflate an effect or per- haps “find” an effect when there is not one), subgroup analyses, and small treatment effects. The latter is especially true for the modern “mega” RCT with thousands of subjects where a statistically significant effect may be seen, but the absolute effect of the benefit is very small. Analysis of the number needed to treat is often helpful for clinicians trying to discern the true size or importance of such statistical findings.

^†

Endpoints tend to be more clinically relevant, although perhaps less precise, in RCTs as compared with clinical investigations. Physiological parameters are measured with great precision in the CRC. Endpoints in RCT may be quite definite (death) or subject to regional practice patterns (“need” for and access to revascularization). Composite endpoints (a smorgasbord of events that may be possibly affected by the treatment in question) may further obscure what is actually happening in the RCT.

The RCT tends to average out effects across the study population. It fails to differ- entiate between subjects who are responders (and there may be heterogeneity of response among study subjects) and nonresponders. The assumption of uniformity is much less likely to be true in RCT. Post hoc analysis may identify subgroups with dif- ferent responses, though rarely are findings from such analyses clear cut or valid. There is generally too much heterogeneity in the population studied to differentiate distinct subgroups that benefit from those who do not. Fishing expeditions testing innumerable subgroups often lead to chance findings that cannot be confirmed in subsequent trials.

The RCT may also be plagued by the nature of the modern clinical investigator. The major endpoint trials are usually carried out by clinical trialists with both CI and PI backgrounds. Trials not funded by industry (where a new indication for a drug may not be at stake) are less tightly monitored during the actual performance of the trial, so the investigator is left more to his integrity to carry out the trial in a rigorous fashion.

Studies carried out strictly under the auspices of industry have the advantage of more rigorous monitoring to meet FDA guidelines for potential new or expanded treatment indications, but a higher likelihood that much of the work is done in the PI environ- ment. As noted earlier, such physicians are often involved in the clinical investigation enterprise primarily for economic reasons and without adequate training in conducting studies or interest in the process other than financial remuneration. Many of the PI physicians can do an acceptable job. However, improper enrollment, protocol viola- tions, and failure to recognize or appropriately categorize adverse effects are some of the problems that plague the modern clinical investigation enterprise dominated by PIs.

One may readily ask, if the quality of the investigator is suboptimal, can large numbers sufficiently obscure poor workmanship? The author has increasing doubts about large

*As in the recent statin acute coronary syndrome trials, PROVE-IT (N Engl J Med. 2004;

350:1495–504) and A to Z (JAMA. 2004;292:1307–1316) where >30% of subjects dropped out over 2 yr or less.

†

As in the CAPRIE trial with clopidogrel where the results are statistically significant but the

number needed to treat is approx 200 (Lancet. 1996; 348:1329–1339).

pharmaceutical company funded trials where a majority of the subjects are enrolled by the PI entrepreneur. Although rampant fraud has been documented (11), wholesale chicanery is probably rare. However, the fact that economics are now the paramount interest in most industry funded RCTs (by both the sponsor and most of the investi- gators) should raise concerns about potential compromises to the integrity of the data generated.

What is the physician to do with contradictory evidence from RCTs? For the clini- cal investigation situation, it is much more likely that one or more plausible explana- tions may be readily identifiable. The answers in the RCT environment are much less clear because of the many potential sources of influence on trial results, as well as the possibility that the prevailing hypothesis is actually incorrect. Publication bias (pub- lishing only trials finding statistically significant treatment effects) is particularly prob- lematic in industry-funded trials (12), in that they may provide the impression of false uniformity.

THE ROLE OF CLASSICAL CLINICAL INVESTIGATION IN THE 21ST CENTURY

Previously, we asked two questions: With the change in the hierarchy of evidence for the practice of medicine, has the modern “classic” CI fallen on hard times? Has EBM devalued his or her trade to the point that it is largely irrelevant? The author would answer these questions with a resounding, “No.”

As already discussed, in the new paradigm of EBM, the CI will never provide the final answer about the efficacy or safety of treatments. However, without the CI, the entire superstructure of medical science is likely to collapse. Advances in the basic understanding of physiology and pathophysiology will come to a screeching halt. The inductive process of knowledge acquisition will be aborted. The science must advance well before the application of that science can proceed to testing in the RCT. Further- more, from an economic standpoint, it is unjustifiable to contemplate carrying out a large multicenter RCT without encouragement from smaller more rigorous studies.

Similarly, it is ethically unthinkable to subject large populations to interventions with- out having scrupulously evaluated the rationale for their use in a highly controlled environment such as the CRC. In short, the RCT cannot take place without the ground- breaking work of the CI.

Can classic clinical investigation become more like the pragmatic RCT? Definitely not, as its scientific rigor is vital to the integrity of its product, scientific understanding.

Has the pragmatic RCT wandered too far from the moorings of the scientific method as performed most elegantly in humans in CRC? Possibly so, as has been mentioned in the earlier discussion. The RCT, although always likely to be an exercise in major monetary investment, may be too much governed by economic factors instead of scientific ones, particularly when its truth is linked to investors’ returns. The recent fiasco with the coxibs and the selective publication of data (13) gives us ample pause to consider how pristine is the knowledge generated in the current environment. Furthermore, negative tri- als and trials that are not favorable to the industry sponsor’s product (and thus industry’s bottom line) are not likely to be become public.

Whereas the very recent trend for a registry of all trials will hopefully stop this prac-

tice (14), spinmeisters (opinion leaders) are likely to be given the new task of explain-

ing the unexplainable. Only time will tell how this new approach to the dissemination of trial results will play out. The pure CIs are inclined to publish results contrary to their hypotheses, because their motivations come generally from science and knowledge rather than from economics. In fact, paradoxically finding contradictory findings may help the CI’s funding because scientific bodies might conclude that we know even less about the field in question than we thought we did.

The CI is not dead. The rise of EBM has correctly placed the findings from clinical investigation as the shoulders over which patient important effects of promising treat- ments can be tested in RCTs. The displacement of classic clinical investigation from the hierarchy of evidence has not displaced it from its place as an invaluable discipline for the advancement of the science of medicine and patient care.

REFERENCES

1. Montori VM, Guyatt GH. What is evidence-based medicine? Endocrinol Metab Clin North Am 2002;

31:521–526

2. Brown MS, Goldstein JL. Receptor-mediated control of cholesterol metabolism. Science 1976;191:

150–154.

3. Parulkar AA, Pendergrass ML, Granda-Ayala R, Lee TR, Fonseca VA. Nonhypoglycemic effects of thiazolidinediones. Ann Intern Med 2001;134:61–71.

4. Angell M. The Truth About the Drug Companies: How They Deceive Us and What to Do About It.

Random House, New York, NY, 2004.

5. Graham DJ, Drinkard CR, Shatin D. Incidence of idiopathic acute liver failure and hospitalized liver injury in patients treated with troglitazone. Am J Gastroenterol 2003; 8:175–179.

6. Fleming TR, DeMets DL. Surrogate end points in clinical trials: are we being misled? Ann Intern Med 1996; 125: 605–613.

7. Davison S, Davis SR. New markers for cardiovascular disease risk in women: impact of endogenous estrogen status and exogenous postmenopausal hormone therapy. J Clin Endocrinol Metab 2003; 88:

2470–2478.

8. Grady D, Herrington D, Bittner V, et al. For HERS Research Group. Cardiovascular disease outcomes during 6.8 years of hormone therapy: Heart and Estrogen/progestin Replacement. Study follow-up (HERS II). JAMA 2002;288:49–57.

9. Anderson GL, Limacher M, Assaf AR, et al. For the Women’s Health Initiative Steering Committee effects of conjugated equine estrogen in postmenopausal women with hysterectomy: the Women’s Health Initiative randomized controlled trial. JAMA 2004;291:1701–1712.

10. Montori VM, Jaeschke R, Schünemann HJ, et al. Users’ guide to detecting misleading claims in clin- ical research reports. BMJ 2004;329:1093–1096.

11. Davis BR, Furberg CD, Wright JT Jr, Cutler JA, Whelton P; ALLHAT Collaborative Research Group.

ALLHAT: setting the record straight. Ann Intern Med 2004;141:39–46

12. Eichenwald K, Kolata G. A doctor’s drug trials turn into fraud. New York Times, May 17, 1999, Sec- tion A, pp.1.

13. Bhandari M, Busse JW, Jackowski D, et al. Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. CMAJ 2004;170:477–480.

14. Fitzgerald GA. Coxibs and cardiovascular disease. N Engl J Med 2004; 351:1709–1411.

15. DeAngelis CD, Drazen JM, Frizelle FA, et al. Clinical trial registration: a statement from the Inter- national Committee of Medical Journal Editors. JAMA 2004;292:1363–1364.