Chapter 003. Decision-Making in Clinical Medicine (Part 8) Statistical Prediction Models Bayes' theorem, as presented above, deals with a clinical prediction problem that is unrealistically simple relative to most problems a clinician faces. Prediction models, based on multivariable statistical models, can handle much more complex problems and substantially enhance predictive accuracy for specific situations. Their particular advantage is the ability to take into account many overlapping pieces of information and assign a relative weight to each based on its unique contribution to the prediction in question. For example, a logistic regression model to predict the probability of CAD takes into account all of the relevant independent factors from the clinical examination and diagnostic testing instead of the small handful of data that clinicians can manage in their heads or with Bayes' theorem. However, despite this strength, the models are too complex computationally to use without a calculator or computer (although this limit may be overcome once medicine is practiced from a fully computerized platform). To date, only a handful of prediction models have been properly validated. The importance of independent validation in a population separate from the one used to develop the model cannot be overstated. An unvalidated prediction model should be viewed with the same skepticism appropriate for a new drug or medical device that has not been through rigorous clinical trial testing. When statistical models have been compared directly with expert clinicians, they have been found to be more consistent, as would be expected, but not significantly more accurate. Their biggest promise, then, would seem to be to make less-experienced clinicians more accurate predictors of outcome. Decision Support Tools Decision Support Systems Over the past 35 years, many attempts have been made to develop computer systems to help clinicians make decisions and manage patients. Conceptually, computers offer a very attractive way to handle the vast information load that today's physicians face. The computer can help by making accurate predictions of outcome, simulating the whole decision process, or providing algorithmic guidance. Computer-based predictions using Bayesian or statistical regression models inform a clinical decision but do not actually reach a "conclusion" or "recommendation." Artificial intelligence systems attempt to simulate or replace human reasoning with a computer-based analogue. To date, such approaches have achieved only limited success. Reminder or protocol- directed systems do not make predictions but use existing algorithms, such as practice guidelines, to guide clinical practice. In general, however, decision support systems have shown little impact on practice. Reminder systems, although not yet in widespread use, have shown the most promise, particularly in correcting drug dosing and in promoting adherence to guidelines. The full impact of these approaches will only be evaluable when computers are fully integrated into medical practice. Decision Analysis Compared with the methods discussed above, decision analysis represents a completely different approach to decision support. Its principal application is in decision problems that are complex and involve a substantial risk, a high degree of uncertainty in some key area, or an idiosyncratic feature that does not "fit" the available evidence. Five general steps are involved. First, the decision problem must be clearly defined. Second, the elements of the decision must be made explicit. This involves specifying the alternatives being considered, their relevant outcomes, the probabilities attached to each outcome, and the relative desirability (called "utility") of each outcome. Cost can also be assigned to each branch of the decision tree, allowing calculation of cost effectiveness. Typically, the data to populate a decision model are derived from the literature, from unpublished sources, from expert opinion and from other secondary sources. Third, the decision model must be "evaluated" to determine the net long-term health benefits and costs of each strategy being considered. Fourth, the incremental health benefits and costs of the more effective strategies must be calculated. Finally, extensive sensitivity analyses must be used to examine the effects on the results of varying the starting assumptions through plausible alternative values. An example decision tree created to evaluate strategies for screening for human immunodeficiency virus (HIV) infection is shown in Fig. 3-3. Up to 20,000 new cases of HIV infection are believed to be caused each year in the United States by infected individuals who are unaware of their illness. In addition, about 40% of HIV-positive patients progress to AIDS within a year of their diagnosis. Early identification offers the opportunity both to prevent progression to AIDS through use of serial CD4 counts and measurements of viral load linked to selective use of combination antiretroviral therapy and to encourage reduction of risky sexual behavior. The Centers for Disease Control and Prevention (CDC) proposed in 2003 that routine HIV testing should be a part of standard medical care. In a decision- model exploration of this proposed strategy compared with usual care, assuming a 1% prevalence of unidentified HIV infection in the population, routine screening of a cohort of 43-year-old men and women increased life expectancy by 5.5 days and cost $194 per subject screened. The cost-effectiveness ratio for screening relative to usual care was $15,078 per quality-adjusted life year. Results were sensitive to assumptions about the effectiveness of behavior modification on subsequent sexual behavior, the benefits of early therapy of HIV infection and the prevalence and incidence of HIV infection in the population targeted. This model, which required over 75 separate data points, provides novel insights into a clinical management problem that has not been subjected to a randomized clinical trial. The process of building and evaluating decision models is generally too complex for use in real-time clinical management. The potential for this tool therefore lies in the development of a set of published models addressing a particular decision or policy area that can serve to highlight key pressure points in the problem. Although many published models tend to focus excessively on providing an "answer," their better role is to enhance understanding of the most important questions that deserve particular attention in clinical decision-making . Chapter 003. Decision-Making in Clinical Medicine (Part 8) Statistical Prediction Models Bayes' theorem, as presented above, deals with a clinical prediction problem. CAD takes into account all of the relevant independent factors from the clinical examination and diagnostic testing instead of the small handful of data that clinicians can manage in their heads. of HIV infection and the prevalence and incidence of HIV infection in the population targeted. This model, which required over 75 separate data points, provides novel insights into a clinical