 |
 |
Good question which could open or continue multiple message threads. I am assuming the statistical modeling (predictive statistics) should be analyzed scientifically for its reliability like any other prediction rule. If this assumption is wrong I'd like to understand why. I have seen many suggestions for using computer modeling for individualized risk (artificial neural networks being among the most sophisticated of the models) as a "higher order" of evidence than randomized trials. But having a statistical model shown to work for one concept does not necessarily mean that it will work for many other concepts. That would be like establishing that a drug works well for one condition and then concluding other drugs work well for other conditions so we do not need to test other drugs. This could be considered from the "hierarchy of evidence" (levels of evidence) or from the "pyramid of evidence (6S model) which are different constructs for different purposes. My take for levels of evidence (How well do the results of such modeling represent the "truth" compared to other types of evidence?) is that it depends on the type of clinical question. If the question is one of prognosis (What are the likely outcomes or likelihood of outcomes for a specific patient?) then randomized trials are not the type of evidence primarily considered. A validated prediction rule could be the highest level of evidence, and the statistical modeling is a method of providing a prediction rule. If the question is one of treatment (Will taking one action instead of a different action lead to a difference in likelihood of outcomes?) the statistical modeling has the same biases as observational studies. Statistical modeling conducted before the WHI trial (at a time when most of the clinicians and patients believe postmenopausal hormone replacement therapy [HRT] improved cardiovascular outcomes) would likely suggest that HRT would reduce cardiovascular events, similar to data suggested by observational studies. The modeling would likely be biased in a large way by "healthier people" following advice more often including the advice to take HRT, and not being able to account adequately for all the variables identifying "healthier people" (whatever that means). However where (statistical modeling that is individualized) and (extrapolation from randomized trials that is not well matching the individual) fall in relative hierarchy can be debated, with either version being favored by how far or close the randomized trial evidence matches the individual in question. Looking at the 6S model (pyramid of evidence) which is more of a guide of where to look for efficiency in seeking evidence than a hierarchy of the quality of evidence for a specific fact or assertion, the statistical modeling can fit at any level: Systems -- the computerized decision support that occurs automatically within the system being used and not requiring external searching is the most efficient -- a prediction rule incorporated into such a system would place this statistical modeling at the top of the 6S pyramid - no need for a seventh S. Summaries -- resources like DynaMed that synthesize information from all the lower parts of the pyramid and put the whole picture together. These resources can include validated prediction rules or other statistical modeling and make them available at this part of the pyramid. Synopses of Syntheses -- if the syntheses covered the statistical modeling, then the synopsis would cover it Syntheses -- a systematic review of statistical models, or a systematic review of prediction rules including statistical models, would cover it. Synopses of studies -- if the studies covered it, then the synopses would cover it Studies -- the original evidence that the statistical model predicts the outcome in question. If the studies do not exist do we have any reason to expect reliability for the statistical modeling? Brian S. Alper, MD, MSPH Editor-in-Chief, DynaMed (www.ebscohost.com/dynamed) -----Original Message----- From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Ernesto Barrera Sent: Tuesday, September 21, 2010 4:53 PM To: [log in to unmask] Subject: Data mining in Medical Records? I have known a data mining software that calculates predictive models without much statistical knowledge. I wonder if the automatic incorporation of predictive models to computerized medical records, patient risk modeling based on regional or local data could be a future. In that case, could we say that we move towards a model of 7S (predictive Statistics) in the pyramid of evidence? I appreciate your views on this subject. Sincerely, Ernesto Barrera Family Doctor