I hope this isn't too far off topic. If so, please accept my apologies.
Chris Burton writes:
>Producing statistics of physician (or group or unit or whatever)
performance
>seems to be one of the great growth industries in medicine. Graphs of
>performance in just about anything seem to be produced - usually with
>something that looks at first glance like a normal distribution (and almost
>never with any statistical addenda). But I would like to know whether we
can
>use them sensibly as anything other than pictures. In particular when I am
>one of the subjects of the analysis how do I interpret my own performance?
This is a difficult area.
My first comment is that Statistics are great at characterizing the behavior
of groups, but they don't do as well when they try to characterize the
behavior of individuals. With care, you can use Statistics to characterize
individuals, but you want to avoid blindly using the Statistical methods
that have been developed for clinical trials. Just as a simple example, you
should note that characterizing individual behavior is not an activity that
fits in well with the traditional hypothesis driven research. Do you set up
a separate hypothesis for each doctor?
My second comment is that it is tricky to decipher when a deviation is part
of the random fluctuations that are an inherent part of the medical system
and when a deviation is an indication of a special cause that we might want
to investigate and learn from. We all have a tendency to overestimate and
overreact to small deviations that may be nothing more than normal
variation.
My third comment is that when we see deviations, we tend to attribute them
too often to the individual and tend to ignore the environment that the
individual works in. If there are unacceptably large variations in
performance, your first thought ought to be "how do I change the environment
to reduce this variation" but it's human nature instead to say, "who should
I retrain or reprimand".
You might look at some of the comments that W. Edwards Deming has made about
individual variation in job performance. Pay special attention to the famous
"red bead" experiment. Mary Walton has some good books about Deming and his
philosophy (I don't have the exact citation handy, sorry!). Walton's books
about the Deming philosophy are easier to read that Deming's books. Walton
and Deming are talking about business practices, but I believe you can
extrapolate these ideas to medical practice.
Another good book to look at is "Understanding Variation" by Donald Wheeler.
This is a delightful and very easy to read book that explains many of the
problems that businesses have with handling variation in the products they
produce. Again, you need to extrapolate these ideas from a business context
to a health care context. If you think about some of the ways that physician
performance data has been abused and misused, then you will see that these
same types of abuses from a business context in Wheeler's book.
After you've read Walton and/or Wheeler, you may come to the conclusion that
the statistical control chart, a tool widely used in industry, has similar
applicability in health care. I would encourage you to apply control chart
methods to physician performance data as well as a lot of other data that is
not usually examined carefully in the health care context.
I'm working with some nurses at Children's Mercy Hospital to use control
charts to track medication errors, patient complaints, employee accidents,
unplanned sick leave, employee turnover, and additional measures of
organizational safety and effectiveness. It has a lot of potential, in my
opinion, to handle both this type of data as well as the type of data you
are referring to.
I need to add a disclaimer that health care is different from most other
businesses. That doesn't mean that health care can't use control charts, but
it does mean that we can't blindly apply a process developed for industries
that produce fast cars and fast food. So some type of adaptation will be
necessary.
Wheeler, Donald J. (1993) Understanding Variation. The Key to Managing
Chaos. Knoxvile TN: SPC Press, Inc. (ISBN: 0-945320-35-3).
For the beginning student. An insightful introduction about variation in
business processes, how to identify it and how to control it. A must read
for anyone working on improving quality in work processes.
Steve Simon, [log in to unmask], Standard Disclaimer.
STATS - Steve's Attempt to Teach Statistics: http://www.cmh.edu/stats
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|