Dear All
A colleague and I recently published a paper on the Adapted Fresno Test,
intended for use with occupational therapists and other allied health
professionals - this won't be suitable for use with psychiatrists, but
might be of interest to some of the people on this listserve. We
intentionally put quite a lot of details and examples from the test into
the publication, and a full copy can be obtained from me via email.
McCluskey, A., & Bishop, B (2009). The Adapted Fresno Test of Competence
in Evidence-Based Practice. Journal of Continuing Education in the
Health Professions, 29(2), 1-8.
I agree with Connie that the scoring is time consuming -
students/clinicians take about 20 minutes to complete the test, then it
takes another 20 minutes to score the test.
We had problems initially getting good agreement/inter-rater reliability
between raters, and had to expand the examples in the grading
table/matrix before the reliability improved to an acceptable level.
Annie
Annie McCluskey PhD MA DipCOT
Postdoctoral Research Fellow
Community-Based Health Care Research Unit (CBHCRU), Royal Rehabilitation
Centre Sydney, Ryde &
Faculty of Health Sciences, The University of Sydney
Honours Coordinator (Occupational Therapy)
Faculty of Health Sciences
The University of Sydney
Cumberland Campus (Blg J, Room 122)
PO Box 170
Lidcombe
New South Wales
Australia 1825
Email: [log in to unmask]
Ph +61 2 9351 9834 (USyd)
Ph + 61 2 9808 0513 (RRCS, Ryde)
Fax +61 2 9351 9197 (USyd)
Mob 0419 44 7738
Dr McCluskey's fellowship is funded by the University of Sydney, the
Royal Rehabilitation Centre Sydney and the National Stroke Foundation
The Community-Based Health Care Research Unit (CBHCRU) is a joint
initiative of the University of Sydney and the Royal Rehabilitation
Centre Sydney. The CBHCRU is located at Ryde. CBHCRU researchers
investigate the effectiveness of community-based and rehabilitation
interventions, and the well-being of people with traumatic injuries and
long-term chronic conditions.
The National Stroke Foundation (NSF) is a not-for-profit organisation
that works with the public, government, health professionals, patients,
carers and stroke survivors to reduce the impact of stroke on the
Australian community. The NSF is committed to funding high quality
research and has developed a research strategy which addresses gaps in
knowledge and meets priority needs for stroke care.
-----Original Message-----
From: Evidence based health (EBH)
[mailto:[log in to unmask]] On Behalf Of Connie
Schardt
Sent: Monday, 10 August 2009 10:51 PM
To: [log in to unmask]
Subject: Re: Evaluating Educational Outcomes in EBM/Practice Based
Learning
Vinod,
I used the Fresno test to evaluate an EBM class taught to medical
librarians. We did a pre Fresno and post Fresno test. The major concern
is that grading the responses required a lot of time and the ability to
be very objective in assessing the answers to somewhat open ended
questions.
They provide a grading rubric but it still required some interpretation
of responses. I also used the Berlin Questionnaire to evaluate an EBM
Course for medical students. This was easier to use as it is multiple
choice and the grading was very objective (one clear answer). Both get
at information retention. It's the grading that makes the difference.
Connie
Connie Schardt
Medical Center Library
Duke University
DUMC Box 3702
204 Seeley Mudd Bldg
Durham, NC 27710
919-660-1124 (voice)
Attend MLA '10, MLA's annual meeting and exhibition, "Reflect & Connect"
in Washington, D. C. May 21-26, 2010
"Vinod H.
Srihari, M.D."
<vinod.srihari@YA
To
LE.EDU>
[log in to unmask]
Sent by: K
"Evidence based
cc
health (EBH)"
<EVIDENCE-BASED-H
Subject
[log in to unmask] Evaluating Educational Outcomes
in
.UK> EBM/Practice Based Learning
08/08/2009 05:49
AM
Please respond to
"Vinod H.
Srihari, M.D."
<vinod.srihari@YA
LE.EDU>
Dear all -
I teach an EBM/PBL (Practice Based Learning & Improvement) course for
second and third year U.S. psychiatry residents and have been mulling
over how to evaluate the impact of my teaching on resident knowledge and
EBM-related skills. While the more distal behavior is of most interest
to me (i.e. do they actually apply this in patient care?) I am holding
off on that until we have an electronic medical record in place that
will allow some feasible way to audit practice.
I have been reviewing instruments and guided by a systematic review
(Shaneyfelt T, Baum KD, Bell D, et al. Instruments for Evaluating
Education in Evidence-Based Practice: A Systematic Review. JAMA.
2006;296(9):1116-1127) have been looking at the Fresno. I have queried
two psychiatry educators on adaptations they have made to the Fresno
vignettes to make this instrument relevant for adult psychiatry
residents....is there anyone on this list who is aware of other such
adaptations of Fresno or the Berlin instrument?
I have also been considering a multiple choice assessment focused on
clinical epi/biostats knowledge (as used in the very interesting article
by Windish DM, Huot SJ, Green ML. Medicine Residents' Understanding of
the Biostatistics and Results in the Medical Literature. JAMA.
2007;298(9):1010-1022). While this does not get at EBM skills in a
broader sense, it has advantages as a way to get a sense of whether
there is minimal proficiency in biostats (althougn I would take issue
with a couple of the items on this score).
I would welcome specific or more general comments and look forward to
seeing those of you who will be at TEBM (Oxford) next month...
Thanks,
Vinod Srihari
--
Vinod H. Srihari, MD
Assistant Professor
Department of Psychiatry
Yale University School of Medicine
Director, Specialized Treatment in Early Psychosis (STEP)
www.clinicaltrials.gov ID: NCT00309452
Staff Psychiatrist
Connecticut Mental Health Center (CMHC)
Phone (203) 974-7816
Fax (203) 974-7052
Mail: 34 Park Street
CMHC
New Haven, CT 06519
U.S.A.
|