Great discussion.
Re Martin's question re outcomes (and I'd want to do a literature search before going any further of course):
But setting that aside, and this may be incredibly naïve, but are there four categories, and are each important?
1. Process measures: in this case, how many clinical questions asked and successfully answered; or time to answer a predetermined question; or quality of answer to preset question (e.g. if the question is about antibiotics in otitis media if the "search engine" doesn't put Paul et al's Cochrane review at or very near the top it doesn't get many marks from me).
2. Qualitative. Too obvious for this group but you know what I mean. Participant experience is an important measure. And I'd include both patients and healthcare practitioners of course.
3. Quantitative. Here I'd be looking for data relating to at least a change in clinical practice. Pragmatically it might have to be measured on a before and after basis but surely we can focus on some key gaps between evidence and practice and try and bridge them by making "search engines" available and discussing how to use them "optimally" on those areas?
4. Pastoral. Different, I'd argue from qualitative. Here I'm thinking about how individuals feel about their work and role (or from patient's perspective how they feel about their illness) in a holistic sense, taking a much more rounded view about their worth, approach to CPD, staying up to date, how consultations are going now. Qualitative approaches likely, but much deeper questions.
I too have a summer student but only for 3 weeks, but I think we could be persuaded to play in the sandpit as long as this is reasonably small scale.
Best wishes
Neal
Neal Maskrey
Director of Evidence Based Therapeutics
National Prescribing Centre
Liverpool UK
L69 3GF
-----Original Message-----
From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Martin Dawes, Dr.
Sent: 18 February 2008 17:29
To: [log in to unmask]
Subject: Re: Google vs Trip/Cochrane, and the promise of mahalo - i use looknow.com
When teaching medical students I find a level of confusion similar or greater than that expressed in the recent flurry of correspondence. It also sounds as if we are using our experience to guide the discussion rather than the evidence.
If there is little evidence which I believe to be the case then maybe we as the leaders of EB practice should do the research
P: 10 clinical cases with 3 questions each:
I/C: 4 search engines to be tested and compared
O: help!!!
It is the outcome of the study that is so difficult to determine.
Speed is easy -
Validity & Level of evidence- likelihood of finding highest level evidence matched with validity within the first 20 hits.
Maybe we need to assess the papers using this as a new rubric - a two dimensional representation of the validity (Jahad or other validity tool) against level of evidence (http://www.cebm.net/index.aspx?o=1025)
Maybe we should plot them on the equivalent to a ROC plot but call it a Clinical Utility Score. Or maybe someone has already done this I and I did not find it on my search engines of choice!
I have a summer student: of course she has no idea what I am planning - and it will probably take 6 months just to get the question sorted let alone the trial design but anyone else want to play in the sand box?
As to search engines - I use looknow.com & TRIP
Martin
Dr Martin Dawes
Chair Family Medicine
McGill University
515 Pine Avenue West, Montreal
Quebec, Canada
Tel 514 398 7375 x0469
Fax 514 398 4202
*** This communication is confidential. Please request explicit permission before copying or forwarding. If you are not the intended recipient, you are notified that any distribution or copying of this email is strictly prohibited. If you have received this email in error, please notify us immediately by return email and delete all copies of the message. Thank you. ***
-----Original Message-----
From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Arin Basu
Sent: 17 February 2008 23:35
To: [log in to unmask]
Subject: Google vs Trip/Cochrane, and the promise of mahalo
Hi Kamlesh,
Your points about Google are well taken, but there is a difference
between searching Google and searching for information through
databases like Trip/cochrane/dynamed/clinical evidence. When you put a
search term in Google, Google's spider finds documents through a
relevance matching process that relates how many other documents
actually link to the document of highest relevance score and then
draws the list it builds. It is both useful and has problems that some
search engine optimizer gurus have exploited so that related pages
that link to the original page moves up all the time and you tend to
get the same set of popular, well linked to pages. Google is therefore
a great tool when you are starting out to scan the horizon, but when
you start asking specific questions that require quality controlled
focused response, think again.
With Google, you do not necessarily get to see pages that have most
relevant information for _your_ specific need, nor can you expect
pages that you retrieve using Google search to be quality checked in
terms of their content. Although the search results, if done within a
reasonable period of time of each other, can be very reliable (you
get to see the same pages that you retrieved in your first attempt at
search). On the other hand, when you search Cochrane/Trip/Dynamed/,
there is an additional layer of curation and hard coding behind the
process including a peer review, in most cases, and relevant quality
control.
There is perhaps a need for a search engine that is as powerful as
Google and as reliable as the human curated databases such as the ones
we are discussing here. Recently, I checked out mahalo -- a very small
search engine, but looked to me promising because it attempts to blend
the best of both worlds. Who knows how far it will go, but looked good
to me when I checked it out for the first time. Here's the link:
http://www.mahalo.com
Best,
Arin
On Feb 17, 2008 11:14 PM, Kamlesh Bhargava <[log in to unmask]> wrote:
>
>
> On Feb 17, 2008 2:12 PM, Kamlesh Bhargava <[log in to unmask]> wrote:
> > I agree with Kev. It depends.
> >
> > For articles which I know the titles of I would do a Google search.
> >
> > Also other like information about drugs I have found Google as a useful
> resource, advantage being the speed and free access.
> >
> > Uptodate, tripdatabse, cochrane in that order are my preferences for
> clinical information.
> >
> > Pubmed Clinical queries yeilds good information.
> >
> > PMC Pubmed central gives free full text articles in Pubmed.
> >
> > Kamlesh
> >
> >
> >
> > On Feb 17, 2008 1:28 PM, k.hopayian <[log in to unmask]> wrote:
> >
> > >
> > > It depends on what you are searching for and why. So students need to
> know the advantages and disadvantages of different search filters and
> databases.
> > >
> > > For example, is your question about diagnosis? If yes, then Cochrane
> won't help.
> > > Are you looking for a guide to practice? In which you may want
> guidelines.
> > > Systematic reviews summarise evidence. If there is one on your subject,
> great! But if not, you may need to go to original papers. Even if there is
> an SR, it may be too long and detailed for your purposes so you may prefer a
> commentary on the review.
> > >
> > > If you are planning an in-depth review, for example, for departmental
> policy, you may want to see original papers too.
> > >
> > > So I teach students, "it all depends".
> > >
> > > Most of us, most of the time, will be satisfied with searches where
> filters have been written for us and which cover a wide set of data. One
> such is TRIP, already mentioned. Another is SUMSearch
> http://sumsearch.uthscsa.edu/ This returns items in a heirachical fashion so
> that the user can pick down the list from general texts, guidelines, reviews
> (including the excellent commentaries on reviews, DARE), to original
> research. SUMSearch also has the facility for focussing the search onto
> diagnosis, aetiology etc.
> > > PUBMED clinical queries has a similar facility but its search is
> limited to Medline and Cochrane.
> http://www.ncbi.nlm.nih.gov:80/entrez/query/static/clinical.shtml
> > >
> > > B/W, Kev Hopayian
> > >
> > >
> > >
> > >
> > >
> > >
> > > -------on 16/2/08 18:35, Moacyr Roberto Cuce Nobre at [log in to unmask]
> wrote:
> > >
> > >
> > >
> > >
> > >
> > > Dear all
> > >
> > > When you are ?teaching? how to search evidence, what do you recommend
> ?first?: Cochrane, PubMed (Free Internet Access - contains Cochrane Reviews)
> or some search engines like TRIP Database ( Free Internet Access - contains
> both)?
> > >
> > > Remember that PubMed contains a lot of article in other languages beside
> english!
> > >
> > > Thanks for comments
> > >
> > > --
> > > Moacyr
> > >
> > > _______________________________________
> > > Moacyr Roberto Cuce Nobre, MD, MS, PhD.
> > > Diretor da Unidade de Epidemiologia Clínica do InCor
> > > Faculdade de Medicina da Universidade de São Paulo - Brasil
> > > 55 11 3069 5941 (fone/fax)
> > > 55 11 9133 1009 (celular)
> > >
> > >
> > >
> > > --
> > >
> > >
> > >
> >
> >
> >
> > --
> > Kamlesh Bhargava
>
>
>
> --
> Kamlesh Bhargava
Legal Disclaimer:
--------------------------------------------------------------------------------
This email is confidential and intended solely for the use of the individual
to whom it is addressed. Any views or opinions presented are solely those
of the author and do not necessarily represent those of the Organisation. If
you are not the intended recipient, be advised that you have received
this email in error and that any use, dissemination, forwarding, printing
or copying of this email is strictly prohibited. If you have received this
email in error please notify the sender. This e-mail has been checked
for viruses using anti-virus software
--------------------------------------------------------------------------------
|