When teaching medical students I find a level of confusion similar or greater than that expressed in the recent flurry of correspondence. It also sounds as if we are using our experience to guide the discussion rather than the evidence.

If there is little evidence which I believe to be the case then maybe we as the leaders of EB practice should do the research

P: 10 clinical cases with 3 questions each:
I/C: 4 search engines to be tested and compared
O: help!!!
It is the outcome of the study that is so difficult to determine. 
Speed is easy - 
Validity & Level of evidence- likelihood of finding highest level evidence matched with validity within the first 20 hits.
Maybe we need to assess the papers using this as a new rubric - a two dimensional representation of the validity (Jahad or other validity tool) against level of evidence (http://www.cebm.net/index.aspx?o=1025)
Maybe we should plot them on the equivalent to a ROC plot but call it a Clinical Utility Score. Or maybe someone has already done this I and I did not find it on my search engines of choice!

I have a summer student: of course she has no idea what I am planning - and it will probably take 6 months just to get the question sorted let alone the trial design but anyone else want to play in the sand box?


As to search engines - I use looknow.com & TRIP

Martin




Dr Martin Dawes
Chair Family Medicine
McGill University
515 Pine Avenue West, Montreal
Quebec, Canada
Tel 514 398 7375 x0469
Fax 514 398 4202
 
 

*** This communication is confidential. Please request explicit permission before copying or forwarding. If you are not the intended recipient, you are notified that any distribution or copying of this email is strictly prohibited. If you have received this email in error, please notify us immediately by return email and delete all copies of the message. Thank you. ***

-----Original Message-----
From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Arin Basu
Sent: 17 February 2008 23:35
To: [log in to unmask]
Subject: Google vs Trip/Cochrane, and the promise of mahalo

Hi Kamlesh,

Your points about Google are well taken, but there is a difference
between searching Google and searching for information through
databases like Trip/cochrane/dynamed/clinical evidence. When you put a
search term in Google, Google's spider finds documents through a
relevance matching process that relates how many other documents
actually link to the document of highest relevance score and then
draws the list it builds. It is both useful and has problems that some
search engine optimizer gurus have exploited so that related pages
that link to the original page moves up all the time and you tend to
get the same set of popular, well linked to pages. Google is therefore
a great tool when you are starting out to scan the horizon, but when
you start asking specific questions that require quality controlled
focused response, think again.

With Google, you do not necessarily get to see pages that have most
relevant information for _your_ specific need, nor can you expect
pages that you retrieve using Google search to be quality checked in
terms of their content. Although the search results, if done within a
reasonable period of  time of each other, can be very reliable (you
get to see the same pages that you retrieved in your first attempt at
search). On the other hand, when you search Cochrane/Trip/Dynamed/,
there is an additional layer of curation and hard coding behind the
process including a peer review, in most cases, and relevant quality
control.

There is perhaps a need for a search engine that is as powerful as
Google and as reliable as the human curated databases such as the ones
we are discussing here. Recently, I checked out mahalo -- a very small
search engine, but looked to me promising because it attempts to blend
the best of both worlds. Who knows how far it will go, but looked good
to me when I checked it out for the first time. Here's the link:

http://www.mahalo.com


Best,
Arin



On Feb 17, 2008 11:14 PM, Kamlesh Bhargava <[log in to unmask]> wrote:
>
>
> On Feb 17, 2008 2:12 PM, Kamlesh Bhargava <[log in to unmask]> wrote:
> > I agree with Kev. It depends.
> >
> > For articles which I know the titles of I would do a Google search.
> >
> > Also other like information about drugs I have found Google as a useful
> resource, advantage being the speed and free access.
> >
> > Uptodate, tripdatabse, cochrane in that order are my preferences for
> clinical information.
> >
> > Pubmed Clinical queries yeilds good information.
> >
> > PMC Pubmed central gives free full text articles in Pubmed.
> >
> > Kamlesh
> >
> >
> >
> > On Feb 17, 2008 1:28 PM, k.hopayian <[log in to unmask]> wrote:
> >
> > >
> > > It depends on what you are searching for and why. So students need to
> know the advantages and disadvantages of different search filters and
> databases.
> > >
> > > For example, is your question about diagnosis? If yes, then Cochrane
> won't help.
> > > Are you looking for a guide to practice? In which you may want
> guidelines.
> > > Systematic reviews summarise evidence. If there is one on your subject,
> great! But if not, you may need to go to original papers. Even if there is
> an SR, it may be too long and detailed for your purposes so you may prefer a
> commentary on the review.
> > >
> > > If you are planning an in-depth review, for example, for departmental
> policy, you may want to see original papers too.
> > >
> > > So I teach students, "it all depends".
> > >
> > > Most of us, most of the time, will be satisfied with searches where
> filters have been written for us and which cover a wide set of data. One
> such is TRIP, already mentioned. Another is SUMSearch
> http://sumsearch.uthscsa.edu/ This returns items in a heirachical fashion so
> that the user can pick down the list from general texts, guidelines, reviews
> (including the excellent commentaries on reviews, DARE), to original
> research. SUMSearch also has the facility for focussing the search onto
> diagnosis, aetiology etc.
> > > PUBMED clinical queries  has a similar facility but its search is
> limited to Medline and Cochrane.
> http://www.ncbi.nlm.nih.gov:80/entrez/query/static/clinical.shtml
> > >
> > > B/W, Kev Hopayian
> > >
> > >
> > >
> > >
> > >
> > >
> > > -------on 16/2/08 18:35, Moacyr Roberto Cuce Nobre at [log in to unmask]
> wrote:
> > >
> > >
> > >
> > >
> > >
> > > Dear all
> > >
> > > When you are ?teaching? how to search evidence, what do you recommend
> ?first?: Cochrane, PubMed (Free Internet Access - contains Cochrane Reviews)
> or some search engines like TRIP Database ( Free Internet Access - contains
> both)?
> > >
> > > Remember that PubMed contains a lot of article in other languages beside
> english!
> > >
> > > Thanks for comments
> > >
> > > --
> > > Moacyr
> > >
> > > _______________________________________
> > > Moacyr Roberto Cuce Nobre, MD, MS, PhD.
> > > Diretor da Unidade de Epidemiologia Clínica do InCor
> > > Faculdade de Medicina da Universidade de São Paulo - Brasil
> > > 55 11 3069 5941 (fone/fax)
> > > 55 11 9133 1009 (celular)
> > >
> > >
> > >
> > > --
> > >
> > >
> > >
> >
> >
> >
> > --
> > Kamlesh Bhargava
>
>
>
> --
> Kamlesh Bhargava