Jo et al,
a couple of comments to make here - just like in EBM the starting
point is the question - so I'd try a version of PICO
In what Population with what problem or issue does this test (I
analogue) do better than that alternative (C) in assessing for what
purpose (O)
Its also worth having a look at the Cochrane Diagnostic Test Accuracy
Reviews - some of these relate to standardised questionnaire measures
and do a nice job of assessing what in psychometric terms might be
called criterion validity. Depression screening tools are an example.
Another useful sceptical view is in Howard Garb's review of decision
making in clinical psychology
(http://dx.doi.org/10.1146/annurev.clinpsy.1.102803.143810 )
It is perhaps worth thinking about the Emotional Stroop paradigm from
a psychometric perspective ( - see for eg my colleagues Blake Dear et
al paper in press in the Journal of Pain
http://dx.doi.org/10.1016/j.jpain.2011.07.003 for a psychometric view
on the dot-probe task) but I would see it as more a cognitive task
than a standardised questionnaire.
While we are on the topic I think its worth separating standardised
questionnaires (often called psychometric tests) from the theories of
measuring "psychological" states, abilities, attitudes, and knowledge
and the broadly statistical procedures (psychometrics) that are used
to evaluate measures of otherwise unobservable constructs. When we
measure & assess we measure with error and uncertainty - its just
knowing how large the error is. Here the use of techniques like factor
analysis to evaluate diagnostic criteria for syndromal disorders is a
useful example.
And a final plea before developing your own questionnaire - do that
literature search - there are sooo many questionnaires out there with
partially adequate empirical work ups - its better to use one of them
and contribute something to improving it than to start from scratch
and develop yet another.
Andrew
Date: Thu, 17 Nov 2011 20:52:28 +0000
From: jo kirkpatrick <[log in to unmask]>
Subject: Re: Psychometric Testing?
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="-332380854-939649030-1321563148=:22601"
---332380854-939649030-1321563148=:22601
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Dear Robin, William, Helena, Amy and all=0A=0A=0AThanks to everyone for you=
r thoughts on this complicated issue. It is clear that the accuracy of thes=
e assessments spans from almost supernaturally good to not much better than=
guesswork, depending on the validity of the items; the reliability of the =
measures, and the skill of the interpreter. In my small experience I have u=
sed Positive And Negative Attitude Scales and Profile Of Moods but the best=
was the Emotional Stroop Task, which once I had a baseline was uncanny in =
its accuracy of measuring depression and the reverse EST for positive mood.=
=0A=0A=0AI am sure William is right when he states that these tools can wor=
k well, but some, probably the one size fits all types are basically flawed=
, such as the one that found Robin to be totally ruthless in pursuit of his=
own interests. As Williaam pointed out - assessments are only as good as t=
he assessors, if they lack adequate training or the necessary intuition and=
innate skills the assessment will be flawed. I also with William that test=
s must be fit for the purpose and the best would be specially written or ad=
apted for the task, rather than picked from a long list. When the time come=
s I will definitely seek the advice of an expert to help me to design my ow=
n tests and to analyse and interpret the results. I also need to find tests=
to use for triangulation and validation and I am very grateful to Helena f=
or the filter she designed.=C2=A0 =0A=0A=0AThe most telling comment for me =
as a psychologist was Robin describing his attempt to 'tone it down' when h=
e recognised ruthlessness in himself. This is not =0Ahow a ruthless person =
would react, if anything they would step it up a =0Afew notches. They are n=
ot ashamed of being ruthless, in fact they =0Agratefully relish the quality=
, even if they pay lip-service to having a =0Asocial conscience. In semanti=
c terms determination, dedication and =0Aruthlessness are very similar, the=
main difference is in how =0Amuch integrity one is prepared to =0Alose in =
order to be a success. Ruthless people lack serious concern =0Aabout others=
who might get hurt in the process and rationalise the way =0Athey feel abo=
ut this. Dedicated and determined people will find a way to achieve their g=
oals without losing their integrity or hurting others, =0Aeven if it takes =
longer, ruthless people will take the fastest route =0Aregardless of who ge=
ts hurt, provided they are quite sure that they can =0Aget away with it.=0A=
=0A=0ABest wishes Jo
--
====================================================
Andrew Baillie MPsychol PhD
Associate Professor & Director of Clinical Psychology Training
Centre for Emotional Health
Psychology Department
Macquarie University NSW 2109 AUSTRALIA
Room 713 Building C3A
Ph +61 2 9850 9436
Fax +61 2 9850 8062
mailto:[log in to unmask]
====================================================
This message is intended for the addressee named and may contain
confidential information. If you are not the intended recipient,
please delete it and notify the sender. Views expressed in this
message are those of the individual sender, and are not necessarily
the views of Macquarie University.
====================================================
|