Hi everyone,
I am forwarding a posting from another list (copied below) about the
use of incorrect, misleading, or out-of-date information in
peer-reviewed scientific journals that may be of interest in light of
some of our recent discussions here.  That post reminded me of a book
about statistics that has just been published and that some list
members may be interested in.  I've read some reviews and it looks
well done-- DAMNED LIES AND STATISTICS: UNTANGLING NUMBERS FROM THE
MEDIA, POLITICIANS, AND ACTIVISTS, by Joel Best, U. Cal. Press 2001.
I think it's likely that Best touches on many of the concerns about
the so-called 'grey literature' that we have discussed on the list in
previous weeks.

As an aside, it was interesting for me to learn that Best's book is
the likely source for Steve Verdon's July 9 email last month:
<http://www.jiscmail.ac.uk/cgi-bin/wa.exe?A2=ind0107&L=enviroethics&O=A&P=4056>.

An excerpted chapter appeared in the Chronicle of Higher Education
("Telling the Truth About Damned Lies and Statistics," May 4 2001
issue, B7-B9).  In particular Best uses an example that resembles the
mourning dove statistic discussed below.  A dissertation prospectus
that Best was reading started off with a carefully footnoted
statement, "Every year since 1950, the number of American children
gunned down has doubled."  When Best went to the original journal
source, the statement read exactly the same: "Every year since 1950 .
. .  has doubled."  Best estimates that if that were true, the figure
for children shot to death in 1995 when the article was published
would be 35 trillion.  When he checked *that* author's source, he
found that this statistic started out in life originally in a
Children's Defense Fund (CDF) report as: "The number of American
children killed each year by guns has doubled since 1950."  Best
examines how a relatively innocuous statistic--after all, the
American population had roughly doubled in the same period--took on a
life of its own and became what he calls a "mutant" statistic.  Best
comments:  "Certainly, the article's author didn't ask many probing,
critical questions about the C.D.F.'s claim.  Impressed by the
statistic, the author repeated it--well, meant to repeat it.
Instead, by rewording the C.D.F.'s claim, the author created a mutant
statistic, one garbled almost beyond recognition" (CHE, B8).

I think that Best's conclusions about statistics relate both to the
post below about mourning dove population dynamics but also to our
discussions here over the past few weeks.  Best writes:

        "Some statistics are born bad--they aren't good from the
start, because they are based on nothing more than guesses or dubious
data.  Other statistics mutate; they become bad after being mangled
(as in the case of the author's creative rewording).  Either way, bad
statistics are potentially important: They can be used to stir up
public outrage or fear; they can distort our understanding of the
world; and they can lead us to make poor policy choices" (CHE, B8).

At the risk of sounding defensive, I think Best here is making the
same basic point that Evil Steve and I were simply trying to make
last month in the various threads about statistics, NYC recycling,
etc.  And I believe this is the point that Missouri biologist John
Schulz is making below when he writes: "[W]hen incorrect, misleading,
or out-of-date information is allowed to be published in peer
reviewed scientific journals other people can use this information
for their own political agenda."

Anyway . . .  fyi and fwiw.  It's interesting to see people on other
lists addressing many of the same issues we deal with (quarrel
about?) <grin> here.  Well, you know what Chesterton said . . . the
trouble with a quarrel, he said, is that it interrupts a good
argument.   :-)   later,

Jim T.


--- begin forwarded text


Status: U
Date:         Tue, 7 Aug 2001 14:28:35 -0500
Reply-To: John Schulz <[log in to unmask]>
Sender: TWS-L Discussion List <[log in to unmask]>
From: John Schulz <[log in to unmask]>
Subject:      Quality Control in the Publication Process?
To: [log in to unmask]

Like most folks, my pile of reading material is large, and it takes
several months to get around to reading everything.  Much to my
enjoyment, I stumbled upon an article about mourning doves in the
October, 2000 of the Journal of Wildlife Management
(64(4):1004-1008); I read the article closely, and found some
surprises.

My intent with this email note is point out some issues that must be
resolved with The Wildlife Society (TWS) publications.  My use of a
recent Journal of Wildlife Management (Factors influencing mourning
dove nest success in CRP fields) article is not intended to impugn
the authors, but to highlight my case for a few reforms in the peer
review process.  I apologize ahead of time if this comment appears to
be an attack on the authors or mean spirited; it not intended as such.

My case is simple: when incorrect, misleading, or out-of-date
information is allowed to be published in peer reviewed scientific
journals other people can use this information for their own
political agenda.  For example, the opening sentence in the article
reads, "Mourning dove populations in the Central Management Unit
(CMU) have increased in recent years (Dolton and Smith 1998)."  The
sentence is false, or misleading at best.  The actual 1998 report
which was cited in the article shows a nonsignificant 2-year
(1997-98) increase of 1.7% (90% CI -4.3 to 7.3) for the entire CMU;
hardly an increasing population trend in recent years.  Also, the
1998 report shows that over the previous 10 and 33 year periods,
population trends have decreased significantly 1.1% (P<0.1) and -0.4%
(P<0.05) respectively.  Again, this hardly indicates an increasing
CMU mourning dove population.  Giving the authors the benefit of the
doubt, mourning dove population trends in Kansas showed a
nonsignificant increase of 3.0% (90% CI -1.1 to 7.1) for the 1989-98
period; however, their statement in the published paper refers to the
entire 14 state Central Management Unit.  The editors or reviewers
should have caught this obvious error, and flaw in the overall logic
of the paper (i.e., CRP is possibly responsible for increasing dove
populations).

Secondly, the authors state that they wanted to compare their nesting
data to previously published reports from the Great Plains and across
North America.  With only one exception, their comparison did not
include any published work dealing with doves nesting in CRP fields
or grassland habitat studies after 1993.  There have been numerous
and valuable contributions in the Journal of Wildlife Management
about ground nesting doves, and authors and reviewers missed this
major flaw in the paper.  I quickly looked on my bookshelf and found
4-5 papers related to ground nesting doves using various
open-land/grassland habitats in less 5 minutes (JWM 61:318-325, Avian
use and vegetation characteristics of conservation reserve program
fields; JWM 61:644-655, Bird use and nesting in conventional,
minimum-tillage, and organic cropland; JWM 62:474-484, Songbird
community composition and nesting success in grazed and ungrazed
pinyon-juniper woodlands; JWM 63:1009-1017, Effects of deterrents on
avian abundance and nesting density in electrical substations in
Oklahoma; J. Field Onithol. 69:299-305, Mourning nesting habitat and
nest success in central Missouri).  A comparison of more recent and
applicable data would have been more helpful and pertinent to the
topic at hand.

So what's the big deal?  This is just another example highlighting
the need to improve the publication process for wildlife related
journals.  I propose the following suggestions.  First, I think TWS
should seriously consider the use of double-blind reviews (similar to
some domestic poultry literature) where neither the reviewer or
author know each other's identity.  For example, the author's name
and affiliation would be placed on a separate page which would be
removed at the editor's office prior to shipping in the manuscript
out for review.  Second, I urge reviewers to seriously undertake a
thorough review of the paper that has been assigned to them.  In my
example, anyone familiar with mourning doves should have caught the
mistake in the first sentence of the paper.  I've had some reviewers
recommend rejection of a paper because they had a personal moral
disagreement with a certain management activity unrelated to study at
hand.

My third and last point.  Other biologists in my office will say,
"Ya, it's just the luck the of the draw sometimes, you never know
which AE will get the paper or what the reviewers will say."
Hopefully, random chance is not part of the scientific peer review
process.

I'd be interested in the ideas of others in The Wildlife Society.

John H. Schulz
Mo. TWS Chapter, Past-President

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To unsubscribe from this list, send a message to
[log in to unmask] with the words "signoff tws-l" in the body
of the message.

--- end forwarded text