Ken,
Thanks for the detailed (as usual) and quite exhaustive post. My comments,
such as they are, are embedded.
On 5 June 2015 at 16:05, Ken Friedman <[log in to unmask]> wrote:
> Don pointed to some of the problems of the publishing system. There are
> better ways to make journals work, but these require editors to examine
> articles carefully prior to peer review. A significant number of flawed
> articles get through both because reviewers are over-worked and because
> editors send material to review that should not be sent on. If editors
> ensured that articles conformed to journal standards prior to review, this
> would reduce pressure on reviewers. Editors might also pay attention to
> serious reviews — it has been my experience that a serious, developmental
> review takes about six hours, and I have seen on several occasions that
> editors simply accept articles despite significant flaws. This is a
> separate problem to authors who don’t care about the issues in a review,
> but simply pass articles along to the next journal without improvements —
> or modify the title and do nothing else.
>
It might also help if new editors were given some mentorship in these
matters. When I decided to try my hand at it, it was assumed that I
already knew exactly what to do. Fortunately, my curiosity drove me to
check the papers I was assigned in tandem with searching for reviewers.
This led me to discover independently the value of editor-as-filter for
journals. I don't know what other practices I might be dropping the ball
on. Perhaps it's a function of the journal with which I am involved - I
would imagine some top shelf journals have very stringent practices about
accepting a new editor and their training if needed.
>
> Part of this has to do with publication metrics. It involves the wish (or
> need) for authors to publish something, perhaps anything, in an effort to
> represent that they are doing research. This is quite different to the
> desire to make a contribution to the knowledge of the field.
>
> This also involves the appetite of journals for content. Once a journal
> exists, it must fill pages. There is some flexibility, but there is
> generally a minimum page count. While I can say that I have seen some silly
> work creep into print, I acknowledge that there are legitimate differences
> of opinion on what is worth publishing in the way of scholarship and
> research. Even so, the standards in the design field often fall below the
> standards in other fields.
>
> This is exacerbated by accepting bad conference papers. One of the key
> reasons for so many bad conference papers is that conferences must often
> have a specific number of presenters to break even. As a result, conference
> organisers often accept bad papers — including papers that all reviewers
> have stated lie outside the range of acceptable research.
>
> An increasing number of conference organisers use a trick based on a
> supposedly rigorous point system. Reviewers are asked to assign points for
> different criteria on a scale of 1 to 100 or some similar system. When the
> conference plan begins, organisers state that they will only accept papers
> above 70 of 100 points. Then, when all the reviewing is done, they
> calculate how the number of accepted papers will affect the budget. On
> several occasions, I have seen organisers drop the number of required
> points dramatically to reach the financial break-even point. If it takes
> dropping the score on accepted papers to 35 of 100 points to break even,
> that is the decision.
>
This really brings out the systems aspects of things. If Terry is
listening somewhere, I'm confident he'd agree that this is a systems
problem, and that the first step would be to develop an accurate (albeit
possibly qualitative) model of the whole academic publishing enterprise.
It might be that from such a model, opportunities to exert pressure at
"leverage points" (per Meadows) would yield improvements without causing
chaos.
Additionally, I wonder if we don't have too many conferences. I mean - one
way to help ensure that bad conference papers don't see the light of day is
to limit the total number of spaces. Perhaps we need to start organizing
fewer, but larger conferences, such that there will be a net decrease in
total number of papers presented. Having fewer conferences could help
standardize practices around the review process too. Economies of scale
and all that. It would be a delicate matter to find the right balance, but
I don't think it's impossible.
>
> This has nothing to do with predatory publishing or fake conferences —
> these problems and tricks appear in journals and conferences that are, for
> the most part, legitimate.
>
> Other issues also come into play. Journal publishing firms must build up
> their package of journals to make an offering in the library market. And
> there are still more reasons for publishing journals with sometimes
> troubling content. I know one journal publishing firm that added a few
> questionable journals to its offering simply to create a bigger package of
> journals as part of the preparation for a corporate sale. More journals
> meant a higher sale price.
>
> There are ways around this. They require serious thought and occasional
> tough choices. The question is what we want for the field.
>
> And this involves a wide range of additional questions in a field where
> people do not always wish to address the kinds of question on research and
> philosophy of science that older fields wrestled with and acted on long ago.
>
So... at the risk of being recursive, perhaps a series of workshops to
bring people together to study the problem?
> Or so I believe.
>
I agree with you.
\V/_ /fas
*Prof. Filippo A. Salustri, Ph.D., P.Eng.*
Email: [log in to unmask]
http://deseng.ryerson.ca/~fil/
-----------------------------------------------------------------
PhD-Design mailing list <[log in to unmask]>
Discussion of PhD studies and related research in Design
Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
-----------------------------------------------------------------
|
