Dear Ali,

Thanks for these. Assessments and rankings are a plague and a bother. There are many good reasons question them — and to question any specific ranking system. Any system you can use is prone to being gamed. When that happens, the assessment may only measure some factors — and a gamed system may not even measure what it purports to measure.

My colleagues and I did that first journal ranking article as a matter of self-preservation in a system where the journals of all other disciplines were ranked in ways that helped the relevant schools and faculties while our journals were invisible. The more detailed article published in Design Studies continued the process.

Now, as a journal editor, I worry about rankings for two reasons. First, many people are required by their universities or by their departments to consider rankings when they place articles with journals. Second, even though we are funded as a not-for-profit journal by a university and a university press, our university has a strategic plan for us that requires us to meet certain ranking goals. Elsevier serves as our publisher, but we are in reality a diamond open access journal because Tongji University pays all author fees as well as paying Elsevier for other services. The university also supports an editorial office through the D&I Publishing Platform and they pay for extensive design services that enable us to produce a well-designed attractive journal in full colour without any extra charge to authors for colour plates, while authors with colour material are free to use as many illustrations as they wish. All of this costs a great deal — it comes with a price for the editorial team, and that price is our need to worry about rankings.

The entire set if league tables and rankings that plagues the university world now is compounded by many factors. Foremost among these is the fact that there are somewhere between 14,000 and 22,000 universities in the world today. (The number differs depending on who is counting.) That is many times more than the world’s 2,000 to 3,000 universities in the 1950s — and fewer than 1,000 only a century before that. What universities are and what they do is heavily contested, but many want or require their staff to engage in research as the price of academic work. How do they know what research is good, valid, or worth doing? Often they don’t. They may not even care. But they do care about representing the research done as good, valid, and worth doing. Finding metrics to make this representation is therefore significant.

For individual faculty members, the ability to contribute to university metrics is also significant. With the growth of the adjunct class and casual employment, and the dramatic growth of administration, there are fewer tenure track and full time faculty jobs than in the past. Those who aspire to such jobs are forced to demonstrate their worthiness by using the kinds of quantification that universities can measure.

All of this gives rise to many quandaries.

While I plan to read Juan Pablo Pardo-Guerra’s book, The Quantified Scholar, my suspicion is that a good diagnosis of the problem doesn’t lead to a solution is a situation where so many different groups and individuals have a stake in keeping the problem what it is today.

Yours,

Ken

Ken Friedman, Ph.D., D.Sc. (hc), FDRS | Editor-in-Chief | 设计 She Ji. The Journal of Design, Economics, and Innovation | Published by Tongji University in Cooperation with Elsevier | URL: http://www.journals.elsevier.com/she-ji-the-journal-of-design-economics-and-innovation/

Chair Professor of Design Innovation Studies | College of Design and Innovation | Tongji University | Shanghai, China | Email  [log in to unmask] | Academia https://tongji.academia.edu/KenFriedman | D&I http://tjdi.tongji.edu.cn 

—

Ali Ilhan wrote:

Date:    Wed, 5 Apr 2023 07:48:39 -0400
From:    Ali Ilhan <[log in to unmask]>
Subject: Re: Design research journal rankings

Dear all,

I have not come across a newer ranking article yet, but you may find the
article below on why it is notoriously difficult to assess
research/journals in humanities (which I think  can easily be applied to
design):
https://www.mdpi.com/2078-2489/11/11/540

Now some tangents:

Are all the articles in predaotry journals useless:

https://direct.mit.edu/qss/article/doi/10.1162/qss_a_00242/114726/Are-papers-published-in-predatory-journals

Preprint version:
https://osf.io/preprints/642ad/

and

This is an excellent book about the “ills” of quantifying research
assesement (the book focuses on British research excellence framework):

https://cup.columbia.edu/book/the-quantified-scholar/9780231197816

“Combining interviews and original computational analyses, The Quantified
Scholar provides a compelling account of how scores, metrics, and
standardized research evaluations altered the incentives of scientists and
administrators by rewarding forms of scholarship that were closer to
established disciplinary canons. In doing so, research evaluations
amplified publication hierarchies and long-standing forms of academic
prestige to the detriment of diversity. Slowly but surely, they reshaped
academic departments, the interests of scholars, the organization of
disciplines, and the employment conditions of researchers.”

All the best,

Ali


-----------------------------------------------------------------
PhD-Design mailing list  <[log in to unmask]>
Discussion of PhD studies and related research in Design
Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
-----------------------------------------------------------------