About the coefficient alpha’s saga: when we compute a reliability coefficient statistic (alpha or alternative formula) to estimate the reliability of variables, we assume that the numbers interred into the formula are measurements (i.e., addition and multiplication are meaningful operations). But if we acknowledge the default hypothesis that these numbers are not measurements (a testable hypothesis), what is the usefulness of reliability?

Stéphane Vautier

Octogone-Lordat, Université de Toulouse

Blog : http://epistemo.hypotheses.org/

Just out … for those who still use ‘psychometric’ estimates of reliability (not me).

A very nicely writen article; nice straightforward explanations, and very handy as a source of formulae, explanations, and examples of the ‘old’ with the ’new’. R-code is provided as a ‘how to’ using a variety of R modules -  in an appendix.

McNeish, D. (2017). Thanks coefficient alpha, we'll take it from here. Psychological Methods (http://dx.doi.org/10.1037/met0000144 ), In Press, , 1-23.

Abstract

Empirical studies in psychology commonly report Cronbach’s alpha as a measure of internal consistency reliability despite the fact that many methodological studies have shown that Cronbach’s alpha is riddled with problems stemming from unrealistic assumptions. In many circumstances, violating these assumptions yields estimates of reliability that are too small, making measures look less reliable than they actually are. Although methodological critiques of Cronbach’s alpha are being cited with increasing frequency in empirical studies, in this tutorial we discuss how the trend is not necessarily improving methodology used in the literature. That is, many studies continue to use Cronbach’s alpha without regard for its assumptions or merely cite methodological articles advising against its use to rationalize unfavorable Cronbach’s alpha estimates. This tutorial first provides evidence that recommendations against Cronbach’s alpha have not appreciably changed how empirical studies report reliability. Then, we summarize the drawbacks of Cronbach’s alpha conceptually without relying on mathematical or simulation-based arguments so that these arguments are accessible to a broad audience. We continue by discussing several alternative measures that make less rigid assumptions hich provide justifiably higher estimates of reliability compared to Cronbach’s alpha. We conclude with empirical examples to illustrate advantages of alternative measures of reliability including omega total, Revelle’s omega total, the greatest lower bound, and Coefficient H. A detailed software appendix is also provided to help researchers implement alternative methods.

Translational Abstract

Scales are commonly used in psychological research to measure directly unobservable constructs like motivation or depression. These scales are comprised of multiple items, each aiming to provide information about various aspects of the construct of interest. Whenever a scale is used in a psychological study, it is important to report on its reliability. Since the 1950s, the primary method for capturing reliability has been Cronbach’s alpha, a method whose status is perhaps best exemplified by its place as one of the most cited scientific articles of all-time, in any field. Despite its overwhelming popularity, the underlying assumptions of Cronbach’s alpha have been questioned recently in the statistical literature because these assumptions were commonplace 65 years ago but have largely disappeared from more modern statistical methods for constructing scales. Though the ideas in these statistical articles have the potential to significantly alter how psychological research is conducted and reported, recommendations from the statistical literature have yet to permeate the psychological literature. In this article, the goal is to demonstrate why Cronbach’s alpha is no longer the optimal method for reporting on reliability. To differentiate this article from articles appearing in the statistical literature, we approach issues with Cronbach’s alpha with very little focus on mathematical or computational detail so that the deficiencies of Cronbach’s alpha are illustrated in words and examples rather than proofs and simulations so that these ideas can impact a larger group of researchers—namely, the researchers who most often report Cronbach’s alpha.

Regards .. Paul

Chief Research Scientist

Cognadev.com

__________________________________________________________________________________

W: www.cognadev.com

W: www.pbarrett.net

E: [log in to unmask]

M: +64-(0)21-415625