Burke,
In some sense you question is of little import. I don't mean to denigrate you, or your thought process by saying this, but we must remember that there is nothing "magical" about a p<0.05. There is no science behind the choice of 0.05 as indicating significance as compared to any other value. What is the difference between a p<0.04 which we say is significant and one that is <0.06 which we say is not significant? They both differ form the magical 0.05 by 0.01! The choice of 0.05 comes from R.A. Fisher who pulled the value out of the air. It is, I believe far better to give the effect size along with a measure its precision (i.e. the SE) and a p value, or perhaps better the effect and a 95% confidence interval around the effect without getting tied into knots determining what is statistically significant and what is not. It is all to easy to fall into the trap of saying that one will pay attention to a test associated with a p<0.05 (or <=0.05) and ignore results with any larger p value.
We must also remember statistical significance does not mean a result is important, and conversely.
John
John David Sorkin M.D., Ph.D.
Chief, Biostatistics and Informatics
University of Maryland School of Medicine Division of Gerontology
Baltimore VA Medical Center
10 North Greene Street
GRECC (BT/18/GR)
Baltimore, MD 21201-1524
(Phone) 410-605-7119
(Fax) 410-605-7913 (Please call phone number above prior to faxing)>>> Burke Johnson <[log in to unmask]> 1/7/2010 5:23 PM >>>
Hello list members,
I have a question for you: Would you advocate "case 1" or "case 2" below (or do you have a preferred "case 3")?
Case 1.
If p is less than or equal to alpha, then reject null.
If p is greater than alpha, then fail to reject null.
Case 2.
If p is less than alpha, then reject null.
If p is greater than or equal to alpha, then fail to reject null.
As you can see, for completeness I'm asking for your thoughts about the highly unlikely (but possible) situation where p=alpha.
For example, when using an alpha level of .05, what would you do in the unlikely situation where the observed p-value is equal to .05 (i.e., alpha is set at .05 and the observed p=.05 to as many places as the computer prints out).
If you recommended case 1, I have a follow-up question about rounding: What observed p-value would you consider close enough to be considered "equal to .05" in the procedure? (The late Jacob Cohen offered a convention that a p-value of .00 to .05 was sufficiently small, but .051-1.00 was not sufficiently small to reject the null).
Thanks in advance for your thoughts!
Burke Johnson
You may leave the list at any time by sending the command
SIGNOFF allstat
to [log in to unmask], leaving the subject line blank.
Confidentiality Statement:
This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.
You may leave the list at any time by sending the command
SIGNOFF allstat
to [log in to unmask], leaving the subject line blank.
|