Print

Print
Hi Terry,
I think the answer is...."no."

The reason is that no number of covariates can address some key unknown confounders, like attitude or intent.  Imagine a study of healthy lifestyle--one may find identical demographics and comorbidity etc but an attitude towards health behaviors may, either in and of itself be predictive of outcome or at least be associated with a number of unknown confounders.  In theory if one knew everything then the answer could be, I suppose, yes.  But one cant know everything or enough.   And obs studies often address questions that are different from those in trials. Miguel Hernan has written a lot about this, that I find quite useful.  For example, though not exactly on topic, the considerations are related  http://annals.org/article.aspx?articleid=1735165

best,
Rich Saitz


On 21 March 2014 09:40, Mayer, Dan <[log in to unmask]> wrote:

Hi Terry and list.


This is a great question.   I’ve always thought of propensity matching as a way of doing a pre-test adjustment and so apply the rules that I would use in looking at data that has been adjusted for whatever.

 

I have two main rules:

1.       The number of outcome events should be at least 20 times the number of variables that are used in the adjustment.

2.       There should not be any ‘obvious’ variables that were ‘left out’ of the adjustment process.   This is a bit subjective and I always ask about educational level and socioeconomic status of the participants, things that are usually not added to any adjustments and which in some situations are likely would lead to biased results.

 

The bottom line is that observational studies need to take so many more precautions in order to have low levels of bias and that is why they are classified as low quality to start out on the GRADE quality criteria.

 

Hope that this helps.

Best wishes,

 

Dan

 

 

Dan Mayer, MD

Professor of Emergency Medicine

Albany Medical College

Mail Code 34

47 New Scotland Ave.

Albany, NY, 12208

phone: 518-262-6180

FAX: 518-262-5029

Mobile: 518-461-3191

 

 

 

From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Shaneyfelt, Terry
Sent: Thursday, March 20, 2014 12:24 PM
To: [log in to unmask]
Subject: Re: [EXTERNAL] RE: Is propensity matching equivalent to randomization?

 

Is there a number of confounding variables you can account for after which you consider the likelihood of unrecognized confounding variables is extremely low?

 

This is really the crux of my question. I cant remember the exact reference but we read a large propensity matched study for journal club and a lot of variables were in the propensity model. As I thought about it I couldn’t think of anything else that could be a confounder that was not covered by what they put in the propensity model. I know the propensity model doesn’t predict the outcome but it contained everything that could be a potential confounder of the outcome. The residents asked me if this was as good as a RCT. I wonder the same thing.

 

 

From: Brian Alper MD [mailto:[log in to unmask]]
Sent: Thursday, March 20, 2014 10:26 AM
To: Shaneyfelt, Terry
Cc: [log in to unmask]
Subject: [EXTERNAL] RE: Is propensity matching equivalent to randomization?

 

Randomization balances the unrecognized variables (or at least increases the likelihood that the unrecognized variables will be balanced and not biasing results in an unseen way).

 

How many recognized covariaties must be balanced to have a “balancing” of the unrecognized variables?

 

It seems to me like the more propensity matching that is done the more balancing you get but how do you know when you have “reasonable certainty” for the effect on the unknown variables?

 

Is there a number of confounding variables you can account for after which you consider the likelihood of unrecognized confounding variables is extremely low?

 

I don’t have a statistical answer – just trying to rephrase the question to understand it in terms of reaching high likelihood of certainty in the estimates of effect.

 

From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Shaneyfelt, Terry
Sent: Thursday, March 20, 2014 11:10 AM
To: [log in to unmask]
Subject: Is propensity matching equivalent to randomization?

 

This may seem like a silly question but is propensity matching in a large observational study equivalent to randomization? Consider the propensity score to have been developed using a lot of covariates (everything you could think of and measure). As you have probably seen, large propensity matched cohorts will be similar to each other within tenths of percentage points on various characteristics. If you develop your propensity score with enough covariates what else is potentially missing that could be a confounder of the effect?

----------------------------------------- CONFIDENTIALITY NOTICE: This email and any attachments may contain confidential information that is protected by law and is for the sole use of the individuals or entities to which it is addressed. If you are not the intended recipient, please notify the sender by replying to this email and destroying all copies of the communication and attachments. Further use, disclosure, copying, distribution of, or reliance upon the contents of this email and attachments is strictly prohibited. To contact Albany Medical Center, or for a copy of our privacy practices, please visit us on the Internet at www.amc.edu.




--
Richard Saitz
Editor Evidence-Based Medicine

BMJ, BMA House, Tavistock Square, London, WC1H 9JR
T: +1 617 414 7744
E: [log in to unmask]
W: ebm.bmj.com
Twitter: @EvidBaseMed_BMJ 
www.facebook.com/Evidence.Based.Medicine

BMJ advances healthcare worldwide by sharing knowledge and expertise to improve experiences, outcomes and value. This email and any attachments are confidential. If you have received this email in error, please delete it and kindly notify us. If the email contains personal views then BMJ accepts no responsibility for these statements. The recipient should check this email and attachments for viruses because the BMJ accepts no liability for any damage caused by viruses. Emails sent or received by BMJ may be monitored for size, traffic, distribution and content. BMJ Publishing Group Limited trading as BMJ. A private limited company, registered in England and Wales under registration number 03102371. Registered office: BMA House, Tavistock Square, London WC1H 9JR, UK.