For a quasi-experimental impact study of a job training program, we are using propensity score matching and a difference-in-difference analysis. We have elected to use kernel matching because it gives us the closest match. Since kernel matching compares the treated with a weighted average of all untreated persons (i.e. with the highest weight given to those with propensity scores closest to the treated), we’re a little stuck in terms of interpreting statistical significance. For the difference-in-difference, Stata only includes a t-stat. Since t-stats require one to know the degrees of freedom to determine significance, the question is how to determine degrees of freedom in this instance.
Should we be calculating the degrees of freedom based on the number of control group members with a weight greater than zero? Or is it technically correct to use the entire (trimmed) control group regardless of whether or not weight is being applied? Our sample size is fairly large (over 500 for the treatment group and about 5000 for the total trimmed control group).
Any insights would be much appreciated.
You may leave the list at any time by sending the command
SIGNOFF allstat
to [log in to unmask], leaving the subject line blank.
|