Hello again
 
Many, many thanks to all who responded to my request yesterday .. With the
subject title: "Testing non-nested regression Rsquares". I received about
10 responses in all - which is impressive.

Short answer is that the problem as defined might indeed be conceived of as
a nested regression issue, along with the secondary proposition that the
AIC or BIC information indices might also be worth using in this context,
and the third suggestion to use a bootstrap approach to develop empirical
thresholds for significance.

I've posted some of the key replies below which elaborate on each of these
approaches: I've not posted names etc. in order to assure confidentiality
...

One book I found most useful - in fact absolutely riveting frankly - was:

Burnham, K.P. and Anderson, D.R. (2003) Model Selection and Multimodel
Inference: A Practical Information-Theoretic Approach 2nd Edition. New
York: Springer. ISBN: 0-387-95364-7


-Reply #1-
This actually a "nonlinear" regression problem, so I'm not surprised at the
confusion.  My research field, so I've done this alot, esp with
econometrics application, where it is popular.  Are you using SAS/ 

Here's the model, where K is the unknown transition point,
for X < K, E(Y) = b0 + b1*X
for X > K, E(Y) = g0 + g1*X, where g0 = b0 + (b1 - g1)*K

That's the Full model.
The Reduced model is just a line (E(Y) = b0 + b1*X),
then put the SSE's into the usual formula.

The Full model has 4 unknown parameters, the reduced one has just two.


-Reply #2-
A simple linear regression is indeed nested in a split-line or break-point
regression: 
1. E(y) = a + b.x 
2. E(y) = a + b.x + c.x.(x>d), where d is the break-point, and c is the
change in slope 
Thus the MS associated with the difference, divided by the residual MS,
should have an F statistic with (2, r) d.f., where r is the number of
residual d.f. 


-Reply #3-
So long as the predictor (X) variable is the same, I think the break-point
model can be considered a more elaborate version of the simple regression
(or the simple regression is the same as a breakpoint regrssion where the
gradients in the two segments are constrained to be the same);  the break
point model has added one or two extra parameters (one if the location of
the breakpoint is fixed, two if you are estimating it from the data)

For comparing non-nested regression models  you can use the Adjusted
R-square, Akaike's Information Criterion (AIC), or Mallow's Cp  (Hmm, what
would be a good reference?  Draper & Smith, Applied Regression Analysis,
Wiley, covers Cp and Adjusted R-Square)  Not sure if there are any formal
*tests* for these or not.


-Reply #4-
I'm not so sure this is non-nested. If the slopes of all the line segments
are the same then the break-point model becomes the same as the linear
regression model. Therefore it's nested, surely? Slightly more tricky is
the change in df but i guess 2 per break-point if you're allowing x as well
as y position of the break-point(s) to be fitted to the data.


-Reply #5-
I think that the answer will depend very much on the form of the second
model, if the first is -- as in your example -- the simpler. Standard
properties of tests (eg LR tests) will not apply in the context of a
break-point (segmented) regression, assuming that the break-point parameter
is estimated. (If given, then standard Rsquare applies because the model
can be fit as linear regression with an added covariate.)A general approach
would be to use a bootstrap evaluation of significance, essentially
simulating data from the null fitted model.


-Reply #6-
This is not possible for separate families of hypotheses.
  See the paper
  Williams, D.A. -1970 -Discriminating between regression models to
determine the pattern of enzyme syntheses in synchronous cell cultures.
Biometrics 28, 23-32
  It deals with your problem using what is now called parametric bootstrap.
  For a review and recente refences in nonnested models look
  Separated families of hypotheses, vol 7 of Encyclopedia of Biostatistics
pg 4881-4886


Once again, many thanks to all who responded ... 

Regards .. Paul
_________________________________________________________________
Paul Barrett    Tel: +64 (0)9-373-7599 x82143     Mob: 021-415625
Adjunct Professor of Psychometrics,  University of Auckland, NZ
Adjunct Assoc. Prof. of Psychology,  University of Canterbury, NZ
   
email: [log in to unmask]
       [log in to unmask] 
       [log in to unmask]
web:   www.pbarrett.net