Hi all,
I am working with Boosting Trees (using STATISTICA Data Miner) for a
classification problem with four possible outcomes. I have run the same
Boosting algorithm over different random subsamples of the same data set and
tested its performance also on different subsamples.
On each of these runs, I got different performances (measured as the % of
corrected classified items on each category) and also different relative
importance values for the predictors.
The category where I am getting more "instability" in the results represents
only about 2% of the overall population (all categories).
Does it make sense to assess the final performance of the model as an
average of the % of corrected classify over all runs? The same for relative
importance, should I average the relative importance of each predictor over
all runs?
I know that this would depend on the particularities of the data and the
stability of the results, but how many times should I run the same model on
different random subsample of the data, and average the results as suggested
above to obtain a "decent" estimate for the overall expected performance for
the model?
Thank you very much,
Leo.
|