Backward stepwise elimination: A model-based method for nonlinear dimension reduction

By Changtong Luo

machine learning dimension reduction feature selection backward stepwise elimination

In multidimensional data modeling, dimension reduction is not intuitive. Forward feature selection is usually deceptive. That is, a strongly related feature may have a small correlation coefficient (near zero) to the objective, especially when the target model is nonlinear. Therefore, we suggest using backward stepwise elimination (BSE) for dimension reduction. BSE is an iterative model-based method. It starts from an all-feature model and iteratively eliminates the feature which does not significantly affect the goodness of the surrogate model until no more features could be eliminated without downgrading the model goodness. The surrogate model could be constructed with the extreme learning machine, Kriging, or some other machine learning techniques. The goodness of a model could be evaluated with the coefficient of determination $R^2$, or Nash-Sutcliffe coefficient E, etc.

Attachment: An_illustrative_example.jpg (140 KB)

Comments

Please log in to add a comment.