Parsimonious network design and feature selection through node pruning

Abstract

We propose a node saliency measure and a back-propagation type of algorithm to compute the node saliencies. A node-pruning procedure is then presented to remove insalient nodes in the network to create a parsimonious network. The optimal/suboptimal subset of features are simultaneously selected by the network. The performance of the proposed approach for feature selection is compared with Whitney's feature selection method. One advantage of the node-pruning procedure over classical feature selection methods is that the node-pruning procedure can simultaneously "optimize" both the feature set and the classifier, while classical feature selection methods select the "best" subset of features with respect to a fixed classifier.