A Local Learning Rule That Enables Information Maximization for Arbitrary Input Distributions

Abstract

This note presents a local learning rule that enables a network to maximize the mutual information between input and output vectors. The network's output units may be nonlinear, and the distribution of input vectors is arbitrary. The local algorithm also serves to compute the inverse C-1 of an arbitrary square connection weight matrix.