Optimal linear filters for nonlinear magnetic recording channels

Abstract

The problem of specifying the optimal linear receiving filter (OLRF) for nonlinear channels is considered. This filter minimizes the mean-squared error (MSE) between the samples taken from its output signal and corresponding known reference samples. The MSE attained by the unrealizable OLRF establishes a lower bound on the MSE achievable with any linear filter. Transfer functions of optimal and suboptimal filters were determined for realistic models of digital magnetic-saturation recording channels and three- or five-level reference samples. Optimal filters provide the best tradeoff between reduction of noise and elimination of linear and nonlinear intersymbol interference (ISI) in the signal samples. For the hypothetical situation of vanishing noise in the read-back signal, the OLRF removes all ISI at the sampling instant at the expense of increased timing-phase sensitivity and larger variance in the signal envelope.