Adaptive nonlinear equalizer using a mixture of Gaussian-based online density estimator

This paper introduces a new adaptive nonlinear equalizer relying on a radial basis function (RBF) model, which is designed based on the minimum bit error rate (MBER) criterion, in the system setting of the intersymbol interference channel plus cochannel interference (CCI). Our proposed algorithm is referred to as the online mixture of Gaussian-estimator-aided MBER (OMG-MBER) equalizer. Specifically, a mixture of Gaussian-based probability density function (pdf) estimator is used to model the pdf of the decision variable, for which a novel online pdf update algorithm is derived to track the incoming data. With the aid of this novel online mixture of Gaussian-based sample-by-sample updated pdf estimator, our adaptive nonlinear equalizer is capable of updating its equalizer's parameters sample by sample to aim directly at minimizing the RBF nonlinear equalizer's achievable bit error rate (BER). The proposed OMG-MBER equalizer significantly outperforms the existing online nonlinear MBER equalizer, known as the least bit error rate equalizer, in terms of both the convergence speed and the achievable BER, as is confirmed in our simulation study.