The present invention relates to a pattern recognition system which uses data fusion to combine data from a plurality of extracted features and a plurality of classifiers. Speaker patterns can be accurately verified with the combination of discriminant based and distortion based classifiers. A novel approach using a training set of a "leave one out" data can be used for training the system with a reduced data set. Extracted features can be improved with a pole filtered method for reducing channel effects and an affine transformation for improving the correlation between training and testing data.