DATA PROCESSING METHOD, SYSTEM AND RELATED DEVICE

摘要

Disclosed are a data processing method and system, and a related device, used for improving the speed and accuracy for solving parameters in an hLDA model. The method according to an embodiment of the present invention comprises: sending initial global statistical information to each slave node, and merging local statistical information received from each slave node, to obtain new global statistical information; if Gibbs sampling performed on the slave nodes is finished, calculating, according to the new global statistical information, probability distribution between documents and topics and probability distribution between the topics and words; establishing, according to the calculated probability distribution, a likelihood function of a text set, and maximizing the likelihood function, to obtain a new hLDA hyper-parameter; and if the iteration for solving the hLDA hyper-parameter is converged, calculating and outputting, according to the new hLDA hyper-parameter, probability distribution between the documents and the topics and probability distribution between the topics and the words.