Efficient and fault-tolerant distributed algorithm for learning latent factor models through matrix factorization

摘要

A method for estimating model parameters. The method comprises receiving a data set related to a plurality of users and associated content, partitioning the data set into a plurality of sub data sets in accordance with the users so that data associated with each user are not partitioned into more than one sub data set, storing each of the sub data sets in a separate one of a plurality of user data storages, each of said data storages being coupled with a separate one of a plurality of estimators, storing content associated with the plurality of users in a content storage, where the content storage is coupled to the plurality of estimators so that the content in the content storage is shared by the estimators, and estimating, asynchronously by each estimator, one or more parameters associated with a model based on data from one of the sub data sets.

主权项

1. A method implemented on a computer having at least one processor, a storage, and a communication platform for providing personalized content, comprising:
receiving a data set including information on interactions between a plurality of users and a plurality of pieces of content, wherein the plurality of users include multiple sets of users; partitioning the data set into a plurality of sub data sets each of which corresponds to a separate set of the multiple sets of users; associating each of the plurality of sub data sets with a corresponding one of a plurality of estimators; storing one or more model parameters in a shared storage, wherein the shared storage is coupled to the plurality of estimators so that the plurality of estimators can asynchronously access the one or more model parameters; estimating, asynchronously by each of the plurality of estimators, the one or more model parameters based on corresponding one of the plurality of sub data sets associated with the estimator and the stored one or more model parameters; updating, asynchronously by each of the plurality of estimators, the one or more model parameters stored in the shared storage based on the asynchronously estimated one or more model parameters; obtaining information related to a user; and providing content personalized with respect to the user based on the information related to the user and the one or more model parameters stored in the shared storage.