Online asynchronous reinforcement learning from concurrent customer histories

摘要

In one embodiment, an indication of a Decision Request or an Update Request may be received, where the Update Request is activated independent of user activity. A user state pertaining to at least one user may be received, obtained, accessed or constructed. For the Decision Request, one or more actions may be scored according to one or more value functions associated with a computing device, a policy associated with the computing device may be applied to identify one of the scored actions as a decision, and an indication of the decision may be provided or applied. For the Update Request, the one or more value functions and/or the policy may be updated. An indication of updates to the one or more value functions and/or an indication of updates to the policy may be provided.

主权项

1. An apparatus, comprising;
one or more computing devices, each of the computing devices having one or more processors and memories configured to perform a method of asynchronous reinforcement learning (RL), including:
obtaining an indication of a Decision Request;receiving, obtaining, accessing or constructing a user state pertaining to at least one user; andin response to the Decision Request:
scoring a plurality of actions according to one or more value functions based, at least in part, upon the user state;applying a policy to identify one of the scored actions as a decision; andproviding an indication of the decision or applying the decision to the at least one user;obtaining an indication of an Update Request, the Update Request being activated independent of user activity;receiving, obtaining, accessing or constructing a further user state pertaining to the at least one user; andin response to the Update Request:
updating at least one of: the one or functions and the policy based, at least in part, upon the further user state,wherein the Decision Request is activated in response to an event timer and the event timer operates to periodically generate Decision Requests, wherein a frequency with which the event timer generates the Decision Requests is based at least in part, upon a period of time from a last user event pertaining to the at least one user or from a last user action, the last user action including the providing of the indication of the decision or the applying of the decision to the at least one user.