Systems and methods for conducting experiments on applications

摘要

Systems and methods for conducting a set of experiments on an application having a plurality of users are provided. Each experiment is identified by a respective experiment identification (ID) and is associated with a respective group of experimental variations of the application. A system includes an entropy source module that assigns a user group number to a designated user. The user group number is less than or equal to an entropy cap, which is less than an expected entropy, which represents a total number of different states of the experiments. The system includes an experiment module that determines a designated one of the experiments to be conducted on the application. The designated experiment is identified by a designated experiment ID and is associated with a designated group of experimental variations of the application. The experiment module assigns a designated variation to the designated user based on the user group number.

主权项

1. A system for conducting a set of experiments on an application having a plurality of users, wherein each experiment is identified by a respective experiment identification (ID) and is associated with a respective group of experimental variations of the application, the system comprising:
one or more processors; and a memory containing processor-executable instructions that, when executed by the one or more processors, cause the system to: assign a user group number to a designated one of the plurality of users, the user group number being less than or equal to an entropy cap, the entropy cap being less than an expected entropy associated with the set of experiments, the expected entropy representing a total number of different states of the set of experiments, each of the different states comprising a respective experimental variation of each experiment; determine a designated one of the set of experiments to be conducted on the application in connection with the designated user, wherein the designated experiment is identified by a designated experiment ID and is associated with a designated group of experimental variations of the application; and assign a designated one of the designated group of experimental variations to the designated user based on the user group number.