IMPLEMENTING A RESET POLICY DURING A SEQUENTIAL VARIATION TEST OF CONTENT

摘要

A variation testing system for performing variation testing of content is disclosed. The variation testing system allows an experimenter to continuously monitor the results of the variation test. In order to ensure that the experimenter is provided with valid results, the variation testing system may reset the variation test.

主权项

1. A computer-implemented method for conducting a variation. experiment on a web page, the method comprising:
receiving indications that the web page is provided to a plurality of client devices, the web page associated, with a first variation of the web page and a second variation of the web page that is distinct from the first variation of the web page, and each of the plurality of client devices receive either the first variation of the web page or the second variation of the web page; and responsive to receiving each indication:
determining a difference in conversion rate of user interactions between the first variation of the web page and the second variation, of the web page, anddetermining whether to reset the variation experiment based on the determined difference in conversion rate.