AUTOMATED EXPERIMENT SCHEDULING

摘要

Various embodiments are generally directed to an automated A/B testing system using a combination of classical and alternative statistical analysis to control the performance A/B tests. An apparatus includes a processor circuit; and logic to commence collection of a specified number of samples of user responses to multiple versions of a user interface in a test, employ an alternative statistical analysis to analyze the samples as they are collected, determine whether a proposed version elicits a statistically significant improvement in user response over a control version, terminate the test before the specified number of samples are collected in response to a statistically significant improvement over the control version elicited by the proposed version, and complete collection of the specified number of samples and employ a classical statistical analysis to analyze the samples in response to the proposed version not eliciting the statistically significant improvement. Other embodiments are described and claimed.

主权项

1. An apparatus comprising:
a processor circuit; and logic to:
commence collection of a specified number of samples of user responses to multiple versions of a user interface in a test;employ an alternative statistical analysis to analyze the samples as the samples are collected;determine whether a proposed version of the multiple versions elicits a statistically significant improvement in user response over a control version of the multiple versions;terminate the test before the specified number of samples are collected in response to a statistically significant improvement over the control version elicited by the proposed version; andcomplete collection of the specified number of samples and employ a classical statistical analysis to analyze the samples in response to the proposed version not eliciting the statistically significant improvement.