OUTLIER DATA DETECTION

摘要

Online experimentation has been widely used to evaluate an effect of a new feature of an online product on user engagement. One challenge is that an existence of outliers can often complicate the analysis of such experimental results. Thus, a procedure is provided herein to detect and remove outliers from experimental results. The procedure can use statistical tests based on parametric distributions of sample maximum or minimum. These tests can be performed using an inward testing procedure to identify multiple outliers. Finally, these filtered test results can be used to control delivery of a new feature of an online product.

主权项

1. A system, comprising:
A/B testing circuitry, configured to:
test a feature of an online product for an effect of the feature on user engagement with the online product; and communicate user engagement data associated with the effect to requesting circuitry; outlier removal circuitry, configured to:
request user engagement data from the A/B testing circuitry;detect an outlier value in the user engagement data;remove the outlier value in the user engagement data;communicate the user engagement data subsequent to removal of the outlier to requesting circuitry; andwith respect to the detection of the outlier value,
estimate parameters of a distribution of the user engagement data using a hurdle model and a negative binomial distribution;identify a maximum value or a minimum value in the user engagement data according to the estimated parameters; anddetermine whether the maximum value or the minimum value is an outlier value according to a distribution of sample maximum or a distribution of sample minimum, respectively; and feature delivery circuitry, configured to:
request user engagement data from the outlier removal circuitry; and deliver a corresponding feature of an online product according to the user engagement data.