Flexible Experiment Framework for Hierarchical Data

摘要

Methods, systems, and apparatus include computer programs encoded on a computer-readable storage medium for conducting an experiment for a campaign. A method includes: identifying a campaign including one or more content items and current selection criteria; receiving an indication of a proposed modification to be made to the campaign; storing the proposed modification as a draft campaign including the proposed modification in a first structure that includes the differences that are proposed to the campaign and a second structure that represents the campaign with the proposed modifications applied; determining a split to be applied as between using current campaign parameters or current selection criteria for a first portion of received traffic and using the proposed modification in the draft campaign for a second portion of the received traffic; and conducting an experiment based on using received traffic and in accordance with the split.

主权项

1. A method comprising:
identifying a campaign, the campaign including one or more groups of content items and current selection criteria for each group, wherein the current selection criteria define conditions that are to be satisfied for an individual content item to be eligible to be presented responsive to a received request; receiving an indication of a proposed modification to be made to one or more current campaign parameters or the current selection criteria associated with a group; storing the proposed modification as a draft campaign including storing the proposed modification in a first structure that includes only the differences that are proposed to the campaign and a second structure that represents the campaign with the proposed modifications applied; determining a split to be applied as between using the current campaign parameters or current selection criteria for a first portion of received traffic and using the proposed modification in the draft campaign for a second portion of the received traffic; and conducting an experiment based on using received traffic and in accordance with the split including delivering impressions based on both the current campaign parameters or current selection criteria and the proposed modification including serving content in accordance with the campaign and the draft campaign.