Method and system for processing measurement data for website statistics

摘要

Methods and systems for storing web-site visitor data in a specific bucket among a plurality of buckets based on a unique visitor identifier. The methods and systems may dynamically change the number of buckets based on certain conditions, such as when one of the buckets reaches a predetermined size. Several processors may be used in parallel to process the visitor data from two or more of the buckets simultaneously, for example to generate or obtain report data. The systems and methods may aggregate the report data into a result table.

主权项

1. A system for processing measurement data for website statistics, comprising:
a plurality of processors, at least some of the plurality of processors are implemented at least partially by hardware; and a memory system, operably connected to the plurality of processors, containing instructions that cause the plurality of processors to perform operations comprising:
receiving measurement data from a website, wherein the measurement data comprises a plurality of visitor data indicating at least activities associated with corresponding visitors of the website;selecting a first bucket of a plurality of buckets based on a hash value calculated from a unique user identifier of a first unique visitor, wherein the first bucket is associated with a first plurality of hash values, wherein a bucket contains at least a log file associated with at least an activity of a particular visitor;storing all visitor data for the first unique visitor in the first bucket of the plurality of buckets;selecting a second bucket of the plurality of buckets based on a hash value calculated from a unique user identifier of a second unique visitor, wherein the second bucket is associated with a second plurality of hash values different from the first plurality of hash values;storing all visitor data for the second unique visitor in the second bucket of the plurality of buckets;dynamically changing an amount of the plurality of buckets when at least one of the first bucket and the second bucket reaches a predetermined size, wherein the amount is based on a size of the website;
receiving a report request, from a client device, that comprises a query for report data on the first unique visitor and the second unique visitor;in response to receiving the report request, processing the plurality of visitor data stored in the plurality of buckets using the plurality of processors to obtain the report data, wherein:
the first bucket and all the visitor data for the first unique visitor are processed by a first processor of the plurality of processors;the second bucket and all the visitor data for the second unique visitor are processed by a second processor of the plurality of processors; andthe first processor and the second processor are processing the first and second buckets simultaneously; andproviding the report data to the client device.