Scalable testing in a production system with autoscaling

摘要

A network-based production service is configured to process client requests for the production service via a network, capture production request data defining the requests and store the production request data in a data store. A test system comprising one or more controllers creates test jobs according to a test plan for testing the production service. The test plan creates a test profile for using specified production request data to simulate a load on the production service. Each job created by the test plan specifies a portion of production request data. A job queue receives and queues test jobs from one or more controllers configured to add test jobs to the job queue according to the test plan. Workers access jobs from the job queue and the production request data from the data store as specified in each job and replay the production request data to the production service.

主权项

1. A system, comprising:
a network-based production service implemented by one or more computers, wherein the production service is configured to:
process requests from clients for the production service via a network; andcapture production request data defining the requests; a data store configured to store the production request data captured at the production service; a test system implemented by one or more computers configured to implement:
one or more controllers configured to create test jobs according to a test plan for testing a production service, wherein the test plan specifies production request data to simulate a load on the production service, wherein the production request data was previously captured at the production service, and wherein each job specifies a portion of production request data;a job queue configured to receive and queue test jobs from the one or more controllers according to the test plan; andone or more workers to access jobs from the job queue and replay production request data to the production service as specified by the jobs; and a load generator configured to scale the number of said one or more workers in response to one or more metrics.