System and method for throttling service requests having non-uniform workloads

摘要

A system that provides services to clients may receive and service requests, various ones of which may require different amounts of work. The system may determine whether it is operating in an overloaded or underloaded state based on a current work throughput rate, a target work throughput rate, a maximum request rate, or an actual request rate, and may dynamically adjust the maximum request rate in response. For example, if the maximum request rate is being exceeded, the maximum request rate may be raised or lowered, dependent on the current work throughput rate. If the target or committed work throughput rate is being exceeded, but the maximum request rate is not being exceeded, a lower maximum request rate may be proposed. Adjustments to the maximum request rate may be made using multiple incremental adjustments. Service request tokens may be added to a leaky token bucket at the maximum request rate.

主权项

1. A method, comprising:
performing by a computer system that provides storage services to clients:
receiving a plurality of requests to read or write data on behalf of one or more clients;servicing a portion of the requests, wherein the portion of the plurality of service requests that is serviced is dependent on a maximum request rate;determining that the rate at which the plurality of requests was received exceeds the maximum request rate;in response to said determining, adjusting the maximum request rate, wherein said adjusting is dependent on an observed data transfer rate required to satisfy the portion of the requests, and wherein the amount of data transferred in servicing each of the requests in the portion of the requests is non-uniform; andsubsequent to said adjusting:
receiving one or more additional requests to read or write data on behalf of the one or more clients; andservicing a portion of the additional requests, wherein the portion of the additional requests that is serviced is dependent on the adjusted maximum request rate.