Performing an operation using multiple services

摘要

Some embodiments provide a method for distributing an operation for processing by a set of background services. The method automatically determines a number of background services for performing an operation. The method partitions the operation into several sub-operations. The method distributes the several sub-operations across the determined number of background services.

主权项

1. A non-transitory machine readable medium storing a program which when executed by at least one processing unit of a particular physical machine having a plurality of processing units distributes an operation for processing by a set of background services, the program comprising sets of instructions for:
maintaining a service configuration table comprising a mapping of operation properties and hardware resources to a number of processing units used by each background service and a maximum number of background services for performing the operation; detecting a set of the hardware resources of the particular physical machine and a set of properties for the operation; performing a lookup on the service configuration table using the detected set of the hardware resources and the detected set of the properties for the operation in order to calculate a number of background services to instantiate to perform the operation; instantiating the calculated number of background services on the particular physical machine, each of the background services utilizing a different one or more of the processing units of the particular physical machine; partitioning the operation into a number of sub-operations that is larger than the number of background services; initially distributing one sub-operation to each instantiated background service on the particular physical machine; and each time a particular background service completes a sub-operation, distributing one undistributed sub-operation to the particular background service until all of the sub-operations are distributed.