Systems and methods for large-scale randomized optimization for problems with decomposable loss functions

摘要

Systems and methods directed toward processing optimization problems using loss functions, wherein a loss function is decomposed into at least one stratum loss function, a loss is decreased for each stratum loss function to a predefined stratum loss threshold individually using gradient descent, and the overall loss is decreased to a predefined threshold for the loss function by appropriately ordering the processing of the strata and spending appropriate processing time in each stratum. Other embodiments and aspects are also described herein.

主权项

1. A method comprising:
decomposing a primary loss function into stratum loss functions, each of the stratum loss functions having a non-zero weight applied thereto; decreasing a stratum loss to a predefined stratum loss threshold for each of the stratum loss functions by processing each of the stratum loss functions individually using gradient descent, the non-zero weights remaining as non-zero weights throughout said decreasing; and decreasing a primary loss to a predefined primary loss threshold of the primary loss function by processing each of the stratum loss functions according to a stratum sequence, the sequence being chosen to establish convergence to at least one of: stationary points of the primary loss function, and chain-recurrent points of the primary loss function.