STOCHASTIC CONTROL OPTIMIZATION FOR SENDER-BASED FLOW CONTROL IN A DISTRIBUTED STATEFUL MESSAGING SYSTEM

摘要

A method and system for controlling message flow in distributed stream processing. State transition probabilities in a Markov model having one state per staleness value of data are determined for sending or withholding updates of data to subscribers using expected message rates from an information provider. A cost function annotates each state transition in the model with a state transition cost for each decision to "send" or "withhold". A propagation policy specifying whether to send or withhold the message is determined for each state. The propagation policy is then deployed. If a new message comprising an update of data is received during a lapsed time unit, a staleness value of the data held by subscribers is increased. The propagation policy is used to determine whether to send or withhold the message. If the message should be sent, the message is propagated and the staleness value of the data is reset.