Identifying likely faulty components in a distributed system

摘要

In general, techniques are described for automatically identifying likely faulty components in massively distributed complex systems. In some examples, snapshots of component parameters are automatically repeatedly fed to a pre-trained classifier and the classifier indicates whether each received snapshot is likely to belong to a fault and failure class or to a non-fault/failure class. Components whose snapshots indicate a high likelihood of fault or failure are investigated, restarted or taken off line as a pre-emptive measure. The techniques may be applied in a massively distributed complex system such as a data center.

主权项

1. A method of predicting component failure, the method comprising:
receiving, by a communication protocol and with a virtual network controller that includes an analytics plane to analyze operations of a plurality of components in one or more virtual networks, a first parameter set from each of the components, wherein a parameter set from a component includes one or more quantitative parameters that each describes a state of the component; receiving, by the communication protocol and with the virtual network controller, an indication of detected component failure for one or more of the components; training, with the virtual network controller and using the first parameter sets and the indication of detected component failure, a trainable automated classifier to develop a classifying structure that distinguishes between component parameter sets that logically associate with a detected component failure and component parameter sets that do not logically associate with a detected component failure; receiving, by the communication protocol and with the virtual network controller, a second parameter set from each of the components; and predicting, with the virtual network controller using the trainable automated classifier and the classifying structure, a failure of a first one of the components.