Policy-driven automatic network fault remediation

摘要

A policy-driven automatic network remediation service is described, which resides on the network and is triggered when a network fault is detected. Once triggered, the service automatically connects to network devices in the topological locale of the detected fault and collects diagnostic information from the affected area, running diagnostics which are appropriate to the fault type. The service can validate a set of preconditions prior to taking remedial action. For example, the service can empirically validate that the network topology is actually as expected and that automatic remediation would be safe and would not compromise network availability or redundancy. Diagnostic information can be recorded in a trouble ticket to support post-event auditing. Once the preconditions have been validated, the service can automatically take corrective action based on the type of the fault, such as shutting down an interface on a particular network device.

主权项

1. A computer-implemented method for policy-driven automatic network fault remediation, the method comprising:
receiving a notification by a network remediation service, the notification indicating that an error has been detected in at least one of a plurality of devices on a network, the error being associated with a particular error type; inspecting a policy by the network remediation service, the policy controlling how remediation of the error is performed; logging in to a device on the network on which the error has been detected to have occurred by a network remediation service; checking a set of preconditions to determine whether a corrective action should be performed on the device; causing a set of diagnostic commands to be executed on the device by the network remediation service; performing the corrective action by the network remediation service, the corrective action being based on the error type of the error; and causing the network remediation service to suspend itself in response to a determination that a predetermined number of corrective actions have been performed as specified in the policy, wherein the determination that the predetermined number of corrective actions have been performed includes tracking a number of interfaces or devices that have been shut down by the network remediation service.