Methods for detecting split brain in a distributed system

摘要

A method and apparatus for detecting split brain in a distributed system is provided. After determining that a rogue instance is no longer an active member of the cluster, a recovery instance detects activity associated with a redo log that is updated by the rogue instance to store log records that describe changes made by the rogue instance to data associated with the cluster.

主权项

1. A method for detecting a split brain condition in a cluster of a plurality of instances comprising:
determining, by one or more second instances of the plurality of instances, that a first instance of the plurality of instances is presumably no longer an active member of the cluster; said first instance updating a redo log with log records that describe changes made to a shared data resource by the first instance; performing, by the one or more second instances of said plurality of instances, instance recovery of the first instance after determining that the first instance is no longer an active member of the cluster, wherein instance recovery comprises performing a first pass of the redo log and detecting, during a second pass of the redo log, any new activity in the redo log since the first pass; and based on the new activity, determining that the previously-presumed inactive first instance, is an active member of a disjoint cluster; wherein the one or more second instances are running on one or more computing devices.