Method for detection of redundant messages in a message stream

摘要

The method involves storing messages during a time window of predetermined period, and concatenating the messages. A pattern dictionary is updated from the result of iterative pattern search on executed concatenation. Each pattern is associated to a frequency of occurrence in the concatenation result. Messages having a pattern of length and frequency of occurrence greater than threshold are marked.