SYSTEMS AND METHODS FOR ESTIMATING USER JUDGMENT BASED ON PARTIAL FEEDBACK AND APPLYING IT TO MESSAGE CATEGORIZATION

摘要

Messages in a first and second plurality of messages are respectively classified using a first and second classifier into message categories in a set of message categories, with messages in the first and second plurality of messages being associated with message reputation carriers in a plurality of message reputation carriers. The classified messages are delivered to recipients and message category correction events are collected. Correction weights are determined for correction types associated with the set of message categories using the initial message categorizations and the category correction events. At least a subset of the calculated correction weights is used to determine a probability or likelihood that a particular message reputation carrier in the plurality of carriers is associated with a first message category in the set of message categories. The particular carrier is whitelisted to the first message category when the calculated probability or likelihood satisfies a whitelisting criterion.

主权项

1. A method of whitelisting a first message reputation carrier to a first message category in a set of message categories, the method comprising:
at a computer system having one or more processors, and memory storing one or more programs for execution by the one or more processors:
classifying each message in a first plurality of messages using a first classifier, thereby independently identifying an initial message category in the set of message categories for each respective message in the first plurality of messages, wherein the first plurality of messages includes, for each respective message reputation carrier in a plurality of message reputation carriers, at least one message associated with the respective message reputation carrier and wherein the plurality of message reputation carriers includes the first message reputation carrier;classifying each message in a second plurality of messages using a second classifier, thereby independently identifying an initial message category in the set of message categories for each respective message in the second plurality of messages, wherein the second plurality of messages includes, for each respective message reputation carrier in the plurality of message reputation carriers, at least one message associated with the respective message reputation carrier;delivering the first and second plurality of messages to a plurality of recipients with a designation of the message category of each respective message in the first and second plurality of messages, as respectively determined by the first and second classifier;collecting a plurality of recipient initiated message category correction events for messages in the first and second plurality of messages;determining a correction weight for each respective correction type associated with the set of message categories using at least (i) the initial message category for each respective message in the first plurality of messages assigned by the first classifier, (ii) the initial message category for each respective message in the second plurality of messages assigned by the second classifier and (iii) the plurality of recipient initiated message category correction events;using the correction weight for each correction type associated with a message category in the set of message categories to determine a probability or likelihood that the first message reputation carrier is associated with the first message category in the set of message categories; andwhitelisting the first message reputation carrier to the first message category when the calculated probability or likelihood satisfies a whitelisting criterion.