COLLABORATIVE CREATION OF ANNOTATION TRAINING DATA

摘要

Collaborative annotation of a document. A GUI is generated by a host computer that displays a document to annotate. The GUI is transmitted by the host computer to a plurality of client computers. Real time communication modes are established by the host computer between the client computers, each including communication channels between all the client computers. The host computer receives annotations entered into the GUIs at the client computers, each annotation identifying a span of words in the document to annotate and including an associated annotation type. The host computer updates the GUI to include for display the received annotations and inter-rater agreement statistics indicating the level of agreement between annotations received from each client computer and annotations received from the other client computers. The updated GUI is then transmitted by the host computer to the plurality of client computers.

主权项

1. A method for collaborative annotation of a document, the method comprising:
generating, by a host computer, a graphical user interface (GUI) that displays a document to annotate; transmitting, by the host computer to a plurality of client computers, the GUI; establishing, by the host computer, a plurality of real time communication modes between the client computers, each communication mode including communication channels between all client computers; receiving, by the host computer from the plurality of client computers, a plurality of annotations entered into the GUIs at the client computers, each annotation identifying a span of words in the document to annotate and including an associated annotation type; updating, by the host computer, the GUI to include for display the received plurality of annotations and inter-rater agreement statistics indicating the level of agreement between annotations received from each client computer and annotations received from the other client computers; and transmitting, by the host computer to the plurality of client computers, the updated GUI.