Providing an annotation linking related entities in onscreen content

摘要

Systems and methods are provided for providing insight for entities in mobile onscreen content. For example, a method includes receiving, from a mobile device, an indication of selection of a first entity represented by a visual cue in first annotation data for a screen capture image of a screen of the mobile device and determining entities related to the first entity in a graph-based data store. The method may also include identifying a second entity in the screen capture image that is one of the entities related to the first entity, generating second annotation data, the second annotation data including a visual element linking the first entity and the second entity, and providing the second annotation data for display with the screen on the mobile device.

主权项

1. A system comprising:
a graph-based data store of entities connected by edges; at least one processor; and memory storing instructions that, when executed by the at least one processor, cause the system to:
receive a selection of a first entity represented by a visual cue in first annotation data for a screen capture image of a screen of a mobile device;determine entities related to the first entity in the graph-based data store;identify a second entity in the screen capture image that is one of the entities related to the first entity;generate second annotation data, the second annotation data including a visual element linking the first entity and the second entity; andprovide the second annotation data for display with the screen on the mobile device.