Adaptive Multimodal Communication Assist System

摘要

A computer implemented method and system for assisting a user to learn and/or communicate in a visual communication language in one or more modes is provided. The multimodal communication assist application, provided on a user's computing device, determines the user's characteristic information based on one or more selected multimodal communication mappers. The multimodal communication assist application determines a delay factor based on the characteristic information. The multimodal communication assist application captures a modal input in one of the modes from the user via an interactive interface based on the delay factor and the characteristic information. The multimodal communication assist application processes and transforms the captured modal input in one of the modes into a modal output in another one or more of the modes and renders the modal output to the user via the interactive interface. The multimodal communication assist application generates learning components and testing components for the user.