A dactylology telephone system having a dactylology function, through which an auditorily impaired person can talk by dactylology with a non-auditorily-impaired person who does not know dactylology. A dactylology telephone set is provided on the auditorily impaired person side and the movement of the fingers inputted through a dactylology input means is recognized as dactylology and translated into the Japanese sentences. The Japanese character strings are converted into synthesized voices and sent to the video telephone set on the non-handicapped person side through a network. The voices inputted through the video telephone set are recognized and are translated into dactylology and a dactylology animation is generated and displayed on the TV screen of the dactylology telephone set. Therefore, an aurally handicapped person using dactylology can talk with a non-handicapped person who is at a remote place and does not know dactylology through an existing network.