The future where nonverbal information opens the door to rich and natural communication

The infrastructure of “telecommunication with nonverbal information” connects various people at a distance. If a person who wants to express her/his joy with his/her body, s/he may not be able to do so due to physical reasons. In such cases, an avatar or a robot can express her/his joy on his/her behalf. Even if a person is unable to understand another person's emotional expression due to his/her inability to see or hear, the robot might be able to recognize it using sensors equipped on the robot and convert it into information that the person can understand and convey to him/her. Such technologies will help to realize an accessible and inclusive society that accommodates diversity in terms of disabilities, culture, gender, etc., and we hope to contribute to this goal. In order to achieve this, however, this requires interdisciplinary research over a certain long period of time that brings together knowledge from a variety of fields.

A model of telecommunication

To enrich telecommunication

Telecommunication can be modeled as a person in a real space communicating with a person far away in a real space using telecommunication technology. However, the current telecommunication system lacks important information, so it cannot be said to provide rich communication. Therefore, we are trying to realize rich and natural communication by realizing a system that can successfully transmit subtle nonverbal information.

Nonverbal Information

Very wide range of nonverbal information

Nonverbal information refers to all information other than verbal information (spoken and written). In addition to verbal information exchanged superficially in communication, there are various other types of nonverbal information exchanged, such as facial expressions, body movements, eye contact, clothing, interpersonal distance, and the atmosphere of a place. It is reported that more than 90% of the information in emotional communication is conveyed not by words but by nonverbal information, and it is known that nonverbal information plays a major role in communication.
Nonverbal information is very diverse, ranging from those directly related to a person’s emotions to those dependent on social relationships, and from those that can be easily conveyed to some extent by turning on a camera to those for which a means of successful communication has not yet been established.