Social Interaction Assistant

Social Interaction Assistant

Social Interaction Assistant

August, 2011

Contact

Members

Effective communication requires a shared context. In face-to-face interactions, parts of this shared context are the number and location of people, their facial expression, head pose, eye contact, and movements of each person engaged in a conversation. Faces serve an essential function in social interactions by providing cues to attention, understanding and emotion. Faces also provide contextual information such as identity, age, family origin and personal health. In face-to-face communication, people with visual impairments are not privy to this visual information, nor are they necessarily aware of their own facial expressions. This lack of access to facial information negatively impacts employment opportunities and has a tendency to socially isolate people who are visually impaired.

The goal of the Interaction Assistant project is to provide a user who is visually impaired with information that can improve their social interactions with sighted counterparts. The system consists of a camera mounted on a pair of sun glasses which the user wears. The video from the camera is analyzed to extract information that can help the user better understand other people in their surroundings. An important goal of the system is to leverage the capabilities of the user and only provide information which is not already available by other means. The work also concentrates on developing various delivery covert mechanisms that allow the user to interpret the social information without interfering with the social interaction itself.

Funding Sources

National Science Foundation NSF Award 1116360;
CUbiC Technology and Research Initiative Fund

Members

Dr. Sethuraman "Panch" Panchanathan

Dr. Sethuraman "Panch" Panchanathan

Director, National Science Foundation

Dr. Troy L. McDaniel

Dr. Troy L. McDaniel

Assistant Professor, The Polytechnic School; Director, HAPT-X Laboratory; Director, Center for Cognitive Ubiquitous Computing (CUbiC); PI, NSF-NRT grant program, Citizen-Centered Smart Cities and Smart Living

Taruni Kancharla

Taruni Kancharla

Masters student researcher

Hemanth Venkateswara

Hemanth Venkateswara

Assistant Research Professor, School of Computing Informatics and Decision Systems Engineering; Associate Director, Center for Cognitive Ubiquitous Computing (CUbiC)

Hiranmayi Ranganathan

Hiranmayi Ranganathan

Ph.D Student Researcher

Prasanth Lade

Prasanth Lade

Ph.D. Student Researcher

Shantanu Bala

Shantanu Bala

Undergrad Student Researcher