MAScreen: Augmenting Speech with Visual Cues of Lip Motions, Facial Expressions, and Text Using a Wearable Display (2020)

Personal protective equipment, particularly face masks, have be- come increasingly common with the rise of global health issues, such as fine-dust storms and pandemics. Face masks, however, also degrade speech intelligibility by effectively occluding visual cues, such as lip motions and facial expressions. In this paper, we propose MAScreen, a wearable LED display in the shape of a mask, which is capable of sensing lip motion and speech and provides real-time visual feedback of the mouth behind the mask.

References

Hyein Lee, Yoonji Kim, and Andrea Bianchi. 2020. MAScreen: Augmenting Speech with Visual Cues of Lip Motions, Facial Expressions, and Text Using a Wearable Display. To appear in SIGGRAPH Asia Emerging Technologies 2020.

paper    doi


Back to projects