GeSTICS
A Multimodal Corpus for Studying Gesture Synthesis in Two-party Interactions with Contextualized Speech
IVA 2024
- Gaoussou Youssouf Kebe1
- Mehmet Deniz Birlikci1
- Auriane Boudin2
- Ryo Ishii3
- Jeffrey M. Girard4
- Louis-Philippe Morency1
1Carnegie Mellon University 2Aix Marseille Univ, CNRS, LPL, LIS 3NTT Corporation 4University of Kansas
Dataset Overview
We present GeSTICS, a novel multimodal corpus designed to facilitate the study of gesture synthesis in two-party interactions with contextualized speech. GeSTICS comprises audiovisual recordings of post-game sports interviews, capturing both verbal and non-verbal communication aspects.
Features
Speech Features
- Whisper Transcriptions
- eGeMAPS Acoustic Features (88)
- LIWC (93) & VADER Lexical Features
Visual Features
- Body Pose Landmarks (33 points)
- Hand Keypoints (21 per hand)
- Face Landmarks (468) & AUs (52)
Contextual Metadata
Situational Factors
- Game Score Difference
- Game Quality Index
- Game Importance
- Team Strength Difference
Individual Factors
- Race
- Origin
- Team Role
- Age
Download
The GeSTICS dataset will be available for download soon. Please check back later or contact us for more information.
Applications
The GeSTICS dataset is designed to enhance the generation of realistic nonverbal behaviors in:
Virtual Agents
Improve the naturalness of virtual agent interactions in various applications.
Animated Characters
Enhance the realism of animated characters in games and interactive media.
Human-Robot Interaction
Develop more natural and intuitive interactions between humans and robots.
Bibtex
@inproceedings{kebe2024gestics, title={GeSTICS: A Multimodal Corpus for Studying Gesture Synthesis in Two-party Interactions with Contextualized Speech}, author={Kebe, Gaoussou Youssouf and Birlikci, Mehmet Deniz and Boudin, Auriane and Ishii, Ryo and Girard, Jeffrey M. and Morency, Louis-Philippe}, booktitle={ACM International Conference on Intelligent Virtual Agents (IVA '24)}, year={2024}, address={GLASGOW, United Kingdom}, month={September}, publisher={ACM}, doi={10.1145/3652988.3673917}, isbn={979-8-4007-0625-7/24/09} }
License
For full license details, please contact the authors.