Şen, Mehmet Umut and Perez-Rosas, Veronica and Yanıkoğlu, Berrin and Abouelenien, Mohamed and Burzo, Mihai and Mihalcea, Rada (2020) Multimodal deception detection using real-life trial data. IEEE Transactions on Affective Computing . ISSN 1949-3045 Published Online First http://dx.doi.org/10.1109/TAFFC.2020.3015684
![Warning](/style/images/warning.png)
There is a more recent version of this item available.
Official URL: http://dx.doi.org/10.1109/TAFFC.2020.3015684
Abstract
Hearings of witnesses and defendants play a crucial role when reaching court trial decisions. Given the high-stakes nature of trial outcomes, developing computational models that assist the decision-making process is an important research venue. In this paper, we address the identification of deception in real-life trial data. We use a dataset consisting of videos collected from public court trials. We explore the use of verbal and non-verbal modalities to build a multimodal deception detection system that aims to discriminate between truthful and deceptive statements provided by defendants and witnesses. In particular, three complementary modalities (visual, acoustic and linguistic) are evaluated for the classification of deception at the subject level. The final classifier is obtained by combining the three modalities via score-level classification, achieving 83.05% accuracy in subject-level deceit detection. To place our results in perspective, we present a human deception detection study where we evaluate the human capability of detecting deception using different modalities and compare the results to the developed system. The results show that our system outperforms the average non-expert human capability of identifying deceit.
Item Type: | Article |
---|---|
Divisions: | Faculty of Engineering and Natural Sciences > Academic programs > Computer Science & Eng. Faculty of Engineering and Natural Sciences |
Depositing User: | Berrin Yanıkoğlu |
Date Deposited: | 03 Sep 2021 16:18 |
Last Modified: | 03 Sep 2021 16:18 |
URI: | https://research.sabanciuniv.edu/id/eprint/42388 |
Available Versions of this Item
- Multimodal deception detection using real-life trial data. (deposited 03 Sep 2021 16:18) [Currently Displayed]