Multimodal deception detection using real-life trial data
Şen, Mehmet Umut and Perez-Rosas, Veronica and Yanıkoğlu, Berrin and Abouelenien, Mohamed and Burzo, Mihai and Mihalcea, Rada (2020) Multimodal deception detection using real-life trial data. IEEE Transactions on Affective Computing . ISSN 1949-3045 Published Online First http://dx.doi.org/10.1109/TAFFC.2020.3015684
Full text not available from this repository.
Official URL: http://dx.doi.org/10.1109/TAFFC.2020.3015684
Hearings of witnesses and defendants play a crucial role when reaching court trial decisions. Given the high-stakes nature of trial outcomes, developing computational models that assist the decision-making process is an important research venue. In this paper, we address the identification of deception in real-life trial data. We use a dataset consisting of videos collected from public court trials. We explore the use of verbal and non-verbal modalities to build a multimodal deception detection system that aims to discriminate between truthful and deceptive statements provided by defendants and witnesses. In particular, three complementary modalities (visual, acoustic and linguistic) are evaluated for the classification of deception at the subject level. The final classifier is obtained by combining the three modalities via score-level classification, achieving 83.05% accuracy in subject-level deceit detection. To place our results in perspective, we present a human deception detection study where we evaluate the human capability of detecting deception using different modalities and compare the results to the developed system. The results show that our system outperforms the average non-expert human capability of identifying deceit.
Repository Staff Only: item control page