title
  

Multi-modal deception detection from videos

Warning The system is temporarily closed to updates for reporting purpose.

Şen, Mehmet Umut (2020) Multi-modal deception detection from videos. [Thesis]

[img]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
1146Kb

Official URL: http://risc01.sabanciuniv.edu/record=b2486381 (Table of contents)

Abstract

Hearings of witnesses and defendants play a crucial role when reaching court trial decisions. Given the high-stakes nature of trial outcomes, developing computational models that assist the decision-making process is an important research venue. In this thesis, we address the deception detection in real-life trial videos. Using a dataset consisting of videos collected from concluded public court trials, we explore the use of verbal and non-verbal modalities to build a multimodal deception detection system that aims to classify the defendant in a given video as deceptive or not. Three complementary modalities (visual, acoustic and linguistic) are evaluated separately for the classification of deception. The final classifier is obtained by combining the three modalities via score-level classification, achieving 83.05% accuracy. Multimodal analysis of trial videos involves many challenges. Prior to developing the final deception detection system, we have worked on sub-problems that would be helpful on improving deception detection performance. High volume of background sounds in a video decreases the quality of the speech features, and it results in low speech recognition performance. We developed a neural network based single-channel source separation model to extricate the speech from the mixed sound recording. Word embeddings, is the state-of-art technique in processing of textual data. In addition to evaluating pretrained word embeddings in developing the deception system for English, we have also worked on learning word embeddings for Turkish and used them for categorizing text documents. This work can be applied in future for a deception system in Turkish

Item Type:Thesis
Uncontrolled Keywords:deception detection. -- multi-modal. -- word embeddings. -- document classification. -- speech source separation . -- aldatmaca kestirimi. -- çoklu-modalite. -- kelime temsilleri. -- doküman sınıflandırma.-- konuşma kaynak ayırımı.
Subjects:T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK7800-8360 Electronics
ID Code:41204
Deposited By:IC-Cataloging
Deposited On:01 Nov 2020 14:42
Last Modified:01 Nov 2020 14:42

Repository Staff Only: item control page