Shaun Canavan, PhD

Shaun Canavan, PhD

Associate Professor, Computer Science & Engineering

Posts by Collection

affective

Impact of Culture on Affect

,

Abstract

It has been shown that emotions are learned in a cultural way and expressions are often used to help convey these emotional states. Considering this, in this work, we investigate multimodal cultural behavior differences across 6 different cultures. More specifically, we investigate head pose, action units, and facial landmarks in British, Chinese, German, Greek, Hungarian, and Serbian cultures. Along with this, we also investigate the differences along valence and arousal dimensions for these cultures. To conduct this investigation, we evaluate the SEWA multimodal and multi-cultural dataset. We find varying differences exist that are impacted by culture, context, and modality. Based on these findings, we also perform context classification that takes into account these differences in culture. We show that incorporating culture into our pipeline improves classification performance.

Expression Recognition Across Age

,

Abstract

Expression recognition is an important and growing field in AI. It has applications in fields including, but not limited to, medicine, security, and entertainment. A large portion of research, in this area, has focused on recognizing expressions of young and middle-age adults with less focus on children and elderly subjects. This focus can lead to unintentional bias across age, resulting in less accurate models. Considering this, we investigate the impact of age on expression recognition. To facilitate this investigation, we evaluate two state-of-the-art datasets, that focus on different age ranges (children and elderly), namely EmoReact and ElderReact. We propose a Siamese-network based approach that learns the semantic similarity of expressions relative to each age. We show that the proposed approach, to expression recognition, is able to generalize across age. We show the proposed approach is comparable to or outperforms current state-of-the-art on the EmoReact and ElderReact datasets.

Gaze-based Classification of Autism Spectrum Disorder

,

Abstract

People with autism spectrum disorder (ASD) display impairments in social interaction and communication skills, as well as restricted interests and repetitive behaviors, which greatly affect daily life functioning. Current identification of ASD involves a lengthy process that requires an experienced clinician to assess multiple domains of functioning. Considering this, we propose a method for classifying multiple levels of risk of ASD using eye gaze and demographic feature descriptors such as a subject’s age and gennder. We construct feature descriptors that incorporate the subject’s age and gender, as well as features based on eye gaze patterns. We also present an analysis of eye gaze patterns validating the use of the selected hand-crafted features. We assess the efficacy of our descriptors to classify ASD on a National Database for Autism Research dataset, using multiple classifiers including a random forest, C4.5 decision tree, PART, and a deep feedforward neural network.

Latent Space Representation of Emotion

,

Abstract

Facial expression Recognition is a growing and important field that has applications in fields such as medicine, security, education, and entertainment. While there have been encouraging approaches that have shown accurate results on a wide variety of datasets, in many cases it is still a difficult problem to explain the results. Considering this, in this paper, we propose to model flow-based latent representations of facial expressions, which allows us to further analyze the features and grants us more granular control over which features are produced for recognition. Our work is focused on posed facial expressions with a tractable density of the latent space. We investigate the behaviour of these tractable latent space features in the case of subject dependent and independent expression recognition. We employ a flow-based generative approach with minimal supervision introduced during training and observe that traditional metrics give encouraging results. When subject independent expressions are evaluated, a shift towards a stochastic nature, in the probability space, is observed. We evaluate our flow-based representation on the BU-EEG dataset showing our approach provides good separation of classes, resulting in more explainable results.

Quantified Facial Affect Analysis

,

Abstract

The quantification of visual affect data (e.g. face images) is essential to build and monitor automated affect modeling systems efficiently. Considering this, this work proposes quantified facial Temporal-expressiveness Dynamics (TED) to quantify the expressiveness of human faces. The proposed algorithm leverages multimodal facial features by incorporating static and dynamic information to enable accurate measurements of facial expressiveness. We show that TED can be used for high-level tasks such as summarization of unstructured visual data, and expectation from and interpretation of automated affect recognition models. To evaluate the positive impact of using TED, a case study was conducted on spontaneous pain using the UNBC-McMaster spontaneous shoulder pain dataset. Experimental results show the efficacy of using TED for quantified affect analysis.

asd

biometrics

Continuous Authentication

, University of South Florida

Research into age-aware continuous authentication. This work was supported by the NSF.

USF CA Dataset

, University of South Florida

Video and physiologial signals for continuous authentication

datasets

hci

Gesture Recognition

, University of South Florida

Gesture Recognition for sign language recognition.

news

ACII Paper Accepted Permalink

News, Affective Computing and Intelligent Interaction, 2024

Multimodal Behavior Analysis and Impact of Culture on Affect.

ECCV Paper Accepted Permalink

News, European Conference on Computer Vision

FlowCon: Out-of-Distribution Detection using Flow-Based Contrastive Learning paper accepted to ECCV 2024.

research

Biometrics Permalink

Research, University of South Florida

Research into subject identification and age-aware continuous user authentication.

sponsors

students

Saandeep Aathreya Permalink

PhD Student, University of South Florida

PhD student investigating latent space representations of emotion.

Rupal Agarwal Permalink

PhD Student, University of South Florida

PhD student investigating ASD screening in children.

Anis Elebiary Permalink

PhD Student, University of South Florida

PhD student investigating bias in expression recognition.

Liza Jivnani Permalink

Undergraduate Student, University of South Florida

Undergraduate student investigating human behavior during stressful situations.

teaching

COT 6405 - Introduction to Theory of Algorithms

Graduate course, University of South Florida, Fall, 2024

Analysis techniques for algorithms. Characterizing algorithms in terms of recurrence relations, solutions of recurrence relations, upper and lower bounds. Graph problems, parallel, algorithms, NP completeness and approximation algorithms, with relationship to practical problems.

CAP 4687 & CAI 5627

Undergraduate and graduate course, University of South Florida, Spring, 2025

The study of systems that can recognize, interpret, process, and simulate human affect. Topics may include, but are not limited to, physiology of emotion, lie detection, wearable devices, music, and ethical concerns associated with affective computing.