Using biometric data to detect the emotional state of social media user as a first step towards early anxiety detection and intervention
PDF

Keywords

machine learning
social media
medical devices

Abstract

Intense emotions often suppress reason, leading to potentially impulsive or harmful behaviors, such as sending negative messages to friends when  not in a favorable mental state or causing physical or emotional harm. Since intense emotions are connected to physiological responses, this project aims to develop a software solution that utilizes AI image recognition to help prevent or reduce harmful behaviors by constantly measuring three physiological signals--heart rate, galvanic skin response, and skin temperature--during social media use, and providing alerts and suggestions to the user when a negative emotion is sensed. The target market for the AI is teenagers, as they are more prone to displaying impulsive behaviors. Using data acquired while watching movies, surfing social media, viewing short internet videos, various studies have proven the connections between the physiological data and emotional states. This project will use AI image recognition routines to make the connections between physiological data and emotional states.

PDF

References

Dzedzickis, A., Kaklauskas A., Bucinskas V. (2020). Human Emotion Recognition: Review of Sensors and Methods. NCBI. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7037130/

Gatti E., Calzolari E., Maggioni E., Obrist M. Emotional ratings and skin conductance response to visual, auditory and haptic stimuli. Sci. Data. 2018;5:180120. doi: 10.1038/sdata.2018.120.

Jeremy et al. (2020). Fastai. GitHub. https://github.com/fastai/fastai

Koelstra, S., Muhl, C., Soleymani, M., Jong-Seok Lee, Yazdani, A., Ebrahimi, T., … Patras, I. (2012). DEAP: A Database for Emotion Analysis ;Using Physiological Signals. IEEE Transactions on Affective Computing, 3(1), 18–31. doi:10.1109/t-affc.2011.15

Langereis, G., (2010). Photoplethysmography (PPG) system. Seeedstudio. https://github.com/SeeedDocument/Grove-Ear-clip_Heart_Rate_Sensor/raw/master/res/PPG%20Sensor%20System.pdf

Lisetti, C.L., Nasoz, F. Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals. EURASIP J. Adv. Signal Process. 2004, 929414 (2004). https://doi.org/10.1155/S1110865704406192

Tamura T., Maeda Y., Sekine M., Yoshida M. Wearable Photoplethysmographic Sensors—Past and Present. Electronics. 2014;3:282–302. doi: 10.3390/electronics3020282.

tape software. (2020, September 8). Cross-platform inference using fast.ai models. Cross-platform inference using fast.ai models - tape software. https://tapesoftware.net/fastai-onnx/

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Copyright (c) 2021 Zoey, Emily Sam