SensAI+Expanse Emotional Valence Prediction Studieswith Cognition and Memory Integration
Nuno A. C. HenriquesBioISI
Ciências/ULisboaPortugal
Helder CoelhoBioISI
Ciências/ULisboaPortugal
Leonel Garcia-MarquesCICPSI
Psicologia/ULisboaPortugal
COGNITIVE 2020, 24–29 October, Nice, France
N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 1 / 14
Bio
Nuno A. C. Henriqueshttps://nunoachenriques.net/
PhD in Cognitive Science from ULisboaMSc in Informatics Engineering from FCT/NOVA
Chief Artificial Intelligence Officer at MettaNoonPropTech Start-up Advisor at Unlockit
Developing socially conscious opportunities tocreatively apply Sensory AI and more. Thinking asa data and information architect, engineer,scientist, and strategist towards efficient innovation.
It all started with the ZX Spectrum 48k andnever stopped from coding search engines,architecting information systems, engineeringdatabases, Cloud, Web, and mobiledevelopment integrations. Further, on roboticssoftware, GPS-based navigation, live videohuman face detection, and IoT (mobile)sensors’ data acquisition. Bridgingstate-of-the-art algorithms and techniquestowards automated machine learning,explainable, and efficient predictions incontext regarding human emotions.
N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 2 / 14
Challenge
Emotions and Human-Agent Interaction
“[...] if we want computers to be genuinely intelligentand to interact naturally with us, we must give computers
the ability to recognize, understand,even to have and express emotions [...]”1
1Picard, R. W. (1997). Affective Computing. MIT Press.N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 3 / 14
Challenge
Research Problem
Behaviour Is modified by affective states.Interaction May be subject to change or bias.Prediction When, where, and more context
may improve the dyadic bonding.
N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 4 / 14
Cognition and Memory Platform
Architecture, Data, Flow: SensAI+ExpanseArchitectureReconstructResampleAlign time
GeolocationClusters
Global grid
Wide alignSplit train/testClass balance
Auto adapt, train cross val, params, learn
Prediction model / estimator / personEmotional valence in context
Sentiment analysis
N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 5 / 14
Cognition and Memory Prediction Results
Best Model on Average2
(0.0, 0.1]
(0.1, 0.2]
(0.2, 0.3]
(0.3, 0.4]
(0.4, 0.5]
(0.5, 0.6]
(0.6, 0.7]
(0.7, 0.8]
(0.8, 0.9]
(0.9, 1.0]
Score range
0
5
10
15
20
25
Num
ber
ofen
titi
es(e
ligib
lep
opul
atio
n=31
)
Entities prediction performanceF1 score
Extreme Gradient Boosting
F1 score Prediction performed wellin most cases.Efficient energy use vs.Multi-Layer Perceptron
1/10 duration.Best F1 = 0.91.
Per class probability.Explainable.Each person provides adistinct data set.
2Henriques, N. A. C., Coelho, H., & Garcia-Marques, L. (2019). SensAI+Expanse Adaptation on Human Behaviour Towards Emotional Valence Prediction.1–6. http://arxiv.org/abs/1912.10084v4
N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 6 / 14
Cognition and Memory Prediction Results
3-Class Probabilistic Prediction: Example for Entity 24
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0mean(|SHAP value|) (average impact on model output magnitude)
mgrs_1000_33LUL2509mgrs_1000_33LUL0522mgrs_1000_33LUL2512mgrs_1000_33LUL0514mgrs_1000_33LUL0423mgrs_1000_33LUL0422mgrs_1000_33LUL0624
moment_day_quartermoment_hourmoment_dow
Feature (N=10) overall influence for entity 24
Evidence of time-related feature impact.Location competing with time in some cases.
N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 7 / 14
Cognition and Memory Prediction Results
Time and Space Competing Features: Results
Overall temporal dimension sensitivity.Most influential (prediction model):
Weekday: 64.5%Hour: 25.8%Location: 9.7%
Prediction of idiosyncratic factors.Emotional valence changes in context.Adding new features may reveal otherrelevant factors (e.g., sports).
Hand-picked sample: Entity 24
N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 8 / 14
Behaviour Study Method
Participants
< day < week < 4 weeks ≥ 4 weeksDuration range
0
5
10
15
20
25
Num
ber
ofen
titi
es
Entities retention(eligible 49/57)
by data collect duration range and gender
gender
Female
Male Age [10, 70), median 34.Females and males.33 retained (≥ 4 weeks).Africa, America, Asia,Europe.
N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 9 / 14
Behaviour Study Method
Design, Procedure, and Demographics
[10, 34) [34, 70)Age range
0
2
4
6
8
10
12
14
16
Num
ber
ofen
titi
es
Entities(eligible: 49/57)
by age range dichotomy (median=34) and gender
gender
Female
Male
Worldwide access using afree Android app.Neutral messages(age, gender).Chromatically consistent.Negative | Neutral | PositiveSensorial and non-invasiveartificial agent.
N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 10 / 14
Behaviour Study Results
Behaviour Aggregated
Monday Tuesday Wednesday Thursday Friday Saturday SundayWeekday
0
100
200
300
400
500
600
700
800
Rep
orts
Emotional valence reportsTotal by weekday(population=49)
negative
neutral
positive
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23Hour
0
50
100
150
200
250
300
Rep
orts
Emotional valence reportsTotal by hour
(population=49)
negative
neutral
positive
N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 11 / 14
Behaviour Study Results
Behaviour Differences
Female Male[10, 34)
Female Male[34, 70)
Age range and gender
0.0
0.2
0.4
0.6
0.8
1.0
Val
ence
Emotional valence reportPercentage by age range dichotomy (median=34) and gender
(population=49)
negative
neutral
positive
[10, 34) vs. [34, 70)Evidence of differences.
p = 1.161 × 10−30
[10, 34) F. vs. [34, 70) F.Older group less negative.
p = 5.539 × 10−14
[34, 70) F. vs. [34, 70) M.Female more positive.
p = 7.027 × 10−67
Mann-Whitney U, α = 0.05
N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 12 / 14
Conclusion
Summary
A novel system for studiesregarding emotional valencechanges in context.Mobile sensing agent withadaptation and learningcapabilities.Age range and gender neutral.Robust to idiosyncratic factors.Potentially free of known bias3.Open source code and openscience.
3Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2-3), 61–83.https://doi.org/10.1017/S0140525X0999152X
N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 13 / 14
Acknowledgements
Thanks
Humans Incognito participants. Advisors. Family and friends. Lab mates.Funding Universidade de Lisboa [PhD support grant between May 2016 and April 2019].
Fundação para a Ciência e Tecnologia [UIDB/04046/2020 Research Unit grantfrom FCT, Portugal (to BioISI)].
Logistics MAS/BioISI laboratory. European Grid Infrastructure (EGI) andNCG-INGRID-PT (Portugal).
N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 14 / 14
Recommended
