Click here to load reader

SensAI+Expanse Emotional Valence Prediction Studies with ... ... CognitionandMemory PredictionResults 3-ClassProbabilisticPrediction:ExampleforEntity24 0.00.51.01.52.02.53.03.54.0

  • View
    1

  • Download
    0

Embed Size (px)

Text of SensAI+Expanse Emotional Valence Prediction Studies with ... ... CognitionandMemory...

  • SensAI+Expanse Emotional Valence Prediction Studies with Cognition and Memory Integration

    Nuno A. C. Henriques BioISI

    Ciências/ULisboa Portugal

    [email protected]

    Helder Coelho BioISI

    Ciências/ULisboa Portugal

    [email protected]

    Leonel Garcia-Marques CICPSI

    Psicologia/ULisboa Portugal

    [email protected]

    COGNITIVE 2020, 24–29 October, Nice, France

    N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 1 / 14

  • Bio

    Nuno A. C. Henriques https://nunoachenriques.net/

    PhD in Cognitive Science from ULisboa MSc in Informatics Engineering from FCT/NOVA

    Chief Artificial Intelligence Officer at MettaNoon PropTech Start-up Advisor at Unlockit

    Developing socially conscious opportunities to creatively apply Sensory AI and more. Thinking as a data and information architect, engineer, scientist, and strategist towards efficient innovation.

    It all started with the ZX Spectrum 48k and never stopped from coding search engines, architecting information systems, engineering databases, Cloud, Web, and mobile development integrations. Further, on robotics software, GPS-based navigation, live video human face detection, and IoT (mobile) sensors’ data acquisition. Bridging state-of-the-art algorithms and techniques towards automated machine learning, explainable, and efficient predictions in context regarding human emotions.

    N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 2 / 14

    https://nunoachenriques.net/

  • Challenge

    Emotions and Human-Agent Interaction

    “[...] if we want computers to be genuinely intelligent and to interact naturally with us, we must give computers

    the ability to recognize, understand, even to have and express emotions [...]”1

    1Picard, R. W. (1997). Affective Computing. MIT Press. N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 3 / 14

  • Challenge

    Research Problem

    Behaviour Is modified by affective states. Interaction May be subject to change or bias. Prediction When, where, and more context

    may improve the dyadic bonding.

    N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 4 / 14

  • Cognition and Memory Platform

    Architecture, Data, Flow: SensAI+ExpanseArchitecture Reconstruct Resample Align time

    Geolocation Clusters

    Global grid

    Wide align Split train/test Class balance

    Auto adapt, train cross val, params, learn

    Prediction model / estimator / person Emotional valence in context

    Sentiment analysis

    N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 5 / 14

  • Cognition and Memory Prediction Results

    Best Model on Average2

    (0. 0,

    0.1 ]

    (0. 1,

    0.2 ]

    (0. 2,

    0.3 ]

    (0. 3,

    0.4 ]

    (0. 4,

    0.5 ]

    (0. 5,

    0.6 ]

    (0. 6,

    0.7 ]

    (0. 7,

    0.8 ]

    (0. 8,

    0.9 ]

    (0. 9,

    1.0 ]

    Score range

    0

    5

    10

    15

    20

    25

    N um

    b er

    of en

    ti ti

    es (e

    lig ib

    le p

    op ul

    at io

    n= 31

    )

    Entities prediction performance F1 score

    Extreme Gradient Boosting

    F1 score Prediction performed well in most cases. Efficient energy use vs. Multi-Layer Perceptron

    1/10 duration. Best F1 = 0.91.

    Per class probability. Explainable. Each person provides a distinct data set.

    2Henriques, N. A. C., Coelho, H., & Garcia-Marques, L. (2019). SensAI+Expanse Adaptation on Human Behaviour Towards Emotional Valence Prediction. 1–6. http://arxiv.org/abs/1912.10084v4

    N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 6 / 14

    http://arxiv.org/abs/1912.10084v4

  • Cognition and Memory Prediction Results

    3-Class Probabilistic Prediction: Example for Entity 24

    0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 mean(|SHAP value|) (average impact on model output magnitude)

    mgrs_1000_33LUL2509 mgrs_1000_33LUL0522 mgrs_1000_33LUL2512 mgrs_1000_33LUL0514 mgrs_1000_33LUL0423 mgrs_1000_33LUL0422 mgrs_1000_33LUL0624

    moment_day_quarter moment_hour moment_dow

    Feature (N=10) overall influence for entity 24

    Evidence of time-related feature impact. Location competing with time in some cases.

    N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 7 / 14

  • Cognition and Memory Prediction Results

    Time and Space Competing Features: Results

    Overall temporal dimension sensitivity. Most influential (prediction model):

    Weekday: 64.5% Hour: 25.8% Location: 9.7%

    Prediction of idiosyncratic factors. Emotional valence changes in context. Adding new features may reveal other relevant factors (e.g., sports).

    Hand-picked sample: Entity 24

    N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 8 / 14

  • Behaviour Study Method

    Participants

    < day < week < 4 weeks ≥ 4 weeks Duration range

    0

    5

    10

    15

    20

    25

    N um

    b er

    of en

    ti ti

    es

    Entities retention (eligible 49/57)

    by data collect duration range and gender

    gender

    Female

    Male Age [10, 70), median 34. Females and males. 33 retained (≥ 4 weeks). Africa, America, Asia, Europe.

    N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 9 / 14

  • Behaviour Study Method

    Design, Procedure, and Demographics

    [10, 34) [34, 70) Age range

    0

    2

    4

    6

    8

    10

    12

    14

    16

    N um

    b er

    of en

    ti ti

    es

    Entities (eligible: 49/57)

    by age range dichotomy (median=34) and gender

    gender

    Female

    Male

    Worldwide access using a free Android app. Neutral messages (age, gender). Chromatically consistent. Negative | Neutral | Positive Sensorial and non-invasive artificial agent.

    N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 10 / 14

  • Behaviour Study Results

    Behaviour Aggregated

    Monday Tuesday Wednesday Thursday Friday Saturday Sunday Weekday

    0

    100

    200

    300

    400

    500

    600

    700

    800

    R ep

    or ts

    Emotional valence reports Total by weekday (population=49)

    negative

    neutral

    positive

    0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Hour

    0

    50

    100

    150

    200

    250

    300

    R ep

    or ts

    Emotional valence reports Total by hour

    (population=49)

    negative

    neutral

    positive

    N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 11 / 14

  • Behaviour Study Results

    Behaviour Differences

    Female Male [10, 34)

    Female Male [34, 70)

    Age range and gender

    0.0

    0.2

    0.4

    0.6

    0.8

    1.0

    V al

    en ce

    Emotional valence report Percentage by age range dichotomy (median=34) and gender

    (population=49)

    negative

    neutral

    positive

    [10, 34) vs. [34, 70) Evidence of differences.

    p = 1.161 × 10−30

    [10, 34) F. vs. [34, 70) F. Older group less negative.

    p = 5.539 × 10−14

    [34, 70) F. vs. [34, 70) M. Female more positive.

    p = 7.027 × 10−67

    Mann-Whitney U, α = 0.05

    N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 12 / 14

  • Conclusion

    Summary

    A novel system for studies regarding emotional valence changes in context. Mobile sensing agent with adaptation and learning capabilities. Age range and gender neutral. Robust to idiosyncratic factors. Potentially free of known bias3. Open source code and open science.

    3Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2-3), 61–83. https://doi.org/10.1017/S0140525X0999152X

    N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 13 / 14

    https://doi.org/10.1017/S0140525X0999152X

  • Acknowledgements

    Thanks

    Humans Incognito participants. Advisors. Family and friends. Lab mates. Funding Universidade de Lisboa [PhD support grant between May 2016 and April 2019].

    Fundação para a Ciência e Tecnologia [UIDB/04046/2020 Research Unit grant from FCT, Portugal (to BioISI)].

    Logistics MAS/BioISI laboratory. European Grid Infrastructure (EGI) and NCG-INGRID-PT (Portugal).

    N.A.C.Henriques (ULisboa) SensAI+Expanse Cognition Towards Prediction COGNITIVE 2020 14 / 14

    Challenge Cognition and Memory Platform Prediction Results

    Behaviour Study Method Results

    Conclusion Acknowledgements

Search related