Interactive Inference of Mental States

Understanding a child’s social emotional state throughout an educational activity is crucial for a positive learning outcome.  As such, we develop machine learning models that help social robots understand the cognitive states of children.  Current state-of-the art approaches design inference models as passive processes of observing the nonverbal expressions of people. But these approaches are missing how those expressions are embedded in a greater interaction context. Our partially observable Markov decision process (POMDP) model captures how the meaning of an expression changes based on the cue context. This allows us to more accurately estimate a listener’s level of attention and engagement in a peer-to-peer storytelling interaction.  We model social robots  as having an active role in the inference process  by generating social cues that elicit nonverbal responses from children to better understand their underlying cognitive state.

Cognitive inference is an interactive process!