1
To Appear in the Journal of Applied Artificial Intelligence
Probabilistic Assessment of User’s Emotions in Educational Games
Cristina Conati
Department of Computer Science
University of British Columbia
Vancouver, BC V6T1Z4
conati@cs.ubc.ca
, +604-8224632
Abstract
We present a probabilistic model to monitor a user’s emotions and engagement during the
interaction with educational games. We illustrate how our probabilistic model assesses affect by
integrating evidence on both possible causes of the user’s emotional arousal (i.e., the state of the
interaction) and its effects (i.e., bodily expressions that are known to be influenced by emotional
reactions). The probabilistic model relies on a Dynamic Decision Network to leverage any indirect
evidence on the user’s emotional state, in order to estimate this state and any other related variable
in the model. This is crucial in a modeling task in which the available evidence usually varies with
the user and with each particular interaction. The probabilistic model we present is to be used by
decision theoretic pedagogical agents to generate interventions aimed at achieving the best tradeoff
between a user’s learning and engagement during the interaction with educational games.
2
1. Introduction
In recent years, there has been an increasing interest in studying how to make computers
more “sociable” by enabling them to both display their own emotions and react to the
user’s emotions. Building computers that display emotions in a natural and meaningful
way is already a challenging endeavor, since it requires formalizing concepts and
mechanisms that are often still under investigation in emotional psychology. But building
computers that recognize a user’s emotions is even more challenging, as is proven by the
fact that even human beings are not always proficient in this task. The challenge is due to
the high level of ambiguity that exists in the mapping between emotional states and the
factors that can be used to detect them. For instance, different people can have different
emotional reactions to the same stimulus, and the variability depends upon traits that are
not always easily observable, such as a person’s goals, preferences, expectations and
personality. Emotions can be recognized because they often have observable effects on a
user’s behavior and bodily expressions. But the mapping between emotions and their
observable effects also depends on often hidden traits of a person, as well as on the
context of the interaction. Furthermore, observable effects of emotions are not always
easily recognizable by a computer (i.e., subtle changes in facial expression and intonation).
Existing approaches have tackled the challenge of recognizing user’s affect by trying to
reduce the ambiguity in the modeling task. This has been achieved either by focusing on
recognizing a specific emotion in a fairly constraining interaction (Healy and Picard, 2000;
Hudlicka and McNeese, 2002) or by assessing only lower level dimensions of emotional
reaction, such as its intensity and valence
1
(Ball and Breeze, 2000).
In this paper, we present an approach to modeling user affect designed to assess a variety
of emotional states during interactions in which knowing the details of a user’s emotional
reaction can enhance a system capability to interact with the user effectively. Instead of
reducing the uncertainty in emotion recognition by constraining the task and the
granularity of the model, our approach explicitly encodes and processes this uncertainty
by relying on probabilistic reasoning. In particular, we use Dynamic Decision Networks
(Dean and Kanazawa, 1989; Russell and Norvig, 1995) to represent in a unifying
framework the probabilistic dependencies between possible causes and emotional states
(including the temporal evolution of these states), and between emotional states and the
user’s bodily expressions they can affect. Our goal is to create a model of user affect that
can generate as accurate an assessment as possible, by leveraging any existing information
on the user’s emotional state, but that can also explicitly express the uncertainty of its
predictions when little or ambiguous information is available.
We discuss our model in the context of the interaction with pedagogical agents designed to
improve the effectiveness of computer-based educational games (which we will simply call
educational games throughout the paper). In the rest of the paper, we first describe why
detecting emotions is important for educational games. We then introduce Dynamic
Decision Networks (DDN) and illustrate how they can be used to enable pedagogical
agents for educational games to generate interactions tailored to both the user’s learning
and emotional state. Next, we describe in detail the DDN underlying our model of user
1
Valence measures whether the emotion generated a positive or negative feeling
3
affect and how it integrates in a principled way different sources of ambiguous
information on the user’s emotional state. We end with an overview of related work,
discussion and conclusions.
2. Emotionally Intelligent Agents for Educational Games
Several authors have suggested the potential of video and computer games as educational
tools (e.g., Silvern, 1986; Malone and Lepper, 1987). However, empirical studies have
shown that, while educational games are usually highly engaging, they often do not trigger
the constructive reasoning necessary for learning (Conati and Fain Lehman, 1993; Klawe,
1998).
An explanation of these findings is that it is often possible to learn how to play an
educational game effectively without necessarily reasoning about the target domain
knowledge (Conati and Fain Lehman, 1993). Possibly, for many students the high level of
engagement triggered by the game activities acts as a distraction from reflective cognition.
This seems to happen especially when the game is not integrated with external activities
that help ground the game experience into the learning one. Also, educational games are
usually highly exploratory in nature, and empirical studies on exploratory learning
environments have shown that these environments tend to be effective only for those
students that already possess the learning skills necessary to benefit from autonomous
exploration (e.g., Shute, 1993).
To overcome the limitations of educational games, we are working on designing intelligent
pedagogical agents that, as part of game playing, can generate tailored interventions aimed
at stimulating a student’s reasoning if they detect that the student is failing to learn from
the game. “As part of game playing” is the key point in the design of these agents. The
main advantage of educational games versus more traditional computer-based tutors is that
the former tend to generate a much higher level of students’ positive emotional
engagement, thus making the learning experience more motivating and appealing. In order
not to lose this advantage, it is crucial that the interventions of pedagogical agents be
consistent with the spirit of the game and consider the players’ emotional state, in addition
to their learning. On the one hand, these pedagogical agents need to make sure that a
student learns as much as possible from the game. On the other hand, they also need to
avoid interventions that make the student start seeing the interaction with the game more as
an educational chore than as a fun activity. Thus, at any point during the player interaction
with the game, a pedagogical agent may need to consider the tradeoff between the
player’s learning and entertainment when deciding how to act. The more information the
agent has on the student’s learning and emotional state, the more focused and effective its
actions can be. We formalize this behavior by designing our pedagogical agents as decision
theoretic agents (Howard and Matheson, 1977; Russell and Norvig, 1995) that select
actions so as to maximize the outcome in terms of a student’s learning and emotional
engagement, as we describe in the next section.
3. Decision-theoretic Pedagogical Agents
In a decision-theoretic model (Howard and Matheson, 1977), an agent’s preferences over
world states S are expressed by a utility function U(S), which assigns a single number to
4
express the desirability of a state. Furthermore, for each action a available to the agent,
and for each possible outcome state S’ of that action, P(S’|E, a) represents the agent’s
belief that action a will result in state S’, when the action is performed in a state identified
by evidence E. The expected utility of an action a is then computed as
Learning
S
t
i
Emotional
State
Student Traits
U(
S
)
t
i
Bodily
Expressions
Student
Action
t
i
+1
Agent
Action
Sensors
Learning
S
t
i
Emotional
State
Student Traits
Bodily
Expressions
Sensors
Figure 1: DDN to model the decision of a pedagogical agent
EU(A) = Σ
S’
P(S’|E, a)U(S’)
A decision-theoretic agent selects the action that maximizes this value when deciding how
to act.
Decision Networks (DNs), or influence diagrams (Henrion, Breeze and Horvitz, 1991), are
an extension of Bayesian Networks (Pearl, 1988) that allow modeling decision-theoretic
behavior. In addition to nodes representing probabilistic events in the world, a DN includes
nodes representing an agent’s decision points and utilities. By relying on propagation
algorithms for Bayesian networks, DNs allow computing the agent’s action (or sequence
of actions) with maximum expected utility given the available evidence on the current state
of the world.
Dynamic Decision Networks (DDNs) add to DNs the capability of modeling environments
that change over time. Figure 1 shows how a DDN can be used to define the behavior of
pedagogical agents that take into account both the student’s learning and emotional
reactions when deciding how to act. This DDN models behavior over two time slices, to
answer the question: given the student’s state S
ti
at time t
i
, what is the agent’s action that
will maximize the agent’s expected utility at time t
i+1
, defined in terms of the student’s
learning and emotional state at that time?
In a DDN, the links between variables in different time slices indicate that the values of
these variables evolve over time and that the value at time t
i
influences the value at time
5
t
i+1
. In Figure 1, this is the case for the random variables Learning and Emotional State
representing a student’s learning and emotional state, respectively. The links between
Learning nodes, for example, model the fact that a student is likely to know a given
concept at time t
i+1
if she knew it at time t
i
. The links between Emotional State nodes
encode that a student is more likely to feel a given emotion at time t
i+1
if something that
can trigger that emotion happens and the student was already feeling that emotion at time
t
i
. The shaded nodes in Figure 1 represent random variables for which evidence is
available to update the student model at a given time slice. In Figure 1, this evidence
includes the student’s game action at time t
i
, as well as the output of sensors for
monitoring the student’s affective response at time t
i
and
t
i+1
(we will say more about these
sensors in a later section). The rectangular node in time slice t
i+1
represents the agent’s
available actions at that time, while the hexagonal node represents the agent’s utility. To
compute the agent’s action with highest expected utility in this time slice, the DDN
computes the expected value of each action given the evidence currently available at time
slice t
i
. The agent’s decision node is then set to the action with the highest expected utility,
and new evidence on the student’s emotional reactions in collected to assess what
emotional state the agent’s action actually generated.
The links from the Learning and Emotional State nodes to the utility node in Figure 1
indicate that an agent’s utility function is defined over the student’s learning and emotional
states. By varying this utility function, we can define agents that play different roles in the
game. So, for instance, the utility function of a tutoring-oriented agent will assign higher
values to states characterized by high levels of student learning, giving less importance to
the student’s emotional engagement. In contrast, the utility function of a game-oriented
agent will value more those states in which the student is positively engaged.
In the rest of the paper, we will concentrate on illustrating the part of the DDN that
assesses the user’s emotional state, to show how a probabilistic model can deal with the
high level of uncertainty involved in this still largely unexplored user modeling task. For
simplicity, we will ignore any relation between emotional state and learning, as well as
details on how assessment of learning is performed.
4. A Dynamic Decision Network for Modeling Affect
Figure 2 shows two time slices of the DDN that forms our model of student affect. The
nodes in Figure 2 represent classes of variables in the actual DDN. As the figure shows, the
network includes variables that represent both causes and effects of emotional reactions.
Being able to combine evidence on both causes and effects aims to compensate for the fact
that often evidence on causes or effects alone is insufficient to accurately assess the user’s
emotional state, as we illustrate in the next subsection.
4.1 Uncertainty in Modeling Affect
Although emotions often visibly affect a person’s behaviour and expressions, the effects of
emotions are not always discriminating enough to allow a precise diagnosis of the
emotional states that generated them. For example, some accentuated facial expressions
and prosody features can be quite indicative of specific emotional states such as fear, joy
or anger (Ekman, 1993; Murray and Arnott, 1993). However, whether these intense
emotion expressions arise usually depends on the intensity of the emotion, on the user’s
personality and on the interaction context. For instance, an introvert person might have a
