Tutoring an Entire Game with Dynamic Strategy Graphs: The Mixed-Initiative Sudoku Tutor
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
20 JOURNAL OF COMPUTERS, VOL. 2, NO. 1, FEBRUARY 2007
Tutoring an Entire Game with Dynamic Strategy
Graphs: The Mixed-Initiative Sudoku Tutor
Allan Caine and Robin Cohen
David R. Cheriton School of Computer Science, University of Waterloo, Waterloo, Canada
Email: {adcaine, rcohen}@cs.uwaterloo.ca
Abstract— In this paper, we develop a mixed-initiative intel-
ligent tutor for the game of Sudoku called MITS. We begin
by developing a characterization of the strategies used in
Sudoku as the basis for teaching the student how to play. In
order to reason about interaction with the student, we also
introduce a student modeling component motivated by the
mixed-initiative model of Fleming and Cohen that tracks
what the student knows and understands. In contrast to
other systems for tutoring games, we are able to interact
with students to complete an entire game. This is achieved
by retaining a model of acceptable next moves (called a
strategy graph) and dynamically adjusting this model as the
student plays the game. We present the overall architecture
of the system followed by an explanation of the modules that
encapsulate the rules of Sudoku. We also outline formulae
for reasoning about interaction with the student that support
mixed-initiative where either the system or the student can
elect to direct the playing of the game. An implementation
of the system is discussed, including examples of MITS Figure 1. A solved Sudoku Puzzle.
interacting with students in order to tutor the game. To
conclude, we discuss how this research is useful not only to
gain insight into how to tutor students about strategy games 2) allow for strategy graphs to be computed dynami-
but also to understand how to support mixed-initiative cally while tutoring students about a game;
interaction in tutorial settings.
3) present an architecture for mixed-initiative tutoring
Index Terms— Intelligent Tutoring Systems, Mixed-Initiative of games;
Systems, Strategy Games, Student Modeling 4) indicate how to adjust a framework for reasoning
about interaction for mixed-initiative systems for
I. I NTRODUCTION the setting of intelligent tutoring; and
5) provide insight into what is required in order to
Sudoku is a strategy game, developed in Japan, that tutor the game of Sudoku itself.
is growing in popularity worldwide. The game consists
We begin with a discussion of related work in the
of trying to fill in empty cells in a partially completed
areas of intelligent tutoring systems for games and mixed-
9 × 9 grid1 , with a clear set of rules about the possible
initiative systems, looking more specifically at mixed-
entries in a cell (using the numbers of 1 to 9 only) and the
initiative tutoring and at models for reasoning about
conflicts to avoid when completing cells (exactly one of
interaction in mixed-initiative settings. We argue that the
any number in any one column, row and particular 3 × 3
choice of Sudoku is effective in order to design a system
blocks of the grid). A sample solved Sudoku puzzle is
that can tutor an entire game and we provide details on
displayed in Figure 1 with the 3 × 3 grids delineated by
how to characterize Sudoku game play for the purpose of
darker lines.
tutoring.
In this paper we present MITS: Mixed-Initiative Intelli-
Next we present the overall architecture of MITS and
gent Tutoring System for Sudoku. The main contributions
discuss how this extends the set of modules proposed
of our research will be to:
in [2], in a critical way, to support mixed-initiative in-
1) demonstrate how to tutor a student for an entire teraction and tutoring of an entire game board.
strategy game; We then move on to discuss, at a high level, the
processing algorithm underlying MITS. Our discussion
This paper is based on “MITS: A Mixed-Initiative Intelligent Tutoring
System for Sudoku,” by A. Caine and R. Cohen, which appeared in clarifies the first phase used to build up information
AI’06, Québec City, Québec, Canada, June 2006 [1]. about the student, and the second phase where the system
Financial assistance was received from the Natural Sciences and reasons about interacting with the student as the student
Engineering Research Council of Canada (NSERC).
1 We focus on 9 × 9 Sudokus as found in newspapers although larger attempts to complete a Sudoku game board. The details
Sudokus are possible. of the processing to capture mixed-initiative reasoning
© 2007 ACADEMY PUBLISHER
JOURNAL OF COMPUTERS, VOL. 2, NO. 1, FEBRUARY 2007 21
for the purposes of tutoring and to tutor the specifics of in that graph can serve to characterize the student
Sudoku are revealed in a separate section. as novice or expert;
Details of the implemented system are discussed in 2) UMRAO cannot switch its interpretation of the
a section that includes examples of interactions with student’s abilities during a tutoring session; as such
students illustrating the proposed GUI interface. We con- it cannot adapt during the game play; and
clude with a summary of the main contributions of the 3) the strategy graph that represents how to correctly
work contrasting with other research in the areas of play the game is computed off-line because of the
intelligent tutoring and mixed-initiative systems. great deal of time needed to compute it.
This paper extends our earlier work presented in [1] We elected to study a game that was less open-ended
in the following ways. Our assumptions about the game than chess, Sudoku, where the student’s moves could be
of Sudoku are clarified; this constitutes the starting point interpreted in terms of a small fixed number of possible
of our proposed approach for tutoring the game. The strategies. We wanted to emphasize the opportunity for
overall processing of the MITS system is characterized interaction during the tutoring session and to explore the
as a set of distinct phases including actions and iterations circumstances under which interaction should take place,
with terminating conditions. The strategies of Sudoku based on a modeling of the current state of the student.
and the modules used within the MITS architecture for As such, we made sure that the student model is updated
capturing these strategies are described in a detailed and with each move to reflect the abilities of the student. In
comprehensive manner. The most important new design contrast to UMRAO, our strategy graph displays what is
decisions introduced for both the architecture and im- allowable as the next move in the game (as in Figure 4).
plementation of the system are identified and discussed We update the strategy graph only when the student has
to emphasize the contributions of our research. We also made an acceptable move; the graph reflects the student’s
include an extensive set of examples to illustrate the progress in solving the Sudoku puzzle. While the student
central processing algorithm covering various scenarios model reflects what the student knows and understands,
where tutoring may arise. the strategy graph reflects what the student has solved. In
the end, the interaction provided in the system depends on
II. R ELATED W ORK the student and the state of the game, which is adjusted
According to Freedman [3], an intelligent tutoring as the student becomes more skillful in playing the game.
system has four parts: a model of the domain, a model Research in the field of mixed-initiative systems is
of the student, a model of the learning environment, and aimed at determining how best to allow systems and
a teaching model. The model of the domain is what is users to work together in order to collaboratively solve
being taught. The model of the student is a representation problems. Some researchers have argued that intelligent
of the student by the tutor in the computer system. The tutoring is best addressed as a mixed-initiative activity,
model of the learning environment is essentially the user- where students have some control over how the tutoring
interface. Finally, the teaching model is a representation of will proceed as in [4]. Others like [5], [6] have explored
how the material is taught to the student. These important more specifically how to manage tutorial dialogues with
elements are retained in our proposed architecture for students in a coherent manner. Still others have examined
tutoring students in the game of Sudoku. how to predict when the student will take the initiative; an
We also take as a starting point some research on the example is [7]. A further key research question brought
topic of teaching students about endgames of chess, the up in [8], [9] is how to model what the student is learning.
UMRAO system [2]. This system includes two major Our investigation of mixed-initiative tutoring is distinct.
components: the Expert and the Tutor. The Expert is We are focused on how to reason about interacting with
responsible for selecting the specific game board to be a student while attempting to tutor an entire game. What
played and generating the strategy graph — a graph of the student knows and is learning will be captured in
all the possible next moves from a given game board simplified terms and in the context of the strategies
position. Attached to each node is an explanation of underlying the game. The issues of managing an ongoing
the move. The Tutor uses the strategy graph to evaluate dialogue will also be less of a concern, because each new
the moves made by the student, providing feedback and interaction will be selected based on the effort needed to
suggesting future moves to attempt. UMRAO incorporates progress in completing the game puzzle with the student.
as well a graphical interface that displays both the game We take as a starting point for the mixed-initiative
board being considered and a the running feedback from modeling in MITS the research of Fleming and Co-
the Tutor. hen [10]. This work emphasizes the value of modeling
UMRAO is a valuable starting point for our research to the user when reasoning about interaction in a decision-
develop a tutoring system for Sudoku, but it has a number theoretic framework that weighs the benefits and costs of
of shortcomings that we attempt to address in our model, interaction at any given point in time.
as follows: The first model of Fleming and Cohen [10] makes
1) it has no explicit student model. Instead, Gadwal et a distinction between what a user knows and what a
al. [2] claim that the student is modeled in terms of user understands or can be made to understand through
the strategy graph — the level of play demonstrated interaction. We elect to model the students in our Sudoku
© 2007 ACADEMY PUBLISHER22 JOURNAL OF COMPUTERS, VOL. 2, NO. 1, FEBRUARY 2007
tutoring system according to these parameters. In partic- extent the student knows and understands each of these
ular, we will characterize the strategies required to solve four strategies towards the successful completion of the
Sudoku game puzzles and track the probability that the game. For the sake of brevity, we label these strategies
student knows these strategies, or is understanding them, s1 . . . s4 with detailed examples of the use of these
through previous interaction with the tutor. strategies presented in Section V.
Another key element of the Fleming and Cohen model • Rows and Columns s1 We look at the numbers in
is the utility of an action chosen either by the system the current rows and columns and determine what
without the benefit of interaction with the user, or by the must be left by a process of elimination;
system once the user has had a chance to respond and to • Blocks s2 This strategy is like s1 except that we look
provide information. In MITS, we will adjust this factor at the 3 × 3 block to eliminate possibilities;
to consider instead the utility of each move in the game • Pointing Pairs s3 In this strategy, we look for pairs
with respect to completing the puzzle in the most efficient of numbers2 in the same row, column, or block which
way. then cannot be possibilities in any other cells of that
A final component of the Fleming and Cohen work row, column, or block; and
is the concept of cost. Interaction does generate possible • Block-Line Reduction s4 In this strategy, we com-
bother to the user (as discussed in [11]). In MITS, the pare the values needed in a particular block to the
costs will be as a result of generating explanations during values needed in any row or any column which
the course of tutoring. intersects that block.
It is important to note that the model of Fleming and We are also careful to make a distinction between what
Cohen was designed for collaborative problem solving are referred to as playable and unplayable cells. At any
environments, and, as such, it did not specifically examine given stage of the game, it is important to reason from the
the importance of interacting in order to enable the user known values of the other cells of the puzzle to determine
to learn during the collaborative process. The importance for each cell whether it is possible to currently fill that cell
of tracking what the student does not know and needs to with one value (i.e. the cell is playable) or whether there
be taught will be introduced within our model as part of is more than one possible value for the cell (i.e. the cell
the reasoning about interaction. is ambiguous and unplayable). For example, consider the
In the sections that follow, MITS will be presented in 3×3 block g7-i9 of the Sudoku in Figure 2(a).3 By virtue
detail and the value of the new proposals for intelligent of the three 8’s in cells c7, d9 and i6, there must be an 8
tutoring and mixed-initiative will be clarified. in cell g8. The three 5’s in cells c9, e7 and g3 dictate that
there is a 5 in cell i8. So, cells g8 and i8 are playable cells.
III. O UR P ROPOSED M ODEL : MITS Their values are shown in bold in Figure 2(b). It would
In this section, we present our system MITS, which will be improper to play a 4 into cell i9 because that cell can
tutor a student playing the game of Sudoku using a mixed- be either a 4 or a 9 (shown as smaller numbers in that
initiative approach. We begin with a characterization of cell); cell i9 is unplayable. The same holds true of cell
the game of Sudoku and its tutoring environment followed g7, which currently can be either 6 or 4. Understanding
by a discussion of the proposed architecture of MITS and playable and unplayable cells enables the player to make
a description of the algorithm used by MITS to reason moves in a correct and logical order.
about interacting with the student.
B. Architecture of MITS
A. Sudoku Strategies In Figure 3, we present the proposed architecture for
Recall that the game of Sudoku is played beginning MITS.
with a partially completed 9 × 9 grid of numbers (each The Puzzle Library can be thought of as a database of
of which is from 1 to 9) requiring the player to correctly Sudoku puzzles. An example of such a puzzle is given in
fill the remaining values of the grid in accordance with Figure 2(a).
the rules: that there is only one of any number in any The Expert can be thought of as the game’s referee.
column, row or delineated 3 × 3 block of the grid. The Expert knows the game’s rules and enforces them.
To tutor Sudoku, we first of all restrict ourselves to The Expert has two main tasks: generating the Sudoku
games where there is only one solution to the Sudoku Strategy Graph (SSG), and generating the Sudoku Skill
puzzle. The aim is for the student to learn to play Sudoku Matrix (SSM). As will be explained in Subsection III-D,
sufficiently well so that the student will consistently the SSG indicates what the student can do; the SSM how
choose the correct value for each empty cell (and make the student can do it. The SSM will be discussed in more
effective decisions about which cells to try to fill as play detail in Subsection III-E.
progresses), culminating in a completely filled and correct 2 It is also possible to have pointing triplets or quadruplets, but they
grid. occur less in practice than pointing pairs. So, we confine ourselves to
We make the design decision of focusing on the four pointing pairs in this paper.
3 By convention, the columns of the Sudoku puzzle are labeled with
basic strategies that a player can use in order to complete letters with “a” on the left to “i” on the right. The rows are numbered
the open cells of the Sudoku puzzle, modeling to what from 1 to 9 with 1 at the top and 9 at the bottom.
© 2007 ACADEMY PUBLISHERJOURNAL OF COMPUTERS, VOL. 2, NO. 1, FEBRUARY 2007 23
1 2 3 4 7 costs, the player will be tutored. On the other hand, if the
2 1 3 4 costs exceed or equal the benefits, MITS will invite the
3 3 6 1 2 5 student to take the initiative and play independently.
4 4 8 9 3 When the student is taking the initiative, the Tutor will
5 2 7 8 6 communicate the move made by the student to the Student
6 6 1 5 8 Reasoner. The Expert will reply with information about
7 8 5 9 1 3 the correctness or incorrectness of the move. When the
8 4 9 7 student has attempted an incorrect move, MITS will re-
9 5 8 6 2 take the initiative and begin a process of tutoring.
a b c d e f g h i While MITS has some similarities with the UMRAO
(a) Partially Completed Sudoku model of Gadwal et al. [2], there are also some important
1 2 3 4 7 differences, introduced in order to enable the tutoring of
an entire game and to support a mixed-initiative model of
2 1 3 4
interaction between the student and the system.
3 3 6 1 2 5
In UMRAO, the strategy graph and the student model
4 4 8 9 3 are essentially combined. In MITS, to support the mixed-
5 2 7 8 6 initiative paradigm, it is imperative to be able to assess
6 6 1 5 8 the student’s ability to play Sudoku, independent of any
64
7 8 5 9 1 3 consideration of what moves may be made next. We
8 4 9 8 7 5 introduce a separate Student Model and also include
9 94 a Student Reasoner in order to select the part of the
9 5 8 6 2
a b c d e f g h i puzzle to present to the student during tutoring. Since
(b) Sudoku with some playable and unplayable cells we maintain a dynamic strategy graph, this allows the
Figure 2. Sudoku Boards Student Reasoner to constantly monitor the game for the
best moves and strategies to tutor the student, in order
to make the student’s learning experience most effective;
i.e. moves that draw the student closest to successful
completion of the game. The Tutor is relieved of the
responsibility for reading the strategy graph and is instead
focused upon the dialogue with the student.
C. Reasoning about Interaction in MITS
Because we are using a mixed-initiative paradigm, we
have the problem that the Tutor needs to know when to
intervene and offer help, and when to permit the user
to play without assistance. We argue that the model of
Fleming and Cohen [10] can be easily incorporated into
MITS, because we have focused our modeling of students
Figure 3. Our Proposed Model. in terms of four basic strategies of playing the game.
Generally, the Tutor will first take the initiative and
strictly control the solving of the puzzle in an effort to
Once the Expert has generated the SSG and SSM, they impart some initial knowledge to the student; see Subsec-
are sent to the Student Reasoner. The Student Reasoner tion III-F. As more moves are made by the student and
can be thought of as an educational consultant. It uses the witnessed by the Student Model, the Student Reasoner
Student Model, explained in Subsection III-F, to suggest can develop a prediction of the success rate of the student
to the Tutor the move to propose to the student. Once a in solving a Sudoku puzzle without assistance from the
correct move is made (i.e. the student has selected the Tutor. As the probability that the student can solve the
correct value) the Tutor reports the move to the Expert, puzzle without assistance rises, the Student Reasoner will
who updates the game. The Expert re-computes the SSG instruct the Tutor to allow the student to take the initiative.
and SSM and sends them to the Student Reasoner in Even while the student is taking the initiative, the system
anticipation of the next move. will continue to monitor the student’s performance. If the
The Student Model keeps track of MITS’s belief that student’s performance should degrade during unassisted
the user understands the four strategies used to solve play, the Tutor can resume assisting the student.
Sudoku puzzles in terms of acquiring the skill to apply In accordance with the model of Fleming and Co-
them correctly. Using these probabilities, the Student hen [10], a cost-benefit analysis is conducted to determine
Reasoner can assess the benefit and the cost associated if assistance should be given by the Tutor. In general, if
with taking the initiative and tutoring a player regarding the benefit of giving assistance exceeds the cost, the Tutor
a particular move. Generally, if the benefits outweigh the gives assistance.
© 2007 ACADEMY PUBLISHER24 JOURNAL OF COMPUTERS, VOL. 2, NO. 1, FEBRUARY 2007
Bi,(x,y) = [1 − PUK (si )][PUU (si ) + (1 − PUU (si ))PUMU (si )]∆U(x,y) (1)
# of times strategy si was used correctly
PUK (si ) = (2)
# of times strategy si was used
# of times strategy si was explained and understood
PUU (si ) = (3)
# of times strategy si was explained
# of times si was explained in detail and understood
PUMU (si ) = (4)
# of times si was explained in detail.
# of cells known unambiguously after move − # of cells known unambiguously before move
∆U(x,y) = (5)
81
Ci,(x,y) = Ci,(x,y) simple explanation + (1 − PUU (si ))Ci,(x,y) detailed explanation (6)
Wi,(x,y) = Bi,(x,y) − Ci,(x,y) (7)
The benefit of giving assistance for a move into cell brackets takes into account the expectation that the player
(x, y) using strategy i is given by Equation (1) where an understands the strategy. All other things being equal,
explanation of the variables used in Equation (1) follows. the benefit is inversely related to PUK (si ). For exam-
While play is underway, the Student Reasoner records ple, suppose that MITS believes with complete certainty
the answers provided by the student in the empty cells. that the player understands strategy si , which implies
For each cell, one of the strategies would have been that PUK (si ) = 1. Regardless of the other values in
employed by the student. Consequently, the student model Equation (1), Bi,(x,y) = 0. Intuitively, this result makes
can keep simple tallies per Equation (2) to (4) for i ∈ sense. If the player understands the strategy and MITS is
1 . . . 4. completely convinced of that fact, then there is no benefit
As given in Equation (2), PUK (si ) is the probability in tutoring the student about the strategy, si , regardless
that the student knows strategy i. The use of the factor of the cell (x, y).
1 − PUK (si ) in Equation (1) is opposite to the use in [10]: The second part of Equation (1) in the second set of
the benefit of giving advice is inversely related to the square brackets addresses MITS’s expectation that the
probability that the student already knows strategy i. player can be tutored if the need should arise. All other
Furthermore, as in Equation (3), PUU (si ) is the probabil- things being equal, it is more beneficial to choose to tutor
ity that the student would understand the advice given for the player regarding a move where MITS believes that
strategy si and PUMU (si ) in Equation (4) is the probability the student can understand an explanation or be made to
that the student could be made to understand the strategy understand an explanation.
si . The third and final part of Equation (1), ∆U(x,y) , takes
At the outset, MITS does not know if a student would into account the utility of the move. All things being
understand the advice given or if it would be able to equal, MITS should choose to tutor the student regarding
make the student understand the advice given. So, these a move which is most beneficial in terms of completing
probabilities are initialized to 0.5. These probabilities the game. In other words, Equation (1) balances three
would be adjusted, up and down, with actual experience factors:
as the student interacts with MITS and makes correct
or incorrect moves. As illustrated in further detail in • MITS’s belief that the student understands the strat-
Section IV, explaining a strategy occurs when MITS takes egy underlying the move in cell (x, y);
the initiative to direct a user to fill a particular cell and a • MITS’s belief that the student can be tutored regard-
detailed explanation occurs when the student is provided ing the move in cell (x, y); and
with additional hints for filling a cell. • the strategic importance of the move in cell (x, y).
The change in utility that a particular cell, (x, y), Cost would be measured as follows according to
contributes to the game is computed by Equation (5). Equation (6) for some strategy i ∈ 1 . . . 4. The costs
Simply put, ∆U is a measurement of the extent to which a Ci,(x,y) simple and Ci,(x,y) detailed would be each set on a
particular move advances the game towards the solution if scale (0, 1] and would be in proportion to the difficulty
∆U > 0, or the extent of a digression from the solution if of explaining the strategy to the user. Some of the
∆U < 0. It is also possible for ∆U = 0, which means that four strategies are easier to explain than others. The
the move failed to disambiguate any other cells. In brief, first factor, Ci,(x,y) simple , is the basic cost involved in
Equation (1) suggests that: i) it is beneficial to interact if offering an explanation of the user. The second factor,
the student lacks knowledge and either would understand (1 − PUU (si ))Ci,(x,y) detailed explanation , relates to the cost of
or could be made to understand the strategy that needs a detailed explanation weighted by MITS’s expectation
to be explained; and ii) it is more beneficial to interact that the detailed explanation will have to be given. Note
about moves that resolve more of the game board. that when the probability that the user understands is high,
The first part of Equation (1) in the first set of square the cost of a detailed explanation is lower than when the
© 2007 ACADEMY PUBLISHERJOURNAL OF COMPUTERS, VOL. 2, NO. 1, FEBRUARY 2007 25
probability that the user understands is low. Suppose a player inserts an 8 in square e5 of Fig-
Equation (7) is a measure of net worth: the difference ure 4(a). Prior to that move, there were 37 unambiguous
between the benefit and the cost. In general, if the cells. After making that move and with reference to Fig-
Student Reasoner can find a move into cell (x, y) and ure 4(b), there are 39 unambiguous cells, which implies
a corresponding strategy i such that Wi,(x,y) > 0, then that ∆U(e,5) = 39−37
81 = 0.025.
MITS will take the initiative and tutor the student in As well, an SSG can become contradicted. Suppose
regards to move (mx , my ) and strategy i, where (mx , my ) a player plays “1” in cell i8 from the board state in
refers to filling cell (x, y) with its value. Since there Figure 4(b). The strategy graph that would result is
may be more than one move and corresponding strategy illustrated in Figure 4(c). This play is improper because
satisfying the condition Wi,(x,y) > 0, MITS makes its “1” is not a possibility for cell i8, because a “1” already
choice such that (i, (x, y)) = argmaxı̂,x̂,ŷ Wı̂,(x̂,ŷ) . appears in cell f8. Before the move is made, there are
On the other hand, if ∀ i, x, y Wi,(x,y) ≤ 0, then turning 39 unambiguous cells; afterwards, there are 34; so then
over the initiative to the student is indicated; the student ∆U(i,8) = 34−39
81 = −0.062. So, whenever ∆U(x,y) < 0
should be allowed to play independently and without the student requires assistance.
assistance.
So, under this approach, an explanation would be given
E. Sudoku Skill Matrix
if Bi,(x,y) > Ci,(x,y) for some playable move (mx , my )
using strategy i. However, because of the manner in which It is not enough for the Expert to simply draw up
we have defined utility, U , we must make an exception the SSG. It must also compute the Sudoku Skill Matrix
to the ordinary cost-benefit rule whenever ∆U(x,y) < 0 (SSM). The SSM is a 9 × 9 matrix each entry of which is
for some move (mx , my ) made by the student during an integer from 0 to 4. The numbers 1 to 4 indicate the
unassisted play. If ∆U(x,y) < 0, then the benefit calcu- strategy (or skill) that the student must use to solve the
lated according to Equation (1) will be negative. Since given position on the Sudoku board. If the Expert assigns
the cost is always non-negative, the cost-benefit rule a zero to any entry of the SSM, the zero signifies that
is never met under these circumstances. Yet, whenever either the cell has already been solved out, or that the
∆U(x,y) < 0, the student has committed an error and cell is ambiguous and any attempt to solve the cell would
MITS must intervene to correct the improper play. We be premature. The SSM related to Figure 4(a) is given in
discuss in Section VI why it may not be advisable to Figure 5.
allow students to continue to play after making a wrong
move in order to effectively tutor the game of Sudoku. 1 0 0 0 0 0 0 0 2 0
The ultimate goal of the student model is Bi,(x,y) ≤ 2 0 0 0 0 4 0 0 0 4
Ci,(x,y) for every plausible move (mx , my ) and strategy 3 0 0 0 0 0 0 4 0 0
i from the current board state. This means that the 4 0 0 0 0 0 0 0 0 0
student is playing Sudoku independently. Yet, we cannot 5 0 0 0 4 1 0 1 0 0
uncompromisingly apply the rule ∆U(x,y) > 0 and 6 0 0 0 0 0 0 0 0 0
Bi,(x,y) > Ci,(x,y) ∀ i, (x, y) as a condition of giving 7 0 0 0 0 0 0 0 1 0
advice. Consider that case where the student wants advice 8 4 0 0 0 0 0 0 0 0
when the student is playing independently. For example, 9 0 1 0 0 0 0 0 0 0
the student may be playing a puzzle at a high level a b c d e f g h i
of difficulty and has managed to “get stuck.” In this
Figure 5. A Sudoku Skill Matrix for the SSG in Figure 4(a).
case, the tutor computes argmaxı̂,x̂,ŷ Wı̂,(x̂,ŷ) , in order
to determine the most worthwhile move, (mx , my ), using
Potentially, there might be more than one strategy
strategy i to discuss with the user. An example of a student
available for solving out a cell in the Sudoku board.
requesting help is displayed in Section V-B.1.
So, the programmer will need to provide for storage for
additional integers in each cell. In the case of multiple
D. Sudoku Strategy Graphs strategies, we must consider two cases: the tutor has the
Sudoku Strategy Graphs (SSG’s) are straightforward to initiative and the student has the initiative.
read. A typical strategy graph is given in Figure 4(a). If the tutor has the initiative, then the strategy i will be
A cell with a single large number means that the cell selected such that PUK (si ) is a minimum. Consequently,
was either an initial clue or it has been subsequently and Equation (1) is maximized all other things being equal.
correctly solved out by the student. A cell with one small The tutor will be focused upon the strategy that the
number means that the cell is unambiguous, and, for the student knows the least.
current board state, the cell can be solved out. If a cell If the student has the initiative, the problem is more
in the SSG has two or more small numbers, the cell is complex because the issue of plan recognition arises. If
ambiguous. The numbers appearing in the cell are the the student gives the right answer, the tutor does not know
current possibilities, but the actual answer is unknown. A which strategy the student used. The tutor could query
student who attempts to solve out an ambiguous cell is the user to find out, but queries following a proper move
in fact committing an error. might be viewed as an annoyance to the user. We take the
© 2007 ACADEMY PUBLISHER26 JOURNAL OF COMPUTERS, VOL. 2, NO. 1, FEBRUARY 2007
(a) First board Position (b) SSG after an “8” is played in cell e5 (c) SSG after a “1” improperly played in i8
Figure 4. Sudoku Strategy Graphs.
view that it would be better to simply not trouble the user number of ambiguous cells from the first board state. The
even though the probability PUK (si ) will not be updated number of ambiguous cells can be determined from the
for that play. On the other hand, if the user supplies puzzle’s first strategy graph.
the wrong answer, the tutor can re-take the initiative and MITS would start a new student off with a puzzle
choose the strategy i such that PUK (si ) is minimized over having the least degree of difficulty advancing the student
all plausible strategies. The tutor has moved the focus of upwards to the most difficult puzzle. When a puzzle is
discourse to the strategy least understood by the user. solved, it would be recorded by the student model to
prevent the puzzle from being selected again. The student
model would also store the level of difficulty of the
F. Initializing the Student Model last solved puzzle for reference in making future puzzle
There are two ways in which a student model can be selections. Note that this procedure is conducted once per
initialized. The student can provide the parameters, or the student.
system can develop the model during play. We take the
position that the latter approach is best.
The user is modeled using 12 probabilities PUK (si ), G. The Algorithm
PUU (si ), and PUMU (si ) for i ∈ 1 . . . 4. We concede that The MITS Algorithm is given in Figure 6. The first
the user might be able to provide the four probabilities phase of the algorithm is the first repeat loop from lines 2
PUK (si ) provided that the user actually understands the to 8. It involves establishing the Student Model. In this
strategies s1 . . . s4 . The danger is that the user might phase, MITS observes the student’s progress. However,
believe incorrectly that they understand a strategy and the game is under the control of MITS.
overestimate one or more or the probabilities PUK (si ). The second phase is in the second repeat loop: lines 10
Until a sufficiently large sample of moves is recorded to 34. It commences whenever MITS is satisfied it has
by the student model, these probabilities cannot be used. established a fair representation of the student in the
However, the Student Reasoner keeps game play under student model. In line 13, MITS computes ∆U for all 81
strict tutor control selecting a variety of possible moves cells. If any cell, (x, y), has the property that ∆U(x,y) ≥
as the subject of discourse which exemplifies all four 0, it means that the cell is playable. So, in lines 11 to 15,
strategies s1 . . . s4 . Once a sufficiently large sample of MITS computes the benefit and cost of those cells as
moves is obtained by the student model, the mixed B(x,y) and C(x,y) using Equations (1) and (6) respectively.
initiative model can be brought on-line and used actively. At the if-statement at line 16, MITS decides if it should
We have already suggested that the eight other proba- take the initiative. MITS takes the initiative whenever it
bilities PUU (si ) and PUMU (si ) should be all initialized to finds a positive Wi,(x,y) as computed per Equation (7);
0.5. MITS cannot anticipate how well its discourse with there is at least one move that is beneficial to discuss
the student will fare. Likewise, since the student has yet with the user. MITS will tutor the move with the greatest
to interact with MITS, the student cannot anticipate how net worth. Note that when a move (mx , my ) is chosen
well he or she will understand the MITS’ advice. So, it to tutor the student, MITS will know the strategy that
makes no logical sense for the student to provide those enables filling in the value of the chosen cell. This strategy
probabilities to the student model. will be suggested whenever MITS engages in detailed
Unlike UMRAO, the Student Reasoner recommends explanation to the student. If there is more than one
the next puzzle to be played. In UMRAO, the Expert possible strategy, MITS will be explaining the one that
makes the decision. To account for this difference, the maximizes the value of Wi,(x,y) .
puzzles would be ranked by their level of difficulty. The The student takes the initiative in line 20 whenever
difficulty level of a Sudoku puzzle is a function of the Wi,(x,y) ≤ 0 ∀ i, x, y, for under these circumstances there
© 2007 ACADEMY PUBLISHERJOURNAL OF COMPUTERS, VOL. 2, NO. 1, FEBRUARY 2007 27
is no benefit in discussing any of the moves with the user. 1 /* PHASE ONE */;
When the player has the initiative, MITS must anticipate 2 repeat
four possible scenarios: 3 if game over then
• The student requests assistance: In this case as in 4 Select a new game from the puzzle library
line 23, MITS must re-take the initiative and begin 5 Select move for the student to attempt;
tutoring the student regarding move (mx , my ) such 6 Build up the Student Model;
that Wi,(x,y) is maximum. 7 /* No reasoning about mixed initiative */
• The student does not request assistance, and: 8 until sufficient statistics on student ;
– The student makes a play into an unplayable 9 /* PHASE TWO */;
cell: An unplayable move is characterized by 10 repeat
∆U(x,y) < 0; see line 26. MITS must intervene 11 for x ← 1 to 9 do
and prohibit the move; otherwise, the game 12 for y ← 1 to 9 do
cannot be concluded successfully. 13 Compute ∆U(x,y) ;
– The student makes a play into a playable cell, 14 Compute Bi,(x,y) and Ci,(x,y) ;
and: 15 /* for those i indicated by the SSM */;
∗ Inserts the wrong number: In this case per 16 if ∃ i, x, y 3 Wi,(x,y) > 0 then
line 28, the student must be tutored on the 17 /* MITS Takes the initiative */;
correct play. The student’s probabilities must 18 i, x, y ← argmaxı̂,x̂,ŷ Wı̂,(x̂,ŷ) ;
be penalized to reflect the incorrect play. 19 Tutor the move into cell (x, y);
∗ Inserts the correct number: As in line 31,
the student has made the correct play into 20 else
a playable cell. The student is rewarded for 21 /* Student has the initiative */;
the correct play by having the probabilities 22 /* ∀i, x, y Wi,(x,y) ≤ 0 */;
associated with that play increased. 23 if student requests help then
24 i, x, y ← argmaxı̂,x̂,ŷ Wı̂,(x̂,ŷ) ;
After one of these four cases have been dealt with
25 Tutor the move into cell (x, y);
by MITS, MITS loops back to line 10 unless the game
is concluded. MITS re-computes the ∆U(x,y) ’s, B(x,y) ’s, 26 else if student moves into cell
and C(x,y) ’s in lines 13 to 15. Since the probabilities may (x, y) 3 ∆U(x,y) < 0 then
have been changed in lines 30 or 32, it is possible that the 27 MITS intervenes and prohibits
condition in line 16 may change from the last iteration. To 28 else if student puts wrong number into (x, y)
put it another way, MITS may retake the initiative if the then
player played poorly in the last iteration (line 30); MITS 29 Tutor student (x, y) to correct error;
may relinquish the initiative to the player if the player 30 Penalize student’s probabilities;
played well in the last iteration (line 32). 31 else
32 /* Student has made a correct move */;
33 Reward the student’s probabilities;
IV. G RAPHICAL U SER I NTERFACE
The Graphical User Interface is illustrated in Fig- 34 until game concluded ;
ure 7(a). We have opted for a simple GUI patterned after Figure 6. The MITS Algorithm
UMRAO. The playing board is to the left; the discourse
window to the right. In Figure 7(a), the student model
is being built and the Tutor has the initiative. The test this a simple explanation (refer to Equation (3)), because
concerns strategy s4 , Box-Line Reduction. Since there is the Tutor merely indicates where the move ought to be
an 8 in each of columns g and h and the columns intersect made, but does not explain the logic. If the student cannot
the block in the lower right, the user ought to be able to determine the value of the cell, then the Tutor will resort
conclude that the value in cell i9 is an 8. We call this a to a detailed explanation similar to Figure 7(a).
detailed explanation (refer to Equation (4)) because the
Tutor specifically names the other cells needed in making
V. S AMPLE S ESSION
the logical deduction that i9 is “8.”. Detailed explanation
occurs during initial tutoring. In this section, we wish to demonstrate MITS under a
Assume that the user correctly plays 8 in cell i9. MITS number of scenarios. The scenarios we wish to consider
continues to control the tutoring and directs the student are:
to examine cell g7. This is a test of another variation of 1) MITS takes the initiative, and
strategy s4 . Here, the student must recognize that there a) Illustrates Strategy s1 — rows and columns
exists a 2 in each of rows 8 and 9, and a 2 in column b) Illustrates Strategy s2 — Blocks
h. So, it follows that g7 is 2. The Tutor is trying to c) Illustrates Strategy s3 — Pointing Pairs
ascertain if the student can use what was learned in board d) Illustrates Strategy s4 — Block-Line Reduc-
Figure 7(a) to solve cell g7 in board Figure 7(b). We call tion
© 2007 ACADEMY PUBLISHER28 JOURNAL OF COMPUTERS, VOL. 2, NO. 1, FEBRUARY 2007
some thoughts on how we may also encourage students
to acquire a deeper understanding of the strategies of
Sudoku.
1) MITS Illustrates Strategy s1 : Strategy s1 is the row
and column strategy. From Figure 8, it is possible to
determine the position of the 9 in column g. The 9’s in
i7, h5 and a1 eliminate all possibilities other than g2; so
g2 = 9.
1 9 2 8 5
2 7 3 2 6
(a) Sudoku GUI — Move #1 Detailed Explanation 3 8 1 4 5
4 6 1 8
5 1 2 4 3 7 9
6 7 2 5
7 4 5 5 9
8 9 3 2 7
9 4 8 1 3
a b c d e f g h i
Figure 8. Sudoku for row and column strategy
Consider a possible interaction between MITS and the
Player as depicted in Figure 9. The notation M>> means
(b) Sudoku — Move #2 Simple Explanation MITS is displaying a message on the GUI; P>> means
the player is interacting with the GUI. In this example, the
Figure 7. The MITS Graphical User Interface.
user initially gives the wrong answer, g1; MITS reminds
the player that there is already a 9 in row 1 and asks for
2) The player takes the initiative, and another response. The player responds correctly.
a) The player requests assistance; M>> Where is the value 9 located in
b) The player makes a play into an unplayable column g?
cell; P>> g1=9
c) The player makes a play into a playable cell M>> No, there is already a 9 in row 1
but enters the wrong number into that cell; by a1=9. Try again.
d) The player makes a play into a playable cell P>> g2=9
and enters the correct number into that cell. M>> Correct!
For the purposes of our illustration, we assume that the
Figure 9. Possible Interaction over rows and column strategy
∆U(x,y) ’s, B(x,y) ’s, and C(x,y) ’s in lines4 13 to 15 have
been computed. 2) MITS Illustrates Strategy s2 : This strategy is the
The examples presented below would all be cast in the blocks strategy where we try to determine a value by
GUI presented in Section IV. The interaction from MITS elimination. Suppose that we placed a 4 in c1 because of
would appear in the column on the right hand side of the 4 in b7, and a 5 in b2 because of a 5 in h3. This
the screen and the student would interact by clicking on situation after these two moves is depicted in Figure 10.
certain empty cells and filling in certain values. In order to Then, by a process of elimination, the value in b3 must
reduce the display of screenshots below, we simply show be 6 for it is the only value left which is still needed by
the initial game board that would appear and convert the the block a1-c3.
student’s turns into actual responses within a dialogue A possible interaction is given in Figure 11. Because
with MITS. of the simplicity of the situation, the student gives the
correct answer at once.
A. MITS takes the initiative 3) MITS Illustrates Strategy s3 : To illustrate the point-
ing pairs strategy, s3 , consider Figure 12. In the 3 × 3
Recall that this situation is characterized by ∃ i, x, y 3
block from g4-i6, we need a digit 1. However, it is not
Wi,(x,y) > 0 for some playable move (mx , my ); see
clear if the 1 is in cell i4 or i5. On the other hand, if
line 16. MITS focuses on encouraging the student to
we consider the block from d4-f6, the 1 is in either e4 or
put the correct value into a cell using the SSM in order
f4 as denoted by the two tiny 1’s. Those two 1’s are the
to determine the strategy at hand. See Section VI for
pointing pair, which indicate that there can be no other 1
4 For the sake of brevity in this section, whenever we refer to a line on row 4. Consequently, the 1 in the g4-i6 block must be
number, we are referring to a line number with respect to Figure 6. in cell i5 – not i4.
© 2007 ACADEMY PUBLISHERJOURNAL OF COMPUTERS, VOL. 2, NO. 1, FEBRUARY 2007 29
1 9 2 4 8 5 1 7 6 5 4 8
2 7 5 3 2 6 2 9 6 1 2 3
3 8 1 4 5 3 5 4 8 7 6
4 6 1 8 4 4 1 1
8 5
5 1 2 4 3 7 9 5 8 7 2 6 3
6 7 2 5 6 1 5
7 4 5 5 9 7 3 2 5 1
8 9 3 2 7 8 2 5
9 4 8 1 3 9 6 7 5 2 3
a b c d e f g h i a b c d e f g h i
Figure 10. Sudoku for blocks strategy Figure 12. Sudoku for pointing pairs strategy
M>> All of the cells in block a1:c3
are known except b3. M>> Where must the 1’s go in block
M>> What is the value of cell b3? d4-f6?
P>> b3=6 P>> e4=1 or f4=1
M>> Correct! M>> Correct! So, where is the 1 in
block g4-i6?
Figure 11. Interaction using the blocks strategy P>> i5=1
M>> Correct!
A possible interaction with MITS is given in Figure 13. Figure 13. Interaction for pointing pairs
Note that this strategy is somewhat more subtle. MITS
must lead up to it by first getting the student to locate the 1 9 2 8 5
pointing pairs by asking where the 1’s must go in block 2 7 3 2 6
d4-f6. Only then can MITS ask the student where the 1 3 8 1 4 5
is located in block g4-i6. 4 6 1 8
4) MITS Illustrates Strategy s4 : In the block-line re- 5 1 2 4 3 7 9
duction strategy, we compare values needed in a particular
6 7 2 5
block to the values needed in a particular row or column.
Consider column b and block a1-c3 in Figure 14. Clearly, 7 4 5 5 9
block a1-c3 needs a 4. On the other hand, column b 8 9 3 2 7
does not need a 4; a 4 already appears in cell b7. So, 9 4 8 1 3
by elimination, the 4 needed by block a1-c3 must appear a b c d e f g h i
somewhere other than column b; it must appear in cell Figure 14. Block Line Reduction
c1.
A possible interaction over the block line reduction is M>> Where is the 4 in block a1-c3?
given in Figure 15. Because of its simplicity, the player P>> c1=4
gives the correct answer at once. M>> Correct!
Figure 15. Possible interaction over block line reduction
B. The Player takes the initiative
The player takes the initiative whenever 1 4 9 8
∀i, x, y Wi,(x,y) ≤ 0 as per line 20. The player 2 9 2 4
takes the initiative because the foregoing conclusion 3 5 7 4 6
does not indicate a need to tutor the player. However, 4 3 9 1
while the player is taking the initiative, there are four 5 3
possibilities as outlined below:
6 9 7 3
1) Requests Help: This situation arises whenever the
student has been playing well, but the player desires a 7 8 7 1 4
hint. This situation can be found on line 23. 8 3 2 8
In Figure 16, we show a Sudoku rated for advanced 9 2 4 5
players. It is a difficult Sudoku because there are few a b c d e f g h i
open moves. So, a possible interaction with MITS where Figure 16. Advanced Sudoku
the player has the initiative is given in Figure 17. As it
turns out, h7=2 by the block-line strategy. P>> help
2) Unplayable Move: Unlike Section V-A, we must M>> There is a move at cell h7
consider the case where the player might make a move
Figure 17. Interaction with MITS using help
into a cell which is unplayable. By contrast, MITS will
© 2007 ACADEMY PUBLISHER30 JOURNAL OF COMPUTERS, VOL. 2, NO. 1, FEBRUARY 2007
never suggest that a player make a move into an un- against the player when the player is penalized for the
playable cell. incorrect play.
The situation is illustrated in line 26. It it characterized Consider again the Sudoku in Figure 16. Further, let
in two ways: cell (x, y) is unplayable if ∆U(x,y) < 0 or us suppose that the user recognizes that there is a play to
if the SSM at cell (x, y) is zero5 . MITS must intervene. cell f5, which there truly is. Suppose that due to a lapse
Otherwise, if the move is allowed to stand, the game is in concentration, the user plays f5=8. The correct move
guaranteed to end in a deadlock. is f5=9 by recognizing the three 9’s in cells a6, h4, and
On the other hand, no penalty is possible. Since there d1. The interaction might be as in Figure 19.
is no strategy associated with an unplayable move, there
P>> f5=8
is no strategy to penalize. Since the probabilities will
M>> That’s not quite right. Try again.
not change, the ∆U(x,y) ’s, B(x,y) ’s, and C(x,y) ’s won’t
P>> f5=7
change. Instead, MITS will simply suggest that the stu-
M>> Notice that a6=9, h4=9, and d1=9.
dent try another move because the cell that the player
Try again.
tried is unplayable. The player retains control and takes
P>> f5=9
the initiative on the next move.
M>> Correct. Since you had difficulty
Reconsider the advanced Sudoku in Figure 16. Now, with block-line, let me suggest the
in block a4-c6, it is certainly the case that the 1 is not next move. What is h7?
in cells a4, b4, and c4. Yet, consider the interaction with P>> h7=2
MITS in Figure 18 where the player elects to play a 1 in M>> Correct. Keep playing.
cell a5.
Figure 19. User makes a wrong play into a playable cell
P>> a5=1
M>> There is no open move in that Notice that when the user makes an incorrect response
cell. Try again the second time, MITS gives an extra hint. Once the
player does give the correct answer, the initiative shifts
Figure 18. Interaction where user makes an unplayable move
to MITS since it believes that a discussion to reinforce
the ideas of the block-line strategy is worthwhile. So, it
Note that MITS does nothing more than point out that directs the player’s attention to cell h7. We assume that the
no move exists. Furthermore, it does not matter that it player answers correctly on the first attempt. Satisfied that
may come to pass that a5=1, because MITS discourages the user understands the block-line strategy, MITS shifts
guessing. However, since the benefits and costs of inter- the initiative back to the player and invites the player to
action have not changed, the player is invited to try again. play again.
3) Playable Move — Wrong Entry: This situation can 4) Playable Move — Correct Entry: This situation,
be found at line 28. Here, the student has found a move depicted in line 32, is the most desirable. The player
which is playable, but, unfortunately, enters the wrong has played correctly and independently into a playable
number into the cell. In this case, MITS assumes that cell. MITS rewards the student by increasing the player’s
the strategy is not understood, penalizes the student’s probabilities for the strategy that was employed to solve
probabilities for that strategy, takes the initiative, and that cell. Since some of the player’s probabilities increase,
tutors the student. After the tutoring is complete, MITS some B(x,y) ’s will decrease. Consequently, the condition
may or may not return control back to the user depending ∀i, x, y Wi,(x,y) ≤ 0 remains true. The player retains the
upon the condition ∃ i, x, y 3 Wi,(x,y) > 0. Since initiative on the next move. Intuitively, this result makes
some of the student’s probabilities were reduced, some sense: if the player is playing correctly and independently,
of the B(x,y) ’s will increase. So, the foregoing condition any intervention by MITS would be unnecessary and
might be made true in which case MITS would retain possibly unwelcome to the player.
the initiative until the foregoing condition was made false Again, reconsider the Sudoku in Figure 16. For MITS,
again. this is the easiest case to consider. It need only record
This result is intuitively correct. If the error is major the move and ask the player to play again. Suppose the
in nature, then it ought to be the case that ∃ i, x, y 3 player correctly plays f6=9. A possible interaction is given
Wi,(x,y) > 0; so, MITS should take the initiative on the in Figure 20.
next move. On the other hand, if the error is minor in
nature such that in spite of a decrease in the player’s P>> f6=9
probabilities it is still the case that ∀i, x, y Wi,(x,y) ≤ 0, M>> Correct! Play again
then MITS will not over-react to a minor error. The player Figure 20. Interaction where the user makes a correct play into a
will take the initiative on the next move in spite of the playable cell
minor error. What constitutes a major or a minor error
depends upon the extent to which the probabilities shift
VI. C ONCLUSIONS AND F URTHER R ESEARCH
5 To be clear, these conditions are equivalent: ∆U
(x,y) < 0 iff the
In developing MITS, a system for tutoring a stu-
SSM at (x, y) is zero. dent about Sudoku, we have designed an architecture to
© 2007 ACADEMY PUBLISHERJOURNAL OF COMPUTERS, VOL. 2, NO. 1, FEBRUARY 2007 31
support mixed-initiative interaction during tutoring with at that what move long ago generated the difficulty, it
a student being modeled and advised about an entire would be challenging to progressively step through the
game. This is in contrast to other efforts to tutor games various contexts that led from that move to the final
that are restricted to endgames only (eg. [2]). Using deadlock.
dynamic strategy graphs, we are able to model the student Since we do inform students when they have made
progressively during game play. incorrect moves, it may be useful for future work to
The domain of Sudoku is generally helpful for in- explore the possibility of pulling a student away from
vestigating mixed-initiative tutoring because there is an a puzzle entirely, if it is proving to be too challenging
intuitive interpretation of the utility of a move in terms (i.e. a vast number of repeated wrong moves), replacing
of the ability to disambiguate any open cells in the grid. this with a somewhat simpler puzzle to tutor.
We are then able to make use of this term of utility to Another avenue for future work is to do a second pass
critique the actions of the student leading to intervention with students, going over puzzles already completed in
from the tutor when the student lacks knowledge and is order to convey the essence of the basic strategies of
following paths that have low utility. Sudoku, to have students acquire this deep knowledge.
We have been able to introduce some innovative Note that it would be challenging to simply “give” the
changes to the model of Fleming and Cohen for the design students the strategies to begin with, because they are
of mixed-initiative systems [10] in order to apply it to the best understood with illustrative examples. We therefore
problem of intelligent tutoring. Providing for a model of decided to have the student experience the examples by
whether a user understands or can be made to understand playing the puzzles.
when engaged in dialogue leads to a tutorial system that
tracks the understanding of the student based in part on R EFERENCES
past attempts. In addition, the need to ensure that we are [1] A. Caine and R. Cohen, “MITS: A mixed-initiative intel-
also enabling learning serves to adjust the decisions about ligent tutoring system for sudoku,” AI 2006, June 2006.
interaction in the mixed-initiative model. [2] D. Gadwal, J. E. Greer, and G. I. McCalla, “Tu-
In contrast to other researchers focused on developing toring bishop-pawn endgames: an experiment in using
knowledge-based chess as a domain for intelligent tutor-
mixed-initiative tutoring systems [4]–[8] we specify how ing,” Applied Intelligence, vol. 3, pp. 207 – 224, 1993.
to integrate a definitive decision-theoretic model for rea- [3] R. Freedman, “What is an intelligent tutoring system?”
soning about interaction with the student. In addition, we Intelligence, vol. 11(3), 2000.
work directly from a student model that has been built up [4] S. Carberry, “Discourse initiative: Its role in intelligent
dynamically in previous sessions with the student. Within tutoring systems,” Working Notes of the AAAI Spring
Symposium on Computational Models of Mixed Initiative
the context of Sudoku, we are able as well to condense Interaction, pp. 10 – 15, 1997.
the modeling of the student into a representation of the [5] F. Shah and M. Evens, “Student initiatives and tutor
student’s abilities with respect to the central strategies for responses in a medical tutoring system,” Working Notes
playing the game. of the AAAI Spring Symposium on Computational Models
of Mixed Initiative Interaction, pp. 138 – 144, 1997.
There are several avenues for future research. In par- [6] R. Freedman, “Degrees of mixed-initiative interaction in
ticular, developing a more sophisticated Student Reasoner an intelligent tutoring system,” in AAAI 1997 Spring
and a more detailed student model would both be helpful Symposium on Computational Model for Mixed-Initiative
in order to deliver more customized tutoring to the stu- Interaction., 1997.
dent. For instance, the Student Reasoner could identify [7] J. Beck, P. Jia, J. Sison, and J. Mostow, “Predicting student
help-request behavior in an intelligent tutor for reading,”
patterns of difficulty in a student’s strategy graph in order in Proceedings of User Modeling 2003, 2003.
to predict values for the PUU (si ) and PUMU (si ) variables [8] M. Manske and C. Conati, “Modelling learning in an
manipulated in the formulae. Another suggestion is to educational game,” in Proceedings of AIED 2005, 2005.
estimate more precisely the value of certain factors, such [9] A. Baena, M. Belmonte, and L. Mandow, “An intelligent
as the expected number of interactions to explain a given tutor for a web-based chess course,” in Lecture Notes in
Computer Science, P. Brusilovsky, O. Stock, and C. Strap-
strategy by analyzing the student’s current knowledge parava, Eds., vol. 1982/2000, Aug. 2000.
and past behavior. The suggestion of allowing for either [10] M. Fleming and R. Cohen, “A user modeling approach
very basic or more detailed commentary when advising to determining system initiative in mixed-initiative ai sys-
a student is yet another area where more intelligent tems,” Proceedings of User Modeling 2001, July 2001.
algorithms may be designed mapping a certain range of [11] R. Cohen, M. Cheng, and M. Fleming, “Why bother about
bother: Is it worth it to ask the user?” AAAI’05 Fall Sym-
values of student modeling factors with a proposed level posium on Mixed-Initiative Problem-Solving Assistants,
of detail for the interaction. 2005.
Note that we elected to have MITS interact with
students whenever wrong moves were attempted (see
Section V-B.1). We did not allow the student to continue
to play a game with wrong moves because there are
Sudoku puzzles where it may take a long number of
moves before it is apparent that the game cannot be
completed. While it would be possible to show the student
© 2007 ACADEMY PUBLISHERYou can also read
