Creating mathematics quiz questions with randomised parameters poses some
interesting technical and educational problems. Some of these are explored
in a case study on implementing a series of questions on introductory matrix
algebra, for use in the MacQTeX quiz system, developed by the Mathematics
Department, Macquarie University, Sydney.
The MacQTeX Randomised Quiz System1 developed
at Macquarie University, Sydney, provides online quizzes in PDF2,
3 format for mathematics, having randomised question
parameters which allow students to practice many examples of the same quiz.
The quizzes are automatically marked and contain fully worked solutions which
aid students in understanding the mathematical concepts involved. The questions
may be multiple choice, or accept answers in numeric form, text or as mathematical
expressions. The checking of answers, marking and calculating the score is
done using the JavaScript4,5 engine in Adobe Acrobat, so it is possible to make
quizzes which are fully functional but independent of a server.
The quizzes are generated using pdfLaTeX6,
randomisation is achieved using perl9 and the PDF forms functionality is generated by a heavily
customised version of D.P. Story's exerquiz8 package for LaTeX7. A quiz has a LaTeX7 source, which loads the LaTeX7 input files for each question as needed. The question
source contains TeX macros in place of the random parameters, and loads a
file defining their replacement values. Each question also has a perl9
source, which is used to supply the TeX definitions for the random parameters.
When a quiz is generated, the perl9
and LaTeX7 sources are assembled, the perl9
sources run, creating the TeX definitions, then the LaTeX7
job is run, using these definitions to supply the values associated with the
TeX macros. (Fig 1.)
Figure 1. The mechanism for creating a quiz.
The decision to implement multiple choice questions or fill-in-the-answer
style questions should be based on educational considerations. Fill-in questions
are always easier to program, as there is no need to deal with distractors,
but students generally prefer multiple choice.
Multiple choice is often seen as a poor substitute for having the student
do all the work himself. However there is value in the student learning to
recognise the correct answer without having to perform the full calculation.
The amount of variety in the question will limit this occurring unless the
student does numerous examples, which is desirable of course.
Distractors for multiple choice questions should be tempting, and based
on mistakes that students commonly make. Many questions are unsuitable for
randomised multiple choice, as the form of the correct answer may be too obvious,
the errors students make are due to careless arithmetic rather than typical
misunderstandings, or it may be that a student who does not understand the
material cannot make any progress on the question at all.
Numeric answers for fill-in-the-answer style questions present few problems,
but if the answer is a mathematical expression then the complexity of the
syntax required to enter it should not place excessive demands on the student.
For elementary mathematics (eg first year university non-mathematics majors),
students cannot be expected to enter much more than a number, or perhaps a
simple expression such as x*(x-5). More advanced students, particularly
those also studying computing, can be expected to enter more complicated expressions,
perhaps involving nested parentheses and special syntax for common functions.
There are several other considerations which must be taken into account
when designing randomised questions. The possibility of duplicate multiple
choice answers must be eliminated, it is not uncommon to come up with the
right answer using the wrong method. If a quiz is to have several examples
of the same question then the system must check for duplicate parameter sets.
The amount of variety obtained by randomisation depends on the nature of
the question, and on the feasability of its implementation. A broad question
may have several types of solution. It must be decided whether it is more
useful educationally to treat them separately or together. Quite often separating
the most common case, or the case that is usually taught first, is useful,
as teachers can then allow the students to practice this before having the
additional demand of recognising the particular case of the general problem.
Separating the cases can also lead to quicker implementation.
The worked solutions must be designed in such a way that they provide a
model from which students can learn to express mathematics well. Succinctness
must be balanced against spoon-feeding - mathematicians like concise elegant
solutions, but students do not like to have steps left out, even if these
are nothing more than basic arithmetic. From a programming point of view,
a succinct solution requires fewer intermediate results to be generated, and
takes less space. In fact, it is in presenting the worked solution where most
of the programming effort lies, when designing a question. When a solution
would take up more than the available space, a series of lead up questions
can be used in order to develop the required techniques.
Random parameters should be chosen to avoid instances of a question which
are trivial, or overly tedious, thus keeping the level of difficulty of the
question fairly constant. Certain parameters will lead to expressions such
as 1x or x0, which look rather silly. These must
be filtered out before the quiz is typeset. Other typesetting issues include
variation in the number of steps in the solution, and alternate text depending
on the type of solution.
Whilst there are efficient and elegant perl9 library functions available for many of the calculations
the students are asked to perform, these are often unsuitable for use in generating
quiz questions as they provide only the answer, but none of the intermediate
results. In some cases these functions can be modified to output the intermediate
steps (in TeX format if necessary), but quite often the algorithms used are
not suitable for hand calculation and are not those that students learn. Hence
it may be necessary to write a new library, and this provides the opportunity
to tailor the algorithms to produce output in the most convenient form. Below
are brief descriptions of the library functions developed for use in MacQTeX
for introductory matrix algebra.
any_matrix(m,n)- simply returns a m x n matrix with arbitrary entries.
-
square_matrix(dim,inv,top_left,steps)
- returns a square matrix with
dim rows and columns. The parameter
inv is 0 for a singular matrix, 1 for an invertible matrix.
top_left indicates that the top left entry should be 1. steps
requests that the intermediate matrices generated by row reduction are returned
as well. The function begins by generating an arbitrary upper triangular
matrix, with non-zero diagonal entries or one zero on the diagonal according
to inv. Then the process of row reduction is reversed, to produce
what appears to be an arbitrary matrix, as well as the intermediate matrices
between this arbitrary matrix and the upper triangular matrix.
make_tex_matrix(M)- returns a string in TeX format suitable for
pmatrix, vmatrix,
smallmatrix etc.
mul_matrix(A,B,steps)- multiplies the matrices A and B. If
steps
is present, it also returns a matrix of TeX formatted expanded products,
which is useful in demonstrating to students how matrix multiplication is
performed. mul_matrix relies of the functions transpose(M)
and dot(u,v).
transpose(M)- simply transposes the matrix M.
dot(u,v,steps)- computes the dot product of vectors u and v. This is needed
for matrix multiplication. It also returns the expanded products if
steps
is set to 1.
determinant(steps,M)- computes the determinant of the matrix M using cofactor expansion.
If
steps = 1 then the entire process of cofactor expansion
is returned as a TeX string. This is not an efficient way to compute a determinant,
but it is included for the purpose of demonstrating the process of cofactor
expansion. First it locates the row or column with the most zeroes, by calling
find_zeroes(M), then recursively builds up the TeX formatted
cofactor expansion.
find_zeroes(M)- locates the row or column of M which contains the greatest number
of zeroes. It returns a string which encodes the position of this row or
column and the number of zeroes it contains.
-
A demonstration of these functions can be found on http://rutherg
len.ics.mq.edu.au/~macqtex/caa/matrices.html.
In addition to a new perl9 library,
some modifications to the JavaScript4,5 which controls the answer input fields in the quiz
documents was needed. D.P. Story's original exerquiz8 package provides a vector syntax. This was modified
to allow nested vectors, to serve as rows of a matrix. To shorten the notation
when fractions are present, common factors may be taken outside the matrix
as a whole, or outside a single row, although the latter is not standard written
notation. The answer is checked not only to match the matrix entries to the
correct answer, but also to make sure the matrix has the correct dimensions,
in which case the student is given feedback that the form of the answer is
wrong. In the case of an undefined answer (such as an undefined matrix product),
the word undefined may be entered.
Matrix addition and multiplication
Given matrices A and B the students are asked to find rA
+ sB, for real r and s, and type the answer. This is quite
straightforward, the matrices A and B are generated by calls
to any_matrix(m,n), keeping m and n constant. The
only restriction is that r and s are non-zero. The matrices
are kept small so that the syntax for the answer is not overly demanding,
and since students generally have little difficulty with this type of question,
there is no need to provide great variety in the dimension of the matrices.
For matrix multiplication, the students are asked to find the matrix product
AB, with matrices of various dimensions. Much can go wrong with matrix
multiplication, so there is plenty of scope for multiple choice distractors.
Typical errors include calculating BA, multiplying corresponding entries
(for matrices with the same dimensions), incorrectly identifying an undefined
product, or attempting an undefined product by ignoring the problematic entries.
Again, any_matrix(m,n) is used to generate the matrices, and
the dimensions of A and B are chosen so that BA is always
defined, but occasionally AB is not. If A and B have
the same dimensions then the distractor which multiplies corresponding entries
is included. When AB is defined, the distractor stating that it is
not, is included. When AB is not defined, the distractor which attempts
the undefined product is included.
Matrix inversion
There is no need to solve the problem of matrix inversion in general. The
question begins half way through the process of converting the extended matrix
(A|I3) to reduced row echelon form to get (I3|A-1).
Again, square_matrix(3,1,1,1) provides the row operations needed
to achieve row echelon form, conveniently avoiding fractions, and guarantees
that the A is invertible. Continuing to reduced row echelon form does
involve fractions, but the tedium can be minimized by restricting the size
of the determinant of the original matrix. Care is taken to present all fractions
in their lowest terms.
In the solution, the extended matrices are shown for each new column of
zeroes below or above the diagonal of A, along with the row operations
used. The student is also reminded to check that AA-1 =
I.
Errors students make in matrix inversion are usually to do with careless
arithmetic or poor strategy in row operations. A student who does not understand
the algorithm usually cannot get started on the problem. This makes finding
suitable distractors difficult, so the question is implemented as a fill-in-the-answer.
The JavaScript4,5 controlling the syntax for
entering the answer allows the student to take common factors out of the matrix,
or out of a single row, even though this is not standard notation. Hopefully
this will not encourage some students to use this notation in their written
work!
Systems of linear equations
As students generally learn to solve systems with a unique solution before
being introduced to those with infinitely many solutions or no solutions,
it was decided to treat the unique solution case separately. The other two
types of system are presented together in the same question, where the randomisation
is set up so that an inconsistent system occurs with probability 1/4, giving
students more practice at dealing with the arbitrary variables in the infinitely
many solutions case.
To generate a suitable matrix and a unique solution in integers, square_matrix(3,1,1,1)
returns a 3x3 invertible matrix A0, an upper triangular
matrix A2 from which A0 was obtained by
reverse row reduction, and A1, the matrix between A0
and A2. The solutions, (x, y, z), of the system are
random non-zero integers, and the constants in the system are obtained using
these and A0. The multipliers used in the gaussian elimination
are easily read from A0 and A1, and so
the constants for each step are easily computed.
For an inconsistent system, square_matrix(3,0,1,1) returns
a 3x3 singular matrix A0, along with the corresponding A1
and A2. The constants in the system are chosen so that the
zero row of the A2 corresponds to a non-zero constant. The
linear system with infinitely many solutions also uses square_matrix(3,0,1,1),
but the zero row of A2 is given a zero constant.
In each case, the augmented matrices are presented in the solution, along
with the row operations used, then partial solutions and back substitution
are shown, and finally the solution. Variable text is needed depending on
whether or not the system is inconsistent. The statement of the question includes
instructions on the correct syntax for the unique solution or on how to enter
the arbitrary variable.
Determinants
The determinant of a 2x2 matrix is very straightforward, and is handled
in the obvious way. For larger matrices, students usually learn cofactor expansion,
as well as being shown how to exploit row and column operations to reduce
the amount of work needed. Four questions have been designed, the first asks
students to find the determinant of a 2x2 matrix, the second a 3x3 using cofactor
expansion, the third is again a 3x3 matrix, but uses row and column operations,
and the fourth does this for a 4x4 matrix.
The 3x3 matrix with cofactor expansion is begun with any_matrix(3,3)
providing an arbitrary 3x3 matrix M. Some of the entries of M
are randomly changed to zero, so that students learn to expand along the row
or column with the most zeroes. determinant(1,M) returns all
the steps in the cofactor expansion method, exploiting the zeroes, as a TeX
string, along with the value of the determinant. Of course this is not the
most efficient way to compute a determinant, but it is necessary in order
to reproduce what the students are expected to do by hand.
Finding the determinant of a 3x3 matrix using row or column operations is
quite easy by hand, but designing a useful question presents some interesting
technical problems. Firstly, the students should learn to use both row and
column operations, so the matrices are sometimes transposed, although internally
the work is done as row operations using square_matrix(3,int rand 4,1,1).
The matrix is singular with probability 1/4. Next, by insisting that the matrix
has 1 as the top left entry and then multiplying the first row by a constant,
it is possible to demonstrate taking a common factor from a row (column),
but any matrix which has another row with a common factor is rejected in order
to keep the solution sufficiently short. Additionally, the rows (columns)
are sometimes swapped, to show how this changes the sign of the determinant.
Again the question is generated from the answer, which of course is the
product of the diagonal entries in the original upper triangular matrix. Because
of the arbitrary row swaps (which may appear as column swaps) and constant
multiples of a row (which may appear as a column), the number of steps in
the solution is variable. This is dealt with at the time of writing the TeX
definitions file, after the main processing of the random parameters is complete.
For the determinant of a 4x4 matrix, the matrix is designed as above, but
an extra row and column are included, such that one of these duplicates an
existing row or column except for one entry. The extra row and column are
then randomly positioned in the matrix. The remaining processing of the question
is similar to the 3x3 case above.
The need to write a library of functions almost from scratch may seem a
daunting prospect. In reality its functions can be made to output data in
exactly the format needed, and the use of optional parameters to request extra
output such as intermediate results is possible, making the design of quiz
questions significantly quicker and simpler. Because the functions are generally
based on the algorithms students learn to perform by hand, they are not necessarily
very efficient, but this is not a major disadvantage as most of the time taken
to generate a quiz is used for the typesetting, not the randomisation process.
This project has received funding in the form of a Targeted Flagship Grant
from the Centre for Flexible Learning, Macquarie University, Sydney, and from
the Division of Information and Communication Sciences, Macquarie University,
Sydney, as well as an equipment grant from Apple Computer, Australia Pty Ltd,
via the Apple Universities Consortium.
-
F. Griffin. Maths CAA Series, LTSN Maths, Stats & OR Network, University
of Birmingham, UK, January 2004. http://ltsn.mathstore.ac.uk/articles/maths-caa-series/jan2004/
- Adobe Systems Inc.; PDF Reference second edition
Adobe Portable Document Format Version 1.3; July, 2000.
- Adobe Systems Inc.; pdfmark Reference Manual, Version:
Acrobat 5.0; Technical Note #5150; Adobe Developer Relations; Revised:
June 25, 2001.
- Adobe Systems Inc.; Acrobat Forms JavaScript Object
Specification, Version 5.0.5; Technical Note #5186; Revised: September
14, 2001.
- Netscape Communications Corporation; Core JavaScript
Reference, 2001; available online at http://develop
er.netscape.com/docs/manuals/javascript.html.
- Han, The Thanh; pdf-TeX, free software for
generating documents in PDF format, based on the TeX typesetting system.
Available for all computing platforms; see http://www.tug.org/applications/p
dftex/.
- Lamport, Leslie; LaTeX, a Document Preparation System.
This is free software available for all computing platforms. Consult the
TeX User's Group (TUG) website, at http://www.tug.org/.
- Story, Donald; exerquiz & AcroTeX, packages
for including special effects in PDF documents, using TeX and LaTeX.
Dept. of Mathematics and Computer Science, University of Akron. Software
available online from http://www.math.uakron.edu
/~dpstory/webeq.html.
- Wall, Larry; perl, a general purpose scripting
language for all computing platforms. This is Free Software, available
from http://www.perl.com/.
