Using Action-Object-Input Scheme for Better Error Diagnosis and Assessment in Expression Manipulation Tasks

by

Rein Prank, Marina Issakova, Dmitri Lepp, Vahur Vaiksaar

Institute of Computer Science, University of Tartu

{rein.prank, marina.issakova, dmitri, vax}@ut.ee

Introduction

Expression manipulation is one of the central skills needed for solving tasks in practically all fields of mathematics. The work with expressions is also very important in programming, as well as in spreadsheets or database queries. The school curriculum contains hundreds of tasks with fractions, monomials, polynomials, equations and systems of equations. In comparison with many other topics, the solutions of expression manipulation exercises are long. The length and order of solution steps depend on the pupil’s decisions. The result of application of the correct transformation rules usually differs from the result of application of the most frequent malrules only by some symbols. During the paper-and-pencil training process the teachers are not able to discover all the mistakes and correct them in time. Even tests are often returned to the students with some mistakes uncorrected. Permanent need for quick analysis of large amounts of detailed information indicates that the training and testing could be improved by using computerised environments. What type of environments do we need?

Many basic types of expression manipulation tasks are taught together with some solution algorithms. When the student solves such task, he should at each solution step

1)      choose a transformation rule corresponding to a certain operation in the algorithm (or some simplification or calculation rule known earlier),

2)      select the operands (certain parts of expressions or equations) for this rule,

3)      replace them with the result of the operation.

Some other tasks (such as factorisation of multi-variable polynomials or integration) are taught without having direct algorithm but the solutions are expected to consist of steps of the same mental structure. It is clear that mistakes can be made and feedback or hints are necessary at any of the three stages of the step. For proper learning of expression manipulation as well as for assessment and diagnosis of knowledge gaps, the student should make all the necessary decisions and calculations at each solution step, and the program should be able to understand the mistakes. Is it possible using existing expression manipulation software?

The answer seems to be “No”. Universities and also schools of many countries use computer algebra systems as basic software for the work with expressions. Using Maple, Mathematica or Derive, the student chooses for the step only the transformation rule and in some cases part of the expression. The transformation itself is made by the computer and it is impossible to make transformation mistakes. In many cases the program itself selects operands. In addition, the commands of computer algebra programs are very powerful (Simplify, Expand, Factorise, Solve, Differentiate, Integrate) and enable to solve many tasks of the school and university mathematics in one step without intellectual participation of the student. There exist some rule-based learning environments that have more detailed rules extracted from textbook algorithms [1, 9], but they also apply the rules mainly automatically. It means that such programs can be used for learning the algorithms but not training low-level work.

Completely orthogonal approach is used in APLUSIX [5] where the student simply enters the solution line by line and the program checks each step. As working with paper-and-pencil, the student makes in this interface all the decisions and the opportunity to make mistakes is not restricted. But this interface does not give the program any information about the intentions of the student and allows making in one step arbitrarily complex multi-rule conversions. As a result, it is very hard to provide a diagnosis of the mistakes that would go beyond the message Not equivalent. The student himself has to understand the reasons of the message and the program cannot give qualitative information about the errors for the teacher.

In the University of Tartu the first intelligent expression manipulation environment (for normal forms and expressibility with {Ø,&}, {Ø,Ú} and {Ø,É} in Propositional Logic) was implemented in 1988-90 in DOS text mode [6, 7]. In this environment the student marks for each step a subformula and (depending on the working mode) selects a transformation rule from the menu or enters the string replacing this subformula. The program checks the correctness of the subformula selection (order of operations), applicability of the rule (or syntactical correctness of the entered part and equivalence with the marked part) and achievement of the goal. We used this program for exercises and tests for many years until a Java version together with addition of tasks of Predicate logic was implemented [8]. In the middle of nineties we tried to design in Turbo Pascal 7 an analogue for polynomials in school mathematics but we were unable to create satisfactory screen representation and an editor for expressions. During the subsequent ten years the computer algebra community has solved those problems and we can already speak about some standards.

In 2004 we launched the T-algebra project to design and implement an interactive learning environment for expression manipulation tasks in four areas:

a)      calculation of the values of numerical expressions;

b)      operations with fractions;

c)      solving of linear equations, inequalities and linear equation systems;

d)      simplification and factoring of polynomials.

Our main goal is to design an environment that is able to understand the intentions of the student and to diagnose separately erroneous choice of transformation and inaccurate execution.

Action-Object-Input Interface

In T-algebra the students solve problems step by step. At each step the student should apply a certain rule, but unlike other rule-based environments T-algebra does not apply the rule automatically. The student has to make all the essential decisions himself when applying the rule. A special solution step dialogue was designed for that purpose [2, 4]. Each solution step in T‑algebra consists of three stages:

1)      selecting a transformation rule (action),

2)      marking the parts of the expression (object),

3)      entering the result of the application of the selected rule (input).

Hereafter we will refer to this scheme as the Action-Object-Input scheme, after its three stages. In this scheme the student first decides which rule to apply and selects it from the list of available rules, and then selects parts of the expression to apply the rule to them. T-algebra copies unchanged parts of the expression and lets the student to enter the result of application of the rule.

Figure 1 shows the problem solution window of T-algebra when choosing a rule. The main part of the window contains previous problem solution steps and a virtual keyboard for marking the objects. On the right side is a menu of possible actions. The lower part includes instructions for the student in this particular situation.

Figure 1. The problem-solution window of the T-algebra program

When solving the same problem on paper according to the algorithm taught at school the student would first open parentheses, second move all variable terms to the left and all constant terms to the right, then combine like terms, etc. The solution process in T-algebra follows exactly the same steps (same rules are applied). When making the first step on paper the student would first decide which rule to apply (open parentheses) and then think which parentheses to open – multiply by a constant (in this example there is only one such pair). Then he would write the whole resulting equation on the next line in his solution. When applying the same rule in T-algebra the student has to record his decisions directly. The example on Figure 2 shows different stages of application of the rule Open parentheses:

1)      At the first stage the student selects the rule Open parentheses from the rule list – the program allows selecting any rule without checking whether it is possible to apply such a transformation at that stage or not.

2)      Then the student marks the part of the expression – the product that contains the constant and the parentheses to open. The program checks whether the selected parts of the expression are the constant and the parentheses expression of the same product.

3)      After confirmation of the marking, the program copies unchanged parts of the expression onto the next line and asks the student to enter the result. The third stage has the largest selection of potential mistakes, because the student must apply the rule for marked parts and enter the result.

Figure 2. Three stages of the step

There are three different input modes implemented for the third stage of each step: free input, structured input and partial input [3]. Depending on the input mode used, T-algebra offers different number and types of boxes for entering the result of application of the rule.

In the free mode the whole resulting subexpression is entered into one single box. This is the most general input mode and it is implemented in the same way for all the available rules. Only a few restrictions on the kind of expression are applied (for example, only equation and inequality signs are not permitted when a monomial or polynomial expression should be entered).

The structured and partial input modes are rule-specific. Their implementation depends on the rule and objects selected – having this information, T-algebra calculates the resulting expression itself and offers the corresponding structure for filling out. It offers separate boxes for single numbers, coefficients of monomials, variables and powers. In the partial input mode T-algebra fills some parts of the result automatically – the user has to enter only the essential parts of the result (for example, in the case of combining like terms only the coefficient of the monomial and the operation sign should be entered, variables and their powers are filled automatically). By offering the structure, T-algebra helps the student to some extent and provides a possibility for indicating the exact position of an error.

Figure 3 shows three different input modes of the rule Combine like terms applied to expression . The student selects  and  as operands. Different modes are shown in the picture from left to right: free input (the whole monomial into one box), structured input (the structure for entering the whole monomial is given) and partial input (boxes are given only for input of rule-specific components, the sign and coefficient in the current example). In the picture the boxes are already filled with the correct result.

Figure 3. Three different input modes of rule Combine like terms

The three-stage dialogue has several advantages over environments with pure input when it comes to generation of feedback and diagnostics:

1)      the program can explicitly and without guessing evaluate the student’s decisions about the operation and operands,

2)      the information received from the first two stages can be used for generating hints at the third stage and for predicting the structure of the input (in structured and partial mode),

3)      the program can compute the result corresponding to the input of the first two stages, and compare it with the input of the third stage,

4)      the formulated error messages point exactly to incorrect parts.

Consider now the diagnostics more precisely.

Diagnostics Possibilities

In a pure-input dialogue the first object of checking is the syntactical correctness of the whole expression and then the equivalence of the received expression/equation/system with the previous stage of the solution. It is very hard to diagnose more precisely the reasons of non‑equivalence without knowing the intentions of the student. Suitability of a step for any particular task can be measured indirectly – it is possible to detect what subgoals of the solution algorithm are already reached and how many steps are needed to reach any of the remaining goals. What can we check knowing Action, Object and Input?

1. Choice of the rule

It is possible to check the first two issues immediately after a rule has been selected:

1)      Is it possible to apply this rule to the actual expression?

2)      Does the choice of the rule correspond to the algorithm to be learned for actual task type (if such exists)?

In T-algebra we prefer to check the rule only after marking of operands. Sometimes the students know the algorithm well and after finishing one step automatically select the next one. We want to give them the possibility to cancel the step themselves if they see that the rule cannot be applied. Concerning the second issue our program displays a warning if the selected rule does not correspond to the „official” algorithm and is not a simplification rule. The student can continue or cancel the rule.

2. Marking of operands

Before any rule-associated considerations the program should check that the marked parts are syntactically correct subexpressions. If the selected part itself is not a correct expression then usually the student has made some technical mistake with the mouse or some oversight error with details that are not very important for using the rule (for instance, including only one of the two brackets). However, incompetent pupils can mark completely meaningless segments of the expression. If the marked part itself is an expression (syntactically correct) but not a subexpression (mathematically incorrect) then the situation is usually serious. The student has misunderstood the order of operations or some other syntactical convention.

After checking the general syntactical suitability, the program can detect misunderstanding of a particular rule, checking the suitability of the marked part(s) for this rule. In many rules the operands should have a specific form. The like terms for combining should be monomials. Some rules can be applied only to polynomials with more than one member, etc.

Further, many rules have some compatibility requirements for their operands. The pupils can try to cancel the numbers that do not have a common factor (Figure 4) or to combine terms that are not similar.

Figure 4. Error message during marking of the operands (unsuitable operands)

Finally, there are requirements about the positions of operands. The operand cannot be a member of some product in the case of additive moving to the other side of the equation (Figure 5). Like terms for combining should belong to the same sum.

Figure 5. Error message during marking of the operands (operands in unsuitable positions)

The mistakes described here are quite common for pupils. Precise detection of these errors is quite simple when the operands and the rule are identified explicitly. Conversely, it is very difficult to guess automatically the causes of wrong expression if the program can analyse only the result but the mistake is caused by inconsistent choice of operands.

3. Syntactical correctness and equivalence

The programs with a pure-input interface can check only the next entered line as a whole. On the one hand they do not know what part of the expression was the object of conversion (some other parts of the expression may have been changed as a result of a mistake). On the other hand namely the correctness of the whole next line and its equivalence with the previous line are necessary for the correctness of the step. However, the essence of expression manipulation is replacing some subexpression(s) with an equivalent subexpression. Correspondingly, the main object of checking in T-algebra is the entered part of the expression. Syntactical correctness of the entered expression does not guarantee syntactical correctness of the whole resulting expression and the equivalence of the entered part with the marked part does not guarantee equivalence of complete expressions. As we can see below, there may be cases when the entered part is correct/equivalent but the whole expression is not and such facts help us to detect some common mistakes.

Our program always checks the syntactical correctness of the entered part of the expression and requires correction if needed. T-algebra checks also the syntactical correctness of the whole resulting expression because in some cases the position of the operands causes additional requirements. For example, we combine -2x+5x to 3x, but if the same objects were chosen for combining like terms in y-2x+5x, then +3x should be entered as result.

Next, the equivalence of the entered part with the marked part is checked. In addition the program should check here for one more possible mistake. If the operation of the next level has higher priority than the main operation of the entered expression then the latter should be included in brackets. For instance, although (a+b)c = ac+bc, the brackets should be added if this operation was made in 2·(a+b)c.

The program also checks whether the entered parts are equivalent to the parts calculated by the T-algebra. If the expressions are not equivalent, the program checks the correctness of each entered part to produce a more specific diagnosis. In most cases T-algebra is able to diagnose the exact position of an error and show it to the student (red box on the figure 6).

Figure 6. Diagnosis of the exact position of the error

4. Rule-specific checks

In many cases the rule and the operands determine the resulting expression much more precisely than modulo equivalence of expressions. For example, the result of combining like terms should be a monomial; the result of multiplication of two polynomials should contain certain number of monomials (Figure 7), etc. The cancelled factors in the numerator and denominator of the fraction should be less than the corresponding members of the original fraction, etc. Thus the explicit information about the operation and the operands enables to check two further issues (without a clear distinction in the case of some rules):

a)      Does the result have the form required by this particular rule?

b)      Is the operation really performed?

Figure 7. Error message during entering of the result (result with unsuitable form)

The exact content of the requirements a) and b) depends on the particular rule and should be implemented for each rule separately.

Assessment Scheme

In the rule-based Action-Object-Input interface both positive and negative aspects of the student’s performance can be measured in quite a natural way. We describe here the assessment of a solution of one task only.

The positive aspect indicates how far the student has progressed in creating the solution. If the answer was accepted then it is natural to give full points (if a step is accepted in the T-algebra environment then it must be correct). If the student did not reach the end of the solution then it is possible to see how much of the solution path the student did cover. T-algebra implements two tools for such measuring. First, it is possible to check what stages of the “official” solution algorithm were reached and how much remains to do. The second tool is an automated solution of the task starting from the initial expression/equation/system or from the endpoint of the student’s solution.

In order to get a numerical measure, we can assign static weights to each step of the algorithm for the certain task type. However, we can also calculate the weights of the algorithm steps dynamically (for certain task) by counting the number of solution steps and rules used in the solution created automatically by T-algebra. In this case the weight of each algorithm step depends on the amount of work to be done (the number of rules that should be applied). This weight can be even zero or have some default minimal value if this step requires no application of any rules. There are also some "creative" problem types without a prescribed algorithm (factorisation) and problem types that are devoted to drilling one single step of the algorithm and assume iterated use of only one rule (for example Combine like terms). In this case the progress can be measured by comparing the length of the path from the last accepted expression to the final result and the length of the full solution created by T-algebra.

Consider now the assessment of the negative aspect. We saw that many (most) mistakes get quite clear diagnostics. We have grouped similar mistakes into a smaller number of types. The program counts the number of mistakes of each type. The program has some default penalties (defined in percents of the price of one step in the solution created by T-algebra) for each type. For example, if the teacher gives the task the weight 8 points, the length of the solution of T‑algebra is 7 steps and the penalty rate is 50% then the program subtracts 4/7 points in this task for each mistake of this type. The teacher can also assign his own penalties. The penalties can be given also for asking hints or for automatic execution of the first, second and third stage of the step. In this way the program can calculate the points that the student collects for each task.

Rule-based standardisation of the steps makes it possible also to take into account the length of the student's solution (compared to automated solution) but we have not implemented this possibility.

References

   [1]    Beeson, M. (1998). Design Principles of Mathpert: Software to support education in algebra and calculus. In: Kajler, N. (ed.): Computer-Human Interaction in Symbolic Computation. Springer-Verlag, 89-115.

   [2]    Issakova, M. & Lepp, D. (2004). Rule dialogue in problem solving environment T-algebra. In: Proceedings TIME-2004: Montreal International Symposium on Technology and its Integration into Mathematics Education. Montreal, Canada, 16p.

   [3]    Issakova, M., Lepp, D., Prank, R. (2005). Input Design in Interactive Learning Environment T-algebra. In: Proceedings ICALT-2005: The 5th IEEE International Conference on Advanced Learning Technologies. Kaohsiung, Taiwan, 489-491.

   [4]    Lepp, D. (2005). Extended Solution Step Dialogue in Problem Solving Environment T-Algebra. In: Proceedings of the 7th International Conference on Technology in Mathematics Teaching, Bristol, UK, Vol. 1, 267-274.

   [5]    Nicaud, J., Bouhineau, D., & Chaachoua, H. (2004). Mixing microworld and cas features in building computer systems that help students learn algebra. International Journal of Computers for Mathematical Learning, 5(2), 169-211.

   [6]    Prank, R. (1991). Using Computerised Exercises on Mathematical Logic. Informatik-Fachberichte, Vol. 292, Springer-Verlag, 34-38.

   [7]    Prank, R. & Viira, H. (1991). Algebraic Manipulation Assistant for Propositional Logic. Computerised Logic Teaching Bulletin, St Andrews Univ, 4(1), 13-18.

   [8]    R.Prank & V.Vaiksaar. (2003). Expression manipulation environment for exercises and assessment. 6th International Conference on Technology in Mathematics Teaching. Volos-Greece, October 2003, 342-348.

   [9]    Ravaglia, R., Alper, T., Rozenfeld, M. & Suppes, P. (1998). Succesful pedagogical applications of symbolic computation. In: Kajler, N. (ed.) Computer-Human Interaction in Symbolic Computation, Springer-Verlag, 61-88.