Implementing a Parsons-style Exam Problem

This guide describes two approaches taken to writing a Parsons problem:

A simple version using the normal code editor, demo-uni-finalexam-unscramble-lines;
A version using the new Parsons Problem editor, demo-uni-finalexam-parsons

If you cannot access these problems you need to request read access to our "Demo University" content via uni-support@groklearning.com.

What is a Parsons Problem?

A Parsons Problem asks a student to re-arrange lines of code to implement the required program behaviour. There are several variants - a good summary of Parsons Problems can be found in A Review of Research on Parsons Problems by Du et al..

Default Editor Implementation

This variant described here makes some design choices, either for ease of implementation or for pedagogical reasons.

For simplicity, the editor allows students to edit the lines of code.
All lines are provided without indentation, and the student is required to indent the each line as appropriate (this is mostly relevant for languages with whitespace semantics, such as Python, but could also be required on the basis of good style).
To avoid hinting from the syntax highlighting (e.g., about indentation or loop constructs), a plain-text editor is used.

Problem Description Content

The problem content should include a description of the desired behaviour. It could include examples of expected outcomes. To prevent students editing or running code, examples should be use the readonly and norun flags.

```norun;readonly<br>import csv<br>import itertools<br>```

The lines of code that the student should rearrange can be presented in the content panel as a reference, either as a single code block or as individual lines. Either way, use the readonly and norun flags as above.

Guidelines

It's useful to include some specific instructions to the student about how to complete the problem in Grok. E.g.,

> ## Guidelines 
> The lines of the function have been provided for you in the editor panel. Cut and paste the lines to rearrange them into the correct order, and indent them appropriately.
> * To submit, click the Mark button.
> * You can submit as many times as you like, but you will be assessed on you last submission.
> * If you encounter problems with the editor you can reset the editor file.

which is displayed as

Editor Workspace

The intended outcome here is to provide a text editor containing the lines to unscramble, plus optionally some read-only lines, as shown here:

Use un-runnable Text Language type

In order to prevent the code from being run, we can change the problem language from a Python to a plain text problem, via the button shown here

In the dialog you can choose "Text" as the language, and save.

Main submission file

By convention, the file into which the student enters their answers for a Text problem is called submission.txt. The .txt suffix instructs the editor not to use any syntax highlighting. This file should contain the scrambled lines that the user is required to rearrange.

Read-only lines

If you want to include some lines in the submission file that the student can't edit, you can use the text-marker API (see Read only and other text markers). In the example here, the imports are fixed at the top of the page. In the file settings the File Metadata (note, the Show advanced options? flag must be clicked at in the admin page for this to be visible) is set as shown:

This specifies that the first three lines (lines 0 to 2, inclusive) are read-only (the default option for text marker regions). The extra options prevent the user from selecting anywhere on the these lines, which avoids the student accidentally adding text to them before or after the read-only characters.

Solution

Even if you don't intend to make solutions available to students, it's worth adding at least one version of a working solution to the problem, since this makes validation of any tests much easier.

Testing

Acknowledge submission

Students should receive positive confirmation that they have submitted something.

If this is the only test, you can configure the common options to always pass by default:

Then include a single validation test that simply displays a confirmation message:

Line order

Manual marking can be made easier by checking that lines are in the correct order (even if indentation is wrong). This can be checked with an Assess test to prevent the student getting feedback during the exam.

Test Case Label (e.g.): line-order
Test Case Type: Assess
"On Pass" Message: Checking line order.
"On Fail" Message/"On Error" Message: (Automatic "Regular" message generated from On-Pass message)
Files:
1. This program dumps the lines of answer.txt to stdout.
  - File Type Normal In
  - File Path: program.py
  - File Content:
```
import sys
filename = sys.argv[1] if len(sys.argv)>1 else 'answer.txt'
with open(filename) as s:
  for line in s:
    line = line.strip()
    if line!="" and line[0]!='#':
      print(line)
			
```
2. Reference output.
  - File Type: stdout
  - File Content: The correct solution to the problem as outputted by the above "program.py" script
Test Options: If the common options are Always Pass, override the Test Options for this test to be
- Driver: Python 3 IO
- Output Checker: Differ

Example Outcomes

Manual marking can also be assisted by making sure that the submission functions as expected, using a few example situations.

Test Case Label (e.g.): example-outcomes
Test Case Type: Assess
"On Pass" Message: Checking example outcomes.
"On Fail" Message/"On Error" Message: (Automatic "Regular" message generated from On-Pass message)

Files:

This program dumps the output for the different test parameters, run through the code in answer.txt, to stdout.

File Type Normal In
File Path: program.py

File Content:

import sys
filename = sys.argv[1] if len(sys.argv)>1 else 'answer.txt'

with open(filename) as f:
  exec(f.read())

# This is just an example for this particular problem
# Write something specific to your own problem here.
# Note that the 'superheroes.csv' must also be included in the
# test workspace
for s in [('superheroes.csv', 1970)]:
  print(f">>> race_list('{s[0]}', {s[1]})")
  print(race_list(s[0], s[1]))

Reference output.
- File Type: stdout
- File Content: The correct solution to the problem as outputted by the above "program.py" script.

Test Options: If the common options are Always Pass, override the Test Options for this test to be
- Driver: Python 3 IO
- Output Checker: Differ

Parsons Problems Editor Implementation

The Parsons problem editor allows students to graphically construct a solution using a selection of line options. The editor dynamically updates the contents of the default file (program.py for a Python problem), allowing arbitrary tests to be applied to the code upon submissions just as for a normal code problem.

Change the Editor

To change from the default code editor, hit the Change button as indicated:

Then choose "Parsons" as the desired editor and then select Save.

Create the Parsons file

Add a new file parsons.json to the workspace and then click the "Edit Parsons problem" button to launch the Parsons editor.

Edit the problem

Within the editor you can populate the problem with the lines of code. Hit "Add Line" to create a new line to be included in the solution space, and insert the code for each line into the available textbox. Drag the line via the handle on the left in order to adjust the indentation. When you're all done, click "Insert Code" to update the "parsons.json" file.

Other options

If you want to provide additional lines to the problem space that are not part of the solution, use "Add Distractor" instead of "Add Line".
Changing "Indentable" to "Flat" prevents students from changing the indentation of lines, simplifying the problem.
Changing "Single Indentation" to "Double Indentation" alters the tab spacing from 2 to 4 spaces.
Changing "Random order" to "Initial order" forces the editor to present the available lines in the order provided, rather than randomising them. This can be useful in a Parsons problem where the solution is evaluated separately, but has little value in a slide-based editor.
Changing "Runnable" to "Non runnable" prevents the student from running the code. This is sometimes desirable in an exam.