Skip to content

Instantly share code, notes, and snippets.

Last active May 4, 2022 02:48
Show Gist options
  • Save legowerewolf/f34835c07acb06527358155e9ca66c4c to your computer and use it in GitHub Desktop.
Save legowerewolf/f34835c07acb06527358155e9ca66c4c to your computer and use it in GitHub Desktop.
Grading automation script for CptS 355

Grading helper

Basic use

  1. Make a new folder.
  2. Into this folder, move/copy the following:
    • helper.ps1
    • (from Canvas)
    • Your grading program files, probably written using a unit test framework. This example will use
  3. Open a PowerShell terminal in this folder.
  4. Run this command: helper.ps1

Advanced use

The script has some more advanced features made available with some command-line switches. Run Get-Help helper.ps1 -Detailed for more on those.

Testing Python with VS Code

VS Code has a testing interface built in. The Python extension provides a connection between it and any Python unit tests in your workspace.

You can install the Python extension by searching for ms-python.python in the Extensions sidebar panel.

To enable unittest unit tests for VS Code and correctly configure them for the workspace managed by the script, place this in your .vscode/settings.json file:

	"python.testing.unittestArgs": ["-v", "-s", "./workspace", "-p", "*"],
	"python.testing.unittestEnabled": true

and rename your grading test file to end with (Or change the last argument in the python.testing.unittestArgs setting.)

Adding support for another language

Right now, the script has support for Python and Haskell.

To add support for other languages, you need to know (1) what the file extension is for that language, and (2) what the executable is that will do anything it needs to to run that file.

For example, Python has the .py file extension and the python executable. Haskell has .hs and runghc.

Knowing that, find the $program variable declaration in the script and add a line to that block, using the established pattern. (The -match operator is a regex check; PowerShell uses ` as an escape character.)

#Requires -Version 7.0
Created by Duncan Gibson. This script is shared under the terms of the
CC Attribution-NonCommercial-ShareAlike 4.0 International license, which
may be found here:
The PowerShell help system can be invoked to show you how to use this
script by running `Get-Help helper.ps1`
This script was designed to help automate the grading of homework
assignments for Dr. Sakire Arslan Ay's CptS 355 class.
It expects there to be a "" file from Canvas in the working
directory. You can acquire one of these files by going to the assignment in
Canvas and clicking "Download Submissions" in the panel on the right.
It also expects you to have a file of unit tests that check your assignment.
This script will unzip the file and group the contents by
student. Then it will copy each student's files to a subdirectory named
"workspace", one student at a time. Submitted .zip files are a special
case; they will be unzipped and their contents added to the workspace.
The test file and other provided files will likewise be copied to the
Once all files for a student are in place, it will run the test file,
display its outputs, and pause. If you need to re-run the tests for a
student for some reason, open another terminal and run the test file in the
workspace. Workspace files are safe to edit without affecting other
The workspace is reset before the next student's submission is loaded.
WARNING: This script does not make guarantees about the safety of submitted
files. If you're feeling paranoid, you should probably use a VM.
This script is currently configured to run Haskell and Python programs.
More languages can be supported by adding a case for them in the
configuration section of the script, right below the parameter declaration.
param (
A list of files that we provide that should be in the workspace for
tests to run.
[string[]]$TesterProvidedFiles = @(),
A list of files that we expect to be in the submission. If a given file
isn't detected, the grader will be prompted to check the workspace and
correct the filename.
This list should, at minimum, include files that are depended on by the
main test file.
[string[]]$ExpectedFiles = @()
#################### START CONFIGURATION ###################
This specifies the program that gets run for the selected test file. You
shouldn't have to edit this unless Sakire uses a language other than Python or
$program = {
if ($TestFileName -match ".*`.hs") { return "runghc" }
if ($TestFileName -match ".*`.py") { return "python" }
##################### END CONFIGURATION ####################
# helper function for the script's interface
function Start-Pause {
param (
[ValidateLength(1, 1)]
Write-Output $Prompt
Start-Sleep -s 1
while ($Host.UI.RawUI.ReadKey('NoEcho,IncludeKeyDown').Character -notlike $ResumeChar) {}
# real start of the script
# verify and fix configuration
if ($TesterProvidedFiles.GetType().FullName -eq "System.String") { $TesterProvidedFiles = , $TesterProvidedFiles }
if ($ExpectedFiles.GetType().FullName -eq "System.String") {
$ExpectedFiles = , $ExpectedFiles
# purge existing files
Remove-Item "./submissions/" -Recurse -ErrorAction SilentlyContinue
Remove-Item "./workspace/" -Recurse -ErrorAction SilentlyContinue
# prep files and folder structure
Expand-Archive "./" -DestinationPath "./submissions/"
# group files by student
$submissiongroups = Get-ChildItem "./submissions/" | Where-Object Name -NotLike "*test*" | ForEach-Object -Begin { $groups = @{} } -Process {
$key = $_.Name.Split("_")[0]
if ($groups.ContainsKey($key)) {
$groups[$key] += $_
else {
$groups.Add($key, @($_))
} -End { $groups }
# loop over each student
foreach ($submission in ($submissiongroups.GetEnumerator() | Sort-Object -Property Name)) {
# reset the workspace
Remove-Item "./workspace/" -Recurse -Force -ErrorAction SilentlyContinue
New-Item -Path "./workspace" -ItemType Directory >$null
# copy our provided files to the workspace
foreach ($file in $TesterProvidedFiles) {
Copy-Item $file -Destination "./workspace/"
# copy the student's files to the workspace
foreach ($file in $submission.Value) {
$destname = $file.Name.Split("_")[-1]
$destname = $destname -replace "-\d+\.", "."
Copy-Item $file -Destination "./workspace/$destname"
if ($destname -like "*.zip") {
Expand-Archive "./workspace/$destname" -DestinationPath "./workspace/zipped/"
$zip_contents = Get-ChildItem "./workspace/zipped/"
if ($zip_contents.count -eq 1 -and $zip_contents[0].Mode -like "d*") {
$zip_contents = $zip_contents[0] | Get-ChildItem
$zip_contents | ForEach-Object { Move-Item -Path $_.FullName -Destination ".\workspace\" }
Remove-Item -Recurse .\workspace\zipped\
Remove-Item "./workspace/$destname"
# check that the submission has the expected filenames
foreach ($expected_file_name in $ExpectedFiles) {
if (-not (Test-Path "./workspace/$expected_file_name")) {
Start-Pause -ResumeChar 't' -Prompt "Warning! No $expected_file_name detected! Correct this, then press [t] to continue."
# copy the test file to the workspace
Copy-Item $TestFileName -Destination "./workspace/"
Write-Output "Loaded files for $($submission.Key)."
# set current working directory to the workspace
Set-Location "./workspace"
# run the test file
Invoke-Expression "$program $TestFileName"
# return to the original directory
Write-Output "Above results are for $($submission.Key)."
Start-Pause -ResumeChar 't' -Prompt (("=" * 20) + " Press [t] to continue to next student. " + ("=" * 20))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment