Blackboard-Pool-Generator.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Generate Pool files for upload into Blackboard"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Software to use\n",
    "\n",
    "This jupyter notebook uses a SageMath 9.0 kernel. It should be fairly easy to adapt this to python 3.7+, as that is basically what it is. In principle, you could achieve the same functionality using any other programming language, and even using a spreadsheet software. The description below applies to those alternatives as well.\n",
    "\n",
    "## Limitations\n",
    "\n",
    "This code will create pools of multiple choice question variations. These can contain a limited amount of LaTeX formulas via [MathJax](https://www.mathjax.org/). At this pointimages is not supported in any form, so the tests are text only.\n",
    "\n",
    "\n",
    "## The output generated by the code in this notebook\n",
    "\n",
    "The format for multiple choice question pools generated below is basically \n",
    "\n",
    "- a list of individual question/answer variations\n",
    "- each such variation is a list of the format\n",
    "    - question text\n",
    "    - correct answer\n",
    "    - wrong answer 1\n",
    "    - wrong answer 2\n",
    "    - wrong answer 3\n",
    "\n",
    "A fourth answer \"None of the others\" will be added automatically. The resulting list of variations is converted into a TAB separated text table (TSV format) with one question per row, each row starting with \"MC\", and for each answer stating either \"correct\" or \"incorrect\". This format, along with that of other possible question types not discussed here, is defined [here](https://uonline.newcastle.edu.au/webapps/blackboard/execute/viewExtendedHelp?helpkey=question_upload&pluginId=_1202_1&ctx=course&course_id=_1408028_1).\n",
    "\n",
    "The correct and wrong answers 1--3 are shuffled randomly, the fourth option \"None of the others\" is added at the end. \n",
    "\n",
    "## Uploading the pool file to blackboard\n",
    "\n",
    "To import the resulting pool file into Blackboard, go to \n",
    "`Course Management` -> `Course Tools` -> `Tests, Surveys, and Pools` -> `Pools`. Now click `Build Pool`. I recommend the filename of the TSV file you are about to upload as the question name, and to have uniform naming scheme like `MATH1110-Quiz 01-Question 1.txt`. It is absolutely important that your filename ends in `.txt` or Blackboard will not accept it. Click `submit`. \n",
    "\n",
    "On the next screen click `Upload questions`, then browse to your pool file and upload it. Do not bother with the points per question, as Blackboard does not actually use this setting anyways. On the following screen you see a list of your questions. Do not change any settings here, it is not worth your time, trust me.\n",
    "\n",
    "## Creating a test in Blackboard\n",
    "\n",
    "To create a test, you need multiple pool files as described above and as generated below. One per question that you want to have in your test. Each pool will provide random variants for a single question.\n",
    "\n",
    "To create a test in a \"Content Area\" (Blackboard lingo for a page), click on `Assessments` -> `Test` -> `Create` and give it a name.\n",
    "\n",
    "__Now the important part__: In both the `Description` and `Instructions` fields, do first click the HTML button so that a HTML source editor window pops up. Paste the following text and click submit.\n",
    "\n",
    "    <div>\n",
    "    <script type=\"text/javascript\" async=\"\" src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-MML-AM_CHTML\"></script>\n",
    "    </div>\n",
    "\n",
    "If you do not do this, your test will not show formulas typeset in LaTeX correctly. \n",
    "\n",
    "Submit the title/description/instructions page (don't bother putting actual descriptions or instructions in here, it is more efficient to keep these separate in a text area, centrally for all tests, in the enclosing Content Area; this is particularly true if you have to delete and regenerate your test multiple times, while debugging it).\n",
    "\n",
    "On the next screen click `Reuse question` -> `Create Random Block`. Under Criteria then choose the file you have uploaded, and in that same column but further below select `All Pool Questions`. Then confirm. \n",
    "\n",
    "Now your test has one question. Add the other questions in the same way. \n",
    "\n",
    "Lastly, with your test loaded with a number of questions, assign points per question. Confirm.\n",
    "\n",
    "Now it remains to schedule your test. Don't forget to make it available. \"Force completion\" seems like a reasonable approach to replicate in-class tests that have a time limit. Notable is that you can define exceptions, like additional time, for particular students _or groups_, e.g. all your students with reasonable adjustment plans, or those with approved adverse circumstances."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a random pool in SageMath"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import a few python libraries with helper functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import itertools, functools\n",
    "import random as py_random\n",
    "import re"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the central function that converts lists of questions into the right format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('MC',\n",
       " 'Rewriting the expression \\\\(\\\\mbox{Re} (z^{2} + \\\\overline{z} + 1) = -1\\\\) using the complex variable \\\\(z=x+iy\\\\) (with \\\\(x,y\\\\in\\\\mathbb{R}\\\\)), we obtain',\n",
       " '\\\\(4 \\\\, x^{2} - 4 \\\\, y^{2} + 3 \\\\, x + 2 = \\\\left(-1\\\\right)\\\\)',\n",
       " 'incorrect',\n",
       " '\\\\(2 \\\\, x^{2} - 2 \\\\, y^{2} + 3 \\\\, x + 2 = 1\\\\)',\n",
       " 'incorrect',\n",
       " '\\\\(x^{2} - y^{2} + x + 1 = \\\\left(-1\\\\right)\\\\)',\n",
       " 'correct',\n",
       " '\\\\(3 \\\\, x^{2} - 3 \\\\, y^{2} + 4 \\\\, x + 2 = 1\\\\)',\n",
       " 'incorrect',\n",
       " 'None of the others',\n",
       " 'incorrect')"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def make_BB_row(row):\n",
    "    '''Convert a list-like object of the form (question,correct_answer,wrong_answer_1,...,wrong_answer_3)\n",
    "    that may contain formulas typeset in LaTeX and linebreaks into a a tuple of the format \n",
    "    (MC, question, random_answer1, \"correct/incorrect\", ..., random_answer4, \"correct/incorrect\", \"None of the others\", \"incorrect\").\n",
    "    \n",
    "    The shuffling of the answers and the insertion of the last option is automatic and different \n",
    "    in each invocation of this function. All formulas are escaped to play nicely with MathJax. All \n",
    "    newlines are replaced with normal spaces, to comply with the format requirements of Blackboard.'''\n",
    "    row = map(lambda s: re.sub(r',',r', ',s), row)\n",
    "    row = map(lambda s: re.sub(r'\\\\([a-z]+)\\b', r' \\\\\\1 ',s), row)\n",
    "    row = map(lambda s: re.sub(r'}{', r'} {',s), row)\n",
    "    row = map(lambda s: re.sub(r'([=<>/+-])', r' \\1 ',s), row)\n",
    "\n",
    "    row = map(lambda s: re.sub(r'\\$\\$(.*?)\\$\\$',r'\\\\[\\1\\\\]',s), row)\n",
    "    row = map(lambda s: re.sub(r'\\$(.*?)\\$',r'\\\\(\\1\\\\)',s), row)\n",
    "    row = map(lambda s: re.sub(r'\\\\displaystyle\\b',r' ',s), row)\n",
    "    row = map(lambda s: re.sub(r'\\\\dfrac\\b',r'\\\\frac',s), row)\n",
    "    row = map(lambda s: re.sub('[\\n ]+',' ',s), row)\n",
    "    question, *answers = tuple(row)\n",
    "    answers = list(zip(answers, ('correct',*('incorrect',)*3)))\n",
    "    py_random.shuffle(answers)\n",
    "    answers = itertools.chain(*answers)\n",
    "    answers = list(answers)\n",
    "    return ('MC',question, *answers, 'None of the others', 'incorrect')\n",
    "\n",
    "make_BB_row(('''Rewriting the expression \n",
    "                $\\\\mbox{Re} (z^{2} + \\\\overline{z} + 1) = -1$ \n",
    "                using the complex variable $z=x+iy$ (with $x,y\\\\in\\\\mathbb{R}$), we obtain''',\n",
    "              '\\\\(x^{2} - y^{2} + x + 1 = \\\\left(-1\\\\right)\\\\)',\n",
    "              '\\\\(3 \\\\, x^{2} - 3 \\\\, y^{2} + 4 \\\\, x + 2 = 1\\\\)',\n",
    "              '\\\\(4 \\\\, x^{2} - 4 \\\\, y^{2} + 3 \\\\, x + 2 = \\\\left(-1\\\\right)\\\\)',\n",
    "              '\\\\(2 \\\\, x^{2} - 2 \\\\, y^{2} + 3 \\\\, x + 2 = 1\\\\)'),)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now to the generation of an example question\n",
    "\n",
    "We start by randomly generating questions and correct answers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "question_answer_list = [\n",
    "    (  \n",
    "        f\"What is the derivative $f'(x)$ of\\n $$f(x) = {latex(f)}$$?\", # Question\n",
    "        f\"${latex(diff(f,x))}$\" # correct answer\n",
    "    )\n",
    "    for f in [\n",
    "        sum(randint(1,9)*x**k for k in range(5)) for variation in range(100)] # make 100 random polynomials\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The trick now is to ensure that answers and questions are unique. We achieve this by essentially converting both to sets (keys of a dictionary, which must be unique). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# now make the questions unique\n",
    "question_answer_list = dict(question_answer_list) \n",
    "\n",
    "# make the answers unique by changing keys for values in the dictionary\n",
    "question_answer_list = {v:k for k,v in question_answer_list.items()}\n",
    "\n",
    "# switch back\n",
    "question_answer_list = {v:k for k,v in question_answer_list.items()}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that the answers are unique, we randomly select answers of other variants as distractors for each variant."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# now add wrong answers\n",
    "all_answers = set(question_answer_list.values())\n",
    "question_answer_list = [(\n",
    "    question,\n",
    "    correct_answer,\n",
    "    *py_random.sample(all_answers.difference({correct_answer}), 3)\n",
    ") for question, correct_answer in question_answer_list.items()]\n",
    "\n",
    "\n",
    "# question_answer_list is now in the required format"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we inspect the fruits of our work."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(\"What is the derivative $f'(x)$ of\\n $$f(x) = 3 \\\\, x^{4} + 9 \\\\, x^{3} + 2 \\\\, x^{2} + 3 \\\\, x + 6$$?\",\n",
       "  '$12 \\\\, x^{3} + 27 \\\\, x^{2} + 4 \\\\, x + 3$',\n",
       "  '$12 \\\\, x^{3} + 9 \\\\, x^{2} + 14 \\\\, x + 2$',\n",
       "  '$8 \\\\, x^{3} + 24 \\\\, x^{2} + 16 \\\\, x + 5$',\n",
       "  '$28 \\\\, x^{3} + 21 \\\\, x^{2} + 14 \\\\, x + 2$'),\n",
       " (\"What is the derivative $f'(x)$ of\\n $$f(x) = 5 \\\\, x^{4} + 5 \\\\, x^{3} + x^{2} + 2 \\\\, x + 7$$?\",\n",
       "  '$20 \\\\, x^{3} + 15 \\\\, x^{2} + 2 \\\\, x + 2$',\n",
       "  '$16 \\\\, x^{3} + 9 \\\\, x^{2} + 16 \\\\, x + 2$',\n",
       "  '$16 \\\\, x^{3} + 15 \\\\, x^{2} + 16 \\\\, x + 1$',\n",
       "  '$8 \\\\, x^{3} + 3 \\\\, x^{2} + 8 \\\\, x + 4$'),\n",
       " (\"What is the derivative $f'(x)$ of\\n $$f(x) = 7 \\\\, x^{4} + 9 \\\\, x^{3} + 6 \\\\, x^{2} + x + 5$$?\",\n",
       "  '$28 \\\\, x^{3} + 27 \\\\, x^{2} + 12 \\\\, x + 1$',\n",
       "  '$8 \\\\, x^{3} + 12 \\\\, x^{2} + 6 \\\\, x + 1$',\n",
       "  '$32 \\\\, x^{3} + 15 \\\\, x^{2} + 8 \\\\, x + 3$',\n",
       "  '$24 \\\\, x^{3} + 21 \\\\, x^{2} + 12 \\\\, x + 2$'),\n",
       " (\"What is the derivative $f'(x)$ of\\n $$f(x) = x^{4} + 4 \\\\, x^{3} + 5 \\\\, x^{2} + 8 \\\\, x + 3$$?\",\n",
       "  '$4 \\\\, x^{3} + 12 \\\\, x^{2} + 10 \\\\, x + 8$',\n",
       "  '$36 \\\\, x^{3} + 6 \\\\, x^{2} + 6 \\\\, x + 1$',\n",
       "  '$28 \\\\, x^{3} + 27 \\\\, x^{2} + 12 \\\\, x + 1$',\n",
       "  '$20 \\\\, x^{3} + 21 \\\\, x^{2} + 12 \\\\, x + 2$'),\n",
       " (\"What is the derivative $f'(x)$ of\\n $$f(x) = 9 \\\\, x^{4} + 3 \\\\, x^{3} + 2 \\\\, x^{2} + 4 \\\\, x + 5$$?\",\n",
       "  '$36 \\\\, x^{3} + 9 \\\\, x^{2} + 4 \\\\, x + 4$',\n",
       "  '$8 \\\\, x^{3} + 24 \\\\, x^{2} + 16 \\\\, x + 5$',\n",
       "  '$12 \\\\, x^{3} + 3 \\\\, x^{2} + 16 \\\\, x + 2$',\n",
       "  '$28 \\\\, x^{3} + 21 \\\\, x^{2} + 14 \\\\, x + 2$')]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# inspect first five variants\n",
    "question_answer_list[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now write the random questions into a Blackboard pool file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('my-first-random-pool.txt','w') as f:\n",
    "    f.write(\"\\n\".join(\"\\t\".join(make_BB_row(row)) for row in question_answer_list))    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lastly, for the sake of documentation, we show the first few lines of our pool file here. This is what Blackboard will see."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MC\tWhat is the derivative \\(f'(x)\\) of \\[f(x) = 3 \\, x^{4} + 9 \\, x^{3} + 2 \\, x^{2} + 3 \\, x + 6\\]?\t\\(12 \\, x^{3} + 27 \\, x^{2} + 4 \\, x + 3\\)\tcorrect\t\\(12 \\, x^{3} + 9 \\, x^{2} + 14 \\, x + 2\\)\tincorrect\t\\(8 \\, x^{3} + 24 \\, x^{2} + 16 \\, x + 5\\)\tincorrect\t\\(28 \\, x^{3} + 21 \\, x^{2} + 14 \\, x + 2\\)\tincorrect\tNone of the others\tincorrect\r\n",
      "MC\tWhat is the derivative \\(f'(x)\\) of \\[f(x) = 5 \\, x^{4} + 5 \\, x^{3} + x^{2} + 2 \\, x + 7\\]?\t\\(16 \\, x^{3} + 15 \\, x^{2} + 16 \\, x + 1\\)\tincorrect\t\\(16 \\, x^{3} + 9 \\, x^{2} + 16 \\, x + 2\\)\tincorrect\t\\(8 \\, x^{3} + 3 \\, x^{2} + 8 \\, x + 4\\)\tincorrect\t\\(20 \\, x^{3} + 15 \\, x^{2} + 2 \\, x + 2\\)\tcorrect\tNone of the others\tincorrect\r\n",
      "MC\tWhat is the derivative \\(f'(x)\\) of \\[f(x) = 7 \\, x^{4} + 9 \\, x^{3} + 6 \\, x^{2} + x + 5\\]?\t\\(28 \\, x^{3} + 27 \\, x^{2} + 12 \\, x + 1\\)\tcorrect\t\\(24 \\, x^{3} + 21 \\, x^{2} + 12 \\, x + 2\\)\tincorrect\t\\(32 \\, x^{3} + 15 \\, x^{2} + 8 \\, x + 3\\)\tincorrect\t\\(8 \\, x^{3} + 12 \\, x^{2} + 6 \\, x + 1\\)\tincorrect\tNone of the others\tincorrect\r\n",
      "MC\tWhat is the derivative \\(f'(x)\\) of \\[f(x) = x^{4} + 4 \\, x^{3} + 5 \\, x^{2} + 8 \\, x + 3\\]?\t\\(36 \\, x^{3} + 6 \\, x^{2} + 6 \\, x + 1\\)\tincorrect\t\\(20 \\, x^{3} + 21 \\, x^{2} + 12 \\, x + 2\\)\tincorrect\t\\(28 \\, x^{3} + 27 \\, x^{2} + 12 \\, x + 1\\)\tincorrect\t\\(4 \\, x^{3} + 12 \\, x^{2} + 10 \\, x + 8\\)\tcorrect\tNone of the others\tincorrect\r\n",
      "MC\tWhat is the derivative \\(f'(x)\\) of \\[f(x) = 9 \\, x^{4} + 3 \\, x^{3} + 2 \\, x^{2} + 4 \\, x + 5\\]?\t\\(8 \\, x^{3} + 24 \\, x^{2} + 16 \\, x + 5\\)\tincorrect\t\\(28 \\, x^{3} + 21 \\, x^{2} + 14 \\, x + 2\\)\tincorrect\t\\(12 \\, x^{3} + 3 \\, x^{2} + 16 \\, x + 2\\)\tincorrect\t\\(36 \\, x^{3} + 9 \\, x^{2} + 4 \\, x + 4\\)\tcorrect\tNone of the others\tincorrect\r\n",
      "MC\tWhat is the derivative \\(f'(x)\\) of \\[f(x) = 2 \\, x^{4} + 5 \\, x^{3} + 6 \\, x^{2} + 8 \\, x + 8\\]?\t\\(8 \\, x^{3} + 15 \\, x^{2} + 12 \\, x + 8\\)\tcorrect\t\\(12 \\, x^{3} + 24 \\, x^{2} + 16 \\, x + 5\\)\tincorrect\t\\(32 \\, x^{3} + 3 \\, x^{2} + 16 \\, x + 5\\)\tincorrect\t\\(36 \\, x^{3} + 6 \\, x^{2} + 6 \\, x + 1\\)\tincorrect\tNone of the others\tincorrect\r\n",
      "MC\tWhat is the derivative \\(f'(x)\\) of \\[f(x) = 8 \\, x^{4} + 7 \\, x^{3} + 6 \\, x^{2} + 9 \\, x + 8\\]?\t\\(24 \\, x^{3} + 21 \\, x^{2} + 12 \\, x + 2\\)\tincorrect\t\\(24 \\, x^{3} + 18 \\, x^{2} + 12 \\, x + 3\\)\tincorrect\t\\(16 \\, x^{3} + 27 \\, x^{2} + 8 \\, x + 7\\)\tincorrect\t\\(32 \\, x^{3} + 21 \\, x^{2} + 12 \\, x + 9\\)\tcorrect\tNone of the others\tincorrect\r\n",
      "MC\tWhat is the derivative \\(f'(x)\\) of \\[f(x) = 9 \\, x^{4} + 5 \\, x^{3} + 7 \\, x^{2} + 8 \\, x + 9\\]?\t\\(20 \\, x^{3} + 9 \\, x^{2} + 14 \\, x + 1\\)\tincorrect\t\\(4 \\, x^{3} + 6 \\, x^{2} + 16 \\, x + 9\\)\tincorrect\t\\(36 \\, x^{3} + 15 \\, x^{2} + 14 \\, x + 8\\)\tcorrect\t\\(20 \\, x^{3} + 15 \\, x^{2} + 2 \\, x + 2\\)\tincorrect\tNone of the others\tincorrect\r\n",
      "MC\tWhat is the derivative \\(f'(x)\\) of \\[f(x) = 2 \\, x^{4} + 4 \\, x^{3} + 3 \\, x^{2} + x + 7\\]?\t\\(32 \\, x^{3} + 27 \\, x^{2} + 2 \\, x + 4\\)\tincorrect\t\\(36 \\, x^{3} + 9 \\, x^{2} + 16 \\, x + 3\\)\tincorrect\t\\(8 \\, x^{3} + 12 \\, x^{2} + 6 \\, x + 1\\)\tcorrect\t\\(4 \\, x^{3} + 12 \\, x^{2} + 10 \\, x + 8\\)\tincorrect\tNone of the others\tincorrect\r\n",
      "MC\tWhat is the derivative \\(f'(x)\\) of \\[f(x) = 9 \\, x^{4} + x^{3} + x^{2} + 8 \\, x + 1\\]?\t\\(36 \\, x^{3} + 24 \\, x^{2} + 10 \\, x + 1\\)\tincorrect\t\\(36 \\, x^{3} + 3 \\, x^{2} + 2 \\, x + 8\\)\tcorrect\t\\(20 \\, x^{3} + 24 \\, x^{2} + 18 \\, x + 3\\)\tincorrect\t\\(4 \\, x^{3} + 24 \\, x^{2} + 8 \\, x + 2\\)\tincorrect\tNone of the others\tincorrect\r\n"
     ]
    }
   ],
   "source": [
    "! head my-first-random-pool.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "SageMath 9.0",
   "language": "sage",
   "name": "sagemath"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}