cnx-baking-exercises-problems.md

Missing Exercises and Baking Problems

Issues

1. Alina's team reported that a book they are working on has missing exercises

   

   https://staging.cnx.org/a/content-status/b8d5607f-7db7-47ed-8ea7-8cedb24284f0

   showed that the content baked successfully:

   Version: 2.2     Created: 2019-11-13 17:01:36.727661-06:00     Recipe: 59b98fbdbe876512046a81998d9fc43727a4e8eb     State: SUCCESS
   Message:
   

2. and then afterwards they baked it again and:

   Version: 3.1     Created: 2019-11-14 15:03:52.575973-06:00     Recipe: 59b98fbdbe876512046a81998d9fc43727a4e8eb     State: FAILURE
   Message:
   
   Traceback (most recent call last):
     File "/var/cnx/venvs/publishing/local/lib/python2.7/site-packages/celery/app/trace.py", line 385, in trace_task
       R = retval = fun(*args, **kwargs)
     File "/var/cnx/venvs/publishing/local/lib/python2.7/site-packages/cnxpublishing/tasks.py", line 31, in __call__
       return super(PyramidAwareTask, self).__call__(*args, **kwargs)
     File "/var/cnx/venvs/publishing/local/lib/python2.7/site-packages/celery/app/trace.py", line 648, in __protected_call__
       return self.run(*args, **kwargs)
     File "/var/cnx/venvs/publishing/local/lib/python2.7/site-packages/cnxpublishing/db.py", line 74, in wrapped
       return func(*args, **kwargs)
     File "/var/cnx/venvs/publishing/local/lib/python2.7/site-packages/cnxpublishing/subscribers.py", line 168, in baking_processor
       bake(binder, recipe_id, publisher, message, cursor=cursor)
     File "/var/cnx/venvs/publishing/local/lib/python2.7/site-packages/cnxpublishing/db.py", line 70, in wrapped
       return func(*args, **kwargs)
     File "/var/cnx/venvs/publishing/local/lib/python2.7/site-packages/cnxpublishing/bake.py", line 74, in bake
       binder = collate_models(binder, ruleset=recipe, includes=includes)
     File "/var/cnx/venvs/publishing/local/lib/python2.7/site-packages/cnxepub/collation.py", line 61, in collate
       raw_html = io.BytesIO(bytes(html_formatter))
     File "/var/cnx/venvs/publishing/local/lib/python2.7/site-packages/cnxepub/formatters.py", line 299, in __str__
       return self.__bytes__()
     File "/var/cnx/venvs/publishing/local/lib/python2.7/site-packages/cnxepub/formatters.py", line 303, in __bytes__
       self.build()
     File "/var/cnx/venvs/publishing/local/lib/python2.7/site-packages/cnxepub/formatters.py", line 274, in build
       proc(elem)
     File "/var/cnx/venvs/publishing/local/lib/python2.7/site-packages/cnxepub/formatters.py", line 423, in _replace_exercises
       if exercise['total_count'] == 0:
   KeyError: u'total_count'
   

Analysis

Both issues are related to exercises, the first one shows that baking succeeded but exercises are not found on the exercise server. The second one happens in _replace_exercises.

We can find the exercises server that is used on staging by looking at cnx-deploy:

exercises_domain: exercises-staging.openstax.org

Reproducing the Error Locally

For errors related to exercises, we should be able to reproduce the error by using neb get and neb assemble.

Start a nebuchadnezzar container: (If you're using a locally installed neb, you can skip this part)

docker pull openstax/nebuchadnezzar
docker run -ti --rm -v $PWD:/out -w /out openstax/nebuchadnezzar /bin/bash

Find out the legacy collection id for the book by going to:

https://staging.cnx.org/contents/b8d5607f-7db7-47ed-8ea7-8cedb24284f0

and then click on (or have a look at the link at) "CNX Author | Legacy site" on the top right hand corner:

https://legacy-staging.cnx.org/content/m79617/1.2/?collection=col30444/1.7&legacy=true

The collection id is col30444.

So now run neb get:

neb get staging col30444 1.3

After that, neb assemble:

neb assemble --exercise-host exercises-staging.openstax.org col30444_1.3.1 col30444_1.3.1_assembled

It would show you the same KeyError as the server.

Using the Exercises API

Have a look at the screenshot for the first error again, the missing exercise is on page 1.5 Columbian Exchange.

Look at the "CNX Author | Legacy Site" link on the top right hand corner:

https://legacy-staging.cnx.org/content/m79621/1.2/?collection=col30444/1.7&legacy=true

The module is m79621, you can have a look at the module content at col30444_1.3.1/m79621/index.cnxml. The part that has the exercise looks like this:

<section id="fs-idm236276144" class="ap-practice">
<title>AP Practice Questions</title>
<exercise id="fs-idm237110128">
<problem id="fs-idm211969360">
<para id="fs-idm175428624"><link class="os-embed" url="#exercise/Ch01-N-ColumbExch-AP01"/></para>
</problem>
</exercise>
</section>

The code that contructs the exercises API is in cnx-publishing:

exercise_url_template = '{baseUrl}/api/exercises?q={field}:"{{itemCode}}"'

The variables are in cnx-deploy:

embeddables.exercise.base_url = https://{{ exercises_domain }}
embeddables.exercise.match =
  #ost/api/ex/,tag
  #exercise/,nickname
  #exercises/,nickname

So to fetch the exercise for #exercise/Ch01-N-ColumbExch-AP01, the url should be https://exercises-staging.openstax.org/api/exercises?q=nickname:Ch01-N-ColumbExch-AP01.

When I debugging the problem, this URL returned 500 Internal Server Error and that's why the exercises code failed.

Solution

Devops fixed the 500 Internal Server Error but the exercise was still missing on exercises-staging.openstax.org (the same url would work if I replace exercises-staging.openstax.org with exercises.openstax.org). Dante suggested syncing the exercises databases again (he said the tutor database is not the same as the exercises database so there might have been some confusion) and now it is fixed.