reedstrm

#Possible models of the new workspace

• Everything I could possibly edit (everything I have rights to edit - problematic for prolific authors, org accounts, and managers)
• Everything I could possibly edit that currently has a draft in flight
• Everything I have expressed an interest in editing (permissions + another bit)

Does new content have a diff. set of conditions?

• I don't thing so: adding an new editor (acl change) is the same for either, and is not a role request
• N.B. We do not display to the public in any place who is allowed to edit a given piece of content.

reedstrm

Recommendation for code versioning for OpenStax (née Connexions) code.

1. use git tags, of the form: v[numeric-version]
2. we tag complete (or at least, stable) features, for roll out to demo, qa, sprint and production a. exception: dev will roll from HEAD of master b. tags will be on the master branch, post merge (if other patchfix tags are needed, devops will tag on staging/production) c. Tags are done by developers, primarily, and include "in development" extra version info (see below)
   1. git describe at minimum
   2. bump versions as needed
3. devops will finalize tags (and versions) at rollout

reedstrm

A reasonable tool to generate SQL diffs for our databases is Pyrseas. Basically this tool works with existing databases, to generate a YAML representation that is more easily diffed, and can be used to auto-generate an SQL update file.

Pyrseas is a python module, and can be installed into your dev virtualenv w/ pip:

pip install pyrseas

This gives you two commands: dbtoyaml and yamltodb

To generate an sql update, you'll need a YAML representation of the new state, and the database to be updated. Let's assume you've

reedstrm

mjs="""{"publishers": [{"website": null, "hasAccepted": true, "surname": "Reedstrom", "firstname": "Ross", "title": "Dr", "emails": ["ross.reedstrom@rice.edu", "ross@cnx.org"], "id": "reedstrm", "requester": "reedstrm", "assignmentDate": "2015-02-10T14:57:51.543583-06:00", "fullname": "Ross Reedstrom", "type": "cnx-id", "email": ""}], "version": "draft", "derivedFrom": null, "abstract": "<div xmlns=\"http://www.w3.org/1999/xhtml\" xmlns:bib=\"http://bibtexml.sf.net/\" xmlns:data=\"http://dev.w3.org/html5/spec/#custom\" xmlns:epub=\"http://www.idpf.org/2007/ops\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\" xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:lrmi=\"http://lrmi.net/the-specification\" class=\"description\" itemprop=\"description\" data-type=\"description\">\n        <div xmlns=\"http://www.w3.org/1999/xhtml\" xmlns:bib=\"http://bibtexml.sf.net/\" xmlns:data=\"http://dev.w3.org/html5/spec/#custom\" xmlns:epub=\"http://www.idpf.org/2007/ops\" xmlns:rdf=\"http://www.w3.org/1999/02/22-

reedstrm

PREAMBLE

We the developers of OpenStax/Connexions, in order to form a more readable URL, establish user harmony, ensure URL persistance, and support all the use cases, do suggest the following id hashing strategy for content objectIds.

ID FORMAT

The canonical form for an id is a uuid-v4, in the 8n-4n-4n-4n-12n format, e.g. 6f0881fc-4d30-43e1-9a3b-52a210ab5980 Python code for generating these:

  >>> import uuid

>>> u=uuid.uuid4()

reedstrm

WITH RECURSIVE t(node, title, path,value, depth, corder) AS (
    SELECT nodeid, title, ARRAY[nodeid], documentid, 1, ARRAY[childorder]
    FROM trees tr, latest_modules m
    WHERE  tr.documentid = m.module_ident AND
      tr.parent_id IS NULL
UNION ALL
    SELECT c1.nodeid, c1.title, t.path || ARRAY[c1.nodeid], c1.documentid, t.depth+1, t.corder || ARRAY[c1.childorder] /* Recursion */
    FROM trees c1 JOIN t ON (c1.parent_id = t.node)
    WHERE not nodeid = any (t.path)
)

reedstrm

It is possible using the session_exec extension to postgresql to run a single function at session creation time

CREATE FUNCTION workon(venv text) RETURNS void
    LANGUAGE plpythonu
    AS $_$
    import os
    import sys

reedstrm

When publishing a new minor version of an existing book (i.e. the structure of the book does not change, but the content of one or more pages does), the new book will need to be cooked. This will generate new versions of all the composite pages, that is, pages that are defined by the cooking recipe, not by the raw content, such as end-of-chapter exercises, end-of-book index or glossary, etc. Ideally, these new pages should retain their uuid, and only get a new version number. This is tricky, since the pages do not exist in the raw content that is sent to be cooked, so the rulesets cannot set thier ids. However, the rulesets do set a locally unique class on each of these pages. This, combined with which chapter a given composite page is in, is enough to identify that particular page across minor versions. So a combination of chapter-identifier and composite-page class can be used as a key to lookup uuids from the soon-to-be-one-version-old book tree, and assign them to the parallel page of the newly-cooked

reedstrm

Baking and Recipes

Baking has become a critical step for publishing - however, it is not planned for it to cause publishing to fail. This is partly due to how long baking may take - much like PDF generation, it can take several hours, if there are external parts, like exercises, that need to be fetched and incorporated into the pages.

This slightly odd situtation (is baking a publish blocker, or not?) comes about partly due to derived copies - publishing a new version of content impacts not only the immediate book, but any book that shares pages with it. It was decided that failure of related books to bake is not a publication blocker, so by extension, it is not for any book. However, we want to be careful to not break existing books by publishing new content, but we cannot test every book that contains a given page.

Selection of Recipe and Fallback

So, we will implement two ways to mitigate this potential problem. First, will we keep track of when and with what recipe a given book successfully ba