philschatz/gist:8149250

## gistfile1.md

      
    Raw
  

              gistfile1.md
            
          
    First, Links!


CSS Coverage of all CCAP books


Once you have the HTML, each book takes 3-5min for coverage, 10-min for regression-prep, and 1min for diff/regression test


Diff of Precalculus PR
Polyfills vs Browser rendering

Use css-polyfills to do HTML+CSS diff


 Build a single file
 include using <style> or <link> tags
 Make it work for small example
 Make it work for whole book
 canonicalize styles (sort rules alphabetically)
 Update the "Case for css-polyfills" gist

Some useful stats:

 HTML+CSS diff: show differences visually
 HTML+CSS diff: commandline stats to stderr
 Total # of selectors
 Total # of uncovered selectors
 Total # of times selectors applied???
 List of uncovered selectors
 Total # of covered elements (including pseudoelements)

API:

 p = new CSSPolyfills()
 p.setPlugins([])
 p.run($root, cssStr, cb)
 p.runTree($root, cssTree, cb)
 p.on () ->

Add routes to cnx-archive


 use Accept header for /content/[UUID]@[VER] to send JSON or HTML back
 add a /content/[UUID]@[VER].snippet for Varnish

CSS Refactorings


 Refactor ccap-numbering.less to use slots
 Refactor ccap-psychology or ccap-economics to use ccap-precalc slots

Blog posts


 Set Up Jekyll


 css-polyfills


 css coverage


 building travis-ci (started with http://vzmind.tumblr.com/post/9412611799/why-travis but very old)


 octokit.js


 Code coverage in Node and the browser with the same unit tests


 Building an eBook using GitHub


 Commit hooks


 Collaboration with github (merging conflicts)


 css logic vs presentation


octokit.js


 Support paged results
 Add NodeJS-style callbacks
 Rename to octokat.js

Xinclude Notes:

xinclude cannot be easily described with just CSS because of the DOM change and pulling out the overridden title.
Would need something like:
// change this to http://archive.cnx.org/ if need be (the trailing slash is IMPORTANT)
body { string-set: root-url ''; }

a.xref[href] {
  //Perform the xinclude
  x-include: root-url attr(href);

  &:pre-xinclude(:not(.autogenerated-content)) {
    string-set: overridden-title content(contents);
    & > .title { content: string(overridden-title); }
  }
}

Minimal Javascript Notes:

Selectors:


jQuery selectors like :has(> .title) are converted to additional class on the element .has-child-title in the CSS


Simple selectors like :before are left as-is if the content:  is simple (only contains text or the counter() or attr() functions)


:outside converts the selector from a b:outside { to a .x-outside.123 > b { } and a .x-outside.123 { }


Same for nested selectors


Also, each one generates accompanying jQuery(sizzle) code.


Content:


if simple, leave in CSS


otherwise, generates jQuery(sizzle) code


move-to:

replaces the selector with .123
to ensure no styling shows up on pending():

ALL original selectors get :not(.moved) added to each element
Selector for content: pending() gets .moved added to it


content: target-counter() is annoying and requires re-simulating counters for all targets


prepass over LESS:

mark if there are ANY pending() calls (requires annotating ALL selectors)
mark a rule as complex if:

contains a "complex" selector (:has():outside:before)
contains a "complex" function in content:


Then, split out "vanilla" rules.
Blog Posts

Introducing CSS Polyfills

Tags: CSS enhancement jQuery selectors Paged Media,
CSS is meant to style structured HTML.
But,


sometimes CSS is not powerful enough to style our Textbooks
we want authors to customize their books but to not want them to write (or run) arbitrary JavaScript

We publish books in various formats and use CSS to style them.
Books online are in an HTML format that closely resembles O'Reilly's HTMLBook and PDFs use an HTML format generated from Docbook.
The Problem

We use Docbook for PDFs partly because we need to move content around (ie collating exercises at the end of a chapter, making an index) and XSLT provides a way to move XML around.
Unfortunately, this means 4 things:

developers need to learn XSLT
we must regression-test all of our books whenever we fix a bug or add a feature
CSS for the PDF is different for ePUB and online
numbering things like exercises is different in a PDF than online

Fortunately, there are a few W3C Drafts that help fill in some of the gaps: Generated Content for Paged Media and CSS3 Generated and Replaced Content Module.
Intersection of Some CSS Features:


Feature
EPUB
Browsers
PrinceXML (PDF)


:before
no
yes
yes


counter-increment:
no
yes
yes


content:
no
partial
yes


target-text()
no
no
yes


page-break-*:
no
no
yes


move-to:
no
no
no


:outside:before
no
no
no


To replace Docbook and have one CSS file to style the various formats we need to support all of these features and note a few differences:

PDF is generated using a single large HTML file (CSS needs to operate on all chapters)
ePUB needs to be chunked into multiple HTML files (ideally using CSS page-break-*)
Online, a single HTML file can be viewed outside the context of a book

Browsers

Ideally, we would be able to get access to all of these unsupported selectors and rules using the CSS Document Object Model but browsers only expose the selectors and rules they understand.
The Solution

Enter CSS-Polyfills.
The project uses LessCSS and jQuery to parse the CSS file and "bake" the changes into the HTML.
With it you can do things not possible to describe using CSS supported by browsers. For example, you can style an element based on children inside:
.note:has(> .title) { /* Give it a fancier border */ }

Or, you can automatically generate a glossary at the and of a chapter based on terms in the chapter:
.term > .definition { move-to: glossary-area; }
.chapter:after {
  content: pending(glossary-area);
}

In another post, I'll go over some of the "Freebies" that come out of this project like CSS Coverage and CSS+HTML Diffs for regression testing.
Bonuses

By parsing the CSS file and "baking" the styles into the HTML there are a few "freebies" that come out.
Easy CSS Coverage

As a free perk, you can easily generate Coverage data for your CSS files by transforming a HTML and CSS file from the commandline and filtering stderr.
HTML+CSS Diffs

To do regression tests on books we merely need to generate the "baked" HTML file twice, once with the old CSS and once with the new CSS.
Then, a quick XSLT file can compare the two and generate a version of the page with <span> tags marking the places where styling changed.
See https://github.com/philschatz/css-diff for a package that does this.
CSS Coverage and CSS Diffs

In this post I'll build on the CSS Polyfills project mentioned earlier.
CSS Coverage

Code coverage is important because code should be exercised to prevent "bitrot".
There are a couple tools that do coverage for CSS files but for groups that use extensions to CSS (like publishers) the coverage will always be incomplete.
Fortunately, CSS Polyfills can output coverage information by just adding a listener. With it you can find the number of selectors that did not match any elements. If you run it after every unit test then viola, CSS coverage for your tests!
CSS Diffs

Our textbooks are frequently hundreds of pages long and use a single CSS file, so making a CSS change can change content in unexpected places.
Again, CSS Polyfills and a little XSLT file makes this easy.
CSS-Diff will take an HTML and CSS file and produce an HTML file with all the styling "baked" in.
Then, the provided XSLT file can compare 2 "baked" HTML files and inject <span> tags whenever the styles differ.
Style Baker code (shortened for brevity):
class StyleBaker
  rules:
    # Match all rules and squirrel the styles away
    '*': (env, name) ->
      $context.addClass('js-polyfill-styles')
      styles = $context.data('js-polyfill-styles') or {}
      rules = []
      for arg, i in arguments
        continue if i < 2
        rules.push(arg.eval(env).toCSS(env))

      styles[name] ?= style
      $context.data('js-polyfill-styles', styles)

# Run Polyfills and bake in the styles
polyfills = new CSSPolyfills(new StyleBaker())
polyfills.run $root, lessFile, (err, newCSS) ->

  # Bake in the styles.
  console.log('Baking styles...')
  $root.find('.js-polyfill-styles').each (i, el) ->
    $el = $(el)
    style = []
    for ruleName, ruleStr of $el.data('js-polyfill-styles')
      style.push("#{ruleName}:#{ruleStr}; ")

    $el.attr('style', style.join(''))

See the CSS-Diff project to run it from the commandline.
Octokit.js

Octokit.js is a JavaScript client that interacts with GitHub using their API.
Some unique features:

works in a browser or in NodeJS
uses Promises
written in CoffeeScript
Mocha unit tests that actually talk to GitHub (and run in the browser or commandline)
Code Coverage using BlanketJS (that runs in the browser or commandline)

API Support:

all repo operations (including create/remove)
Teams, Ogniations, and Permisions
eTags for caching results
Listeners on rate limit changes
Committing multiple files at once

Finding a test and coverage framework that works both in a browser and on the commandline was a bit challenging but I'll return to that in another post.
This library also uses jQuery.ajax and jQuery.Deferred.
Building an eBook using GitHub

At Connexions we build textbooks. To do it we have roughly 4 services:

Published Repository
Authoring Repository
Generate exports (EPUB, PDF)
Website

And we have a bunch of Open Source code for each of these.
As someone who uses GitHub and Travis daily, I wondered how much of this can GitHub (and Travis) do for us?
It turns out, quite a bit. If each book is a repository whose contents is an Unzipped EPUB:

the master branch is the published version of the book
tags are various versions of a book
gh-pages branch can be used as a Website
"Download as Zip" is the EPUB version of the book
The editor can "save" via the GitHub API
Travis-CI can be used to generate a PDF (and "push" to AWS or some other place)

So, I took our editor and made it save EPUB files to GitHub and viola! a book editor using GitHub!
See GitHub Book Editor for the code.
See Book Editor Demo to play around with the editor.
Splitting Hairs in CSS

We've all heard the rule "HTML is for structure and CSS is for presentation".
This is why tags like <b> have been deprecated in favor of styling like font-weight: bold;.
Well, we have somewhat complicated CSS for our textbooks and are in the process of simplifying the underlying HTML format (from Docbook-generated to HTMLBook).
Each book has many different "parts" that are styled differently or tweaked slightly.
Some Examples of "parts" common to all books but tweaked differently:
[Insert napkin image here]
Slots and Skeletons

We split our CSS up into slots (mixins containing rules) and skeletons
Feature	EPUB	Browsers	PrinceXML (PDF)
`:before`	no	yes	yes
`counter-increment:`	no	yes	yes
`content:`	no	partial	yes
`target-text()`	no	no	yes
`page-break-*:`	no	no	yes
`move-to:`	no	no	no
`:outside:before`	no	no	no