Jinja2 is a templating language and parsing module built by Armin Ronacher of Flask, Click and Werkzeug fame (among many other accomplishments). Jinja2 is used in a variety of projects, including Flask itself, Ansible and Pelican (a Python static website generator). I've grown quite accustomed to the Jinja2 syntax, and I really love the project.
Jinja2 itself has some interesting syntax for handling whitespace. According to its documentation, a template can control whitespace around template control flow blocks by the placement of a - adjacent to block delimiters. For example,
{%- if True %}
Hello
{% endif %}
will have different whitespace properties than
{% if True %}
Hello
{% endif %}
While Jinja2's documentation is fine and I believe it adequately describes the whitespace logic, I still find using this feature somewhat confusing and unintuive. In the rest of this Gist, I intend to provide a motivating example for some problems that can arise when using whitespace controls, and then a short Python script that can actually help me visualize how Jinja2 works.
Cookiecutter is a project generator built by Audrey R. Greenfield of "Two Scoops of Django" fame. Cookiecutter takes a project template (or "cookiecutter"), which has a minimal amount of metadata and a scaffolding directory containing Jinja2 templates. When the user invokes cookiecutter with a "cookiecutter" (typically specified as a github project, git url or local filesystem directory), the user answers a series of questions that specify which values will be applied to the templates.
After trying out several "cookiecutters", I found that Audrey's own cookiecutter-pypackage fit best with what I wanted, and so I started using that for most of my Python projects. However, after some use, I noticed that some tools I have been using like PyInstaller were not included in the generated project, so I decided to fork her cookiecutter and make the necessary modifications.
In particular, I modified the requirements_dev.txt file to the following:
bumpversion==0.5.3
wheel==0.29.0
watchdog==0.8.3
flake8==2.6.0
tox==2.3.1
coverage==4.3.1
Sphinx==1.4.8
{%- if cookiecutter.use_pypi_deployment_with_travis == 'y' -%}
cryptography==1.7
PyYAML==3.11
{%- endif %}
{%- if cookiecutter.use_pytest == 'y' -%}
pytest==3.0.5
{%- endif %}
{%- if 'no' not in cookiecutter.command_line_interface|lower %}
git+https://github.com/pyinstaller/pyinstaller.git@8892e117bfcec197572e1d0decf17bf26723f071
{%- endif %}
This, unfortunately, renders as:
bumpversion==0.5.3
wheel==0.29.0
watchdog==0.8.3
flake8==2.6.0
tox==2.3.1
coverage==4.3.1
Sphinx==1.4.8pytest==3.0.5
git+https://github.com/pyinstaller/pyinstaller.git@8892e117bfcec197572e1d0decf17bf26723f071
which, due to the line Sphinx==1.4.8pytest==3.0.5, is invalid syntax for a requirements.txt file.
I could keep fiddling with the cookiecutter definition and generate several projects, but this seemed time-consuming and suboptimal. As Raymond Hettinger says, "There must be a better way!"
Instead, I want to create a Jinja2 template that demonstrates all of the different combinations of ticks, and then render it to gain a better intuition about whitespace handling.
First, let's consider that each if-endif block will have two separate delimiters, each with a start and an end, and thus four possible tick locations. Let's model each tick/no-tick choice as a bit (in this case, we'll just use the strings '0' and '1'. Since we have four bits, we have 2**4 == 16 possible combination of those four bits. A '1' bit will print '-', and a '0' bit will print '', the empty string.
Furthemore, I'd like to have some more intuition about differences between different blocks generated by subsequent bit strings. Gray codes are a good way to generate a sequence of bit strings, where each successive bit string is only changed by a single bit. For example, the bit string 0010 might be succeeded by 0110 or 0011; however, 0010 cannot be succeeded by 0100, since bits at both indices 1 and 2 are changed.
I'd also like to pre-store the Gray codes themesleves in a list for easy access.
import string
BITS = 4
def tick(c):
if c == '1':
return '-'
return ''
def gray_code(bits):
if bits < 1:
return []
if bits == 1:
return ['0', '1']
previous = gray_code(bits - 1)
return ['0' + i for i in previous] + ['1' + i for i in reversed(previous)]
gray_codes = list(gray_code(BITS))
Next, I want to have a way to print out some content that breaks up the bit strings. I'd rather not have a situation where I see two successive bit strings glommed onto one another, e.g. 01100111. I'm going to do this by creating a generator function that produces successive alphabet characters e.g. 'a', 'b', 'c', ... 'z', 'A', 'B', ..., 'Z' and then wraps over to 'aa', 'bb', .., 'ZZ', 'aaa', .... (Yes, this is a bit overengineered for what we need, since we know that we'll never need more than 16 characters, but can't an engineer be forgiven for overengineering in a GitHub Gist? 😉)
def next_letter():
i = 1
while True:
for l in string.ascii_letters:
yield l*i
i += 1
Finally, let's make the code to generate the template. It's not pretty, but it gets the job done, and I find it reasonably readable, at least for the purposes of this one-off script.
def make_template_string():
f_str = (''.join(
'{{%'
+ tick(gray_codes[i][0])
+ ' if True '
+ tick(gray_codes[i][1])
+ '%}}\n'
+ '{' + str(i) + '}\t' + gray_codes[i] + '\n'
+ '{{%'
+ tick(gray_codes[i][2])
+ ' endif '
+ tick(gray_codes[i][3])
+ '%}}\n'
for i in range(2**BITS)
))
template_str = f_str.format(*islice(next_letter(), 2**BITS)).strip()
return template_str
Alright, that's it! Now we can simply run the script with:
tpl_str = make_template_string()
tpl = jinja2.Template(tpl_str)
print(tpl_str)
print('=============')
print(tpl.render())
From there, we get the output
{% if True %}
a 0000
{% endif %}
{% if True %}
b 0001
{% endif -%}
{% if True %}
c 0011
{%- endif -%}
{% if True %}
d 0010
{%- endif %}
{% if True -%}
e 0110
{%- endif %}
{% if True -%}
f 0111
{%- endif -%}
{% if True -%}
g 0101
{% endif -%}
{% if True -%}
h 0100
{% endif %}
{%- if True -%}
i 1100
{% endif %}
{%- if True -%}
j 1101
{% endif -%}
{%- if True -%}
k 1111
{%- endif -%}
{%- if True -%}
l 1110
{%- endif %}
{%- if True %}
m 1010
{%- endif %}
{%- if True %}
n 1011
{%- endif -%}
{%- if True %}
o 1001
{% endif -%}
{%- if True %}
p 1000
{% endif %}
=============
a 0000
b 0001
c 0011
d 0010
e 0110
f 0111g 0101
h 0100
i 1100
j 1101
k 1111l 1110
m 1010
n 1011
o 1001
p 1000
Looking at this output, I can see that the tick itself clears the closest whitespace. A block that begins {%- if True %} will clear all whitespace preceding the start of the delimiter, whereas a block that begins {% if True -%} will clear all whitespace following the end of the delimiter.
Based on this output, I'm choosing the 0110 combination to use as my default template syntax that doesn't add extra newlines, which corresponds to
{% if True -%}
content
{%- endif %}
I had a good time doing this, and I enjoy using Python to solve these kinds of problems with little one-off scripts. I may keep tinkering with this script and try out a non-Gray code version, but for now, I'll use this. I'm still a bit curious about how two consecutive blocks that have conflicting whitespace definitions (such as the f and g blocks above) actually work - does a "no whitespace" tick on the outside of a block definition take precedence over any adjacent block definition? Does order matter? These are questions I'll keep trying to answer.
import string
from itertools import islice
import jinja2
BITS = 4
def tick(c):
if c == '1':
return '-'
return ''
def gray_code(bits):
if bits < 1:
return []
if bits == 1:
return ['0', '1']
previous = gray_code(bits - 1)
return ['0' + i for i in previous] + ['1' + i for i in reversed(previous)]
gray_codes = list(gray_code(BITS))
def next_letter():
i = 1
while True:
for l in string.ascii_letters:
yield l*i
i += 1
def make_template_string():
f_str = (''.join(
'{{%'
+ tick(gray_codes[i][0])
+ ' if True '
+ tick(gray_codes[i][1])
+ '%}}\n'
+ '{' + str(i) + '}\t' + gray_codes[i] + '\n'
+ '{{%'
+ tick(gray_codes[i][2])
+ ' endif '
+ tick(gray_codes[i][3])
+ '%}}\n'
for i in range(2**BITS)
))
template_str = f_str.format(*islice(next_letter(), 2**BITS)).strip()
return template_str
if __name__ == '__main__':
tpl_str = make_template_string()
tpl = jinja2.Template(tpl_str)
print('=============')
print(tpl_str)
print(tpl.render())