Skip to content

Instantly share code, notes, and snippets.

@alexiswl
Created October 23, 2021 04:26
Show Gist options
  • Save alexiswl/7fe1b3a90d86313c9785992bd0ce6e19 to your computer and use it in GitHub Desktop.
Save alexiswl/7fe1b3a90d86313c9785992bd0ce6e19 to your computer and use it in GitHub Desktop.
Adding comments to ruamel
Display the source blob
Display the rendered blob
Raw
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"collapsed": true,
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"## Installation\n",
"\n",
"pip install ruamel.yaml==0.17.4\n",
"OR\n",
"conda install -c conda-forge ruamel.yaml==0.17.4"
]
},
{
"cell_type": "markdown",
"source": [
"## Importation"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "code",
"execution_count": 1,
"outputs": [],
"source": [
"# Regular imports\n",
"from copy import deepcopy\n",
"\n",
"# Yaml loaders and dumpers\n",
"from ruamel.yaml.main import \\\n",
" round_trip_load as yaml_load, \\\n",
" round_trip_dump as yaml_dump\n",
"\n",
"# Yaml commentary\n",
"from ruamel.yaml.comments import \\\n",
" CommentedMap as OrderedDict, \\\n",
" CommentedSeq as OrderedList\n",
"\n",
"# For manual creation of tokens\n",
"from ruamel.yaml.tokens import CommentToken\n",
"from ruamel.yaml.error import CommentMark"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "code",
"execution_count": 2,
"outputs": [
{
"data": {
"text/plain": "ordereddict([('employees', [ordereddict([('name', 'Jeffrey Bezos'), ('job title', 'CEO'), ('annual salary (USD)', 1000000000000)]), ordereddict([('name', 'John Smith'), ('job title', 'factory worker'), ('annual salary (USD)', 20000)])])])"
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"yaml_load(\"\"\"\n",
"employees:\n",
" # Start with CEO\n",
" - name: Jeffrey Bezos # Goes by Jeff\n",
" job title: CEO # / Entrepreneur, born in 1964...\n",
" annual salary (USD): 1000000000000 # This is too much\n",
" # List of factory workers below\n",
" - name: John Smith\n",
" job title: factory worker\n",
" annual salary (USD): 20000 # Probably deserves a raise\n",
"\"\"\")"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "code",
"execution_count": 3,
"outputs": [],
"source": [
"from pprint import pprint"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "code",
"execution_count": 4,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{ 'employees': [ordereddict([('name', 'Jeffrey Bezos'), ('job title', 'CEO'), ('annual salary (USD)', 1000000000000)]), ordereddict([('name', 'John Smith'), ('job title', 'factory worker'), ('annual salary (USD)', 20000)])]}\n"
]
}
],
"source": [
"pprint(yaml_load(\"\"\"\n",
"employees:\n",
" # Start with CEO\n",
" - name: Jeffrey Bezos # Goes by Jeff\n",
" job title: CEO # / Entrepreneur, born in 1964...\n",
" annual salary (USD): 1000000000000 # This is too much\n",
" # List of factory workers below\n",
" - name: John Smith\n",
" job title: factory worker\n",
" annual salary (USD): 20000 # Probably deserves a raise\n",
"\"\"\"), indent=4, )"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "markdown",
"source": [
"## Globals"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "code",
"execution_count": 5,
"outputs": [],
"source": [
"INDENTATION = 2\n",
"RESET_COMMENT_LIST = [None, [], None, None]"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "markdown",
"source": [
"## Creating a Yaml Dict from scratch\n",
"\n",
"A little tedious since the types need to be specified on each nested list and each dict"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "code",
"execution_count": 6,
"outputs": [],
"source": [
"shopping_list = OrderedDict({\n",
" \"Shopping List\": OrderedDict({\n",
" \"eggs\": OrderedDict({\n",
" \"type\": \"free range\",\n",
" \"brand\": \"Mr Tweedy\",\n",
" \"amount\": 12\n",
" }),\n",
" \"milk\": OrderedDict({\n",
" \"type\": \"pasteurised\",\n",
" \"litres\": 1.5,\n",
" \"brands\": OrderedList([\n",
" \"FarmFresh\",\n",
" \"FarmHouse gold\",\n",
" \"Daisy The Cow\"\n",
" ])\n",
" })\n",
" })\n",
"})"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "code",
"execution_count": 6,
"outputs": [],
"source": [],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "markdown",
"source": [],
"metadata": {
"collapsed": false
}
},
{
"cell_type": "code",
"execution_count": 7,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Shopping List:\n",
" eggs:\n",
" type: free range\n",
" brand: Mr Tweedy\n",
" amount: 12\n",
" milk:\n",
" type: pasteurised\n",
" litres: 1.5\n",
" brands:\n",
" - FarmFresh\n",
" - FarmHouse gold\n",
" - Daisy The Cow\n",
"\n"
]
}
],
"source": [
"print(yaml_dump(shopping_list, indent=2, block_seq_indent=2))"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "markdown",
"source": [
"## Importing from a python object\n",
"\n",
"Much easier and probably the form of your existing object.\n",
"Dump and reload is the simplest solution!"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "code",
"execution_count": 8,
"outputs": [],
"source": [
"# Original object\n",
"shopping_list = {\n",
" \"Shopping List\": {\n",
" \"eggs\": {\n",
" \"type\": \"free range\",\n",
" \"brand\": \"Mr Tweedy\",\n",
" \"amount\": 12\n",
" },\n",
" \"milk\": {\n",
" \"type\": \"pasteurised\",\n",
" \"litres\": 1.5,\n",
" \"brands\": [\n",
" \"FarmFresh\",\n",
" \"FarmHouse gold\",\n",
" \"Daisy The Cow\"\n",
" ]\n",
" }\n",
" }\n",
"}\n",
"\n",
"# To yaml\n",
"shopping_list = yaml_load(yaml_dump(shopping_list), preserve_quotes=True)"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "code",
"execution_count": 9,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"<class 'ruamel.yaml.comments.CommentedMap'>\n"
]
}
],
"source": [
"print(type(shopping_list))"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "markdown",
"source": [
"If we dump our yaml now it will look like this"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "code",
"execution_count": 10,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Shopping List:\n",
" eggs:\n",
" type: free range\n",
" brand: Mr Tweedy\n",
" amount: 12\n",
" milk:\n",
" type: pasteurised\n",
" litres: 1.5\n",
" brands:\n",
" - FarmFresh\n",
" - FarmHouse gold\n",
" - Daisy The Cow\n",
"\n"
]
}
],
"source": [
"print(yaml_dump(shopping_list, indent=INDENTATION, block_seq_indent=INDENTATION))"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "markdown",
"source": [
"## Annotating the Data Structure\n",
"\n",
"Now we can have some fun, lets add some comments to the shopping list"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "markdown",
"source": [
"### Add a header"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "code",
"execution_count": 11,
"outputs": [],
"source": [
"shopping_list.yaml_set_start_comment(\"Shopping Lists for date: 21 Oct 2021\")"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "code",
"execution_count": 12,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# Shopping Lists for date: 21 Oct 2021\n",
"Shopping List:\n",
" eggs:\n",
" type: free range\n",
" brand: Mr Tweedy\n",
" amount: 12\n",
" milk:\n",
" type: pasteurised\n",
" litres: 1.5\n",
" brands:\n",
" - FarmFresh\n",
" - FarmHouse gold\n",
" - Daisy The Cow\n",
"\n"
]
}
],
"source": [
"print(yaml_dump(shopping_list,\n",
" indent=INDENTATION,\n",
" block_seq_indent=INDENTATION))"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "markdown",
"source": [
"## Add a comment before a key"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "code",
"execution_count": 13,
"outputs": [],
"source": [
"shopping_list.get(\"Shopping List\").\\\n",
" yaml_set_comment_before_after_key(key=\"eggs\",\n",
" before=\"Please don't forget \"\n",
" \"eggs!\")"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "code",
"execution_count": 14,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# Shopping Lists for date: 21 Oct 2021\n",
"Shopping List:\n",
"# Please don't forget eggs!\n",
" eggs:\n",
" type: free range\n",
" brand: Mr Tweedy\n",
" amount: 12\n",
" milk:\n",
" type: pasteurised\n",
" litres: 1.5\n",
" brands:\n",
" - FarmFresh\n",
" - FarmHouse gold\n",
" - Daisy The Cow\n",
"\n"
]
}
],
"source": [
"print(yaml_dump(shopping_list, indent=INDENTATION, block_seq_indent=INDENTATION))"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "markdown",
"source": [
"Ugh, this isn't ideal! That comment should be indented. Let's delete and make sure to use the indent parameter"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "markdown",
"source": [
"We need to delete the eggs list in the Comment section"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "code",
"execution_count": 15,
"outputs": [],
"source": [
"shopping_list.get(\"Shopping List\").ca.\\\n",
" items[\"eggs\"] = deepcopy(RESET_COMMENT_LIST)"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "code",
"execution_count": 16,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# Shopping Lists for date: 21 Oct 2021\n",
"Shopping List:\n",
" eggs:\n",
" type: free range\n",
" brand: Mr Tweedy\n",
" amount: 12\n",
" milk:\n",
" type: pasteurised\n",
" litres: 1.5\n",
" brands:\n",
" - FarmFresh\n",
" - FarmHouse gold\n",
" - Daisy The Cow\n",
"\n"
]
}
],
"source": [
"print(yaml_dump(shopping_list, indent=INDENTATION, block_seq_indent=INDENTATION))"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "markdown",
"source": [
"## Recreate but use the indent option"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "code",
"execution_count": 17,
"outputs": [],
"source": [
"object_depth = 1 # Shopping List\n",
"\n",
"shopping_list.get(\"Shopping List\").\\\n",
" yaml_set_comment_before_after_key(\n",
" key=\"eggs\",\n",
" before=\"Don't forget the eggs\",\n",
" indent=object_depth*INDENTATION\n",
" )"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "code",
"execution_count": 18,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# Shopping Lists for date: 21 Oct 2021\n",
"Shopping List:\n",
" # Don't forget the eggs\n",
" eggs:\n",
" type: free range\n",
" brand: Mr Tweedy\n",
" amount: 12\n",
" milk:\n",
" type: pasteurised\n",
" litres: 1.5\n",
" brands:\n",
" - FarmFresh\n",
" - FarmHouse gold\n",
" - Daisy The Cow\n",
"\n"
]
}
],
"source": [
"print(yaml_dump(shopping_list, indent=INDENTATION, block_seq_indent=INDENTATION))"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "markdown",
"source": [
"## Add a comment after a key\n",
"\n",
"Note the `after_indent` parameter used"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "code",
"execution_count": 19,
"outputs": [],
"source": [
"object_depth = 2 # Shopping List -> Eggs\n",
"shopping_list.get(\"Shopping List\").\\\n",
" yaml_set_comment_before_after_key(\n",
" key=\"eggs\",\n",
" after=\"Please don't forget eggs!\",\n",
" after_indent=object_depth*INDENTATION\n",
" )"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "code",
"execution_count": 20,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# Shopping Lists for date: 21 Oct 2021\n",
"Shopping List:\n",
" # Don't forget the eggs\n",
" eggs:\n",
" # Please don't forget eggs!\n",
" type: free range\n",
" brand: Mr Tweedy\n",
" amount: 12\n",
" milk:\n",
" type: pasteurised\n",
" litres: 1.5\n",
" brands:\n",
" - FarmFresh\n",
" - FarmHouse gold\n",
" - Daisy The Cow\n",
"\n"
]
}
],
"source": [
"print(yaml_dump(shopping_list, indent=INDENTATION, block_seq_indent=INDENTATION))"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "markdown",
"source": [
"## Lets add an end-of-line comment to a key\n",
"\n",
"Observe the `milk.litres` key, let's note that two litres is too much"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "code",
"execution_count": 21,
"outputs": [],
"source": [
"shopping_list.get(\"Shopping List\").\\\n",
" get(\"milk\").\\\n",
" yaml_add_eol_comment(\n",
" key=\"litres\",\n",
" comment=\"2 litres is too much milk!\"\n",
" )"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "code",
"execution_count": 22,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# Shopping Lists for date: 21 Oct 2021\n",
"Shopping List:\n",
" # Don't forget the eggs\n",
" eggs:\n",
" type: free range\n",
" brand: Mr Tweedy\n",
" amount: 12\n",
" milk:\n",
" type: pasteurised\n",
" litres: 1.5 # 2 litres is too much milk!\n",
" brands:\n",
" - FarmFresh\n",
" - FarmHouse gold\n",
" - Daisy The Cow\n",
"\n"
]
}
],
"source": [
"print(yaml_dump(shopping_list, indent=INDENTATION, block_seq_indent=INDENTATION))"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "markdown",
"source": [
"## Add an inline-comment to the milk brand:\n",
"\n",
"Lets note that `FarmHouse gold` and `Daisy The Cow` are last resort options,\n",
"by using the before parameter on the index of the FarmHouse gold item of the list\n",
"\n"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "code",
"execution_count": 23,
"outputs": [],
"source": [
"object_depth = 3 # Shopping List -> Milk -> Brands\n",
"shopping_list.get(\"Shopping List\").\\\n",
" get(\"milk\").\\\n",
" get(\"brands\").\\\n",
" yaml_set_comment_before_after_key(\n",
" key=1,\n",
" before=\"Last Resorts\",\n",
" indent=object_depth*INDENTATION\n",
" )"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"print(yaml_dump(shopping_list, indent=INDENTATION, block_seq_indent=INDENTATION))"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n",
"is_executing": true
}
}
},
{
"cell_type": "markdown",
"source": [
"## Lets add an eol comment to an item in a list"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"object_depth = 3 # Shopping List -> Milk -> Brands\n",
"shopping_list.get(\"Shopping List\").\\\n",
" get(\"milk\").\\\n",
" get(\"brands\").\\\n",
" yaml_add_eol_comment(\n",
" key=1,\n",
" comment=\"Too creamy\"\n",
" )"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n",
"is_executing": true
}
}
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"shopping_list.get(\"Shopping List\").get(\"milk\").get(\"brands\").ca"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n",
"is_executing": true
}
}
},
{
"cell_type": "markdown",
"source": [],
"metadata": {
"collapsed": false
}
},
{
"cell_type": "markdown",
"source": [
"Oh no! We're going to have to use the ca attribute of the brands object to add this"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"print(shopping_list.get(\"Shopping List\").\n",
" get(\"milk\").\n",
" get(\"brands\").ca)"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n",
"is_executing": true
}
}
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"# If we need to manually add in a comment token we will need to set the COMMENT_LIST as a default\n",
"if 1 not in shopping_list.get(\"Shopping List\").get(\"milk\").get(\"brands\").ca.items.keys():\n",
" shopping_list.get(\"Shopping List\").get(\"milk\").get(\"brands\").ca.items[1] = deepcopy(RESET_COMMENT_LIST)"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n",
"is_executing": true
}
}
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"shopping_list.get(\"Shopping List\").\\\n",
" get(\"milk\").\\\n",
" get(\"brands\").ca.\\\n",
" items[1][0] = CommentToken(value=\" # Too creamy\\n\",\n",
" start_mark=CommentMark(0),\n",
" end_mark=None)"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n",
"is_executing": true
}
}
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"shopping_list.get(\"Shopping List\").get(\"milk\").get(\"brands\").ca.items[2] = deepcopy(RESET_COMMENT_LIST)"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n",
"is_executing": true
}
}
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"shopping_list.get(\"Shopping List\").get(\"milk\").get(\"brands\").ca.items[2][0] = CommentToken(value=\" # Poor taste\\n\",\n",
" start_mark=CommentMark(0),\n",
" end_mark=None)"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n",
"is_executing": true
}
}
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"print(yaml_dump(shopping_list, indent=INDENTATION, block_seq_indent=INDENTATION))"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n",
"is_executing": true
}
}
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.6"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
name: ruamel
channels:
- conda-forge
- defaults
dependencies:
- ruamel.yaml=0.17.4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment