a demonstration of `anyconfig` for loading and dumping config files. https://mybinder.org/v2/gist/tonyfast/383b5ccee549dd8c51d81662f4e19aac/master

2019-10-29-anyconfig-first-look.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> python-anyconfig is a MIT licensed python library provides common APIs to load and dump configuration files in various formats with some useful features such as contents merge, templates, query, schema validation and generation support.\n",
    "\n",
    "[Github](https://github.com/ssato/python-anyconfig/)\n",
    "[Documentation](https://python-anyconfig.readthedocs.io)\n",
    "\n",
    "\n",
    "This notebook demonstrates some features of `anyconfig` for loading and dumping data.  We raise a comparsion to `traitlets.config` at the end where a backend for python config files is created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [],
   "source": [
    "    import anyconfig, pydantic, munch, json, toolz"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`anyconfig` can load to multiple formats."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [],
   "source": [
    "    config = anyconfig.loads(\"\"\"a: 10\"\"\", ac_parser='yaml')\n",
    "    config.update(anyconfig.loads(\"\"\"{\"b\": 10}\"\"\", ac_parser='json'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'a': {'q': 10}}"
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "    anyconfig.loads(\"\"\"a.q=10\"\"\", ac_parser='toml')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`anyconfig` allows `jinja2` templates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'a': 'aaa'}"
      ]
     },
     "execution_count": 134,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "    anyconfig.loads(\"a: {{ a|default('aaa') }}\\n\", ac_parser='yaml', ac_template=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'a': 3}"
      ]
     },
     "execution_count": 135,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "    anyconfig.loads(\"a: {{ a|default('aaa') }}\\n\", ac_parser='yaml', ac_template=True, ac_context={'a': 3})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`anyconfig` can dump to multiple formats."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"a\": 10, \"b\": 10}\n"
     ]
    }
   ],
   "source": [
    "    print(anyconfig.dumps(config, 'json'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a: 10\n",
      "b: 10\n",
      "\n"
     ]
    }
   ],
   "source": [
    "    print(anyconfig.dumps(config, 'yaml'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`anyconfig` provides `jsonschema` validation.  We'll use `pydantic` to generate schema."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [],
   "source": [
    "    class Schema(pydantic.BaseModel): a: int; b: int"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [],
   "source": [
    "    valid, _ = anyconfig.validate(config, Schema.schema())\n",
    "    assert Schema and valid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [],
   "source": [
    "    class BadSchema(pydantic.BaseModel): a: int; b: str"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10 is not of type 'string'\n",
      "\n",
      "Failed validating 'type' in schema['properties']['b']:\n",
      "    {'title': 'B', 'type': 'string'}\n",
      "\n",
      "On instance['b']:\n",
      "    10\n"
     ]
    }
   ],
   "source": [
    "    valid, errors = anyconfig.validate(config, BadSchema.schema())\n",
    "    assert not valid\n",
    "    print(errors)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In `traitlets` we could load configurations from python files.  Below we add similar functionality to `anyconfig` for a python backend."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {},
   "outputs": [],
   "source": [
    "    def load(object, **c):\n",
    "        if hasattr(object, 'read'): object = object.read()\n",
    "        c['c'] = c.get('c', munch.Munch())\n",
    "        return exec(object, c, c) or c['c']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {},
   "outputs": [],
   "source": [
    "    class Python(anyconfig.backend.base.StringStreamFnParser):\n",
    "        _cid = \"python\"\n",
    "        _type = \"python\"\n",
    "        _extensions = [\"python\"]\n",
    "        _ordered = True\n",
    "        _load_opts = _dump_opts = _dict_opts = [\"_dict\"]\n",
    "\n",
    "        _load_from_string_fn = anyconfig.backend.base.to_method(load)\n",
    "        _load_from_stream_fn = anyconfig.backend.base.to_method(load)\n",
    "        _dump_to_string_fn = anyconfig.backend.base.to_method(toolz.compose('c = '.__add__, json.dumps))\n",
    "        _dump_to_stream_fn = anyconfig.backend.base.to_method(json.dump)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Loading a python config."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'a': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]}"
      ]
     },
     "execution_count": 148,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "    data = anyconfig.loads(\"\"\"c.a = list(range(10))\"\"\", ac_parser=Python); data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Writing a python config."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'c = {\"a\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]}'"
      ]
     },
     "execution_count": 149,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "    anyconfig.dumps(data, ac_parser=Python)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}

requirements.txt

pyyaml
toml
jmespath
jsonschema
jinja2
munch
toolz
pydantic
anyconfig

That Python parser is dope! Goodbye, traitlets.config!