Skip to content

Instantly share code, notes, and snippets.

@mariogeiger

mariogeiger/tutorial.ipynb

Created November 15, 2022 21:09
Show Gist options
  • Save mariogeiger/160848d2d67e6d8292271a243bb6f40e to your computer and use it in GitHub Desktop.
Save mariogeiger/160848d2d67e6d8292271a243bb6f40e to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
tutorial.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Boilerplate"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import jax\n",
"import matplotlib.pyplot as plt\n",
"\n",
"jax.config.update(\"jax_enable_x64\", True)\n",
"np.set_printoptions(precision=4, suppress=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Installation\n",
"\n",
"To install the package, run the following command:\n",
"\n",
"```\n",
"pip install git+https://github.com/e3nn/e3nn-jax.git\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"import jax\n",
"import jax.numpy as jnp\n",
"import e3nn_jax as e3nn"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Functions for rotations\n",
"\n",
"### parameterizations\n",
"- Euler angles\n",
"- Rotation matrices\n",
"- Axis-angle\n",
"- Quaternions\n",
"\n",
"### functions\n",
"- Convertions between parameterizations\n",
"- Compositions of rotations\n",
"- Random rotations\n",
"- Inverse rotation"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(DeviceArray([0.7821, 0.6875, 1.4887, 5.9424, 1.002 , 5.3127, 0.0833,\n",
" 2.333 , 2.1504, 3.9292], dtype=float64),\n",
" DeviceArray([0.4508, 1.5341, 1.291 , 2.6191, 1.2815, 1.9438, 2.1814,\n",
" 2.0649, 2.0775, 1.1074], dtype=float64),\n",
" DeviceArray([4.4497, 1.0236, 1.8509, 0.0122, 3.4505, 5.3254, 5.8178,\n",
" 3.4122, 2.5171, 5.5721], dtype=float64))"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.rand_angles(jax.random.PRNGKey(0), (10,))"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(DeviceArray([-0.0449, -0.8229, 0.5665], dtype=float64),\n",
" DeviceArray(2.2121, dtype=float64))"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.rand_axis_angle(jax.random.PRNGKey(0), ())"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(DeviceArray([0.4486, 0.8926, 0.045 ], dtype=float64),\n",
" DeviceArray(0.4466, dtype=float64, weak_type=True))"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.angles_to_axis_angle(alpha=0.1, beta=0.2, gamma=0.3)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(DeviceArray([0.4459, 0.894 , 0.0447], dtype=float64),\n",
" DeviceArray(0.2235, dtype=float64))"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.compose_axis_angle(\n",
" axis1=jnp.array([1, 0, 0]),\n",
" angle1=0.1,\n",
" axis2=jnp.array([0, 1, 0]),\n",
" angle2=0.2,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Irreps\n",
"\n",
"Representation of the group $O(3)$\n",
"\n",
"### functions\n",
"- simplify\n",
"- sort\n",
"- filter\n",
"\n",
"$$ D'(g) = A D(g) A^{-1} $$"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"10x0e"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.Irreps([(10, (0, 1))])"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"64x0e+32x1e+32x1o"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.Irreps(\"64x0e + 32x1e + 32x1o\")"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"6"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.Irreps(\"2x1o\").dim"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"4"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.Irreps(\"2x0e + 2x1o\").num_irreps"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"4x0e"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.Irreps(\"2x0e + 2x0e\").simplify()"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"sort(irreps=2x0e+2x0e+1x1o, p=(0, 2, 1), inv=(0, 2, 1))"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.Irreps(\"2x0e + 1o + 2x0e\").sort()"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"DeviceArray([[ 0.8776, 0.4034, 0.259 ],\n",
" [ 0. , 0.5403, -0.8415],\n",
" [-0.4794, 0.7385, 0.4742]], dtype=float64)"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.Irreps(\"1e\").D_from_angles(0.5, 1.0, 0.0)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# IrrepsArray\n",
"\n",
"The `IrrepsArray` is a class that represents a tensor `ndarray` with a given representation of rotation `Irreps`."
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"3x0e+1x1o [300. -10. -5. 1. 0. 0.]"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"array = jnp.array([300.0, -10.0, -5.0, 1.0, 0.0, 0.0])\n",
"irreps = e3nn.Irreps(\"3x0e + 1o\")\n",
"\n",
"x = e3nn.IrrepsArray(irreps, array)\n",
"x"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"3x0e+1x1o [[300. -10. -5. 1. 0. 0.]]"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x.reshape((1, 6))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- binary operations\n",
"- indexing\n",
"- e3nn.mean\n",
"- e3nn.norm\n",
"- e3nn.normal\n",
"- axis-to-mul and mul-to-axis\n",
"- slice-by-* functions\n",
"- sorted\n",
"- simplify\n",
"- transform-by-* functions"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x0e [4.]"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.IrrepsArray(\"0e\", jnp.array([3.0])) + 1.0 # (assumed as 0e)"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"3x0e [301. -9. -4.]"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x[:3] + 1.0"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"3x0e+1x1o [301. -9. -4. 2. 1. 1.]"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x + e3nn.IrrepsArray(\"3x0e + 1o\", jnp.ones(6))"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"6"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.Irreps(\"3x0e + 1o\").dim"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"4"
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.Irreps(\"3x0e + 1o\").num_irreps"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x1o [1. 0. 0.]"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x[3:]"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x1o [1. 0. 0.]"
]
},
"execution_count": 26,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# s0 s1 s2 [v00 v01 v02]\n",
"\n",
"x.slice_by_dim[-3:]"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x1o [1. 0. 0.]"
]
},
"execution_count": 27,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# s0 s1 s2 [v0]\n",
"\n",
"x.slice_by_mul[-1:]"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x1o [1. 0. 0.]"
]
},
"execution_count": 29,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# s [v]\n",
"\n",
"x.slice_by_chunk[-1:]"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"2x0e+1x1e+2x0e"
]
},
"execution_count": 30,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x = e3nn.IrrepsArray(\"2x0e + 1e + 2x0e\", jnp.ones(2 + 3 + 2))\n",
"\n",
"x.irreps"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"4x0e+1x1e [1. 1. 1. 1. 1. 1. 1.]"
]
},
"execution_count": 31,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x.sorted().simplify()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"2x0e [1. 1.]"
]
},
"execution_count": 32,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x.slice_by_chunk[:1]"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"2x1e [ 3. 4. 5. 12. 14. 16.]"
]
},
"execution_count": 33,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x = e3nn.IrrepsArray(\"2x0e\", jnp.array([1.0, 2.0]))\n",
"y = e3nn.IrrepsArray(\"2x1e\", jnp.array([3.0, 4.0, 5.0, 6.0, 7.0, 8.0]))\n",
"\n",
"x * y"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {},
"outputs": [],
"source": [
"x = e3nn.IrrepsArray(\"0e + 1e\", jnp.array([1.0, 2.0, 3.0, 4.0]))\n",
"y = e3nn.IrrepsArray(\"1e + 0e\", jnp.array([3.0, 4.0, 5.0, 6.0]))\n",
"\n",
"# x * y ---> ValueError: x * y with both x and y non scalar and ambiguous. Use e3nn.elementwise_tensor_product or e3nn.tensor_product instead."
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {},
"outputs": [],
"source": [
"x = e3nn.IrrepsArray(\"2x1e\", jnp.array([3.0, 4.0, 5.0, 6.0, 7.0, 8.0]))\n",
"y = e3nn.IrrepsArray(\"2x1e\", jnp.array([3.0, 4.0, 5.0, 6.0, 7.0, 8.0]))\n",
"\n",
"# x * y ---> ValueError: x * y with both x and y non scalar and ambiguous. Use e3nn.elementwise_tensor_product or e3nn.tensor_product instead."
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(3, 256)"
]
},
"execution_count": 40,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x = e3nn.normal(\"64x0e + 64x1o\", jax.random.PRNGKey(0), (3,))\n",
"\n",
"x.shape"
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(3, 64, 4)"
]
},
"execution_count": 41,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x.mul_to_axis().shape"
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(3, 256)"
]
},
"execution_count": 42,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x.reshape((-1, -1)).shape"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x1o [2.1239 1.4495 2.7181]"
]
},
"execution_count": 43,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.IrrepsArray(\"1o\", jnp.array([1.0, 2.0, 3.0])).transform_by_angles(0.1, 0.2, 0.3)"
]
},
{
"cell_type": "code",
"execution_count": 44,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x0e+1x1o+2x1e [6 0 1 2 3 4 5 7 8 9]"
]
},
"execution_count": 44,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.IrrepsArray(\"1o + 1e + 0e + 1e\", jnp.arange(10)).sorted().simplify()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Tensor products\n",
"\n",
"- spherical harmonics\n",
"- tensor product\n",
"- elementwise product\n",
"- tensor square\n",
"- reduced tensor product basis\n",
"- symmetric contraction"
]
},
{
"cell_type": "code",
"execution_count": 45,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x1e+1x1o [ 3. 4. 5. 12. 18. 24.]"
]
},
"execution_count": 45,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x = e3nn.IrrepsArray(\"0e + 1e\", jnp.array([1.0, 2.0, 3.0, 4.0]))\n",
"y = e3nn.IrrepsArray(\"1e + 0o\", jnp.array([3.0, 4.0, 5.0, 6.0]))\n",
"\n",
"e3nn.elementwise_tensor_product(x, y)"
]
},
{
"cell_type": "code",
"execution_count": 46,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x0e+1x0o+1x1o+1x1e+1x1e+1x2e\n",
"[21.9393 6. 12. 18. 24. 3. 4. 5. -0.7071\n",
" 1.4142 -0.7071 15.5563 12.0208 -0.8165 21.9203 9.8995]"
]
},
"execution_count": 46,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.tensor_product(x, y)"
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x0o+1x1o+1x1o+1x2o\n",
"[ 3.4641 0.7071 -1.4142 0.7071 100. 100. 100. 2.8284\n",
" 2.1213 0. 3.5355 1.4142]"
]
},
"execution_count": 47,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x = e3nn.IrrepsArray(\"1o\", jnp.array([1.0, 1.0, 1.0]))\n",
"y = e3nn.IrrepsArray(\"1e + 0e\", jnp.array([1.0, 2.0, 3.0, 100.0]))\n",
"\n",
"e3nn.tensor_product(x, y)"
]
},
{
"cell_type": "code",
"execution_count": 48,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"DeviceArray([[ 1., 2., 3., 100.],\n",
" [ 1., 2., 3., 100.],\n",
" [ 1., 2., 3., 100.]], dtype=float64)"
]
},
"execution_count": 48,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"jnp.einsum(\"i,j\", x.array, y.array)"
]
},
{
"cell_type": "code",
"execution_count": 49,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"DeviceArray([ 3.4641, 100. , 100. , 100. , 0.7071, -1.4142,\n",
" 0.7071, 2.8284, 2.1213, -0. , 3.5355, 1.4142], dtype=float64)"
]
},
"execution_count": 49,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"u = e3nn.reduced_tensor_product_basis(\"ij\", i=\"1o\", j=\"1e + 0e\")\n",
"\n",
"jnp.einsum(\"i,j,ijk->k\", x.array, y.array, u.array)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": 50,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"2x1o"
]
},
"execution_count": 50,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x = e3nn.IrrepsArray(\"2x1o\", jnp.array([1.0, 2.0, 3.0, 4.0, 5.0, 6.0]))\n",
"x.irreps"
]
},
{
"cell_type": "code",
"execution_count": 51,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"4x0e+4x1e+4x2e\n",
"[ 8.0829 18.4752 18.4752 44.456 -0. 0. 0. -2.1213 4.2426\n",
" -2.1213 2.1213 -4.2426 2.1213 -0. 0. 0. 4.2426 2.8284\n",
" -0.8165 8.4853 5.6569 12.7279 9.1924 -0.8165 19.0919 9.8995 12.7279\n",
" 9.1924 -0.8165 19.0919 9.8995 33.9411 28.2843 -0.8165 42.4264 14.1421]"
]
},
"execution_count": 51,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.tensor_product(x, x)"
]
},
{
"cell_type": "code",
"execution_count": 52,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x0e+2x0e+1x1e+1x2e+2x2e\n",
"[18.4752 3.6148 19.8813 -2.1213 4.2426 -2.1213 12.7279 9.1924 -0.8165\n",
" 19.0919 9.8995 3. 2. -0.5774 6. 4. 24. 20.\n",
" -0.5774 30. 10. ]"
]
},
"execution_count": 52,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.tensor_square(x)"
]
},
{
"cell_type": "code",
"execution_count": 53,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x2e [3.873 3.873 0. 3.873 0. ]"
]
},
"execution_count": 53,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x = e3nn.IrrepsArray(\"1o\", jnp.array([1.0, 1.0, 1.0]))\n",
"\n",
"e3nn.spherical_harmonics(2, x, False, normalization=\"component\")"
]
},
{
"cell_type": "code",
"execution_count": 54,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x2e [1. 1. 0. 1. 0.]"
]
},
"execution_count": 54,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.tensor_square(x)[1:]"
]
},
{
"cell_type": "code",
"execution_count": 55,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x0o+1x1o+1x2o+1x0e\n",
"[ 3.4641 0.7071 -1.4142 0.7071 2.8284 2.1213 0. 3.5355\n",
" 1.4142 200. ]"
]
},
"execution_count": 55,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x = e3nn.IrrepsArray(\"1o + 0e\", jnp.array([1.0, 1.0, 1.0, 2.0]))\n",
"y = e3nn.IrrepsArray(\"1e + 0e\", jnp.array([1.0, 2.0, 3.0, 100.0]))\n",
"\n",
"e3nn.elementwise_tensor_product(x, y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\"Normalization\" topic below explains the difference between `e3nn.tensor_product`, `e3nn.tensor_square` and `e3nn.spherical_harmonics`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Linear\n",
"\n",
"Where the weight matrices are..."
]
},
{
"cell_type": "code",
"execution_count": 56,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"2x0e [46.1297 -5.4155]"
]
},
"execution_count": 56,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import haiku as hk\n",
"\n",
"@hk.without_apply_rng\n",
"@hk.transform\n",
"def model(x):\n",
" x = e3nn.tensor_square(x)\n",
" x = e3nn.Linear(\"2x0e\")(x)\n",
" return x\n",
"\n",
"x = e3nn.IrrepsArray(\"0e + 1o\", jnp.array([10.0, 1.0, 1.0, 1.0]))\n",
"params = model.init(jax.random.PRNGKey(0), x)\n",
"y = model.apply(params, x)\n",
"\n",
"y"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Grad\n",
"\n",
"The `e3nn.grad` function is a wrapper around `jax.grad` that takes into account the representation of rotation of the input and output of the function."
]
},
{
"cell_type": "code",
"execution_count": 57,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"2x0e+2x1o [ 9.2907 -1.1702 -0.216 -0.216 -0.216 0.2902 0.2902 0.2902]"
]
},
"execution_count": 57,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"e3nn.grad(model.apply, 1)(params, x)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Gate Activation Function\n",
"\n",
"The `e3nn.gate` function is a simple way to apply activation functions to non-scalar quantities."
]
},
{
"cell_type": "code",
"execution_count": 58,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x0e+1x1o [15.3485 1.8337 3.6675 5.5012]"
]
},
"execution_count": 58,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x = e3nn.IrrepsArray(\"2x0e + 1o\", jnp.array([10.0, 5.0, 1.0, 2.0, 3.0]))\n",
"\n",
"e3nn.gate(x)"
]
},
{
"cell_type": "code",
"execution_count": 59,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"DeviceArray(15.3485, dtype=float64)"
]
},
"execution_count": 59,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"gelu = e3nn.normalize_function(jax.nn.gelu)\n",
"gelu(jnp.array(10.0))"
]
},
{
"cell_type": "code",
"execution_count": 60,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"DeviceArray([1.8337, 3.6675, 5.5012], dtype=float64)"
]
},
"execution_count": 60,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sigmoid = e3nn.normalize_function(jax.nn.sigmoid)\n",
"\n",
"sigmoid(jnp.array(5.0)) * jnp.array([1.0, 2.0, 3.0])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Radial Basis Functions\n",
"\n",
"- Bessel\n",
"- One hot functions\n",
"- polynomial envelopes\n",
"- smooth envelope"
]
},
{
"cell_type": "code",
"execution_count": 61,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.legend.Legend at 0x142195e10>"
]
},
"execution_count": 61,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABn80lEQVR4nO3deVwU9R/H8dfucoOgqIAoivd9H+SVR5iW2Z1a/tQsO0wrs0Ot1MrKDi0rNctSszQtyy5NM9TMI80DT7zPVMATEJBr5/fHJkWpsbgwHO/n47EPh9mZ73x2wvbtzHy/X4thGAYiIiIiJrGaXYCIiIiUbAojIiIiYiqFERERETGVwoiIiIiYSmFERERETKUwIiIiIqZSGBERERFTKYyIiIiIqdzMLiA37HY7x48fp1SpUlgsFrPLERERkVwwDIOkpCRCQ0OxWi9//aNIhJHjx48TFhZmdhkiIiKSB0ePHqVSpUqXfb9IhJFSpUoBjg/j7+9vcjUiIiKSG4mJiYSFhWV/j19OkQgjF2/N+Pv7K4yIiIgUMf/1iIUeYBURERFTKYyIiIiIqRRGRERExFRF4pkREZGiyDAMMjMzycrKMrsUkXxhs9lwc3O76mE3FEZERPJBeno6J06cICUlxexSRPKVj48PFSpUwMPDI89tKIyIiLiY3W7n4MGD2Gw2QkND8fDw0ICNUuwYhkF6ejonT57k4MGD1KxZ84oDm12JwoiIiIulp6djt9sJCwvDx8fH7HJE8o23tzfu7u4cPnyY9PR0vLy88tSOHmAVEcknef1XokhR4orfc/1NEREREVM5HUZWrlxJjx49CA0NxWKx8M0331xx+6+//pouXbpQvnx5/P39ad26NUuWLMlrvSIiIlLMOB1GkpOTady4MZMnT87V9itXrqRLly4sWrSIjRs30qlTJ3r06MHmzZudLlZERAqvXbt2cc011+Dl5UWTJk3MLueqhIeHM3HiRLPLKDGcfoD1hhtu4IYbbsj19v/8j/nqq6/y7bff8v3339O0aVNnDy8iIoXUmDFj8PX1Zffu3fj5+TFz5kyGDh3KuXPnzC5NCrkC701jt9tJSkoiMDDwstukpaWRlpaW/XNiYmK+1PLZb4c5eCqZPhGVqVbeL1+OISJSUuzfv5/u3btTpUoVs0uRIqbAH2AdP34858+fp2fPnpfdZty4cQQEBGS/wsLCXF6H3W7wwcr9fLzqIJ0n/ELfj9exeHssmVl2lx9LRMQwDFLSM015GYaR6zrnz59Pw4YN8fb2pmzZskRGRpKcnIzdbuell16iUqVKeHp60qRJExYvXpy9n8ViYePGjbz00ktYLBY6duzIgAEDSEhIwGKxYLFYeOGFF/7z+GlpaTz11FNUrFgRX19fIiIiWLFiRfb7M2fOpHTp0ixZsoS6devi5+dHt27dOHHiBAA//fQTXl5e/7oa8/jjj9O5c+fsn1etWkX79u3x9vYmLCyMxx57jOTk5MvWdeTIEW655Rb8/Pzw9/enZ8+exMXFZb//wgsv0KRJEz744IPsLt09e/YkISEhRzsfffQRdevWxcvLizp16jBlypT/PCclQYFeGZkzZw4vvvgi3377LUFBQZfdbuTIkQwbNiz758TExHwJJC/d0oDP1h5m2e54ft17il/3niLE34t7IirTu1UYQaXy1l9aROSfUjOyqDfanIf3d77UFR+P//7f/YkTJ7j77rt54403uO2220hKSuLXX3/FMAzeeecdJkyYwAcffEDTpk2ZPn06N998Mzt27KBmzZqcOHGCyMhIunXrxlNPPYWPjw8zZsxg9OjR7N69GwA/v/++Aj1kyBB27tzJ3LlzCQ0NZcGCBXTr1o1t27ZRs2ZNAFJSUhg/fjyffvopVquV//3vfzz11FPMnj2b6667jtKlS/PVV19x//33A5CVlcW8efN45ZVXAMcVnG7duvHyyy8zffp0Tp48yZAhQxgyZAgzZsz4V012uz07iPzyyy9kZmYyePBgevXqlSMo7du3jy+++ILvv/+exMRE7r//fh555BFmz54NwOzZsxk9ejSTJk2iadOmbN68mQceeABfX1/69+//n+emOCuwMDJ37lwGDhzIl19+SWRk5BW39fT0xNPTM1/rsVotdKodRKfaQRw9k8Kc9Uf44vejxCZe4K2le3hv2V5ubFiBfq2r0KxyGY2eKCLF3okTJ8jMzOT222/PvtXSsGFDwHFVe/jw4fTu3RuA119/neXLlzNx4kQmT55MSEgIbm5u+Pn5ERISAkBAQAAWiyX75/9y5MgRZsyYwZEjRwgNDQXgqaeeYvHixcyYMYNXX30VgIyMDKZOnUr16tUBR4B56aWXAMdcKb1792bOnDnZYSQqKopz585xxx13AI6r73369GHo0KEA1KxZk3fffZcOHTrw/vvv/2vgrqioKLZt28bBgwez/2E8a9Ys6tevz++//07Lli0BuHDhArNmzaJixYoAvPfee3Tv3p0JEyYQEhLCmDFjmDBhArfffjsAVatWZefOnXzwwQcKIwVxkM8//5z77ruPuXPn0r1794I4pFPCAn0Y3q0OQyNr8uO2WGatPcSmI+f4Nvo430Yfp36oP/3bhHNz41C83G1mlysiRZC3u42dL3U17di50bhxY6677joaNmxI165duf7667nzzjux2WwcP36ctm3b5ti+bdu2bNmyxWV1btu2jaysLGrVqpVjfVpaGmXLls3+2cfHJzuIAFSoUIH4+Pjsn/v06cM111zD8ePHCQ0NZfbs2XTv3p3SpUsDsGXLFrZu3Zp9xQIct9EuDuNft27dHMePiYkhLCwsxxX6evXqUbp0aWJiYrLDSOXKlbODCEDr1q2x2+3s3r2bUqVKsX//fu6//34eeOCB7G0yMzMJCAjIy+kqVpwOI+fPn2ffvn3ZPx88eJDo6GgCAwOpXLkyI0eO5NixY8yaNQtw3Jrp378/77zzDhEREcTGxgKOIWQL238ATzcbtzatyK1NK7LtjwRmrT3Ed1uOs+N4Is/M38q4RTHc3aoy/7umCqGlvc0uV0SKEIvFkqtbJWay2WwsXbqUNWvW8NNPP/Hee+/x3HPPsXTp0gI5/vnz57HZbGzcuBGbLWeA+vstHnd39xzvWSyWHM/FtGzZkurVqzN37lwGDRrEggULmDlzZo7jPPTQQzz22GP/qqFy5cou+jQ5nT9/HoBp06YRERGR471/ftaSyOm/GRs2bKBTp07ZP198tqN///7MnDmTEydOcOTIkez3P/zww+z7a4MHD85ef3H7wqphpQDevKsxz95Yl3kbjvLp2sMcO5fKlBX7+WDlAbrWD2ZA26q0qKJbOCJSfFgsFtq2bUvbtm0ZPXo0VapUISoqitDQUFavXk2HDh2yt129ejWtWrW6bFseHh5kZWXl+thNmzYlKyuL+Ph42rdvf1Wfo0+fPsyePZtKlSphtVpzXJVv1qwZO3fupEaNGrlqq27duhw9epSjR49mXx3ZuXMn586do169etnbHTlyJPtqDMBvv/2G1Wqldu3aBAcHExoayoEDB+jTp89VfbbiyOkw0rFjxys+mf3PgPH3h3uKojK+HjzcoToD21Xl55h4PllziLUHTrNoWyyLtsXSoKI/A9pU5abGFfB0U7oVkaJr3bp1REVFcf311xMUFMS6des4efIkdevW5emnn2bMmDFUr16dJk2aMGPGDKKjo3Pc6vin8PBwzp8/T1RUFI0bN8bHx+eKEwfWqlWLPn360K9fPyZMmEDTpk05efIkUVFRNGrUyKnb/H369OGFF17glVde4c4778zxHOLw4cO55pprGDJkCAMHDsTX15edO3eydOlSJk2a9K+2IiMjadiwIX369GHixIlkZmbyyCOP0KFDB1q0aJG9nZeXF/3792f8+PEkJiby2GOP0bNnz+xnZl588UUee+wxAgIC6NatG2lpaWzYsIGzZ8/m6LRRIhlFQEJCggEYCQkJZpdiGIZhxJxIMIbP32LUem6RUWX4D0aV4T8YzccuNSYu3WOcSrpgdnkiYrLU1FRj586dRmpqqtmlOGXnzp1G165djfLlyxuenp5GrVq1jPfee88wDMPIysoyXnjhBaNixYqGu7u70bhxY+PHH3/MsX/jxo2NMWPG5Fj38MMPG2XLljWAf713Kenp6cbo0aON8PBww93d3ahQoYJx2223GVu3bjUMwzBmzJhhBAQE5NhnwYIFxqW+zlq1amUAxrJly/713vr1640uXboYfn5+hq+vr9GoUSPjlVdeyX6/SpUqxttvv5398+HDh42bb77Z8PX1NUqVKmXcddddRmxsbPb7Y8aMMRo3bmxMmTLFCA0NNby8vIw777zTOHPmTI7jzp4922jSpInh4eFhlClTxrj22muNr7/++j/PS2F2pd/33H5/WwzDiQ7oJklMTCQgIICEhAT8/f3NLifbmeR0Pl9/hFlrDxGX6BikzcPNyu1NK3Jfu6rUCi5lcoUiYoYLFy5w8OBBqlatmucp1aVoeeGFF/jmm2+Ijo42u5QCd6Xf99x+f2vW3qsQ6OvB4E41WDW8M+/0bkKjSgGkZ9qZ+/tRrn97Jf2mr2flnpNODTgkIiJS0hTuR7uLCHeblVuaVOTmxqFsOHyWj389yE87Y1m55yQr95ykTkgpBravxs2NQ/FwU/4TkZLp119/veLcZhd7nEjJo9s0+eTI6RSmrz7IFxuOkpLueJo8qJQn97YNp09EFQK83f+jBREpqnSb5tJSU1M5duzYZd/Pbe8WKVxccZtGYSSfJaRkMGf9EWauOZj9XImvh427W1XmvnZVNV6JSDGkMCIliZ4ZKQICfNwZ1LE6vz7TmQl3NaZ2cCmS07P4aNVBrn1jOU/Mi2ZXbP7MSiwiIlIU6JmRAuLhZuWO5pW4vVlFVuw5yYe/HGDtgdMs2HyMBZuP0al2eQZ1rEHLcA2iJiIiJYvCSAGzWP6aoG/rH+f44JcDLNp+guW7T7J890maVS7NoI41uK5OEFarQomIiBR/uk1jokaVSjO5TzOWPdmRu1tVxsNmZdORczwwawM3vPMr32w+RmaW3ewyRURE8pXCSCFQtZwv425vyKoRnXi4Q3X8PN3YHZfE0HnRdJqwgs9+O8yFjNzP7yAiIlKUKIwUIkGlvBhxQx1Wj+jMU9fXItDXg6NnUnn+m+1c+8ZyPvr1ACnpmWaXKSJyRTNnzqR06dJml+FSK1aswGKxcO7cOVOOf+211zJnzpwCP+6IESN49NFH8/04CiOFUIC3O0M612T18M6M6VGPCgFexCel8fLCGNq9vpzJy/eReCHD7DJFRKQAfPfdd8TFxdG7d+8c69euXUvnzp3x9fXF39+fa6+9ltTU1Fy3u2LFCm655RYqVKiAr68vTZo0+dfEh0899RSffPIJBw4ccMlnuRyFkULM28PGgLZV+eXpTrx2e0OqlPXhTHI6by7ZTbvXljHx5z0kpCiUiIgUZ++++y4DBgzAav3rK3vt2rV069aN66+/nvXr1/P7778zZMiQHNv8lzVr1tCoUSO++uortm7dyoABA+jXrx8//PBD9jblypWja9euvP/++y79TP+kMFIEeLhZ6d2qMlHDOjCxVxNqBvmReCGTiT/vpd3ryxi/ZDdnk9PNLlNErsQwID3ZnJcTY1t27NiRIUOGMGTIEAICAihXrhyjRo3KMcfW2bNn6devH2XKlMHHx4cbbriBvXv3XrK9Q4cOYbVa2bBhQ471EydOpEqVKtjt//2Q/vbt27nhhhvw8/MjODiYvn37curUqRw1P/bYYzzzzDMEBgYSEhLCCy+8kP3+PffcQ69evXK0mZGRQbly5Zg1axYAdrudcePGUbVqVby9vWncuDHz58+/Yl1fffUV9evXx9PTk/DwcCZMmJDj/fDwcMaOHcvdd9+Nr68vFStWZPLkyTm2OXfuHAMHDqR8+fL4+/vTuXNntmzZkv3+yZMnWbZsGT169Mix3xNPPMFjjz3GiBEjqF+/PrVr16Znz554enr+5/m86Nlnn2Xs2LG0adOG6tWr8/jjj9OtWze+/vrrHNv16NGDuXPn5rrdvFDX3iLEzWbl1qaOOXB+3B7Le8v2sis2iUnL9zFj9UH6tQnngfbVCPT1MLtUEfmnjBR4NdScYz97HDx8c735J598wv3338/69evZsGEDDz74IJUrV+aBBx4A4N5772Xv3r189913+Pv7M3z4cG688UZ27tyJu3vOqS7Cw8OJjIxkxowZtGjRInv9jBkzuPfee//zX/Lnzp2jc+fODBw4kLfffpvU1FSGDx9Oz549WbZsWY6ahw0bxrp161i7di333nsvbdu2pUuXLvTp04e77rqL8+fP4+fnB8CSJUtISUnhtttuA2DcuHF89tlnTJ06lZo1a7Jy5Ur+97//Ub58eTp06PCvujZu3EjPnj154YUX6NWrF2vWrOGRRx6hbNmy3Hvvvdnbvfnmmzz77LO8+OKLLFmyhMcff5xatWrRpUsXAO666y68vb358ccfCQgI4IMPPuC6665jz549BAYGsmrVKnx8fKhbt252m/Hx8axbt44+ffrQpk0b9u/fT506dXjllVdo165dbv4TX1ZCQkKOYwG0atWKP/74g0OHDhEeHn5V7V+OhoMvwux2g592xvFu1F52nnCM4urrYaN/m3AGKpSImOaSw2OnJxeJMNKxY0fi4+PZsWNH9gCMI0aM4LvvvmPnzp3s3buXWrVqsXr1atq0aQPA6dOnCQsL45NPPuGuu+5i5syZDB06NPthzy+++IKHH36YEydO4OnpyaZNm2jRogUHDhz4zy+3l19+mV9//ZUlS5Zkr/vjjz8ICwtj9+7d1KpVi44dO5KVlcWvv/6avU2rVq3o3Lkzr732GpmZmVSoUIG33nqLvn37Ao6rJXa7nblz55KWlkZgYCA///wzrVu3zm5j4MCBpKSkMGfOHFasWEGnTp04e/YspUuXpk+fPpw8eZKffvope/tnnnmGhQsXsmPHDsARxOrWrcuPP/6YvU3v3r1JTExk0aJFrFq1iu7duxMfH5/jikaNGjV45plnePDBB5k4cSLvvfce+/fvz37/t99+o3Xr1gQGBjJ+/HiaNGnCrFmzmDJlCtu3b6dmzZq5+m/9T1988QV9+/Zl06ZN1K9fP3v9xe/gFStWXDKYuWI4eF0ZKcKsVgvdGoTQtX4wP8fEM/HnPew4nsiUFfv5ZM0h+rcJ58Frq1HaR6FExHTuPo5QYNaxnXDNNdfkGAm6devWTJgwgaysLGJiYnBzcyMiIiL7/bJly1K7dm1iYmIu2d6tt97K4MGDWbBgAb1792bmzJl06tQpV//K3rJlC8uXL8++ovF3+/fvp1atWgA0atQox3sVKlQgPj4eADc3N3r27Mns2bPp27cvycnJfPvtt9m3Hvbt20dKSkr21YqL0tPTadq06SXriomJ4ZZbbsmxrm3btkycOJGsrCxsNhtAjnBz8eeJEydmf7bz589TtmzZHNukpqZmh4/U1NR/fcFfvLX10EMPMWDAAACaNm1KVFQU06dPZ9y4cZes+UqWL1/OgAEDmDZtWo4gAuDt7ZhDLSUlxel2c0thpBiwWCx0qRdMZN2gf4WSWWsPc1+7qtzfrqpmChYxk8Xi1K2S4sTDw4N+/foxY8YMbr/9dubMmcM777yTq33Pnz9Pjx49eP311//1XoUKFbKX/3l7yGKx5HgepU+fPnTo0IH4+HiWLl2Kt7c33bp1yz4GwMKFC6lYsWKOdpx5BsNZ58+fp0KFCqxYseJf713sGl2uXDnOnj2b472Ln7tevXo51tetW5cjR444Xccvv/xCjx49ePvtt+nXr9+/3j9z5gwA5cuXd7rt3FIYKUb+Hkp+2hnH20v3sCs2iXej9jJz9UEeaF+NAe2q4uep/+wicnnr1q3L8fNvv/1GzZo1sdls1K1bl8zMTNatW5fjNs3u3bv/9eX4dwMHDqRBgwZMmTKFzMxMbr/99lzV0qxZM7766ivCw8Nxc8v7/7vatGlDWFgY8+bN48cff+Suu+7KDjD16tXD09OTI0eOXPI2xKXUrVuX1atX51i3evVqatWqlX1VBBzn7u9+++237GcymjVrRmxsLG5ubpe9StS0aVNiY2M5e/YsZcqUARy3f0JDQ9m9e3eObffs2cMNN9yQq/ovWrFiBTfddBOvv/46Dz744CW32b59O+7u7v+6YuJSRhGQkJBgAEZCQoLZpRQpWVl2Y+HW40bkhBVGleE/GFWG/2A0fekn48Nf9hup6ZlmlydSbKWmpho7d+40UlNTzS7FaR06dDD8/PyMJ554wti1a5cxZ84cw9fX15g6dWr2NrfccotRr14949dffzWio6ONbt26GTVq1DDS09MNwzCMGTNmGAEBAf9qu02bNoaHh4fx8MMP57qeY8eOGeXLlzfuvPNOY/369ca+ffuMxYsXG/fee6+RmZmZXfPjjz+eY79bbrnF6N+/f451zz33nFGvXj3Dzc3N+PXXX//1XtmyZY2ZM2ca+/btMzZu3Gi8++67xsyZMw3DMIzly5cbgHH27FnDMAxj48aNhtVqNV566SVj9+7dxsyZMw1vb29jxowZ2W1WqVLF8Pf3N15//XVj9+7dxqRJkwybzWYsXrzYMAzDsNvtRrt27YzGjRsbS5YsMQ4ePGisXr3aePbZZ43ff//dMAzDyMzMNMqXL298//33Oep9++23DX9/f+PLL7809u7dazz//POGl5eXsW/fvlyf22XLlhk+Pj7GyJEjjRMnTmS/Tp8+nWO7MWPGGJ07d75sO1f6fc/t97fCSAmQmWU3vtn8h9HpzeXZoaTVK0uNWWsPGWkZWWaXJ1LsFPUw8sgjjxgPP/yw4e/vb5QpU8Z49tlnDbvdnr3NmTNnjL59+xoBAQGGt7e30bVrV2PPnj3Z718ujHz88ccGYKxfv96pmvbs2WPcdtttRunSpQ1vb2+jTp06xtChQ7Nrym0Y2blzpwEYVapUyfF5DMMRDCZOnGjUrl3bcHd3N8qXL2907drV+OWXXwzD+HcYMQzDmD9/vlGvXj3D3d3dqFy5svHmm2/maLNKlSrGiy++aNx1112Gj4+PERISYrzzzjs5tklMTDQeffRRIzQ01HB3dzfCwsKMPn36GEeOHMne5plnnjF69+79r/Mybtw4o1KlSoaPj4/RunXrfwWsDh06/Osc/F3//v0N4F+vDh065Niudu3axueff37ZdlwRRtSbpgTJzLLz9aZjvBO1l2PnHKP0hQV6M6xLLW5uXBGbZgkWcYkr9S4o7Dp27EiTJk2yH7J0pbFjx/Lll1+ydetWl7ddGIWHhzN06FCGDh16Ve3ExsZSv359Nm3aRJUqVXK9X5UqVXjxxRdzdDV21o8//siTTz7J1q1bL3ubzBW9aTToWQniZrPSs2UYy57qwEu31Kd8KU+OnknliXlbuPGdX/l5ZxxFIJuKSBFz/vx5tm/fzqRJkwpknpPiJiQkhI8//tiph1N37NhBQEDAJR9IdUZycjIzZsy4qud1ckNhpATydLPRr3U4K5/uxPBudfD3cswSPHDWBu54fw3rD54xu0QRKUaGDBlC8+bN6dixI/fdd1+O9x5++GH8/Pwu+Xr44YdNqrjwufXWW2nfvn2ut69fvz5bt251anj4S7nzzjtzdOPOL7pNIySkZDB15X5mrD7IhQxHV7jr6gTxTLc61A4pZXJ1IkVPUb5NU9Di4+NJTEy85Hv+/v4EBQUVcEXiLA16Ji4R4OPO8G51GNAmnHei9jL396NE7Ypn2e54bm9aiWHX16JiaW+zyxSRYigoKEiBQ3SbRv4S5O/FK7c1ZOkT13JjwxAMA77a9Aedxq9g3KIYElI1Q7CIM4rAhWeRq+aK33OFEfmXauX9mNKnOQseaUNE1UDSM+18sPIAHd5czserDpKWmWV2iSKF2sXBtPJz+GyRwuLi7/k/R8F1hp4ZkSsyDIPlu+MZt2gXe+MdQyaHBXozvFsdujeskGP+ChH5y4kTJzh37hxBQUH4+Pjo74oUO4ZhkJKSQnx8PKVLl84xPP9Fuf3+VhiRXMnMsjN/4x+8tXQP8UlpADSrXJrnutejeZUyJlcnUvgYhkFsbGz2zLUixVXp0qUJCQm5ZOBWGJF8kZKeyYcrD/DBLwdIzXDcruneqAIjutUhLNC5mUFFSoKsrCwyMvS8lRRP7u7uOebi+SeFEclXcYkXmPDTbr7c+AeGAR42K/e1q8rgTtUp5aXZgUVERGFECsiO4wm8uiiG1ftOA1DOz5Onu9bizuZhGl5eRKSEUxiRAmMYBlEx8byyKIaDp5IBqFfBn9E96nFNtbImVyciImZRGJECl55pZ9baQ7wTtZekC5kAdG9YgZE31qFSGT1PIiJS0iiMiGnOJKfz1tLdzFl3BLsBnm5WHu5QnYc7VMfb4/IPOomISPGiMCKm23k8kRe/38G6PyfeCw3w4vmb6nFDg0t3ARMRkeJFYUQKBcMwWLQtllcXxXDsXCoAbWuU5YUe9akZrEn4RESKM4URKVRS07OY+st+3v9lP+mZdtysFga0Deex62qqK7CISDGV2+9vzU0jBcLbw8YTXWrx8xMd6FIvmEy7wbRfD9J5wi98G31ME4qJiJRgujIiplixO54Xv9+Z3RW4dbWyvHSLbt2IiBQnujIihVrH2kEsHtqep66vhZe7lbUHTnPDO78yblEMyWmZZpcnIiIFSGFETOPpZmNI55osfaIDkXUdt24+WHmALm/9wk87Ys0uT0RECojCiJguLNCHj/q34OP+LahUxpvjCRd48NONDPzkd/44m2J2eSIiks+cDiMrV66kR48ehIaGYrFY+Oabb/5znxUrVtCsWTM8PT2pUaMGM2fOzEOpUtxdVzeYpU90YHCn6rjbLPwcE0+Xt1Yy9Zf9ZGTZzS5PRETyidNhJDk5mcaNGzN58uRcbX/w4EG6d+9Op06diI6OZujQoQwcOJAlS5Y4XawUf94eNp7uWodFj7WnVdVAUjOyeO3HXfR4bxWbj5w1uzwREckHV9WbxmKxsGDBAm699dbLbjN8+HAWLlzI9u3bs9f17t2bc+fOsXjx4lwdR71pSibDMJi/8Q9eWRTDuZQMLBboe00Vnu5aW2OTiIgUAbn9/nbL70LWrl1LZGRkjnVdu3Zl6NChl90nLS2NtLS07J8TExPzqbgpcO5I/rRdGFxyyHXL5d/LXmf5c/lv21qsOddfXPf39Rd/ttr+WrbYwHrxT9uff7o5lq0Xl90df9rcwObx58sdi82Du6p6EDmwOm8vP8w3204xb+1elmw/wUu3NqRr/RBXnzERETFBvoeR2NhYgoODc6wLDg4mMTGR1NRUvL29/7XPuHHjePHFF/O7NNixAP5Yn//HkatSBngJeMnrzxUZcOELd87bvPH28cXm4QPuPuDhCx4X//RzvLz8wbMUePqDVwB4lQbv0n/+WcaxbNXkfSIiZsr3MJIXI0eOZNiwYdk/JyYmEhYW5voDNbkbqrZ3fbtmuOLdNuMK2xh/W2/k3Oaf6w3737b58+eL6y++Z/9z2chy/GnPcizb/1xnzwJ7pmM5K9OxbM8Ee4bjvax0yMr485XueGWmQVZajqq9LBmOfc5f7VUziyOU+JYDn7KOP/2CwS8ESgU7lktVgIBKju00wZ+IiMvlexgJCQkhLi4ux7q4uDj8/f0veVUEwNPTE09Pz/wuDVrcl//HENcwjL+CSWYae4+d5K1F0RyJP4M3abQI9eT+VsGU98yEjGRIOw/p5yEtCS4kQloiXEiAC+cgNQFSz0J6EmBA6hnH67+4+4B/qCOYlAn/26sqBFZzXIURERGn5XsYad26NYsWLcqxbunSpbRu3Tq/Dy3FicUCbp6OF1Czdnneq1GHmWsOMf6n3Ww4ZueTH2w83bU2/duEY7Pm4gpGVoYjlCSfgpTTkHLKsXw+DpJi//oz6QQkn4SMFDi9z/G6FL8QKFcTytVyvILqQnAD8C3rwhMhIlL8ON2b5vz58+zb5/ifcdOmTXnrrbfo1KkTgYGBVK5cmZEjR3Ls2DFmzZoFOLr2NmjQgMGDB3PfffexbNkyHnvsMRYuXEjXrl1zdUz1ppErOXI6heFfbWXtgdMANKtcmjfubEyNID/XHSTjAiQec7zOHYWzh/72OugIK5dTqgIE13cEk4rNILSZ4+qKbvmISDGX2+9vp8PIihUr6NSp07/W9+/fn5kzZ3Lvvfdy6NAhVqxYkWOfJ554gp07d1KpUiVGjRrFvffe6/IPIyWX3W4w9/ejvLoohvNpmXi4WRkaWZMH21fDzVYAAw1fSIBT++DUnr9ecdsdYeVSfMtDxeZQqSVUaeMIKO5el95WRKSIyrcwYgaFEcmt4+dSeXbBNlbsdlypaBxWmvF3NjJvNuC0JIiPgdhtELsVjm2EuJ2OB3j/zubhCCdV2kC1jhAWkX1LSkSkqFIYkRLLMAy+2nSMF7/fQdIFx1WSJyJr8UD7qgVzleS/ZKQ6wskfG+DIWsfrn7d53H2gSluo3glqXu94FkVEpIhRGJESLzbhAiO/3sryv10lmXCXi58lcQXDgDMH4PAaOPQr7F8OyfE5tylbA2rfALW7Q1grjY0iIkWCwogIfw0p/9IPO0m6kImnm5VnutVhQJtwrLnpcWMGw4C4HXBgOeyLgkOrHGOqXORTDurdDA3ugMptHCPciogUQgojIn9zIiGV4V9tY+Uex1WSiKqBjL+rMWGBPiZXlgsXEmHfz7B7Eez9yfGw7EWlQqH+bdCoJ4Q2Ma1EEZFLURgR+QfDMJi97givLoohJT0LXw8bo3vUo2eLMCxFpZttVgYc/AW2L4CY7yHtb8EkpCE07QeN7nKMFisiYjKFEZHLOHw6mae+3MLvh84CEFk3mNfuaEg5vyLWeyUzzXHFZNuXsGuhY4RaAJun4zZOy4GOXjlFJWiJSLGjMCJyBVl2g49+PcCEn/aQnmWnrK8Hr93RiC71gv9758Io5Qxs/QI2f+oY3+SiCk3gmkGOWznqKiwiBUxhRCQXYk4k8sS8aHbFJgHQu2UYo26qh69noZxD8r8ZBhzfDBumO8LJxQkGfYMcV0paPQA+gebWKCIlhsKISC5dyMjiraV7mPbrAQwDwsv6MLF3U5qElTa7tKuTfBo2zoDfP3LMrwPg4Qct74fWQ8AvyNz6RKTYUxgRcdLa/ad58otojidcwGa18ERkTQZ1rJG7SfcKs6wM2PktrHr7r1s4bl7QrB+0e8IxE7GISD5QGBHJg4SUDJ79ZhsLtzquJLQKD+StXo2pVKYIdAH+L4YBexbDyvFwbINjnZsXRDwEbYfq9o2IuJzCiEgeGYbB15uOMfrb7SSnZ1HKy41xtzfkpkbF5AqCYcDBlbBinGMoegDPAGj3OEQ8DB6+5tYnIsWGwojIVTp8Opmh86LZfOQc4Hi4dXSPevh4FNGHW//JMGDvUoh68a/bN34h0OUlxyBq6hIsIldJYUTEBTKy7Ez8eQ9TVuzHMKB6eV/eu7sZ9UKL0e+h3Q7b58Oyl+HcYce6Sq3gxjcgtKm5tYlIkaYwIuJCa/ad4okvoolLTMPDZuW57nXp17pK0Rm5NTcy02DtZMczJRnJgAWa9YXrxoBvObOrE5EiSGFExMXOJKfzzPwt/BzjmFG3a/1g3rijMQE+7iZX5mKJx2HpGNj2heNn70DoNg4a9dKtGxFxisKISD4wDIMZqw8x7scYMrIMKpb25r17mtKscjGcC+bwWlj4JMTvcPxcIxJuehtKVza3LhEpMnL7/a25x0WcYLFYuK9dVb4a1IbKgT4cO5dKz6lr+eCX/djthT7XO6dKa3joF+j8PNg8HPPgTL4GfpvqeM5ERMRFdGVEJI8SL2Qw8uu/xiS5rk4QE3o2prSPh8mV5YOTe+D7x/7qClz1Wrj1fQioZG5dIlKo6cqISD7z93Jn0t1NeeW2Bni4WYnaFU/3d1ex+chZs0tzvfK14N5FcON4cPdxjFPyfhvY/pXZlYlIMaAwInIVLBYLfSKq8PWgNlQp++dtmw/WMmP1QYrARUfnWK2OifYe+hVCm8GFBJh/H3z9oGNZRCSPFEZEXKBBxQC+f7QdNzQIISPL4MXvdzJ4ziaSLmSYXZrrlasB9/8E1z4DFitsnQdT28PxaLMrE5EiSmFExEX8vdyZ0qcZY3rUw91mYdG2WG6ZvJo9cUlml+Z6Nnfo/BwMWOzoXXPuMHx8PWyY4RjZVUTECQojIi5ksVgY0LYq8x5qTYUALw6cTOaWSav5NvqY2aXlj8oR8NBKqHUDZKXBD0NhwcOQnmx2ZSJShCiMiOSDZpXL8MOj7WhXoxypGVk8Pjea0d9uJz2zGHaJ9S4DvedA5ItgscHWuTDtOji93+zKRKSIUBgRySdl/Tz55L5WPNa5BgCz1h6m94driUu8YHJl+cBqhXZDof/34BcMJ2NgWmc48IvZlYlIEaAwIpKPbFYLw66vzfR7W1DKy41NR85x03ur+P3QGbNLyx/hbR29bSq2gAvn4NPb4PePzK5KRAo5hRGRAtC5TjDfD2lH7eBSnExK4+4Pf+OTNYeKX/dfgFLBcO9CaNgTjCzHkPILn4SsYtizSERcQmFEpICEl/NlweA29GgcSqbdYMx3O3jyyy1cyMgyuzTXc/eC2z90zPiLxXF1ZPadcCHR7MpEpBBSGBEpQD4ebrzbuwnPd6+LzWrh603H6PXBWk4kpJpdmutZLNB+GPSeDe6+cGAFzLwRkuLMrkxEChmFEZECZrFYGNi+GrPua0VpH3e2/JFAj/dWs6G4PkdSpzsMWAi+5SF2G3wcCaf2mV2ViBQiCiMiJmlboxzfD2lHnZBSnDqfxt3TfmP2usNml5U/Qps6Rm0tUxXOHYHp18MfG8yuSkQKCYUREROFBfrw9SNt6N6wAhlZBs8t2M6ob7aTkVUMxyMJrAb3L3UEk5TT8EkP2Pez2VWJSCGgMCJiMh8PNybd05Snu9bGYoFPfztMv4/XczY53ezSXM+vPPT/AapfBxkp8PndsGuh2VWJiMkURkQKAYvFwuBONfiwbwt8PWysPXCaWyavZm9xnNfG0w/ungt1b4asdPiiH+xYYHZVImIihRGRQqRLvWC+fqQtYYHeHDmTwm1T1hAVUwx7n7h5wJ0zoOFdYM+E+ffBlrlmVyUiJlEYESlkaoeU4tvB7YioGsj5tEwGztrAR78eKH4DpNnc4LYPoGlfMOyOCfY2zjS7KhExgcKISCEU6OvBZwMjuLtVZQwDXl4Yw7MLthW/B1utNujxLrR8ADDg+8dh06dmVyUiBUxhRKSQcrdZefW2BjzfvS4WC3y+/ij9p68nIaWYDatutcKNb8I1jzh+/u5R2PqluTWJSIFSGBEpxC4OkPZRP8eDrWv2n+a2Kas5eCrZ7NJcy2KBrq9Ci/sAAxY8BDu/NbsqESkgCiMiRcB1dYOZP6gNFUt7c+BUMrdPWV38Zv61WODGCdCkj2OCvfn3we7FZlclIgVAYUSkiKhbwZ8Fg9vQuFIAZ1My6DNtHd9GHzO7LNeyWuHm96DBnY5eNl/0hf3Lza5KRPKZwohIERJUyou5D7bm+nrBpGfZeXxuNJOW7S1ePW2sNrhtKtTt4RiHZN7/4Phms6sSkXykMCJSxHh72Hj/f80Z2K4qAON/2sMz87cWr542Nne442Oo2gHSz8Nnd8Lp/WZXJSL5RGFEpAiyWS08f1M9xt5SH6sFvtz4B/d/soHzaZlml+Y6bp7Q6zMIaQQpp+DT2yCpGA4AJyJ5CyOTJ08mPDwcLy8vIiIiWL9+/RW3nzhxIrVr18bb25uwsDCeeOIJLly4kKeCReQvfVuH81H/Fni721i55yQ9p64lLrEY/d3y8of/ffXnbL+HYfYdcCHR7KpExMWcDiPz5s1j2LBhjBkzhk2bNtG4cWO6du1KfHz8JbefM2cOI0aMYMyYMcTExPDxxx8zb948nn322asuXkSgc51g5j10DeX8PNh5IpHbp6wpXnPa+AVB36/BtzzEboN5fSCzGE4iKFKCOR1G3nrrLR544AEGDBhAvXr1mDp1Kj4+PkyfPv2S269Zs4a2bdtyzz33EB4ezvXXX8/dd9/9n1dTRCT3GlUqzdeD2lKtnC/HzqVyx/trWHfgtNlluU5gNegzHzz84OBK+GEoFKeHdkVKOKfCSHp6Ohs3biQyMvKvBqxWIiMjWbt27SX3adOmDRs3bswOHwcOHGDRokXceOONlz1OWloaiYmJOV4icmWVy/rw1aA2tKhShsQLmfSdvp7F20+YXZbrhDaBnp+AxQrRs2HV22ZXJCIu4lQYOXXqFFlZWQQHB+dYHxwcTGxs7CX3ueeee3jppZdo164d7u7uVK9enY4dO17xNs24ceMICAjIfoWFhTlTpkiJVebPOW2urxdMeqadQbM38elvh80uy3VqRMINbziWo16EHd+YWo6IuEa+96ZZsWIFr776KlOmTGHTpk18/fXXLFy4kLFjx152n5EjR5KQkJD9Onr0aH6XKVJseLk7uv7eE+GYZG/UN9uZ8NPu4jMWSasHIOJhx/KCh+CPjebWIyJXzc2ZjcuVK4fNZiMuLmf3uri4OEJCQi65z6hRo+jbty8DBw4EoGHDhiQnJ/Pggw/y3HPPYbX+Ow95enri6enpTGki8jc2q4VXbm1AcCkv3v55D+8t28fJpDRevrUBbrZi0KO/66tw5iDsXQKf94YHlkFpXUEVKaqc+r+Sh4cHzZs3JyoqKnud3W4nKiqK1q1bX3KflJSUfwUOm80GUHz+pSZSCFksFh6PrMmrtzXEaoG5vx9l8JxNXMjIMru0q2e1wZ0fQ3ADSI6Hz++G9BSzqxKRPHL6n0jDhg1j2rRpfPLJJ8TExDBo0CCSk5MZMGAAAP369WPkyJHZ2/fo0YP333+fuXPncvDgQZYuXcqoUaPo0aNHdigRkfxzT0Rl3v9fczzcrCzZEceAGb+TdCHD7LKunmcpuHsu+JSDuG3w3aPqYSNSRDl1mwagV69enDx5ktGjRxMbG0uTJk1YvHhx9kOtR44cyXEl5Pnnn8disfD8889z7NgxypcvT48ePXjllVdc9ylE5Iq61g9h5oCWPDhrI2sPnObuab8xc0AryvkV8duhpcMcPWxm3QLb5zt63LR51OyqRMRJFqMI3CtJTEwkICCAhIQE/P39zS5HpMjafiyB/tPXczo5narlfJl1XyvCAn3MLuvqrfsAfnzG0e33f19D9U5mVyQi5P77uxg8ySYiudWgYgBfPtyaiqW9OXgqmZ4frGVf/Hmzy7p6rR6ExveAYYf598HZYtSdWaQEUBgRKWGqlffjq0FtqBHkx4mEC/T6YC3bjyWYXdbVsVjgprchtCmknoG5ffRAq0gRojAiUgKFBHjxxUOtaVgxgNPJ6dw97Tc2HDpjdllXx93LMcvvxQdaf3za7IpEJJcURkRKqEBfD2Y/EEGr8ECSLmTS9+P1rNxz0uyyrk5AJbhrhuPZkc2fwebZZlckIrmgMCJSgvl7ufPJfa3oUKs8qRlZ3P/J7yzZcempHYqMqtdCxz+nm1j4JMTtNLceEflPCiMiJZy3h41p/VpwY8MQMrIMHpm9ie+3HDe7rKvT/kmo3hkyU+GLfpBWDB7SFSnGFEZEBA83K+/2bsrtTSuSZTd4fO5mvtxQhOeEslrh9mlQKhRO74UfhmpANJFCTGFERABws1kZf1dj7m5VGbsBT8/fyqdrD5ldVt75loM7p4PFBtu+hI0zzK5IRC5DYUREslmtFl69rQH3tgkHYNS3O/jo1wPmFnU1qrSGyDGO5cUjIT7G3HpE5JIURkQkB4vFwpge9RjUsToALy+MYeov+02u6iq0eQxqdIHMCzD/fsi4YHZFIvIPCiMi8i8Wi4Vnutbm8etqAvDaj7uYvHyfyVXlkcUCt04B3/IQvwN+HmN2RSLyDwojInJJFouFJ7rU4skutQB4c8lu3vl5r8lV5ZFfENw61bG8birsWWJuPSKSg8KIiFzRo9fV5JlutQF4++c9TPhpN0Vgfs1/qxkJ1zziWP5mECQV8fFURIoRhRER+U+PdKzBczfWBeC9Zft4a+meohlIIl+A4IaQctoRSOx2sysSERRGRCSXHri2GqNuqgc4AsnbRTGQuHnCnR+DmzfsXwbrPzC7IhFBYUREnHB/u6o8391xheTdZft4uyg+Q1K+NnR9xbH88wtwco+p5YiIwoiIOGlg+2p/BZKovUz8uQh+mbe4D6pf5+juu+AhyMo0uyKREk1hREScNrB9texnSCb+vLfo9bKxWOCWSeAVAMc3waq3zK5IpERTGBGRPHng2mo8e2MdwNHL5v0VRWxgNP9QuHG8Y/mX1+F4tKnliJRkCiMikmcPXludp7s6uv2+vnhX0Rs6vuFdUO8WsGfCgoc1OquISRRGROSqDO5UI3uk1pcXxhStyfUsFuj+NvgGwckYWP6y2RWJlEgKIyJy1YZG1syey2bUtzuYu/6IyRU5wbcs3PyuY3nNJDj6u7n1iJRACiMictUuzmVzf7uqAIxcsI0Fm/8wuSon1L4BGvUGDPh2MGSmmV2RSImiMCIiLmGxWHi+e136ta6CYcBTX25l8fYiNOR6t3GO2zWndsMvb5hdjUiJojAiIi5jsVh4oUd97mxeiSy7waOfb2LF7nizy8odn0DoPsGxvOptOLHF3HpEShCFERFxKavVwut3NKJ7owpkZBk89OlG1h04bXZZuVPvZqh3KxhZjts1WRlmVyRSIiiMiIjL2awW3u7ZhM51gkjLtHP/JxuIPnrO7LJy58Y3wbsMxG6D1RPNrkakRFAYEZF84eFmZUqfZrSpXpbzaZn0n76e3bFJZpf13/yC4IY/nxn55Q2IjzG3HpESQGFERPKNl7uNaf1a0LRyaRJSM+j78TqOnkkxu6z/1vAuqNUNstLh+8fBbje7IpFiTWFERPKVr6cbM+9tRZ2QUsQnpdHno3XEJxbykU4tFsfDrB5+cHQdbJppdkUixZrCiIjkuwAfd2bd14rKgT4cOZNC34/Xcy4l3eyyriygEnR+3rG89AVIKkLdlEWKGIURESkQQf5efHZ/BEGlPNkdl8SAmb+TnJZpdllX1upBCG0KaQmweITZ1YgUWwojIlJgKpf14dP7IwjwdmfzkXMMmr2J9MxC/DyG1QY93gGLDXYsgD1LzK5IpFhSGBGRAlU7pBQzB7TE293Gyj0neXr+Fux2w+yyLq9CY7hmkGN54ZOQdt7cekSKIYURESlwTSuX4f3/NcPNauHb6OO8vDAGwyjEgaTTsxBQGRKOwopxZlcjUuwojIiIKTrWDmL8XY0BmL76IO//st/kiq7Awxduesux/NsUx4BoIuIyCiMiYppbm1bk+e51AXhj8W6++P2oyRVdQc0uUO8WMOyO2zUae0TEZRRGRMRUA9tX4+EO1QEYuWAbUTFxJld0BV3HgbuvY+yRLXPMrkak2FAYERHTDe9WO3um38FzNrH5yFmzS7q0gIrQ8c8uvktHQ8oZc+sRKSYURkTEdBaLhXG3N6RDrfJcyHBMrHfwVLLZZV3aNYOgfF1IOQ3LxppdjUixoDAiIoWCu80xsV7DigGcSU6n//T1nExKM7usf7O5Q/fxjuUNM+DYRnPrESkGFEZEpNDw9XRj+r0ts4eNv6+wjtIa3g4a9QKMPx9mzTK7IpEiTWFERAqV8qU8+eS+VgT6erDtWAKD52wiM6sQ9lzpMhY8/eH4Ztg40+xqRIo0hRERKXSqlvPl4/4t8HK3smL3SUZ9u6PwDYpWKvivifSWjdXDrCJXQWFERAqlppXL8G7vplgs8Pn6I4VzULQW90NQPUg9C8tfNbsakSIrT2Fk8uTJhIeH4+XlRUREBOvXr7/i9ufOnWPw4MFUqFABT09PatWqxaJFi/JUsIiUHNfXD2H0TfUAx6Bo30YfM7mif7C5wQ2vO5Y3fAyx282tR6SIcjqMzJs3j2HDhjFmzBg2bdpE48aN6dq1K/Hx8ZfcPj09nS5dunDo0CHmz5/P7t27mTZtGhUrVrzq4kWk+BvQtir3t6sKwNNfbmXdgdMmV/QPVa+Ferc6Rmb9cTgUtttJIkWAxXDyRmxERAQtW7Zk0qRJANjtdsLCwnj00UcZMWLEv7afOnUqb775Jrt27cLd3T1PRSYmJhIQEEBCQgL+/v55akNEii673eCR2ZtYvCOWAG93vn6kDdXL+5ld1l/OHYVJLSEzFe6cAQ1uN7sikUIht9/fTl0ZSU9PZ+PGjURGRv7VgNVKZGQka9euveQ+3333Ha1bt2bw4MEEBwfToEEDXn31VbKyLt8VLi0tjcTExBwvESm5rFYLE3s3oWnl0iSkZnDfzN85k5xudll/KR0G7Yc5ln8aBemFdMA2kULKqTBy6tQpsrKyCA4OzrE+ODiY2NjYS+5z4MAB5s+fT1ZWFosWLWLUqFFMmDCBl19++bLHGTduHAEBAdmvsLAwZ8oUkWLIy93GtH4tqFTGm8OnU3jo0w2kZRai8T3aPAqlK0PiH7DqbbOrESlS8r03jd1uJygoiA8//JDmzZvTq1cvnnvuOaZOnXrZfUaOHElCQkL26+jRQjyTp4gUmHJ+nsy4tyWlPN34/dBZRny1rfB0+XX3hq5/9qhZ/S6cPWRqOSJFiVNhpFy5cthsNuLics6qGRcXR0hIyCX3qVChArVq1cJms2Wvq1u3LrGxsaSnX/oyq6enJ/7+/jleIiIANYNLMeV/zbBZLSzYfIz3lu0zu6S/1LkJqnaArDRYOsbsakSKDKfCiIeHB82bNycqKip7nd1uJyoqitatW19yn7Zt27Jv3z7s9r9GUNyzZw8VKlTAw8Mjj2WLSEnWvmZ5xt7SAIC3lu4pPF1+LRbH1RGLFXZ+A4cv/SydiOTk9G2aYcOGMW3aND755BNiYmIYNGgQycnJDBgwAIB+/foxcuTI7O0HDRrEmTNnePzxx9mzZw8LFy7k1VdfZfDgwa77FCJS4twTUZkH2v/Z5Xf+VjYfOWtyRX8KaQDN+jmWF48AeyEcyl6kkHE6jPTq1Yvx48czevRomjRpQnR0NIsXL85+qPXIkSOcOHEie/uwsDCWLFnC77//TqNGjXjsscd4/PHHL9kNWETEGSNuqEtk3SDSM+08MGsjx8+lml2SQ6fnwaMUnIiGrXPNrkak0HN6nBEzaJwREbmc82mZ3Pn+GnbFJlGvgj/zB7XGx8PN7LJg9TuwdDSUqgBDNoBnIRoXRaSA5Ms4IyIihY2fpxsf9W9BOT8Pdp5I5Il50djtheDfWBEPQ5lwSDrhCCYiclkKIyJS5FUq48MHfZvjYbOyZEccE5buNrskcPOELmMdy2vedYzSKiKXpDAiIsVC8yqBvHZHQwAmL99fOHrY1O0BVdpB5gWIetHsakQKLYURESk2bm9WiYc7VAfgmflb2frHOXMLslig6yuABbZ9Ccc2mluPSCGlMCIixcrTXWvTuU4QaZl2Hpy1kfjEC+YWFNoEGvd2LP80SrP6ilyCwoiIFCs2q4V3ejehRpAfsYkXeOizjebPYdP5eXDzgsOrYfcic2sRKYQURkSk2Cnl5c60fi3w93Jj85FzPLdgu7lz2ARUgtZ/DvS4dDRkZZhXi0ghpDAiIsVS1XK+TO7TDKsF5m/8g+mrD5lbUNuh4FMOTu+DjTPNrUWkkFEYEZFiq33N8jzXvR4Ary6KYc2+U+YV4+UPnf6cKmPFOLiQYF4tIoWMwoiIFGv3tQ3n9mYVybIbDJ6ziaNnUswrpll/KFcLUk7DqrfNq0OkkFEYEZFizWKx8OptDWlYMYCzKRk89OlGUtNNeqDV5g5dXnIsr52igdBE/qQwIiLFnpe7jQ/6Nqesr2PI+OFfbTXvgdZa3SC8PWSlwfJXzalBpJBRGBGREiG0tDdT+jTDzWrhuy3H+ejXg+YUYrFAlz9HY93yOcRuN6cOkUJEYURESoyIamUZdZPjgdZxP8aw2qwHWis2h/q3AQb8/II5NYgUIgojIlKi9GtdhTubV8JuwKOfb+bYuVRzCuk8CqxusG8pHPjFnBpECgmFEREpUSwWCy/f2oAGFf05k5zOoM82ciHDhAday1aHFvc5lpeOBru94GsQKSQURkSkxPFyt/F+n+aU9nFn6x8JvPDdDnMKufYZ8PCDE9Gw42tzahApBBRGRKRECgv04d3eTbFYYO7vR5m7/kjBF+FXHto+7lheNhYy0wu+BpFCQGFEREqsa2uV56nrawMw+tsdbDl6ruCLaD0Y/ILh7CHYML3gjy9SCCiMiEiJNqhDdbrUCyY9y86gzzZyJrmAr054+ELHP4eJX/kGXEgs2OOLFAIKIyJSolmtFib0bEzVcr4cT7jA0HnRZNkLeEC0pn2hbA3HMPFrJxfssUUKAYURESnx/L3cef9/zfByt7Jyz0neW7a3YAuwuTm6+gKsnQTnTxbs8UVMpjAiIgLUCfHn1dsaAvBO1F5W7I4v2ALq3QKhzSD9PKx8s2CPLWIyhRERkT/d3qwS90RUxjBg6Lzogh0QzWKByBccyxumwxmThqsXMYHCiIjI34y+qR4NKwZwLiWDR2ZvIi2zAAdEq9YBqnUCe4Ym0ZMSRWFERORvvNxtTOnTjABvd7YcPccrC2MKtoCLV0e2fQmx2wr22CImURgREfmHsEAf3u7VGIBZaw/zw9bjBXfw0CZQ/3bAgKiXCu64IiZSGBERuYTOdYIZ1LE6ACO+2sbBU8kFePDnHZPo7f0JDq0uuOOKmERhRETkMp7sUotW4YGcT8vkkdmbCm5CvbLVoVk/x3LUi2AU8LgnIgVMYURE5DLcbFbeu6cpZX09iDmRyIvfF+CEetc+A27ecHQd7FlScMcVMYHCiIjIFQT7ezGxdxMsFvh8/VEWbP6jYA7sXwEiHnIsR70EdnvBHFfEBAojIiL/oX3N8jzauSYAz369nX3x5wvmwO2GgmcAxO+A7fML5pgiJlAYERHJhcevq0mb6mVJzchiyJwCen7Euwy0fcyxvPwVyCzgSfxECojCiIhILtisFib2akI5Pw92xSbx0g87C+bA1wwC3yA4ewg2zyqYY4oUMIUREZFcCvL34u1ejudH5qw7wvdbCmD8EQ9fuPZpx/Ivb0J6Sv4fU6SAKYyIiDihfc3yDO5YA4CRX2/j8OkCGH+k+b1QujKcj4X1H+T/8UQKmMKIiIiThkbWpGV4Gc6nZTJkzub8n7/GzQM6PutYXvU2pJ7L3+OJFDCFERERJ7nZrLx7d1PK+Liz7VgCr/24K/8P2qgnlK8DFxJgzXv5fzyRAqQwIiKSBxUCvJnQ0zF/zYzVh/h5Z1z+HtBqcwwTD/Db+3A+Pn+PJ1KAFEZERPKoc51g7m9XFYCn528hNuFC/h6wzk0Q2gwykuHXt/L3WCIFSGFEROQqPNOtNg0q+nM2JYPH524my56P88hYLHDdaMfyho/h3NH8O5ZIAVIYERG5Cp5uNt67uxm+HjbWHTzD5OX78veA1TpCeHvISodfXs/fY4kUEIUREZGrVLWcL2NvbQDAxJ/3sP7gmfw72N+vjkTPgVN78+9YIgVEYURExAVub1aJ25tWxG7A43M3cy4lH4duD2sFtW4AIwuWv5p/xxEpIAojIiIu8tKtDahazpcTCRcY8dU2DCMfnx+52LNmx9dwYkv+HUekAOQpjEyePJnw8HC8vLyIiIhg/fr1udpv7ty5WCwWbr311rwcVkSkUPPzdOPd3k1xt1lYvCOWeb/n4wOmIQ2gwZ2O5WWv5N9xRAqA02Fk3rx5DBs2jDFjxrBp0yYaN25M165diY+/cp/3Q4cO8dRTT9G+ffs8FysiUtg1rBTAU9fXBuDF73eyL/58/h2s07NgscHeJXBkXf4dRySfOR1G3nrrLR544AEGDBhAvXr1mDp1Kj4+PkyfPv2y+2RlZdGnTx9efPFFqlWrdlUFi4gUdg+0r0bbGmVJzcjisc/zcbj4stWhaR/HctRLkJ+3hUTykVNhJD09nY0bNxIZGflXA1YrkZGRrF279rL7vfTSSwQFBXH//ffn6jhpaWkkJibmeImIFBVWq4W3ejahjI87O08k8ubi3fl3sA7DweYBh1fBgeX5dxyRfORUGDl16hRZWVkEBwfnWB8cHExsbOwl91m1ahUff/wx06ZNy/Vxxo0bR0BAQPYrLCzMmTJFREwX7O/Fm3c6hov/aNVBftlzMn8OFFAJWvz5Dz1dHZEiKl970yQlJdG3b1+mTZtGuXLlcr3fyJEjSUhIyH4dPapRBkWk6ImsF0y/1lUAeOrLLZw+n5Y/B2o/DNx94fhm2PVD/hxDJB85FUbKlSuHzWYjLi7nhFBxcXGEhIT8a/v9+/dz6NAhevTogZubG25ubsyaNYvvvvsONzc39u/ff8njeHp64u/vn+MlIlIUPXtjXWoG+XEyKY3h+dXd1y8IrhnkWF72Ctjz6RkVkXziVBjx8PCgefPmREVFZa+z2+1ERUXRunXrf21fp04dtm3bRnR0dPbr5ptvplOnTkRHR+v2i4gUe17uNt7p3RQPm5WfY+KYs/5I/hyozaPgFQAnY2Db/Pw5hkg+cfo2zbBhw5g2bRqffPIJMTExDBo0iOTkZAYMGABAv379GDlyJABeXl40aNAgx6t06dKUKlWKBg0a4OHh4dpPIyJSCNUL9eeZbo7uvmN/yKfuvt6loe3jjuUVr0JmPo4AK+JiToeRXr16MX78eEaPHk2TJk2Ijo5m8eLF2Q+1HjlyhBMnTri8UBGRouy+tlVpV6McFzLsDJ23mfRMu+sPEvEw+AbB2UOw+VPXty+STyxGvo5X7BqJiYkEBASQkJCg50dEpMiKS7xA14krOZeSwcMdqjPihjquP8i6D+DHZ8AvBB6PBndv1x9DJJdy+/2tuWlERApIsL8Xr93eCIAPVu5n7f7Trj9I83shIAzOx8L63A+pIGImhRERkQLUrUEIvVuGYRjw5BfRJKRmuPYAbp7QcYRjedVbcEGDRkrhpzAiIlLARt1UjyplfTiecIEx3253/QEa9YZytSD1LKyd7Pr2RVxMYUREpID5errxdq8m2KwWvok+zndbjrv2ADY36PScY3ntJEjOh9tBIi6kMCIiYoJmlcswuFMNAJ5fsI3j51Jde4C6N0OFxpB+3nG7RqQQUxgRETHJo51r0DisNIkXMnnqyy3Y7S7s3Gi1QufRjuX10yDhmOvaFnExhREREZO426xM7NUEb3cba/afZvrqg649QI3roEpbyEqDX153bdsiLqQwIiJioqrlfBl1Uz0A3li8m92xSa5r3GKB6/68OrL5Mzh96fnARMymMCIiYrK7W4VxXZ0g0rPsDJ0XTVqmCye6q3wN1OwKRhYse9l17Yq4kMKIiIjJLBYL4+5oSKCvBzEnEpn4817XHuC6UY4/d3wNJ7a4tm0RF1AYEREpBIJKefHqbQ0B+OCX/fx+6IzrGg9pCA3udCxHjXVduyIuojAiIlJIdGsQwp3NK2E3YNgX0ZxPy3Rd452eBasb7FsKh9e4rl0RF1AYEREpRMb0qEfF0t4cPZPK2O93uq7hstWhaV/H8s8vQuGfI1VKEIUREZFCpJSXOxN6NsZigXkbjrJ0Z5zrGu/wDLh5wdHfYM8S17UrcpUURkRECplrqpXlgfbVABj59VZOn09zTcP+oRDxkGN52Viw213TrshVUhgRESmEhnWpRe3gUpw6n86zC7ZhuOq2Stuh4BkAcdth+3zXtClylRRGREQKIS93G2/1aoy7zcKSHXF8vclFw7n7BELbxxzLy8ZCpouuuohcBYUREZFCqn5oAEMjawHwwnc7OOaqyfSuGQR+wXDuCGyY4Zo2Ra6CwoiISCH20LXVaFa5NElpmTz1hYsm0/PwhY4jHMsr34Q0Fw5BL5IHCiMiIoWYm83KhJ6OyfTWHjjNjDWHXNNw074QWB1STsGaSa5pUySPFEZERAq5quV8ebZ7XQDeWLyLffHnr75Rm/tfw8SvnQTnT159myJ5pDAiIlIE/C+iMtfWKk9app1hX0STkeWCbrn1boXQppB+3nG7RsQkCiMiIkWAxWLhjTsa4e/lxtY/EpiyfL8rGoXIFxzLG6bDmYNX36ZIHiiMiIgUESEBXoy9tQEA7y3by7Y/Eq6+0WodoVonsGfA8leuvj2RPFAYEREpQm5uHMqNDUPItBsM+yKaCxlZV9/oxasj276E49FX356IkxRGRESKEIvFwsu3NqScnyd7488z4afdV99oaBNo2NOxvHSUJtGTAqcwIiJSxAT6evD6HQ0B+GjVQdYdOH31jXZ+HmwecHAl7Pv56tsTcYLCiIhIEXRd3WB6tqiEYcBT87dwPi3z6hosUwVaPehYXjoa7C64/SOSSwojIiJF1Kib6lGxtDdHz6TyysKYq2+w/ZPgFQDxO2HL51ffnkguKYyIiBRRpbzcefOuRgB8vv4Iy3fHX12DPoHQ/inH8rJXID3lKisUyR2FERGRIqxN9XIMaBsOwPD5WzmXkn51DbZ6EALCIOk4rHv/6gsUyQWFERGRIm54tzpUK+9LfFIao7/dcXWNuXtB5z+Hif/1bUg+dfUFivwHhRERkSLOy93GWz2bYLNa+G7LcRZuPXF1DTa8C0IaQXoSrHjNNUWKXIHCiIhIMdAkrDSPdKwOwPPfbCM+6ULeG7Naoeufo7FumA4nXTCWicgVKIyIiBQTj3auSf1Qf86mZDDyq20YVzN4WdVroXZ3MLLgp1GuK1LkEhRGRESKCQ83K2/1bIKHzUrUrni+3PDH1TXY5SWwusHeJbB/mWuKFLkEhRERkWKkdkgpnry+FgAvfr+Do2euontuuRrQ8gHH8pLnNRCa5BuFERGRYmZg+2q0DC9DcnoWT325Bbv9Km7XdHgGvEpD/A7Y/JnLahT5O4UREZFixma1MP6uxvh42Fh38Awz1hzKe2M+gdBhuGN52cuQluSSGkX+TmFERKQYqlLWl+e61wXg9cW72Bd/FSGi5UAIrAbJ8bDqbRdVKPIXhRERkWLqnlaV6VCrPOmZdp6Yt4WMLHveGnLzgC5jHctrJsHZQy6rUQQURkREii2LxcLrdzQiwNudbccSmLRsX94bq9Pd0d03Kw1+et51RYqgMCIiUqyFBHgx9tYGAExavo/oo+fy1pDFAt1eB4sNYr6HAytcVqOIwoiISDF3c+NQejQOJctuMGxeNKnpeeyiG1wPWt7vWP5xBGRluq5IKdEURkRESoCxt9Qn2N+TA6eSeX3xrrw31HEkeAfCyRjY8LHrCpQSLU9hZPLkyYSHh+Pl5UVERATr16+/7LbTpk2jffv2lClThjJlyhAZGXnF7UVExPVK+3jwxp2NAZi55hC/7j2Zt4Z8AqHzn8+MLH8Fkk+7qEIpyZwOI/PmzWPYsGGMGTOGTZs20bhxY7p27Up8fPwlt1+xYgV33303y5cvZ+3atYSFhXH99ddz7Nixqy5eRERyr0Ot8vS9pgoAT3+5lYSUjLw11PxeCG4IFxJg2VjXFSgllsVwcialiIgIWrZsyaRJkwCw2+2EhYXx6KOPMmLEiP/cPysrizJlyjBp0iT69euXq2MmJiYSEBBAQkIC/v7+zpQrIiJ/k5KeSfd3V3HwVDI3Nw7l3bub5q2hQ6th5o2ABR76BSo0dmmdUjzk9vvbqSsj6enpbNy4kcjIyL8asFqJjIxk7dq1uWojJSWFjIwMAgMDL7tNWloaiYmJOV4iInL1fDzceLtXE2xWC99tOc630Xm8Sh3eFurfDhiw8Emw53EMExGcDCOnTp0iKyuL4ODgHOuDg4OJjY3NVRvDhw8nNDQ0R6D5p3HjxhEQEJD9CgsLc6ZMERG5giZhpXm0cw0Anv9mO8fPpeatoa6vgIcf/PE7bP7UhRVKSVOgvWlee+015s6dy4IFC/Dy8rrsdiNHjiQhISH7dfTo0QKsUkSk+BvSqQaNw0qTdCGTJ7/I42R6/qGO3jUAP4/Rw6ySZ06FkXLlymGz2YiLi8uxPi4ujpCQkCvuO378eF577TV++uknGjVqdMVtPT098ff3z/ESERHXcbNZmdirCd7uNtYeOM301Qfz1lDEQxBUH1LPOgKJSB44FUY8PDxo3rw5UVFR2evsdjtRUVG0bt36svu98cYbjB07lsWLF9OiRYu8VysiIi5TtZwvz9/kmEzvjcW72R2bh8n0bO7QfYJjefOncGSdCyuUksLp2zTDhg1j2rRpfPLJJ8TExDBo0CCSk5MZMGAAAP369WPkyJHZ27/++uuMGjWK6dOnEx4eTmxsLLGxsZw/f951n0JERPLknlaVua5OEOlZdh6fu5kLGXkYnbVKa2jyP8fywmEamVWc5nQY6dWrF+PHj2f06NE0adKE6OhoFi9enP1Q65EjRzhx4kT29u+//z7p6enceeedVKhQIfs1fvx4130KERHJE4vFwmt3NKKsrwe7YpN4Y/HuvDXU5UXwKg1x22H9hy6tUYo/p8cZMYPGGRERyV/LdsVx38wNAHxyXys61CrvfCMbZsAPQx09bB75DUqrJ2RJly/jjIiISPHUuU4w/Vs7Rmd96sstnD6f5nwjzfpDWASkn3fcrin8/9aVQkJhREREABh5Y11qBvlxMimN4V9txekL51Yr9HgXbB6w9yfY/lX+FCrFjsKIiIgA4OVu453eTfGwWfk5Jp7P1h1xvpGgOtD+Kcfyj89o7BHJFYURERHJVi/Un2e61Qbg5R92sjcuD9192z0B5etCymlY8qyLK5TiSGFERERyuK9tVdrXLEdapp0hc/LQ3dfNA25+D7DA1rmw7+d8qVOKD4URERHJwWq18FbPJpTz82R3XBIvL9zpfCNhLSHiYcfy909AmsaWkstTGBERkX8pX8qTt3o2BuCz347w47YT/7HHJXR+HgIqQ8IR+PkF1xYoxYrCiIiIXNK1tcrzcIfqAAz/ait/nE1xrgFPP7j5Hcfy79Ng/3IXVyjFhcKIiIhc1pPX16JJWGkSL2Ty+NxoMrPszjVQvTO0HOhY/nYwpJ5zeY1S9CmMiIjIZbnbrLx3d1NKebqx8fBZ3v55j/ONdHkJAqtB4jH4cbjri5QiT2FERESuKCzQh3F3NARg8vL9rNgd71wDHr5w61SwWB29a2K+z4cqpShTGBERkf90U6NQ/ndNZQCemBfN8XOpzjVQOQLaPu5Y/n4onD/p2gKlSFMYERGRXBl1Uz0aVgzgbEoGg+dsIj3TyedHOo6E4AaQcgq+f1xz10g2hREREckVTzcbU/o0o5SXG5uPnOO1H3c514CbJ9w2FazusHsh/P5R/hQqRY7CiIiI5FpYoA8T7nKMPzJ99UHnxx8JaQhdXnQsL3kOTmx1cYVSFCmMiIiIU66vH8KD11YD4Jn5Wzl4Ktm5Bq55BGp1g6w0mD9Ao7OKwoiIiDjv6a61aRlehqS0TB6ctYHzaZm539ligVumQKlQOL0PFj2Vf4VKkaAwIiIiTnO3WZl8TzOCSnmyN/48T32xBcOZB1J9y8KdHzu6+275HKLn5F+xUugpjIiISJ4E+Xvx/v+a426zsHhHLJOX73OugSptHD1sABY+CSd3u75IKRIURkREJM+aVynDS7c0AGDC0j0s2xXnXAPtn4Sq10JGCsz7H1xIzIcqpbBTGBERkatyd6vK3BNRGcOAx+dGO/dAq9UGt3/keH7k1B74+kGwOzl+iRR5CiMiInLVXuhRn+ZVypB0IZMHZm0gITUj9zuXCoben4HNE/b8CCtezb9CpVBSGBERkavm4Wbl/T7NCPH3Yl/8eR6ZvZEMZ2b4rdgcerzjWF75Juz8Nn8KlUJJYURERFwiyN+Lj+9tgY+HjdX7TvP8gu3O9bBpcrdjDBKABYMgbkf+FCqFjsKIiIi4TP3QACbd0xSrBeZtOMrUXw4410CXsVC1A2Qkw+d3a0K9EkJhREREXKpznWDG9KgPwOuLd7HImSHjbW5w10woEw7nDsPsOzVCawmgMCIiIi7Xv00497YJB+CJedFsPHw29zv7BEKfr8CnLJyIhi/6QZYTD8RKkaMwIiIi+WLUTfW4rk4QaZl2BsxYT8wJJ8YQKVcD7vkS3H1gfxR8OwScef5EihSFERERyRc2q4X37mlKiyplSLyQSd+P13PImTFIKjWHnrPAYoOtc+HnF/KtVjGXwoiIiOQbHw83Pr63JXUr+HPqfBp9PlpHbMKF3DdQswvcMsmxvHoirH43X+oUcymMiIhIvgrwdmfWfa2oWs6XY+dS+d/H6ziTnJ77BprcA9eNcSwvHQWrJuZLnWIehREREcl35Ut58un9ragQ4BgUrd/0dZxLcSKQtHvir0n1fh7jGBhNig2FERERKRCVyvjw6f0RlPX1YPuxRHp/+Bsnk9Jyt7PFAh1HQOfnHT8vexlWvJZ/xUqBUhgREZECUyPIj7kPXkNQKU92xSbR64O1nEhIzX0D1z4NkS86lleMc4QS9bIp8hRGRESkQNUMLsUXD7WmYmlvDpxK5q6pazlyOiX3DbQbCte/4lhe+SZ8NwQynbjlI4WOwoiIiBS48HK+zHvoGsLL+vDH2VR6frCWvXFJuW+gzRDoPgEsVtj8GXx2O6Scyb+CJV8pjIiIiCkqlfHhi4daUzPIj9jEC9w+ZQ3Ld8fnvoGWA+GeL8CjFBz6FT6KhFP78q9gyTcKIyIiYpogfy/mPdSaVuGBJKVlcv/M35m28kDuZ/ut2QXu/wkCKsOZ/fDRdbB/ef4WLS6nMCIiIqYK9PXgs4ER9G4Zht2AVxbF8PT8raRlZuWugeB68EAUVGwBF87Bp7fB0tF6jqQIURgRERHTebhZGXd7Q8b0qIfVAvM3/kHvD3/L/YOtfkFw7w/QrD9gwOp34ONIOLknX+sW11AYERGRQsFisTCgbVVmDmiFv5cbm4+co9s7K5m97nDubtu4e8PN70Kvz8A7EE5sgQ+uhd8/Ars9/z+A5JnFyPWNOfMkJiYSEBBAQkIC/v7+ZpcjIiL57OiZFJ78cgvrDzp6yHSoVZ7X72hESIBX7hpIPAHfDIIDfz4/UrEFdHsNwlrmU8VyKbn9/lYYERGRQsluN5i++iBvLNlNeqYdfy83nulWh14tw3C35eLCvt0O66Y6BkbL+HO24IY9IfIFCKiYr7WLg8KIiIgUC/vikxj2xRa2/pEAQHhZH568vjbdG1bAarX8dwNJsRA1FqJnAwa4+0DEw9DqQfCvkL/Fl3AKIyIiUmxkZNn57LfDTFq2j9N/zvhbP9Sfp66vTYda5XMXSo5vhh9HwNHfHD9b3aHB7XDNIxDaJP+KL8Fy+/2dpwdYJ0+eTHh4OF5eXkRERLB+/forbv/ll19Sp04dvLy8aNiwIYsWLcrLYUVEpIRyt1kZ0LYqvzzTiScia+Hn6caO44kMmPk71765nHd+3ssfZ/+j501oU7hvMfSaDZVbgz0Dts6DDzvA9Btgw3RIiiuYDyQ5OH1lZN68efTr14+pU6cSERHBxIkT+fLLL9m9ezdBQUH/2n7NmjVce+21jBs3jptuuok5c+bw+uuvs2nTJho0aJCrY+rKiIiI/N3p82lMWbGfL34/SlJaJuCY2Ldt9XJ0bRBCRNVAapT3u/IVk2Ob4LcpsGMB2DP/XGmBsFZQ5ybHgGrlaoHVlv8fqJjKt9s0ERERtGzZkkmTJgFgt9sJCwvj0UcfZcSIEf/avlevXiQnJ/PDDz9kr7vmmmto0qQJU6dOdemHERGRkiU1PYslO2L5YsNR1uw/neO9Mj7utAwPpHmVMlQp60tYoDdhgT74e7nnbCTxOGyZC7t+gGMbc77n4QcVGjuuqoQ2hTJVHQ+/+gaBVaNj/Jfcfn+7OdNoeno6GzduZOTIkdnrrFYrkZGRrF279pL7rF27lmHDhuVY17VrV7755pvLHictLY20tLTsnxMTE50pU0RESghvDxu3Nq3IrU0rcvRMCt9GH+O3A2fYePgsZ1My+GlnHD/tzHnrJcDbnbJ+HpTydMPX0w0/Tzd8PDrhVvo6AkudpEHSKuon/EpYyg480s/D4dWO199kWdxIdC9Psnsg6VYv0q3eZFi9ybB5YccNw2LBwPHiz+XCrvKNTxEaXtuUYzsVRk6dOkVWVhbBwcE51gcHB7Nr165L7hMbG3vJ7WNjYy97nHHjxvHiiy86U5qIiJRwYYE+DOlckyGdHQ+8bj+WwPqDZ9h6LIE/zqRw9GwqZ5LTSUjNICE14wottQRaYsVODcsxGlkP0MhygHrWw1S0nCKIs9jIpEz6Ccqknyioj5fvdp3qXTTCSEEZOXJkjqspiYmJhIWFmViRiIgUJe42K00rl6Fp5TI51p9Py+SPsymcTc4gOS2T5PRMzqdlkpKWRabdwG4YZGYZZNntOJ5hqIVhdCIeiPtzjcXIxC/9FP5pcXhnnsM96wLu9lTHK+sCFiPLcU3EsMOf10eKgqpBlU07tlNhpFy5cthsNuLicl7yiouLIyQk5JL7hISEOLU9gKenJ56ens6UJiIi8p/8PN2oE6JnDwsbp56+8fDwoHnz5kRFRWWvs9vtREVF0bp160vu07p16xzbAyxduvSy24uIiEjJ4vRtmmHDhtG/f39atGhBq1atmDhxIsnJyQwYMACAfv36UbFiRcaNGwfA448/TocOHZgwYQLdu3dn7ty5bNiwgQ8//NC1n0RERESKJKfDSK9evTh58iSjR48mNjaWJk2asHjx4uyHVI8cOYL1b92d2rRpw5w5c3j++ed59tlnqVmzJt98802uxxgRERGR4k3DwYuIiEi+yNfh4EVERERcRWFERERETKUwIiIiIqZSGBERERFTKYyIiIiIqRRGRERExFQKIyIiImIqhRERERExlcKIiIiImMrp4eDNcHGQ2MTERJMrERERkdy6+L39X4O9F4kwkpSUBEBYWJjJlYiIiIizkpKSCAgIuOz7RWJuGrvdzvHjxylVqhQWi8Vl7SYmJhIWFsbRo0c1581V0Hl0DZ1H19B5dA2dR9co6efRMAySkpIIDQ3NMYnuPxWJKyNWq5VKlSrlW/v+/v4l8pfE1XQeXUPn0TV0Hl1D59E1SvJ5vNIVkYv0AKuIiIiYSmFERERETFWiw4inpydjxozB09PT7FKKNJ1H19B5dA2dR9fQeXQNncfcKRIPsIqIiEjxVaKvjIiIiIj5FEZERETEVAojIiIiYiqFERERETFVsQ8jkydPJjw8HC8vLyIiIli/fv0Vt//yyy+pU6cOXl5eNGzYkEWLFhVQpYWbM+dx2rRptG/fnjJlylCmTBkiIyP/87yXFM7+Pl40d+5cLBYLt956a/4WWEQ4ex7PnTvH4MGDqVChAp6entSqVUt/t3H+PE6cOJHatWvj7e1NWFgYTzzxBBcuXCigagunlStX0qNHD0JDQ7FYLHzzzTf/uc+KFSto1qwZnp6e1KhRg5kzZ+Z7nYWeUYzNnTvX8PDwMKZPn27s2LHDeOCBB4zSpUsbcXFxl9x+9erVhs1mM9544w1j586dxvPPP2+4u7sb27ZtK+DKCxdnz+M999xjTJ482di8ebMRExNj3HvvvUZAQIDxxx9/FHDlhYuz5/GigwcPGhUrVjTat29v3HLLLQVTbCHm7HlMS0szWrRoYdx4443GqlWrjIMHDxorVqwwoqOjC7jywsXZ8zh79mzD09PTmD17tnHw4EFjyZIlRoUKFYwnnniigCsvXBYtWmQ899xzxtdff20AxoIFC664/YEDBwwfHx9j2LBhxs6dO4333nvPsNlsxuLFiwum4EKqWIeRVq1aGYMHD87+OSsrywgNDTXGjRt3ye179uxpdO/ePce6iIgI46GHHsrXOgs7Z8/jP2VmZhqlSpUyPvnkk/wqsUjIy3nMzMw02rRpY3z00UdG//79FUYM58/j+++/b1SrVs1IT08vqBKLBGfP4+DBg43OnTvnWDds2DCjbdu2+VpnUZKbMPLMM88Y9evXz7GuV69eRteuXfOxssKv2N6mSU9PZ+PGjURGRmavs1qtREZGsnbt2kvus3bt2hzbA3Tt2vWy25cEeTmP/5SSkkJGRgaBgYH5VWahl9fz+NJLLxEUFMT9999fEGUWenk5j9999x2tW7dm8ODBBAcH06BBA1599VWysrIKquxCJy/nsU2bNmzcuDH7Vs6BAwdYtGgRN954Y4HUXFzoe+bSisREeXlx6tQpsrKyCA4OzrE+ODiYXbt2XXKf2NjYS24fGxubb3UWdnk5j/80fPhwQkND//UXsCTJy3lctWoVH3/8MdHR0QVQYdGQl/N44MABli1bRp8+fVi0aBH79u3jkUceISMjgzFjxhRE2YVOXs7jPffcw6lTp2jXrh2GYZCZmcnDDz/Ms88+WxAlFxuX+55JTEwkNTUVb29vkyozV7G9MiKFw2uvvcbcuXNZsGABXl5eZpdTZCQlJdG3b1+mTZtGuXLlzC6nSLPb7QQFBfHhhx/SvHlzevXqxXPPPcfUqVPNLq1IWbFiBa+++ipTpkxh06ZNfP311yxcuJCxY8eaXZoUA8X2yki5cuWw2WzExcXlWB8XF0dISMgl9wkJCXFq+5IgL+fxovHjx/Paa6/x888/06hRo/wss9Bz9jzu37+fQ4cO0aNHj+x1drsdADc3N3bv3k316tXzt+hCKC+/jxUqVMDd3R2bzZa9rm7dusTGxpKeno6Hh0e+1lwY5eU8jho1ir59+zJw4EAAGjZsSHJyMg8++CDPPfccVqv+bZsbl/ue8ff3L7FXRaAYXxnx8PCgefPmREVFZa+z2+1ERUXRunXrS+7TunXrHNsDLF269LLblwR5OY8Ab7zxBmPHjmXx4sW0aNGiIEot1Jw9j3Xq1GHbtm1ER0dnv26++WY6depEdHQ0YWFhBVl+oZGX38e2bduyb9++7DAHsGfPHipUqFAigwjk7TympKT8K3BcDHiGpjjLNX3PXIbZT9Dmp7lz5xqenp7GzJkzjZ07dxoPPvigUbp0aSM2NtYwDMPo27evMWLEiOztV69ebbi5uRnjx483YmJijDFjxqhrr+H8eXzttdcMDw8PY/78+caJEyeyX0lJSWZ9hELB2fP4T+pN4+DseTxy5IhRqlQpY8iQIcbu3buNH374wQgKCjJefvllsz5CoeDseRwzZoxRqlQp4/PPPzcOHDhg/PTTT0b16tWNnj17mvURCoWkpCRj8+bNxubNmw3AeOutt4zNmzcbhw8fNgzDMEaMGGH07ds3e/uLXXuffvppIyYmxpg8ebK69hrFvGuvYRjGe++9Z1SuXNnw8PAwWrVqZfz222/Z73Xo0MHo379/ju2/+OILo1atWoaHh4dRv359Y+HChQVcceHkzHmsUqWKAfzrNWbMmIIvvJBx9vfx7xRG/uLseVyzZo0RERFheHp6GtWqVTNeeeUVIzMzs4CrLnycOY8ZGRnGCy+8YFSvXt3w8vIywsLCjEceecQ4e/ZswRdeiCxfvvyS/7+7eO769+9vdOjQ4V/7NGnSxPDw8DCqVatmzJgxo8DrLmwshqHrayIiImKeYvvMiIiIiBQNCiMiIiJiKoURERERMZXCiIiIiJhKYURERERMpTAiIiIiplIYEREREVMpjIiIiIipFEZERETEVAojIiIiYiqFERERETGVwoiIiIiY6v+VQwBGLmPG5gAAAABJRU5ErkJggg==",
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"x = jnp.linspace(0.0, 1.1, 100)\n",
"\n",
"plt.plot(x, e3nn.soft_envelope(x), label=\"soft_envelope\")\n",
"plt.plot(x, e3nn.poly_envelope(6, 2)(x), label=\"poly_envelope(6, 2)\")\n",
"\n",
"plt.legend()\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Normalization\n",
"\n",
"If the data distribution at **input** satisfies the following conditions:\n",
"\n",
"$$ \\langle x_i^2 \\rangle = 1 $$\n",
"\n",
"Then the function $f$ is **component normalized** if the output distribution satisfies the same condition:\n",
"\n",
"$$ \\langle f(x)_i^2 \\rangle = 1 $$"
]
},
{
"cell_type": "code",
"execution_count": 62,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x0e+1x0e+1x0e [0.966 2.9739 4.906 ]"
]
},
"execution_count": 62,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x = e3nn.normal(\"1o\", jax.random.PRNGKey(0), (10_000,))\n",
"y = e3nn.normal(\"1o\", jax.random.PRNGKey(1), (10_000,))\n",
"\n",
"e3nn.mean(e3nn.norm(e3nn.tensor_product(x, y), squared=True), axis=0)"
]
},
{
"cell_type": "code",
"execution_count": 63,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x0e+1x0e [0.9995 4.9973]"
]
},
"execution_count": 63,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x = e3nn.normal(\"1o\", jax.random.PRNGKey(0), (10_000,))\n",
"\n",
"e3nn.mean(e3nn.norm(e3nn.tensor_square(x), squared=True), axis=0)"
]
},
{
"cell_type": "code",
"execution_count": 64,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1x0e+1x0e+1x0e [1. 3. 5.]"
]
},
"execution_count": 64,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x = e3nn.normal(\"1o\", jax.random.PRNGKey(0), (10_000,))\n",
"x = x / e3nn.norm(x)\n",
"\n",
"e3nn.mean(e3nn.norm(e3nn.spherical_harmonics([0, 1, 2], x, False, normalization=\"component\"), squared=True), axis=0)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# The `.list` optimization"
]
},
{
"cell_type": "code",
"execution_count": 65,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[None]"
]
},
"execution_count": 65,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import haiku as hk\n",
"\n",
"x = e3nn.IrrepsArray(\"1o\", jnp.array([1.0, 0.0, 0.0]))\n",
"\n",
"foo = hk.transform(lambda x: e3nn.Linear(\"1e\")(x))\n",
"\n",
"w = foo.init(jax.random.PRNGKey(0), x)\n",
"foo.apply(w, jax.random.PRNGKey(0), x).list"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.10.7 ('base')",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.8"
},
"orig_nbformat": 4,
"vscode": {
"interpreter": {
"hash": "f26faf9d33dc8b83cd077f62f5d9010e5bc51611e479f12b96223e2da63ba699"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment