Created
October 21, 2019 15:24
-
-
Save ankschoubey/fd46436315561b7a27c7463ce61c695e to your computer and use it in GitHub Desktop.
20191021_broadcasting_and_rules.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"nbformat": 4, | |
"nbformat_minor": 0, | |
"metadata": { | |
"colab": { | |
"name": "20191021_broadcasting_and_rules.ipynb", | |
"provenance": [], | |
"collapsed_sections": [], | |
"include_colab_link": true | |
}, | |
"kernelspec": { | |
"name": "python3", | |
"display_name": "Python 3" | |
} | |
}, | |
"cells": [ | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"id": "view-in-github", | |
"colab_type": "text" | |
}, | |
"source": [ | |
"<a href=\"https://colab.research.google.com/gist/ankschoubey/fd46436315561b7a27c7463ce61c695e/20191021_broadcasting_and_rules.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>" | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"id": "yPSRgrrmTvaH", | |
"colab_type": "text" | |
}, | |
"source": [ | |
"FastAI Course v3 Lesson 8 Notes: https://forums.fast.ai/t/lesson-8-notes/41442/22\n", | |
"\n", | |
"Broadcasting Numpy: https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html\n", | |
"\n", | |
"Broadcasting Pytorch: https://pytorch.org/docs/stable/notes/broadcasting.html\n", | |
"\n", | |
"---\n", | |
"\n", | |
"\n", | |
"This is one of my random practice notebook.\n", | |
"\n", | |
"Most/all content/ideas from this ntoebook are not my own but from source are links above. #NoPlagiarism\n", | |
"\n", | |
"---\n", | |
"\n", | |
"> Subject to certain constraints, the smaller array is “broadcast” across the larger array so that they have compatible shapes. \n", | |
"\n", | |
"> Broadcasting provides a means of vectorizing array operations so that looping occurs in C instead of Python. \n", | |
"\n", | |
"> It does this without making needless copies of data and usually leads to efficient algorithm implementations.\n" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "P9rVf9l9TOE2", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"import torch" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "Kt5oQjt3U2wR", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"def print_tensor_info(x):\n", | |
" print(f'{x}')\n", | |
" print(f'\\nShape: {x.shape}')\n", | |
" print(f'\\nStride: {x.stride()}')\n", | |
" print(f'\\nStorage: {x.storage()}')" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "0fC9vMawVp7G", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"x = torch.ones(3,3)\n", | |
"print_tensor_info(x)\n", | |
"\n", | |
"# the stride, tells when it is going\n", | |
"# row to row, it takes 3 steps (constant values)\n", | |
"# column to column it takes 1 step" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "rgaV9rtEUiPj", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"m = torch.Tensor([1,2,3])\n", | |
"print_tensor_info(m)" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "RXIYZ7a9UrjR", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"broadcasted = m.expand_as(x)\n", | |
"print(f'Original: {m}')\n", | |
"print_tensor_info(broadcasted)\n", | |
"\n", | |
"# the stride, tells when it is going\n", | |
"# row to row, it takes 0 steps (constant values)\n", | |
"# column to column it takes 1 step \n", | |
"\n", | |
"#INSIGHT: take a look at storage" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"id": "WDe6zQ8VYgye", | |
"colab_type": "text" | |
}, | |
"source": [ | |
"### Broadcasting Rules\n", | |
"\n", | |
"> Two tensors are “broadcastable” if the following rules hold:\n", | |
"> - Each tensor has at least one dimension." | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "wjP4FZVhV5PO", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"m = torch.Tensor(0)\n", | |
"m, m.shape" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "1GKZt_mC2QjM", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"m.expand_as(x)" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"id": "S4jbjGh82B7V", | |
"colab_type": "text" | |
}, | |
"source": [ | |
"> - When iterating over the dimension sizes, starting at the trailing dimension, the dimension sizes must either be equal, one of them is 1, or one of them does not exist." | |
] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"id": "z6HrPiAL2v5t", | |
"colab_type": "text" | |
}, | |
"source": [ | |
"#### example 1: both shape are one" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "u-M1SkUM23iu", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"x.shape" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "mHlQjiqa2uYL", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"m = torch.Tensor(1,1) # of of them is 1\n", | |
"m, m.shape" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "zWIaSeVe27KG", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"m.expand_as(x)" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"id": "vKAA_zJe3KQR", | |
"colab_type": "text" | |
}, | |
"source": [ | |
"### example 2: of dimension is one other is equal" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "ARPv1ebT3Iiu", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"x.shape" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "Y1FJ15Gq3a71", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"m=torch.Tensor(3,1)\n", | |
"m, m.shape" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "VOa8clqe3mrG", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"m.expand_as(x)" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "RCHquDDU3pnz", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"m=torch.Tensor(1,3)\n", | |
"m, m.shape" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "NVOe20IT3unS", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"m.expand_as(x)" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"id": "_c_lNIrs3z71", | |
"colab_type": "text" | |
}, | |
"source": [ | |
"### example 3: less number of dimensions" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "eBeD6lA13v7C", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"m=torch.Tensor(1)\n", | |
"m, m.shape" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "ybFRgzpw39cp", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"m.expand_as(x)" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"id": "Wp4dssGA4FQa", | |
"colab_type": "text" | |
}, | |
"source": [ | |
"example 4: different dimensions. None is one." | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "XXiI4AIO4FAU", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"m=torch.Tensor(2,2)\n", | |
"m, m.shape" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "mvhzN7O64EVt", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"m.expand_as(x)" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "markdown", | |
"metadata": { | |
"id": "KH2UJ5CK4UKv", | |
"colab_type": "text" | |
}, | |
"source": [ | |
"Example 6: One of the dimension is 1 but other is not" | |
] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "e1aomdFm4AzZ", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"m=torch.Tensor(2,1)\n", | |
"m, m.shape" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
}, | |
{ | |
"cell_type": "code", | |
"metadata": { | |
"id": "IGTYEEhk4c6_", | |
"colab_type": "code", | |
"colab": {} | |
}, | |
"source": [ | |
"m.expand_as(x)" | |
], | |
"execution_count": 0, | |
"outputs": [] | |
} | |
] | |
} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment