Skip to content

Instantly share code, notes, and snippets.

@ankschoubey

ankschoubey/20191021_broadcasting_and_rules.ipynb

Created October 21, 2019 15:24
Show Gist options
  • Save ankschoubey/fd46436315561b7a27c7463ce61c695e to your computer and use it in GitHub Desktop.
Save ankschoubey/fd46436315561b7a27c7463ce61c695e to your computer and use it in GitHub Desktop.
Download ZIP
20191021_broadcasting_and_rules.ipynb
Display the source blob
Display the rendered blob
Raw
20191021_broadcasting_and_rules.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "20191021_broadcasting_and_rules.ipynb",
"provenance": [],
"collapsed_sections": [],
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/ankschoubey/fd46436315561b7a27c7463ce61c695e/20191021_broadcasting_and_rules.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "yPSRgrrmTvaH",
"colab_type": "text"
},
"source": [
"FastAI Course v3 Lesson 8 Notes: https://forums.fast.ai/t/lesson-8-notes/41442/22\n",
"\n",
"Broadcasting Numpy: https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html\n",
"\n",
"Broadcasting Pytorch: https://pytorch.org/docs/stable/notes/broadcasting.html\n",
"\n",
"---\n",
"\n",
"\n",
"This is one of my random practice notebook.\n",
"\n",
"Most/all content/ideas from this ntoebook are not my own but from source are links above. #NoPlagiarism\n",
"\n",
"---\n",
"\n",
"> Subject to certain constraints, the smaller array is “broadcast” across the larger array so that they have compatible shapes. \n",
"\n",
"> Broadcasting provides a means of vectorizing array operations so that looping occurs in C instead of Python. \n",
"\n",
"> It does this without making needless copies of data and usually leads to efficient algorithm implementations.\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "P9rVf9l9TOE2",
"colab_type": "code",
"colab": {}
},
"source": [
"import torch"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "Kt5oQjt3U2wR",
"colab_type": "code",
"colab": {}
},
"source": [
"def print_tensor_info(x):\n",
" print(f'{x}')\n",
" print(f'\\nShape: {x.shape}')\n",
" print(f'\\nStride: {x.stride()}')\n",
" print(f'\\nStorage: {x.storage()}')"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "0fC9vMawVp7G",
"colab_type": "code",
"colab": {}
},
"source": [
"x = torch.ones(3,3)\n",
"print_tensor_info(x)\n",
"\n",
"# the stride, tells when it is going\n",
"# row to row, it takes 3 steps (constant values)\n",
"# column to column it takes 1 step"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "rgaV9rtEUiPj",
"colab_type": "code",
"colab": {}
},
"source": [
"m = torch.Tensor([1,2,3])\n",
"print_tensor_info(m)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "RXIYZ7a9UrjR",
"colab_type": "code",
"colab": {}
},
"source": [
"broadcasted = m.expand_as(x)\n",
"print(f'Original: {m}')\n",
"print_tensor_info(broadcasted)\n",
"\n",
"# the stride, tells when it is going\n",
"# row to row, it takes 0 steps (constant values)\n",
"# column to column it takes 1 step \n",
"\n",
"#INSIGHT: take a look at storage"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "WDe6zQ8VYgye",
"colab_type": "text"
},
"source": [
"### Broadcasting Rules\n",
"\n",
"> Two tensors are “broadcastable” if the following rules hold:\n",
"> - Each tensor has at least one dimension."
]
},
{
"cell_type": "code",
"metadata": {
"id": "wjP4FZVhV5PO",
"colab_type": "code",
"colab": {}
},
"source": [
"m = torch.Tensor(0)\n",
"m, m.shape"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "1GKZt_mC2QjM",
"colab_type": "code",
"colab": {}
},
"source": [
"m.expand_as(x)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "S4jbjGh82B7V",
"colab_type": "text"
},
"source": [
"> - When iterating over the dimension sizes, starting at the trailing dimension, the dimension sizes must either be equal, one of them is 1, or one of them does not exist."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "z6HrPiAL2v5t",
"colab_type": "text"
},
"source": [
"#### example 1: both shape are one"
]
},
{
"cell_type": "code",
"metadata": {
"id": "u-M1SkUM23iu",
"colab_type": "code",
"colab": {}
},
"source": [
"x.shape"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "mHlQjiqa2uYL",
"colab_type": "code",
"colab": {}
},
"source": [
"m = torch.Tensor(1,1) # of of them is 1\n",
"m, m.shape"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "zWIaSeVe27KG",
"colab_type": "code",
"colab": {}
},
"source": [
"m.expand_as(x)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "vKAA_zJe3KQR",
"colab_type": "text"
},
"source": [
"### example 2: of dimension is one other is equal"
]
},
{
"cell_type": "code",
"metadata": {
"id": "ARPv1ebT3Iiu",
"colab_type": "code",
"colab": {}
},
"source": [
"x.shape"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "Y1FJ15Gq3a71",
"colab_type": "code",
"colab": {}
},
"source": [
"m=torch.Tensor(3,1)\n",
"m, m.shape"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "VOa8clqe3mrG",
"colab_type": "code",
"colab": {}
},
"source": [
"m.expand_as(x)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "RCHquDDU3pnz",
"colab_type": "code",
"colab": {}
},
"source": [
"m=torch.Tensor(1,3)\n",
"m, m.shape"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "NVOe20IT3unS",
"colab_type": "code",
"colab": {}
},
"source": [
"m.expand_as(x)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "_c_lNIrs3z71",
"colab_type": "text"
},
"source": [
"### example 3: less number of dimensions"
]
},
{
"cell_type": "code",
"metadata": {
"id": "eBeD6lA13v7C",
"colab_type": "code",
"colab": {}
},
"source": [
"m=torch.Tensor(1)\n",
"m, m.shape"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "ybFRgzpw39cp",
"colab_type": "code",
"colab": {}
},
"source": [
"m.expand_as(x)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "Wp4dssGA4FQa",
"colab_type": "text"
},
"source": [
"example 4: different dimensions. None is one."
]
},
{
"cell_type": "code",
"metadata": {
"id": "XXiI4AIO4FAU",
"colab_type": "code",
"colab": {}
},
"source": [
"m=torch.Tensor(2,2)\n",
"m, m.shape"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "mvhzN7O64EVt",
"colab_type": "code",
"colab": {}
},
"source": [
"m.expand_as(x)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "KH2UJ5CK4UKv",
"colab_type": "text"
},
"source": [
"Example 6: One of the dimension is 1 but other is not"
]
},
{
"cell_type": "code",
"metadata": {
"id": "e1aomdFm4AzZ",
"colab_type": "code",
"colab": {}
},
"source": [
"m=torch.Tensor(2,1)\n",
"m, m.shape"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "IGTYEEhk4c6_",
"colab_type": "code",
"colab": {}
},
"source": [
"m.expand_as(x)"
],
"execution_count": 0,
"outputs": []
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment