slots and dataclass

slots-and-dataclass.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/hansonkim/6d5d61e1c33b66b0556d4535fa91a21f/slots-and-dataclass.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Class memory 점유 및 실행시간\n",
        "\n",
        "python class 내부적으로 dict 형태로 데이터를 저장하는 관계로 메모리 사용량이 높다.\n",
        "이를 줄이기 위한 방안으로 __slots__ 변수를 사용하면 dict 형태로 저장하는 것을 막아 메모리를 절약할 수 있다.\n",
        "dataclass 형태의 간소한 class 작성을 선호하는데 [PEP557](https://peps.python.org/pep-0557/) 에 따르면 repr, init, eq, ne, gt 등의 함수를 대신하여 작성해주는 역할만 한다.\n",
        "dataclass 어노테이션을 사용하여도 dict 로 저장하지 않을 것이라 기대하며,\n",
        "1. dataclass 를 사용한 경우\n",
        "1. slots 만 사용한 경우\n",
        "1. attrs 를 사용한 경우\n",
        "1. 일반적인 방법으로 작성한 경우\n",
        "에 대하여 timeit, memit 을 통해 실행시간 및 메모리 사용양을 비교해본다."
      ],
      "metadata": {
        "id": "a_yW-bJ5-gfN"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jv812nJ8wbjc",
        "outputId": "c792e249-22e0-4d17-8f04-bde973e5782e"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: memory_profiler in /usr/local/lib/python3.7/dist-packages (0.61.0)\n",
            "Requirement already satisfied: attrs in /usr/local/lib/python3.7/dist-packages (22.1.0)\n",
            "Requirement already satisfied: psutil in /usr/local/lib/python3.7/dist-packages (from memory_profiler) (5.4.8)\n"
          ]
        }
      ],
      "source": [
        "pip install memory_profiler attrs"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%load_ext memory_profiler"
      ],
      "metadata": {
        "id": "XwvqsJTWw1Ar",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8a1d5da1-8db6-4148-e8d1-1ca32fdd16fe"
      },
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The memory_profiler extension is already loaded. To reload it, use:\n",
            "  %reload_ext memory_profiler\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from dataclasses import dataclass\n",
        "from attrs import define\n",
        "import gc"
      ],
      "metadata": {
        "id": "1o2Ig6VIBAm4"
      },
      "execution_count": 31,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# dataclass and slots"
      ],
      "metadata": {
        "id": "NvNwzvXn11TB"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "gc.collect()\n",
        "\n",
        "@dataclass\n",
        "class DataClassAndSlots:\n",
        "  __slots__=[\"a\", \"s\"]\n",
        "  a:int\n",
        "  s:str\n",
        "\n",
        "%timeit -n 5 -r 10 test = [DataClassAndSlots(i, str(i)) for i in range(100000)]\n",
        "%memit -r 10 test = [DataClassAndSlots(i, str(i)) for i in range(100000)]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "woc26udgzJSk",
        "outputId": "a2878dec-5621-4e6a-8232-c6ebd4d7b735"
      },
      "execution_count": 47,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "51.5 ms ± 2.18 ms per loop (mean ± std. dev. of 10 runs, 5 loops each)\n",
            "peak memory: 248.77 MiB, increment: 0.00 MiB\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# only dataclass"
      ],
      "metadata": {
        "id": "OvpZ_PUj3o2u"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "gc.collect()\n",
        "\n",
        "@dataclass\n",
        "class OnlyDataclass:\n",
        "  a:int\n",
        "  s:str\n",
        "\n",
        "%timeit -n 5 -r 10 test = [OnlyDataclass(i, str(i)) for i in range(100000)]\n",
        "%memit -r 10 test = [OnlyDataclass(i, str(i)) for i in range(100000)]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wYcqwWhz3rCW",
        "outputId": "3ddd2025-3395-47ed-b2b9-e48abdbd0a35"
      },
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "69.7 ms ± 4.78 ms per loop (mean ± std. dev. of 10 runs, 5 loops each)\n",
            "peak memory: 259.32 MiB, increment: 38.50 MiB\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# only slots"
      ],
      "metadata": {
        "id": "RBaOQYdz18NO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "gc.collect()\n",
        "\n",
        "class OnlySlots:\n",
        "  __slots__=[\"a\", \"s\"]\n",
        "  def __init__(self, a, s):\n",
        "    self.a = a\n",
        "    self.s = s\n",
        "\n",
        "%timeit -n 5 -r 10 test = [OnlySlots(i, str(i)) for i in range(100000)]\n",
        "%memit -r 10 test = [OnlySlots(i, str(i)) for i in range(100000)]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "r4tv9AToxx7K",
        "outputId": "1717acca-0603-4855-d355-d37e01e1aa00"
      },
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "53.8 ms ± 1.29 ms per loop (mean ± std. dev. of 10 runs, 5 loops each)\n",
            "peak memory: 222.25 MiB, increment: 9.00 MiB\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Attrs"
      ],
      "metadata": {
        "id": "4gzNC7Ib9cpA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "gc.collect()\n",
        "\n",
        "@define\n",
        "class AttrClass:\n",
        "    a: int\n",
        "    s: str\n",
        "\n",
        "%timeit -n 5 -r 10 test = [OnlySlots(i, str(i)) for i in range(100000)]\n",
        "%memit -r 10 test = [OnlySlots(i, str(i)) for i in range(100000)]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "K2pXn4719fJV",
        "outputId": "88b0c1fa-c855-48dc-ef31-12c94ce2dac1"
      },
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "50.3 ms ± 1.41 ms per loop (mean ± std. dev. of 10 runs, 5 loops each)\n",
            "peak memory: 238.35 MiB, increment: 16.35 MiB\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# normal"
      ],
      "metadata": {
        "id": "ZWlpW_Kt1-ph"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "gc.collect()\n",
        "\n",
        "class Normal:\n",
        "  def __init__(self, a, s):\n",
        "    self.a = a\n",
        "    self.s = s\n",
        "\n",
        "%timeit -n 5 -r 10 test = [Normal(i, str(i)) for i in range(100000)]\n",
        "%memit -r 10 test = [Normal(i, str(i)) for i in range(100000)]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0XtQwVbtyCaP",
        "outputId": "1633070b-aebb-4484-9cbf-66f85cc2bfa1"
      },
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "72 ms ± 4.77 ms per loop (mean ± std. dev. of 10 runs, 5 loops each)\n",
            "peak memory: 275.61 MiB, increment: 42.09 MiB\n"
          ]
        }
      ]
    }
  ]
}