Skip to content

Instantly share code, notes, and snippets.

@AseiSugiyama

AseiSugiyama/kubeflowpipelinessdkv2artifact.ipynb

Created June 29, 2021 12:40
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save AseiSugiyama/28d4245f4a7f7d0601af54cf3354aaab to your computer and use it in GitHub Desktop.
Save AseiSugiyama/28d4245f4a7f7d0601af54cf3354aaab to your computer and use it in GitHub Desktop.
Download ZIP
KubeflowPipelinesSDKV2Artifact.ipynb
Display the source blob
Display the rendered blob
Raw
kubeflowpipelinessdkv2artifact.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "KubeflowPipelinesSDKV2Artifact.ipynb",
"provenance": [],
"authorship_tag": "ABX9TyPre2Ti7M6a2FV7hka0z0KE",
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/AseiSugiyama/28d4245f4a7f7d0601af54cf3354aaab/kubeflowpipelinessdkv2artifact.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "GaEsYSMzNm1i"
},
"source": [
"## Setup"
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"id": "_4aLln5ALXkU",
"outputId": "dd3cd4d1-5293-4a94-a441-2755315c0714"
},
"source": [
"!pip install kfp"
],
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"text": [
"Collecting kfp\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/d6/21/a895fe3958d052e77afdbbc56e21e831a7e52ea39074c6784bc265907c0c/kfp-1.6.4.tar.gz (225kB)\n",
"\u001b[K |████████████████████████████████| 235kB 7.6MB/s \n",
"\u001b[?25hCollecting absl-py<=0.11,>=0.9\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/bc/58/0aa6fb779dc69cfc811df3398fcbeaeefbf18561b6e36b185df0782781cc/absl_py-0.11.0-py3-none-any.whl (127kB)\n",
"\u001b[K |████████████████████████████████| 133kB 28.1MB/s \n",
"\u001b[?25hCollecting PyYAML<6,>=5.3\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/7a/a5/393c087efdc78091afa2af9f1378762f9821c9c1d7a22c5753fb5ac5f97a/PyYAML-5.4.1-cp37-cp37m-manylinux1_x86_64.whl (636kB)\n",
"\u001b[K |████████████████████████████████| 645kB 21.6MB/s \n",
"\u001b[?25hCollecting google-cloud-storage<2,>=1.20.0\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/92/b7/10f7086520bc7f4e80e10dce9c41f5317e7c6a7e1234bf2e1fe7c7193130/google_cloud_storage-1.39.0-py2.py3-none-any.whl (103kB)\n",
"\u001b[K |████████████████████████████████| 112kB 35.3MB/s \n",
"\u001b[?25hCollecting kubernetes<13,>=8.0.0\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/50/55/84aaadb5f8b4d2641f28eb7373feb610f4b609cc9cf223bcd494fd66a647/kubernetes-12.0.1-py2.py3-none-any.whl (1.7MB)\n",
"\u001b[K |████████████████████████████████| 1.8MB 30.1MB/s \n",
"\u001b[?25hRequirement already satisfied: google-api-python-client<2,>=1.7.8 in /usr/local/lib/python3.7/dist-packages (from kfp) (1.12.8)\n",
"Requirement already satisfied: google-auth<2,>=1.6.1 in /usr/local/lib/python3.7/dist-packages (from kfp) (1.31.0)\n",
"Collecting requests-toolbelt<1,>=0.8.0\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/60/ef/7681134338fc097acef8d9b2f8abe0458e4d87559c689a8c306d0957ece5/requests_toolbelt-0.9.1-py2.py3-none-any.whl (54kB)\n",
"\u001b[K |████████████████████████████████| 61kB 9.3MB/s \n",
"\u001b[?25hRequirement already satisfied: cloudpickle<2,>=1.3.0 in /usr/local/lib/python3.7/dist-packages (from kfp) (1.3.0)\n",
"Collecting kfp-server-api<2.0.0,>=1.1.2\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/34/15/72c49038a9ab4b8d22b961b4e02dd300d5f36cb3933891270725b95a834c/kfp-server-api-1.6.0.tar.gz (52kB)\n",
"\u001b[K |████████████████████████████████| 61kB 8.7MB/s \n",
"\u001b[?25hCollecting jsonschema<4,>=3.0.1\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/c5/8f/51e89ce52a085483359217bc72cdbf6e75ee595d5b1d4b5ade40c7e018b8/jsonschema-3.2.0-py2.py3-none-any.whl (56kB)\n",
"\u001b[K |████████████████████████████████| 61kB 8.6MB/s \n",
"\u001b[?25hRequirement already satisfied: tabulate<1,>=0.8.6 in /usr/local/lib/python3.7/dist-packages (from kfp) (0.8.9)\n",
"Requirement already satisfied: click<8,>=7.1.1 in /usr/local/lib/python3.7/dist-packages (from kfp) (7.1.2)\n",
"Collecting Deprecated<2,>=1.2.7\n",
" Downloading https://files.pythonhosted.org/packages/fb/73/994edfcba74443146c84b91921fcc269374354118d4f452fb0c54c1cbb12/Deprecated-1.2.12-py2.py3-none-any.whl\n",
"Collecting strip-hints<1,>=0.1.8\n",
" Downloading https://files.pythonhosted.org/packages/72/a5/1b8e2990d3f2b24a83d682607a0de5a745ad24de1b2bca33b1561c303a9c/strip-hints-0.1.9.tar.gz\n",
"Collecting docstring-parser<1,>=0.7.3\n",
" Downloading https://files.pythonhosted.org/packages/b0/d9/fff7bdbc6bcd949dbf8bd64967a07d79f2767ae147fc5c1f0b2c8a063a77/docstring_parser-0.8.1.tar.gz\n",
" Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
" Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
" Preparing wheel metadata ... \u001b[?25l\u001b[?25hdone\n",
"Collecting kfp-pipeline-spec<0.2.0,>=0.1.8\n",
" Downloading https://files.pythonhosted.org/packages/09/c4/0c750431805f1282c733ac3fd5758d36bfaba75c113eb392fbda91a4cc7a/kfp_pipeline_spec-0.1.8-py3-none-any.whl\n",
"Collecting fire<1,>=0.3.1\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/11/07/a119a1aa04d37bc819940d95ed7e135a7dcca1c098123a3764a6dcace9e7/fire-0.4.0.tar.gz (87kB)\n",
"\u001b[K |████████████████████████████████| 92kB 12.9MB/s \n",
"\u001b[?25hCollecting protobuf<4,>=3.13.0\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/4c/53/ddcef00219f2a3c863b24288e24a20c3070bd086a1e77706f22994a7f6db/protobuf-3.17.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.0MB)\n",
"\u001b[K |████████████████████████████████| 1.0MB 45.9MB/s \n",
"\u001b[?25hRequirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from absl-py<=0.11,>=0.9->kfp) (1.15.0)\n",
"Collecting google-cloud-core<2.0dev,>=1.4.1\n",
" Downloading https://files.pythonhosted.org/packages/88/a7/b74266a6fd888d91a6f517c574c17425731181fe44e2df1e414d4b77fbe8/google_cloud_core-1.7.1-py2.py3-none-any.whl\n",
"Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /usr/local/lib/python3.7/dist-packages (from google-cloud-storage<2,>=1.20.0->kfp) (2.23.0)\n",
"Collecting google-resumable-media<2.0dev,>=1.3.0\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/11/96/4360dc70bef5559b3faf3deeda97aae7d10ff7660d41fd233eb792e7d09f/google_resumable_media-1.3.1-py2.py3-none-any.whl (75kB)\n",
"\u001b[K |████████████████████████████████| 81kB 7.9MB/s \n",
"\u001b[?25hRequirement already satisfied: certifi>=14.05.14 in /usr/local/lib/python3.7/dist-packages (from kubernetes<13,>=8.0.0->kfp) (2021.5.30)\n",
"Requirement already satisfied: python-dateutil>=2.5.3 in /usr/local/lib/python3.7/dist-packages (from kubernetes<13,>=8.0.0->kfp) (2.8.1)\n",
"Requirement already satisfied: setuptools>=21.0.0 in /usr/local/lib/python3.7/dist-packages (from kubernetes<13,>=8.0.0->kfp) (57.0.0)\n",
"Requirement already satisfied: requests-oauthlib in /usr/local/lib/python3.7/dist-packages (from kubernetes<13,>=8.0.0->kfp) (1.3.0)\n",
"Requirement already satisfied: urllib3>=1.24.2 in /usr/local/lib/python3.7/dist-packages (from kubernetes<13,>=8.0.0->kfp) (1.24.3)\n",
"Collecting websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/ca/5f/3c211d168b2e9f9342cfb53bcfc26aab0eac63b998015e7af7bcae66119d/websocket_client-1.1.0-py2.py3-none-any.whl (68kB)\n",
"\u001b[K |████████████████████████████████| 71kB 7.9MB/s \n",
"\u001b[?25hRequirement already satisfied: httplib2<1dev,>=0.15.0 in /usr/local/lib/python3.7/dist-packages (from google-api-python-client<2,>=1.7.8->kfp) (0.17.4)\n",
"Requirement already satisfied: uritemplate<4dev,>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from google-api-python-client<2,>=1.7.8->kfp) (3.0.1)\n",
"Requirement already satisfied: google-api-core<2dev,>=1.21.0 in /usr/local/lib/python3.7/dist-packages (from google-api-python-client<2,>=1.7.8->kfp) (1.26.3)\n",
"Requirement already satisfied: google-auth-httplib2>=0.0.3 in /usr/local/lib/python3.7/dist-packages (from google-api-python-client<2,>=1.7.8->kfp) (0.0.4)\n",
"Requirement already satisfied: rsa<5,>=3.1.4; python_version >= \"3.6\" in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.1->kfp) (4.7.2)\n",
"Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.1->kfp) (0.2.8)\n",
"Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.1->kfp) (4.2.2)\n",
"Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema<4,>=3.0.1->kfp) (21.2.0)\n",
"Requirement already satisfied: pyrsistent>=0.14.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema<4,>=3.0.1->kfp) (0.17.3)\n",
"Requirement already satisfied: importlib-metadata; python_version < \"3.8\" in /usr/local/lib/python3.7/dist-packages (from jsonschema<4,>=3.0.1->kfp) (4.5.0)\n",
"Requirement already satisfied: wrapt<2,>=1.10 in /usr/local/lib/python3.7/dist-packages (from Deprecated<2,>=1.2.7->kfp) (1.12.1)\n",
"Requirement already satisfied: wheel in /usr/local/lib/python3.7/dist-packages (from strip-hints<1,>=0.1.8->kfp) (0.36.2)\n",
"Requirement already satisfied: termcolor in /usr/local/lib/python3.7/dist-packages (from fire<1,>=0.3.1->kfp) (1.1.0)\n",
"Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-storage<2,>=1.20.0->kfp) (3.0.4)\n",
"Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-storage<2,>=1.20.0->kfp) (2.10)\n",
"Collecting google-crc32c<2.0dev,>=1.0; python_version >= \"3.5\"\n",
" Downloading https://files.pythonhosted.org/packages/fc/ae/b6efa1019e18c6c791f0f5cd93b2ff40f8f06696dbf04db39ec0f5591b1e/google_crc32c-1.1.2-cp37-cp37m-manylinux2014_x86_64.whl\n",
"Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib->kubernetes<13,>=8.0.0->kfp) (3.1.1)\n",
"Requirement already satisfied: pytz in /usr/local/lib/python3.7/dist-packages (from google-api-core<2dev,>=1.21.0->google-api-python-client<2,>=1.7.8->kfp) (2018.9)\n",
"Requirement already satisfied: packaging>=14.3 in /usr/local/lib/python3.7/dist-packages (from google-api-core<2dev,>=1.21.0->google-api-python-client<2,>=1.7.8->kfp) (20.9)\n",
"Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from google-api-core<2dev,>=1.21.0->google-api-python-client<2,>=1.7.8->kfp) (1.53.0)\n",
"Requirement already satisfied: pyasn1>=0.1.3 in /usr/local/lib/python3.7/dist-packages (from rsa<5,>=3.1.4; python_version >= \"3.6\"->google-auth<2,>=1.6.1->kfp) (0.4.8)\n",
"Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata; python_version < \"3.8\"->jsonschema<4,>=3.0.1->kfp) (3.4.1)\n",
"Requirement already satisfied: typing-extensions>=3.6.4; python_version < \"3.8\" in /usr/local/lib/python3.7/dist-packages (from importlib-metadata; python_version < \"3.8\"->jsonschema<4,>=3.0.1->kfp) (3.7.4.3)\n",
"Requirement already satisfied: cffi>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from google-crc32c<2.0dev,>=1.0; python_version >= \"3.5\"->google-resumable-media<2.0dev,>=1.3.0->google-cloud-storage<2,>=1.20.0->kfp) (1.14.5)\n",
"Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=14.3->google-api-core<2dev,>=1.21.0->google-api-python-client<2,>=1.7.8->kfp) (2.4.7)\n",
"Requirement already satisfied: pycparser in /usr/local/lib/python3.7/dist-packages (from cffi>=1.0.0->google-crc32c<2.0dev,>=1.0; python_version >= \"3.5\"->google-resumable-media<2.0dev,>=1.3.0->google-cloud-storage<2,>=1.20.0->kfp) (2.20)\n",
"Building wheels for collected packages: docstring-parser\n",
" Building wheel for docstring-parser (PEP 517) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for docstring-parser: filename=docstring_parser-0.8.1-cp37-none-any.whl size=19678 sha256=e4cf748094b910778d02a7eef10992a0c29db78be3c420a5ebae2de16f52d9fa\n",
" Stored in directory: /root/.cache/pip/wheels/b0/6a/da/4827c7f057a80d9ea35ca5002c0357a46aa86e3c31a7a1357f\n",
"Successfully built docstring-parser\n",
"Building wheels for collected packages: kfp, kfp-server-api, strip-hints, fire\n",
" Building wheel for kfp (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for kfp: filename=kfp-1.6.4-cp37-none-any.whl size=307978 sha256=e98156d072519d568dcb59806de8b43d70b292d98f9a19d1a1323e90bfa7849d\n",
" Stored in directory: /root/.cache/pip/wheels/01/59/8c/768960df398127c54155e79d7cdfdf4e2bb6c9ec1f81f94402\n",
" Building wheel for kfp-server-api (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for kfp-server-api: filename=kfp_server_api-1.6.0-cp37-none-any.whl size=92524 sha256=23d4f6d3852bfa694f7ee5cf6aa252a4ce025a3d016897db39478681d1b12177\n",
" Stored in directory: /root/.cache/pip/wheels/b1/bd/39/ca712322b001ac20bfac7dbc85d85b8ebba5d22201fe4674d0\n",
" Building wheel for strip-hints (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for strip-hints: filename=strip_hints-0.1.9-py2.py3-none-any.whl size=21017 sha256=e3999d7cde732ef48febd4c0b72eacd16575b8b8470f60e24abe766b7e8cdd6c\n",
" Stored in directory: /root/.cache/pip/wheels/1a/6c/b0/2e38ee5cf60250da7f47962f8ab888672d0956774f79ee624f\n",
" Building wheel for fire (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for fire: filename=fire-0.4.0-py2.py3-none-any.whl size=115943 sha256=dc36a829f5024436c74cac3bf0fded8fedc588e1f3b6c80f98899e5fdd6d789c\n",
" Stored in directory: /root/.cache/pip/wheels/af/19/30/1ea0cad502dcb4e66ed5a690279628c827aea38bbbab75d5ed\n",
"Successfully built kfp kfp-server-api strip-hints fire\n",
"\u001b[31mERROR: nbclient 0.5.3 has requirement jupyter-client>=6.1.5, but you'll have jupyter-client 5.3.5 which is incompatible.\u001b[0m\n",
"\u001b[31mERROR: google-cloud-bigquery 1.21.0 has requirement google-resumable-media!=0.4.0,<0.5.0dev,>=0.3.1, but you'll have google-resumable-media 1.3.1 which is incompatible.\u001b[0m\n",
"Installing collected packages: absl-py, PyYAML, google-cloud-core, google-crc32c, google-resumable-media, google-cloud-storage, websocket-client, kubernetes, requests-toolbelt, kfp-server-api, jsonschema, Deprecated, strip-hints, docstring-parser, protobuf, kfp-pipeline-spec, fire, kfp\n",
" Found existing installation: absl-py 0.12.0\n",
" Uninstalling absl-py-0.12.0:\n",
" Successfully uninstalled absl-py-0.12.0\n",
" Found existing installation: PyYAML 3.13\n",
" Uninstalling PyYAML-3.13:\n",
" Successfully uninstalled PyYAML-3.13\n",
" Found existing installation: google-cloud-core 1.0.3\n",
" Uninstalling google-cloud-core-1.0.3:\n",
" Successfully uninstalled google-cloud-core-1.0.3\n",
" Found existing installation: google-resumable-media 0.4.1\n",
" Uninstalling google-resumable-media-0.4.1:\n",
" Successfully uninstalled google-resumable-media-0.4.1\n",
" Found existing installation: google-cloud-storage 1.18.1\n",
" Uninstalling google-cloud-storage-1.18.1:\n",
" Successfully uninstalled google-cloud-storage-1.18.1\n",
" Found existing installation: jsonschema 2.6.0\n",
" Uninstalling jsonschema-2.6.0:\n",
" Successfully uninstalled jsonschema-2.6.0\n",
" Found existing installation: protobuf 3.12.4\n",
" Uninstalling protobuf-3.12.4:\n",
" Successfully uninstalled protobuf-3.12.4\n",
"Successfully installed Deprecated-1.2.12 PyYAML-5.4.1 absl-py-0.11.0 docstring-parser-0.8.1 fire-0.4.0 google-cloud-core-1.7.1 google-cloud-storage-1.39.0 google-crc32c-1.1.2 google-resumable-media-1.3.1 jsonschema-3.2.0 kfp-1.6.4 kfp-pipeline-spec-0.1.8 kfp-server-api-1.6.0 kubernetes-12.0.1 protobuf-3.17.3 requests-toolbelt-0.9.1 strip-hints-0.1.9 websocket-client-1.1.0\n"
],
"name": "stdout"
},
{
"output_type": "display_data",
"data": {
"application/vnd.colab-display-data+json": {
"pip_warning": {
"packages": [
"google"
]
}
}
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "VFv7InU5Np7q"
},
"source": [
"from google.colab import auth\n",
"auth.authenticate_user()"
],
"execution_count": 1,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "j5MoNNly3Kyh"
},
"source": [
"## Analysis of `io_types` and `type_utils`\n",
"\n",
"### `io_types.Artifact`, `io_types.Model`,`io_types.Dataset`\n",
"\n",
"`io_types` contains definitions of class of artifacts. We can build Python artifact object using `io_types`. Python object of `Artifact`, `Model`, `Dataset` have same schema defined at [pipelines/sdk/python/kfp/dsl/type_schemas](https://github.com/kubeflow/pipelines/tree/master/sdk/python/kfp/dsl/type_schemas)."
]
},
{
"cell_type": "code",
"metadata": {
"id": "Fo2L0HXYeGxg"
},
"source": [
"from kfp.dsl.io_types import Artifact, Model, Dataset, Metrics, ClassificationMetrics, SlicedClassificationMetrics\n",
"artifact = Artifact(name=\"sample_artifact\", uri=\"/tmp/sample_artifact.json\", metadata=None)\n",
"model = Model(\n",
" name=\"sample_model\",\n",
" uri=\"/tmp/sample_model.pb\",\n",
" metadata={\n",
" \"framework\":\"TensorFlow\",\n",
" \"framework_version\":\"2.5.0\",\n",
" }\n",
")\n",
"dataset = Dataset(\n",
" name=\"sample_dataset\",\n",
" uri=\"/tmp/sample_dataset.csv\",\n",
" metadata={\n",
" \"payload_format\": \"\",\n",
" \"container_format\": \"\",\n",
" } \n",
")"
],
"execution_count": 60,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"id": "-jQ7J7N0eUVa",
"outputId": "671c5a57-b358-46d3-d1df-c5b2d2f75f7a"
},
"source": [
"json.dumps(vars(artifact))"
],
"execution_count": 53,
"outputs": [
{
"output_type": "execute_result",
"data": {
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
},
"text/plain": [
"'{\"uri\": \"/tmp/sample_artifact.json\", \"name\": \"sample_artifact\", \"metadata\": {}}'"
]
},
"metadata": {
"tags": []
},
"execution_count": 53
}
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"id": "71GPuZzaKNop",
"outputId": "2d8eca83-f0d0-4bcc-bec1-170836c797c8"
},
"source": [
"json.dumps(vars(model))"
],
"execution_count": 54,
"outputs": [
{
"output_type": "execute_result",
"data": {
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
},
"text/plain": [
"'{\"uri\": \"/tmp/sample_model.pb\", \"name\": \"sample_model\", \"metadata\": {\"framework\": \"TensorFlow\", \"framework_version\": \"2.5.0\"}}'"
]
},
"metadata": {
"tags": []
},
"execution_count": 54
}
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"id": "N32aVh4IKS9K",
"outputId": "d9720c7d-7b5c-47e2-86d4-53285c7640b2"
},
"source": [
"json.dumps(vars(dataset))"
],
"execution_count": 55,
"outputs": [
{
"output_type": "execute_result",
"data": {
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
},
"text/plain": [
"'{\"uri\": \"/tmp/sample_dataset.csv\", \"name\": \"sample_dataset\", \"metadata\": {\"payload_format\": \"\", \"container_format\": \"\"}}'"
]
},
"metadata": {
"tags": []
},
"execution_count": 55
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "LWgVmRJ8LWfy"
},
"source": [
"### `io_types.Metrics`\n",
"\n",
"Unlike other artifacts (`Artifact`, `Dataset`, `Model`), `io_types.Metrics` dose not follow the schema defined in [pipelines/metrics.yaml](https://github.com/kubeflow/pipelines/blob/master/sdk/python/kfp/dsl/type_schemas/metrics.yaml). YAML file describes a entity of `metadata`. To add a metric entity to `metadata`, we can use `log_metric` method."
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"id": "NoqhhKxDKX-U",
"outputId": "6aea55c3-e6eb-4970-fbaf-d7df306b00ad"
},
"source": [
"metrics = Metrics(\n",
" name=\"simple_metric\",\n",
" uri=\"/tmp/simple_metric.json\",\n",
" metadata={}\n",
")\n",
"metrics.log_metric(\n",
" metric=\"accuracy\",\n",
" value=0.8\n",
")\n",
"\n",
"json.dumps(vars(metrics))"
],
"execution_count": 59,
"outputs": [
{
"output_type": "execute_result",
"data": {
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
},
"text/plain": [
"'{\"uri\": \"/tmp/sample_dataset.csv\", \"name\": \"sample_dataset\", \"metadata\": {\"accuracy\": 0.8}}'"
]
},
"metadata": {
"tags": []
},
"execution_count": 59
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "RHVKYMySNKqY"
},
"source": [
"### `io_types.ClassificationMetrics`\n",
"\n",
"`io_types.ClassificationMetrics` is an `Artifact` but has a complexed `metadata`. `metadata` of `io_types.ClassificationMetrics` is able to following keys:\n",
"\n",
"- `confidenceMetrics` : Logs of data points in the ROC curve\n",
"- `'confusionMatrix'` : Dict that represents confusion matrics\n",
" - `annotationSpecs` : names of the categories\n",
" - `rows` : List of floats for each low of confusion matrics.\n",
"\n",
"[confidence_metrics.yaml](https://github.com/kubeflow/pipelines/blob/master/sdk/python/kfp/dsl/type_schemas/confidence_metrics.yaml) describes the schema of `confidenceMetrics` and [confusion_matrix.yaml](https://github.com/kubeflow/pipelines/blob/master/sdk/python/kfp/dsl/type_schemas/confusion_matrix.yaml) describes the schema of `confusionMatrix`. \n",
"\n",
"To set the `confidenceMatrics`, we can use [log_roc_curve](https://github.com/kubeflow/pipelines/blob/a1d513eb412f3ffd44edf82af2fa7edb05c3b952/sdk/python/kfp/dsl/io_types.py#L162). To set the `confusionMatrix`, we can use [log_confusion_matrix](https://github.com/kubeflow/pipelines/blob/a1d513eb412f3ffd44edf82af2fa7edb05c3b952/sdk/python/kfp/dsl/io_types.py#L252).\n",
"\n",
"\n",
"To check the schema of `ClassificationMetrics.metadata`, visit [classification_metrics.yaml](https://github.com/kubeflow/pipelines/blob/master/sdk/python/kfp/dsl/type_schemas/classification_metrics.yaml).\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "YT3Lr37XaP4z"
},
"source": [
"classification_metrics = ClassificationMetrics(\n",
" name=\"simple_metric\",\n",
" uri=\"/tmp/simple_metric.json\", \n",
")"
],
"execution_count": 85,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "H0ohh8MNjY6-"
},
"source": [
"from sklearn.metrics import confusion_matrix, roc_curve\n",
"from sklearn.datasets import load_iris\n",
"from sklearn.model_selection import train_test_split\n",
"from sklearn.ensemble import RandomForestClassifier\n",
"\n",
"iris = load_iris()\n",
"X = iris.data\n",
"y = iris.target\n",
"class_names = iris.target_names\n",
"\n",
"X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=24)\n",
"\n",
"model = RandomForestClassifier()\n",
"model.fit(X_train, y_train)\n",
"y_pred = model.predict(X_test)\n",
"\n",
"matrix = confusion_matrix(y_test, y_pred)\n",
"fpr, tpr, thresholds = roc_curve(y_test, y_pred, pos_label=0)"
],
"execution_count": 86,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "BB_pO95-kXrW"
},
"source": [
"# Note: To serialize JSON format, it is required to convert from numpy array into python array.\n",
"classification_metrics.log_confusion_matrix(\n",
" categories=class_names.tolist(),\n",
" matrix=matrix.tolist()\n",
")\n",
"classification_metrics.log_roc_curve(\n",
" fpr=fpr.tolist(), tpr=tpr.tolist(), threshold=thresholds.tolist()\n",
")"
],
"execution_count": 87,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 86
},
"id": "1tBgtt-doBAK",
"outputId": "22197840-6ae4-4940-fbb4-254827e23b11"
},
"source": [
"json.dumps(vars(classification_metrics))"
],
"execution_count": 88,
"outputs": [
{
"output_type": "execute_result",
"data": {
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
},
"text/plain": [
"'{\"uri\": \"/tmp/simple_metric.json\", \"name\": \"simple_metric\", \"metadata\": {\"confusionMatrix\": {\"annotationSpecs\": [{\"displayName\": \"setosa\"}, {\"displayName\": \"versicolor\"}, {\"displayName\": \"virginica\"}], \"rows\": [{\"row\": [12, 0, 0]}, {\"row\": [0, 7, 1]}, {\"row\": [0, 1, 17]}]}, \"confidenceMetrics\": [{\"confidenceThreshold\": 3, \"recall\": 0.0, \"falsePositiveRate\": 0.0}, {\"confidenceThreshold\": 2, \"recall\": 0.0, \"falsePositiveRate\": 0.6923076923076923}, {\"confidenceThreshold\": 1, \"recall\": 0.0, \"falsePositiveRate\": 1.0}, {\"confidenceThreshold\": 0, \"recall\": 1.0, \"falsePositiveRate\": 1.0}]}, \"_categories\": [\"setosa\", \"versicolor\", \"virginica\"], \"_matrix\": [{\"row\": [12, 0, 0]}, {\"row\": [0, 7, 1]}, {\"row\": [0, 1, 17]}], \"_confusion_matrix\": {\"annotationSpecs\": [{\"displayName\": \"setosa\"}, {\"displayName\": \"versicolor\"}, {\"displayName\": \"virginica\"}], \"rows\": [{\"row\": [12, 0, 0]}, {\"row\": [0, 7, 1]}, {\"row\": [0, 1, 17]}]}}'"
]
},
"metadata": {
"tags": []
},
"execution_count": 88
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "CsrX65vPoASX"
},
"source": [
"### `io_types.ClassificationMetrics`\n",
"\n",
"`io_types.ClassificationMetrics` is another complexed version of `Artifacts`. It represents sliced `ClassificationMetrics` by the slice label.\n",
"\n",
"Even though there is class definition of `io_types.ClassificationMetrics`, currently, we have no documents, test, and samples."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "yK1er00LxQRX"
},
"source": [
"### `type_utils`\n",
"\n",
"`type_utils` are utilitiy functions to get specs protobuf from python object. These utility functions are used to build specs alike `ComponentSpec` internally. Generally, KFP SDK user might not use this util functions."
]
},
{
"cell_type": "code",
"metadata": {
"id": "jRjl3iUjeibQ"
},
"source": [
"from kfp.dsl.type_utils import get_artifact_type_schema\n",
"from typing import Type\n",
"def get_artifact_schema_from_class(artifact: Type[Artifact]):\n",
" artifact_type_schema = get_artifact_type_schema(artifact)\n",
" return artifact_type_schema.schema_title"
],
"execution_count": 40,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"id": "b7c6d-qfewzR",
"outputId": "b12930a9-964c-4b4a-a554-607cc301571d"
},
"source": [
"model = Model()\n",
"get_artifact_schema_from_class(model.__class__)"
],
"execution_count": 41,
"outputs": [
{
"output_type": "execute_result",
"data": {
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
},
"text/plain": [
"'system.Model'"
]
},
"metadata": {
"tags": []
},
"execution_count": 41
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1de4i49iNiG5"
},
"source": [
"## Build simple input/output compont\n",
"\n",
"### ComponentSpec\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "0bbEBwLcL25u"
},
"source": [
"!writefile component.yaml\n",
"name: sample_io_component\n",
"outputs:\n",
"- {name: train_data_path, type: dataset}\n",
"- {name: eval_data_path, type: dataset}\n",
"implementation:\n",
" container:\n",
" image: ${tagged_name}\n",
" args:\n",
" - {outputUri: train_data_path}\n",
" - {outputUri: eval_data_path}"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "TAwyi5OZPNZX",
"outputId": "4417a669-2ac0-44ff-97b9-00e106679693"
},
"source": [
"from kfp.v2 import dsl, compiler"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Writing confusion_matrix.py\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "K61dLWNlRgjo",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "1745bc0c-cb09-48d0-8832-be5a09661c10"
},
"source": [
"!gsutil cat gs://bradley-playground/sfpd/predictions/part-0"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"BucketNotFoundException: 404 gs://bradley-playground bucket does not exist.\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "TiPyMTb5Pamq"
},
"source": [
"%%writefile Dockerfile\n",
"FROM gcr.io/deeplearning-platform-release/tf2-cpu.2-5\n",
"\n",
"WORKDIR /home\n",
"COPY . .\n",
"ENTRYPOINT [\"python\", \"confusion_matrix.py\"]"
],
"execution_count": null,
"outputs": []
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment