pip_modules.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5ba9b165-8669-4b35-915a-1d857892c3f7",
   "metadata": {},
   "source": [
    "# Distributing closed source Co-Simulations\n",
    "\n",
    "When an IP core is ready to ship, it is typically compiled into a python installable wheel package. This allows to deliver a closed source binary for evaluation to a development partner or end customer.\n",
    "\n",
    "This notebook explains how to install a compiled IP core via a given URL and run tests against ist.\n",
    "\n",
    "## Installing a IP core module via pip\n",
    "\n",
    "Run pip to install a wheel from a specific URL:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "acd4f65a-aab7-483d-b134-f5912eae2fc7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Defaulting to user installation because normal site-packages is not writeable\n",
      "Collecting masoc==0.0\n",
      "  Using cached https://section5.ch/cyrite/masoc-0.0-cp310-cp310-linux_x86_64.whl (89 kB)\n",
      "Requirement already satisfied: Cython in /home/cyrite/.local/lib/python3.10/site-packages (from masoc==0.0) (0.29.36)\n",
      "\u001b[33mWARNING: You are using pip version 21.2.4; however, version 24.2 is available.\n",
      "You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip install https://section5.ch/cyrite/masoc-0.0-cp310-cp310-linux_x86_64.whl"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f1d9dd6-383f-4f13-9a53-63fcd502bbd9",
   "metadata": {},
   "source": [
    "We might have dependencies from our own, private library, which we delivered to a development partner as closed source as well. We include it in the search path to make sure we use our local source version:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "8d32c700-5608-4b00-b6af-18068d6f88e7",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.insert(0, \"../../\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45769bea-b0c5-41c7-911b-0aba9ff73181",
   "metadata": {},
   "source": [
    "## Running the co-simulation\n",
    "\n",
    "The above pip install may have installed several modules from the 'masoc' package. In this example, we have a top level package `fifobuffer` which we can import as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "3e9d820c-9151-46f9-8f82-7b82df12412f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from fifobuffer import test_fifobuffer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d62fcc3-45a1-4dd9-9283-89877d117b69",
   "metadata": {},
   "source": [
    "We run the default test by calling `test_fifo()` from test_fifobuffer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "6b9f64fb-45a3-42b1-b02d-df3308ef224b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[7;35m Declare obj 'fifo_wrapper' in context '(_TestFifo 'PktFifo')'(<class 'fifobuffer.test_fifobuffer._TestFifo'>) \u001b[0m\n",
      "\u001b[7;35m Declare obj 'dv_rising' in context '(_TestFifo 'PktFifo')'(<class 'fifobuffer.test_fifobuffer._TestFifo'>) \u001b[0m\n",
      "USING READ DELAY\n",
      "\u001b[7;35m Declare obj 'pktfifo' in context '(_TestFifo 'PktFifo')'(<class 'fifobuffer.test_fifobuffer._TestFifo'>) \u001b[0m\n",
      " ADDR_W: use default 6 \n",
      " SYN_RAMTYPE: use default block_ram \n",
      "DEBUG LIB ALL ELEM (_TestFifo 'PktFifo')\n",
      "DEBUG MAIN ELAB [Instance fifo_wrapper I/F: [// ID: fifo_wrapper_0 ]]\n",
      " DEBUG Elaborating component  (_TestFifo 'PktFifo') lfsr8u_1u_1_23u_1u_8 \n",
      "\u001b[32m Adding module with name `lfsr8` \u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[7;31mDEBUG: not handling type <class 'myirl.kernel._types.SigType'> in co-simulation. Your Cosimulation may not run correctly.\u001b[0m\n",
      "\u001b[7;31mDEBUG: not handling type <class 'myirl.kernel._types.SigType'> in co-simulation. Your Cosimulation may not run correctly.\u001b[0m\n",
      "\u001b[7;31mDEBUG: not handling type <class 'myirl.emulation.myhdl_intbv.intbv'> in co-simulation. Your Cosimulation may not run correctly.\u001b[0m\n",
      "/home/pyosys/src/myhdl2/myhdl.v2we/howto/../../myirl/kernel/components.py:203: UserWarning: @component `pktfifo` interface :Output flushing not driven\n",
      "  base.warnings.warn(self._err(\"Output %s not driven\" % \\\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " DEBUG Elaborating component  (_TestFifo 'PktFifo') pktfifo_obj__TestFifou_1_FifoPortIn_obj_module_8_in_FifoPortOut_obj_module_8_inu_1u_1u_1_5 \n",
      "\u001b[32m Adding module with name `pktfifo` \u001b[0m\n",
      " DEBUG Elaborating component  (_TestFifo 'PktFifo') dv_rising_obj__TestFifou_1u_1u_1 \n",
      "\u001b[32m Adding module with name `dv_rising` \u001b[0m\n",
      "\u001b[32m Adding module with name `fifo_wrapper` \u001b[0m\n",
      "\u001b[7;34m FINALIZE implementation `fifo_wrapper` of `fifo_wrapper` \u001b[0m\n",
      "\u001b[7;34m FINALIZE implementation `fifo_wrapper` of `fifo_wrapper` \u001b[0m\n",
      "\n",
      "-- Running command `tee -q hierarchy -top \\fifo_wrapper' --\n",
      "\n",
      "-- Running command `ls; check' --\n",
      "\n",
      "2. Executing CHECK pass (checking for obvious problems).\n",
      "Found and reported 0 problems.\n",
      "\n",
      "-- Running command `hierarchy -check' --\n",
      "\n",
      "3. Executing HIERARCHY pass (managing design hierarchy).\n",
      "\n",
      "-- Running command `write_verilog debug.v' --\n",
      "\n",
      "4. Executing Verilog backend.\n",
      "\n",
      "4.1. Executing BMUXMAP pass.\n",
      "\n",
      "4.2. Executing DEMUXMAP pass.\n",
      "Open for writing: fifo1.vcd\n",
      "\u001b[7;35m CXXRTL context: SKIP INTERFACE ITEM `self` \u001b[0m\n",
      "\u001b[7;35m CXXRTL context: SKIP INTERFACE ITEM `TX` \u001b[0m\n",
      "DEBUG: RUNNING RX SEQUENCE\n",
      "RESET DONE\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[7;34mAttemping cold import of\u001b[0m .runtime.fifo_wrapper fifobuffer\n",
      "\u001b[32mCosimulation: co_count not connected to backend\u001b[0m\n",
      "\u001b[32mCosimulation: co_count_r not connected to backend\u001b[0m\n",
      "\u001b[32mCosimulation: co_fillcount not connected to backend\u001b[0m\n",
      "\u001b[32mCosimulation: co_ready_d not connected to backend\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RUN FLUSH START\n"
     ]
    }
   ],
   "source": [
    "test_fifobuffer.test_fifo()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b94a671-96de-4099-8b21-f2621b28f062",
   "metadata": {},
   "source": [
    "The simulation has created a `.vcd` file. We can download it and examine it in **gtkwave**, for example."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d34e88be-a4fb-4173-b5b1-0683505a8c8d",
   "metadata": {},
   "source": [
    "## Creating own tests\n",
    "\n",
    "When creating own tests for a blackbox unit under test (UUT), it is best to derive from the `TestFifo` factory class and insert your own test bench.\n",
    "\n",
    "However, there's an important detail to note: Because the simulation runtime DLL is accessed as a python module relative to the module this class resides in, we must *explicitely* register the distribution path below by registering `test_fifobuffer.__name__` below.\n",
    "\n",
    "In our derived class, we reuse the original test bench, but change the `fifo_check` subroutine:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e6ea2710-aa2f-4a8d-ada4-92f0e68dc187",
   "metadata": {},
   "outputs": [],
   "source": [
    "from cyhdl import *\n",
    "\n",
    "class TestFifo(test_fifobuffer._TestFifo):\n",
    "    @cyrite_method.sequence\n",
    "    def fifo_check(self, tap):\n",
    "        if self.reading:\n",
    "            out = self.fifobuf.pop()\n",
    "            print(\"> %02x\" % out)\n",
    "            if out != tap.data1:\n",
    "                print(\"DATA MISMATCH (wants: %s, is: %s)\" % (hex(out), hex(int(tap.data1))))\n",
    "\n",
    "                raise ValueError(\"Simulation failure\")\n",
    "\n",
    "            self.co_count.next = self.co_count + 1\n",
    "    \n",
    "        yield None\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "199f4e79-64a9-401e-ab13-93611ff4bcd4",
   "metadata": {},
   "source": [
    "Then we instance this class and create a test bench instance as well. Note that we force the signature of the module to be ignored, because it is altered by your derived class. In this case a warning is emitted instead of an exception."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "db70cd61-88a1-48ac-bbc1-4ce329c4b047",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[7;35m Declare obj 'fifo_wrapper' in context '(TestFifo 'PktFifo')'(<class '__main__.TestFifo'>) \u001b[0m\n",
      "\u001b[7;35m Declare obj 'dv_rising' in context '(TestFifo 'PktFifo')'(<class '__main__.TestFifo'>) \u001b[0m\n",
      "USING READ DELAY\n",
      "\u001b[7;35m Declare obj 'pktfifo' in context '(TestFifo 'PktFifo')'(<class '__main__.TestFifo'>) \u001b[0m\n",
      " ADDR_W: use default 6 \n",
      " SYN_RAMTYPE: use default block_ram \n",
      "\u001b[32m Module PktFifo: Existing instance lfsr8, rename to lfsr8_1 \u001b[0m\n",
      "DEBUG LIB ALL ELEM (TestFifo 'PktFifo')\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[7;31mDEBUG: not handling type <class 'myirl.kernel._types.SigType'> in co-simulation. Your Cosimulation may not run correctly.\u001b[0m\n",
      "\u001b[7;31mDEBUG: not handling type <class 'myirl.kernel._types.SigType'> in co-simulation. Your Cosimulation may not run correctly.\u001b[0m\n",
      "\u001b[7;31mDEBUG: not handling type <class 'myirl.emulation.myhdl_intbv.intbv'> in co-simulation. Your Cosimulation may not run correctly.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "t = TestFifo()\n",
    "tb = t.test_fifo(TX = False)\n",
    "tb.ignore_signature = True"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61bb8b15-a701-442c-a11c-38ccd7307356",
   "metadata": {},
   "source": [
    "Then we register the module package runtime:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "6bc3c6b1-cc2c-44dc-9656-b212657a4694",
   "metadata": {},
   "outputs": [],
   "source": [
    "tb.register_distpath(test_fifobuffer.__name__)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bdd9bc6-ff6a-47af-9123-59cbb6336dd2",
   "metadata": {},
   "source": [
    "...and run the test bench. Note again, `recompile` must explicitely be set `False`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "3946f756-c84b-488d-b828-a61c8cd71f56",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DEBUG MAIN ELAB [Instance fifo_wrapper I/F: [// ID: fifo_wrapper_0 ]]\n",
      " DEBUG Elaborating component  (TestFifo 'PktFifo') lfsr8u_1u_1_23u_1u_8 \n",
      "\u001b[32m Adding module with name `lfsr8_1` \u001b[0m\n",
      " DEBUG Elaborating component  (TestFifo 'PktFifo') pktfifo_obj_TestFifou_1_FifoPortIn_obj_module_8_in_FifoPortOut_obj_module_8_inu_1u_1u_1_5 \n",
      "\u001b[32m Adding module with name `pktfifo` \u001b[0m\n",
      " DEBUG Elaborating component  (TestFifo 'PktFifo') dv_rising_obj_TestFifou_1u_1u_1 \n",
      "\u001b[32m Adding module with name `dv_rising` \u001b[0m\n",
      "\u001b[32m Adding module with name `fifo_wrapper` \u001b[0m\n",
      "\u001b[7;34m FINALIZE implementation `fifo_wrapper` of `fifo_wrapper` \u001b[0m\n",
      "DEBUG: Ignore signature mismatch: fifo_wrapper_obj_TestFifou_1u_1u_1u_1u_1u_1u_1u_1u_1u_1_0_TestAccessPort_obj_type_out != fifo_wrapper_obj__TestFifou_1u_1u_1u_1u_1u_1u_1u_1u_1u_1_0_TestAccessPort_obj_type_out\n",
      "Open for writing: test2.vcd\n",
      "\u001b[7;35m CXXRTL context: SKIP INTERFACE ITEM `self` \u001b[0m\n",
      "\u001b[7;35m CXXRTL context: SKIP INTERFACE ITEM `TX` \u001b[0m\n",
      "DEBUG: RUNNING RX SEQUENCE\n",
      "RESET DONE\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[7;34mAttemping cold import of\u001b[0m .runtime.fifo_wrapper fifobuffer\n",
      "\u001b[32mCosimulation: co_count not connected to backend\u001b[0m\n",
      "\u001b[32mCosimulation: co_count_r not connected to backend\u001b[0m\n",
      "\u001b[32mCosimulation: co_fillcount not connected to backend\u001b[0m\n",
      "\u001b[32mCosimulation: co_ready_d not connected to backend\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 17\n",
      "> 2e\n",
      "> 5d\n",
      "> bb\n",
      "> 77\n",
      "> ef\n",
      "> df\n",
      "> be\n",
      "> 7d\n",
      "> fb\n",
      "> f6\n",
      "> ec\n",
      "> d9\n",
      "> b2\n",
      "> 65\n",
      "> ca\n",
      "> 95\n",
      "> 2b\n",
      "> 57\n",
      "> af\n",
      "> 5e\n",
      "> bd\n",
      "> 7b\n",
      "> f7\n",
      "> ee\n",
      "> dd\n",
      "RUN FLUSH START\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tb.run(8000, debug = False, wavetrace = \"test2.vcd\", recompile = False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f93d10e9-7cd5-4088-81bb-8783334de880",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}