APS2019DataCollection.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# APS March Meeting 2019 Talk: Data Collection\n",
    "\n",
    "Karalekas et al., **F27.00003** : *Compiler tools for hybrid quantum-classical algorithms*\n",
    "\n",
    "This notebook contains the data collection from my talk \"[Compiler tools for hybrid-quantum-classical algorithms](https://meetings.aps.org/Meeting/MAR19/Session/F27.3)\" at the international physics conference, **APS March Meeting**, in Boston. Before the data can be collected, you must have booked QPU time on [Quantum Cloud Services](https://rigetti.com/qcs). Running the notebook itself requires `numpy`, `pandas`, `pyquil`, and `tqdm` to run. Additionally, it depends on a CSV file that contains the timing data from our previous quantum computing platform, the Forest Web API. This file should be included in a tarball `timing-data.tgz` that is in the same GitHub Gist as this notebook (it will need to be unzipped before this will run). Once all these requirements are satisfied, you should be able to re-collect the QCS data yourself! For the data analysis component, check out [APS2019DataAnalysis.ipynb](https://gist.github.com/karalekas/1cb4bacb1a1607b8796955f2fa569c20), which is also available as a GitHub Gist."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Determine Average Job Statistics\n",
    "\n",
    "We begin by extracting information about the median number of qubits, CZ gates, and RX gates run on the Forest Web API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_forest_web_api = pd.read_csv('timing-data/forest_web_api.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Median number of qubits = 3.0\n"
     ]
    }
   ],
   "source": [
    "print(f'Median number of qubits = {np.median(df_forest_web_api.nQ)}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Median number of CZ gates = 2.0\n"
     ]
    }
   ],
   "source": [
    "print(f'Median number of CZ gates = {np.median(df_forest_web_api.nCZ)}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Median number of RX gates = 14.0\n"
     ]
    }
   ],
   "source": [
    "print(f'Median number of RX gates = {np.median(df_forest_web_api.nRX)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Design a \"Representative\" Quil Program\n",
    "\n",
    "Using the average Forest Web API job statistics we just determined, we design a Quil program to run on Quantum Cloud Services. Because programs pass through `quilc`, we verify that the programs have the desired number of CZs and RXs after that compiler pass."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyquil import get_qc, Program\n",
    "from pyquil.gates import CZ, MEASURE, RESET, RX\n",
    "from pyquil.quilbase import Gate, Pragma"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "qpu = get_qc('Aspen-1-3Q-D')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def count_CZs(program: Program) -> int:\n",
    "    count = 0\n",
    "    for instruction in program.instructions:\n",
    "        if isinstance(instruction, Gate) and instruction.name == 'CZ':\n",
    "            count += 1\n",
    "    return count\n",
    "\n",
    "\n",
    "def count_RXs(program: Program) -> int:\n",
    "    count = 0\n",
    "    for instruction in program.instructions:\n",
    "        if isinstance(instruction, Gate) and instruction.name == 'RX':\n",
    "            count += 1\n",
    "    return count"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Standard (Non-Parametric) Program"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PRAGMA INITIAL_REWIRING \"GREEDY\"\n",
      "DECLARE ro BIT[3]\n",
      "RX(0.1) 0\n",
      "RX(0.2) 1\n",
      "CZ 0 1\n",
      "RX(0.1) 0\n",
      "RX(0.2) 1\n",
      "RX(0.3) 2\n",
      "CZ 1 2\n",
      "RX(0.2) 1\n",
      "RX(0.3) 2\n",
      "MEASURE 0 ro[0]\n",
      "MEASURE 1 ro[1]\n",
      "MEASURE 2 ro[2]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "standard_program = Program(Pragma('INITIAL_REWIRING', freeform_string='GREEDY'))\n",
    "ro = standard_program.declare('ro', 'BIT', 3)\n",
    "theta = [0.1, 0.2, 0.3]\n",
    "standard_program.inst(RX(theta[0], 0),\n",
    "                      RX(theta[1], 1),\n",
    "                      CZ(0, 1),\n",
    "                      RX(theta[0], 0),\n",
    "                      RX(theta[1], 1),\n",
    "                      RX(theta[2], 2),\n",
    "                      CZ(1, 2),\n",
    "                      RX(theta[1], 1),\n",
    "                      RX(theta[2], 2),\n",
    "                      MEASURE(0, ro[0]),\n",
    "                      MEASURE(1, ro[1]),\n",
    "                      MEASURE(2, ro[2]))\n",
    "print(standard_program)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CZs in quilc-compiled standard program = 2\n"
     ]
    }
   ],
   "source": [
    "print(f'CZs in quilc-compiled standard program = {count_CZs(qpu.compiler.quil_to_native_quil(standard_program))}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RXs in quilc-compiled standard program = 14\n"
     ]
    }
   ],
   "source": [
    "print(f'RXs in quilc-compiled standard program = {count_RXs(qpu.compiler.quil_to_native_quil(standard_program))}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parametric Program"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PRAGMA INITIAL_REWIRING \"GREEDY\"\n",
      "RESET\n",
      "DECLARE ro BIT[3]\n",
      "DECLARE theta REAL[3]\n",
      "RX(theta[0]) 0\n",
      "RX(theta[1]) 1\n",
      "CZ 0 1\n",
      "RX(theta[0]) 0\n",
      "RX(theta[1]) 1\n",
      "RX(theta[2]) 2\n",
      "CZ 1 2\n",
      "RX(theta[1]) 1\n",
      "RX(theta[2]) 2\n",
      "MEASURE 0 ro[0]\n",
      "MEASURE 1 ro[1]\n",
      "MEASURE 2 ro[2]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "parametric_program = Program(Pragma('INITIAL_REWIRING', freeform_string='GREEDY'))\n",
    "parametric_program.inst(RESET())\n",
    "ro = parametric_program.declare('ro', 'BIT', 3)\n",
    "theta = parametric_program.declare('theta', 'REAL', 3)\n",
    "parametric_program.inst(RX(theta[0], 0),\n",
    "                        RX(theta[1], 1),\n",
    "                        CZ(0, 1),\n",
    "                        RX(theta[0], 0),\n",
    "                        RX(theta[1], 1),\n",
    "                        RX(theta[2], 2),\n",
    "                        CZ(1, 2),\n",
    "                        RX(theta[1], 1),\n",
    "                        RX(theta[2], 2),\n",
    "                        MEASURE(0, ro[0]),\n",
    "                        MEASURE(1, ro[1]),\n",
    "                        MEASURE(2, ro[2]))\n",
    "print(parametric_program)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CZs in quilc-compiled parametric program = 2\n"
     ]
    }
   ],
   "source": [
    "print(f'CZs in quilc-compiled parametric program = {count_CZs(qpu.compiler.quil_to_native_quil(parametric_program))}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RXs in quilc-compiled parametric program = 14\n"
     ]
    }
   ],
   "source": [
    "print(f'RXs in quilc-compiled parametric program = {count_RXs(qpu.compiler.quil_to_native_quil(parametric_program))}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Collect the Data on Quantum Cloud Services\n",
    "\n",
    "Using the representative Quil program, we take data on QCS with standard compilation, parametric compilation, and parametric compilation + active qubit reset for the same top ten numbers of shots. We then additionally take some data for higher numbers of shots, in order to understand the asymptotic behavior of the shot rates available on QCS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "from pyquil.api import QuantumComputer\n",
    "from tqdm import tqdm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "def run_standard_program(qpu: QuantumComputer, thetas: list, trials: int) -> list:\n",
    "    qpu.reset()\n",
    "    times = []\n",
    "    for theta in tqdm(thetas):\n",
    "        start = time.time()\n",
    "        # build standard program\n",
    "        program = Program(Pragma('INITIAL_REWIRING', freeform_string='GREEDY'))\n",
    "        ro = program.declare('ro', 'BIT', 3)\n",
    "        theta_parameter = [theta, theta, theta]\n",
    "        program.inst(RX(theta_parameter[0], 0),\n",
    "                     RX(theta_parameter[1], 1),\n",
    "                     CZ(0, 1),\n",
    "                     RX(theta_parameter[0], 0),\n",
    "                     RX(theta_parameter[1], 1),\n",
    "                     RX(theta_parameter[2], 2),\n",
    "                     CZ(1, 2),\n",
    "                     RX(theta_parameter[1], 1),\n",
    "                     RX(theta_parameter[2], 2),\n",
    "                     MEASURE(0, ro[0]),\n",
    "                     MEASURE(1, ro[1]),\n",
    "                     MEASURE(2, ro[2]))\n",
    "        program.wrap_in_numshots_loop(trials)\n",
    "        # compile every time\n",
    "        binary = qpu.compile(program)\n",
    "        # run standard binary\n",
    "        results = qpu.run(binary)\n",
    "        times.append(time.time() - start)\n",
    "    return times"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "def run_parametric_program(qpu: QuantumComputer, thetas: list, trials: int, active_reset: bool) -> list:\n",
    "    qpu.reset()\n",
    "    program = Program(Pragma('INITIAL_REWIRING', freeform_string='GREEDY'))\n",
    "    if active_reset:\n",
    "        program.inst(RESET())\n",
    "    # build parametric program\n",
    "    ro = program.declare('ro', 'BIT', 3)\n",
    "    theta_parameter = program.declare('theta', 'REAL', 3)\n",
    "    program.inst(RX(theta_parameter[0], 0),\n",
    "                 RX(theta_parameter[1], 1),\n",
    "                 CZ(0, 1),\n",
    "                 RX(theta_parameter[0], 0),\n",
    "                 RX(theta_parameter[1], 1),\n",
    "                 RX(theta_parameter[2], 2),\n",
    "                 CZ(1, 2),\n",
    "                 RX(theta_parameter[1], 1),\n",
    "                 RX(theta_parameter[2], 2),\n",
    "                 MEASURE(0, ro[0]),\n",
    "                 MEASURE(1, ro[1]),\n",
    "                 MEASURE(2, ro[2]))\n",
    "    program.wrap_in_numshots_loop(trials)\n",
    "    # compile only once\n",
    "    binary = qpu.compile(program)\n",
    "    times = []\n",
    "    for theta in tqdm(thetas):\n",
    "        start = time.time()\n",
    "        # run and provide parameter values\n",
    "        results = qpu.run(binary, memory_map={'theta': [theta, theta, theta]})\n",
    "        times.append(time.time() - start)\n",
    "    return times"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_pts = 1000\n",
    "trials_list = [1, 10, 50, 100, 500, 1000, 2000, 5000, 8000, 10000]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Collect Data for Standard Compilation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1000/1000 [03:21<00:00,  5.56it/s]\n",
      "100%|██████████| 1000/1000 [03:21<00:00,  5.59it/s]\n",
      "100%|██████████| 1000/1000 [03:26<00:00,  5.44it/s]\n",
      "100%|██████████| 1000/1000 [03:30<00:00,  5.25it/s]\n",
      "100%|██████████| 1000/1000 [04:13<00:00,  4.24it/s]\n",
      "100%|██████████| 1000/1000 [05:07<00:00,  3.48it/s]\n",
      "100%|██████████| 1000/1000 [06:53<00:00,  2.49it/s]\n",
      "100%|██████████| 1000/1000 [12:12<00:00,  1.40it/s]\n",
      "100%|██████████| 1000/1000 [17:32<00:00,  1.03s/it]\n",
      "100%|██████████| 1000/1000 [21:04<00:00,  1.24s/it]\n"
     ]
    }
   ],
   "source": [
    "rows_sc = []\n",
    "\n",
    "for trials in trials_list:\n",
    "    times = run_standard_program(qpu, thetas=np.linspace(0, 4 * np.pi, num_pts), trials=trials)\n",
    "    \n",
    "    for t in times:\n",
    "        rows_sc.append({'trials': trials, 'time': t})\n",
    "    \n",
    "    df_qcs_standard_compilation = pd.DataFrame(rows_sc)\n",
    "    df_qcs_standard_compilation.to_csv('qcs_standard_compilation.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Collect Data for Parametric Compilation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1000/1000 [00:35<00:00, 27.95it/s]\n",
      "100%|██████████| 1000/1000 [00:36<00:00, 27.35it/s]\n",
      "100%|██████████| 1000/1000 [00:40<00:00, 24.47it/s]\n",
      "100%|██████████| 1000/1000 [00:46<00:00, 21.70it/s]\n",
      "100%|██████████| 1000/1000 [01:29<00:00, 11.24it/s]\n",
      "100%|██████████| 1000/1000 [02:22<00:00,  7.01it/s]\n",
      "100%|██████████| 1000/1000 [04:09<00:00,  4.01it/s]\n",
      "100%|██████████| 1000/1000 [09:26<00:00,  1.77it/s]\n",
      "100%|██████████| 1000/1000 [14:44<00:00,  1.13it/s]\n",
      "100%|██████████| 1000/1000 [18:15<00:00,  1.10s/it]\n"
     ]
    }
   ],
   "source": [
    "rows_pc = []\n",
    "\n",
    "for trials in trials_list:\n",
    "    times = run_parametric_program(qpu, thetas=np.linspace(0, 4 * np.pi, num_pts), trials=trials, active_reset=False)\n",
    "    \n",
    "    for t in times:\n",
    "        rows_pc.append({'trials': trials, 'time': t})\n",
    "    \n",
    "    df_qcs_parametric_compilation = pd.DataFrame(rows_pc)\n",
    "    df_qcs_parametric_compilation.to_csv('qcs_parametric_compilation.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Collect Data for Parametric Compilation and Active Qubit Reset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1000/1000 [00:36<00:00, 27.13it/s]\n",
      "100%|██████████| 1000/1000 [00:37<00:00, 26.81it/s]\n",
      "100%|██████████| 1000/1000 [00:37<00:00, 26.55it/s]\n",
      "100%|██████████| 1000/1000 [00:38<00:00, 25.93it/s]\n",
      "100%|██████████| 1000/1000 [00:44<00:00, 22.39it/s]\n",
      "100%|██████████| 1000/1000 [00:52<00:00, 18.96it/s]\n",
      "100%|██████████| 1000/1000 [01:08<00:00, 14.58it/s]\n",
      "100%|██████████| 1000/1000 [01:56<00:00,  8.53it/s]\n",
      "100%|██████████| 1000/1000 [02:43<00:00,  6.11it/s]\n",
      "100%|██████████| 1000/1000 [03:15<00:00,  5.09it/s]\n"
     ]
    }
   ],
   "source": [
    "rows_ar = []\n",
    "\n",
    "for trials in trials_list:\n",
    "    times = run_parametric_program(qpu, thetas=np.linspace(0, 4 * np.pi, num_pts), trials=trials, active_reset=True)\n",
    "    \n",
    "    for t in times:\n",
    "        rows_ar.append({'trials': trials, 'time': t})\n",
    "    \n",
    "    df_qcs_active_qubit_reset = pd.DataFrame(rows_ar)\n",
    "    df_qcs_active_qubit_reset.to_csv('qcs_active_qubit_reset.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Collect High Shots (HS) Data on Quantum Cloud Services\n",
    "\n",
    "We define high shots (HS) to be numbers of shots above the \"top ten\" that we are using for the roundtrip time comparisons in the talk."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_pts_hs = 1000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Standard Compilation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1000/1000 [38:44<00:00,  2.30s/it]\n",
      "100%|██████████| 1000/1000 [1:31:46<00:00,  5.47s/it]\n"
     ]
    }
   ],
   "source": [
    "rows_sc_hs = []\n",
    "trials_list_hs = [20000, 50000]\n",
    "\n",
    "for trials in trials_list_hs:\n",
    "    times = run_standard_program(qpu, thetas=np.linspace(0, 4 * np.pi, num_pts_hs), trials=trials)\n",
    "    \n",
    "    for t in times:\n",
    "        rows_sc_hs.append({'trials': trials, 'time': t})\n",
    "    \n",
    "    df_qcs_standard_compilation_hs = pd.DataFrame(rows_sc_hs)\n",
    "    df_qcs_standard_compilation_hs.to_csv('qcs_standard_compilation_hs.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parametric Compilation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1000/1000 [35:51<00:00,  2.15s/it]\n",
      "100%|██████████| 1000/1000 [1:28:42<00:00,  5.32s/it]\n"
     ]
    }
   ],
   "source": [
    "rows_pc_hs = []\n",
    "trials_list_hs = [20000, 50000]\n",
    "\n",
    "for trials in trials_list_hs:\n",
    "    times = run_parametric_program(qpu, thetas=np.linspace(0, 4 * np.pi, num_pts_hs), trials=trials, active_reset=False)\n",
    "    \n",
    "    for t in times:\n",
    "        rows_pc_hs.append({'trials': trials, 'time': t})\n",
    "    \n",
    "    df_qcs_parametric_compilation_hs = pd.DataFrame(rows_pc_hs)\n",
    "    df_qcs_parametric_compilation_hs.to_csv('qcs_parametric_compilation_hs.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Active Qubit Reset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1000/1000 [05:54<00:00,  2.82it/s]\n",
      "100%|██████████| 1000/1000 [13:44<00:00,  1.21it/s]\n",
      "100%|██████████| 1000/1000 [26:50<00:00,  1.61s/it]\n"
     ]
    }
   ],
   "source": [
    "rows_ar_hs = []\n",
    "trials_list_hs = [20000, 50000, 100000]  # we go out further for active qubit reset\n",
    "\n",
    "for trials in trials_list_hs:\n",
    "    times = run_parametric_program(qpu, thetas=np.linspace(0, 4 * np.pi, num_pts_hs), trials=trials, active_reset=True)\n",
    "    \n",
    "    for t in times:\n",
    "        rows_ar_hs.append({'trials': trials, 'time': t})\n",
    "    \n",
    "    df_qcs_active_qubit_reset_hs = pd.DataFrame(rows_ar_hs)\n",
    "    df_qcs_active_qubit_reset_hs.to_csv('qcs_active_qubit_reset_hs.csv', index=False)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}

timing-data.tgz