First example of a notebook based report

notebook_report.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Characterization Metric Report: Job 1163"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## By Simon Krughoff and Michael Wood-Vasey -- DMTR-999"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Abstract\n",
    "\n",
    "This brief report describes measurements of interest that were carried out for Job 1163 of the Science Pipeline.\n",
    "\n",
    "In short, the astrometric performance is excellent in comparison to both the per cycle ramps as well as the design level KPMs. The photometric performance is not as good. The inclusion of jointcal in the processing in future cycles will improve photometric performance. Current reprocessing of HSC data using a precursor to jointcal called meas_mosaic shows significant improvement over single epoch processing.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Introduction\n",
    "\n",
    "Measured using validation_data_hsc, which consists of 8 HSC engineering images: 2 r-band, 4 i’-band, and 2 y-band. Measurements were made on individual, separately-processed, single frame images: Jointcal and/or meas_mosaic were not run. For comparison, we provide the SRD required “design” value of each metric as defined in the Science Requirements Docu- ment [LPM-17](https://ls.st/LPM-17), and, where available, the target for this release as defined in the Data Man- agement Development Milestone Roadmap [LDM-240](https://ls.st/LDM-240). All values were computed using the `examples/runHscTest.sh` script in the validate_drp package.\n",
    "\n",
    "Some KPMs (AF1, AD1) involve thresholds that are different for “design”, “minimum”, and “stretch” specifications. Thus comparing one of these metrics against a given target number is a two-level process. Both the threshold used in the calculation is dependent on the specifi- cations, and the requirement on the computed number is dependent on the specifications.\n",
    "\n",
    "The metrics in this report have all been computed relative to the “design” thresholds. The values of these KPMs would be different if computed against different thresholds.\n",
    "\n",
    "Note also that the photometric performance of the pipelines in the y-band is an under estmate of expected delivered performance. For these tests, the y-band data was calibrated with z-band photometry. This is due to the lack of a reference catalog containing y-band informa- tion at this time. We recognize that the bandpass mismatch is certainly not the only source of scatter in the y-band photometry. These metric measurements are still worth noting in this report as a historical benchmark to track relative performance.\n",
    "\n",
    "The per cycle target numbers come from the “KPMs” sheet of [LDM-240](LDM-240)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Photometric Performance\n",
    "\n",
    "_Submitted by Simon Krughoff and Michael Wood-Vasey_\n",
    "\n",
    "procCalRep corresponds to requirement OSS-REQ-0275 (defined in [LSE-30](http://ls.st/LSE-30)). All other photo- metric performance metrics follow LSS-REQ-0093 ([LSE-29](http://ls.st/LSE-29)) and [LPM-17](http://ls.st/LPM-17) table 14."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from astropy.table import Table, Column\n",
    "import numpy as np\n",
    "band_map = {'HSC-R':'r', 'HSC-I':'i'}\n",
    "jobs_ids = {'HSC-R':1661, 'HSC-I':1660}\n",
    "data = {'Metric':['PA1', 'PA1', 'AM1', 'AM1'], 'band':['HSC-R', 'HSC-I', 'HSC-R', 'HSC-I'], 'metric value':[np.nan, np.nan, np.nan, np.nan],\n",
    "        'spec operator':['<=', '<=', '<=', '<='], 'spec value':[np.nan, np.nan, np.nan, np.nan]}\n",
    "table = Table(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "jobs = {}\n",
    "for band, job_id in jobs_ids.items():\n",
    "    r = requests.get(\"https://squash-restful-api.lsst.codes/job/%i\"%job_id)\n",
    "    jobs[band] = r.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_get_str(metric_name, band):\n",
    "    if metric_name.startswith('AM'):\n",
    "        get_str = 'https://squash-restful-api.lsst.codes/spec/validate_drp.%s.design'%(metric_name)\n",
    "    elif metric_name.startswith('PA'):\n",
    "        get_str = 'https://squash-restful-api.lsst.codes/spec/validate_drp.%s.hsc_design_%s'%(metric_name, band)\n",
    "    else:\n",
    "        raise ValueError('Only AM and PA metrics supported currently')\n",
    "    \n",
    "    return get_str"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "units = []\n",
    "for i, v in enumerate(zip(table['Metric'], table['band'])):\n",
    "    for measurement in jobs[v[1]]['measurements']:\n",
    "        if measurement['metric'] == 'validate_drp.'+v[0]:\n",
    "            table['metric value'][i] = measurement['value']\n",
    "            units.append(measurement['unit'])\n",
    "            \n",
    "            r = requests.get(make_get_str(v[0], band_map[v[1]]))\n",
    "            result = r.json()\n",
    "            table['spec value'] = result['threshold']['value']\n",
    "\n",
    "col = Column(units, name='unit')\n",
    "table.add_column(col)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "table.show_in_notebook()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "LSST",
   "language": "python",
   "name": "lsst"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}