BL Python tutorial

tutorial.ipynb

voyager.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Breakthrough Listen: Voyager 1 Observations\n",
    "\n",
    "[Voyager 1](https://en.wikipedia.org/wiki/Voyager_1) is the most distant man-made object from Earth. Launched by NASA in 1977, it has travelled at fantastic speed (roughly 17,000 m/s), past the outer boundaries of our Solar System and into interstellar space (>12.5 billion miles from the Sun).\n",
    "\n",
    "Remarkably, 38 years on, Voyager 1 is still sending telemetry data from the depths of interstellar space. This makes it a great systems test for the Breakthrough Listen signal processing pipeline.\n",
    "\n",
    "In this tutorial, we load, read, and plot some Breakthrough Listen (BL) observations of Voyager 1. The data were taken using the [Robert C. Byrd Greenbank Telescope](https://science.nrao.edu/facilities/gbt) in West Virginia.\n",
    "\n",
    "### About this tutorial\n",
    "\n",
    "This tutorial introduces you to BL filterbank data. It is intended for intermediate to advanced users, who have experience with Python, Numpy and basic astronomy. You'll need to have [Jupyter](http://www.jupyter.org) installed, along with a scientific Python installation (numpy, scipy, matplotlib, and [astropy](http://www.astropy.org)).\n",
    "\n",
    "\n",
    "### About the data\n",
    "\n",
    "We used the Greenbank X-band receiver (8.0-11.6 GHz) on December 30, 2015, to observe the known position of Voyager 1. The BL digital signal processing system saves digitized data in a 'raw' format, which we have converted into 'filterbank' format using our `gpuspec` code (see guppi2spectra.c in https://github.com/UCBerkeleySETI/gbt_seti/tree/master/src ). For *advanced users* who want to start from scratch, the specific command is:\n",
    "\n",
    "```\n",
    "time /gbt_seti/bin/gpuspec -i ./blc3_2bit_guppi_57386_VOYAGER1_0004.0000.raw \\\n",
    "    -B 2 -f 1032192 -t 15 -V -o /datax2/scratch/dprice/\n",
    "```\n",
    "\n",
    "For the purposes of this tutorial, we suggest that you download the 504 MB file [`voyager_f1032192_t300_v2.fil`](https://storage.googleapis.com/gbt_fil/voyager_f1032192_t300_v2.fil) from the BL data archive.\n",
    "\n",
    "#### Filterbank format \n",
    "\n",
    "The voyager data is stored in *filterbank format*, a simple binary file format that is detailed in the [SIGPROC user guide](http://sigproc.sourceforge.net/sigproc.pdf). For this tutorial, we've provided a simple Python class to load and interpret the filterbank file into a [numpy](http://www.numpy.org) array. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's get started!\n",
    "\n",
    "Firstly, let's setup the notebook and import the `Filterbank()` class to read the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python2.7/site-packages/matplotlib/__init__.py:872: UserWarning: axes.color_cycle is deprecated and replaced with axes.prop_cycle; please use the latter.\n",
      "  warnings.warn(self.msg_depr % (key, alt_key))\n"
     ]
    }
   ],
   "source": [
    "import pylab as plt\n",
    "from blimpy import Filterbank"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's read the observation data using `Filterbank()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "obs = Filterbank('voyager_f1032192_t300_v2.fil')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can get basic info about the file by reading the filterbank header. The `info()` method will do this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    telescope_id :                                6\n",
      "     rawdatafile : ./blc3_2bit_guppi_57386_VOYAGER1_0004.0000.raw\n",
      "     source_name :                         VOYAGER1\n",
      "          tstart :                    57386.8649074\n",
      "       data_type :                                1\n",
      "          nchans :                         66060288\n",
      "           ibeam :                                1\n",
      "           tsamp :                      105.6964608\n",
      "            foff :               -2.83831641788e-06\n",
      "         src_raj :                     17:11:58.728\n",
      "         src_dej :                      11:56:57.48\n",
      "          nbeams :                                1\n",
      "        az_start :                              0.0\n",
      "            fch1 :                    8588.96484233\n",
      "        za_start :                              0.0\n",
      "      machine_id :                               20\n",
      "            nifs :                                1\n",
      "           nbits :                               32\n",
      "\n",
      "Num ints in file :                                2\n",
      "      Data shape :                 (2, 1, 66060288)\n",
      "Start freq (MHz) :                    8401.46484517\n",
      " Stop freq (MHz) :                    8588.96484233\n"
     ]
    }
   ],
   "source": [
    "obs.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The specifics are discussed in the detailed in the [SIGPROC user guide](http://sigproc.sourceforge.net/sigproc.pdf). Briefly, astronomers use a Celestial coordinate system to specify the location of objects in outer space. The `src_raj` and `src_dej` specify the [J2000](https://en.wikipedia.org/wiki/Epoch_%28astronomy%29#Julian_years_and_J2000) coordinates, in terms of Right Ascension and Declination (RA & DEC), toward which the telescope is pointing. `tstart` specifies the [Julian Date](https://en.wikipedia.org/wiki/Julian_day) of the observation, and `fch1` and `foff` specify the frequency of the first and frequency increment of each data channel respectively, in MHz. \n",
    "\n",
    "\n",
    "We can also look directly at the header and data, which are stored as objects in the `Filterbank()` class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'telescope_id': 6, 'rawdatafile': './blc3_2bit_guppi_57386_VOYAGER1_0004.0000.raw', 'source_name': 'VOYAGER1', 'tstart': 57386.864907407406, 'data_type': 1, 'nchans': 66060288, 'ibeam': 1, 'tsamp': 105.6964608, 'foff': -2.838316417875744e-06, 'src_raj': <Angle 17.199646666666666 hourangle>, 'src_dej': <Angle 11.9493 deg>, 'nbeams': 1, 'az_start': 0.0, 'fch1': 8588.964842330843, 'za_start': 0.0, 'machine_id': 20, 'nifs': 1, 'nbits': 32}\n",
      "(2, 1, 66060288)\n"
     ]
    }
   ],
   "source": [
    "print obs.header\n",
    "print obs.data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data has the shape (2, 66060288), meaning that is has 2 time integrations, and  66,060,288 channels, over a bandwidth of 187 MHz. For the record, that's a lot of channels! The corresponding channel bandwidth is far more than that required for most astrophysical observations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotting the data\n",
    "\n",
    "The `Filterbank()` class provides a `plot_spectrum()` function to plot the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "extracting integration 0...\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAicAAAGUCAYAAADu5oKbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAMTQAADE0B0s6tTgAAIABJREFUeJzs3Xd4FNX6B/DvJiEJgSgtFBEVpSmKAlIEBENoASJFAoZe\npCNXEVCQKkWaoBTBghQRJSoISqIiBAteFLyiUn/3eqUoBgIkSIAQkuzvj9zd7JQ9Z86WZDHfz/P4\nyGZ2Z87MzpzznvecmbXZ7XY7iIiIiAJEUFEXgIiIiMgVgxMiIiIKKAxOiIiIKKAwOCEiIqKAwuCE\niIiIAgqDEyIiIgooDE6IiIgooDA4ISIMHjwYkyZNMl02duxYTJs2DQCwa9cu9OvXDw0bNkSTJk3Q\nu3dvbN++3fneTz/9FHXr1sXPP/+sWUdubi569uyJcePGaf6+du1a1KlTB8nJyabb/vXXXzFx4kS0\natUK9evXR4sWLTB+/HicOHHC+Z7x48fj3nvvRYMGDZz/1a9fHw0aNMDBgwcBAC1btsT999+vWd6i\nRQtMmzYN2dnZhu2uXr0ao0ePtnDkiMgv7ERU7H3++ef2+vXr269cuaL5+/nz5+1169a1Hz161L5m\nzRp78+bN7du2bbNnZmbac3Nz7bt377ZHR0fbFy1a5PzMc889Z2/fvr09KyvL+bdly5bZ27Zta798\n+bJm/R06dLBPnTrV3qtXL0OZfvrpJ3uDBg3sr7/+uj0jI8Nut9vtZ86csS9atMjevHlz59/Gjx9v\nnzt3rnD/WrZsad+5c6fmb//973/tbdq0sS9btsz5t2vXrtkXLVpkr127tn306NHCdRKR/zBzQkRo\n3bo1IiMjDRmMDz/8EPXq1UPZsmXx0ksvYcWKFYiLi0OpUqUQFBSEVq1a4Y033sCaNWvw73//GwAw\nZcoU5OXlYeHChQCAgwcPYvXq1ViyZAkiIiKc6967dy8uX76M5557Dr/++it++eUXzbZnzJiBfv36\nYejQobj55psBABUrVsQzzzyDvn37Ij09XWkf7bqHYVevXh3R0dE4evSo8299+vTBH3/8gV69eimt\nm4h8628RnGRmZiIuLg6nT5+29P6lS5di+fLlztc5OTl4/vnn0alTJ8TFxRlS0kR/d8HBwYiPj8fm\nzZs1f//ggw/Qp08ffP3116hcuTLuv/9+w2fvuusuPPDAA/j8888BAKVKlcLChQuxadMmfP/993j+\n+efx1FNPoW7duprPvffee4iPj0dERAS6du2KdevWOZf9/vvvOHz4sNsgYcSIEbjjjju82ufDhw/j\n888/R7NmzZx/W7FiBRYvXoyyZct6tW4i8s4NH5wcOHAACQkJOH78uPS9ly5dwuTJk7FmzRrN3995\n5x0AwPbt2/Hyyy9jwoQJyMvL80dxiQJWz5498dNPP+HkyZMACjIb7du3x7lz5xAVFeX2sxUrVkRa\nWprz9f3334+RI0di6NChuOWWWzBgwADN+8+fP4+UlBT07NkTAPD444/js88+c67jzJkzsNlsqFKl\nivMzb731Fho1aoTGjRujfv36zswMAGzcuBGNGzd2/teoUSN0795ds83x48ejcePGeOCBB1C3bl1M\nmzYNI0eORO/evTX7QURF74YPThITEzFjxgxNpbJnzx7Ex8eje/fuGDlypLPC27FjB+68804MGjRI\ns46UlBR069YNQH4vsEqVKvjXv/5VeDtBFAAqVqyI6OhoZ/bk/fffR3x8PEJCQhAVFSXMTP7xxx+o\nUKGC5m8jRozAtWvXMGrUKMP7ExMTkZOTg8ceewwtWrTAgAEDkJub6+woONZ19uxZ52cGDx6Mffv2\n4fvvv0ebNm2Qk5PjXNa7d298//33zv/27dtnyAK99NJL+P777/HNN9+gS5cuuHz5Mtq0aaN4lIio\nMNzwwcncuXPRsGFD53hyeno6Fi5ciNWrV2Pz5s3o2LEjZs2aBQDo3r07nnjiCQQFaXf7zJkzqFSp\nkvN1VFQUzpw5U3g7QRQgEhISsHXrVmRkZGDXrl3OYZVHHnkE6enp+Oc//2n4zNGjR3Ho0CG0a9dO\n83fHdaa/3ux2O95//33Mnj0bW7dudf43ffp0JCYmIjs7G7fffjtq1KiBDz74wGf75qgjSpcujdmz\nZ6NChQoYNmyYJsghosBwwwcnej/99BNOnz6NAQMGoGvXrnjzzTfx+++/Cz9jNoRjs9n8VUSigPXQ\nQw8hLCwMc+bMQYsWLVC5cmUAQLly5TBx4kSMHz8eH3/8MS5fvoysrCykpKRg1KhR6N+/P2rVqmVp\nG7t378alS5fQqVMnlC9f3vlf165dcf36dWzbtg0AMGfOHKxduxbLly93ZlDS0tKwatUq7Nixw6sh\nmKCgICxYsAAnTpzQzD8josAQUtQF8LXc3Fw8+OCDePXVVwEA169fx8WLF4WfqVKlCs6ePYtq1aoB\nyK8AHZUyUXHz+OOPY/78+Xjrrbc0f+/Tpw9uu+02rF69GrNmzYLdbkeNGjUwbtw4dO7c2XRdZkH+\npk2b0L59e4SGhmr+HhYWhs6dO2P9+vXo0aMH6tWrh82bN+O1117D448/joyMDISHh+OBBx7A8uXL\n0aJFC+dnN27ciPfff9/52m63w2azYdy4cejbt69pOapUqYLnnnsOM2bMQPv27XH33XcrHSci8h+b\nXX9/3Q2qdevW2LBhA0JDQ/Hoo49i48aNuOOOO/D666/jl19+wbJly5zvdfSUxowZAwBYs2YN/vOf\n/2DOnDn49ddfMXToUHz22WcoUaJEkewLERFRcVYomZM1a9Y4J6fdd999eOGFFxASUrDpM2fOYMKE\nCTh//jyioqKwePFilCtXTmkbjp5RhQoVMHfuXDz99NPIy8tD+fLlsWDBAuFn+/TpgxdeeAGdO3eG\nzWbDiy++yMCEiIioiPg9c/Lzzz9jypQpeP/99xEWFoaJEyeibt26mlsLR48ejXbt2qFLly748MMP\n8e233+Kll17yZ7GIiIgoQPl9QuzNN9+MadOmISwsDABQp04d/Pnnn87lOTk5+O6775xj1l27dsXu\n3buRm5vr76IRERFRAPJ7cHL77bfjwQcfBJD/4KUNGzYgJibGuTwjIwOlS5dGcHAwgPwnVUZGRuLC\nhQv+LhoREREFoEK7W+f333/HiBEj8Pjjj6NRo0bOv5vdxmu32w3PRjCTlnbJp2UkIiIi/4uKihQu\nL5TnnBw5cgS9e/dGQkIChg0bpllWrlw5ZGZmOoOU3NxcXLlyBWXKlCmMohEREVGA8XtwcuHCBQwd\nOhTTpk1Dnz59DMtDQkLQqFEj54OXtm3bhsaNGzuHeYiIiKh48fvdOkuWLMH69etxxx13OB+M1KpV\nK5w/fx6tW7dGdHQ0UlNTMWnSJKSlpaFMmTJYtGiRpYegcViHiIjoxiMb1rmhH8LG4ISIiOjGExBz\nToiIiIisYnBCREREAYXBCREREQUUBidEREQUUBicEBERUUBhcEJEREQBhcEJERHRDSQl5QsMGdIP\nAwf2xoABCdi48W3nstWrX8PPPx8Qfn7u3JlITv7Ep2V6+OFG8jcpKLTf1iEiIiLvnDuXhhUrXsGa\nNRsRGRmJrKwsjBkzDLfddjtatGiJAwf+hYYNfRsoWGGz2Xy6PgYnREREFs2ePR3fffdPv6y7SZOH\nMGXKTOF7MjIykJubi8uXLyMyMhLh4eF4/vkZCA0NRVLSxzh69AjmzZuN2bPnIywsDAsWzMFff/2F\niIiS+Mc/JqBOnbud68rOzsaECU+hceMm6NNnAL7/fi/eeGMl8vLyEBUVhQkTJqN8+QpYs+YNfPll\nCoKDg3H33fdg/PhJftl/VwxOiIiIbhA1atRE8+Yt8fjjXVGzZm00aPAg2rRpj+rV70TVqrciOfkT\nPPHECNSoURNDhw5A374D0KpVaxw6dBBTpkzEe+9tAQDk5ORgypSJaNKkKXr37o+LFzOwcuVSLF36\nGiIjI/H5559iyZKFmDRpKrZs+QBbt34KAFi8eAHS0s4iKqqiX/eTwQkREZFFssxGYRg//jkMHPgE\n9u3bi717v8XIkYMxZcpMPPJIDADAbrfj6tWr+OOP39GqVWsAQN269+Kmm27GyZPHAQBvvrkSdjvw\nwgvzAACHDh1Eamoq/vGPEbDb7bDbgaCgIJQqVRrVq9+JIUP6oXnzh/HYYz39HpgADE6IiIhuGN9+\n+w2uXr2CmJh2iI3tjNjYzvj444+QlPSJMzgBALs9z/BZuz0POTk5AIDWrdvBbrdj1arleOqp8cjL\ny8UDD9THiy++BCA/s3LxYgYA4JVXVuLgwZ/x3Xf/xLhxYzB9+mzcf399v+4n79YhIiK6QYSHh+ON\nN1YiNTUVQH6W5Pjx/6JGjZoAgODgYOTm5iIiohSqVq2K3bt3AgAOHvwF58+fx5131gAA1KpVG8OG\njcTXX+/GoUMHcc899+LgwV9w6tRJAMC7727A4sXzkZr6JwYMSECtWnUwZMhwNGrUBP/5z78N5fL1\nbwgzc0JERHSDaNDgQQwc+ASeffYp5Obmwm63o1Gjphg8eBgAoFmzFli48EVMnjwNU6fOwsKFc7Fm\nzRsoUSIUc+YsQEhIQbMfEVEKTz75NObNewFr1mzEpEnTMH36JOTl2VG2bFlMmzYLZcuWQ5s27TB4\ncB+Eh5dE5cqV0bFjnKFcvr5bx2b3dbhTiNLSLhV1EYiIiEhRVFSkcDmHdYiIiCigMDghIiKigMLg\nhIiIiAIKgxMiIiIKKAxOiIiIKKAwOCEiIqKAwuCEiIiIAgqDEyIiIgooDE6IiIgooDA4ISIiooDC\n4ISIiIgCCoMTIiIiCigMToiIiMhvLlw4j99++6/SZwotOMnMzERcXBxOnz5tWHbq1Cn07t0bXbt2\nRb9+/fDnn38WVrGIiIjIj/r06YFRo4YgIyPd8mcKJTg5cOAAEhIScPz4cdPlL7/8MuLi4vDRRx+h\nffv2WLx4cWEUi4iIiApJWtpZy+8tlOAkMTERM2bMQMWKFU2XlypVCpcvXwYAZGVloWTJkoVRLCIi\nIgpAIYWxkblz5wIA7Ha76fLBgwejT58+WL9+PXJzc/Hee+8VRrGIiIgoABX5hFi73Y6RI0di9uzZ\n+OqrrzB9+nSMGjWqqItFRERERaTIg5P09HRkZGQgOjoaANCuXTucO3cO6enWJ84QERHR30eRByfl\nypVDREQE9u3bBwD44YcfUKpUKZQtW7aIS0ZERERFoVDmnDjYbDbnv6dMmYKYmBhER0dj2bJlmDlz\nJrKyshAREYFly5YVZrGIiIgogNjs7map3gDS0i4VdRGIiIhIIDY2f9rG0qWrULNmbQBAVFSk8DNF\nPqxDRERE5IrBCREREQUUBidEREQUUBicEBERUUBhcEJEREQBhcEJERERBRQGJ0RERBRQGJwQERFR\nQGFwQkRERAGFwQkREREFFAYnREREFFAYnBAREVFAYXBCREREAYXBCREREQUUBidEREQUUBicEBER\nUUBhcEJEREQBhcEJERERBRQGJ0RERBRQGJwQERFRQGFwQkRERAGFwQkREREFFAYnREREFFAYnBAR\nEVFAYXBCREREAYXBCREREQUUBidEREQUUBicEBERUUBhcEJEREQBpdCCk8zMTMTFxeH06dOGZWlp\naRg+fDi6du2K3r17448//iisYhEREVGAKZTg5MCBA0hISMDx48dNl0+cOBExMTH46KOP0KVLFyxY\nsKAwikVEREQBqFCCk8TERMyYMQMVK1Y0LEtPT8exY8fQs2dPAEC3bt3wzDPPFEaxiIiIKACFFMZG\n5s6dCwCw2+2GZadOnUKVKlUwb948/POf/0SVKlUwffr0wigWERERBaAinxCbk5ODQ4cOoUmTJti6\ndStiYmIwceLEoi4WERERFZEiD06ioqIQERGB6OhoAEBcXBwOHjxYxKUiIiKiolLkwUm1atVwyy23\nYPfu3QCA3bt345577inaQhEREVGRKdTgxGazOf89ZcoUpKSkAACWL1+O1atXIy4uDmvXrnXOUSEi\nIqLix2Y3m6V6g0hLu1TURSAiIiKB2Nj8aRtLl65CzZq1AQBRUZHCzxT5sA4RERGRKwYnREREFFAY\nnBAREVFAYXBCREREAYXBCREREQUUBidEREQUUBicEBERUUBhcEJEREQBhcEJERERBRQGJ0RERBRQ\nGJwQERFRQGFwQkRERAGFwQkREREFFAYnREREFFAYnBAREVFAYXBCREREAYXBCREREQUUBidEREQU\nUBicEBERUUBhcEJEREQBhcEJERERBRQGJ0RERBRQGJwQERFRQGFwQkRERAGFwQkREREFFAYnRERE\nFFAYnBAREVFAYXBCREREAYXBCREREQWUQgtOMjMzERcXh9OnT7t9z+HDh3HfffcVVpGIiIgoABVK\ncHLgwAEkJCTg+PHjbt+TlZWFWbNmIScnpzCKRERERAGqUIKTxMREzJgxAxUrVnT7nnnz5mHQoEGF\nURwiIiIKYCGFsZG5c+cCAOx2u+nyXbt24dq1a2jXrp3b9xAREVHxUCjBici5c+ewcuVKrF27FgBg\ns9mKtkBERERUpIr8bp2UlBRcvHgRffv2RdeuXWG329GtWzdkZmYWddGIiIioCBR55iQ+Ph7x8fHO\n13Xq1MGWLVuKsERERERUlAo1c+I6ZDNlyhSkpKQI30NERETFT6FmTnbu3On89+zZs03fc+TIkcIq\nDhEREQWgIp9zQkREROSKwQkREREFFAYnREREFFAYnBAREVFAYXBCREREAYXBCREREQUUBidEREQU\nUBicEBERUUBhcE