1177137

{"nbformat_minor": 0, "cells": [{"source": "##Hello's Hello World\nA brief attempt to get data for LOOP_SHARING_STATE_CHANGE and LOOP_TWO_WAY_MEDIA_CONN_LENGTH. See this bug] for more details. You need to go to http://telemetry-dash.mozilla.org/, sign in with Persona and create a Spark Cluster. It will take time to start, but the landing page will have details on how to get to this notebook.", "cell_type": "markdown", "metadata": {}}, {"execution_count": 1, "cell_type": "code", "source": "import ujson as json\nimport matplotlib.pyplot as plt\nimport pandas as pd\nimport numpy as np\nimport plotly.plotly as py\nfrom moztelemetry import get_pings, get_pings_properties, get_one_ping_per_client, get_clients_history\n%pylab inline", "outputs": [{"output_type": "stream", "name": "stdout", "text": "Populating the interactive namespace from numpy and matplotlib\n"}], "metadata": {"collapsed": false, "trusted": true}}, {"source": "## Histograms: LOOP_TWO_WAY_MEDIA_CONN_LENGTH_1 and LOOP_SHARING_STATE_CHANGE_1\nLet's fetch some pings first for LOOP_TWO_WAY_MEDIA_CONN_LENGTH_1 and LOOP_SHARING_STATE_CHANGE_1 (ideally I would have a build-id range, but not sure who to ask right now). The API returns two histograms for each submission and each of the above:\nthe aggregate of the parent and child histograms", "cell_type": "markdown", "metadata": {}}, {"execution_count": 6, "cell_type": "code", "source": "pings = get_pings(sc, app=\"Firefox\", channel=\"nightly\", submission_date=(\"20150505\", \"20150705\"), fraction=1, schema=\"v4\")\nhistograms = get_pings_properties(pings,[\"payload/histograms/LOOP_SHARING_STATE_CHANGE_1\", \"payload/histograms/LOOP_TWO_WAY_MEDIA_CONN_LENGTH_1\"], with_processes=False)", "outputs": [], "metadata": {"collapsed": false, "trusted": true}}, {"execution_count": 5, "cell_type": "code", "source": "help(get_pings)\n", "outputs": [{"output_type": "stream", "name": "stdout", "text": "Help on function get_pings in module moztelemetry.spark:\n\nget_pings(sc, **kwargs)\n    Returns a RDD of Telemetry submissions for a given filtering criteria.\n    \n    Depending on the value of the 'schema' argument, different filtering criteria\n    are available. By default, the 'v2' schema is assumed (classic Telemetry).\n    The unified Telemetry/FHR submissions are available by selecting the 'v4' schema.\n    \n    If schema == \"v2\" then:\n    :param app: an application name, e.g.: \"Firefox\"\n    :param channel: a channel name, e.g.: \"nightly\"\n    :param version: the application version, e.g.: \"40.0a1\"\n    :param build_id: a build_id or a range of build_ids, e.g.:\n                     \"20150601000000\" or (\"20150601000000\", \"20150610999999\")\n    :param submission_date: a submission date or a range of submission dates, e.g:\n                            \"20150601\" or (\"20150601\", \"20150610\")\n    :param fraction: the fraction of pings to return, set to 1.0 by default\n    :param reason: submission reason, set to \"saved_session\" by default, e.g: \"saved_session\"\n    \n    If schema == \"v4\" then:\n    :param app: an application name, e.g.: \"Firefox\"\n    :param channel: a channel name, e.g.: \"nightly\"\n    :param version: the application version, e.g.: \"40.0a1\"\n    :param build_id: a build_id or a range of build_ids, e.g.:\n                     \"20150601000000\" or (\"20150601000000\", \"20150610999999\")\n    :param submission_date: a submission date or a range of submission dates, e.g:\n                            \"20150601\" or (\"20150601\", \"20150610\")\n    :param source_name: source name, set to \"telemetry\" by default\n    :param source_version: source version, set to \"4\" by default\n    :param doc_type: ping type, set to \"saved_session\" by default\n    :param fraction: the fraction of pings to return, set to 1.0 by default\n\n"}], "metadata": {"collapsed": false, "trusted": true}}, {"execution_count": 7, "cell_type": "code", "source": "histograms.take(5)", "outputs": [{"execution_count": 7, "output_type": "execute_result", "data": {"text/plain": "[{'payload/histograms/LOOP_SHARING_STATE_CHANGE_1': None,\n  'payload/histograms/LOOP_TWO_WAY_MEDIA_CONN_LENGTH_1': None},\n {'payload/histograms/LOOP_SHARING_STATE_CHANGE_1': None,\n  'payload/histograms/LOOP_TWO_WAY_MEDIA_CONN_LENGTH_1': None},\n {'payload/histograms/LOOP_SHARING_STATE_CHANGE_1': None,\n  'payload/histograms/LOOP_TWO_WAY_MEDIA_CONN_LENGTH_1': None},\n {'payload/histograms/LOOP_SHARING_STATE_CHANGE_1': None,\n  'payload/histograms/LOOP_TWO_WAY_MEDIA_CONN_LENGTH_1': None},\n {'payload/histograms/LOOP_SHARING_STATE_CHANGE_1': None,\n  'payload/histograms/LOOP_TWO_WAY_MEDIA_CONN_LENGTH_1': None}]"}, "metadata": {}}], "metadata": {"collapsed": false, "trusted": true}}, {"source": "A lot of Nones! Let's get a sample of some non Nones. Are they many non None packets? Not realy ...", "cell_type": "markdown", "metadata": {}}, {"execution_count": 8, "cell_type": "code", "source": "nonone1 = histograms.filter(lambda p: p[\"payload/histograms/LOOP_TWO_WAY_MEDIA_CONN_LENGTH_1\"] is not None)\nnonone2 = histograms.filter(lambda p: p[\"payload/histograms/LOOP_SHARING_STATE_CHANGE_1\"] is not None)", "outputs": [], "metadata": {"collapsed": true, "trusted": true}}, {"execution_count": 9, "cell_type": "code", "source": "print(nonone1.count())\nprint(nonone2.count())", "outputs": [{"output_type": "stream", "name": "stdout", "text": "512\n157\n"}], "metadata": {"collapsed": false, "trusted": true}}, {"source": "There are only 512 submissions that have non missing LOOP_TWO_WAY_MEDIA_CONN_LENGTH_1!!!", "cell_type": "markdown", "metadata": {}}, {"execution_count": null, "cell_type": "code", "source": "", "outputs": [], "metadata": {"collapsed": true, "trusted": true}}], "nbformat": 4, "metadata": {"kernelspec": {"display_name": "Python 2", "name": "python2", "language": "python"}, "language_info": {"mimetype": "text/x-python", "nbconvert_exporter": "python", "version": "2.7.9", "name": "python", "file_extension": ".py", "pygments_lexer": "ipython2", "codemirror_mode": {"version": 2, "name": "ipython"}}}}