Bug 1331702 - Add Wow64 sanity check

Bug 1331702 Validation.ipynb

Bug 1331702 Validation.py

# coding: utf-8

# # Bug 1331702 - Add Wow64 sanity check

# The test dataset was generated against [commit `51c6849`](https://github.com/mozilla/telemetry-batch-view/pull/183/commits) using the following parameters.
# 
# ```
# spark-submit \
# --master yarn \
# --deploy-mode client \
# --class com.mozilla.telemetry.views.MainSummaryView \
# target/scala-2.11/telemetry-batch-view-1.1.jar \
# --bucket net-mozaws-prod-us-west-2-pipeline-analysis/amiyaguchi/main_summary_testing \
# --from 20170101 \
# --to 20170101 \
# --channel nightly
# ```

# In[12]:

df = spark.read.parquet("s3://net-mozaws-prod-us-west-2-pipeline-analysis/"
                        "amiyaguchi/main_summary_testing/main_summary/v3/"
                        "submission_date_s3=20170101/")


# The new schema should now contain `is_wow64`.

# In[13]:

"is_wow64" in df.columns


# Let's also check that the values exist. The operating system should be Windows because this attribute is Windows only.

# In[34]:

df.groupby("is_wow64").count().show()


# In[23]:

wow64 = df.where(df.is_wow64.isNotNull()).select("os", "is_wow64")


# In[24]:

wow64.count()


# In[25]:

wow64.show()