acviana

#!/usr/bin/env python

import requests
import json

with open('driver-en.js', 'r') as f:
    data = f.readlines()

output_list =[]
for counter, record in enumerate(data):

acviana

Unix Commands

How many requests are in the log file?

$ wc example.log
 9999999 173321954 1629465770 example.log

9,999,999 assuming 1 per line.

What percentage of requests from iPads resulted in HTTP status code 500?

acviana

Numpy

Load the Numpy module:

import numpy as np

Use the Numpy genfromtxt function to load the data, manually defining the column names.

acviana

Data Analysis in the Shell

The shell provides several utilities that aren't great for large scale analysis but can be useful for quick data analysis.

Reading Data

There are various ways to display human-readable file.

• less

acviana

def check_date_format(date):
    '''
    Check if that date string has a 'MMM DD YYYY' format. Return a boolean.
    '''
    output = True
    if isinstance(date,unicode) == False:
        return False
    month, day, year = date.split()
    if set(month) >= set(string.ascii_letters):
        return False

acviana

'''
This module is a script demonstrating some basic features of the 
SQLAlchemy ORM declarative base. First shared with the Baltimore
Python meetup 06/19/13.

It requires either a MySQL instance or SQLite to run as written but 
you can modify it run with any SQL flavor SQLAlchemy supports.

Alex C. Viana
alexcostaviana [at] gmail [dot] com

acviana

Keybase proof

I hereby claim:

• I am acviana on github.
• I am acv (https://keybase.io/acv) on keybase.
• I have a public key ASARGh9PzYhkW7ifTJAzViwVjjlHNXkPKGOIS5vBgON-fgo

To claim this, I am signing this object: