This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Query to select comments that received more than 10 replies | |
query_popular = """ | |
SELECT parent, COUNT(id) | |
FROM `bigquery-public-data.hacker_news.comments` | |
GROUP BY parent | |
HAVING COUNT(id) > 10 | |
""" | |
# Set up the query (cancel the query if it would use too much of | |
# your quota, with the limit set to 1 GB) | |
safe_config = bigquery.QueryJobConfig(maximum_bytes_billed=1e9) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from google.cloud import bigquery | |
# Create a "Client" object | |
client = bigquery.Client() | |
# Construct a reference to the "hacker_news" dataset | |
dataset_ref = client.dataset("hacker_news", project="bigquery-public-data") | |
# API request - fetch the dataset | |
dataset = client.get_dataset(dataset_ref) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
if getattr(request, 'start_date'): | |
start_date = datetime.datetime.strptime(request.start_date, '%Y-%m-%d') | |
search_dict['applicant_time__gte'] = start_date | |
if getattr(request, 'end_date'): | |
end_date = datetime.datetime.strptime(request.end_date, '%Y-%m-%d') + datetime.timedelta(days=1) | |
search_dict['applicant_time__lte'] = end_date |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
class ValueProcessUtils(object): | |
@classmethod | |
def pop_none(cls, **kwargs): | |
params = dict() | |
for key, value in kwargs.iteritems(): | |
if value is not None: | |
params.update({key: value}) | |
return params |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
price_extremes = reviews.groupby('variety').price.agg([min, max]) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
iris["Species"].value_counts() | |
# or | |
reviews_written = reviews.groupby('taster_twitter_handle').size() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def stars(row): | |
if row.country == 'Canada': | |
return 3 | |
elif row.points >= 95: | |
return 3 | |
elif row.points >= 85: | |
return 2 | |
else: | |
return 1 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
n_trop = reviews.description.map(lambda desc: "tropical" in desc).sum() | |
n_fruity = reviews.description.map(lambda desc: "fruity" in desc).sum() | |
descriptor_counts = pd.Series([n_trop, n_fruity], index=['tropical', 'fruity']) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bargain_idx = (reviews.points / reviews.price).idxmax() | |
bargain_wine = reviews.loc[bargain_idx, 'title'] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
reviews[reviews.country.isin(['Australia', 'New Zealand'])] |