Skip to content

Instantly share code, notes, and snippets.

View tnbe21's full-sized avatar

tnbe21 tnbe21

  • Tokyo, Japan
  • 18:03 (UTC +09:00)
  • X @tnbe21
View GitHub Profile
@tnbe21
tnbe21 / grep.py
Created June 26, 2021 20:25
Cloud Dataflow code sample
#!/usr/bin/env python
"""
Copyright Google Inc. 2016
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@tnbe21
tnbe21 / find_correlation_between_rain_and_bicycle_rentals.sql
Last active February 20, 2021 09:02
Using BigQuery to do Analysis
/**
result: [
{
"num_trips": "28598.0",
"rainy": false
},
{
"num_trips": "19503.0",
"rainy": true
}
@tnbe21
tnbe21 / aggregations.sql
Created February 13, 2021 09:33
google_public_taxirides_dataset_aggregation
WITH streaming_data AS (
SELECT
timestamp,
TIMESTAMP_TRUNC(timestamp, HOUR, 'UTC') AS hour,
TIMESTAMP_TRUNC(timestamp, MINUTE, 'UTC') AS minute,
TIMESTAMP_TRUNC(timestamp, SECOND, 'UTC') AS second,
ride_id,
latitude,
longitude,
@tnbe21
tnbe21 / logistic_regression.sql
Created February 9, 2021 01:03
SQLs of the different model_types in BigQuery ML
CREATE OR REPLACE MODEL `ecommerce.classification_model_2`
OPTIONS
(model_type = 'logistic_reg', labels = ['will_buy_on_return_visit']) AS
WITH all_visitor_stats AS (
SELECT
fullvisitorid,
IF (COUNTIF(totals.transactions > 0 AND totals.newVisits IS NULL) > 0, 1, 0) AS will_buy_on_return_visit
FROM `data-to-insights.ecommerce.web_analytics`
GROUP BY fullvisitorid
#!/usr/bin/env python
"""
Copyright Google Inc. 2016
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
import argparse
import random
import string
parser = argparse.ArgumentParser()
DEFAULT_STR_LEN = 8
parser.add_argument(
"-l", "--string-length", type=int, default=DEFAULT_STR_LEN, help=f"string length (default: {DEFAULT_STR_LEN})")
parser.add_argument("-s", "--include-symbol", action="store_true", help="include symbol characters")
args = parser.parse_args()
from collections import deque
def sort_for_heap_tree(arr):
arr_len = len(arr)
for parent_idx in range(arr_len):
child1_idx = parent_idx * 2 + 1
if child1_idx >= arr_len:
break
child1 = arr[child1_idx]
from collections import deque
def sort_by_quick_sort(arr):
if len(arr) == 1:
return arr
base_value = arr[0]
smaller = deque([arr[0]])
larger = deque()
cnt = 0
def sort_by_merge_sort(arr):
if len(arr) == 1:
return arr
if len(arr) == 2:
if arr[0] < arr[1]:
return arr
else:
return [arr[1], arr[0]]