Skip to content

Instantly share code, notes, and snippets.

Avatar

Kiichi ki-chi

  • Tokyo, Japan
  • 17:18 (UTC +09:00)
View GitHub Profile
@ki-chi
ki-chi / pandas_coalesce.py
Last active Sep 7, 2022
coalesce function by pandas
View pandas_coalesce.py
import pandas as pd
def coalesce(df: pd.DataFrame, *args) -> pd.Series:
return df[list(args)].bfill(axis=1).iloc[:, 0]
@ki-chi
ki-chi / pandas_parquet.dockerfile
Created Jun 30, 2022
pd.read_parquet()の挙動の確認用
View pandas_parquet.dockerfile
FROM python:3.10
RUN pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir pandas==1.4.3 pyarrow==8.0.0 fastparquet==0.8.1
@ki-chi
ki-chi / melt_columns.py
Created Aug 6, 2021
Melt hierarchal columns of pandas dataframe
View melt_columns.py
def melt_columns(df: pd.DataFrame, sep: str="_") -> pd.DataFrame:
"""与えられたデータフレームの列がMultiIndexの場合にそれを非階層なIndexにして返す.
Arguments:
df (pd.DataFrame) : 列を非階層にしたいデータフレーム.
sep (str) : 新たな列名を作るときにセパレーターとして使われる文字列. デフォルトでは"_".
Return:
pd.DataFrame : 列名が非階層なIndexとなっているデータフレーム
View 2020-11-19-224416.py
import time
import numpy as np
import pandas as pd
def np_mode(df):
"""
与えられたDataFrameの各列から最頻値を返す
Original: https://twitter.com/nkay/status/1328231713919496194
"""
View 2020-11-19-225744.jl
# julia> versioninfo()
# Julia Version 1.5.0
# Commit 96786e22cc (2020-08-01 23:44 UTC)
# Platform Info:
# OS: macOS (x86_64-apple-darwin18.7.0)
# CPU: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
# WORD_SIZE: 64
# LIBM: libopenlibm
# LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
@ki-chi
ki-chi / num2int.py
Created Jan 24, 2017
The function for converting tricky-format digits
View num2int.py
## Convert strings of 2-digits number have the following tricky format to an integer.
## Tricky format: the last digit is described as decimal, and the first digit is chosen from {1, ..., 9, A, ..., Z, a, ..., z}.
## The min is 0(="00") and the max is 619(="z9").
## Example: "15" -> 15, "A0" -> 100, "Z9" -> 359, "a0" -> 360, "c5" -> 385
def num2int(num):
if num.isdigit():
return int(num)
elif num[0].isupper():
return int(num[0], 36) * 10 + int(num[1])
@ki-chi
ki-chi / matplotlib_hokkaido.ipynb
Created Jan 22, 2017
matplotlib_hokkaido.ipynb
View matplotlib_hokkaido.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@ki-chi
ki-chi / jsm2pandas.py
Created Oct 3, 2016
Convert instances of jsm into Pandas DataFrame.
View jsm2pandas.py
import pandas as pd
import jsm
import datetime
def get_historical_price(stockcode, start_date=None, end_date=None):
q = jsm.Quotes()
if start_date is None or end_date is None:
jsm_data = q.get_historical_prices(stockcode)
else:
jsm_data = q.get_historical_prices(stockcode, jsm.DAILY, start_date, end_date, jsm.DAILY)
@ki-chi
ki-chi / cluster_coef_for_unweighted_undirected_graph.py
Last active Dec 12, 2015
重み無し無向グラフのクラスター係数を隣接行列から求める
View cluster_coef_for_unweighted_undirected_graph.py
# -*- coding: utf-8 -*-
from __future__ import print_function
import numpy as np
# it's for unweighted and undirected graph.
# Adjency matrix of the graph
adjency_mat = np.array([[0, 1, 1, 1, 0, 0],
[1, 0, 1, 0, 0, 1],
[1, 1, 0, 1, 1, 0],
[1, 0, 1, 0, 1, 0],
@ki-chi
ki-chi / nikkei225_crawler.py
Last active Aug 29, 2015
日経平均株価の日次データを取得するクローラー
View nikkei225_crawler.py
#!/usr/bin/python
# coding: utf-8
# 日経平均資料室(http://indexes.nikkei.co.jp/nkave/archives/data)の日次データの差分をクロールして追加するプログラム
import urllib2
import lxml.html
import csv
import re
import time
from datetime import datetime as dt