Skip to content

Instantly share code, notes, and snippets.

🌴
On vacation

catindog/nardtree GINK03

🌴
On vacation
Block or report user

Report or block GINK03

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@GINK03
GINK03 / gist:007eeaad1f0d0d93e93d0b40e64825e9
Created Apr 2, 2017
データ分析者のElixir, Kotlin入門
View gist:007eeaad1f0d0d93e93d0b40e64825e9
# データ分析に使える言語はPython, Rだけではない
@GINK03
GINK03 / char_vec.py
Created Apr 3, 2017
char level 絵文字を意識したfasttextによるベクトル化
View char_vec.py
DATA_SIZE = 300000
def _make_char_index():
char_index = {}
for char in open('./char_level.txt', 'r').read().replace('\n', ' ').split():
emoji = re.compile(u'['
u'\U0001F300-\U0001F5FF'
u'\U0001F600-\U0001F64F'
u'\U0001F680-\U0001F6FF'
u'\u2600-\u26FF\u2700-\u27BF]+',
re.UNICODE)
View keras-rnn-char-fasttext-conditiona-rnn.py
counter = 0
def build_model(mode=None, maxlen=None, output_dim=None):
print('Build model...')
def _scheduler(epoch):
global counter
counter += 1
rate = learning_rates[counter]
#model.lr.set_value(rate)
print(model.optimizer.lr)
#print(counter, rate)
View crazy-format-parser.py
import os
import math
import sys
import itertools
import subprocess
import glob
import csv
from collections import OrderedDict as dict
import pickle
import re
View polarity
This file has been truncated, but you can view the full file.
ダイエー -4.211883314885654
がっかり -3.724952240087231
最悪 -3.629671589687795
二度と -3.615183142062377
期待はずれ -3.364096361814979
在庫管理 -3.251811276002615
シーブリーズ -3.243134607447971
返金 -3.223751242139063
江尾 -3.142244830633572
View adhoc.kt
import com.treasuredata.client.*
import com.google.common.base.Function
import org.msgpack.core.MessagePack
import org.msgpack.core.MessageUnpacker
import org.msgpack.value.ArrayValue
import org.msgpack.core.MessageFormat
import com.treasuredata.client.model.*
import java.io.InputStream
import java.io.File
import kotlin.String
@GINK03
GINK03 / food2stuff.md
Last active Apr 23, 2017
resnetで料理の材料を当てていく
View food2stuff.md

Resnetで料理の材料をあてていく:food2stuff

モチベーション

  • ResNetを使ってみたい
  • 強い転移学習方法を教わったのでやってみたい
  • img2cal(画像からカロリーを当てるタスク)などに転用できるかもと思った

food2stuffって?

  • stuffって材料って意味らしいです
  • 画像からなんの構成品目でできており、なんの食材でできているかベクトル表現で出力します
View td.txt
date_time varchar date_time
ip varchar ip
request_uri varchar request_uri
ipao97_value varchar ipao97_value
referer varchar referer
useragent varchar useragent
tuuid varchar tuuid
account_id bigint account_id
data_owner_id bigint data_owner_id
os varchar os
View console.sh
$ td job:show 138717728 -f csv -o 138717728.csv --column-header
JobID : 138717728
Status : success
Type : hive
Database : tech_batch
Priority : NORMAL
Retry limit : 0
Output :
Query : SELECT * FROM tech_batch.latest_60days WHERE TD_TIME_RANGE (time, '2017-04-01', null, 'JST') LIMIT 1000000000
CPU time : 3h 13m 33s 930ms
View treasuredata-json-download.md

存在を知らなかったが、jsonでダウンロードできるらしい

まじか

$ td job:show 138717728 -f json -o 138717728.json

こんな感じ

You can’t perform that action at this time.