Skip to content

Instantly share code, notes, and snippets.

@sudodo
sudodo / python_parallel_processing.py
Created September 23, 2016 03:09
Python parallel processing sample
def map_f(w, model):
return model.get_vector(w)
def get_vector_matrix_for_w2v_model(model, vocab=None):
"""Create vector matrix for word2vec model"""
if vocab is None:
vocab = model.vocab
# vect_mat = np.array([model.get_vector(w) for w in vocab])
p = Pool(multiprocessing.cpu_count() - 1)
@sudodo
sudodo / remove_nonsense_words.py
Last active September 23, 2016 03:04
Detect if string is non sense word.
import mojimoji
def is_non_sense_word(word):
"""return True if non sense word like a@~ """
hankaku_word = mojimoji.zen_to_han(word.decode("utf-8"), kana=False)
#半角記号を含む語は削除
pattern = re.compile("[!-/:-@[-`{-~]")
if pattern.search(hankaku_word) is not None:
return True
@sudodo
sudodo / not_list_comprehension_with_multithreading.py
Created March 11, 2015 03:00
Python 2.x (not) list comprehension with multithreading
import multiprocessing
import itertools
def get_trajectory(arg1, arg1):
return [999,888,777,666]
def get_traj_as_star(arg):
return get_trajectory(arg[1], arg[2])
def run_multi_particle(num_particle, num_time, num_mesh, multi_thread=True):
@sudodo
sudodo / add_line.sh
Created March 10, 2015 00:52
Add a line at the begging of large file
#! /bin/bash
echo "index,datetime,count" > newfile.csv
cat original_file.csv >> newfile.csv
@sudodo
sudodo / remove_symbol.rb
Last active August 29, 2015 14:04
Remove symbol
# encoding: utf-8
def remove_symbol str
wide_symbol = "、 。 , . ・ : ; ? ! ゛ ゜ ´ ` ¨ ^  ̄ _ ヽ ヾ ゝ ゞ 〃 仝 々 〆 〇 ー ― ‐ / \ ~ ∥ | … ‥ ‘ ’ “ ” ( ) 〔 〕 [ ] { } 〈 〉 《 》 「 」 『 』 【 】 + - ± × ÷ = ≠ < > ≦ ≧ ∞ ∴ ♂ ♀ ° ′ ″ ℃ ¥ $ ¢ £ % # & * @ § ☆ ★ ○ ● ◎ ◇ ◆ □ ■ △ ▲ ▽ ▼ ※ 〒 → ← ↑ ↓ 〓"
str.gsub!(/[!-\/:-@\[-`{-~]*/,"") # => remove word consisting of only numeric or special chars like 123, `"$#
str.gsub!(/[#{wide_symbol}]/,"")
# str.gsub!(/[!。、*・]/,"")
str.gsub!(/\p{Symbol}/,"")
str
end
@sudodo
sudodo / rspec_macher_system_call.rb
Created March 2, 2012 04:12
rspec macher for system comamnd call.
RSpec::Matchers.define :call_system_command do |expected|
match do |actual|
actual.should_receive(:'`').with(/#{expected}/) #|| actual.should_receive(:system).with(expected)
end
end
@sudodo
sudodo / show_erb_gen.rake
Created February 22, 2012 04:27
Rake task to print a table for show.html.erb' in a Rails app.
#encoding: utf-8
require 'rake'
task :show_table_gen, :arg1 do |t, arg|
# TODO eval(arg[]) is NOT secure code.
model_name = eval(arg['arg1']) # => NOT secure code...
str = <<-EOS
<table>
EOS
@sudodo
sudodo / act.like.vim.tmux.conf
Created February 9, 2012 05:26
.tmux.conf act like vim
# act like vim
setw -g mode-keys vi
bind h select-pane -L
bind j select-pane -D
bind k select-pane -U
bind l select-pane -R
bind-key -r C-h select-window -t :-
bind-key -r C-l select-window -t :+
@sudodo
sudodo / how_to_install_tmux_from_source_jp
Created February 3, 2012 05:20
tmuxのソースからのインストール方法
1. libeventの導入
安定版のソース(自分は2.0.16)
http://libevent.org/
手順はここを参照
https://github.com/libevent/libevent
普通に
./configure -> make -> sudo make install
でできます。
@sudodo
sudodo / the_hacker_way_jp
Created February 2, 2012 04:01
The Hacker Way 日本語翻訳版
The Hacker Way
ハッカーのいきかた
原文
The Hacker Way (in Facebook' S-1 Filling.)
http://www.sec.gov/Archives/edgar/data/1326801/000119312512034517/d287954ds1.htm#toc
強い企業にする一環として、世界に大きなインパクトを与えようとし、また他のすばらしい人々から学ぼうとするような偉大な人々にとって最高の場所になるように努めてきました。私たちはHacker Wayと呼ぶユニークなカルチャーやマネジメント手法を構築してきました。