Skip to content

Instantly share code, notes, and snippets.

Avatar

Jungsub Lim jsrimr

  • SNU
  • Seoul, Dongjak-gu
View GitHub Profile
@jsrimr
jsrimr / mnist_DDP.py
Created Feb 5, 2021
pytorch mnist train with multi-gpu using DataParallel
View mnist_DDP.py
from __future__ import print_function
import argparse
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torchvision import datasets, transforms
from torch.optim.lr_scheduler import StepLR
@jsrimr
jsrimr / test_tpu_multiproc.py
Created Feb 5, 2021
MNIST train code using pytorch, TPU(GCP)
View test_tpu_multiproc.py
"""
most code from
https://colab.research.google.com/github/pytorch/xla/blob/master/contrib/colab/mnist-training.ipynb#scrollTo=pTmxZL5ymp8P
"""
import math
from matplotlib import pyplot as plt
M, N = 4, 6
RESULT_IMG_PATH = '/tmp/test_result.png'
@jsrimr
jsrimr / memory_in_multiprocessing.py
Created Oct 20, 2020
multiprocess시에 pipe를 만들면 프로세스들은 각자 pipe에 대고 listen, speak할 수 있다. 한쪽의 process가 pipe의 listen쪽이나 speak쪽을 닫더라도 다른 process는 여전히 이 파이프를 통해 listen, speak 할 수 있다
View memory_in_multiprocessing.py
from multiprocessing import Process, Pipe
import time
remotes, work_remotes = zip(*[Pipe() for _ in range(10)])
def worker(remote, parent_remote):
parent_remote.close()
print(f"closed {id(parent_remote)}, {time.time()}")
# print(f"In worker : {id(remotes[0])} is closed. {remotes[0].closed}")
@jsrimr
jsrimr / 중복제거.sql
Created Sep 28, 2020
db 중복 row 제거 쿼리
View 중복제거.sql
DELETE
FROM log_by_hour
WHERE id IN
(
SELECT id FROM (SELECT max(id) FROM log_by_hour where time BETWEEN FROM_UNIXTIME(1598927371) AND FROM_UNIXTIME(1598956231) GROUP BY instance_id, cmd_id, time, min, max, avg HAVING count(*) > 1) temp_table
)
and time BETWEEN FROM_UNIXTIME(1598927371) AND FROM_UNIXTIME(1598956231)
;
DELETE
@jsrimr
jsrimr / 2types_binSearch.py
Created Sep 10, 2020
binary serach 2가지 유형. Exact match and close match. close match 가 필요한 경우 : https://programmers.co.kr/learn/courses/30/lessons/64062
View 2types_binSearch.py
def find_closest():
"""
max_consecutive_cnt == k 인 경우가 여러개 있을 수 있다. 이 중 stone 을 가장 작게 하는 경우를 찾는다.
일단, max_consecutive_cnt >= k 를 만족하긴 해야한다. 여기를 =mid 로 잡고
나머지 한쪽을 mid-1 이나 mid+1 해서 실험하는 용도로 사용한다.
left 는 항상 k에 못미달하는 구간으로(실패) 업데이트시 max_consecutive_cnt < k 일때 left = mid+1
right 는 항상 k에 도달하는 구간으로(실패) max_consecutive_cnt >= k 에서 업데이트시 right = mid
"""
@jsrimr
jsrimr / executeSqlFile.py
Created Aug 12, 2020
executeSqlFile by python
View executeSqlFile.py
import pymysql
from settings import db_host, db, db_user, db_password
def executeScriptsFromFile(filename):
# Open and read the file as a single buffer
fd = open(filename, 'r')
sqlFile = fd.read()
fd.close()
# all SQL commands (split on ';')
sqlCommands = sqlFile.split(';')
@jsrimr
jsrimr / celery_run.sh
Created Aug 6, 2020
run python module celery with env variables and virtualenv
View celery_run.sh
set -a
source ../.env.local
set +a
/Users/jungsublim/.pyenv/versions/anaconda3-5.1.0/envs/cloaweb/bin/celery -A conf.celery worker -l info -B
@jsrimr
jsrimr / custom.ini
Created Jul 31, 2020
외부에서 iframe 으로 grafana view 할 수 있도록 하는 설정. 모든 권한을 다 할 수 있는 것 같아 조정이 필요해보임. 설정파일 바꾸고 docker restart [grafana_container_name] 하면 됨
View custom.ini
##################### Grafana Configuration Example #####################
#
# Everything has defaults so you only need to uncomment things you want to
# change
# possible values : production, development
;app_mode = production
# instance name, defaults to HOSTNAME environment variable value or hostname if HOSTNAME var is empty
;instance_name = ${HOSTNAME}
View wordBreak2.py
class Trie:
head = {}
def add(self, word):
cur = self.head
for ch in word:
if ch not in cur:
cur[ch] = {}
cur = cur[ch]
# *가 있을 경우, 그 단어가 자료구조에 저장되어 있음을 의미
# *가 없을 경우, 그 단어는 자료구조에 저장되어 있지 않음, 단지 더 긴 단어의 부분으로만 존재함
@jsrimr
jsrimr / Trie.py
Last active Jul 30, 2020
트라이구현
View Trie.py
class Trie:
def __init__(self):
self.head = {}
def add(self, word):
cur = self.head
for ch in word:
if ch not in cur:
cur[ch] = {}
cur = cur[ch]