Skip to content

Instantly share code, notes, and snippets.

goungoun /
Last active Jun 21, 2019
vertex에 없는 edge 빼기
import pandas as pd
vtx=pd.read_csv("vtx.tsv", sep=" ")
edge=pd.read_csv("edge.tsv", sep=" ")
edge = edge[edge[':START_ID'].isin(vtx["id:ID"])]
edge = edge[edge[':END_ID'].isin(vtx["id:ID"])]
edge.to_csv("edge2.tsv", sep=" ", index=False, header = True)
goungoun / JSONSample.scala
Created Oct 17, 2018 — forked from takezoe/JSONSample.scala
Example of scala.util.parsing.json
View JSONSample.scala
import scala.util.parsing.json._
val result = JSON.parseFull("""
{"name": "Naoki", "lang": ["Java", "Scala"]}
result match {
case Some(e) => println(e) // => Map(name -> Naoki, lang -> List(Java, Scala))
case None => println("Failed.")
View gist:4bf71b861b4a195200bccb04b3a8b989
<h3> <p [ngStyle]="{backgroundColor:getColor()}" [ngClass]="{online: 'online'==='online'}">Oozie Workflow Maker </p></h3>
View Qwik-lab
sudo apt-get update
sudo apt-get --fix-missing install python-mpltoolkits.basemap python-numpy python-matplotlib
# gcloud beta pubsub topics create sanidego
# gcloud beta pubsub topics publish sandiego "hello"
from import pubsub
client = pubsub.Client()
topic = client.topic("sandiego")
View GCP datalab
datalab create mydatalabvm --zone us-central1-b
View gist:f49795a11755a5aa20226e5b822317ec
Compute Engine:
Cloud Launcher:
Pricing Philosophy:
View gist:e290cd8b63eba7955006e12fdd346a76
gcloud dataproc clusters create <NAME-OF-YOUR-CLUSTER> --subnet default --zone us-central1-b --master-machine-type n1-standard-2 --master-boot-disk-size 500 --num-workers 2 --worker-machine-type n1-standard-2 --worker-boot-disk-size 500 --project <YOUR-PROJECT-ID>
View big query example.sql
# It is a kind of Column Store
# Avoid using * (star) to return all columns, instead use preview
# Check the amount of the processing size by changing query) (500 MB, 1T..)
# Always with LIMIT
# format converts number into string (cannot add)
# cannot use aliased column in where clause like income
# StandardSQL or legacySQL
goungoun /
Created Apr 10, 2018
GCP with Embulk

아래 자료는 황장군님의 강의자료를 GCP에서 테스트한 결과입니다.


embulk를 리눅스에 설치해보자. jar를 copy 하면 됨

curl --create-dirs -o ~/.embulk/bin/embulk -L ""