Skip to content

Instantly share code, notes, and snippets.

@fonylew
Last active May 12, 2016 11:07
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save fonylew/38e34ae808f7ca65272e1c31d987f422 to your computer and use it in GitHub Desktop.
Save fonylew/38e34ae808f7ca65272e1c31d987f422 to your computer and use it in GitHub Desktop.
# resolve apt-get problem
# remove last two lines from sources.list
sudo head -n -2 /etc/apt/sources.list
# add these two line to .bashrc to config locale
sudo echo "export LC_ALL=en_US.utf8" >>~/.bashrc
sudo echo "export LANGUAGE=en_US.utf8" >>~/.bashrc
# install necessary packages
sudo apt-get install screen vim
# change permission of Docker Socket
sudo chmod 777 /var/run/docker.sock
git clone https://github.com/fonylew/json-pantipdata.git
cd json-pantipdata
# Download from Google Storage
for i in {3159..3279}
gsutil cp gs://pantip-data-source-mapdb/json/data-$i.json .
python cuttofile.py
# enter start and end folder number
python batchpantip.py
# enter start and end folder number
cd ..
mkdir data-copy
cp json-pantipdata/*.csv data-copy
cp json-pantipdata/*.cypher data-copy
git clone https://github.com/fonylew/rneo4j-centrality.git
git clone https://github.com/fonylew/webpantip.git
# Neo4j Database
docker run -it -p 7474:7474 -v "data-copy":/opt/data-copy --name graphdb kbastani/docker-neo4j:2.2.1 /bin/bash
# Rstudio Server
docker run -d --name rstudio -p 8787:8787 –v "rneo4j-centrality":/home/rstudio rocker/rstudio
# Apache HTTPD with PHP
docker run -d -it -p 80:80 --name php -v "webpantip":/var/www/html php
# Import Database
screen
docker exec -it graphdb /bin/bash
cd ~/neo4j
bin/neo4j-shell -path data/graph.db -file /opt/data-copy/index.cypher
bin/neo4j-shell -path data/graph.db -file /opt/data-copy/import.cypher
cp -r data/graph.db /opt/data
exit
exit
# Install required package
docker exec -it rstudio /usr/bin/R
install.packages("RNeo4j")
install.packages("igraph")
install.packages("jsonlite")
install.packages("devtools")
install.packages("plumber")
q()
# Run centrality.R to find centrality in background
screen
docker exec -it rstudio /bin/bash
Rscript /home/rstudio/centrality.R
exit
exit
# Run runapi.R for serving API in background
screen
docker exec -it rstudio /bin/bash
Rscript /home/rstudio/runapi.R
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment