e9t/README.md

## README.md

      
    Raw
  

              README.md
            
          
    다양한 가격대의 서울시내 스시야를 시각화한 지도.
맛집 이름들이 겹치지 않게 하기 위해 forced layout을 적용했습니다.

데이터 수집: 수작업
Author: Lucy Park
License: Apache v2


## crawl.py
#! /usr/bin/python3
# -*- coding: utf-8 -*-

import json

from lxml import html
import requests

MAPAPI = 'http://openapi.map.naver.com/api/geocode.php?key=2994b9957130f5a19b0aa0165bb9fd1e&encoding=utf-8&coord=LatLng&query=%s'

def get_latlon(query):
    root = html.parse(MAPAPI % query)
    lon, lat = root.xpath('//point/x/text()')[0], root.xpath('//point/y/text()')[0]
    return (lat, lon)

def prep(item):
    n, name = item[0].split(' ', 1)
    lat, lon = get_latlon(item[3])
    return {
        'num': n, 'name': name,
        'lat': lat, 'lon': lon,
        'description': item[1],
        'phone': item[2],
        'addr': item[3]
    }

url = 'http://m.wikitree.co.kr/main/news_view.php?id=217101'
query = '서울시 서대문구 창천동 72-36'

r = requests.get(url)
root = html.document_fromstring(r.text)
string = '\n'.join(root.xpath('//div[@id="ct_size"]/div//text()'))

items = []
for i in range(1, 21):
    tmp = string.split('%s.' % i, 1)
    string = tmp[1]
    items.append([j.strip() for j in tmp[0].split('\n') if j and j!='\xa0'])

data = [prep(i[:4]) for i in items[1:]]

with open('places.json', 'w') as f:
    json.dump(data, f)

with open('places2.csv', 'w') as f:
    f.write('type,name,prince,lat,lon,url\n')
    for d in data:
        f.write('siksin,%(name)s,0,%(lat)s,%(lon)s,\n' % d)

## index.html
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<style>
    svg circle {
      opacity: .5;
      stroke: white;
    }
    svg circle:hover {
      stroke: #333;
    }
    svg .axis line, svg .axis path {
        fill: none;
        stroke: #000;
        shape-rendering: crispEdges;
    }
    svg .axis text {
        font: 10px sans-serif;
    }
    svg .municipality {
      fill: #efefef;
      stroke: #fff;
    }
    svg .municipality-label {
      fill: #bbb;
      font-size: 12px;
      font-weight: 300;
      text-anchor: middle;
    }
    svg #map text {
      color: #333;
      font-size: 10px;
      pointer-events: none;
      text-anchor: middle;
    }
    svg #places text {
      color: #777;
      font: 10px sans-serif;
      text-anchor: start;
    }
    #title {
      position: absolute;
      top: 10px;
      left: 650px;
      width: 300px;
      font-family: sans-serif;
      text-align: right;
    }
    #title p {
      font-size: 10pt;
    }
</style>
</head>
<body>
  <div id="title">
      <h2>서울시 스시야 기행</h2>
      <p><a href="http://redfish.egloos.com">레드피쉬</a>, <a href="http://hsong.egloos.com">녹두장군</a> 등 유명 맛집블로거들이 추천하는 서울 시내 스시집!
      점의 색은 스시야의 가격대를 나타내고, 점을 클릭하면 해당 스시야에 대한 소개글로 이동합니다.
      하나씩 정복해봐요... &gt;_&lt;</p>
      <p>
      <a href="https://gist.github.com/e9t/85fcfb53db389696624f">Code</a>
      by <a href="http://lucypark.kr">Lucy Park</a>.
      <br>
      <a href="http://opensource.org/licenses/Apache-2.0">Licensed with Apache 2.0</a>
      </p>
  </div>
  <div id="chart"></div>
  <script src="http://d3js.org/d3.v3.min.js"></script>
  <script src="http://d3js.org/queue.v1.min.js"></script>
  <script src="http://d3js.org/topojson.v1.min.js"></script>
  <script>
  // set params
  var width = 960,
      height = 500;
  var minColor = 'yellow',
      maxColor = 'red';
  var minValue = 5,
      maxValue = 35; // TODO: automate
  var legendWidth = 15,
      legendHeight = 150,
      margin = { left: 40, top: 30 };

  // define color scale
  var colorScale = d3.scale.linear()
      .range([minColor, maxColor]) // or use hex values
      .domain([minValue, maxValue]);

  // define projection and path
  var projection = d3.geo.mercator()
      .center([126.9895, 37.5651])
      .scale(80000)
      .translate([2*width/5, height/2]);

  var path = d3.geo.path().projection(projection);

  // add canvas
  var svg = d3.select("#chart").append("svg")
      .attr("width", width)
      .attr("height", height);

  var map = svg.append("g").attr("id", "map"),
      points = svg.append("g").attr("id", "places"),
      legend = svg.append("g").attr("id", "legend");

  // add legend for colors
  var legendBar = legend.append("defs").append("linearGradient")
      .attr("id", "gradient")
      .attr("x1", "100%")
      .attr("y1", "0%")
      .attr("x2", "100%")
      .attr("y2", "100%")
      .attr("spreadMethod", "pad");

  legendBar.append("stop")
      .attr("offset", "0%")
      .attr("stop-color", maxColor)
      .attr("stop-opacity", 1);

  legendBar.append("stop")
      .attr("offset", "100%")
      .attr("stop-color", minColor)
      .attr("stop-opacity", 1);

  legend.append("rect")
      .attr("width", legendWidth)
      .attr("height", legendHeight)
      .style("fill", "url(#gradient)")
      .style("opacity", 0.5)
      .attr("transform", "translate(" + margin.left + "," + margin.top + ")");

  var y = d3.scale.linear().range([legendHeight, 0]).domain([minValue, maxValue]);
  var yAxis = d3.svg.axis().scale(y).orient("right");

  legend.append("g")
      .attr("class", "axis")
      .attr("transform", "translate(" + (legendWidth + margin.left) + "," + margin.top + ")")
      .call(yAxis)
    .append("text")
      .attr("transform", "rotate(-90)")
      .attr("y", 30)
      .attr("dy", ".71em")
      .style("text-anchor", "end")
      .text("(단위: 만원)");

  // add map
  d3.json("seoul_municipalities_topo_simple.json", function(error, data) {
    var features = topojson.feature(data, data.objects.seoul_municipalities_geo).features;
    map.selectAll('path')
        .data(features)
      .enter().append('path')
        .attr('class', function(d) { console.log(); return 'municipality c' + d.properties.code })
        .attr('d', path);

    map.selectAll('text')
        .data(features)
      .enter().append("text")
        .attr("transform", function(d) { return "translate(" + path.centroid(d) + ")"; })
        .attr("dy", ".35em")
        .attr("class", "municipality-label")
        .text(function(d) { return d.properties.name; })
  });

  // add circles
  d3.csv("places.csv", function(data) {
    var point = points.selectAll("circle")
        .data(data.filter(function(d) { return d.type==="sushi"; }))
      .enter().append("a")
        .attr("xlink:href", function(d) { return d.url });

    point.append("circle")
        .attr("cx", function(d) { return projection([d.lon, d.lat])[0]; })
        .attr("cy", function(d) { return projection([d.lon, d.lat])[1]; })
        .attr("r", 6)
        .style("fill", function(d) { return colorScale(d.price); });

    // add circle labels
    var labels = [],
        labelLinks = [];

    data.forEach(function(d, i) {
      var p = projection([d.lon, d.lat]);
      var node = {
        label: d.name,
        x: p[0],
        y: p[1]
      };
      labels.push({node : node }); labels.push({node : node }); // push twice
      labelLinks.push({ source : i * 2, target : i * 2 + 1, weight : 1, x: 100 });
    });

    var force = d3.layout.force()
        .nodes(labels)
        .links(labelLinks)
        .gravity(0)
        .linkDistance(0)
        .linkStrength(8)
        .charge(-100)
        .size([width, height])
        .on("tick", tick);

    function tick() {
      circleNode.call(updateNode);
      labelNode.each(function(d, i) {
        if(i % 2 == 0) {
          d.x = d.node.x;
          d.y = d.node.y;
        } else {
          var b = this.childNodes[1].getBBox();
          var diffX = d.x - d.node.x,
              diffY = d.y - d.node.y;
          var dist = Math.sqrt(diffX * diffX + diffY * diffY);
          var shiftX = Math.min(0, b.width * (diffX - dist) / (dist * 2));
          var shiftY = 5;
          this.childNodes[1].setAttribute("transform", "translate(" + shiftX + "," + shiftY + ")");
        }
      });
      labelNode.call(updateNode);
    }

    var circleNode = points.selectAll("circle")
        .data(points)
      .enter().append("circle")
        .attr("class", "node")
        .attr("r", 5)
        .style("fill", "#555")
        .style("stroke-width", 3);

    var labelNode = points.selectAll("g")
        .data(force.nodes())
      .enter().append("g")
        .attr("class", "labelNode");

    labelNode.append("circle")
        .attr("r", 0)
        .style("fill", "red");

    labelNode.append("text")
        .text(function(d, i) { return i % 2 == 0 ? "" : d.node.label })
        .style("fill", "#555")
      var updateNode = function() {
        this.attr("transform", function(d) {
          return "translate(" + d.x + "," + d.y + ")";
        });
      }

    force.start();
  });
  </script>
</body>
</html>

## places.csv

          
            type
            name
            price
            lat
            lon
            url

            
              sushi
              이노시시
              6
              37.562562
              126.926975
              http://hsong.egloos.com/3126336

            
              sushi
              스시효
              8.8
              37.522061
              127.041867
              http://hsong.egloos.com/3402467

            
              sushi
              정준호 스시
              7
              37.487505
              126.88886
              http://hsong.egloos.com/3383473

            
              sushi
              오가와
              6
              37.571923
              126.974303
              http://hsong.egloos.com/3421468

            
              sushi
              스시시로
              5
              37.548968
              126.921155
              http://hsong.egloos.com/3098726

            
              sushi
              스시조
              25
              37.564378
              126.980085
              http://egloos.zum.com/redfish/v/1217180

            
              sushi
              슈치쿠
              23
              37.519465
              126.939833
              http://redfish.egloos.com/1354547

            
              sushi
              스시초희
              22
              37.523725
              127.035956
              http://redfish.egloos.com/1360892

            
              sushi
              스시타쿠
              11
              37.5272764
              127.0352359
              http://redfish.egloos.com/1348840

            
              sushi
              아리아께
              22
              37.556978
              127.005914
              http://redfish.egloos.com/1397479

            
              sushi
              스시하꼬
              9
              37.4897
              126.993889
              http://redfish.egloos.com/1313557

            
              sushi
              스시타츠
              22
              37.518042
              127.028216
              http://redfish.egloos.com/1393625

            
              sushi
              스시코우지
              18
              37.522897
              127.039834
              http://redfish.egloos.com/1364412

            
              sushi
              코지마
              35
              37.5257419
              127.0419814
              http://blog.naver.com/mardukas/220178511890

            
              sushi
              기꾸
              4.5
              37.517925
              126.982401
              http://egloos.zum.com/hsong/v/3317958

            
              sushi
              스시선수
              20
              37.522832
              127.036421
              http://redfish.egloos.com/1375236

            
              sushi
              스시산원
              14
              37.5064703
              127.0509131
              http://redfish.egloos.com/1379800

            
              sushi
              미치루
              8
              37.519318
              126.93142
              http://egloos.zum.com/redfish/v/1357195

            
              sushi
              가네끼스시
              4
              37.491172
              126.9249803
              http://redfish.egloos.com/1368486

            
              sushi
              려
              4
              37.4817613
              126.9514527
              http://redfish.egloos.com/1406658

## seoul_municipalities_topo_simple.json

      
Display the source blob

    
Display the rendered blob

    
    Raw
  

              seoul_municipalities_topo_simple.json
            
          
      Loading

      Sorry, something went wrong. Reload?
      Sorry, we cannot display this file.
      Sorry, this file is invalid so it cannot be displayed.
      
          Viewer requires iframe.
      
    
## thumbnail.png

      
    Raw
  

              thumbnail.png
	#! /usr/bin/python3
	# -- coding: utf-8 --

	import json

	from lxml import html
	import requests

	MAPAPI = 'http://openapi.map.naver.com/api/geocode.php?key=2994b9957130f5a19b0aa0165bb9fd1e&encoding=utf-8&coord=LatLng&query=%s'

	def get_latlon(query):
	root = html.parse(MAPAPI % query)
	lon, lat = root.xpath('//point/x/text()')[0], root.xpath('//point/y/text()')[0]
	return (lat, lon)

	def prep(item):
	n, name = item[0].split(' ', 1)
	lat, lon = get_latlon(item[3])
	return {
	'num': n, 'name': name,
	'lat': lat, 'lon': lon,
	'description': item[1],
	'phone': item[2],
	'addr': item[3]
	}

	url = 'http://m.wikitree.co.kr/main/news_view.php?id=217101'
	query = '서울시 서대문구 창천동 72-36'

	r = requests.get(url)
	root = html.document_fromstring(r.text)
	string = '\n'.join(root.xpath('//div[@id="ct_size"]/div//text()'))

	items = []
	for i in range(1, 21):
	tmp = string.split('%s.' % i, 1)
	string = tmp[1]
	items.append([j.strip() for j in tmp[0].split('\n') if j and j!='\xa0'])

	data = [prep(i[:4]) for i in items[1:]]

	with open('places.json', 'w') as f:
	json.dump(data, f)

	with open('places2.csv', 'w') as f:
	f.write('type,name,prince,lat,lon,url\n')
	for d in data:
	f.write('siksin,%(name)s,0,%(lat)s,%(lon)s,\n' % d)
	<!DOCTYPE html>
	<html>
	<head>
	<meta charset="utf-8">
	<style>
	svg circle {
	opacity: .5;
	stroke: white;
	}
	svg circle:hover {
	stroke: #333;
	}
	svg .axis line, svg .axis path {
	fill: none;
	stroke: #000;
	shape-rendering: crispEdges;
	}
	svg .axis text {
	font: 10px sans-serif;
	}
	svg .municipality {
	fill: #efefef;
	stroke: #fff;
	}
	svg .municipality-label {
	fill: #bbb;
	font-size: 12px;
	font-weight: 300;
	text-anchor: middle;
	}
	svg #map text {
	color: #333;
	font-size: 10px;
	pointer-events: none;
	text-anchor: middle;
	}
	svg #places text {
	color: #777;
	font: 10px sans-serif;
	text-anchor: start;
	}
	#title {
	position: absolute;
	top: 10px;
	left: 650px;
	width: 300px;
	font-family: sans-serif;
	text-align: right;
	}
	#title p {
	font-size: 10pt;
	}
	</style>
	</head>
	<body>
	<div id="title">
	<h2>서울시 스시야 기행</h2>
	<p><a href="http://redfish.egloos.com">레드피쉬</a>, <a href="http://hsong.egloos.com">녹두장군</a> 등 유명 맛집블로거들이 추천하는 서울 시내 스시집!
	점의 색은 스시야의 가격대를 나타내고, 점을 클릭하면 해당 스시야에 대한 소개글로 이동합니다.
	하나씩 정복해봐요... >_<</p>
	<p>
	<a href="https://gist.github.com/e9t/85fcfb53db389696624f">Code</a>
	by <a href="http://lucypark.kr">Lucy Park</a>.
	<br>
	<a href="http://opensource.org/licenses/Apache-2.0">Licensed with Apache 2.0</a>
	</p>
	</div>
	<div id="chart"></div>
	<script src="http://d3js.org/d3.v3.min.js"></script>
	<script src="http://d3js.org/queue.v1.min.js"></script>
	<script src="http://d3js.org/topojson.v1.min.js"></script>
	<script>
	// set params
	var width = 960,
	height = 500;
	var minColor = 'yellow',
	maxColor = 'red';
	var minValue = 5,
	maxValue = 35; // TODO: automate
	var legendWidth = 15,
	legendHeight = 150,
	margin = { left: 40, top: 30 };

	// define color scale
	var colorScale = d3.scale.linear()
	.range([minColor, maxColor]) // or use hex values
	.domain([minValue, maxValue]);

	// define projection and path
	var projection = d3.geo.mercator()
	.center([126.9895, 37.5651])
	.scale(80000)
	.translate([2*width/5, height/2]);

	var path = d3.geo.path().projection(projection);

	// add canvas
	var svg = d3.select("#chart").append("svg")
	.attr("width", width)
	.attr("height", height);

	var map = svg.append("g").attr("id", "map"),
	points = svg.append("g").attr("id", "places"),
	legend = svg.append("g").attr("id", "legend");

	// add legend for colors
	var legendBar = legend.append("defs").append("linearGradient")
	.attr("id", "gradient")
	.attr("x1", "100%")
	.attr("y1", "0%")
	.attr("x2", "100%")
	.attr("y2", "100%")
	.attr("spreadMethod", "pad");

	legendBar.append("stop")
	.attr("offset", "0%")
	.attr("stop-color", maxColor)
	.attr("stop-opacity", 1);

	legendBar.append("stop")
	.attr("offset", "100%")
	.attr("stop-color", minColor)
	.attr("stop-opacity", 1);

	legend.append("rect")
	.attr("width", legendWidth)
	.attr("height", legendHeight)
	.style("fill", "url(#gradient)")
	.style("opacity", 0.5)
	.attr("transform", "translate(" + margin.left + "," + margin.top + ")");

	var y = d3.scale.linear().range([legendHeight, 0]).domain([minValue, maxValue]);
	var yAxis = d3.svg.axis().scale(y).orient("right");

	legend.append("g")
	.attr("class", "axis")
	.attr("transform", "translate(" + (legendWidth + margin.left) + "," + margin.top + ")")
	.call(yAxis)
	.append("text")
	.attr("transform", "rotate(-90)")
	.attr("y", 30)
	.attr("dy", ".71em")
	.style("text-anchor", "end")
	.text("(단위: 만원)");

	// add map
	d3.json("seoul_municipalities_topo_simple.json", function(error, data) {
	var features = topojson.feature(data, data.objects.seoul_municipalities_geo).features;
	map.selectAll('path')
	.data(features)
	.enter().append('path')
	.attr('class', function(d) { console.log(); return 'municipality c' + d.properties.code })
	.attr('d', path);

	map.selectAll('text')
	.data(features)
	.enter().append("text")
	.attr("transform", function(d) { return "translate(" + path.centroid(d) + ")"; })
	.attr("dy", ".35em")
	.attr("class", "municipality-label")
	.text(function(d) { return d.properties.name; })
	});

	// add circles
	d3.csv("places.csv", function(data) {
	var point = points.selectAll("circle")
	.data(data.filter(function(d) { return d.type==="sushi"; }))
	.enter().append("a")
	.attr("xlink:href", function(d) { return d.url });

	point.append("circle")
	.attr("cx", function(d) { return projection([d.lon, d.lat])[0]; })
	.attr("cy", function(d) { return projection([d.lon, d.lat])[1]; })
	.attr("r", 6)
	.style("fill", function(d) { return colorScale(d.price); });

	// add circle labels
	var labels = [],
	labelLinks = [];

	data.forEach(function(d, i) {
	var p = projection([d.lon, d.lat]);
	var node = {
	label: d.name,
	x: p[0],
	y: p[1]
	};
	labels.push({node : node }); labels.push({node : node }); // push twice
	labelLinks.push({ source : i * 2, target : i * 2 + 1, weight : 1, x: 100 });
	});

	var force = d3.layout.force()
	.nodes(labels)
	.links(labelLinks)
	.gravity(0)
	.linkDistance(0)
	.linkStrength(8)
	.charge(-100)
	.size([width, height])
	.on("tick", tick);

	function tick() {
	circleNode.call(updateNode);
	labelNode.each(function(d, i) {
	if(i % 2 == 0) {
	d.x = d.node.x;
	d.y = d.node.y;
	} else {
	var b = this.childNodes[1].getBBox();
	var diffX = d.x - d.node.x,
	diffY = d.y - d.node.y;
	var dist = Math.sqrt(diffX * diffX + diffY * diffY);
	var shiftX = Math.min(0, b.width * (diffX - dist) / (dist * 2));
	var shiftY = 5;
	this.childNodes[1].setAttribute("transform", "translate(" + shiftX + "," + shiftY + ")");
	}
	});
	labelNode.call(updateNode);
	}

	var circleNode = points.selectAll("circle")
	.data(points)
	.enter().append("circle")
	.attr("class", "node")
	.attr("r", 5)
	.style("fill", "#555")
	.style("stroke-width", 3);

	var labelNode = points.selectAll("g")
	.data(force.nodes())
	.enter().append("g")
	.attr("class", "labelNode");

	labelNode.append("circle")
	.attr("r", 0)
	.style("fill", "red");

	labelNode.append("text")
	.text(function(d, i) { return i % 2 == 0 ? "" : d.node.label })
	.style("fill", "#555")
	var updateNode = function() {
	this.attr("transform", function(d) {
	return "translate(" + d.x + "," + d.y + ")";
	});
	}

	force.start();
	});
	</script>
	</body>
	</html>
type	name	price	lat	lon	url
sushi	이노시시	6	37.562562	126.926975	http://hsong.egloos.com/3126336
sushi	스시효	8.8	37.522061	127.041867	http://hsong.egloos.com/3402467
sushi	정준호 스시	7	37.487505	126.88886	http://hsong.egloos.com/3383473
sushi	오가와	6	37.571923	126.974303	http://hsong.egloos.com/3421468
sushi	스시시로	5	37.548968	126.921155	http://hsong.egloos.com/3098726
sushi	스시조	25	37.564378	126.980085	http://egloos.zum.com/redfish/v/1217180
sushi	슈치쿠	23	37.519465	126.939833	http://redfish.egloos.com/1354547
sushi	스시초희	22	37.523725	127.035956	http://redfish.egloos.com/1360892
sushi	스시타쿠	11	37.5272764	127.0352359	http://redfish.egloos.com/1348840
sushi	아리아께	22	37.556978	127.005914	http://redfish.egloos.com/1397479
sushi	스시하꼬	9	37.4897	126.993889	http://redfish.egloos.com/1313557
sushi	스시타츠	22	37.518042	127.028216	http://redfish.egloos.com/1393625
sushi	스시코우지	18	37.522897	127.039834	http://redfish.egloos.com/1364412
sushi	코지마	35	37.5257419	127.0419814	http://blog.naver.com/mardukas/220178511890
sushi	기꾸	4.5	37.517925	126.982401	http://egloos.zum.com/hsong/v/3317958
sushi	스시선수	20	37.522832	127.036421	http://redfish.egloos.com/1375236
sushi	스시산원	14	37.5064703	127.0509131	http://redfish.egloos.com/1379800
sushi	미치루	8	37.519318	126.93142	http://egloos.zum.com/redfish/v/1357195
sushi	가네끼스시	4	37.491172	126.9249803	http://redfish.egloos.com/1368486
sushi	려	4	37.4817613	126.9514527	http://redfish.egloos.com/1406658