yurukov

wget -O stops.json "https://varnatraffic.com/Ajax/GetStations" 
echo -n '{"type":"FeatureCollection","features":' > stops.geojson
sed 's_{"id"_\n{"type":"Feature","properties":{"id"_g;s_"text"_"name"_g;s_,"position":{"lat":_},"geometry":{"type":"Point","coordinates":[_g;s_,"lon":_,_g;s_}}_]}}_g' stops.json | sed 's_\[\([0-9.]\+\),\([0-9.]\+\)\]_[\2,\1]_g' >> stops.geojson
echo '}' >> stops.geojson
wget -O site https://varnatraffic.com
grep -o '/bg/Line/Routes/[^"]\+' site  | sed 's_.*/__'| sort -un > linenumbers
for i in `cat linenumbers`; do echo -en $i"\t"; wget -q -O - "https://varnatraffic.com/Ajax/GetLineState?line=$i&direction=0" | grep -o '"id":[0-9]\+,"text"'  | sed 's_"id":\|,"text"__g' | tr '\n' ',' | sed 's_,$__'; echo ""; done > lines.tsv

yurukov

wget -O plovdiv.html "http://transport.plovdiv.bg/desktop/"
grep -o '"number"[^}]\+' plovdiv.html | sed 's_"number":__;s_"\?,"\(name\|lat\|lng\)":"\?_\t_g;s_'"'"'_"_g;s_\\\\__g' | sort -n > stops.tsv
grep 'zul.sel.Listcell' plovdiv.html | grep -v "label:'Линия" | sed "s_\['zul[^{]*{__" | tr '\n' '|' | sed 's_,\[\]\],|__g' | tr '|' '\n' | sed "s_'}label.*'}label:'_\t_;s_'}label:.*_\t_;s_label:'[0-9]*\|'},\[\]\]\]\].*\| __g" > lines.tsv
echo -e "id\tname\tlat\tlng\tlines" > plovdiv.tsv
paste stops.tsv lines.tsv | sed 's_\t\t_\t_;s_"_""_g' | awk -F "\t" '{print $1"\t\""$2"\"\t"$3"\t"$4"\t"$5}'  >> plovdiv.tsv

yurukov

wget https://www.sofiatraffic.bg/interactivecard/lines/1 -O a1
wget https://www.sofiatraffic.bg/interactivecard/lines/2 -O a2
wget https://www.sofiatraffic.bg/interactivecard/lines/3 -O a3
grep -h -o 'for="line[0-9]\+">[^<]\+' a* | sed 's_.*line__;s_">_\t_' > lines
mkdir l
for i in `sed 's_\t.*__' lines|sort -n`; do wget -q -O "l/$i.json" "https://www.sofiatraffic.bg/interactivecard/lines/stops/geo?line_id=$i"; echo -n "."; done
for i in l/*; do echo -e `cat $i` | sed 's_{"geometry":_\n{"geometry":_g' > temp; mv temp $i; done
echo '{"type": "FeatureCollection", "features": [' > stops.geojson
grep -h geometry l/* | sed 's_}}]}_}},_' >> stops.geojson
sed -i '$ s_}},_}}]}_' stops.geojson

yurukov

This is a small util for marking neighboring municipalities.

Dependencies:
jquery, leaflet v101

Data:
https://github.com/yurukov/Bulgaria-geocoding/blob/master/municipalities.geojson

When a municipality is marked as red, you need to click on all blue municialities
which are its neighbors. when that's done, you click on the red municipality itself

yurukov

function download(filename, text) {
  var element = document.createElement('a');
  element.setAttribute('href', 'data:text/plain;charset=utf-8,' + encodeURIComponent(text));
  element.setAttribute('download', filename);
  element.style.display = 'none';
  document.body.appendChild(element);
  element.click();
  document.body.removeChild(element);
}

yurukov

yurukov

SELECT a.timest,a.level, a.isinvalid, (select round(avg(a1.level)) from air_sofia a1 where a1.station=a.station and a1.param=0 and a1.isinvalid=0 and a1.timest>=a.timest - interval 12 hour and a1.timest<a.timest + interval 12 hour) level24,(select count(a2.level) from air_sofia a2 where a2.station=a.station and a2.param=0 and a2.isinvalid=0 and a2.timest>=a.timest - interval 12 hour and a2.timest<a.timest + interval 12 hour) validcount FROM air_sofia a where param=0 and station=1

yurukov

1. Open http://registry.plovdiv.bg/eDogs/default.aspx
2. Select 50 entries per page
3. Copy JS in console and run
4. Back up the downloaded data file
5. Extract all addresses:
  awk -F '\t' '{print $4}' plddogs.tsv | grep -v address | sed 's_,\? \?\(ет\|ап\)\.\? \?[0-9]\+ \?__g;s_ \?№ \?_ _' |sort -u > addr
6. Geotag the addresses:
  php geotag.php addr > addrg
7. Merge geotagged addresses with scaraped data and exclude columns:
  for i in `sed 's_\t_|_g;s_ _\__g' plddogs.tsv | grep -v address`; 

yurukov

I updated my permalink structure from .../Y/m/d/title to .../Y/title. 
Naturaly the share button count was set to 0 due to the new URL. My 
solution was to use the old URL structure for the share buttons only 
for posts before the update. The downside of this is that old posts 
will be shared with the old url structure. Thus when shared, there'll
be another 301. Upside - share count is kept. 

Also, I wanted to optimize the scripts written to the footer by the
share module of jetpack. The registered js file includes all the
scripts for opening the share dialogs in a new window. These are the

yurukov

These are a few commands that could be used to scrape a full group page
from Facebook. One can use the Graph API, but there some users would be
hidden. The JS commands should be run in a browser and scroll through
the page opening up hidden content and comments. I used Chrome. Once
enough content is opened, you should save the page as any other and
analyse it's contents.