Skip to content

Instantly share code, notes, and snippets.

Important notes:
1. Please run this command only during the night when load on the server is low
2. This loads all index files and documents from the opendata portal
3. The script saves your last download time and downloads only those files that are marked as updates after
4. Expected download size should be about 6G
Steps:
1. Open https://kais.cadastre.bg/bg/OpenData
2. Open DevTools
3. Find the second Fetch/XHR call to https://kais.cadastre.bg/bg/OpenData/Read (the one with payload)
diff --git a/index.php b/index.php
index 95c862d..9ba3741 100644
--- a/index.php
+++ b/index.php
@@ -98,6 +98,7 @@
case "nodeinfo/2.1":
nodeinfo(); // Optional. Static.
case rawurldecode( $username ):
+ case "users/" . rawurldecode( $username ):
case "@" . rawurldecode( $username ): // Some software assumes usernames start with an `@`
@yurukov
yurukov / dianabad.geojson
Created January 15, 2024 22:26
Geojson formats for extruded buildings
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@yurukov
yurukov / README.md
Last active February 11, 2023 13:35
Transforms EPSG8122 to WGS84, reduces the precision and simplifies the structure if size is too big

Requires proj4 and simplify-geojson. Receives coordinates in EPSG:8122 projection as regularly used in Bulgarian arcgeo government servers. Transforms to geojson format with reduced precision of less than a meter. Then if the size of the resulting file is bigger than 20k characters, it simplifies it to reduce load on processing later on.

@yurukov
yurukov / bash commands
Last active March 19, 2017 16:54
Данни за спирките и линиите на градския транспорт във Варна
wget -O stops.json "https://varnatraffic.com/Ajax/GetStations"
echo -n '{"type":"FeatureCollection","features":' > stops.geojson
sed 's_{"id"_\n{"type":"Feature","properties":{"id"_g;s_"text"_"name"_g;s_,"position":{"lat":_},"geometry":{"type":"Point","coordinates":[_g;s_,"lon":_,_g;s_}}_]}}_g' stops.json | sed 's_\[\([0-9.]\+\),\([0-9.]\+\)\]_[\2,\1]_g' >> stops.geojson
echo '}' >> stops.geojson
wget -O site https://varnatraffic.com
grep -o '/bg/Line/Routes/[^"]\+' site | sed 's_.*/__'| sort -un > linenumbers
for i in `cat linenumbers`; do echo -en $i"\t"; wget -q -O - "https://varnatraffic.com/Ajax/GetLineState?line=$i&direction=0" | grep -o '"id":[0-9]\+,"text"' | sed 's_"id":\|,"text"__g' | tr '\n' ',' | sed 's_,$__'; echo ""; done > lines.tsv
@yurukov
yurukov / bash commands
Last active April 7, 2022 12:24
Данни за спирките на градския транспорт в Пловдив
wget -O plovdiv.html "http://transport.plovdiv.bg/desktop/"
grep -o '"number"[^}]\+' plovdiv.html | sed 's_"number":__;s_"\?,"\(name\|lat\|lng\)":"\?_\t_g;s_'"'"'_"_g;s_\\\\__g' | sort -n > stops.tsv
grep 'zul.sel.Listcell' plovdiv.html | grep -v "label:'Линия" | sed "s_\['zul[^{]*{__" | tr '\n' '|' | sed 's_,\[\]\],|__g' | tr '|' '\n' | sed "s_'}label.*'}label:'_\t_;s_'}label:.*_\t_;s_label:'[0-9]*\|'},\[\]\]\]\].*\| __g" > lines.tsv
echo -e "id\tname\tlat\tlng\tlines" > plovdiv.tsv
paste stops.tsv lines.tsv | sed 's_\t\t_\t_;s_"_""_g' | awk -F "\t" '{print $1"\t\""$2"\"\t"$3"\t"$4"\t"$5}' >> plovdiv.tsv
@yurukov
yurukov / bash commands
Last active March 7, 2018 13:04
Данни за спирките и линиите на градския транспорт в София
wget https://www.sofiatraffic.bg/interactivecard/lines/1 -O a1
wget https://www.sofiatraffic.bg/interactivecard/lines/2 -O a2
wget https://www.sofiatraffic.bg/interactivecard/lines/3 -O a3
grep -h -o 'for="line[0-9]\+">[^<]\+' a* | sed 's_.*line__;s_">_\t_' > lines
mkdir l
for i in `sed 's_\t.*__' lines|sort -n`; do wget -q -O "l/$i.json" "https://www.sofiatraffic.bg/interactivecard/lines/stops/geo?line_id=$i"; echo -n "."; done
for i in l/*; do echo -e `cat $i` | sed 's_{"geometry":_\n{"geometry":_g' > temp; mv temp $i; done
echo '{"type": "FeatureCollection", "features": [' > stops.geojson
grep -h geometry l/* | sed 's_}}]}_}},_' >> stops.geojson
sed -i '$ s_}},_}}]}_' stops.geojson
This is a small util for marking neighboring municipalities.
Dependencies:
jquery, leaflet v101
Data:
https://github.com/yurukov/Bulgaria-geocoding/blob/master/municipalities.geojson
When a municipality is marked as red, you need to click on all blue municialities
which are its neighbors. when that's done, you click on the red municipality itself
@yurukov
yurukov / scraper.js
Created April 5, 2016 14:05
Scraper for all pharmacies in Bulgaria
function download(filename, text) {
var element = document.createElement('a');
element.setAttribute('href', 'data:text/plain;charset=utf-8,' + encodeURIComponent(text));
element.setAttribute('download', filename);
element.style.display = 'none';
document.body.appendChild(element);
element.click();
document.body.removeChild(element);
}
@yurukov
yurukov / map.geojson
Last active March 3, 2016 14:24
Водоснабдителните зони на София
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.