Skip to content

Instantly share code, notes, and snippets.

@pangoSE
Last active January 13, 2020 06:56
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save pangoSE/5294e835c9bead773e8036603a90b898 to your computer and use it in GitHub Desktop.
Save pangoSE/5294e835c9bead773e8036603a90b898 to your computer and use it in GitHub Desktop.
Quick OSM statistics with bash and jq
#Script to calculate some quick stats on OSM data
#License: GPLv3 or later.
#Author: pangoSE
#invoke like this $ stats.sh x.geojson
#download the geojson from Overpass Turbo -> export or save as from JOSM.
#example query: access!=private and (tourism=alpine_hut or tourism=wilderness_hut or (amenity=shelter and shelter_type!=public_transport)) in sweden
#you can run this query from inside JOSM - see the JOSM wiki.
shelters=$(cat $1 |grep 'shelter",' |wc -l)
lean=$(cat $1 |jq '.features[] .properties | select(.amenity == "shelter") | select(.shelter_type == "lean_to")' | jq -s 'length')
bh=$(cat $1 |jq '.features[] .properties | select(.amenity == "shelter") | select(.shelter_type == "basic_hut")' | jq -s 'length')
ws=$(cat $1 |jq '.features[] .properties | select(.amenity == "shelter") | select(.shelter_type == "weather_shelter")' | jq -s 'length')
ps=$(cat $1 |jq '.features[] .properties | select(.amenity == "shelter") | select(.shelter_type == "picnic_shelter")' | jq -s 'length')
su=$(cat $1 |jq '.features[] .properties | select(.amenity == "shelter") | select(.shelter_type |not)' | jq -s 'length')
ah=$(cat $1 |grep 'alpine_hut"' |wc -l)
wh=$(cat $1 |grep 'wilderness_hut",' |wc -l)
echo Numbers of each type:
echo Shelters - total: $shelters
echo "Shelters - Lean to (sv. vindskydd):" $lean
echo "Shelters - Basic hut (sv. stuga utan uppvärmning):" $bh
echo "Shelters - Weather shelter (sv. väderskydd endast lämpad för kort uppehåll vid dåligt väder):" $ws
echo "Shelters - Picnic shelter (sv. liten byggnad med tak, eldstad och skorsten i mitten):" $ps
echo "Shelters - Unknown type:" $su
echo Wilderness huts: $wh
echo Alpine huts: $ah
echo
echo Lean_to shelters details:
desc=$(cat $1 |jq '.features[] .properties | select(.amenity == "shelter" and .shelter_type=="lean_to") | select(has("description"))' | jq -s 'length')
echo - has description tag: $desc - $(expr $desc \* 100 / $lean)\%
floor=$(cat $1 |jq '.features[] .properties | select(.amenity == "shelter" and .shelter_type=="lean_to") | select(.floor=="wood")' | jq -s 'length')
fire=$(cat $1 |jq '.features[] .properties | select(.amenity == "shelter" and .shelter_type=="lean_to") | select(.fireplace=="yes")' | jq -s 'length')
water=$(cat $1 |jq '.features[] .properties | select(.amenity == "shelter" and .shelter_type=="lean_to") | select(has("water_source"))' | jq -s 'length')
echo - has fireplace=yes: $fire - $(expr $fire \* 100 / $lean)\%
echo - has floor=wood: $floor - $(expr $floor \* 100 / $lean)\%
image=$(cat $1 |jq '.features[] .properties | select(.amenity == "shelter" and .shelter_type=="lean_to") | select(has("image"))' | jq -s 'length')
echo - has image: $image - $(expr $image \* 100 / $lean)\%
snow=$(cat $1 |jq '.features[] .properties | select(.amenity == "shelter" and .shelter_type=="lean_to") | select(.snowmobile=="yes")' | jq -s 'length')
echo - has snowmobile=yes: $snow - $(expr $snow \* 100 / $lean)\%
echo - has water_source tag: $water - $(expr $water \* 100 / $lean)\%
wood=$(cat $1 |jq '.features[] .properties | select(.amenity == "shelter" and .shelter_type=="lean_to") | select(.wood_provided=="yes")' | jq -s 'length')
echo - has wood_provided=yes: $wood - $(expr $wood \* 100 / $lean)\%
echo
echo Basic huts details:
desc=$(cat $1 |jq '.features[] .properties | select(.shelter_type=="basic_hut") | select(has("description"))' | jq -s 'length')
echo - has description tag: $desc - $(expr $desc \* 100 / $bh)\%
fire=$(cat $1 |jq '.features[] .properties | select(.shelter_type=="basic_hut") | select(.fireplace=="yes")' | jq -s 'length')
echo - has fireplace=yes: $fire - $(expr $fire \* 100 / $bh)\%
image=$(cat $1 |jq '.features[] .properties | select(.shelter_type=="basic_hut") | select(has("image"))' | jq -s 'length')
echo - has image: $image - $(expr $image \* 100 / $bh)\%
snow=$(cat $1 |jq '.features[] .properties | select(.shelter_type=="basic_hut") | select(.snowmobile=="yes")' | jq -s 'length')
echo - has snowmobile=yes: $snow - $(expr $snow \* 100 / $bh)\%
water=$(cat $1 |jq '.features[] .properties | select(.shelter_type=="basic_hut") | select(has("water_source"))' | jq -s 'length')
echo - has water_source tag: $water - $(expr $water \* 100 / $bh)\%
wood=$(cat $1 |jq '.features[] .properties | select(.shelter_type=="basic_hut") | select(.wood_provided=="yes")' | jq -s 'length')
echo - has wood_provided=yes: $wood - $(expr $wood \* 100 / $bh)\%
echo
echo Wilderness huts details:
desc=$(cat $1 |jq '.features[] .properties | select(.tourism=="wilderness_hut") | select(has("description"))' | jq -s 'length')
echo - has description tag: $desc - $(expr $desc \* 100 / $wh)\%
fire=$(cat $1 |jq '.features[] .properties | select(.tourism=="wilderness_hut") | select(.fireplace=="yes")' | jq -s 'length')
echo - has fireplace=yes: $fire - $(expr $fire \* 100 / $wh)\%
image=$(cat $1 |jq '.features[] .properties | select(.tourism=="wilderness_hut") | select(has("image"))' | jq -s 'length')
echo - has image: $image - $(expr $image \* 100 / $wh)\%
snow=$(cat $1 |jq '.features[] .properties | select(.tourism=="wilderness_hut") | select(.snowmobile=="yes")' | jq -s 'length')
echo - has snowmobile=yes: $snow - $(expr $snow \* 100 / $wh)\%
water=$(cat $1 |jq '.features[] .properties | select(.tourism=="wilderness_hut") | select(has("water_source"))' | jq -s 'length')
echo - has water_source tag: $water - $(expr $water \* 100 / $wh)\%
wood=$(cat $1 |jq '.features[] .properties | select(.tourism=="wilderness_hut") | select(.wood_provided=="yes")' | jq -s 'length')
echo - has wood_provided=yes: $wood - $(expr $wood \* 100 / $wh)\%
echo
echo Alpine huts details:
desc=$(cat $1 |jq '.features[] .properties | select(.tourism=="alpine_hut") | select(has("description"))' | jq -s 'length')
echo - has description tag: $desc - $(expr $desc \* 100 / $ah)\%
fire=$(cat $1 |jq '.features[] .properties | select(.tourism=="alpine_hut") | select(.fireplace=="yes")' | jq -s 'length')
echo - has fireplace=yes: $fire - $(expr $fire \* 100 / $ah)\%
image=$(cat $1 |jq '.features[] .properties | select(.tourism=="alpine_hut") | select(has("image"))' | jq -s 'length')
echo - has image: $image - $(expr $image \* 100 / $ah)\%
snow=$(cat $1 |jq '.features[] .properties | select(.tourism=="alpine_hut") | select(.snowmobile=="yes")' | jq -s 'length')
echo - has snowmobile=yes: $snow - $(expr $snow \* 100 / $ah)\%
water=$(cat $1 |jq '.features[] .properties | select(.tourism=="alpine_hut") | select(has("water_source"))' | jq -s 'length')
echo - has water_source tag: $water - $(expr $water \* 100 / $ah)\%
wood=$(cat $1 |jq '.features[] .properties | select(.tourism=="alpine_hut") | select(.wood_provided=="yes")' | jq -s 'length')
echo - has wood_provided=yes: $wood - $(expr $wood \* 100 / $ah)\%
@brother
Copy link

brother commented Jan 13, 2020

Some quick thoughts that might help readability of these lines.

Use https://shellcheck.net to avoid common pitfalls in shell syntax, is available as inline help in editors like vs code and so on.

cat $1 |grep 'SOMETHING' |wc -l can be written as grep -c 'SOMETHING' "$1"

$(expr $fire \* 100 / $lean)\%can probably be written $(( fire * 100 / lean )) (I tend to use bash though and not be strict POSIX, the script has no shebang... shellcheck will help you stay out of truoble with bashisms as well if needed to be portable - it's titled bash though)

I am fairly sure the jq fiddling can be slimmed a bit, at least it can read files so there are no need for the cat $1 | ... just tag the file at the end of the command. The better solution would probably be to have a internal function that does the most stuff wrapping the jq command.

myjq() {
local file=$1
local expression=$2
echo $(jq '$expression' $file | jq -s 'length')
}

# replacement for line 87:
wood=$(myjq '.features[] .properties | select(.tourism=="alpine_hut") | select(.wood_provided=="yes")' "$1")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment