This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# | |
# script to load project data from the world bank and generate a training set | |
# for a classifier | |
# | |
# Get a list of projects from the world bank and filter out projects w/o abstracts & themes | |
# | |
# curl -s 'http://search.worldbank.org/api/v2/projects?format=json&fl=project_name,project_abstract,theme_namecode&source=IBRD&rows=50000' | |
# | jq '[.projects[] | select(.project_abstract? and .theme_namecode?)] | map({"text": [.project_name, .project_abstract.cdata] | join(" - "), "themes": .theme_namecode | map(.code)})' | |
# > wb-projects.json | |
# |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Script to get Google CLoud Vision annotations for a bunch of images | |
# Images in /images are expected to be also present in the specified GCS bucket | |
# | |
# setup: | |
# gem install google-cloud-vision | |
# export GOOGLE_CLOUD_PROJECT="XX-XX" | |
# export GOOGLE_CLOUD_KEYFILE="~/path/to/XX-XX.json" | |
# | |
require 'google/cloud/vision' |
OlderNewer