public
Last active — forked from lbosque/cdb_import.sh

Import CartoDB file with cURL

  • Download Gist
cdb_import.sh
Shell
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
#!/bin/bash
CDB_USER=$1
API_KEY=$2
IMPORT_FILE=$3
 
if [[ -z $CDB_USER ]]
then
echo "Missing user"
fi
if [[ -z $API_KEY ]]
then
echo "Missing api key"
fi
if [[ -z $IMPORT_FILE ]]
then
echo "Missing file"
fi
 
v1=$(uname)
 
echo "Sending file..."
if [[ "$v1" = Darwin ]];
then
job_id=`curl -s -F file=@${IMPORT_FILE} "https://${CDB_USER}.cartodb.com/api/v1/imports/?api_key=${API_KEY}" | sed -E 's/\{\"item_queue_id\":([0-9]+).*/\1/'`
else
job_id=`curl -s -F file=@${IMPORT_FILE} "https://${CDB_USER}.cartodb.com/api/v1/imports/?api_key=${API_KEY}" | sed -r 's/\{\"item_queue_id\":([0-9]+).*/\1/'`
fi
 
echo "Waiting for job ${job_id} to be completed..."
while true
do
if [[ "$v1" = Darwin ]];
then
status=`curl -s "https://${CDB_USER}.cartodb.com/api/v1/imports/${job_id}?api_key=${API_KEY}" | sed -E 's/(.*)\"state\":\"([a-z]+)\"(.*)/\2/'`
else
status=`curl -s "https://${CDB_USER}.cartodb.com/api/v1/imports/${job_id}?api_key=${API_KEY}" | sed -r 's/(.*)\"state\":\"([a-z]+)\"(.*)/\2/'`
fi
echo "STATE: ${status}"
if [[ $status == 'complete' ]]
then
echo "Import successful"
break
fi
sleep 2
done
gistfile1.txt
1
./cdb_import.sh <cdb_username> <api_key> <filename>

Bugfix & addition of notification email. The REGEX for sed on the first curl wasn't working for me.

item-queue-id's seem to follow this format now: 9a27y779-0ec9-4cv3-a86d-f64bea4c6416

If you incorporate any of these changes, please credit MapLight (http://maplight.org) as the "sponsor" for the work.

#!/bin/bash

CDB_USER=$1
API_KEY=$2
IMPORT_FILE=$3
NOTIFICATION_EMAIL=$4

if [[ -z $CDB_USER ]]
then
  echo "Missing user"
fi
if [[ -z $API_KEY ]]
then
  echo "Missing api key"
fi
if [[ -z $IMPORT_FILE ]]
then
  echo "Missing file"
fi


v1=$(uname)

echo "Sending file..."
if [[ "$v1" = Darwin ]];
then
  job_id=`curl -s -F file=@${IMPORT_FILE} "https://${CDB_USER}.cartodb.com/api/v1/imports/?api_key=${API_KEY}" | sed -E 's/\{\"item_queue_id\":\"([^"]+)\".*/\1/'`
else
  job_id=`curl -s -F file=@${IMPORT_FILE} "https://${CDB_USER}.cartodb.com/api/v1/imports/?api_key=${API_KEY}" | sed -r 's/\{\"item_queue_id\":\"([^"]+)\".*/\1/'`
fi

echo "Waiting for job ${job_id} to be completed..."

while true
do
  if [[ "$v1" = Darwin ]];
  then
    status=`curl -s "https://${CDB_USER}.cartodb.com/api/v1/imports/${job_id}?api_key=${API_KEY}" | sed -E 's/(.*)\"state\":\"([a-z]+)\"(.*)/\2/'`
  else
    status=`curl -s "https://${CDB_USER}.cartodb.com/api/v1/imports/${job_id}?api_key=${API_KEY}" | sed -r 's/(.*)\"state\":\"([a-z]+)\"(.*)/\2/'`
  fi
  echo "STATE: ${status}"
  if [[ $status == 'complete' ]]
  then
    echo "Import successful"
    if [[ -n $NOTIFICATION_EMAIL ]]
    then
      echo "https://${CDB_USER}.cartodb.com" | mail -s "CartoDB import finished: ${IMPORT_FILE}" "${NOTIFICATION_EMAIL}"
    fi
    break
  fi
  sleep 2
done

Please sign in to comment on this gist.

Something went wrong with that request. Please try again.