Skip to content

Instantly share code, notes, and snippets.

@nhoening
Forked from francoisTemasys/bongo.sh
Last active December 14, 2017 01:40
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 6 You must be signed in to fork a gist
  • Save nhoening/b0c0649282e2fefacaec to your computer and use it in GitHub Desktop.
Save nhoening/b0c0649282e2fefacaec to your computer and use it in GitHub Desktop.
Allowing to pass a query for exporting specific data. Added a LIMIT option to limit the number of returned results. Added a debug switch to see errors.
#!/bin/bash
LOADING=false
DEBUG=/dev/null
usage()
{
cat << EOF
usage: $0 [options] <DBNAME>
OPTIONS:
-h Show this help.
-l Load instead of export
-u Mongo username
-p Mongo password
-H Mongo host string (ex. localhost:27017)
-q Mongo query for export (JSON string)
-L Limit exported results to this amount
-d Show (debug) error messages in console
EOF
}
while getopts "hldu:p:H:q:L:" opt; do
MAXOPTIND=$OPTIND
case $opt in
h)
usage
exit
;;
l)
LOADING=true
;;
d)
DEBUG=/dev/stderr
;;
u)
USERNAME="$OPTARG"
;;
p)
PASSWORD="$OPTARG"
;;
H)
HOST="$OPTARG"
;;
q)
QUERY="$OPTARG"
;;
L)
LIMIT="$OPTARG"
;;
\?)
echo "Invalid option $opt"
exit 1
;;
esac
done
shift $(($MAXOPTIND-1))
if [ -z "$1" ]; then
echo "Usage: $0 [options] <DBNAME>"
exit 1
fi
DB="$1"
if [ -z "$HOST" ]; then
HOST="localhost:27017"
CONN="localhost:27017/$DB"
else
CONN="$HOST/$DB"
fi
ARGS=""
if [ -n "$USERNAME" ]; then
ARGS="-u $USERNAME"
fi
if [ -n "$PASSWORD" ]; then
ARGS="$ARGS -p $PASSWORD"
fi
if $LOADING ; then
echo "*************************** Mongo Import ************************"
echo "**** Host: $HOST"
echo "**** Database: $DB"
echo "**** Username: $USERNAME"
echo "**** Password: $PASSWORD"
echo "*****************************************************************"
if [ ! -f $DB.tar.gz ]; then
echo "Archive $DB.tar.gz to import from could not be found! Aborting ..."
exit
fi
tar -xzf $DB.tar.gz 2>$DEBUG
pushd $DB 2>$DEBUG
for path in *.json; do
collection=${path%.json}
echo "Loading into $DB/$collection from $path"
mongoimport --host $HOST $ARGS -d $DB -c $collection $path --jsonArray
done
popd 2>$DEBUG
rm -rf $DB 2>$DEBUG
else
echo "*************************** Mongo Export ************************"
echo "**** Host: $HOST"
echo "**** Database: $DB"
echo "**** Username: $USERNAME"
echo "**** Password: $PASSWORD"
echo "**** Query: $QUERY"
echo "**** Limit: $LIMIT"
echo "*****************************************************************"
DATABASE_COLLECTIONS=$(mongo $CONN $ARGS --quiet --eval 'db.getCollectionNames()' | sed 's/,/ /g')
mkdir -p tmp/$DB 2>$DEBUG
pushd tmp/$DB 2>$DEBUG
for collection in $DATABASE_COLLECTIONS; do
echo "Exporting collection $collection ..."
mongoexport --host $HOST $ARGS -db $DB -c $collection -q "$QUERY" -o $collection.json >$DEBUG
if [ "$LIMIT" != "" ]; then
head -n $LIMIT $collection.json > $collection.json.tmp && mv $collection.json.tmp $collection.json
echo "limited result set to $LIMIT records"
fi
# TODO: For 2.6, mongoexport supports a limit argument (https://jira.mongodb.org/browse/SERVER-2033).
# Support this native way here as default. Reintroduce the --jsonArray option in that case, probably.
done
pushd .. 2>$DEBUG
tar -czf "$DB.tar.gz" $DB 2>$DEBUG
popd 2>$DEBUG
popd 2>$DEBUG
mv tmp/$DB.tar.gz ./ 2>$DEBUG
rm -rf tmp 2>$DEBUG
fi
@futurist
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment