Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Bash script that connects to a mysql DB to dump column value. It performs this in batches instead of one try. Useful when exporting large amount of data.
#!/bin/bash
# Connects to a MySQL database to dump data in batches.
# Great for dumping large datasets.
LIMIT=1000000
OFFSET=0
MAX_ROWS=1000000000
# This loops until it has read MAX_ROWS
while [ $OFFSET -lt $MAX_ROWS ]
do
echo "Fetching rows starting $OFFSET"
echo "SELECT COL_NAME FROM TABLE_NAME LIMIT $LIMIT OFFSET $OFFSET" | mysql DATABASE_NAME --skip-column-names -B -u MYSQLUSERNAME -pMYSQL_PASSWD >> output.csv
OFFSET=$((OFFSET+LIMIT))
done
# All you have to do is tweak the LIMIT and MAX variable to your liking.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment