public
Created

Shell script for mirroring rubygems.org. Forcibly downloads quick/Marshal.4.8/${GEM}spec.rz, and all the indices (to prevent having to attempt (and subsequently fail - we just can't make it work) to generate_index ourselves). N.B. $MIRRORSCRIPT refers to a ruby script to fetch new gems similar to the script located at the following URL: http://help.rubygems.org/discussions/problems/592-rubygems-mirror-package-is-chronically-missing-gems-please-fix#comment_7994903 The other shell variables should be self-explanatory.

  • Download Gist
mirror_cheater.sh
Shell
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55
#!/bin/bash
 
# define $SHELL_VARS here
 
report_error () {
MESSAGE=$1
SUBJECT=$2
echo "$MESSAGE" | $MAILPROGRAM --subject="$SUBJECT" --toList=$EMAIL
}
 
echo "Starting mirror `date`"
# get new gems, saving output
echo "Mirroring gems"
$MIRRORSCRIPT > $HOME/gem_mirror_out.txt
 
# might as well quit
if [[ $? != 0 ]]; then
report_error "rubygem mirroring failed on $HOST" "$MIRRORSCRIPT failed"
exit
fi
 
# download all the new quick files
cd $MIRRORDIR/quick/Marshal.4.8/
date
echo "Fetching quick{spec.rz}..."
for GEM in $( cat $HOME/gem_mirror_out.txt | grep ^Downloaded | awk '{print $2}' ); do
echo " ... fetching ${GEM}spec.rz"
wget -q http://rubygems.org/quick/Marshal.4.8/${GEM}spec.rz -O ${GEM}spec.rz
if [[ $? != 0 ]]; then
report_error "rubygem quick retrieval failed for '${GEM}spec.rz' on $HOST" "wget failed to retrieve '${GEM}spec.rz'"
exit
fi
done
 
# download the updated indices
cd $MIRRORDIR
date
echo "Fetching {,latest_,prerelease_}specs.4.8{,.gz}..."
for FILE in latest_specs.4.8 prerelease_specs.4.8 specs.4.8; do
echo " ... fetching ${FILE}.gz"
wget -q http://rubygems.org/${FILE}.gz -O ${FILE}.gz
if [[ $? != 0 ]]; then
report_error "rubygem spec fetching failed for '${FILE}.gz' on $HOST" "index fetching ${FILE}.gz failed"
exit
fi
# and expand them
gunzip -c ${FILE}.gz > $FILE
done
 
# sync over gems
date
echo "Rsync'ing mirror"
rsync -avz $MIRRORDIR/ $USER@$MIRRORSERVER:$MIRRORDIR
 
echo "Finished mirror `date`"

Please sign in to comment on this gist.

Something went wrong with that request. Please try again.