A script to validate your local gem cache against the public s3 repositories. If you find mismatches that contain both local and remote values, please post them in comments.

validate_local_cache.sh

#!/usr/bin/env sh

if ! which md5sum > /dev/null; then
  echo Install md5sum
	exit 1
fi

if ! which curl > /dev/null; then
	echo Install curl
	exit 1
fi

home=$(gem env GEM_HOME)
cache=$home/cache

echo This will take a while...

for gem in $cache/*.gem; do
	gemfile=$(basename $gem)

	local=$(md5sum $gem | awk '{print $1}')
	remote=$(curl -s -D - -X HEAD -H 'Connection:close' http://production.cf.rubygems.org/gems/$gemfile | grep 'ETag' | cut -d '"' -f 2)

	if [[ ! $local = $remote ]]; then
		echo $gemfile mismatch. local: $local, remote: $remote
	fi
done

echo All done.

validate_local_cache_openssl.sh

#!/usr/bin/env sh

if ! which openssl > /dev/null; then
  echo Install openssl
	exit 1
fi

if ! which curl > /dev/null; then
	echo Install curl
	exit 1
fi

home=$(gem env GEM_HOME)
cache=$home/cache

echo This will take a while...

for gem in $cache/*.gem; do
	gemfile=$(basename $gem)

	local=$(openssl md5 $gem | awk '{print $2}')
	remote=$(curl -s -D - -X HEAD -H 'Connection:close' http://production.cf.rubygems.org/gems/$gemfile | grep 'ETag' | cut -d '"' -f 2)

	if [[ ! $local = $remote ]]; then
		echo $gemfile mismatch. local: $local, remote: $remote
	fi
done

echo All done.

validate_locate.sh

#!/usr/bin/env sh

if ! which openssl > /dev/null; then
  echo Install openssl
	exit 1
fi

if ! which curl > /dev/null; then
	echo Install curl
	exit 1
fi

echo This will take a while...

for gem in $(locate \*.gem); do
	gemfile=$(basename $gem)

	local=$(openssl md5 $gem | awk '{print $2}')
	remote=$(curl -s -D - -X HEAD -H 'Connection:close' http://production.cf.rubygems.org/gems/$gemfile | grep 'ETag' | cut -d '"' -f 2)

	if [[ ! $local = $remote ]]; then
		echo $gemfile mismatch. local: $local, remote: $remote
	fi
done

echo All done.

If you're on OSX and you don't have md5sum, and you use homebrew, do this:

brew install md5sha1sum

If you're on OSX and you don't have md5sum, and you use homebrew, do this:

brew install md5sha1sum

Alternatively, use the openssl variant..

Alternatively, use the openssl variant..

Or "md5 -r" http://raamdev.com/2008/howto-install-md5sum-sha1sum-on-mac-os-x/

Or "md5 -r" http://raamdev.com/2008/howto-install-md5sum-sha1sum-on-mac-os-x/

Since we're all in the mood for learning :)

Since we're all in the mood for learning :)

Nice tip drnic, but I will just mention you need edit the script to use md5 -r instead of md5sum, since an alias will not work.

Nice tip drnic, but I will just mention you need edit the script to use md5 -r instead of md5sum, since an alias will not work.

If you have a valid locatedb, then validate_locate.sh is probably best, as it'll cover other gemsets etc.

If you have a valid locatedb, then validate_locate.sh is probably best, as it'll cover other gemsets etc.

@raggi Thank you for posting this.

@raggi Thank you for posting this.

Thanks very much for posting this. You've put my mind at ease.

Thanks very much for posting this. You've put my mind at ease.

😞 ./validate_local_cache.sh: 24: ./validate_local_cache.sh: [[: not found

(Ubuntu 12.10 x86_64, zsh 5.0.0 (x86_64-unknown-linux-gnu))

[edit]
Running this with bash ./validate_local_cache.sh makes it happen - perhaps you should amend the she-bang to #!/usr/bin/env bash
[/edit]

😞 ./validate_local_cache.sh: 24: ./validate_local_cache.sh: [[: not found

(Ubuntu 12.10 x86_64, zsh 5.0.0 (x86_64-unknown-linux-gnu))

[edit]
Running this with bash ./validate_local_cache.sh makes it happen - perhaps you should amend the she-bang to #!/usr/bin/env bash
[/edit]

rack-perftools_profiler-0.4.1.gem mismatch. local: 17bc074dcb1f57a0e4b12739397e428b, remote: 4be04a110f5e7fb42a8d7c1ef2766936

rack-perftools_profiler-0.4.1.gem mismatch. local: 17bc074dcb1f57a0e4b12739397e428b, remote: 4be04a110f5e7fb42a8d7c1ef2766936

daemon-kit-0.2.0.gem mismatch. local: e48293321ef54695a33ebbcd849d607d, remote: 2ad8fee42336f519e96f7c6329493251

daemon-kit-0.2.0.gem mismatch. local: e48293321ef54695a33ebbcd849d607d, remote: 2ad8fee42336f519e96f7c6329493251

awesome_bot_factory-0.0.1.gem mismatch. local: b9f7ab361ac31527f9c3a4f560062c9e, remote: d1eeb86996c50f93be882fa7cf66e55e

awesome_bot_factory-0.0.1.gem mismatch. local: b9f7ab361ac31527f9c3a4f560062c9e, remote: d1eeb86996c50f93be882fa7cf66e55e

sahi-1.0.0.gem mismatch. local: c32f744846c751d51f3a4e296432daa8, remote: faff6ce473c47a4bcb5e0c9b5a4def11

robin@robin-harvey-workstation:gem-checker$ locate *.gem | grep sahi | xargs ls -lh
-rw-rw-r-- 1 robin robin 11K Jul 19 2011 /home/robin/sahi/ruby/sahi-1.0.0.gem
-rw-rw-r-- 1 robin robin 11K Jul 19 2011 /home/robin/sahi/ruby/sahi-1.0.1.gem

sahi-1.0.0.gem mismatch. local: c32f744846c751d51f3a4e296432daa8, remote: faff6ce473c47a4bcb5e0c9b5a4def11

robin@robin-harvey-workstation:gem-checker$ locate *.gem | grep sahi | xargs ls -lh
-rw-rw-r-- 1 robin robin 11K Jul 19 2011 /home/robin/sahi/ruby/sahi-1.0.0.gem
-rw-rw-r-- 1 robin robin 11K Jul 19 2011 /home/robin/sahi/ruby/sahi-1.0.1.gem

fission-0.4.0.gem mismatch. local: 0f010d636fbbdc036004710b91e3c2b1, remote: d34073ffa3435be30a5f92856c76ed33

fission-0.4.0.gem mismatch. local: 0f010d636fbbdc036004710b91e3c2b1, remote: d34073ffa3435be30a5f92856c76ed33

Here's an OSX ready script, that also has some hints for folks using bundler. Also OSX md5 works fine, but awk must look at $4 not $1. Anyhoo: https://gist.github.com/4683645

Here's an OSX ready script, that also has some hints for folks using bundler. Also OSX md5 works fine, but awk must look at $4 not $1. Anyhoo: https://gist.github.com/4683645

Much appreciated!

Much appreciated!