Skip to content

Instantly share code, notes, and snippets.

@danielbeardsley
Last active September 17, 2019 16:49
Show Gist options
  • Save danielbeardsley/164cbc12155638514ca0aad0f2c4e24c to your computer and use it in GitHub Desktop.
Save danielbeardsley/164cbc12155638514ca0aad0f2c4e24c to your computer and use it in GitHub Desktop.
memcache key analysis

List and analyze keys in memcache

# Writes a big file with one line per key
> bash dump_keys.sh > all-the-keys

# Get stats on the state of the whole key list
> cat all-the-keys | perl ./aggregate_keys.pl
count:2228244 size:86.50%  fetched:53.66%  bytesize:1948.7MB

# Let the script know how big your cache is
> export CACHE_SIZE_GB=1.948

# Pass a subset of the keys to the script
> grep "part_of_a_key" all-the-keys | perl ./aggregate_keys.pl
count:150729 size:2.85%  fetched:51.13%  bytesize:64.3MB
#!/usr/bin/perl
$gb = $ENV{'CACHE_SIZE_GB'};
if (!$gb) {
die "Set the CACHE_SIZE_GB env variable";
}
while (<>) {
/^key=(\S+) .*fetch=(\S+) .*size=(\d+)/ && {
$sizeBytes += $3,
$count++,
$fetched += ($2 eq "yes") ? 1 : 0,
}
}
$size = ((100.0 * $sizeBytes) / ($gb * 1024 * 1024 * 1024));
$sizeMB = ($sizeBytes / (1024 * 1024)) + "MB";
$fetched = ((100.0 * $fetched) / $count);
printf("count:%i size:%.2f%% fetched:%.2f%% bytesize:%.1fMB\n",
$count, $size, $fetched, $sizeMB);
echo "lru_crawler metadump all" | nc localhost 11211
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment