Skip to content

Instantly share code, notes, and snippets.

@Ianfeather
Last active November 9, 2022 11:39
Show Gist options
  • Save Ianfeather/77bdc6a4855d388d9f946a8f8e2065c2 to your computer and use it in GitHub Desktop.
Save Ianfeather/77bdc6a4855d388d9f946a8f8e2065c2 to your computer and use it in GitHub Desktop.
Calculating Cache Hit Ratio from fastly logs
import { readdir, readFile } from 'node:fs/promises';
// Update this path to point to a directory of log files.
const logPath = './fastly-logs';
const files = await readdir(logPath);
// This map holds pairs of page name and page regex and is used to group requests into page types.
// The regex for each is tested against the url of the request in the logline
/// You should update this map to be relevant for the requests you're interested in.
const types = new Map([
['home', /^\/([?].+)?$/],
['static_assets', /^\/static-assets\/.+/],
['recipe', /^\/recipe\/.+/],
['compilation', /^\/compilation\/.+/],
['article', /^\/article\/.+/]
]);
let logLines = [];
const states = {
'MISS-CLUSTER': 'MISS',
'MISS-WAIT-CLUSTER': 'MISS',
'MISS-WAIT': 'MISS',
'HIT-CLUSTER': 'HIT',
'HIT-SYNTH': 'HIT',
'HIT-WAIT-CLUSTER': 'HIT',
'HIT-STALE': 'HIT',
'HIT-STALE-WAIT': 'HIT',
'HIT-STALE-CLUSTER': 'HIT',
'HIT-WAIT': 'HIT',
'HITPASS': 'HIT'
}
for (const filename of files) {
const rawLogs = await readFile(`${logPath}/${filename}`, 'utf-8');
const structuredLogs = rawLogs.split('\n')
.filter(Boolean)
.map(ll => JSON.parse(ll))
.map(ll => ({ path: ll.http.path, state: ll.network.server.state }))
logLines.push(structuredLogs);
}
const totals = logLines.flat()
.reduce((acc,next) => {
let type;
for (let [name, tester] of types) {
if (next.path.match(tester)) {
type = name;
break;
}
}
if (!type) return acc;
const state = states[next.state] || next.state;
if (acc[type][state]) {
acc[type][state]++;
} else {
acc[type][state] = 1;
}
return acc;
}, [...types.keys()].reduce((obj, page) => ({...obj, [`${page}`]: {}}), {}))
console.log(`Cache Hit Ratios:`);
Object.keys(totals).forEach(type => {
console.log(`${type}: ${(totals[type].HIT / (totals[type].MISS + totals[type].HIT) * 100)}`);
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment