Skip to content

Instantly share code, notes, and snippets.

@Barqawiz
Last active August 12, 2023 19:49
Show Gist options
  • Save Barqawiz/53c7257e942b173357f15a5217e1ac28 to your computer and use it in GitHub Desktop.
Save Barqawiz/53c7257e942b173357f15a5217e1ac28 to your computer and use it in GitHub Desktop.
const llmEvaluation = new LLMEvaluation(openaiChatKey, 'openai');
async function testLLMEvaluation() {
const userInput = "<add your question or prompt here>";
// add idea answers, the more you add the better
const targetAnswers = ["<ideal answer 1>",
"<ideal answer 2>"];
// the models set to compare
const providerSets = [openaiChat, cohereCompletion, llamaChat];
const results = await llmEvaluation.compareModels(userInput, targetAnswers, providerSets);
console.log('The Evaluation Results:', results);
}
(async () => {
await testLLMEvaluation();
})();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment