- ai-rag-chat-evaluator: Repo with scripts to generate sample data based on Azure AI search index and evaluate responses from a chat app using GPT-4 model and custom metrics. Designed for use with azure-search-openai-demo but can be modified for use with other chat apps.
- [📝 Blog post: Can your app say "I don't know?"](https://blog.pamelafox.org/2024/03/evaluating-rag-chat-apps-can-y
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* | |
Countries and dependent territories, 2020 | |
Data adapted from | |
http://www.worldometers.info/world-population/population-by-country/ | |
Does not include rows which had "N.A." values, so some territories are missing. | |
*/ | |
CREATE TABLE countries( | |
name TEXT PRIMARY KEY, | |
population INTEGER, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// Includes functions for exporting active sheet or all sheets as JSON object (also Python object syntax compatible). | |
// Tweak the makePrettyJSON_ function to customize what kind of JSON to export. | |
var FORMAT_ONELINE = 'One-line'; | |
var FORMAT_MULTILINE = 'Multi-line'; | |
var FORMAT_PRETTY = 'Pretty'; | |
var LANGUAGE_JS = 'JavaScript'; | |
var LANGUAGE_PYTHON = 'Python'; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CREATE TABLE pokemon( | |
Number INTEGER, | |
Name TEXT PRIMARY KEY, | |
Type_1 TEXT, | |
Type_2 TEXT, | |
Total INTEGER, | |
HP INTEGER, | |
Attack INTEGER, | |
Defense INTEGER, | |
Sp_Atk INTEGER, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// How to add a bookmarklet to Chrome: https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=chrome%20bookmarklet | |
javascript:(function getScreenshot() {var openScreenshot = function(e) {window.open(e.originalEvent.data);}; $(window).unbind("message.getScreenshot", openScreenshot); $(window).bind("message.getScreenshot", openScreenshot); ScratchpadUI.liveEditor.postFrame({ screenshot: true, screenshotSize: 400 });})(); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* Source: | |
https://www.kaggle.com/nasa/astronaut-yearbook | |
*/ | |
CREATE TABLE astronauts( | |
Name TEXT PRIMARY KEY, | |
Year INTEGER, | |
GroupNum INTEGER, | |
Status TEXT, | |
Birth_Date TEXT, | |
Birth_Place TEXT, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import json | |
import logging | |
import re | |
from typing import Any, AsyncGenerator, Optional, Union | |
import aiohttp | |
import openai | |
from azure.search.documents.aio import SearchClient | |
from azure.search.documents.models import QueryType |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* | |
1st 3 rounds of the NFL 2015 Draft | |
Collected by: https://www.khanacademy.org/profile/BobbyandKaren/ | |
*/ | |
CREATE TABLE picks( | |
id INTEGER PRIMARY KEY, | |
pick_number INTEGER, | |
name TEXT, | |
college TEXT, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* Friendly Card Game Results: | |
During this past winter, a few friends got together every Wednesday night for a friendly game of cards. On some nights they'd play two games, but never the same game twice on the same night. The usual players were Spunky Sam, Marcimus, Winston, and Hopper. Sometimes, one of the friends couldn't make it, so there were only three players. But sometimes they'd call another friend to fill-in. In every game they played, the one with the hightest score was declared the winner. These are their results: | |
Created by: https://www.khanacademy.org/profile/brianduckworth | |
*/ | |
CREATE TABLE card_games(id INTEGER PRIMARY KEY AUTOINCREMENT, | |
date_played TEXT, | |
game_name TEXT, | |
player_name TEXT, | |
score INTEGER); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* | |
Winston's Donut logs | |
This table of logs shows how many donuts Winston eats at each year of his life, | |
plus any particular reason to explain his eating habits. | |
Collected by: https://www.khanacademy.org/profile/mhogwarts/ | |
*/ | |
CREATE TABLE winstons_donut_logs ( | |
id TEXT PRIMARY KEY, | |
status TEXT, | |
years_old INTEGER, |
NewerOlder