Skip to content

Instantly share code, notes, and snippets.

Last active March 2, 2023 23:08
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
What would you like to do?
Quick and dirty GPT Tokenizer in one command


deno run --watch --allow-net main.ts
curl -d 'test world :)' http://localhost:8085


deno run --allow-net=

curl -d 'test world :)' http://localhost:8085
import { Application } from "";
import GPT3Tokenizer from "";
const app = new Application();
app.use(async (ctx) => {
if (ctx.request.method === "POST" && ctx.request.url.pathname === "/") {
// read the body
const body = ctx.request.body({ type: "text" });
const value = await body.value;
const encoded = tokenizer.encode(value);
// respond with json
ctx.response.body = encoded;
} else {
ctx.response.body = "Hello World!";
const tokenizer = new GPT3Tokenizer({ type: "gpt3" }); // or 'codex'
console.log("Server started on port http://localhost:8085");
await app.listen({ port: 8085 });
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment