Skip to content

Instantly share code, notes, and snippets.

@AnsonT
Created May 15, 2023 18:15
Show Gist options
  • Save AnsonT/9b0b6ee195d4cfc6af0eb275aede2512 to your computer and use it in GitHub Desktop.
Save AnsonT/9b0b6ee195d4cfc6af0eb275aede2512 to your computer and use it in GitHub Desktop.
Stream OpenAI response though NextJS API route
import { NextApiRequest, NextApiResponse } from 'next'
import { Readable } from 'node:stream'
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
const { authorization } = req.headers
if (!authorization || authorization !== process.env.AUTHORIZATION_HEADER) {
return res.status(401).json({ error: 'Unauthorized' })
}
const openAIUrl = process.env.OPENAI_URL as string
const headers = new Headers()
headers.set('Content-Type', 'application/json')
headers.set('Authorization', `Bearer ${process.env.OPENAI_API_KEY as string}`)
const body = JSON.stringify({
model: 'gpt-3.5-turbo',
max_tokens: 1024,
temperature: 0.8,
top_p: 1.0,
frequency_penalty: 0,
presence_penalty: 0,
stream: true,
messages: [
{ role: "user", content: req.body.input }
],
})
const response = await fetch(
openAIUrl,
{
method: 'POST',
headers,
body,
})
Readable.fromWeb(response.body as any).pipe(res)
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment