Skip to content

Instantly share code, notes, and snippets.

@kudoh
Last active October 12, 2024 15:40
Show Gist options
  • Save kudoh/c0995ba2233138312c2f412868f196d0 to your computer and use it in GitHub Desktop.
Save kudoh/c0995ba2233138312c2f412868f196d0 to your computer and use it in GitHub Desktop.
OpenAI Realtime API Example
import WebSocket from 'ws';
import { spawn } from 'child_process';
const url = 'wss://api.openai.com/v1/realtime?model=gpt-4o-realtime-preview-2024-10-01';
const ws = new WebSocket(url, {
headers: {
'Authorization': 'Bearer ' + process.env.OPENAI_API_KEY,
'OpenAI-Beta': 'realtime=v1'
}
});
const instructions = `時々俳句や川柳を織り交ぜて返事するアシスタントとして会話してください。
普通の返答に詩的なフレーバーを追加。シンプルなタスク指示でも突然俳句が入ってきます。
例:「タスク完了、おつかれさまです。秋の風、少し寒くて、そっと終わる。」`
const recorder = spawn('sox', [
'--default-device',
'--no-show-progress',
'--rate', '24000',
'--channels', '1',
'--encoding', 'signed-integer',
'--bits', '16',
'--type', 'raw',
'-' // 標準出力
]);
const recorderStream = recorder.stdout;
const player = spawn('sox', [
'--type', 'raw',
'--rate', '24000',
'--encoding', 'signed-integer',
'--bits', '16',
'--channels', '1',
'-', // 標準入力
'--no-show-progress',
'--default-device',
]);
const audioStream = player.stdin;
ws.on('open', () => {
ws.send(JSON.stringify({
type: 'session.update',
session: {
voice: 'echo',
instructions: instructions,
input_audio_transcription: { model: 'whisper-1' },
turn_detection: { type: "server_vad" }
}
}));
recorderStream.on('data', (chunk: Buffer) => {
ws.send(JSON.stringify({
type: 'input_audio_buffer.append',
audio: chunk.toString('base64')
}));
});
});
ws.on('message', (message) => {
const event = JSON.parse(message.toString());
console.log(event.type);
// console.log('EVENT:', JSON.stringify(event, null, 2));
switch (event.type) {
case 'response.audio.delta':
audioStream.write(Buffer.from(event.delta, 'base64'));
break;
case 'response.audio_transcript.done':
case 'conversation.item.input_audio_transcription.completed':
console.log(event.type, event.transcript);
break;
case 'error':
console.error('ERROR', event.error);
break;
}
});
ws.on('error', (error) => {
console.error('WebSocketエラー', { error });
});
ws.on('close', (event) => {
console.log('close connection');
audioStream.end();
recorderStream.pause();
recorder.kill('SIGHUP');
player.kill('SIGHUP');
process.exit();
});
// 標準入力を待機。`q`で終了
process.stdin.resume();
process.stdin.setEncoding('utf8');
process.stdin.on('data', (input: string) => {
if (input.trim().toLowerCase() === 'q') {
console.log('終了します...');
ws.close();
setTimeout(() => {
process.exit();
}, 1000);
}
});