Skip to content

Instantly share code, notes, and snippets.

@dougbtv
Created April 14, 2023 18:09
Show Gist options
  • Save dougbtv/249a31e0d9d671715394a99139a5d3bd to your computer and use it in GitHub Desktop.
Save dougbtv/249a31e0d9d671715394a99139a5d3bd to your computer and use it in GitHub Desktop.
Node.js script to query oobabooga via API
const axios = require('axios');
// Server URL
const server = "http://192.168.50.201:5000/api/v1/generate";
// Generation parameters
// Reference: https://huggingface.co/docs/transformers/main_classes/text_generation#transformers.GenerationConfig
const data = {
'prompt': 'recommend a cheese please',
"max_new_tokens": 100,
"do_sample": true,
"temperature": 0.85,
'top_p': 1,
'typical_p': 1,
'repetition_penalty': 1.01,
'encoder_repetition_penalty': 1,
'top_k': 50,
'min_length': 0,
'no_repeat_ngram_size': 0,
'num_beams': 1,
'penalty_alpha': 0,
'length_penalty': 1,
'early_stopping': false,
'seed': -1,
'add_bos_token': true,
'truncation_length': 2048,
'ban_eos_token': false
};
const headers = {
"Content-Type": "application/json"
};
axios.post(server, data, { headers })
.then(response => {
if (response.status === 200) {
// console.log(response);
const results = response.data.results;
for (const result of results) {
console.log(result.text);
}
} else {
console.log(`Request failed with status code ${response.status}.`);
}
})
.catch(error => {
console.log(`Request failed with error: ${error.message}.`);
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment