Skip to content

Instantly share code, notes, and snippets.

@daveteu
Created October 18, 2023 08:30
Show Gist options
  • Save daveteu/8517ff3845b1fe877d3b311519b67e6f to your computer and use it in GitHub Desktop.
Save daveteu/8517ff3845b1fe877d3b311519b67e6f to your computer and use it in GitHub Desktop.
Axios Read Stream
/*
* How to process stream and NDJson
*/
const matcher = /\r?\n/
const processLine = (line) => {
console.log('line', line)
}
const axiosFetch = (url) => {
const response = await axios.get(url, { responseType: 'stream' })
const stream = response.data;
let buff = "";
stream.on('data', (data) => {
buff += data.toString();
const parts = buff.split(matcher);
buff = parts.pop();
for (const i of parts) {
processLine(JSON.parse(i));
}
})
stream.on('end', () => {
if (buff.length > 0) {
processLine(JSON.parse(buff))
}
})
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment