Skip to content

Instantly share code, notes, and snippets.

@SgtPooki
Last active March 17, 2024 06:09
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save SgtPooki/9efc039d1d349d112d0f505b2ed275c8 to your computer and use it in GitHub Desktop.
Save SgtPooki/9efc039d1d349d112d0f505b2ed275c8 to your computer and use it in GitHub Desktop.
webm streaming req/res of different browsers

While debugging streaming video with the helia-service-worker-gateway, I decided to set up a basic nodejs streaming server that we could ensure a basic test case is working in the browsers, so we knew what to target.

The basic server seems to work in all browsers except for safari.

The steps to generage the attached log files are as follows:

  1. run node demo-video-server.js > <browser>.log
  2. open browser, start a new tab
  3. in tab, enter localhost:8889 into the url and hit enter
  4. stop the video if it plays
  5. stop the demo video server
Server is listening on port 8889
request received: 0
/
GET
headers:
host: localhost:8889
connection: keep-alive
sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Brave";v="122"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "macOS"
upgrade-insecure-requests: 1
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8
sec-gpc: 1
accept-language: en-US,en
sec-fetch-site: none
sec-fetch-mode: navigate
sec-fetch-user: ?1
sec-fetch-dest: document
accept-encoding: gzip, deflate, br
request0 05:05:17.286 readable
request0 05:05:17.287 ended
request0 05:05:17.287 closed
stream0 05:05:17.289 open
response0 05:05:17.289 pipe
response0 05:05:17.419 closed
response0 05:05:17.421 unpipe
request received: 1
/
GET
headers:
host: localhost:8889
connection: keep-alive
sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Brave";v="122"
accept-encoding: identity;q=1, *;q=0
sec-ch-ua-mobile: ?0
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36
sec-ch-ua-platform: "macOS"
accept: */*
sec-gpc: 1
accept-language: en-US,en
sec-fetch-site: same-origin
sec-fetch-mode: no-cors
sec-fetch-dest: video
referer: http://localhost:8889/
range: bytes=0-
request1 05:05:17.435 readable
request1 05:05:17.435 ended
request1 05:05:17.435 closed
stream1 05:05:17.435 open
response1 05:05:17.436 pipe
response1 05:05:19.910 closed
response1 05:05:19.910 unpipe
Server is listening on port 8889
request received: 0
/
GET
headers:
host: localhost:8889
connection: keep-alive
sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Google Chrome";v="122"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "macOS"
dnt: 1
upgrade-insecure-requests: 1
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
sec-fetch-site: none
sec-fetch-mode: navigate
sec-fetch-user: ?1
sec-fetch-dest: document
accept-encoding: gzip, deflate, br, zstd
accept-language: en-US,en;q=0.9
request0 05:04:57.540 readable
request0 05:04:57.542 ended
request0 05:04:57.542 closed
stream0 05:04:57.544 open
response0 05:04:57.544 pipe
response0 05:04:57.623 closed
response0 05:04:57.625 unpipe
request received: 1
/
GET
headers:
host: localhost:8889
connection: keep-alive
sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Google Chrome";v="122"
dnt: 1
accept-encoding: identity;q=1, *;q=0
sec-ch-ua-mobile: ?0
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36
sec-ch-ua-platform: "macOS"
accept: */*
sec-fetch-site: same-origin
sec-fetch-mode: no-cors
sec-fetch-dest: video
referer: http://localhost:8889/
accept-language: en-US,en;q=0.9
range: bytes=0-
request1 05:04:57.632 readable
request1 05:04:57.632 ended
request1 05:04:57.632 closed
stream1 05:04:57.633 open
response1 05:04:57.633 pipe
response1 05:04:57.838 closed
response1 05:04:57.838 unpipe
request received: 2
/
GET
headers:
host: localhost:8889
connection: keep-alive
sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Google Chrome";v="122"
dnt: 1
accept-encoding: identity;q=1, *;q=0
sec-ch-ua-mobile: ?0
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36
sec-ch-ua-platform: "macOS"
accept: */*
sec-fetch-site: same-origin
sec-fetch-mode: no-cors
sec-fetch-dest: video
referer: http://localhost:8889/
accept-language: en-US,en;q=0.9
range: bytes=1048576-
request2 05:04:58.506 readable
request2 05:04:58.506 ended
request2 05:04:58.506 closed
stream2 05:04:58.506 open
response2 05:04:58.506 pipe
response2 05:05:01.368 closed
response2 05:05:01.368 unpipe
/* eslint-disable no-console */
// @ts-check
import http from 'node:http'
import { createVerifiedFetch } from '@helia/verified-fetch'
import { dnsJsonOverHttps } from '@multiformats/dns/resolvers'
import { contentTypeParser } from './dist-tsc/src/lib/content-type-parser.js'
const fetch = await createVerifiedFetch({
gateways: ['http://127.0.0.1:8081'],
routers: ['http://127.0.0.1:8081'],
dnsResolvers: {
'.': dnsJsonOverHttps('https://delegated-ipfs.dev/dns-query')
}
}, {
contentTypeParser
})
/**
*
* @param {http.IncomingMessage} req
* @param {*} reqId
*/
const addLoggingToRequest = (req, reqId) => {
req.on('close', () => {
log(`request${reqId}`, 'closed')
// console.log(`request${reqId} closed`)
})
// don't listen to data event
// req.on('data', (chunk) => {
// console.log(`request${reqId} data: `, chunk)
// })
req.on('end', () => {
log(`request${reqId}`, ' ended')
})
req.on('error', (err) => {
log(`request${reqId}`, ' error: `, er', err)
})
req.on('pause', () => {
log(`request${reqId}`, ' paused')
})
req.on('readable', () => {
log(`request${reqId}`, ' readable')
})
req.on('resume', () => {
log(`request${reqId}`, ' resumed')
})
}
/**
*
* @param {http.ServerResponse<http.IncomingMessage>} res
* @param {number} reqId
*/
const addLoggingToResponse = (res, reqId) => {
res.on('pause', () => {
log(`response${reqId}`, ' paused')
})
res.on('resume', () => {
log(`response${reqId}`, ' resumed')
})
res.on('close', () => {
log(`response${reqId}`, ' closed')
})
res.on('aborted', () => {
log(`response${reqId}`, ' aborted')
})
res.on('error', (err) => {
log(`response${reqId}`, ' error', err)
})
res.on('finish', () => {
log(`response${reqId}`, ' finished')
})
res.on('end', () => {
log(`response${reqId}`, ' ended')
})
res.on('unpipe', () => {
log(`response${reqId}`, ' unpipe')
})
res.on('pipe', () => {
log(`response${reqId}`, ' pipe')
})
}
const log = (id, ...args) => {
console.log(`${id} ${new Date().toISOString().substring(11, 23)}`, ...args)
}
let lastReqId = 0
function logHeaders (reqId, headers) {
console.group(`headers${reqId}: `)
if (headers != null) {
for (const [name, value] of Object.entries(headers)) {
console.log(`${name}: ${value}`)
}
}
console.groupEnd()
}
const server = http.createServer(function (req, res) {
const reqId = lastReqId++
console.group(`server: request received ${reqId}`)
log(`server${reqId}`, req.url)
log(`server${reqId}`, req.method)
logHeaders(reqId, req.headers)
console.groupEnd()
addLoggingToRequest(req, reqId)
addLoggingToResponse(res, reqId)
const vFetchUrl = 'https://bafybeid2sc4k5ynx5yo7wg23fzw2ni4xxuxi7kdq5q3lyurkvglh5wrwmi.ipfs.sw.sgtpooki.com'
log(`server${reqId}`, 'vFetchUrl', vFetchUrl)
const vFetchReqHeaders = new Headers()
for (const [name, value] of Object.entries(req.headersDistinct)) {
if (value == null) {
continue
}
vFetchReqHeaders.set(name, value.join(', '))
}
const abortController = new AbortController()
// if the request stops, cancel the fetch
req.on('aborted', () => {
log(`server${reqId}`, 'req aborted')
abortController.abort()
})
req.on('close', () => {
log(`server${reqId}`, 'req closed')
abortController.abort()
})
const fetchResp = fetch(vFetchUrl, {
signal: abortController.signal,
headers: vFetchReqHeaders,
method: req.method
})
respondWithVfetch({ req, res, fetchResp, reqId })
})
/**
* Respond to a request with a response from `@helia/verified-fetch`
*
* @param {object} param0
* @param {http.IncomingMessage} param0.req
* @param {Promise<Response>} param0.fetchResp
* @param {number} param0.reqId
* @param {http.ServerResponse<http.IncomingMessage>} param0.res
*/
async function respondWithVfetch ({ req, res, fetchResp, reqId }) {
console.group(`respondWithVfetch${reqId}`)
log(`respondWithVfetch${reqId}`, 'start')
let fetchRespValue
try {
fetchRespValue = await fetchResp
} catch (e) {
log(`server${reqId}`, 'fetchRespValue error', e)
res.writeHead(500, {})
res.end(e)
}
if (fetchRespValue == null) {
log(`server${reqId}`, 'fetchRespValue is null')
res.writeHead(500, {})
res.end('fetchRespValue is null')
return
}
log(`respondWithVfetch${reqId}`, 'got fetchRespValue')
logHeaders(reqId, fetchRespValue.headers)
if (!fetchRespValue.ok) {
log(`server${reqId}`, 'fetchRespValue not ok')
res.writeHead(fetchRespValue.status, {})
res.write(fetchRespValue.statusText)
res.write(await fetchRespValue.text())
res.end()
return
}
/**
* @type {import('node:http').OutgoingHttpHeaders}
*/
const headers = {}
for (const [name, value] of fetchRespValue.headers) {
headers[name] = value
}
res.writeHead(fetchRespValue.status, headers)
log(`respondWithVfetch${reqId}`, 'wrote headers', fetchRespValue.status, headers)
const body = await fetchRespValue.arrayBuffer()
// now convert arrayBuffer to nodejs Buffer
const nodeBuffer = Buffer.from(body)
res.end(nodeBuffer)
console.groupEnd()
}
server.listen(8889).on('listening', () => {
console.log('Server is listening on port 8889')
})
/* eslint-disable no-console */
// @ts-check
import fs from 'node:fs'
import http from 'node:http'
import path from 'node:path'
import url from 'node:url'
const __dirname = path.dirname(url.fileURLToPath(import.meta.url))
/**
*
* @param {http.IncomingMessage} req
* @param {*} reqId
*/
const addLoggingToRequest = (req, reqId) => {
req.on('close', () => {
log(`request${reqId}`, 'closed')
// console.log(`request${reqId} closed`)
})
// don't listen to data event
// req.on('data', (chunk) => {
// console.log(`request${reqId} data: `, chunk)
// })
req.on('end', () => {
log(`request${reqId}`, ' ended')
})
req.on('error', (err) => {
log(`request${reqId}`, ' error: `, er', err)
})
req.on('pause', () => {
log(`request${reqId}`, ' paused')
})
req.on('readable', () => {
log(`request${reqId}`, ' readable')
})
req.on('resume', () => {
log(`request${reqId}`, ' resumed')
})
}
/**
*
* @param {http.ServerResponse<http.IncomingMessage>} res
* @param {number} reqId
*/
const addLoggingToResponse = (res, reqId) => {
res.on('pause', () => {
log(`response${reqId}`, ' paused')
})
res.on('resume', () => {
log(`response${reqId}`, ' resumed')
})
res.on('close', () => {
log(`response${reqId}`, ' closed')
})
res.on('aborted', () => {
log(`response${reqId}`, ' aborted')
})
res.on('error', (err) => {
log(`response${reqId}`, ' error', err)
})
res.on('finish', () => {
log(`response${reqId}`, ' finished')
})
res.on('end', () => {
log(`response${reqId}`, ' ended')
})
res.on('unpipe', () => {
log(`response${reqId}`, ' unpipe')
})
res.on('pipe', () => {
log(`response${reqId}`, ' pipe')
})
}
/**
*
* @param {*} file
* @param {Parameters<fs['createReadStream']>[1]} readStreamOptions
* @param res
* @param reqId
*/
const getStreamedResponse = (file, readStreamOptions, res, reqId) => {
const stream = fs.createReadStream(file, readStreamOptions)
.on('open', function () {
log(`response${reqId}`, ' open')
stream.pipe(res)
}).on('error', function (err) {
log(`response${reqId}`, ' error', err)
res.end(err)
}).on('close', function () {
log(`response${reqId}`, ' closed')
}).on('aborted', function () {
log(`response${reqId}`, ' aborted')
}).on('end', function () {
log(`response${reqId}`, ' ended')
})
}
const log = (id, ...args) => {
console.log(`${id} ${new Date().toISOString().substring(11, 23)}`, ...args)
}
let lastReqId = 0
const server = http.createServer(function (req, res) {
const reqId = lastReqId++
console.group(`request received: ${reqId}`)
console.log(req.url)
console.log(req.method)
console.group('headers: ')
if (req.headers) {
for (const [name, value] of Object.entries(req.headers)) {
console.log(`${name}: ${value}`)
}
}
console.groupEnd()
console.groupEnd()
addLoggingToRequest(req, reqId)
addLoggingToResponse(res, reqId)
const file = path.resolve(__dirname, 'sample.webm')
fs.stat(file, function (err, stats) {
if (err) {
if (err.code === 'ENOENT') {
res.statusCode = 404
res.statusMessage = 'File Not Found'
}
return res.end(err)
}
const range = req.headers.range
if (range == null) {
res.writeHead(200, {
'Content-Length': stats.size,
'Content-Type': 'video/webm'
})
return getStreamedResponse(file, {}, res, reqId)
}
const positions = range.replace(/bytes=/, '').split('-')
const start = parseInt(positions[0], 10)
const total = stats.size
const end = positions[1] ? parseInt(positions[1], 10) : total - 1
const chunksize = (end - start) + 1
res.writeHead(206, {
'Content-Range': 'bytes ' + start + '-' + end + '/' + total,
'Accept-Ranges': 'bytes',
'Content-Length': chunksize,
'Content-Type': 'video/mp4'
})
getStreamedResponse(file, { start, end }, res, reqId)
})
})
server.listen(8889).on('listening', () => {
console.log('Server is listening on port 8889')
})
Server is listening on port 8889
request received: 0
/
GET
headers:
host: localhost:8889
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:123.0) Gecko/20100101 Firefox/123.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
accept-language: en-US,en;q=0.7,ko;q=0.3
accept-encoding: gzip, deflate, br
dnt: 1
connection: keep-alive
upgrade-insecure-requests: 1
sec-fetch-dest: document
sec-fetch-mode: navigate
sec-fetch-site: none
sec-fetch-user: ?1
request0 05:05:56.518 readable
request0 05:05:56.519 ended
request0 05:05:56.519 closed
stream0 05:05:56.521 open
response0 05:05:56.521 pipe
request received: 1
/favicon.ico
GET
headers:
host: localhost:8889
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:123.0) Gecko/20100101 Firefox/123.0
accept: image/avif,image/webp,*/*
accept-language: en-US,en;q=0.7,ko;q=0.3
accept-encoding: gzip, deflate, br
dnt: 1
connection: keep-alive
referer: http://localhost:8889/
sec-fetch-dest: image
sec-fetch-mode: no-cors
sec-fetch-site: same-origin
request1 05:05:57.988 readable
request1 05:05:57.988 ended
request1 05:05:57.988 closed
stream1 05:05:57.988 open
response1 05:05:57.988 pipe
stream1 05:05:58.147 ended
response1 05:05:58.148 finished
response1 05:05:58.148 unpipe
request1 05:05:58.148 resumed
response1 05:05:58.148 closed
stream1 05:05:58.148 closed
Server is listening on port 8889
request received: 0
/
GET
headers:
host: localhost:8889
sec-fetch-site: none
connection: keep-alive
upgrade-insecure-requests: 1
sec-fetch-mode: navigate
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.3.1 Safari/605.1.15
accept-language: en-US,en;q=0.9
sec-fetch-dest: document
accept-encoding: gzip, deflate
request0 05:05:39.184 readable
request0 05:05:39.185 ended
request0 05:05:39.185 closed
stream0 05:05:39.186 open
response0 05:05:39.186 pipe
stream0 05:05:39.348 ended
response0 05:05:39.349 finished
response0 05:05:39.349 unpipe
request0 05:05:39.349 resumed
response0 05:05:39.349 closed
stream0 05:05:39.349 closed
request received: 1
/
GET
headers:
host: localhost:8889
sec-fetch-site: same-origin
x-playback-session-id: 5C757166-B2D5-4EC1-956C-F2E58172D282
accept-encoding: identity
accept-language: en-US,en;q=0.9
sec-fetch-mode: no-cors
accept: */*
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.3.1 Safari/605.1.15
referer: http://localhost:8889/
connection: Keep-Alive
range: bytes=0-1
sec-fetch-dest: video
request1 05:05:39.492 readable
request1 05:05:39.492 ended
request1 05:05:39.492 closed
stream1 05:05:39.493 open
response1 05:05:39.493 pipe
stream1 05:05:39.493 ended
response1 05:05:39.493 finished
response1 05:05:39.493 unpipe
request1 05:05:39.493 resumed
response1 05:05:39.493 closed
stream1 05:05:39.493 closed
request received: 2
/
GET
headers:
host: localhost:8889
sec-fetch-site: same-origin
x-playback-session-id: 5C757166-B2D5-4EC1-956C-F2E58172D282
accept-encoding: identity
accept-language: en-US,en;q=0.9
sec-fetch-mode: no-cors
accept: */*
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.3.1 Safari/605.1.15
referer: http://localhost:8889/
connection: Keep-Alive
range: bytes=0-83216544
sec-fetch-dest: video
request2 05:05:39.499 readable
request2 05:05:39.499 ended
request2 05:05:39.499 closed
stream2 05:05:39.499 open
response2 05:05:39.499 pipe
response2 05:05:39.638 closed
response2 05:05:39.638 unpipe
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment