Skip to content

Instantly share code, notes, and snippets.

@samaybhavsar
Created August 30, 2022 14:34
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save samaybhavsar/97d2674536c6f64de8b6d2c43085a347 to your computer and use it in GitHub Desktop.
Save samaybhavsar/97d2674536c6f64de8b6d2c43085a347 to your computer and use it in GitHub Desktop.
Reading stream from "fetch" return chunks of random size, this function transforms the same to chunks of a given size in the browser
// Reading stream from "fetch" return chunks of random size, this function transforms the same to chunks of a given size.
import { decrypt } from "./utils";
export const ChunkerTransformStream = (chunkSize, encryptionKey) => {
chunkSize = chunkSize || 1024 * 1024 * 5;
const SERVER_ENCRYPTION_IV_LENGTH = 12;
const AES_GCM_TAG_SIZE = 16
chunkSize = chunkSize + SERVER_ENCRYPTION_IV_LENGTH + AES_GCM_TAG_SIZE;
let chunk = new Uint8Array();
const transform = async (newChunk, controller) => {
const freeSpaceLength = chunkSize - chunk.length;
if (newChunk.length > freeSpaceLength) {
chunk = Uint8Array.from([...chunk, ...newChunk.slice(0, freeSpaceLength)])
let decryptedData = await decrypt(chunk.buffer, encryptionKey);
controller.enqueue(decryptedData);
chunk = new Uint8Array();
chunk = newChunk.slice(freeSpaceLength)
} else {
chunk = Uint8Array.from([...chunk, ...newChunk])
if (chunk.length == chunkSize) {
let decryptedData = await decrypt(chunk.buffer, encryptionKey);
controller.enqueue(decryptedData);
chunk = new Uint8Array(0);
}
}
}
const flush = async (controller) => {
if (chunk.length != 0) {
let decryptedData = await decrypt(chunk.buffer, encryptionKey);
controller.enqueue(decryptedData);
}
controller.terminate();
}
var chunker = new TransformStream({ transform, flush });
return chunker;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment