Skip to content

Instantly share code, notes, and snippets.

@gildas-lormeau
Created September 12, 2021 20:24
Show Gist options
  • Save gildas-lormeau/731485821c9237b638d01bb4074a6369 to your computer and use it in GitHub Desktop.
Save gildas-lormeau/731485821c9237b638d01bb4074a6369 to your computer and use it in GitHub Desktop.
Basic unzip implementation based on zip.js and Deno
#!/bin/sh
~/.deno/bin/deno run --allow-net --allow-write --allow-read --unstable unzip.js "$@"
/* eslint-disable no-console */
/* global Deno, Intl */
"use strict";
import { parse } from "https://deno.land/std/flags/mod.ts";
import { exists } from "https://deno.land/std/fs/mod.ts";
import { basename, dirname } from "https://deno.land/std/path/mod.ts";
import { ZipReader, HttpReader, Uint8ArrayReader, Uint8ArrayWriter, ERR_HTTP_RANGE, terminateWorkers } from "https://raw.githubusercontent.com/gildas-lormeau/zip.js/master/index.js";
unzip(parse(Deno.args)).catch(error => console.error(error.toString()));
async function unzip(args) {
if (args.l) {
await listEntries(args.l || args._[0]);
} else {
const archive = args._.shift();
if (archive) {
await unzipEntries(archive, args._);
}
}
}
async function unzipEntries(archive, filenames) {
const zipReader = new ZipReader(await getReader(archive));
const entries = await zipReader.getEntries();
let selectedEntries;
if (filenames.length) {
selectedEntries = entries.filter(entry => filenames.includes(entry.filename));
} else {
selectedEntries = entries;
}
await Promise.all(selectedEntries.map(async entry => {
const entryDirectory = dirname(entry.filename);
if (!await exists(entryDirectory)) {
await Deno.mkdir(entryDirectory, { recursive: true });
}
if (!entry.directory) {
await Deno.writeFile(entry.filename, await entry.getData(new Uint8ArrayWriter()));
}
}));
await terminateWorkers();
}
async function listEntries(archive) {
const zipReader = new ZipReader(await getReader(archive));
const entries = await zipReader.getEntries();
let totalSize = 0;
console.log("Archive: ", archive);
let maxNameLength = 0;
const formattedData = entries.map(entry => {
const length = formatLength(entry.uncompressedSize);
const splitDate = entry.lastModDate.toISOString().split("T");
const date = splitDate[0].padStart(11);
const time = splitDate[1].match(/([^:]+:[^:]+):/)[1].padEnd(7);
const name = entry.filename;
totalSize += entry.uncompressedSize;
maxNameLength = Math.max(maxNameLength, length.length);
return { length, date, time, name };
});
console.log("Length".padStart(maxNameLength - 1, " "), " Date Time Name");
const lengthSeparator = "-".padStart(maxNameLength, "-");
console.log(lengthSeparator, " ---------- ----- ----");
formattedData.forEach(({ length, date, time, name }) => console.log(length.padStart(maxNameLength), date, time, name));
console.log(lengthSeparator, " ----");
console.log(formatLength(totalSize));
}
function formatLength(length) {
return new Intl.NumberFormat().format(length);
}
async function getReader(archive) {
if (/^https?:/.test(archive)) {
try {
return new HttpReader(archive, { useRangeHeader: true, forceRangeRequests: true });
} catch (error) {
if (error.message == ERR_HTTP_RANGE) {
try {
return new HttpReader(archive, { useRangeHeader: true });
} catch (error) {
if (error.message == ERR_HTTP_RANGE) {
return new HttpReader(archive);
} else {
throw error;
}
}
} else {
throw error;
}
}
} else {
if (!basename(archive).includes(".")) {
archive += ".zip";
}
return new Uint8ArrayReader(await Deno.readFile(archive));
}
}
@guest271314
Copy link

error: invalid utf-8 sequence of 1 bytes from index 16

@gildas-lormeau
Copy link
Author

gildas-lormeau commented Oct 28, 2023

No stack trace? This error is not thrown by zip.js code.

@guest271314
Copy link

No stack trace. That happened a few days ago while experimenting with unzip.js.

@gildas-lormeau
Copy link
Author

Maybe you're trying to unzip a file with names not encoded in UTF-8?

@guest271314
Copy link

Unfortunately I didn't save exactly what I was doing, and the OS I was testing on is no more. I was probably extracting the contents of the .zip file of Chrome. I'll try again in a few minutes.

@guest271314
Copy link

Just tested again. The code works as expected.

@gildas-lormeau
Copy link
Author

gildas-lormeau commented Oct 28, 2023

That's strange... I'll search on Google the error to see if I can find any clue.

@guest271314
Copy link

I ordinarily try to save the experiments and tests I do. I'm running a live Linux distribution, so I could easily lose all of the tests I run if something goes wrong. I've been trying to fetch, extract, and keep the executable permission of all othe files in the ZIP file that Chrome/Chromium/Chrome-For-Testing includes using only Web API's for the last few days, so you might not find comparable examples in the wild. I'm not sure if anybody is trying to do this to the degree you will find reasons for errors I encounter. I do a modicum of experimentation and testing that may or may not be within the scope of official Web API's, or extension use cases.

@guest271314
Copy link

I'll probably wind up creating a Native Messaging host using your code here, so we can launch and control this for fetching and extracting Chrome/Chromium/Chrome-For-Testing from an arbitrary Web page.

@guest271314
Copy link

Might in fact be something you are doing in the code.

Your code should work like this, and not hang

var {readable, writable} = new TransformStream();
new Blob(['abc']).stream().pipeTo(writable);
try {
  console.log(await new Response(readable).blob());
} catch (e) {
  console.error(e);
}

@guest271314
Copy link

What is parse() used for? What is expected to be passed for listEntries() to be called in unzip()?

@gildas-lormeau
Copy link
Author

parse() is used to parse query parameters, it's a standard module in Deno. When you call the script with "-l", it will list entries of a ZIP read from the filesystem or from a URL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment