Skip to content

Instantly share code, notes, and snippets.

@catboxanon

catboxanon/README.md

Last active April 1, 2024 10:37
Show Gist options
  • Star 26 You must be signed in to star a gist
  • Fork 5 You must be signed in to fork a gist
  • Save catboxanon/ca46eb79ce55e3216aecab49d5c7a3fb to your computer and use it in GitHub Desktop.
Save catboxanon/ca46eb79ce55e3216aecab49d5c7a3fb to your computer and use it in GitHub Desktop.
Download ZIP
/hdg/ catbox.moe userscript
Raw
README.md

/hdg/ catbox.moe userscript

This userscript adds functionality to upload to Catbox directly from 4chan, and to view metadata for Stable Diffusion, TavernAI, and NovelAI, in PNGs, JPEGs, and WebPs.

Prerequisites

You will need both a userscript extension and 4chanX.

Many users have reported the Chrome extension for 4chanX does not work with this userscript. This is likely due to the Chrome extension not firing the required Custom Events for the script to function, either due to a bug, the extension not being up-to-date, or some other unknown factor (I haven't bothered testing since I do not use Chrome). However, you can easily migrate to the userscript version of 4chanX by using the Export and Import feature in the top right of the 4chanX settings panel to migrate your configuration.

Installation

Open this URL: https://gist.github.com/raw/ca46eb79ce55e3216aecab49d5c7a3fb/catbox.user.js

If you have an open tab on 4chan, refresh it.

If you do not recieve an install dialog, you do not have a userscript extension, or it is not functioning properly.

Usage

A CATBOX button is added to the quick reply menu. This will upload images to Catbox and modify the filename to contain a URL hint, i.e. catbox_abc123.png.

Drag-and-drop is also supported by holding the control key.

Download links next to the filename will be updated and labeled in green if the filename matches the Catbox URL hint or NovelAI filename. These can be right-clicked to view/copy the Stable Diffusion/TavernAI/NovelAI metadata (if it exists). Bare .jpg, .png, and .webp Catbox links have the same functionality as well and the cursor will change to give a hint they can be right-clicked.

As of version 3.6.0, this userscript now runs for all boards and threads. If you would like to opt-out of this, you can set the STRICT_CHECK value at the top of the userscript to true, or to make this persistent across userscript updates, set the local storage key CATBOX_HDG_STRICT_CHECK value to true. The previous strict configuration is detailed below.

Currently, this is configured to only run on these boards:

/aco/
/b/
/bant/
/d/
/e/
/h/
/g/
/jp/
/pw/
/trash/
/v/
/vg/
/vt/

The threads also must contain one of the following (not case sensitive):

-----------Thread Title-----------
/aids/
/asdg/
/ddg/
/hdg/
/sdg/
/swarm/
/vtai/
/aicg/
>nai leak speedrun
otaku ai art thread

----------Board Specific----------
/bant/: ai waifus general
/jp/:   ai thread
/pw/:   waifu wrestling alliance

-------------OP Text--------------
waifus.nemusona.com
rentry.org/voldy
rentry.co/voldy
github.com/AUTOMATIC1111/stable-diffusion-webui

You can modify the @match, TITLE_CHECK, and OP_CHECK values to change this behavior.

Notes

Pasting is not supported due to different implementations of the paste event in Chrome and Firefox (Chrome can copy files with metadata, Firefox cannot copy files and can only copy images).

NovelAI image detection is limited to the filename to limit download resources and potential abuse (technically NovelAI image uploads break global rule 17).

No support for vanilla 4chan (removing the 4chanX requirement) is planned to be added at this time. I have no interest in implementing it as I would never personally use it.

I also have no support planned for other imageboard sites at this time. If you want to add support for another site, feel free to make a fork and leave a comment if you wish to possibly see it merged with this userscript, no guarantees though.

Changelog

(2024-02-24) 3.7.0 - Support .webp files

(2024-01-02) 3.6.0 - Run for all boards and threads. Opt-out instructions listed in the Usage section.

(2023-12-18) 3.5.0 - Match 4chan.org for all SFW boards now

(2023-12-03) 3.4.8 - Support for desuarchive.org (thanks to Anonymous)

(2023-12-02) 3.4.7 - Adjust NAI filename regex to account for extra whitespace

(2023-11-26) 3.4.6 - Support checking metadata of all files (only filenames that match specific patterns will get the green color)

(2023-11-25) 3.4.5 - Adjust regex to parse NAI filenames with duplicate file extension(s)

(2023-11-22) 3.4.4 - Adjust CSS attribute selector for NAI filenames

(2023-11-21) 3.4.3 - Adjust regex to parse NAI filenames that were saved with same parameters

(2023-11-19) 3.4.2 - Fix parsing NAI metadata when uploaded to Catbox

(2023-11-19) 3.4.1 - Add loading indicator, fix issue with metadata loading fallback not working

(2023-11-18) 3.4.0 - Initial support for NAI image metadata

(2023-08-21) 3.3.0 - Initial archive website support, Catbox login support (use hotkey CTRL+ALT+x)

(2023-06-11) 3.2.10 - Looser restrictions for /jp/ thread matching

(2023-06-07) 3.2.9 - Fix domain for /bant/

(2023-06-06) 3.2.8 - Fix issue where text checks were not done case insensitive

(2023-06-06) 3.2.7 - Additional title checks, support /b/

(2023-05-30) 3.2.6 - Additional title checks, allow for OP text checks, support /bant/, /pw/, /v/. Also bring back the original title. :^)

(2023-05-29) 3.2.5 - Additional title checks, allow for board-specific title checks

(2023-04-16) 3.2.4 - Make CSS selector not case-sensitive for bare links

(2023-04-16) 3.2.3 - Make CSS selector not case-sensitive for filenames

(2023-04-16) 3.2.2 - Support /trash/ board

(2023-03-19) 3.2.1 - Fixed typo that broke parsing

(2023-03-11) Started recording a proper changelog. JPEG EXIF reading support added. Version bumped to 3.2.0

(2023-03-06) There is now experimental support for TavernAI cards. This will likely be expanded upon in the future with more features and if more card formats are released.

Raw
catbox.user.js
// ==UserScript==
// @name 4chan /hdg/ catbox.moe userscript
// @namespace 4chanhdgcatbox
// @match https://boards.4chan.org/*/thread/*
// @match https://archiveofsins.com/*
// @match https://desuarchive.org/*
// @grant GM.xmlHttpRequest
// @grant GM_xmlhttpRequest
// @version 3.7.0
// @author Anonymous
// @description Upload image directly to catbox.moe from 4chan
// @updateURL https://gist.github.com/raw/ca46eb79ce55e3216aecab49d5c7a3fb/catbox.user.js
// @downloadURL https://gist.github.com/raw/ca46eb79ce55e3216aecab49d5c7a3fb/catbox.user.js
// @require https://cdn.jsdelivr.net/npm/exif-js
// @require https://cdn.jsdelivr.net/npm/exifreader@4.21.0/dist/exif-reader.min.js
// @require https://cdn.jsdelivr.net/npm/pako@2.1.0/dist/pako.min.js
// ==/UserScript==
(async function () {
const STRICT_CHECK = false || JSON.parse(localStorage['CATBOX_HDG_STRICT_CHECK'] || false);
const TITLE_CHECK = [
'/aids/', '/asdg/', '/ddg/', '/hdg/', '/sdg/', '/swarm/', '/vtai/',
'/aicg/',
'>nai leak speedrun', 'otaku ai art thread',
['/jp/', ' ai thread '], ['/jp/', ' ai art '], ['/pw/', 'waifu wrestling alliance'], ['/bant/', 'ai waifus general']
];
const OP_CHECK = [
'waifus.nemusona.com', 'rentry.org/voldy', 'rentry.co/voldy', 'github.com/AUTOMATIC1111/stable-diffusion-webui'
];
const ARCHIVE_CHECK = [
'archiveofsins.com', 'desuarchive.org'
];
const CATBOX_BUTTON_ID = 'qr-catbox-button_userscript';
const RE_CATBOX_URL = /^https?:\/\/files\.catbox\.moe\/([a-z0-9]{6}\.(?:png|jpg|webp))$/i;
const RE_CATBOX_FILENAME = /^catbox_[a-z0-9]{6}\.(?:png|jpg|webp)$/i;
const RE_NAI_FILENAME = /^.*s\-\d+\s*(?:\(\d+\))?(\.png)+\s*$/;
let loaded = false;
let thread_match = false;
let qr_updated = false;
const posts = new Set();
function log(msg) {
console.log(`[4chan /sdg/ catbox.moe userscript] ${msg}`);
}
function getXmlHttpRequest() {
return (typeof GM !== "undefined" && GM !== null ? GM.xmlHttpRequest : GM_xmlhttpRequest);
}
function get(url) {
return new Promise((resolve, reject) => {
getXmlHttpRequest()({
method: 'GET',
url: url,
timeout: 15000,
responseType: 'blob',
onload: function(response) {
resolve(response);
},
onerror: function(error) {
reject(error);
},
ontimeout: function(error) {
reject(error);
}
});
});
}
function post(url, data) {
return new Promise((resolve, reject) => {
getXmlHttpRequest()({
method: 'POST',
url: url,
data: data,
timeout: 15000,
onload: function(response) {
resolve(response);
},
onerror: function(error) {
reject(error);
},
ontimeout: function(error) {
reject(error);
}
});
});
}
function setFontColor(elm, color) {
elm.setAttribute('style', elm.getAttribute('style').replace(/(.+color: )[a-z]+( !.+)/i, `$1${color}$2`));
}
function setCatboxAuth() {
const userhash = window.prompt('Catbox auth hotkey triggered. Please enter your userhash to store your login. Input nothing to remove it.');
localStorage['CATBOX_USERHASH'] = JSON.stringify(userhash || '');
}
function getCatboxAuth() {
const res = JSON.parse(localStorage['CATBOX_USERHASH'] || '{}');
return (typeof res === 'object' ? '' : res || '');
}
async function uploadToCatbox(file) {
const catboxButton = document.querySelector(`#${CATBOX_BUTTON_ID}`);
const submitButton = document.querySelector('#file-n-submit [type=submit][value=Submit]');
const formData = new FormData();
formData.append('reqtype', 'fileupload');
formData.append('fileToUpload', file, file.name);
const userhash = getCatboxAuth();
if (userhash) {
formData.append('userhash', userhash);
}
catboxButton.value = 'Uploading ...';
setFontColor(catboxButton, 'yellow');
// Selector doesn't seem to grab the submit button all the time for some reason
if (submitButton) {
submitButton.disabled = true;
submitButton.style.pointerEvents = 'none';
submitButton.style.opacity = 0.25;
}
try {
log('attempting catbox upload');
const response = await post('https://catbox.moe/user/api.php', formData);
if (response.status === 200 && response.responseText.match(RE_CATBOX_URL)) {
const filenameMatch = response.responseText.match(RE_CATBOX_URL);
log('uploaded');
catboxButton.value = 'uploaded';
setFontColor(catboxButton, 'limegreen');
const filenameInput = document.querySelector('#qr-filename');
const fileEvent = new CustomEvent('QRSetFile', {
detail: {
file: file
}
});
document.dispatchEvent(fileEvent);
filenameInput.value = `catbox_${filenameMatch[1]}`;
filenameInput.dispatchEvent(new Event('input', {bubbles:true}));
} else {
log('upload error');
console.error(response);
catboxButton.value = 'upload error';
setFontColor(catboxButton, 'red');
}
} catch(err) {
log('upload failed');
console.error(err);
catboxButton.value = 'upload failed';
setFontColor(catboxButton, 'red');
}
if (submitButton) {
submitButton.disabled = false;
submitButton.style.pointerEvents = '';
submitButton.style.opacity = 1;
}
setTimeout(() => {
catboxButton.value = 'catbox';
setFontColor(catboxButton, 'inherit');
}, 2000);
}
const loadImage = (blob) => {
return new Promise((resolve, reject) => {
const image = new Image();
image.onload = () => resolve(image);
image.onerror = reject;
image.src = URL.createObjectURL(blob);
});
}
function parseNaiMetadata(metadata) {
metadata = JSON.parse(metadata);
return JSON.stringify(metadata, null, '\t');
}
async function handleMetadataReq(evt, el, href) {
evt.preventDefault();
const box = insertPromptBox(el);
let metadata, chunks;
const hostedOn4chan = el.href.includes('4cdn.org');
if (hostedOn4chan) {
const res = await get(href);
if (res.status != 200) {
return;
}
const image = await loadImage(res.response);
metadata = readInfoFromImageStealth(image);
if (metadata) {
try {
let naiMetadata = JSON.parse(metadata);
metadata = parseNaiMetadata(naiMetadata['Comment']);
} catch {
;
}
}
} else {
const res = await fetch(href);
if (!res.ok || !res.body) {
return;
}
const reader = res.body.getReader();
chunks = [];
let iterCount = 0;
while (true) {
const {done, value} = await reader.read();
if (done || iterCount > 10) {
break;
}
chunks.push(value);
iterCount++;
}
if (href.endsWith('.jpg')) {
let rawJpegMetdata = chunksToArray(chunks);
rawJpegMetdata = EXIF.readFromBinaryFile(rawJpegMetdata.buffer.slice(rawJpegMetdata.byteOffset, rawJpegMetdata.byteLength + rawJpegMetdata.byteOffset));
if (Object.keys(rawJpegMetdata).includes('UserComment')) {
const jpegMetadata = String.fromCharCode(...rawJpegMetdata.UserComment.slice(9).filter((value) => value !== 0));
updatePromptBox(box, jpegMetadata);
}
return;
}
if (href.endsWith('.webp')) {
let rawWebpMetadata = chunksToArray(chunks)?.buffer;
rawWebpMetadata = ExifReader.load(rawWebpMetadata);
if (Object.keys(rawWebpMetadata).includes('UserComment')) {
const webpMetadata = String.fromCharCode(...rawWebpMetadata.UserComment.value.slice(9).filter((value) => value !== 0));
updatePromptBox(box, webpMetadata);
}
return;
}
metadata = await getMetaData(chunks[0]);
}
if (metadata) {
updatePromptBox(box, metadata, hostedOn4chan || metadata.includes('"prompt": "'));
return;
} else if (!hostedOn4chan) {
const fallbackMetadata = await getMetaData(chunksToArray(chunks));
if (fallbackMetadata) {
updatePromptBox(box, fallbackMetadata);
return;
}
}
updatePromptBox(box, 'No metadata found.');
}
function updateLinkHover(link) {
link.setAttribute('style', 'color: limegreen !important;');
link.addEventListener('mouseenter', (evt) => {evt.target.style.filter = 'brightness(250%)'});
link.addEventListener('mouseleave', (evt) => {evt.target.style.filter = 'none'});
}
function updateFilenameLink(link) {
const name = link.getAttribute('download')?.split('_')[1] || link.getAttribute('title')?.split('_')[1];
if (!name) {
log('Failed to parse catbox link filename');
return;
}
const href = `https://files.catbox.moe/${name}`;
log(`parsed catbox link: ${href}`);
link.href = href;
updateLinkHover(link);
}
async function updateDownloadLinks(root=null, limit=100000) {
root = root !== null ? root : document;
const skipExisting = (root == document);
function getPostId(el) {
return el.closest('[data-full-i-d]').getAttribute('data-full-i-d').replace(/\D/g, '');
}
const downloadLinks = Array.from(root.querySelectorAll('.file-info a[href*="4cdn.org"][download^="catbox_" i]')).slice(0, limit);
const naiLinks = Array.from(root.querySelectorAll('a:is([download*="s-1"],[download*="s-2"],[download*="s-3"],[download*="s-4"],[download*="s-5"],[download*="s-6"],[download*="s-7"],[download*="s-8"],[download*="s-9"])[download$="png"]')).slice(0, limit);
const fileDownloadLinks = Array.from(root.querySelectorAll('.file-info a.download-button[href]:not([href*=".catbox.moe/"]):not([download^="catbox_" i])')).slice(0, limit);
for (const link of [...downloadLinks, ...naiLinks]) {
if (skipExisting && posts.has(getPostId(link))) {
continue;
}
if (link.getAttribute('download').match(RE_CATBOX_FILENAME)) {
updateFilenameLink(link);
continue;
}
if (link.getAttribute('download').match(RE_NAI_FILENAME)) {
updateLinkHover(link);
}
}
const catboxLinks = Array.from(root.querySelectorAll('a[href*=".catbox.moe/"]:is([href$=".png" i], [href$=".jpg" i], [href$=".webp" i])')).slice(0, limit);
for (const link of [...catboxLinks, ...fileDownloadLinks]) {
if (skipExisting && posts.has(getPostId(link))) {
continue;
}
log(`found existing link: ${link.href}`);
link.style.cursor = 'help';
link.title = 'Right click to attempt to load & show/hide metadata';
link.addEventListener('contextmenu', async (evt) => {await handleMetadataReq(evt, link, link.href)});
}
if (skipExisting) {
for (const link of [...catboxLinks, ...fileDownloadLinks]) {
posts.add(getPostId(link));
}
}
}
function createMetadataBoxButton(name, desc, rgb, idx=0) {
const btn = document.createElement('div');
btn.title = desc;
btn.classList.add(`catbox-prompt-${name}_userscript`);
btn.style.position = 'absolute';
btn.style.width = '16px';
btn.style.height = '16px';
btn.style.backgroundColor = `rgba(${rgb[0]},${rgb[1]},${rgb[2]},0.5)`;
btn.style.lineHeight = '16px';
btn.style.textAlign = 'center';
btn.style.marginTop = '-8px';
btn.style.marginLeft = '-8px';
btn.style.textIndent = '0px';
btn.style.userSelect = 'none';
btn.style.cursor = 'pointer';
btn.style.color = 'white';
btn.style.left = `${((idx + 1) * 16) + (2 * (idx > 0 ? 1 : 0))}px`;
btn.style.top = '16px';
btn.innerText = name.charAt(0).toUpperCase();
btn.addEventListener('mouseenter', (evt) => {evt.target.style.backgroundColor = `rgba(${rgb[0]},${rgb[1]},${rgb[2]},0.8)`});
btn.addEventListener('mouseleave', (evt) => {evt.target.style.backgroundColor = `rgba(${rgb[0]},${rgb[1]},${rgb[2]},0.5)`});
return btn;
}
function insertPromptBox(el) {
const boxClass = 'catbox-prompt_userscript';
// const blockquote = document.querySelector(`#m${el.closest('.postContainer[data-full-i-d]').getAttribute('data-full-i-d').split('h.')[1]}`);
const blockquote = el.closest(':is(.reply, .op)').querySelector('blockquote');
const exists = blockquote.querySelector(`.${boxClass}`);
if (exists) {
exists.remove();
return;
}
const box = document.createElement('div');
box.classList.add('catbox-prompt_userscript');
box.style.display = 'grid';
box.style.position = 'relative';
box.style.color = 'white';
box.style.backgroundColor = 'rgba(0,0,255,0.2)';
box.style.border = '2px solid rgba(255,255,255,0.2)';
box.style.borderStyle = 'dashed';
box.style.padding = '8px';
box.style.paddingTop = '16px;'
box.style.marginBottom = '16px';
box.style.whiteSpace = 'pre-wrap';
box.style.textIndent = '48px';
box.style.maxHeight = '480px';
box.style.overflowY = 'auto';
box.innerText = 'Attempting to load metadata...';
blockquote.prepend(box);
return box;
}
function updatePromptBox(box, metadata, jsonDisplay = false) {
if (jsonDisplay) {
box.style.fontFamily = 'monospace';
}
box.innerText = metadata;
const closeBtn = createMetadataBoxButton('x', 'Hide', [255,0,0], 0);
closeBtn.addEventListener('click', () => {box.remove()});
const copyBtn = createMetadataBoxButton('copy', 'Copy', [0,127,255], 1);
copyBtn.addEventListener('click', async (evt) => {
await navigator.clipboard.writeText(metadata);
copyBtn.style.backgroundColor = 'rgba(0,255,0,0.5)';
});
box.appendChild(closeBtn);
box.appendChild(copyBtn);
}
async function updateQrWindow() {
if (qr_updated) {
log('qr window already updated');
return;
}
const fileButton = document.querySelector('#qr-file-button');
const catboxButton = fileButton.cloneNode();
catboxButton.id = CATBOX_BUTTON_ID;
catboxButton.value = 'Catbox';
catboxButton.setAttribute('style', 'text-transform: uppercase; font-size: 10px !important; border-radius: 2px !important; color: inherit !important;');
fileButton.parentNode.insertBefore(catboxButton, fileButton.nextSibling);
catboxButton.addEventListener('click', async (evt) => {
const input = document.createElement('input');
input.setAttribute('type', 'file');
input.setAttribute('style', 'display: none !important;');
input.addEventListener('change', async () => {
if (input.files.length > 0) {
await uploadToCatbox(input.files[0]);
}
input.remove();
});
document.body.appendChild(input);
input.click();
});
qr_updated = true;
}
function textCheck(inp, arr) {
for (let title of arr) {
if (Array.isArray(title)) {
if (title.every(substr => inp.includes(substr.toLowerCase()))) {
return true;
}
} else if (inp.indexOf(title.toLowerCase()) != -1) {
return true;
}
}
return false;
}
async function init() {
if (loaded) {
return;
}
loaded = true;
log('initialized userscript');
const pageTitle = document.title.toLowerCase();
const opText = document.querySelector('.post.op .postMessage')?.innerText.toLowerCase() || '';
thread_match = !STRICT_CHECK || textCheck(pageTitle, TITLE_CHECK) || textCheck(opText, OP_CHECK);
if (!thread_match) {
log('Thread does not match criteria, short-circuiting');
return;
}
document.addEventListener('keydown', (evt) => {
if (evt.key == "x" && evt.ctrlKey && evt.altKey) {
setCatboxAuth();
}
});
if (document.querySelector('#qr') && !qr_updated) {
log('qr window already exists -- updating qr window');
await updateQrWindow();
}
window.addEventListener('drop', async (evt) => {
log('drop event');
if (!evt.dataTransfer.files.length || !evt.ctrlKey) {
return;
}
evt.preventDefault();
uploadToCatbox(evt.dataTransfer.files[0]);
});
document.addEventListener('QRDialogCreation', async () => {
log('QR dialog creation event -- attempting to update qr window');
await updateQrWindow();
});
await updateDownloadLinks();
document.addEventListener('PostsInserted', async (evt) => {
if (evt.target.hasAttribute('data-full-i-d')) {
await updateDownloadLinks(evt.target);
} else {
await updateDownloadLinks();
}
});
}
document.addEventListener('4chanXInitFinished', async () => {
log('4chanX init finished');
await init();
});
// 4chanXInitFinished event doesn't seem to fire all the time
window.addEventListener('load', async () => {
if (ARCHIVE_CHECK.includes(window.location.hostname)) {
return;
} else {
setTimeout(async () => {
if (!loaded) {
log('4chanX init never received, using fallback');
await init();
}
}, 5000)
}
});
function fillArchiveWithLinks() {
if (ARCHIVE_CHECK.includes(window.location.hostname)) {
log('Archive website detected');
const catboxLinks = document.querySelectorAll('a.btnr[download^="catbox_" i]');
for (const link of catboxLinks) {
updateFilenameLink(link);
}
}
}
if (document.readyState !== 'loading') {
fillArchiveWithLinks();
} else {
document.addEventListener('DOMContentLoaded', async () => { fillArchiveWithLinks(); })
}
function chunksToArray(inp) {
let data = inp;
if (Array.isArray(inp)) {
let length = 0;
inp.forEach(item => {
length += item.length;
});
data = new Uint8Array(length);
let offset = 0;
inp.forEach(item => {
data.set(item, offset);
offset += item.length;
});
return data;
} else {return inp}
}
/* -----------------------------------------------------------------
/* https://github.com/moonshinegloss/stable-diffusion-discord-prompts
/* ----------------------------------------------------------------- */
// Used for fast-ish conversion between uint8s and uint32s/int32s.
// Also required in order to remain agnostic for both Node Buffers and
// Uint8Arrays.
let uint8 = new Uint8Array(4)
let int32 = new Int32Array(uint8.buffer)
let uint32 = new Uint32Array(uint8.buffer)
const RESOLUTION_UNITS = {UNDEFINED: 0, METERS: 1, INCHES: 2};
/**
* https://github.com/hughsk/png-chunk-text
* Reads a Uint8Array or Node.js Buffer instance containing a tEXt PNG chunk's data and returns its keyword/text:
* @param data
* @returns {{text: string, keyword: string}}
*/
function textDecode (data) {
if (data.data && data.name) {
data = data.data
}
let naming = true
let text = ''
let name = ''
for (let i = 0; i < data.length; i++) {
let code = data[i]
if (naming) {
if (code) {
name += String.fromCharCode(code)
} else {
naming = false
}
} else {
if (code) {
text += String.fromCharCode(code)
}
}
}
return {
keyword: name,
text: text
}
}
/**
* https://github.com/hughsk/png-chunks-extract
* Extract the data chunks from a PNG file.
* Useful for reading the metadata of a PNG image, or as the base of a more complete PNG parser.
* Takes the raw image file data as a Uint8Array or Node.js Buffer, and returns an array of chunks. Each chunk has a name and data buffer:
* @param data {Uint8Array}
* @returns {[{name: String, data: Uint8Array}]}
*/
function extractChunks (data) {
if (data[0] !== 0x89) throw new Error('Invalid .png file header')
if (data[1] !== 0x50) throw new Error('Invalid .png file header')
if (data[2] !== 0x4E) throw new Error('Invalid .png file header')
if (data[3] !== 0x47) throw new Error('Invalid .png file header')
if (data[4] !== 0x0D) throw new Error('Invalid .png file header: possibly caused by DOS-Unix line ending conversion?')
if (data[5] !== 0x0A) throw new Error('Invalid .png file header: possibly caused by DOS-Unix line ending conversion?')
if (data[6] !== 0x1A) throw new Error('Invalid .png file header')
if (data[7] !== 0x0A) throw new Error('Invalid .png file header: possibly caused by DOS-Unix line ending conversion?')
let ended = false
let chunks = []
let idx = 8
while (idx < data.length) {
// Read the length of the current chunk,
// which is stored as a Uint32.
uint8[3] = data[idx++]
uint8[2] = data[idx++]
uint8[1] = data[idx++]
uint8[0] = data[idx++]
// Chunk includes name/type for CRC check (see below).
let length = uint32[0] + 4
let chunk = new Uint8Array(length)
chunk[0] = data[idx++]
chunk[1] = data[idx++]
chunk[2] = data[idx++]
chunk[3] = data[idx++]
// Get the name in ASCII for identification.
let name = (
String.fromCharCode(chunk[0]) +
String.fromCharCode(chunk[1]) +
String.fromCharCode(chunk[2]) +
String.fromCharCode(chunk[3])
)
// The IHDR header MUST come first.
if (!chunks.length && name !== 'IHDR') {
throw new Error('IHDR header missing')
}
// The IEND header marks the end of the file,
// so on discovering it break out of the loop.
if (name === 'IEND') {
ended = true
chunks.push({
name: name,
data: new Uint8Array(0)
})
break
}
// Read the contents of the chunk out of the main buffer.
for (let i = 4; i < length; i++) {
chunk[i] = data[idx++]
}
// Read out the CRC value for comparison.
// It's stored as an Int32.
uint8[3] = data[idx++]
uint8[2] = data[idx++]
uint8[1] = data[idx++]
uint8[0] = data[idx++]
// The chunk data is now copied to remove the 4 preceding
// bytes used for the chunk name/type.
let chunkData = new Uint8Array(chunk.buffer.slice(4))
chunks.push({
name: name,
data: chunkData
})
}
return chunks
}
/**
* read 4 bytes number from UInt8Array.
* @param uint8array
* @param offset
* @returns {number}
*/
function readUint32 (uint8array,offset) {
let byte1, byte2, byte3, byte4;
byte1 = uint8array[offset++];
byte2 = uint8array[offset++];
byte3 = uint8array[offset++];
byte4 = uint8array[offset];
return 0 | (byte1 << 24) | (byte2 << 16) | (byte3 << 8) | byte4;
}
/**
* Get object with PNG metadata. only tEXt and pHYs chunks are parsed
* @param buffer {Buffer}
* @returns {{tEXt: {keyword: value}, pHYs: {x: number, y: number, units: RESOLUTION_UNITS}, [string]: true}}
*/
function readMetadata(buffer){
let result = {};
const chunks = extractChunks(buffer);
chunks.forEach( chunk => {
switch(chunk.name){
case 'tEXt':
if (!result.tEXt) {
result.tEXt = {};
}
let textChunk = textDecode(chunk.data);
result.tEXt[textChunk.keyword] = textChunk.text;
break
case 'pHYs':
result.pHYs = {
// Pixels per unit, X axis: 4 bytes (unsigned integer)
"x": readUint32(chunk.data, 0),
// Pixels per unit, Y axis: 4 bytes (unsigned integer)
"y": readUint32(chunk.data, 4),
"unit": chunk.data[8],
}
break
case 'gAMA':
case 'cHRM':
case 'sRGB':
case 'IHDR':
case 'iCCP':
default:
result[chunk.name] = true;
}
})
return result;
}
function largeuint8ArrToString(uint8arr) {
return new Promise((resolve) => {
const f = new FileReader();
f.onload = function(e) {
resolve(e.target.result);
}
f.readAsText(new Blob([uint8arr]));
})
}
function imageHasAlpha (context, canvas) {
var data = context.getImageData(0, 0, canvas.width, canvas.height).data,
hasAlphaPixels = false;
for (var i = 3, n = data.length; i < n; i+=4) {
if (data[i] < 255) {
hasAlphaPixels = true;
break;
}
}
return hasAlphaPixels;
}
function readInfoFromImageStealth(image) {
let geninfo, items, paramLen;
let r, g, b, a;
const canvas = document.createElement('canvas');
// trying to read stealth pnginfo
const [width, height] = [image.width, image.height];
const context = canvas.getContext('2d');
canvas.width = image.width;
canvas.height = image.height;
context.drawImage(image, 0, 0);
const imageData = context.getImageData(0, 0, width, height);
const data = imageData.data;
let hasAlpha = imageHasAlpha(context, canvas);
let mode = null;
let compressed = false;
let binaryData = '';
let bufferA = '';
let bufferRGB = '';
let indexA = 0;
let indexRGB = 0;
let sigConfirmed = false;
let confirmingSignature = true;
let readingParamLen = false;
let readingParam = false;
let readEnd = false;
for (let x = 0; x < width; x++) {
for (let y = 0; y < height; y++) {
let i = (y * width + x) * 4;
if (hasAlpha) {
[r, g, b, a] = data.slice(i, i+4);
bufferA += (a & 1).toString();
indexA++;
} else {
[r, g, b] = data.slice(i, i+3);
}
bufferRGB += (r & 1).toString();
bufferRGB += (g & 1).toString();
bufferRGB += (b & 1).toString();
indexRGB += 3;
if (confirmingSignature) {
if (indexA === 'stealth_pnginfo'.length * 8) {
const decodedSig = new TextDecoder().decode(new Uint8Array(bufferA.match(/\d{8}/g).map(b => parseInt(b, 2))));
if (decodedSig === 'stealth_pnginfo' || decodedSig === 'stealth_pngcomp') {
confirmingSignature = false;
sigConfirmed = true;
readingParamLen = true;
mode = 'alpha';
if (decodedSig === 'stealth_pngcomp') {
compressed = true;
}
bufferA = '';
indexA = 0;
} else {
readEnd = true;
break;
}
} else if (indexRGB === 'stealth_pnginfo'.length * 8) {
const decodedSig = new TextDecoder().decode(new Uint8Array(bufferRGB.match(/\d{8}/g).map(b => parseInt(b, 2))));
if (decodedSig === 'stealth_rgbinfo' || decodedSig === 'stealth_rgbcomp') {
confirmingSignature = false;
sigConfirmed = true;
readingParamLen = true;
mode = 'rgb';
if (decodedSig === 'stealth_rgbcomp') {
compressed = true;
}
bufferRGB = '';
indexRGB = 0;
}
}
} else if (readingParamLen) {
if (mode === 'alpha' && indexA === 32) {
paramLen = parseInt(bufferA, 2);
readingParamLen = false;
readingParam = true;
bufferA = '';
indexA = 0;
} else if (mode != 'alpha' && indexRGB === 33) {
paramLen = parseInt(bufferRGB.slice(0, -1), 2);
readingParamLen = false;
readingParam = true;
bufferRGB = bufferRGB.slice(-1);
indexRGB = 1;
}
} else if (readingParam) {
if (mode === 'alpha' && indexA === paramLen) {
binaryData = bufferA;
readEnd = true;
break;
}
else if (mode != 'alpha' && indexRGB >= paramLen) {
binaryData = bufferRGB.slice(0, paramLen - indexRGB);
readEnd = true;
break;
}
} else {
// Impossible
readEnd = true;
break;
}
}
if (readEnd) {
break;
}
}
if (sigConfirmed && binaryData) {
// Convert binary string to UTF-8 encoded text
const byteData = new Uint8Array(binaryData.match(/\d{8}/g).map(b => parseInt(b, 2)));
let decodedData;
if (compressed) {
decodedData = pako.inflate(byteData, {to: 'string'});
} else {
decodedData = new TextDecoder().decode(byteData);
}
geninfo = decodedData;
}
return geninfo;
}
async function getMetaData(chunks) {
let meta
try{
meta = readMetadata(chunks)
}catch(_){}
if (meta?.tEXt?.Comment && meta?.tEXt?.Description && meta?.tEXt?.Software) {
return parseNaiMetadata(meta.tEXt.Comment);
}
if(meta?.tEXt?.Dream) {
return `${meta?.tEXt?.Dream} ${meta?.tEXt?.['sd-metadata'] || ''}`
}else if(meta?.tEXt?.parameters) {
return meta?.tEXt?.parameters
}else if(meta?.tEXt?.prompt && meta?.tEXt?.workflow) {
return `prompt\n \n${meta?.tEXt?.prompt}\n \nworkflow\n \n${meta?.tEXt?.workflow}`;
} else if(meta?.tEXt?.chara) {
let charaDef = atob(meta?.tEXt?.chara);
let charaDefJson = JSON.parse(charaDef);
if (charaDefJson && ['name', 'description', 'mes_example', 'first_mes'].every(val => Object.keys(charaDefJson).includes(val))) {
return `Name: ${charaDefJson['name']}\n \nDescription: ${charaDefJson['description']}\n \nMessage example: ${charaDefJson['mes_example']}\n \nFirst message: ${charaDefJson['first_mes']}`;
}
}
// fallback to simple text extraction
const textData = await largeuint8ArrToString(chunks)
const textTypes = ["Dream","parameters"]
if(textData.includes("IDAT") && textTypes.some(x => textData.includes(x))) {
const result = textData.split("IDAT")[0]
.replace(new RegExp(`[\\s\\S]+Xt(${textTypes.join('|')})`),"")
.replace(/[^\x00-\x7F]/g,"")
if(result.length > 50) return result
}
return false;
}
})();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment