Skip to content

Instantly share code, notes, and snippets.

View drdaxxy's full-sized avatar

Niklas K. drdaxxy

  • Hannover, Lower Saxony, Germany
View GitHub Profile

Typical approaches to training, and sampling from, denoising diffusion models yield results whose per-item means match the initial input - i.e. zero when using i.i.d. samples from a standard normal distribution. This has major implications for what outputs can be obtained from popular text-to-image generative models, see e.g. https://twitter.com/apeoffire/status/1624884816851206145 and https://www.crosslabs.org/blog/diffusion-with-offset-noise.

It also means we can reliably produce dark, bright, or tinted images by shifting the input to a desired color.

Now, I was curious what would happen if I made Stable Diffusion denoise an "impossible" image whose mean color exceeds the [0,1] valid RGB range:

init_latent = vae_encode(tensor([1.5, 1.5, 1.5])[None,:,None,None].tile(1,1,512,512)) + sigma_max * randn(1,4,64,64)
{
"diffusion_out":[
"model.diffusion_model.output_blocks.6.0.in_layers.0.bias",
"model.diffusion_model.output_blocks.6.0.in_layers.0.weight",
"model.diffusion_model.output_blocks.6.0.out_layers.0.bias",
"model.diffusion_model.output_blocks.6.0.out_layers.0.weight",
"model.diffusion_model.output_blocks.7.0.in_layers.0.bias",
"model.diffusion_model.output_blocks.7.0.in_layers.0.weight",
"model.diffusion_model.output_blocks.7.0.out_layers.0.bias",
"model.diffusion_model.output_blocks.7.0.out_layers.0.weight",

"Antiprompts" in Craiyon / DALL-E Mega

DALL-E Mega variations of Starry Night, using antiprompts

Current image generation models use a technique called classifier-free guidance[^1] or super conditioning[^2]: At each step, they draw not just what fits the prompt best, but what sets images fitting that description apart from ones that reach similar intermediate results so far but might fit a totally different caption in the end.

To do that, they predict what they'd do next if the prompt was blank and decide based on the difference to what they'd do for the real prompt alone.

There ain't no rule that says we can't change that blank "antiprompt" to something meaningful, though. For Stable Diffusion, others have shown what replacing it can do (to try it, see e.g. AUTOMATIC1111's UI).

@drdaxxy
drdaxxy / parallel_cond_inference_mwe.ipynb
Created June 14, 2022 02:57
Speeding up DALL-E Mega inference with parallel null prompt evaluation
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

Grabbing TechnologyGuide forums via Tapatalk

Most of this applies to any forum using a (sufficiently up-to-date) version of the Tapatalk plugin. Some things may be specific to XenForo 1 or TechnologyGuide's setup. Users and forum administrators may also want to export private data (PMs, restricted forums, edit history...) which is out of scope here.

  • Error responses: 200 OK with body like:
result = /* repsonse body */ {
  result: false,
  result_text: "Need valid forum id!",
  error: "Need valid forum id!"
@drdaxxy
drdaxxy / App.css
Created September 5, 2020 04:13
gpt-3 generated dril tweets but for any keyword you want
/* yolo */
.App {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
}
form {
@drdaxxy
drdaxxy / irunabgm.js
Created July 18, 2020 04:08
script to list Iruna Online BGM + list of URLs for all BGM used in current japanese release + list of URLs for unused BGM between 0 and 2000
'use strict';
// finds all used BGM IDs in Iruna Online
//
// to get tracks, you can rewrite an ID list into URLs:
// <realm resource base>/bgm/<id>.zip
// e.g. 0 => http://rs-jp.iruna.jp/release/bgm/0.zip
// then download and extract with ZIP_PASSWORD
//
// only difference between realms seems to be updates
struct {
// Unknown file header fields have no effect on the animation
uint unk0;
uint unk1;
uint unk2;
uint unk3;
uint unk4;
uint unk5;
uint unk6;
uint unk7;
@drdaxxy
drdaxxy / CrafArchive.cs
Last active March 9, 2018 22:54
OUTDATED, USE https://github.com/drdaxxy/ffxvDitherPatch/blob/master/Craf/CrafArchive.cs INSTEAD. FFXV .earc archive format implementation, including repacking support.
// THIS FILE WILL NO LONGER BE UPDATED
// This code now lives at
// https://github.com/drdaxxy/ffxvDitherPatch/blob/master/Craf/CrafArchive.cs
// --------------------------------------------------------------------------
// THIS FILE WILL NO LONGER BE UPDATED
// Copyright 2018 Niklas K.
@drdaxxy
drdaxxy / shonenmagazine.py
Created November 9, 2017 20:36
pocket.shonenmagazine.com ripper
#!/usr/bin/env python2
# requires lxml, requests, cssselect, pillow
import sys
import os
from io import BytesIO
from lxml import html
import requests
from PIL import Image