AlexKordic

import pynvml, time
from pynvml import *

TEMP_MIN_VALUE = 30.0 # fan is around 30%
TEMP_MAX_VALUE = 65.0 # fan is at 100% onwards
TEMP_RANGE = TEMP_MAX_VALUE - TEMP_MIN_VALUE
def fanspeed_from_t(t):
  if t <= TEMP_MIN_VALUE: return 0.0
  if t >= TEMP_MAX_VALUE: return 1.0
  return (t - TEMP_MIN_VALUE) / TEMP_RANGE

crowsonkb

"""Scalar Preference Optimization."""

import torch
from torch.nn import functional as F


def logp_completion(logits, tokens, mask):
    """Compute the log probabilities of completions given their prompts.

    Args:

rain-1

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

• Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.

harubaru

Waifu Diffusion 1.4 Overview

_{An image generated at resolution 512x512 then upscaled to 1024x1024 with Waifu Diffusion 1.3 Epoch 7.}

Goals

• Improving image generation at different aspect ratios using conditional masking during training. This will allow for the entire image to be seen during training instead of center cropped images, which will allow for better results when generating full body images, portraits, and improving the composition.
• Expanded the input context from 77 tokens to 231 tokens or perhaps to an unlimited amount of tokens. Out of 77 tokens for input, only 75 are useable. This does not give nearly enough room for complex prompting that requires a lot of detail.

jachiam

# Script for converting a HF Diffusers saved pipeline to a Stable Diffusion checkpoint.
# *Only* converts the UNet, VAE, and Text Encoder.
# Does not convert optimizer state or any other thing.
# Written by jachiam

import argparse
import os.path as osp

import torch

trygvebw

import torch
import numpy as np
import k_diffusion as K

from PIL import Image
from torch import autocast
from einops import rearrange, repeat

def pil_img_to_torch(pil_img, half=False):
    image = np.array(pil_img).astype(np.float32) / 255.0

Pyr-000

UMAP maps of internal OpenCLIP ViT-H/14 text representations. Colors are derived from k_means clustering on the unprocessed (1024-dimensional) CLIP feature tensors.

This variant of CLIP is used in StableDiffusion version 2.x. (https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K)

"by <artist>": https://gist.githack.com/Pyr-000/90503e58688a7827278ea801096da933/raw/cd7f59e3d7d25800329ef0213d352e7e8ca8e8c9/OpenCLIP_ViT-H14_rev3_artists_by_map_cluster.html

"in the style of <artist>": https://gist.githack.com/Pyr-000/90503e58688a7827278ea801096da933/raw/cd7f59e3d7d25800329ef0213d352e7e8ca8e8c9/OpenCLIP_ViT-H14_rev3_artists_st_map_cluster.html

kinoc

#  So now you want to finetune that GPT-J-6B on a 3090/TITAN GPU ... okay 
#  More exploratory coding. It uses the Huggingface model port, deepspeed and reads all text/md files from a target directory
#  It is a fragment of a larger system with remote editing, but that's another story
#  This is the raw, training tester. Items to look out for:
#  - uses DeepSpeed and has a DS config
#  - to save space uses SGD instead of ADAM
#  - uses gradient checkpointing
#  - freezes 25% of the layers to fit

# Assumes you can already run https://gist.github.com/kinoc/2d636a68876cd3de7b6e9c9452b61089

Jimmajones

import json
import csv
import matplotlib.pyplot as plt
import matplotlib.ticker as ticker

'''
Difficulty checks are coded in the following format:

[3]
0 Field data

pbaylies

# Modified StyleGAN2 Projector with CLIP, addl. losses, kmeans, etc.
# by Peter Baylies, 2021 -- @pbaylies on Twitter

# Copyright (c) 2021, NVIDIA CORPORATION.  All rights reserved.
#
# NVIDIA CORPORATION and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto.  Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA CORPORATION is strictly prohibited.