Skip to content

Instantly share code, notes, and snippets.

@w-e-w
Last active October 21, 2024 14:16
Show Gist options
  • Save w-e-w/0f37c04c18e14e4ee1482df5c4eb9f53 to your computer and use it in GitHub Desktop.
Save w-e-w/0f37c04c18e14e4ee1482df5c4eb9f53 to your computer and use it in GitHub Desktop.
Stable Diffusion web UI txt2img img2img api example script
from datetime import datetime
import urllib.request
import base64
import json
import time
import os
webui_server_url = 'http://127.0.0.1:7860'
out_dir = 'api_out'
out_dir_t2i = os.path.join(out_dir, 'txt2img')
out_dir_i2i = os.path.join(out_dir, 'img2img')
os.makedirs(out_dir_t2i, exist_ok=True)
os.makedirs(out_dir_i2i, exist_ok=True)
def timestamp():
return datetime.fromtimestamp(time.time()).strftime("%Y%m%d-%H%M%S")
def encode_file_to_base64(path):
with open(path, 'rb') as file:
return base64.b64encode(file.read()).decode('utf-8')
def decode_and_save_base64(base64_str, save_path):
with open(save_path, "wb") as file:
file.write(base64.b64decode(base64_str))
def call_api(api_endpoint, **payload):
data = json.dumps(payload).encode('utf-8')
request = urllib.request.Request(
f'{webui_server_url}/{api_endpoint}',
headers={'Content-Type': 'application/json'},
data=data,
)
response = urllib.request.urlopen(request)
return json.loads(response.read().decode('utf-8'))
def call_txt2img_api(**payload):
response = call_api('sdapi/v1/txt2img', **payload)
for index, image in enumerate(response.get('images')):
save_path = os.path.join(out_dir_t2i, f'txt2img-{timestamp()}-{index}.png')
decode_and_save_base64(image, save_path)
def call_img2img_api(**payload):
response = call_api('sdapi/v1/img2img', **payload)
for index, image in enumerate(response.get('images')):
save_path = os.path.join(out_dir_i2i, f'img2img-{timestamp()}-{index}.png')
decode_and_save_base64(image, save_path)
if __name__ == '__main__':
payload = {
"prompt": "masterpiece, (best quality:1.1), 1girl <lora:lora_model:1>", # extra networks also in prompts
"negative_prompt": "",
"seed": 1,
"steps": 20,
"width": 512,
"height": 512,
"cfg_scale": 7,
"sampler_name": "DPM++ 2M",
"n_iter": 1,
"batch_size": 1,
# example args for x/y/z plot
# "script_name": "x/y/z plot",
# "script_args": [
# 1,
# "10,20",
# [],
# 0,
# "",
# [],
# 0,
# "",
# [],
# True,
# True,
# False,
# False,
# 0,
# False
# ],
# example args for Refiner and ControlNet
# "alwayson_scripts": {
# "ControlNet": {
# "args": [
# {
# "batch_images": "",
# "control_mode": "Balanced",
# "enabled": True,
# "guidance_end": 1,
# "guidance_start": 0,
# "image": {
# "image": encode_file_to_base64(r"B:\path\to\control\img.png"),
# "mask": None # base64, None when not need
# },
# "input_mode": "simple",
# "is_ui": True,
# "loopback": False,
# "low_vram": False,
# "model": "control_v11p_sd15_canny [d14c016b]",
# "module": "canny",
# "output_dir": "",
# "pixel_perfect": False,
# "processor_res": 512,
# "resize_mode": "Crop and Resize",
# "threshold_a": 100,
# "threshold_b": 200,
# "weight": 1
# }
# ]
# },
# "Refiner": {
# "args": [
# True,
# "sd_xl_refiner_1.0",
# 0.5
# ]
# }
# },
# "enable_hr": True,
# "hr_upscaler": "R-ESRGAN 4x+ Anime6B",
# "hr_scale": 2,
# "denoising_strength": 0.5,
# "styles": ['style 1', 'style 2'],
# "override_settings": {
# 'sd_model_checkpoint': "sd_xl_base_1.0", # this can use to switch sd model
# },
}
call_txt2img_api(**payload)
init_images = [
encode_file_to_base64(r"B:\path\to\img_1.png"),
# encode_file_to_base64(r"B:\path\to\img_2.png"),
# "https://image.can/also/be/a/http/url.png",
]
batch_size = 2
payload = {
"prompt": "1girl, blue hair",
"seed": 1,
"steps": 20,
"width": 512,
"height": 512,
"denoising_strength": 0.5,
"n_iter": 1,
"init_images": init_images,
"batch_size": batch_size if len(init_images) == 1 else len(init_images),
# "mask": encode_file_to_base64(r"B:\path\to\mask.png")
}
# if len(init_images) > 1 then batch_size should be == len(init_images)
# else if len(init_images) == 1 then batch_size can be any value int >= 1
call_img2img_api(**payload)
# there exist a useful extension that allows converting of webui calls to api payload
# particularly useful when you wish setup arguments of extensions and scripts
# https://github.com/huchenlei/sd-webui-api-payload-display
@shivang-arya-g5
Copy link

Hello, thnx alot for this helper reference for img2img. I just wanted to ask is "mask" input still working or has it been changed to some other key, because earlier the payload used to work for me now i keep getting this error. Unexpected input. mask is not a valid input option., Ive tried looking at API docs and everywhere it says mask is supposed to work, Do you have any insight on this ?

@w-e-w
Copy link
Author

w-e-w commented Dec 13, 2023

wanted to ask is "mask" input still working ...

did a quick test just now, seems to be working without issue

@ckc1q2w
Copy link

ckc1q2w commented Jan 29, 2024

Hello, you have shown an example args for 'x/y/z plot' in this demo, May I ask if you can give me an example args for 'Prompts from file and textbox'

@w-e-w
Copy link
Author

w-e-w commented Jan 29, 2024

@ckc1q2w

Hello, you have shown an example args for 'x/y/z plot' in this demo, May I ask if you can give me an example args for 'Prompts from file and textbox'

https://gist.github.com/w-e-w/0f37c04c18e14e4ee1482df5c4eb9f53#file-sd-webui-txt2img-img2img-api-example-py-L161-L163

but to be honest I don't know why you would ever use Prompts from file and textbox in api
just used API and send multiple requests to generate the images with the parameters you want, you have more freedom

@MackNcD
Copy link

MackNcD commented Feb 2, 2024

You would use prompts from a file for a program that already has descriptions of images ready to be created for the user, for example, an on-the-fly dungeon crawler game. The point of an API is that it allows for divergent use, allowing you to apply entirely different interfaces and use cases to SD, it's not always about "write description, create image."

@vule20
Copy link

vule20 commented Feb 2, 2024

Hi. Thanks for your great contributions and tutorials. Do you have example payloads for processing multiple images, each with different prompts (batch processing)? I noticed that each sample takes around 10 seconds, and each run takes 3GB of RAM at max. My GPU has 24GB of RAM and I wanna exploit batch processing for higher throughputs. I tried the txt2img API but it only handles 1 request at a time despite receiving many at the same time. I wonder if batch processing is possible or not? Or I have to implement some task processing scheduler such as Celery and use multiple workers at the same time ?

@w-e-w
Copy link
Author

w-e-w commented Feb 2, 2024

@mack

You would use prompts from a file for a program that already has descriptions of images ready to be created for the user, for example, an on-the-fly dungeon crawler game. The point of an API is that it allows for divergent use, allowing you to apply entirely different interfaces and use cases to SD, it's not always about "write description, create image."

you misunderstood what he's asking what he's asking, he's asking about the parameters to use for the Prompts from file or textbox script a buit-in script similer to XYZ grid
basically the function of the script is to let user generate multiple images with different parameters, parameters such as prompts samplers sd model
I don't see a use case of this when using this script when using webui via API, Prompts from file or textbox, only gives you control of a handful of different parameters, while if you use API directly you have control over everything including parameters of extensions, using Prompts from file or textbox only adds an extra layer of complexity while giving you less control

he is not asking about reading prompts from a predetermined file/preset/configuration/template
he's asking about a script

@vincent-dl

Hi. Thanks for your great contributions and tutorials. Do you have example payloads for processing multiple images, each with different prompts (batch processing)? I noticed that each sample takes around 10 seconds, and each run takes 3GB of RAM at max. My GPU has 24GB of RAM and I wanna exploit batch processing for higher throughputs. I tried the txt2img API but it only handles 1 request at a time despite receiving many at the same time. I wonder if batch processing is possible or not? Or I have to implement some task processing scheduler such as Celery and use multiple workers at the same time ?

first you have to understand that there is a difference between batch processing and processing two jobs at once
and with batch processing there's two types of batches batch size and batch count
batch_size is how many images it will process in parallel, using the exact same parameters
(batch count) or in api n_iter is running multiple times for increasing the seed

processing multiple jobs at once it's not supported by webui
lots of parts of the code are only designed for you to only have one job being processed at a time
such as we allows a job to request switching of the current model

just think what chaos what happened if the model is unloaded doing one job by another job

if you really want to do so you could launch multiple instances of webui server, but it doesn't really make much sense doing so on single GPU system
in general when generating "normal size images" your GPU core utilization is basically already at Max
the bottleneck is the processing not the VRAM

while setting a batch size of batch count could reduce job some overhead
it is unlikely that you will see a big performance difference if you use a bigger batch size

but you have multiple GPUs then it may make sense
you just have to distribute it your request among your werbui instances
or use an extension that chains multiple webui toughter https://github.com/papuSpartan/stable-diffusion-webui-distributed

I answered a similar question just recently regarding performance AUTOMATIC1111/stable-diffusion-webui#14784 (comment)

@vule20
Copy link

vule20 commented Feb 5, 2024

@w-e-w thanks for your response

@huangerlin
Copy link

Hello, I would like to ask what the parameters for redrawing the area are. I would like to use redrawing only to mask the area, but it has not taken effect and has been redrawing the entire image

@w-e-w
Copy link
Author

w-e-w commented Feb 25, 2024

Hello, I would like to ask what the parameters for redrawing the area are. I would like to use redrawing only to mask the area, but it has not taken effect and has been redrawing the entire image

are you asking about Inpaint area only masked
irrc the payload are is inpaint_full_res: True

@samlara32
Copy link

Is it possible to add Adetailer to the payload

@w-e-w
Copy link
Author

w-e-w commented Apr 18, 2024

Is it possible to add Adetailer to the payload

yes read the last section

@toshan-luktuke
Copy link

Hi, trying it out on 19th April on Windows 11, Python 3.10.11 and I'm getting a HTTP Error 404: Not Found running the above example script. However when I use the requests module instead of urllib I can get the correct response.
Letting people know in case someone faces a similar issue

@samlara32
Copy link

Hi, trying it out on 19th April on Windows 11, Python 3.10.11 and I'm getting a HTTP Error 404: Not Found running the above example script. However when I use the requests module instead of urllib I can get the correct response. Letting people know in case someone faces a similar issue

hey, can you share your updated code

@ChenXuanting
Copy link

Hi, trying it out on 19th April on Windows 11, Python 3.10.11 and I'm getting a HTTP Error 404: Not Found running the above example script. However when I use the requests module instead of urllib I can get the correct response. Letting people know in case someone faces a similar issue

Hi, I used requests but still got empty response object showing 404. Could you share you code here? Many thanks.

@ChenXuanting
Copy link

@ckc1q2w

Hello, you have shown an example args for 'x/y/z plot' in this demo, May I ask if you can give me an example args for 'Prompts from file and textbox'

https://gist.github.com/w-e-w/0f37c04c18e14e4ee1482df5c4eb9f53#file-sd-webui-txt2img-img2img-api-example-py-L161-L163

but to be honest I don't know why you would ever use Prompts from file and textbox in api just used API and send multiple requests to generate the images with the parameters you want, you have more freedom

Hi, this script is not working anymore. Possible updates?

@muhammadhur2
Copy link

When I make request at http://127.0.0.1:7860/sdapi/v1/txt2img directly using postman or using the above script (with or without requests module) I am getting 404 error. Anyone know how to fix this?

@kid9591
Copy link

kid9591 commented May 22, 2024

Did they remove the txt2img & img2img APIs? Please anyone gives us confirmation, thanks.

@w-e-w
Copy link
Author

w-e-w commented May 22, 2024

Did they remove the txt2img & img2img APIs? Please anyone gives us confirmation, thanks.

api works


... HTTP Error 404: Not Found ...

the API error message is not that descripted, we use 404 for lots of things "not found"

txt2img and img2img

  • Sampler not found
    img2img
  • Init image not found
    interrogate
  • Image not found
  • Model not found

in particular there was some change to the sampler scheduler in 1.9
we did not have the foresight to implement backwards compatibility and result in a 404 if you try to generate an image with a non-existent sampler scheduler

@kid9591
Copy link

kid9591 commented May 22, 2024

@w-e-w
I created a request using Postman and it says "Not found". Can you tell me what is wrong here?

Screenshot 2024-05-22 at 14 01 30

Moreover, at "http://127.0.0.1:7860/docs#/" I cannot see the description for that above 2 APIs. I installed Automatic1111 v1.9.3

@w-e-w
Copy link
Author

w-e-w commented May 22, 2024

Moreover, at "http://127.0.0.1:7860/docs#/" I cannot see the description for that above 2 APIs. I installed Automatic1111 v1.9.3

that's odd, it's showing on mine
image

but since you give me practically zero information I can't really help
if I think your experiencing an issue don't ask here make an issue post

@zhangbool
Copy link

Got the same issue: "... HTTP Error 404: Not Found ..."
I searched the http://127.0.0.1:7860/docs#/, no /sdapi/v1/txt2img was found.
restart for several times, still not working.

So I search all the webui process use the following cmd:
ps -ef | grep webui.sh | grep -v grep | gawk '{ print $2 }' and kill all the pids
then I search the 7860 port use the following cmd:
netstat -antlp | grep LISTEN | grep 7860 and kill the pid again.
then I start webui again, and finally the /sdapi/v1/txt2img was shown and the api test code worked.

I don't know why.
if you guys have the same issue, try to clean all the process and restart with --api.

@FaezehMosayyebi
Copy link

To whom they couldn't find /sdapi/v1/txt2img and /sdapi/v1/img2img endpoint in http://127.0.0.1:7860/docs:
I solved the problem by adding --api to COMMANDLINE_ARGS in webui-user.bat file.

@dreamsdreams
Copy link

To whom they couldn't find /sdapi/v1/txt2img and /sdapi/v1/img2img endpoint in http://127.0.0.1:7860/docs: I solved the problem by adding --api to COMMANDLINE_ARGS in webui-user.bat file.

i want to kiss u

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment