Skip to content

Instantly share code, notes, and snippets.

@martinobettucci

martinobettucci/kohya-online-gui.ipynb Secret

Last active July 20, 2023 11:03
Show Gist options
  • Save martinobettucci/545adb371fcc6a72e8a72e66eae482dc to your computer and use it in GitHub Desktop.
Save martinobettucci/545adb371fcc6a72e8a72e66eae482dc to your computer and use it in GitHub Desktop.
Download ZIP
kohya-online-gui.ipynb
Display the source blob
Display the rendered blob
Raw
kohya-online-gui.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@RimorCosmicam
Copy link

This doesn't work anymore, adding a cell with the following as your 2nd cell fixes it:

%pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 torchtext==0.14.1 torchaudio==0.13.1 torchdata==0.5.1 --extra-index-url https://download.pytorch.org/whl/cu117

@martinobettucci
Copy link
Author

martinobettucci commented May 6, 2023 •
edited

Hello @RimorCosmicam, thank you for your feedback!

I've used the book just this week and it worked flawless: yet I'll consider your addition :-)

Edit: these dependencies of yours are installed during the $TORCH_COMMAND in cell (3). I confirm the notebook is working correctly the way it is, check your notebook environment.
If you experience further problems, please share your log and I will have a look ;-)

@Dat1304
Copy link

Dat1304 commented May 20, 2023 •
edited

can u help me , i have this error, i put the config file and hit train button and this happen :
2023-05-20 12:29:39.311226: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-05-20 12:29:39.493266: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable TF_ENABLE_ONEDNN_OPTS=0.
2023-05-20 12:29:39.549402: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2023-05-20 12:29:40.674587: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda/lib64:/usr/local/cuda-11.1/targets/x86_64-linux/lib/:/usr/local/cuda-11.0/targets/x86_64-linux/lib
2023-05-20 12:29:40.674715: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda/lib64:/usr/local/cuda-11.1/targets/x86_64-linux/lib/:/usr/local/cuda-11.0/targets/x86_64-linux/lib
2023-05-20 12:29:40.674753: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
tensorflow.version
2.10.1
trt.trt_utils._pywrap_py_utils.get_linked_tensorrt_version()
(7, 2, 2)
trt.trt_utils._pywrap_py_utils.get_loaded_tensorrt_version()
2023-05-20 12:29:42.100328: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda/lib64:/usr/local/cuda-11.1/targets/x86_64-linux/lib/:/usr/local/cuda-11.0/targets/x86_64-linux/lib
2023-05-20 12:29:42.100369: F tensorflow/compiler/tf2tensorrt/stub/nvinfer_stub.cc:49] getInferLibVersion symbol not found.
/bin/bash: line 34: 21309 Aborted (core dumped) python <<EOF
import tensorflow
from tensorflow.python.compiler.tensorrt import trt_convert as trt
print('tensorflow.version')
print(tensorflow.version)
print('trt.trt_utils._pywrap_py_utils.get_linked_tensorrt_version()')
print(trt.trt_utils._pywrap_py_utils.get_linked_tensorrt_version())
print('trt.trt_utils._pywrap_py_utils.get_loaded_tensorrt_version()')
print(trt.trt_utils._pywrap_py_utils.get_loaded_tensorrt_version())
import torch
for devid in range(0,torch.cuda.device_count()):
print('torch.cuda.get_device_name()')
print(torch.cuda.get_device_name(devid))

Loading config...
Traceback (most recent call last):
File "/koyah_ss/kohya_venv/lib/python3.10/site-packages/gradio/routes.py", line 394, in run_predict
output = await app.get_blocks().process_api(
File "/koyah_ss/kohya_venv/lib/python3.10/site-packages/gradio/blocks.py", line 1075, in process_api
result = await self.call_function(
File "/koyah_ss/kohya_venv/lib/python3.10/site-packages/gradio/blocks.py", line 884, in call_function
prediction = await anyio.to_thread.run_sync(
File "/koyah_ss/kohya_venv/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/koyah_ss/kohya_venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/koyah_ss/kohya_venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "/koyah_ss/dreambooth_gui.py", line 323, in train_model
msgbox('Image folder does not exist')
File "/koyah_ss/kohya_venv/lib/python3.10/site-packages/easygui/boxes/derived_boxes.py", line 230, in msgbox
return buttonbox(msg=msg,
File "/koyah_ss/kohya_venv/lib/python3.10/site-packages/easygui/boxes/button_box.py", line 95, in buttonbox
bb = ButtonBox(
File "/koyah_ss/kohya_venv/lib/python3.10/site-packages/easygui/boxes/button_box.py", line 147, in init
self.ui = GUItk(msg, title, choices, images, default_choice, cancel_choice, self.callback_ui)
File "/koyah_ss/kohya_venv/lib/python3.10/site-packages/easygui/boxes/button_box.py", line 263, in init
self.boxRoot = tk.Tk()
File "/usr/lib/python3.10/tkinter/init.py", line 2302, in init
self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable

@martinobettucci
Copy link
Author

@Dat1304 It is updated now, it should be fixed.
The error occurred if you used torch 2.0 and has been introduced by the upstream (Torch)

@civet
Copy link

civet commented Jul 20, 2023

everything is ok except the last block...

touch: invalid option -- 'p'
Try 'touch --help' for more information.
Python 3.10.6
tee: /data/requirements.txt: No such file or directory
absl-py==1.4.0
accelerate==0.21.0
aiofiles==23.1.0
aiohttp==3.8.5
aiosignal==1.3.1
albumentations==1.3.0
altair==4.2.2
annotated-types==0.5.0
anyio==3.7.1
appdirs==1.4.4
astunparse==1.6.3
async-timeout==4.0.2
attrs==23.1.0
cachetools==5.3.1
certifi==2023.5.7
charset-normalizer==3.2.0
click==8.1.6
cmake==3.27.0
contourpy==1.1.0
cycler==0.11.0
dadaptation==3.1
diffusers==0.10.2
docker-pycreds==0.4.0
easygui==0.98.3
einops==0.6.0
entrypoints==0.4
exceptiongroup==1.1.2
fairscale==0.4.13
fastapi==0.100.0
ffmpy==0.3.1
filelock==3.12.2
flatbuffers==23.5.26
fonttools==4.41.0
frozenlist==1.4.0
fsspec==2023.6.0
ftfy==6.1.1
gast==0.4.0
gitdb==4.0.10
GitPython==3.1.32
google-auth==2.22.0
google-auth-oauthlib==1.0.0
google-pasta==0.2.0
gradio==3.33.1
gradio_client==0.2.10
grpcio==1.56.2
h11==0.14.0
h5py==3.9.0
httpcore==0.17.3
httpx==0.24.1
huggingface-hub==0.16.4
idna==3.4
imageio==2.31.1
importlib-metadata==6.8.0
install==1.3.5
jax==0.4.13
Jinja2==3.1.2
joblib==1.3.1
jsonschema==4.18.4
jsonschema-specifications==2023.7.1
keras==2.12.0
kiwisolver==1.4.4
lazy_loader==0.3
libclang==16.0.6
-e git+https://github.com/P2Enjoy/kohya_ss.git@2bd39995f94d206d272ce10fa56da0575c6ba47f#egg=library
lightning-utilities==0.9.0
linkify-it-py==2.0.2
lion-pytorch==0.0.6
lit==16.0.6
lycoris-lora==0.1.6
Markdown==3.4.3
markdown-it-py==2.2.0
MarkupSafe==2.1.3
matplotlib==3.7.2
mdit-py-plugins==0.3.3
mdurl==0.1.2
ml-dtypes==0.2.0
mpmath==1.3.0
multidict==6.0.4
mypy-extensions==1.0.0
networkx==3.1
numpy==1.23.5
nvidia-cublas-cu11==11.11.3.6
nvidia-cublas-cu12==12.2.1.16
nvidia-cuda-runtime-cu12==12.2.53
nvidia-cudnn-cu11==8.6.0.163
nvidia-cudnn-cu12==8.9.2.26
nvidia-tensorrt==99.0.0
oauthlib==3.2.2
opencv-python==4.7.0.68
opencv-python-headless==4.8.0.74
opt-einsum==3.3.0
orjson==3.9.2
packaging==23.1
pandas==2.0.3
pathtools==0.1.2
Pillow==10.0.0
prodigyopt==1.0
protobuf==4.23.4
psutil==5.9.5
pyasn1==0.5.0
pyasn1-modules==0.3.0
pydantic==2.0.3
pydantic_core==2.3.0
pydub==0.25.1
Pygments==2.15.1
pyparsing==3.0.9
pyre-extensions==0.0.29
python-dateutil==2.8.2
python-multipart==0.0.6
pytorch-lightning==1.9.0
pytz==2023.3
PyWavelets==1.4.1
PyYAML==6.0.1
qudida==0.0.4
referencing==0.30.0
regex==2023.6.3
requests==2.31.0
requests-oauthlib==1.3.1
rich==13.4.1
rpds-py==0.9.2
rsa==4.9
safetensors==0.3.1
scikit-image==0.21.0
scikit-learn==1.3.0
scipy==1.11.1
semantic-version==2.10.0
sentry-sdk==1.28.1
setproctitle==1.3.2
six==1.16.0
smmap==5.0.0
sniffio==1.3.0
starlette==0.27.0
sympy==1.12
tensorboard==2.12.3
tensorboard-data-server==0.7.1
tensorflow==2.12.0
tensorflow-estimator==2.12.0
tensorflow-io-gcs-filesystem==0.32.0
tensorrt==8.6.1
tensorrt-bindings==8.6.1
tensorrt-libs==8.6.1
termcolor==2.3.0
threadpoolctl==3.2.0
tifffile==2023.7.18
timm==0.6.12
tk==0.1.0
tokenizers==0.13.3
toml==0.10.2
toolz==0.12.0
torch==2.0.1+cu118
torchaudio==2.0.2+cu118
torchmetrics==1.0.1
torchvision==0.15.2+cu118
tqdm==4.65.0
transformers==4.26.0
triton==2.0.0
typing-inspect==0.9.0
typing_extensions==4.7.1
tzdata==2023.3
uc-micro-py==1.0.2
urllib3==1.26.16
uvicorn==0.23.1
voluptuous==0.13.1
wandb==0.15.0
wcwidth==0.2.6
websockets==11.0.3
Werkzeug==2.3.6
wrapt==1.14.1
xformers==0.0.20
yarl==1.9.2
zipp==3.16.2
torch.cuda.get_device_name()
Tesla T4
2023-07-20 10:42:27.356882: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2023-07-20 10:42:28.527402: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-07-20 10:42:32.273033: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
tensorflow.__version__
2.12.0
trt.trt_utils._pywrap_py_utils.get_linked_tensorrt_version()
(8, 4, 3)
trt.trt_utils._pywrap_py_utils.get_loaded_tensorrt_version()
2023-07-20 10:42:37.418709: F tensorflow/compiler/tf2tensorrt/stub/nvinfer_stub.cc:49] getInferLibVersion symbol not found.
/bin/bash: line 45:  6343 Aborted                 (core dumped) python <<EOF
import torch
for devid in range(0,torch.cuda.device_count()):
        print('torch.cuda.get_device_name()')
        print(torch.cuda.get_device_name(devid))

import tensorflow
from tensorflow.python.compiler.tensorrt import trt_convert as trt

print('tensorflow.__version__')
print(tensorflow.__version__)
print('trt.trt_utils._pywrap_py_utils.get_linked_tensorrt_version()')
print(trt.trt_utils._pywrap_py_utils.get_linked_tensorrt_version())
print('trt.trt_utils._pywrap_py_utils.get_loaded_tensorrt_version()')
print(trt.trt_utils._pywrap_py_utils.get_loaded_tensorrt_version())
EOF

Already up to date.
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
LOG INIT
INFO: headless: False
INFO: Load CSS...
Running on local URL:  http://127.0.0.1:7680/
Running on public URL: https
...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment