Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
BackMatting.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@camjac251

This comment has been minimized.

Copy link

camjac251 commented May 17, 2020

Is this script still working? I get some errors with video processing

Segmentation completes successfully Done: colab_inputs_video/input
But with both handheld and fixed, it fails

Traceback (most recent call last):
  File "test_pre_process_video.py", line 123, in <module>
    back_align = alignImages(back, image,mask)
  File "test_pre_process_video.py", line 38, in alignImages
    h, mask = cv2.findHomography(points1, points2, cv2.RANSAC)
cv2.error: OpenCV(3.4.5) /io/opencv/modules/calib3d/src/ptsetreg.cpp:169: error: (-215:Assertion failed) count >= 0 && count2 == count in function 'run'
CUDA Device: 0
Using video mode
Traceback (most recent call last):
  File "test_background-matting_image.py", line 121, in <module>
    bbox=get_bbox(rcnn,R=bgr_img0.shape[0],C=bgr_img0.shape[1])
  File "/content/Background-Matting/functions.py", line 38, in get_bbox
    x1, y1 = np.amin(where, axis=1)
  File "<__array_function__ internals>", line 6, in amin
  File "/usr/local/lib/python3.6/dist-packages/numpy/core/fromnumeric.py", line 2746, in amin
    keepdims=keepdims, initial=initial, where=where)
  File "/usr/local/lib/python3.6/dist-packages/numpy/core/fromnumeric.py", line 90, in _wrapreduction
    return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
ValueError: zero-size array to reduction operation minimum which has no identity
@venkat094

This comment has been minimized.

Copy link

venkat094 commented May 18, 2020

I too got the same error ! The below error is for image, Not sure where is the issue !

CUDA Device: 0
Using image mode
Traceback (most recent call last):
  File "test_background-matting_image.py", line 46, in <module>
    back_img10=cv2.imread(args.target_back); back_img10=cv2.cvtColor(back_img10,cv2.COLOR_BGR2RGB);
cv2.error: OpenCV(3.4.5) /io/opencv/modules/imgproc/src/color.cpp:181: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

@jrdeco560

This comment has been minimized.

Copy link

jrdeco560 commented May 18, 2020

errors

@andreyryabtsev

This comment has been minimized.

Copy link
Owner Author

andreyryabtsev commented May 18, 2020

@camjac251 I think your issue may be similar to senguptaumd/Background-Matting#33 . In order to pre-process hand-held videos, that code tries to find a homography alignment between each frame, which may fail for featureless backgrounds. Unfortunately, the script doesn't report the errors nicely; but maybe you could try a different video, or a fixed cam (no alignment needed) video to see if that works?

@venkat094 Actually your error is different; they look the same because cv2's low level code reports them similarly. I am fairly sure the target background file you are trying to use does not exist. So when cv2.imread opens it it silently reads no data, but when it tries to convert the color space it realizes the image is not actually there.

@camjac251

This comment has been minimized.

Copy link

camjac251 commented May 18, 2020

Thank you for the reply. I'm trying it with the handheld sample video this time so hopefully it works.

I did notice an little error on the
Run the matting algorithm. If your video was captured with a fixed camera, comment the first line and uncomment the second before running.
block
could it be made into

!CUDA_VISIBLE_DEVICES=0 python test_background-matting_image.py -m real-hand-held -i colab_inputs_video/input/ -o colab_inputs_video/output/ -tb colab_inputs_video/background/
# !CUDA_VISIBLE_DEVICES=0 python test_background-matting_image.py -m real-fixed-cam -i colab_inputs_video/input/ -o colab_inputs_video/output/ -tb colab_inputs_video/background/ -b colab_inputs_video/$BM_SOURCE_STILL
@andreyryabtsev

This comment has been minimized.

Copy link
Owner Author

andreyryabtsev commented May 18, 2020

@camjac251 good catch, fixed!

@gchoi

This comment has been minimized.

Copy link

gchoi commented May 20, 2020

Hi, I have error for the following code:
!CUDA_VISIBLE_DEVICES=0 python test_background-matting_image.py -m real-hand-held -i colab_inputs/input/ -o colab_inputs/output/ -tb colab_inputs/background/0001.png

CUDA Device: 0 Using image mode Traceback (most recent call last): File "test_background-matting_image.py", line 121, in <module> bbox=get_bbox(rcnn,R=bgr_img0.shape[0],C=bgr_img0.shape[1]) File "/content/drive/My Drive/Colab-Notebooks/TensorFlow-Examples/033-Back-Matting/Background-Matting/functions.py", line 38, in get_bbox x1, y1 = np.amin(where, axis=1) File "<__array_function__ internals>", line 6, in amin File "/usr/local/lib/python3.6/dist-packages/numpy/core/fromnumeric.py", line 2746, in amin keepdims=keepdims, initial=initial, where=where) File "/usr/local/lib/python3.6/dist-packages/numpy/core/fromnumeric.py", line 90, in _wrapreduction return ufunc.reduce(obj, axis, dtype, out, **passkwargs) ValueError: zero-size array to reduction operation minimum which has no identity

@venkat094

This comment has been minimized.

Copy link

venkat094 commented May 20, 2020

@camjac251 I think your issue may be similar to senguptaumd/Background-Matting#33 . In order to pre-process hand-held videos, that code tries to find a homography alignment between each frame, which may fail for featureless backgrounds. Unfortunately, the script doesn't report the errors nicely; but maybe you could try a different video, or a fixed cam (no alignment needed) video to see if that works?

@venkat094 Actually your error is different; they look the same because cv2's low level code reports them similarly. I am fairly sure the target background file you are trying to use does not exist. So when cv2.imread opens it it silently reads no data, but when it tries to convert the color space it realizes the image is not actually there.

Thanks for your quick reply !!
I did check twice, the target background file is present ! I have just uploaded only one file !

@andreyryabtsev

This comment has been minimized.

Copy link
Owner Author

andreyryabtsev commented May 21, 2020

@andreyryabtsev

This comment has been minimized.

Copy link
Owner Author

andreyryabtsev commented May 21, 2020

In general, colab can be quite finicky (I think someone had their file get lost because the runtime restarted); if anyone has issues like these it may help to either (a) run the repository in a more standard environment or (b) add asserts to double check all the files exist and the segmentations + homography transforms don't fail for whatever reason.

@Flock1

This comment has been minimized.

Copy link

Flock1 commented May 24, 2020

I too got the same error ! The below error is for image, Not sure where is the issue !

CUDA Device: 0
Using image mode
Traceback (most recent call last):
  File "test_background-matting_image.py", line 46, in <module>
    back_img10=cv2.imread(args.target_back); back_img10=cv2.cvtColor(back_img10,cv2.COLOR_BGR2RGB);
cv2.error: OpenCV(3.4.5) /io/opencv/modules/imgproc/src/color.cpp:181: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

I faced this error because of the following:

When I use the segmentation model, it needs to create a model by the name "maskDL" and there it stores the segmented image. Hence, it's not able to read the segmented image because it's not there.

@tolchx

This comment has been minimized.

Copy link

tolchx commented May 25, 2020

Hello, thank you very much for sharing this project, I am having the following error, using video segmentation.
@Flock1 using the online example colab, how do I create a model with the name 'maskDL'?
Using the demo material: #Uncomment and run this if you want to skip uploading your videos (below) and run on the sample videos instead (choose fixed or handheld) It works well

The error I have with my own videos is the following:

CUDA Device: 0
Using video mode
Traceback (most recent call last):
File "test_background-matting_image.py", line 85, in
bg_im0=cv2.imread(os.path.join(data_path, filename.replace('_img','_back'))); bg_im0=cv2.cvtColor(bg_im0,cv2.COLOR_BGR2RGB);
cv2.error: OpenCV(3.4.5) /io/opencv/modules/imgproc/src/color.cpp:181: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

Any help is welcome thanks

@Flock1

This comment has been minimized.

Copy link

Flock1 commented May 27, 2020

@tolchx, this error happens because there is no image that is being read. That's happening probably because there is no folder or a file. Kindly check if the required folder or the image is present.

@tolchx

This comment has been minimized.

Copy link

tolchx commented May 31, 2020

@tolchx, this error happens because there is no image that is being read. That's happening probably because there is no folder or a file. Kindly check if the required folder or the image is present.

Yes, it was that, thank you very much.

folder and python file paths were in other folders,

@tolchx

This comment has been minimized.

Copy link

tolchx commented Jun 1, 2020

Hello, I have an error when I run this section of the code:

#!CUDA_VISIBLE_DEVICES=0 python test_background-matting_image.py -m real-hand-held -i colab_inputs_video/input/ -o colab_inputs_video/output/ -tb colab_inputs_video/background/
!CUDA_VISIBLE_DEVICES=0 python test_background-matting_image.py -m real-fixed-cam -i colab_inputs_video/input/ -o colab_inputs_video/output/ -tb colab_inputs_video/background/ -b colab_inputs_video/$BM_SOURCE_STILL

error:


Traceback (most recent call last):
  File "test_background-matting_image.py", line 92, in <module>
    back_img10=cv2.imread(os.path.join(args.target_back,filename.replace('_img.png','.png'))); back_img10=cv2.cvtColor(back_img10,cv2.COLOR_BGR2RGB);
cv2.error: OpenCV(3.4.5) /io/opencv/modules/imgproc/src/color.cpp:181: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

I did the test with several videos, with different amount of frames, 600/900/1000 frames, for example.
But it always stops at 441 and the error appears.

What could be the cause?

In the folder: Background-Matting / colab_inputs_video / inputs, the remaining frames appear, it does not end at frame 441

Thank you

@Flock1

This comment has been minimized.

Copy link

Flock1 commented Jun 4, 2020

@tolchx, I'm not sure but I think there is a problem with arguments. Instead of --tb, just use --back. Also, how are your results? Can you share them with me because I'm not getting that good results. I had to do some post-processing.

@tolchx

This comment has been minimized.

Copy link

tolchx commented Jun 10, 2020

@tolchx, I'm not sure but I think there is a problem with arguments. Instead of --tb, just use --back. Also, how are your results? Can you share them with me because I'm not getting that good results. I had to do some post-processing.

I had good results with some videos, I'm experimenting with other material, here I uploaded a video, use the algorithm to segment the dancers, and then apply styletransfer https://www.instagram.com/p/CAsK54qgx_9/
I will try the solution you recommend, thanks

@Flock1

This comment has been minimized.

Copy link

Flock1 commented Jun 12, 2020

@tolchx, you are using it for something interesting. Can you elaborate on what you are doing? Also, did you make any changes to the code? Because I had to do some post-processing to get good results.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.