OpenCV Tutorial transparent mask overlay to recognized faces

opencv-tutorial-transparent-mask-overlay-to-recognized-faces.md

As the death toll from the Wuhan coronavirus still risen, I felt anxious and helpless through everyone here were talking and laughing on New Year's Eve. Not sure what should I do and I just wrote this python3 program to make people aware of the importance of wearing respirator masks when defeating the virus.

In this program, we use CascadeClassifier to add respirators masks to the recognized faces in the capture. Before everything start, we have to import the necessary packages:

import argparse
import cv2

We use argparse to construct the argument parse for more flexible usage. The respirator image cloud be found on eBay, Google... and transparent background required as .png format. The haarcascade_frontalface_default.xml could be found at OpenCV official git repository https://github.com/opencv/opencv/tree/master/data/haarcascade.

ap = argparse.ArgumentParser()
ap.add_argument('-c', '--camera', type = int, default = 0, help = 'camera device index')
ap.add_argument('-m', '--mask', default = 'respirator.png', help = 'path of mask .png')
ap.add_argument('-d', '--data', default = 'haarcascade_frontalface_default.xml', help = 'path of haarcascad data model')
ap.add_argument('-w', '--width', type = int, default = 720, help = 'camera frame width')
ap.add_argument('-l', '--height', type = int, default = 480, help = 'camera frame height')
args = vars(ap.parse_args())

Parse the arguments, open camera and load necessary files:

cap = cv2.VideoCapture(args['camera'])
cap.set(cv2.CAP_PROP_FRAME_WIDTH, args['width'])
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, args['height'])
mask = cv2.imread(args['mask'], cv2.IMREAD_UNCHANGED)
face_cascade = cv2.CascadeClassifier(args['data'])

Create tracker bar based on the arguments we may use:

def callBack(val):
    pass

cv2.createTrackbar('scale_factor', 'slider', 10, 100, callBack)
cv2.createTrackbar('min_neighbors', 'slider', 5, 100, callBack)
cv2.createTrackbar('min_size', 'slider', 80, max(args['width'], args['height']), callBack)
cv2.createTrackbar('max_size', 'slider', 360, max(args['width'], args['height']), callBack)
cv2.createTrackbar('mask_y_s_k', 'slider', 5, 100, callBack)
cv2.createTrackbar('mask_y_e_k', 'slider', 10, 100, callBack)

Before start frame processing, let's understand CascadeClassifier deeper. Here I just give a simple reference to the official introduction.

Object Detection using Haar feature-based cascade classifiers is an effective object detection method proposed by Paul Viola and Michael Jones in their paper, It is a machine learning based approach where a cascade function is trained from a lot of positive and negative images. It is then used to detect objects in other images.

Here we will work with face detection. Initially, the algorithm needs a lot of positive images and negative images to train the classifier. Then we need to extract features from it.

Then let's code a function transparentOverlay for blending frame and respirator image first, we loop over all pixels and apply the blending equation:

def transparentOverlay(src, overlay): 
    for col in range(overlay.shape[0]):
        for row in range(overlay.shape[1]):
            if col <= src.shape[0] or row <= src.shape[1]:
                alpha = float(overlay[col][row][3] / 255)
                src[col][row] = alpha * overlay[col][row][:3] + (1 - alpha) * src[col][row]
    return src

Finally we process every frame from capture, recognize the faces, resize the respirators to fit the faces and apply the transparent overlay to each frame:

while cap.isOpened():
    (ret, frame) = cap.read()
    (min_size, max_size) = cv2.getTrackbarPos('min_size', 'slider'), cv2.getTrackbarPos('max_size', 'slider')
    for (x, y, w, h) in face_cascade.detectMultiScale(image = frame, scaleFactor = 1 + cv2.getTrackbarPos('scale_factor', 'slider') / 100, minNeighbors = cv2.getTrackbarPos('min_neighbors', 'slider'), minSize = (min_size, min_size), maxSize = (max_size, max_size)):
        if w > 0 and h > 0:
            (overlay_y_s, overlay_y_e) = int(y + h * cv2.getTrackbarPos('mask_y_s_k', 'slider') / 10), int(y + h * cv2.getTrackbarPos('mask_y_e_k', 'slider') / 10)
            mask_resized = cv2.resize(src = mask, dsize = (w, overlay_y_e - overlay_y_s), interpolation = cv2.INTER_CUBIC)
            transparentOverlay(src = frame[overlay_y_s:overlay_y_e, x:x + w], overlay = mask_resized)
    cv2.imshow(winname = 'respirator', mat = frame)
    if cv2.waitKey(1) > 0:
        break

Forming a sound habit, close capture and destroy windows when program ended:

cap.release()
cv2.destroyAllWindows()