Implement Face Tracking with the dlib DCF-based Tracker: A Beginner Guide
OpenCV Tutorial
Implement Face Tracking with the dlib DCF-based Tracker: A Beginner Guide

Face tracking with the dlib DCF-

based tracker

In the face_tracking_correlation_filters.py script, we perform face tracking using the

dlib frontal face detector for initialization and the dlib DCF-based tracker

DSST for face tracking. In order to initialize the correlation tracker, we execute

the following command:

tracker = dlib.correlation_tracker()

This initializes the tracker with default values (filter_size = 6,num_scale_levels =

5, scale_window_size = 23, regularizer_space = 0.001, nu_space = 0.025, regularizer_scale =

0.001, nu_scale = 0.025, and scale_pyramid_alpha = 1.020). A higher value of filter_size

and num_scale_levels increases tracking accuracy, but it requires more

computational power, increasing CPU processing. The recommended values for

filter_size are 5, 6, and 7, and for num_scale_levels, 4, 5, and 6.

To begin tracking the method, tracker.start_track() is used. In this case, we

perform face detection. If successful, we will pass the position of the face to this

method, as follows:

if tracking_face is False:

gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

# Try to detect a face to initialize the tracker:

rects = detector(gray, 0)

# Check if we can start tracking (if we detected a face):

if len(rects) > 0:

# Start tracking:

tracker.start_track(frame, rects[0])

tracking_face = True

This way, the object tracker will start tracking what is inside the bounding box,

which, in this case, is the detected face.

Additionally, to update the position of the tracked object,

the tracker.update() method is called:

tracker.update(frame)

This method updates the tracker and returns the peak-to-side-lobe ratio, which is

a metric that measures how confident the tracker is. Larger values of this metric

a metric that measures how confident the tracker is. Larger values of this metric

indicate high confidence. This metric can be used to reinitialize the tracker with

frontal face detection.

To get the position of the tracked object, the tracker.get_position() method is

called:

pos = tracker.get_position()

This method returns the position of the object being tracked. Finally, we can

draw the predicted position of the face:

cv2.rectangle(frame, (int(pos.left()), int(pos.top())), (int(pos.right()), int(pos.bottom()))

In this script, we coded the option to reinitialize the tracker if the number 1 is

pressed. If this number is pressed, we reinitialize the tracker trying to detect a

frontal face. To clarify how this script works, the following two screenshots are

included.

In the first screenshot, the tracking algorithm is waiting until a frontal face

detection is performed to initialize the tracking:

In the second screenshot, the tracking algorithm is currently tracking a

previously detected face:

In the previous screenshot you can see that the algorithm is currently tracking

the detected face. You can also see that you can also press the number 1 in order

to re-initialize the tracking.

Related

An Introduction to QR Code Detection in OpenCV

QR code detection To complete this chapter, we are going to learn how to detect QR codes in images. This way, QR codes can also be used as markers for our augmented reality applications. The cv2.detectAndDecode() function both detects and decodes a QR code in the image containing the QR code. The image can be grayscale or color (BGR). This function returns the following: An array of vertices of the found QR code is returned. This array can be empty if the QR code is not found. The rectified and binarized QR code is returned. The data associated with this QR code is returned. In the qr_code_scanner.py script, we make use of the aforementioned function to detect and decode QR codes. The key points are commented next. First, the image is loaded, as follows: image = cv2.imread("qrcode_rotate_45_image.png") Next, we create the QR code detector with the following code: qr_code_detector = cv2.QRCodeDetector() Then, we apply the cv2.detectAndDecode() function, as follows: data, bbox, rectified_qr_code = qr_code_detector.detectAndDecode(image) We check whether the QR code is found before decoding the data and show the detection by using the show_qr_detection() function: if len(data) > 0: print("Decoded Data : {}".format(data)) show_qr_detection(image, bbox) The show_qr_detection() function draws both the lines and the corners of the detected QR code: def show_qr_detection(img, pts): """Draws both the lines and corners based on the array of vertices of the found QR code""" pts = np.int32(pts).reshape(-1, 2) for j in range(pts.shape[0]): cv2.line(img, tuple(pts[j]), tuple(pts[(j + 1) % pts.shape[0]]), (255, 0, 0), 5) for j in range(pts.shape[0]): cv2.circle(img, tuple(pts[j]), 10, (255, 0, 255), -1) The output of the qr_code_scanner.py script can be seen in the next screenshot: In the preceding screenshot, you can see the rectified and binarized QR code (left), and the detected marker (right), with a blue border, and magenta square points highlighting the detection.

Accessing and Manipulating Pixels of Grayscale Images in OpenCV: A Beginner Tutorial

OpenCV with grayscale images Grayscale images have only one channel. Therefore, some differences are introduced when working with these images. We are going to highlight these differences here. Again, we will use the cv2.imread() function to read an image. In this case, the second argument is needed because we want to load the image in grayscale. The second argument is a flag specifying the way the image should be read. The value that's needed for loading an image in grayscale is cv2.IMREAD_GRAYSCALE: # The function cv2.imshow() is used to display an image in a window # The first argument of this function is the window name # The second argument of this function is the image to be shown. # In this case, the second argument is needed because we want to load the image in grayscale. # Second argument is a flag specifying the way the image should be read. # Value needed for loading an image in grayscale: 'cv2.IMREAD_GRAYSCALE'. # load OpenCV logo image: gray_img = cv2.imread('logo.png', cv2.IMREAD_GRAYSCALE) In this case, we store the image in the gray_img variable. If we get the dimensions of the image (using gray_img.shape), we will get only two values that is, rows and columns. In grayscale images, the channel information is not provided: # To get the dimensions of the image use img.shape # If color image, img.shape returns returns a tuple of number of rows, columns and channels # If grayscale, returns a tuple of number of rows and columns. # So, it can be used to check if the loaded image is grayscale or color image. # Get the shape of the image (in this case only two components!): dimensions = gray_img.shape img.shape will return the dimensions of the image in a tuple, like this—(99, 82). A pixel value can be accessed by row and column coordinates. In grayscale images, only one value is obtained (usually called the intensity of the pixel). For example, if we want to get the intensity of the pixel (x=40, y=6), we would use the following code: # You can access a pixel value by row and column coordinates. # For BGR image, it returns an array of (Blue, Green, Red) values. # Get the value of the pixel (x=40, y=6): i = gray_img[6, 40] The pixel values of the image can be also modified in the same way. For example, if we want to change the value of the pixel (x=40, y=6) to black (intensity equals to 0), we would use the following code: # You can modify the pixel values of the image in the same way. # Set the pixel to black: gray_img[6, 40] = 0

Recongnizing Handwritten Digits Using SVM in OpenCV

Handwritten digit recognition using SVM We just saw how to perform handwritten digit recognition using the kNN classifier. The best accuracy was obtained by preprocessing the digits (recall the deskew() function) and computing the HOG descriptor as the feature vector used to describe each digit. Therefore, for the sake of simplicity, the next scripts, which are going to make use of SVM to classify the digits, will use the aforementioned approximation (preprocessing and HOG features). The svm_handwritten_digits_recognition_preprocessing_hog.py script performs the handwritten digit recognition using SVM classification. The key code is shown as follows: # Load all the digits and the corresponding labels: digits, labels = load_digits_and_labels('digits.png') # Shuffle data # Constructs a random number generator: rand = np.random.RandomState(1234) # Randomly permute the sequence: shuffle = rand.permutation(len(digits)) digits, labels = digits[shuffle], labels[shuffle] # HoG feature descriptor: hog = get_hog() # Compute the descriptors for all the images. # In this case, the HoG descriptor is calculated hog_descriptors = [] for img in digits: hog_descriptors.append(hog.compute(deskew(img))) hog_descriptors = np.squeeze(hog_descriptors) # At this point we split the data into training and testing (50% for each one): partition = int(0.5 * len(hog_descriptors)) hog_descriptors_train, hog_descriptors_test = np.split(hog_descriptors, [partition]) labels_train, labels_test = np.split(labels, [partition]) print('Training SVM model ...') model = svm_init(C=12.5, gamma=0.50625) svm_train(model, hog_descriptors_train, labels_train) print('Evaluating model ... ') svm_evaluate(model, hog_descriptors_test, labels_test) In this case, we have used an RBF kernel: def svm_init(C=12.5, gamma=0.50625): """Creates empty model and assigns main parameters""" model = cv2.ml.SVM_create() model.setGamma(gamma) model.setC(C) model.setKernel(cv2.ml.SVM_RBF) model.setType(cv2.ml.SVM_C_SVC) model.setTermCriteria((cv2.TERM_CRITERIA_MAX_ITER, 100, 1e-6)) return model The obtained accuracy is 98.60%, using only 50% of the digits to train the algorithm. Additionally, when using the RBF kernel, there are two important parameters—C and γ. In this case, C=12.5 and γ=0.50625. As before, C and γ are unknown as the best for a given problem (dataset dependent). Therefore, some kind of parameter search must be done. Hence, the goal is to identify good (C and γ) where a grid- search on C and γ is recommended. In the svm_handwritten_digits_recognition_preprocessing_hog_c_gamma.py script, two modifications are carried out in comparison with […]

Implement Skin Segmentation of Images in Different Color Space Using OpenCV

Skin segmentation in different color spaces The aforementioned color spaces can be used in different image processing tasks and techniques. For example, the skin_segmentation.py script implements different algorithms to perform skin segmentation working in different color spaces (YCrCb, HSV, and RGB). This script also loads several test images to see how these algorithms work. The key functions in this script are cv2.cvtColor(), which we have already mentioned on, and cv2.inRange(), which checks whether the elements contained in an array lie between the elements of two other arrays (the lower boundary array and the upper boundary array). Therefore, we use the cv2.inRange() function to segment the colors corresponding to the skin. As you can see, the values defined for these two arrays (lower and upper boundaries) play a critical role in the performance of the segmentation algorithms. In this way, a wide investigation has been carried out in order to set them properly. In this example, the values are obtained from the following research papers: RGB-H-CbCr Skin Color Model for Human Face Detection by Nusirwan Anwar, Abdul Rahman, K. C. Wei, and John See Skin segmentation algorithm based on the YCrCb color space by Shruti D Patravali, Jyoti Waykule, and Apurva Katre Face Segmentation Using Skin-Color Map in Videophone Applications by D. Chai and K.N. Ngan The skin_detectors dictionary has been built to apply all the skin segmentation algorithms to the test images. If we print it, the output will be as follows: index: '0', key: 'ycrcb', value: '<function skin_detector_ycrcb at 0x07B8C030>' index: '1', key: 'hsv', value: '<function skin_detector_hsv at 0x07B8C0C0>' index: '2', key: 'hsv_2', value: '<function skin_detector_hsv_2 at 0x07B8C108>' index: '3', key: 'bgr', value: '<function skin_detector_bgr at 0x07B8C1E0>' You can see that there are four skin detectors defined. In order to call a skin segmentation detector (for example, skin_detector_ycrcb), you must perform the following: detected_skin = skin_detectors['ycrcb'](image) The output of the script can be seen in the following screenshot: You can see the effect of applying different skin segmentation algorithms using several test images to see how these algorithms work under different conditions.

Web Computer Vision Applications Using OpenCV and Flask: A Beginner Tutorial

Web computer vision applications using OpenCV and Flask In this section, we will see how to create web computer vision applications using OpenCV and Flask. We will start with the equivalent Hello world application using OpenCV and Flask. A minimal example to introduce OpenCV and Flask Script hello_opencv.py is coded in order to show how you can use OpenCV to perform a very basic web computer vision application. The code of this script is shown next: # Import required packages: import cv2 from flask import Flask, request, make_response import numpy as np import urllib.request app = Flask(__name__) @app.route('/canny', methods=['GET']) def canny_processing(): # Get the image: with urllib.request.urlopen(request.args.get('url')) as url: image_array = np.asarray(bytearray(url.read()), dtype=np.uint8) # Convert the image to OpenCV format: img_opencv = cv2.imdecode(image_array, -1) # Convert image to grayscale: gray = cv2.cvtColor(img_opencv, cv2.COLOR_BGR2GRAY) # Perform canny edge detection: edges = cv2.Canny(gray, 100, 200) # Compress the image and store it in the memory buffer: retval, buffer = cv2.imencode('.jpg', edges) # Build the response: response = make_response(buffer.tobytes()) response.headers['Content-Type'] = 'image/jpeg' # Return the response: return response if __name__ == "__main__": # Add parameter host='0.0.0.0' to run on your machines IP address: app.run(host='0.0.0.0') The previous code can be explained with the help of the following steps: 1. The first step is to import the required packages. In this example, we have used the route() decorator to bind the canny_processing() function to the /canny URL. Additionally, the url parameter is also needed to perform the GET request correctly. In order to get this parameter, the request.args.get() function is used. 2. The next step is to read the image this URL holds as follows: with urllib.request.urlopen(request.args.get('url')) as url: image_array = np.asarray(bytearray(url.read()), dtype=np.uint8) This way, the image is read as an array. 3. The next step is to convert the image to OpenCV format and perform Canny edge processing, which should be performed on the corresponding grayscale image: # Convert the image to OpenCV format: img_opencv = cv2.imdecode(image_array, -1) […]