Implement Object Tracking with the dlib DCF-based Tracker: A Beginner Guide
OpenCV Tutorial
Implement Object Tracking with the dlib DCF-based Tracker: A Beginner Guide

based tracker

The face_tracking_correlation_filters.py script can be modified to track an arbitrary

object. In this case, we will use the mouse to select the object to track. If we

press 1, the algorithm will start tracking the object inside the pre-defined

bounding box. Additionally, if we press 2, the pre-defined bounding box will be

emptied and the tracking algorithm will be stopped, allowing the user to select

another bounding box.

To clarify how the face_tracking_correlation_filters.py script works, we have

included the next two screenshots. In the first one, we can see that we need to

select a bounding box to start the tracking:

In the second one, we can see the output of an arbitrary frame when the

algorithm is tracking the object:

As you can see in the previous screenshot, the algorithm is tracking the object

inside the bounding box.

Related

Rotating an Image in OpenCV: A Beginner Guide

Rotating an image In order to rotate the image, we make use of the cv.getRotationMatrix2D() function to build the 2 x 3 transformation matrix. This matrix rotates the image at the desired angle (in degrees), where positive values indicate a counterclockwise rotation. Both the center of rotation and the scale factor can also be adjusted. Using these elements in our example, the following transformation matrix is calculated: This expression has the following values: The following example builds the M transformation matrix to rotate 180 degrees with respect to the center of the image with a scale factor of 1 (without scaling). Afterwards, this M matrix is applied to the image, as follows: height, width = image.shape[:2] M = cv2.getRotationMatrix2D((width / 2.0, height / 2.0), 180, 1) dst_image = cv2.warpAffine(image, M, (width, height)) In order to rotate the image, we make use of the cv.getRotationMatrix2D() function to build the 2 x 3 transformation matrix. This matrix rotates the image at the desired angle (in degrees), where positive values indicate a counterclockwise rotation. Both the center of rotation and the scale factor can also be adjusted. Using these elements in our example, the following transformation matrix is calculated: This expression has the following values: The following example builds the M transformation matrix to rotate 180 degrees with respect to the center of the image with a scale factor of 1 (without scaling). Afterwards, this M matrix is applied to the image, as follows: height, width = image.shape[:2] M = cv2.getRotationMatrix2D((width / 2.0, height / 2.0), 180, 1) dst_image = cv2.warpAffine(image, M, (width, height))

Drawing Lines, Rectangles and Circles in OpenCV: A Beginner Tutorial

Basic shapes – lines, rectangles, and circles In the next example, we are going to see how to draw basic shapes in OpenCV. These basic shapes include lines, rectangles, and circles, which are the most common and simple shapes to draw. The first step is to create an image where the shapes will be drawn. To this end, a 400 400 image with the 3 channels (to properly display a BGR image) and a uint8 type (8-bit unsigned integers) will be created: # We create the canvas to draw: 400 x 400 pixels, 3 channels, uint8 (8-bit unsigned integers) # We set the background to black using np.zeros() image = np.zeros((400, 400, 3), dtype="uint8") We set the background to light gray using the colors dictionary: # If you want another background color, you can do the following: image[:] = colors['light_gray'] This canvas (or image) is shown in the next screenshot: Now, we are ready to draw the basic shapes. It should be noted that most drawing functions that OpenCV provides have common parameters. For the sake of simplicity, these parameters are briefly introduced here: img: It is the image where the shape will be drawn. color: It is the color (BGR triplet) used to draw the shape. thickness: If this value is positive, it is the thickness of the shape outline. Otherwise, a filled shape will be drawn. lineType: It is the type of the shape boundary. OpenCV provides three types of line: cv2.LINE_4: This means four-connected lines cv2.LINE_8: This means eight-connected lines shift: This indicates the number of fractional bits in connection with the coordinates of some points defining the shape. In connection with the aforementioned parameters, the cv2.LINE_AA option for lineType produces a much better quality drawing (for example, when drawing text), but it is slower to draw. So, this consideration should be taken into account. Both the eight-connected and the four-connected lines, which are non- antialiased lines, are drawn with the Bresenham algorithm. For the anti-aliased line type, the Gaussian filtering algorithm is used. Additionally, the shift parameter is necessary because many drawing functions can't deal with sub-pixel accuracy. For simplicity in our examples, we are going to work with integer coordinates. Therefore, this value will be set to 0 (shift = 0). However to give you a complete understanding, an example of how to use the shift parameter will also be provided. Remember that, for all the examples included in this section, a canvas has been created to draw all the shapes. This canvas is a 400 x 400 pixel image, with a light gray background. See the previous screenshot, which shows this canvas. Drawing lines The first function we are going to see is cv2.line(). The signature is as follows: img = line(img, pt1, pt2, color, thickness=1, lineType=8, shift=0) This function draws a line on the img image connecting pt1 and pt2: cv2.line(image, (0, 0), (400, 400), colors['green'], 3) cv2.line(image, (0, 400), (400, 0), colors['blue'], 10) cv2.line(image, (200, 0), (200, 400), colors['red'], 3) cv2.line(image, (0, 200), (400, 200), colors['yellow'], 10) After coding these lines, we call the show_with_matplotlib(image, 'cv2.line()') function. The result is shown in the next screenshot: Drawing rectangles The signature for the cv2.rectangle() function is as follows: […]

Calculating Frames of Video Per Second Using OpenCV

Calculating frames per second In the Reading camera frame and video files section, we saw how we can get some properties from the cv2.VideoCapture object. fps is an important metric in computer vision projects. This metric indicates how many frames are processed per second. It is safe to say that a higher number of fps is better. However, the number of frames your algorithm should process every second will depend on the specific problem you have to solve. For example, if your algorithm should track and detect people walking down the street, 15 fps is probably enough. But if your goal is to detect and track cars going fast on a highway, 20-25 fps are probably necessary. Therefore, it is important to know how to calculate the fps metric in your computer vision projects. In the following example, read_camera_fps.py, we are going to modify read_camera.py to output the number of fps. The key points are shown in the following code: # Read until the video is completed, or 'q' is pressed while capture.isOpened(): # Capture frame-by-frame from the camera ret, frame = capture.read() if ret is True: # Calculate time before processing the frame: processing_start = time.time() # All the processing should be included here # ... # ... # End of processing # Calculate time after processing the frame processing_end = time.time() # Calculate the difference processing_time_frame = processing_end - processing_start # FPS = 1 / time_per_frame # Show the number of frames per second print("fps: {}".format(1.0 / processing_time_frame)) # Break the loop else: break First, we take the time before the processing is done: Then, we take the time after all the processing is done: processing_end = time.time() Following that, we calculate the difference: processing_time_frame = processing_end - processing_start Finally, we calculate and print the number of fps: print("fps: {}".format(1.0 / processing_time_frame))

Implement Face Recognition with OpenCV: A Beginner Guide

Face recognition with OpenCV OpenCV provides support to perform face recognition (https://docs.opencv.org/4.0. 1/dd/d65/classcv_1_1face_1_1FaceRecognizer.html). Indeed, OpenCV provides three different implementations to use: Eigenfaces Fisherfaces Local Binary Patterns Histograms (LBPH) These implementations perform the recognition in different ways. However, you can use any of them by changing only the way the recognizers are created. More specifically, to create these recognizers, the following code is necessary: face_recognizer = cv2.face.LBPHFaceRecognizer_create() face_recognizer = cv2.face.EigenFaceRecognizer_create() face_recognizer = cv2.face.FisherFaceRecognizer_create() Once created, and independently of the specific internal algorithm OpenCV is going to use to perform the face recognition, the two key methods, train() and predict(), should be used to perform both the training and the testing of the face recognition system, and it should be noted that the way we use these methods is independent of the recognizer created. Therefore, it is very easy to try the three recognizers and select the one that offers the best performance for a specific task. Having said that, LBPH should provide better results than the other two methods when recognizing images in the wild, where different environments and lighting conditions are usually involved. Additionally, the LBPH face recognizer supports the update() method, where you can update the face recognizer given new data. For the Eigenfaces and Fisherfaces methods, this functionality is not possible. In order to train the recognizer, the train() method should be called: face_recognizer.train(faces, labels) The cv2.face_FaceRecognizer.train(src, labels) method trains the specific face recognizer, where src corresponds to the training set of images (faces), and parameter labels set the corresponding label for each image in the training set. To recognize a new face, the predict() method should be called: label, confidence = face_recognizer.predict(face) The cv2.face_FaceRecognizer.predict(src) method outputs (predicts) the recognition of the new src image by outputting the predicted label and the associated confidence. Finally, OpenCV also provides the write() and read() methods to save the created model and to load a previously created model, respectively. For both methods, the filename parameter sets the name of the model to save or load: cv2.face_FaceRecognizer.write(filename) cv2.face_FaceRecognizer.read(filename) As mentioned, the LBPH face recognizer can be updated using the update() method: cv2.face_FaceRecognizer.update(src, labels) Here, src and labels set the new training examples that are going to be used to update the LBPH recognizer.

Implement Face Detection Using OpenCV: A Beginner Example

Face detection with OpenCV OpenCV provides two approaches for face detection: Haar cascade based face detectors Deep learning-based face detectors The framework proposed by Viola and Jones (see Rapid Object Detection Using a Boosted Cascade of Simple Features (2001)) is an effective object detection method. This framework is very popular because OpenCV provides face detection algorithms based on this framework. Additionally, this framework can also be used for detecting other objects rather than faces (for example, full body detector, plate number detector, upper body detector, or cat face detector). In this section, we will see how to detect faces using this framework. The face_detection_opencv_haar.py script performs face detection using haar feature- based cascade classifiers. In this sense, OpenCV provides four cascade classifiers to use for (frontal) face detection: haarcascade_frontalface_alt.xml (FA1): 22 stages and 20 x 20 haar features haarcascade_frontalface_alt2.xml (FA2): 20 stages and 20 x 20 haar features haarcascade_frontalface_alt_tree.xml (FAT): 47 stages and 20 x 20 haar features haarcascade_frontalface_default.xml (FD): 25 stages and 24 x 24 haar features In some available publications, the authors evaluated the performance of these cascade classifiers using different criteria and datasets. Overall, it can be concluded that these classifiers achieve similar accuracy. That is why, in this script, we will be using two of them (to simplify things). More specifically, in this script, two cascade classifiers (the previously introduced FA2 and FD) are loaded: # Load cascade classifiers: cas_alt2 = cv2.CascadeClassifier("haarcascade_frontalface_alt2.xml") cas_default = cv2.CascadeClassifier("haarcascade_frontalface_default.xml") The cv2.CascadeClassifier() function is used to load a classifier from a file. You can download these cascade classifier files from the OpenCV repository: https:// github.com/opencv/opencv/tree/master/data/haarcascades. Moreover, we have included the two loaded cascade classifier files in the GitHub repository (haarcascade_frontalface_alt2.xml and haarcascade_frontalface_default.xml). The next step is to perform the detection: faces_alt2 = cas_alt2.detectMultiScale(gray) faces_default = cas_default.detectMultiScale(gray) The cv2.CascadeClassifier.detectMultiScale() function detects objects and returns them as a list of rectangles. The final step is to correlate the results using the show_detection() function: img_faces_alt2 = show_detection(img.copy(), faces_alt2) img_faces_default = show_detection(img.copy(), faces_default) The show_detection() function draws a rectangle over each detected face: def show_detection(image, faces): """Draws a rectangle over each detected face""" for (x, y, w, h) in faces: cv2.rectangle(image, (x, y), (x + w, y + h), (255, 0, 0), 5) return image OpenCV also provides the cv2.face.getFacesHAAR() function to detect faces: retval, faces_haar_alt2 = cv2.face.getFacesHAAR(img, "haarcascade_frontalface_alt2.xml") retval, faces_haar_default = cv2.face.getFacesHAAR(img, "haarcascade_frontalface_default.xml") It should be noted that cv2.CascadeClassifier.detectMultiScale() needs a grayscale image, while cv2.face.getFacesHAAR() needs a BGR image as an input. Moreover, cv2.CascadeClassifier.detectMultiScale() outputs the detected faces as a list of rectangles. For example, the output for two detected faces will be like this: [[332 93 364 364] [695 104 256 256]] The cv2.face.getFacesHAAR() function returns the faces in a similar format: […]

Understanding Adaptive Thresholding in OpenCV: A Beginner Tutorial

Adaptive thresholding In the previous section, we have applied cv2.threshold() using a global threshold value. As we could see, the obtained results were not very good due to the different illumination conditions in the different areas of the image. In these cases, you can try adaptive thresholding. In OpenCV, the adaptive thresholding is performed by the cv2.adapativeThreshold() function. The signature for this method is as follows: adaptiveThreshold(src, maxValue, adaptiveMethod, thresholdType, blockSize, C[, dst]) -> dst This function applies an adaptive threshold to the src array (8-bit single-channel image). The maxValue parameter sets the value for the pixels in the dst image for which the condition is satisfied. The adaptiveMethod parameter sets the adaptive thresholding algorithm to use: cv2.ADAPTIVE_THRESH_MEAN_C: The T(x, y) threshold value is calculated as the mean of the blockSize x blockSize neighborhood of (x, y) minus the C parameter cv2.ADAPTIVE_THRESH_GAUSSIAN_C: The T(x, y) threshold value is calculated as the weighted sum of the blockSize x blockSize neighborhood of (x, y) minus the C parameter The blockSize parameter sets the size of the neighborhood area used to calculate a threshold value for the pixel, and it can take the values 3, 5, 7,... and so forth. The C parameter is just a constant subtracted from the means or weighted means (depending on the adaptive method set by the adaptiveMethod parameter). Commonly, this value is positive, but it can be zero or negative. Finally, the thresholdType parameter sets the cv2.THRESH_BINARY or cv2.THRESH_BINARY_INV thresholding types. According to the following formula where T(x, y) is the threshold calculated for each pixel, the thresholding_adaptive.py script applies adaptive thresholding to a test image using the cv2.ADAPTIVE_THRESH_MEAN_C and cv2.ADAPTIVE_THRESH_GAUSSIAN_C methods: Here is the formula for cv2.THRESH_BINARY: Here is the formula for cv2.THRESH_BINARY_INV: The output of this script can be seen in the following screenshot: In the previous screenshot, you can see the output after applying cv2.adaptiveThreshold() with different parameters. As previously mentioned, if your task is to recognize the digits, the adaptive thresholding can give you better thresholded images. However, as you can also see, a lot of noise appears in the image. In order to deal with it, you can apply some smoothing operations (see Cha pter 5, Image Processing Techniques). In this case, we can apply a bilateral filter, because it is highly useful in noise removal while maintaining sharp edges. In order to apply a bilateral filter, OpenCV provides the cv2.bilateralFilter() function. Therefore, we can apply the function before thresholding the image as follows: gray_image = cv2.bilateralFilter(gray_image, 15, 25, 25) The code for this example can be seen in the thresholding_adaptive_filter_noise.py script. The output can be seen in the following screenshot: You can see that applying a smoothing filter is a good solution to deal with noise. In this case, a bilateral filter is applied because we want to keep the edges sharp.