Detecting Facial Landmarks with face_recognition Package: A Beginner Guide
OpenCV Tutorial
Detecting Facial Landmarks with face_recognition Package: A Beginner Guide

face_recognition

The landmarks_detection_fr.py script shows you how to both detect and draw facial

landmarks using the face_recognition package.

In order to detect landmarks, the face_recognition.face_landmarks() function is called,

as follows:

# Detect 68 landmarks:

face_landmarks_list_68 = face_recognition.face_landmarks(rgb)

This function returns a dictionary of facial landmarks (for example, eyes and

nose) for each face in the image. For example, if we print the detected

landmarks, the output is as follows:

[{'chin': [(113, 251), (111, 283), (115, 315), (122, 346), (136, 376), (154, 402), (177, 425), (203, 442), (231, 447), (260, 442), (285, 426), (306, 403), (323, 377), (334, 347), (340, 315), (343, 282), (343, 251)], 'left_eyebrow': [(123, 223), (140, 211), (163, 208), (185, 211), (206, 220)], 'right_eyebrow': [(240, 221), (263, 212), (288, 209), (312, 211), (332, 223)], 'nose_bridge': [(225, 249), (225, 272), (225, 295), (226, 319)], 'nose_tip': [(201, 337), (213, 340), (226, 343), (239, 339), (252, 336)], 'left_eye': [(144, 248), (158, 239), (175, 240), (188, 254), (173, 255), (156, 254)], 'right_eye': [(262, 254), (276, 240), (293, 239), (308, 248), (295, 254), (278, 255)], 'top_lip': [(185, 377), (200, 370), (216, 364), (226, 367), (238, 364), (255, 370), (274, 377), (267, 378), (238, 378), (227, 380), (215, 379), (192, 378)], 'bottom_lip': [(274, 377), (257, 391), (240, 399), (228, 400), (215, 398), (200, 391), (185, 377), (192, 378), (215, 381), (227, 382), (239, 380), (267, 378)]}]

The final step is to draw the detected landmarks:

# Draw all detected landmarks:

for face_landmarks in face_landmarks_list_68:

for facial_feature in face_landmarks.keys():

for p in face_landmarks[facial_feature]:

cv2.circle(image_68, p, 2, (0, 255, 0), -1)

It should be noted that the signature of the face_recognition.face_landmarks() method

is as follows:

face_landmarks(face_image, face_locations=None, model="large")

Therefore, by default, the 68 feature points are detected. If model="small", only 5

feature points will be detected:

# Detect 5 landmarks:

face_landmarks_list_5 = face_recognition.face_landmarks(rgb, None, "small")

If we print face_landmarks_list_5, we get the following output:

[{'nose_tip': [(227, 343)], 'left_eye': [(145, 248), (191, 253)], 'right_eye': [(307, 248), (262, 252)]}]

In this case, the dictionary only contains facial feature locations for both eyes

and the tip of the nose.

The output of the landmarks_detection_fr.py script can be seen in the following

screenshot:

In the screenshot above, you can see the result of drawing both the detected 68

and 5 facial landmarks using face_recognition package.

Related

Implement Face Detection Using cvlib Package: A Beginner Example

Face detection with cvlib For the sake of completeness, we are introducing the cvlib package in this section because it also provides an algorithm for face detection. This library is a simple, high-level and easy-to-use open source computer vision library for Python (https: //github.com/arunponnusamy/cvlib). In order to detect faces with cvlib, you can use the detect_face() function, which will return the bounding boxes and corresponding confidences for all detected faces: import cvlib as cv faces, confidences = cv.detect_face(image) Under the hood, this function is using the OpenCV DNN face detector with pre- trained Caffe models (https://github.com/arunponnusamy/cvlib/blob/master/cvlib/face_det ection.py). Refer to the face_detection_cvlib_dnn.py script for further details. Face detection with cvlib For the sake of completeness, we are introducing the cvlib package in this section because it also provides an algorithm for face detection. This library is a simple, high-level and easy-to-use open source computer vision library for Python (https: //github.com/arunponnusamy/cvlib). In order to detect faces with cvlib, you can use the detect_face() function, which will return the bounding boxes and corresponding confidences for all detected faces: import cvlib as cv faces, confidences = cv.detect_face(image) Under the hood, this function is using the OpenCV DNN face detector with pre- trained Caffe models (https://github.com/arunponnusamy/cvlib/blob/master/cvlib/face_det ection.py). Refer to the face_detection_cvlib_dnn.py script for further details.

Understanding Contrast Limited Adaptive Histogram Equalization (CLAHE) in OpenCV: A Beginner Guide

Contrast Limited Adaptive Histogram Equalization In this section, we are going to see how to apply contrast limited adaptive histogram equalization (CLAHE) to equalize images, which is a variant of adaptive histogram equalization (AHE), in which contrast amplification is limited. The noise in relatively homogeneous regions of the image is overamplified by AHE, while CLAHE tackles this problem by limiting the contrast amplification. This algorithm can be applied to improve the contrast of images. This algorithm works by creating several histograms of the original image, and uses all of these histograms to redistribute the lightness of the image. In the clahe_histogram_equalization.py script, we are applying CLAHE to both grayscale and color images. When applying CLAHE, there are two parameters to tune. The first one is clipLimit, which sets the threshold for contrast limiting. The default value is 40. The second one is tileGridSize, which sets the number of tiles in the row and column. When applying CLAHE, the image is divided into small blocks called tiles (8 x 8 by default) in order to perform its calculations. To apply CLAHE to a grayscale image, we must perform the following: clahe = cv2.createCLAHE(clipLimit=2.0) gray_image_clahe = clahe.apply(gray_image) Additionally, we can also apply CLAHE to color images, very similar to the approach commented in the previous section for the contrast equalization of color images, where the results after equalizing only the luminance channel of an HSV image are much better than equalizing all the channels of the BGR image. In this section, we are going to create four functions in order to equalize the color images by using CLAHE only on the luminance channel of different color spaces: def equalize_clahe_color_hsv(img): """Equalize the image splitting after conversion to HSV and applying CLAHE to the V channel and merging the channels and convert back to BGR """ cla = cv2.createCLAHE(clipLimit=4.0) H, S, V = cv2.split(cv2.cvtColor(img, cv2.COLOR_BGR2HSV)) eq_V = cla.apply(V) eq_image = cv2.cvtColor(cv2.merge([H, S, eq_V]), cv2.COLOR_HSV2BGR) return eq_image def equalize_clahe_color_lab(img): """Equalize the image splitting after conversion to LAB and applying CLAHE to the L channel and merging the channels and convert back to BGR """ cla = cv2.createCLAHE(clipLimit=4.0) L, a, b = cv2.split(cv2.cvtColor(img, cv2.COLOR_BGR2Lab)) eq_L = cla.apply(L) eq_image = cv2.cvtColor(cv2.merge([eq_L, a, b]), cv2.COLOR_Lab2BGR) return eq_image def equalize_clahe_color_yuv(img): """Equalize the image splitting after conversion to YUV and applying CLAHE to the Y channel and merging the channels and convert back to BGR """ cla = cv2.createCLAHE(clipLimit=4.0) Y, U, V = cv2.split(cv2.cvtColor(img, cv2.COLOR_BGR2YUV)) eq_Y = cla.apply(Y) eq_image = cv2.cvtColor(cv2.merge([eq_Y, U, V]), cv2.COLOR_YUV2BGR) return eq_image def equalize_clahe_color(img): """Equalize the image splitting the image applying CLAHE to each channel and merging the results""" […]

Implement Face Recognition with dlib Library: A Beginner Guide

Face recognition with dlib Dlib offers a high-quality face recognition algorithm based on deep learning. Dlib implements a face recognition algorithm that offers state-of-the-art accuracy. More specifically, the model has an accuracy of 99.38% on the labeled faces in the wild database. The implementation of this algorithm is based on the ResNet-34 network proposed in the paper Deep Residual Learning for Image Recognition (2016), which was trained using three million faces. The created model (21.4 MB) can be downloaded from https://github.com/davisking/dlib-models/blob/master/dlib_face_rec ognition_resnet_model_v1.dat.bz2. This network is trained in a way that generates a 128-dimensional (128D) descriptor, used to quantify the face. The training step is performed using triplets. A single triplet training example is composed of three images. Two of them correspond to the same person. The network generates the 128D descriptor for each of the images, slightly modifying the neural network weights in order to make the two vectors that correspond to the same person closer and the feature vector from the other person further away. The triplet loss function formalizes this and tries to push the 128D descriptor of two images of the same person closer together, while pulling the 128D descriptor of two images of different people further apart. This process is repeated millions of times for millions of images of thousands of different people and finally, it is able to generate a 128D descriptor for each person. So, the final 128D descriptor is good encoding for the following reasons: The generated 128D descriptors of two images of the same person are quite similar to each other. The generated 128D descriptors of two images of different people are very different. Therefore, making use of the dlib functionality, we can use a pre-trained model to map a face into a 128D descriptor. Afterward, we can use these feature vectors to perform face recognition. The encode_face_dlib.py script shows how to calculate the 128D descriptor, used to quantify the face. The process is quite simple, as shown in the following code: # Load image: image = cv2.imread("jared_1.jpg") # Convert image from BGR (OpenCV format) to RGB (dlib format): rgb = image[:, :, ::-1] # Calculate the encodings for every face of the image: encodings = face_encodings(rgb) # Show the first encoding: print(encodings[0]) As you can guess, the face_encodings() function returns the 128D descriptor for each face in the image: pose_predictor_5_point = dlib.shape_predictor("shape_predictor_5_face_landmarks.dat") face_encoder = dlib.face_recognition_model_v1("dlib_face_recognition_resnet_model_v1.dat") detector = dlib.get_frontal_face_detector() def face_encodings(face_image, number_of_times_to_upsample=1, num_jitters=1): """Returns the 128D descriptor for each face in the image""" # Detect faces: face_locations = detector(face_image, number_of_times_to_upsample) # Detected landmarks: raw_landmarks = [pose_predictor_5_point(face_image, face_location) for face_location # Calculate the face encoding for every detected face using the detected landmarks for each one: return [np.array(face_encoder.compute_face_descriptor(face_image, raw_landmark_set, raw_landmark_set in raw_landmarks] As you can see, the key point is to calculate the face encoding for every detected […]

An Introduction to OpenCV Modules for Beginners

OpenCV (since version 2) is divided into several modules, where each module can be understood, in general, as being dedicated to one group of computer vision problems. This division can be seen in the next diagram, where the main modules are shown: OpenCV modules are shortly described here: core: Core functionality. Core functionality is a module defining basic data structures and also basic functions used by all other modules in the library. imgproc: Image processing. An image-processing module that includes image filtering, geometrical image transformations, color space conversion, and histograms. imgcodecs: Image codecs. Image file reading and writing. videoio: Video I/O. An interface to video capturing and video codecs. highgui: High-level GUI. An interface to UI capabilities. It provides an interface to easily do the following: Add trackbars to the windows, keyboard commands, and handle mouse events video: Video analysis. A video-analysis module including background subtraction, motion estimation, and object-tracking algorithms. calib3d: Camera calibration and 3D reconstruction. Camera calibration and 3D reconstruction covering basic multiple-view geometry algorithms, stereo correspondence algorithms, object pose estimation, both single and stereo camera calibration, and also 3D reconstruction. features2d: 2D features framework. This module includes feature detectors, descriptors, and descriptor matchers. objdetect: Object detection. Detection of objects and instances of predefined classes (for example, faces, eyes, people, and cars). dnn: Deep neural network (DNN) module. This module contains the following: API for new layers creation Set of built useful layers API to construct and modify neural networks from layers Functionality for loading serialized networks models from different deep learning frameworks ml: Machine learning. The Machine Learning Library (MLL) is a set of classes and methods that can be used for classification, regression, and clustering purposes. flann: Clustering and search in multi-dimensional spaces. Fast Library for Approximate Nearest Neighbors (FLANN) is a collection of algorithms that are highly suited for fast nearest-neighbor searches. photo: Computational photography. This module provides some functions for computational photography. stitching: Images stitching. This module implements a stitching pipeline that performs automatic panoramic image stitching. shape: Shape distance and matching. Shape distance and matching module that can be used for shape matching, retrieval, or comparison. superres: Super-resolution. This module contains a set of classes and methods that can be used for resolution enhancement. videostab: Video stabilization. This module contains a set of classes and methods for video stabilization. viz: 3D visualizer. This module is used to display widgets that provide several methods to interact with scenes and widgets.

Understand Thresholding Algorithms Using scikit-image

Introducing thresholding with scikit- image In order to test scikit-image, we are going to threshold a test image using Otsu's binarization algorithm. In order to try this method, the first step is to import the required packages. In this case, in connection with scikit-image as follows: from skimage.filters import threshold_otsu from skimage import img_as_ubyte The key code to apply Otsu's binarization algorithm with scikit-image is the following: thresh = threshold_otsu(gray_image) binary = gray_image > thresh binary = img_as_ubyte(binary) The threshold_otsu(gray_image) function returns the threshold value based on Otsu's binarization algorithm. Afterwards, with this value, the binary image is constructed (dtype= bool), which should be converted to 8-bit unsigned integer format (dtype= uint8) for proper visualization. The img_as_ubyte() function is used for this purpose. The full code for this example can be seen in the thresholding_scikit_image_otsu.py script. The output can be seen in the following screenshot: We will now try out some thresholding techniques with scikit-image. Trying out more thresholding techniques with scikit-image We are going to threshold a test image comparing Otsu's, triangle, Niblack's, and Sauvola's thresholding techniques. Otsu and triangle are global thresholding techniques, while Niblack and Sauvola are local thresholding techniques. Local thresholding techniques are considered a better approach when the background is not uniform. For more information about Niblack's and Sauvola's thresholding algorithms, see An Introduction to Digital Image Processing (1986) and Adaptive document image binarization (2000), respectively. The full code for this example can be seen in the thresholding_scikit_image_techniques.py script. In order to try these methods, the first step is to import the required packages. In this case, in connection with scikit-image as follows: from skimage.filters import (threshold_otsu, threshold_triangle, threshold_niblack, threshold_sauvola) from skimage import img_as_ubyte In order to perform the thresholding operations with scikit-image, we call each thresholding method (threshold_otsu(), threshold_niblack(), threshold_sauvola(), and threshold_triangle()): # Trying Otsu's scikit-image algorithm: thresh_otsu = threshold_otsu(gray_image) binary_otsu = gray_image > thresh_otsu binary_otsu = img_as_ubyte(binary_otsu) # Trying Niblack's scikit-image algorithm: thresh_niblack = threshold_niblack(gray_image, window_size=25, k=0.8) binary_niblack = gray_image > thresh_niblack binary_niblack = img_as_ubyte(binary_niblack) # Trying Sauvola's scikit-image algorithm: thresh_sauvola = threshold_sauvola(gray_image, window_size=25) binary_sauvola = gray_image > thresh_sauvola binary_sauvola = img_as_ubyte(binary_sauvola) # Trying triangle scikit-image algorithm: thresh_triangle = threshold_triangle(gray_image) binary_triangle = gray_image > thresh_triangle binary_triangle = img_as_ubyte(binary_triangle) The output can be seen in the next screenshot: […]

Accessing and Manipulating Pixels of BGR Images in OpenCV: A Beginner Tutorial

OpenCV with BGR images Now, we are going to see how we can work with BGR images in OpenCV. OpenCV loads the color images so that the blue channel is the first, the green channel is the second, and the red channel is the third. Please see the Accessing and manipulating pixels in OpenCV with grayscale images section to fully understand this concept. First, read the image to work with using the cv2.imread() function. The image should be in the working directory, or a full path to the image should be provided. In this case, we are going to read the logo.png image and store it in the img variable: # The function cv2.imread() is used to read an image from the the working directory # Alternatively, you should provide a full path of the image: # Load OpenCV logo image (in this case from the working directoy): img = cv2.imread('logo.png') After the image has been loaded in img, we will gain access to some properties of the image. The first property we are going to extract from the loaded image is shape, which will tell us the number of rows, columns, and channels (if the image is in color). We will store this information in the dimensions variable for future use: # To get the dimensions of the image use img.shape # img.shape returns a tuple of number of rows, columns and channels (if a colour image) # If image is grayscale, img.shape returns a tuple of number of rows and columns. # So,it can be used to check if loaded image is grayscale or color image. # Get the shape of the image: dimensions = img.shape Another property is the size of the image (img.size is equal to the multiplication of height × width × channels): # Total number of elements is obtained by img.size: total_number_of_elements= img.size The property image datatype is obtained by img.dtype. In this case, the image datatype is uint8 (unsigned char), because values are in the [0 - 255] range: # Get the image datatype: image_dtype = img.dtype To display an image, we will use the cv2.imshow() function to show an image in a window. The window automatically fits to the image size. The first argument to this function is the window name and the second one is the image to be displayed. In this case, since the loaded image has been stored in the img variable, we will use this variable as the second argument: # The function cv2.imshow() is used to display an image in a window # The first argument of this function is the window name # The second argument of this function is the image to be shown. # Each created window should have different window names. # Show original image: cv2.imshow("original image", img) After the image is displayed, the cv2.waitKey() function, which is a keyboard binding function, will wait for a specified number of milliseconds for any keyboard event. The argument is the time in milliseconds. If any key is pressed at that time, the program will continue. If the number of milliseconds is 0 (cv2.waitKey(0)), it will wait indefinitely for a keystroke. Therefore, this function will allow us to see the displayed window waiting for a keystroke: # The function cv2.waitKey(), which is a keyboard binding function, waits for any keyboard event. # This function waits the value indicated by the argument (in milliseconds). # If any keyboard event is produced in this period of time, the program continues its execution # If the value of the argument is 0, the program waits indefinitely until a keyboard event is produced: cv2.waitKey(0) […]