Face Detection

Using the image and cascades

Computer vision is an exciting and growing field. There are tons of interesting problems to solve! One of them is face detection: the ability of a computer to recognize that a photograph contains a human face, and tell you where it is located.

OpenCV cascade breaks the problem of detecting faces into multiple stages. For each block, it does a very rough and quick test. If that passes, it does a slightly more detailed test, and so on. The algorithm may have 30 to 50 of these stages or cascades, and it will only detect a face if all stages pass.

Now we create the cascade and initialize it with our face cascade. This loads the face cascade into memory so it’s ready for use. Remember, the cascade is just an XML file that contains the data to detect faces.

import cv2

imagePath = "faces.png"
cascPath = "haarcascade_frontalface_default.xml"

# Create the haar cascade
faceCascade = cv2.CascadeClassifier(cascPath)

# Read the image
image = cv2.imread(imagePath)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

# Detect faces in the image

faces = faceCascade.detectMultiScale(gray, scaleFactor = 1.1, minNeighbors = 5, minSize = (30,30), flags = 1)

print("Found %d faces " , len(faces))
# Draw a rectangle around the faces

for (x, y, w ,h) in faces:
    cv2.rectangle(image, (x,y), (x+w, y+h), (0,255,0) , 2)

cv2.imshow("Faces found" ,image)
cv2.waitKey(0)

Using the webcam

import cv2
import sys

cascPath = sys.argv[1]

faceCascade = cv2.CascadeClassifier(cascPath)
video_capture = cv2.VideoCapture(0)

while True:
# Capture frame-by-frame
ret, frame = video_capture.read()
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = faceCascade.detectMultiScale(
gray,
scaleFactor=1.1,
minNeighbors=5,
minSize=(30, 30),
flags=cv2.cv.CV_HAAR_SCALE_IMAGE

)

# Draw a rectangle around the faces

for (x, y, w, h) in faces:
    cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)

# Display the resulting frame
cv2.imshow('Video', frame)

if cv2.waitKey(1) & 0xFF == ord('q'):
    break

# When everything is done, release the capture

video_capture.release()
cv2.destroyAllWindows()

The histogram of oriented gradients (HOG) is a feature descriptor used in computer vision and image processing for the purpose of object detection. The technique counts occurrences of gradient orientation in localized portions of an image.In this section, we will take a look at one such feature extraction technique, the Histogram of Oriented Gradients (HOG), which transforms image pixels into a vector representation that is sensitive to broadly informative image features regardless of confounding factors like illumination. We will use these features to develop a simple face detection pipeline, using machine learning algorithms and concepts.

HOG Features

The Histogram of Gradients is a straightforward feature extraction procedure that was developed in the context of identifying pedestrians within images. HOG involves the following steps:

• Optionally pre-normalize images. This leads to features that resist dependence on variations in illumination.
• Convolve the image with two filters that are sensitive to horizontal and vertical brightness gradients. These capture edge, contour, and texture information.
• Subdivide the image into cells of a predetermined size, and compute a histogram of the gradient orientations within each cell.
• Normalize the histograms in each cell by comparing to the block of neighboring cells. This further suppresses the effect of illumination across the image.
• Construct a one-dimensional feature vector from the information in each cell.

A fast HOG extractor is built into the Scikit-Image project, and we can try it out relatively quickly and visualize the oriented gradients within each cell:

A Simple Face Detector

Using these HOG features, we can build up a simple facial detection algorithm with any Scikit-Learn estimator; here we will use a linear support vector machine.

The steps are as follows:

• Obtain a set of image thumbnails of faces to constitute "positive" training samples.
• Obtain a set of image thumbnails of non-faces to constitute "negative" training samples.
• Extract HOG features from these training samples.
• Train a linear SVM classifier on these samples.
• For an "unknown" image, pass a sliding window across the image, using the model to evaluate whether that window contains a face or not.
• If detections overlap, combine them into a single window.

Let's go through these steps and try it out:
1. Obtain a set of positive training samples

Let's start by finding some positive training samples that show a variety of faces. We have one easy set of data to work with—the Labeled Faces in the Wild dataset, which can be downloaded by Scikit-Learn:

from sklearn.datasets import fetch_lfw_people 

faces = fetch_lfw_people() 

positive_patches = faces.images 

positive_patches.shape

Output
(13233, 62, 47)

This gives us a sample of 13,000 face images to use for training.

2. Obtain a set of negative training samples

Next we need a set of similarly sized thumbnails which do not have a face in them. One way to do this is to take any corpus of input images, and extract thumbnails from them at a variety of scales. Here we can use some of the images shipped with Scikit-Image, along with Scikit-Learn's PatchExtractor:

from skimage import data, transform

imgs_to_use = ['camera', 'text', 'coins', 'moon', 'page', 'clock', 'immunohistochemistry', 'chelsea', 'coffee', 'hubble_deep_field'] 

images = [color.rgb2gray(getattr(data, name)()) for name in imgs_to_use]


from sklearn.feature_extraction.image import PatchExtractor 

def extract_patches(img, N, scale=1.0, patch_size=positive_patches[0].shape): 

 extracted_patch_size = tuple((scale * np.array(patch_size)).astype(int))

 extractor = PatchExtractor(patch_size=extracted_patch_size, max_patches=N, random_state=0)   patches = extractor.transform(img[np.newaxis]) 

 if scale != 1: 

     patches = np.array([transform.resize(patch, patch_size) 

     for patch in patches]) 

 return patches 

 negative_patches = np.vstack([extract_patches(im, 1000, scale)      

for im in images for scale in [0.5, 1.0, 2.0]]) 

negative_patches.shape

Output
(30000, 62, 47)

We now have 30,000 suitable image patches which do not contain faces

3.Combine sets and extract HOG features

Now that we have these positive samples and negative samples, we can combine them and compute HOG features. This step takes a little while, because the HOG features involve a nontrivial computation for each image:

from itertools import chain 

X_train = np.array([feature.hog(im) 

for im in chain(positive_patches, negative_patches)]) 

y_train = np.zeros(X_train.shape[0]) 

y_train[:positive_patches.shape[0]] = 1

X_train.shape

Output
(43233, 1215)

We are left with 43,000 training samples in 1,215 dimensions, and we now have our data in a form that we can feed into Scikit-Learn!

4. Training a support vector machine

Next we use the tools  to create a classifier of thumbnail patches. For such a high-dimensional binary classification task, a Linear support vector machine is a good choice. We will use Scikit-Learn's LinearSVC, because in comparison to SVC it often has better scaling for large number of samples.

First, though, let's use a simple Gaussian naive Bayes to get a quick baseline:

from sklearn.naive_bayes import GaussianNB 

from sklearn.cross_validation import cross_val_score 

cross_val_score(GaussianNB(), X_train, y_train)

Output:
array([ 0.9408785 , 0.8752342 , 0.93976823])


We see that on our training data, even a simple naive Bayes algorithm gets us upwards of 90% accuracy. Let's try the support vector machine, with a grid search over a few choices of the C parameter:

from sklearn.svm import LinearSVC

from sklearn.grid_search import GridSearchCV 

grid = GridSearchCV(LinearSVC(), {'C': [1.0, 2.0, 4.0, 8.0]}) 

grid.fit(X_train, y_train) 

grid.best_score_
grid.best_params_
Output
0.98667684407744083
{'C': 4.0}


Let's take the best estimator and re-train it on the full dataset:

model = grid.best_estimator_ model.fit(X_train, y_train)

output:
LinearSVC(C=4.0, class_weight=None, dual=True, fit_intercept=True, intercept_scaling=1, loss='squared_hinge', max_iter=1000, multi_class='ovr', penalty='l2', random_state=None, tol=0.0001, verbose=0)

5. Find faces in a new image

Now that we have this model in place, let's grab a new image and see how the model does. We will use one portion of the astronaut image for simplicity (see discussion of this in Caveats and Improvements), and run a sliding window over it and evaluate each patch:

test_image = skimage.data.astronaut() 

test_image = skimage.color.rgb2gray(test_image) 

test_image = skimage.transform.rescale(test_image, 0.5)

 test_image = test_image[:160, 40:180] 

 plt.imshow(test_image, cmap='gray') 

plt.axis('off');



Next, let's create a window that iterates over patches of this image, and compute HOG features for each patch:

def sliding_window(img, patch_size=positive_patches[0].shape, istep=2, jstep=2, scale=1.0): 

 Ni, Nj = (int(scale * s) for s in patch_size) 

 for i in range(0, img.shape[0] - Ni, istep): 

 for j in range(0, img.shape[1] - Ni, jstep): 

 patch = img[i:i + Ni, j:j + Nj]

 if scale != 1: 

 patch = transform.resize(patch, patch_size) 

 yield (i, j), patch 

indices, patches = zip(*sliding_window(test_image)) 

patches_hog = np.array([feature.hog(patch) for patch in patches]) 

patches_hog.shape

Output:
(1911, 1215)

Finally, we can take these HOG-featured patches and use our model to evaluate whether each patch contains a face:

labels = model.predict(patches_hog) labels.sum()

Output
33.0

We see that out of nearly 2,000 patches, we have found 30 detections. Let's use the information we have about these patches to show where they lie on our test image, drawing them as rectangles:

fig, ax = plt.subplots() 

ax.imshow(test_image, cmap='gray')

ax.axis('off') 

Ni, Nj = positive_patches[0].shape 

indices = np.array(indices) 

for i, j in indices[labels == 1]: 

     ax.add_patch(plt.Rectangle((j, i), Nj, Ni, edgecolor='red', alpha=0.3, lw=2, facecolor='none'))




All of the detected patches overlap and found the face in the image.