Face Recognition on the iPhone

How can I do facial recognition on the iPhone. Could someone provide me with references/articles to point me in the right direction please? I have done research and realised that I need to do face detection first to extract the image and then do facial recognition by comparing it with other images within a database.

I have realised that I have do the face detection by using OpenCV or by utilising iOS 5.0 and upwards to detect the face. I am unsure on the facial recognition (I plan on storing images on a remote database and then doing the comparison against the remote database).

Solution 1:

Face detection

I would use the Haarcascades available in open CV to perform quick and accurate face detection.

http://opencv.willowgarage.com/wiki/FaceDetection

Face recognition

I would use a method such as Principal Component Analysis (PCA) a.k.a eigenfaces.

http://www.cognotics.com/opencv/servo_2007_series/part_5/index.html

That link shows a tutorial on how to get that working with OpenCV - I think this is written for C but i'm sure you can get the basic jist of it.

You could also look at implementing it yourself if you feel brave (it's not too bad)...

http://www.face-rec.org/algorithms/PCA/jcn.pdf

http://blog.zabarauskas.com/eigenfaces-tutorial/

Database

I actually did something similar to you albeit on a PC not an iPhone but its still the same concept. I stored all my images in the database as Blob data types then loaded them into my program when necessary.

Edit

The database is a particularly tricky part of the system as this is where the biggest bottleneck is. In my application, I would go through the following steps...

1. Open application and grab training images from database
2. Generate training set based on these images
3. Once 1 and 2 have been completed the system is very quick as it just performs recognition against the training set.

Fortunately for me, my database server was located on a LAN therefore speed wasn't a problem, however I can see why you have an issue due to the fact that on a mobile device you have a limited data connection (speed/bandwidth). You can compress the images however this may lead to a worse recognition rate, due to image quality reduction and also you will have to decode on the device. There is also the issue of how to expose the remote database to the application, however I do believe this is possible using PHP and JSON (and other technologies, see below).

Retrieving data from a remote database

Maybe you could do an initial synchronize with the database so that the images are cached on the phone? One way or another I think you are probably going to have to have the images on the phone at some point regardless.

Figuring out the best way to store the recognition data/images in the database was one of the biggest challenges I faced so I would be interested to hear if you find a good method.

Solution 2:

As you pointed out, the first step (detection of the face) is easy with iOS 5 and CoreImage.framework. Quick example:

CIImage *image = [CIImage imageWithCGImage:image_ref];
NSDictionary *options = [NSDictionary dictionaryWithObject:CIDetectorAccuracyHigh forKey:CIDetectorAccuracy];
CIDetector *detector = [CIDetector detectorOfType:CIDetectorTypeFace context:nil options:options];
NSArray *features = [detector featuresInImage:image];

for (CIFaceFeature *feature in features)
{
  CGRect face_bounds = [feature bounds];
  CGPoint mouth_position = [feature mouthPosition];
  // do something with these values
}

With regards to the second part of your question (i.e. facial recognition), I shall leave that to someone more qualified than myself to answer. :)

Solution 3:

You probably want to look at the midian project by Pedro Centieiro, which performs Face Recognition on iOS 5 with OpenCV. It's on github at:

• https://github.com/pcentieiro/midian

It uses parts of my libfacerec, so it supports Eigenfaces, Fisherfaces and Local Binary Patterns Histograms for face recognition.

Solution 4:

Face recognition can be implemented as a machine learning algorithm. This book has a chapter that describe this task and how to implement it. Worth the reading!

It uses Single value decomposition (SVD), more specifically the Tensor SVD method.