Does Photos.app do facial recognition locally or are photos sent to Apple for processing?

Photos.app automatically recognizes faces in photos and groups them by person. Are photos sent to Apple for processing or is everything done locally on the machine?

Solution 1:

Facial recognition and other image analysis is done on device. Never done on Apple's servers. Also the results of facial recognition and other images analysis are never send to Apple.

Apple’s system is more focused on privacy than Facebook's or Google’s, with stricter limits on how the data is collected and stored. Onstage, Craig Federighi insisted that Apple’s new system only uses local data, which means the company isn’t storing faceprints on company servers — a crucial point for anyone worried about their faceprint being used for more than just cataloging photos. Faceprints are still being created and used, but it's all happening on your phone, where Apple and the rest of the world can't access it. That’s different from Google and Facebook, which store data on the cloud and get significant scale benefits as a result.

Starting with iOS 11 and macOS High Sierra 10.13 however, facial recognition data will be synced across iCloud Photo Library.

Interview John Gruber with Craig Federighi on how this works:

No, we've been working on it. And it's actually something that's harder to get right right, because when you go through the process of classifying your photos, we offer up and we say "Hey, is this John?" And y'know, "Is this John?" And you say "Yes, yes."

But when you say yes to one face, we're actually saying "Okay, well, there are a thousand other photos we saw that we think are the same person as this, so we're going to count those all as John — but all you really told us is about that one photo.

And so what we actually sync, and support, and so that we get this right — we sync only the specific photos you told us about, and then each… 'cos that's really the truth, y'know, that's what the user really told us.

And then we sync that information, and then each device, because they're independently doing photo recognition are talking that, and re-deriving the — coming to the final result on all your machines to synchronize. So synchronize is actually very little data, and it's all just the truth data, and we won't hold back the right solution.

But getting that architecture completely right was a — took a little longer than we would have liked, and so we're super excited to have it done now.