Computer Vision - filtering convex hulls and convexity defects with OpenCV

Solution 1:

The convex hull at low res can be used to identify the position of the hand as a whole, it is not useful for fingers but does provide a region of interest and appropriate scale.

The higher resolution analysis should then be applied to your approximated contour, it is easy to skip any points that do not pass the "length and angle" criteria from the last two, though you may wish to "average in" instead of "skip entirely".

Your code example is a single pass of calculating convexity defects and then removing them .. that is a logic error .. you need to remove points as you go .. (a) it is faster and simpler to do everything in one-pass (b) it avoids removing points at a first pass and having to add them back later because any removal changes previous calcs.

This basic technique is very simple and so works for a basic open palm. It doesn't intrinsically understand a hand or a gesture though, so tuning the scale, angle and length parameters is only ever going to get you "so far".

References to Techniques: filter length and angle "Convexity defect" Simen Andresen blog http://simena86.github.io/blog/2013/08/12/hand-tracking-and-recognition-with-opencv/

Kinect SDK based C# Library with added finger direction detection http://candescentnui.codeplex.com/ http://blog.candescent.ch/2011/11/improving-finger-detection.html

"Self-growing and organized neural gas" (SGONG) Prof Nikos Papamarkos http://www.papamarkos.gr/uploaded-files/Hand%20gesture%20recognition%20using%20a%20neural%20network%20shape%20fitting%20technique.pdf

Commercial product David Holz & Michael Buckwald founders of "Leap Motion" http://www.engadget.com/2013/03/11/leap-motion-michael-buckwald-interview/