OpenCV tracking using optical flow
As you write, cv::goodFeaturesToTrack
takes an image as input and produces a vector of points which it deems "good to track". These are chosen based on their ability to stand out from their surroundings, and are based on Harris corners in the image. A tracker would normally be initialised by passing the first image to goodFeaturesToTrack and obtaining a set of features to track. These features could then be passed to cv::calcOpticalFlowPyrLK
as the previous points, along with the next image in the sequence and it will produce the next points as output, which then become input points in the next iteration.
If you want to try to track a different set of pixels (rather than features generated by cv::goodFeaturesToTrack
or a similar function), then simply provide these to cv::calcOpticalFlowPyrLK
along with the next image.
Very simply, in code:
// Obtain first image and set up two feature vectors
cv::Mat image_prev, image_next;
std::vector<cv::Point> features_prev, features_next;
image_next = getImage();
// Obtain initial set of features
cv::goodFeaturesToTrack(image_next, // the image
features_next, // the output detected features
max_count, // the maximum number of features
qlevel, // quality level
minDist // min distance between two features
);
// Tracker is initialised and initial features are stored in features_next
// Now iterate through rest of images
for(;;)
{
image_prev = image_next.clone();
feature_prev = features_next;
image_next = getImage(); // Get next image
// Find position of feature in new image
cv::calcOpticalFlowPyrLK(
image_prev, image_next, // 2 consecutive images
points_prev, // input point positions in first im
points_next, // output point positions in the 2nd
status, // tracking success
err // tracking error
);
if ( stopTracking() ) break;
}
cv::calcOpticalFlowPyrLK(..) function uses arguments :
cv::calcOpticalFlowPyrLK(prev_gray, curr_gray, features_prev, features_next, status, err);
cv::Mat prev_gray, curr_gray;
std::vector<cv::Point2f> features_prev, features_next;
std::vector<uchar> status;
std::vector<float> err;
simplest(partial) code to find pixel in next frame :
features_prev.push_back(cv::Point(4, 5));
cv::calcOpticalFlowPyrLK(prev_gray, curr_gray, features_prev, features_next, status, err);
If pixel was successfully found status[0] == 1
and features_next[0]
will show coordinates of pixel in next frame. Value information can be found in this example: OpenCV/samples/cpp/lkdemo.cpp