What is Mean Shift?

Mean shift is a hill climbing algorithm which involves shifting this kernel iteratively to a higher density region until convergence. Every shift is defined by a mean shift vector. The mean shift vector always points toward the direction of the maximum increase in the density.

Consider we have a set of points. We are given a small window and we have to move that window to the area of maximum pixel density. It takes several steps given below:

Steps of Meanshift Object Tracking

During those steps, we use two main methods :

cv2.calBackProject(images, channels, hist, ranges, scale) : This method calculates the back project of the histogram. That is, at each location (x, y) the function collects the values from the selected channels in the input images and finds the corresponding histogram bin. But instead of incrementing it, the function reads the bin value, scales it by scale , and stores in backProject(x,y).
- images : input image. But the image must be in [].
- channels : The list of channels used to compute the back projection. But it must be in [].
- hist : Input histogram that can be dense or sparse.
- ranges : Array of arrays of the histogram bin boundaries in each dimension.
- scale : Optional scale factor for the output back projection.
cv2.meanShift(probImage, window, criteria) : Finds an object on a back projection image.
- probImage : Back projection of the object histogram from calBackProject.
- window : Initial search window.
- criteria : Stop criteria for the iterative search algorithm. It has three parameters as an input which is:
  - type :
    - cv2.TERM_CRITERIA_EPS = Stop iteration until reach at the epsilon.
    - cv2.TERM_CRITERIA_MAX_ITER = Stop iteration until reach at the max_iter.
    - cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER = Stop iteration until one of the two conditions is fulfilled.
  - max_iter : Max iteration number.
  - epsilon : Accuracy.

What is Cam Shift Object Tracking?

It is almost same as meanshift, but it returns a rotated rectangle and box parameters (used to be passed as search window in next iteration). It takes several steps given below:

Steps of Cam Shift

During those steps, we use two main methods :

cv2.calBackProject(images, channels, hist, ranges, scale) : This method calculates the back project of the histogram. That is, at each location (x, y) the function collects the values from the selected channels in the input images and finds the corresponding histogram bin. But instead of incrementing it, the function reads the bin value, scales it by scale , and stores in backProject(x,y).
- images : input image. But the image must be in [].
- channels : The list of channels used to compute the back projection. But it must be in [].
- hist : Input histogram that can be dense or sparse.
- ranges : Array of arrays of the histogram bin boundaries in each dimension.
- scale : Optional scale factor for the output back projection.
cv2.camShift(probImage, window, criteria) : Finds an object on a back projection image.
- probImage : Back projection of the object histogram from calBackProject.
- window : Initial search window.
- criteria : Stop criteria for the iterative search algorithm. It has three parameters as an input which is:
  - type :
    - cv2.TERM_CRITERIA_EPS = Stop iteration until reach at the epsilon.
    - cv2.TERM_CRITERIA_MAX_ITER = Stop iteration until reach at the max_iter.
    - cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER = Stop iteration until one of the two conditions is fulfilled.
  - max_iter : Max iteration number.
  - epsilon : Accuracy.