How to calculate Optical Flow magnitude?

asked 2019-08-15 12:21:09 -0500

Tina  J gravatar image

updated 2019-08-15 12:34:32 -0500

I'm trying to see how big different two given video frames are. My goal is to calculate a single value showing how fast objects inside those frames are moving.

I can calculate Optical Flow matrix below, both the HSV and magnitude matrices. But I don't know how to calculate a average total movement magnitude. How can I calculate it from those matrices?

def optical_flow(one, two):
    one_g = cv2.cvtColor(one, cv2.COLOR_RGB2GRAY)
    two_g = cv2.cvtColor(two, cv2.COLOR_RGB2GRAY)
    hsv = np.zeros((120, 320, 3))
    # set saturation
    hsv[:,:,1] = cv2.cvtColor(two, cv2.COLOR_RGB2HSV)[:,:,1]
    # obtain dense optical flow paramters
    flow = cv2.calcOpticalFlowFarneback(one_g, two_g, flow=None,
                                        pyr_scale=0.5, levels=1, winsize=15,
                                        iterations=2,
                                        poly_n=5, poly_sigma=1.1, flags=0)
    # convert from cartesian to polar
    mag, ang = cv2.cartToPolar(flow[..., 0], flow[..., 1])
    # hue corresponds to direction
    hsv[:,:,0] = ang * (180/ np.pi / 2)
    # value corresponds to magnitude
    hsv[:,:,2] = cv2.normalize(mag,None,0,255,cv2.NORM_MINMAX)
    # convert HSV to int32's
    hsv = np.asarray(hsv, dtype= np.float32)
    rgb_flow = cv2.cvtColor(hsv,cv2.COLOR_HSV2RGB)
    return rgb_flow

The rgb_flow is a 3D array looks like this:

[[[0 0 0]
  [0 0 0]
  [0 0 0]
  ...
  [0 0 0]
  [0 0 0]
  [0 0 0]]

 [[0 0 0]
  [0 0 0]
  [0 0 0]
  ...
  [0 0 0]
  [0 0 0]
  [0 0 0]]


 ...


 [[0 0 0]
  [0 0 0]
  [0 0 0]
  ...
  [0 0 0]
  [0 0 0]
  [0 0 0]]]

And the mag matrix is 2D array like this:

[[3.2825139e-03 3.9561605e-03 4.8938910e-03 ... 3.7310597e-02
  3.2986153e-02 2.5520157e-02]
 [4.9569397e-03 6.3276174e-03 7.7017904e-03 ... 3.9564677e-02
  3.2582227e-02 2.6329078e-02]
 ...

 [6.9548332e-06 8.3683852e-05 6.0906638e-03 ... 8.3484064e-04
  6.4721738e-04 2.9505073e-04]]
edit retag flag offensive close merge delete

Comments

1

avg = np.mean(mag) ??

berak gravatar imageberak ( 2019-08-15 15:28:31 -0500 )edit

Yes, I did that. And then I average this over the whole shot frames. But the results are mehhh...Sometimes a more fast-moving scenes have less values than a more static ones.

Tina  J gravatar imageTina J ( 2019-08-15 15:50:57 -0500 )edit

if you think of it, it's the mean motion of a whole image, including all non-moving pixels (probably the majority)

so, the mean of the magnitude of the flow does not deliver, what you want, but what do you want ?

how fast objects inside those frames are moving.

can you explain ? what do you need it for ? do you need movement directions ? why the optflow ?

I'm trying to see how big different two given video frames are.

maybe much simpler measures like psnr would do better ?

berak gravatar imageberak ( 2019-08-15 16:02:25 -0500 )edit
1

No, my final aim is differentiate fast scenes with slow scenes. Fast camera or fast objects inside, both are OK. Like, a car racing vs a person just standing and talking.

Tina  J gravatar imageTina J ( 2019-08-15 16:11:17 -0500 )edit