Ask Your Question
0

How to calculate Optical Flow magnitude?

asked 2019-08-15 12:21:09 -0500

Tina  J gravatar image

updated 2019-08-15 12:34:32 -0500

I'm trying to see how big different two given video frames are. My goal is to calculate a single value showing how fast objects inside those frames are moving.

I can calculate Optical Flow matrix below, both the HSV and magnitude matrices. But I don't know how to calculate a average total movement magnitude. How can I calculate it from those matrices?

def optical_flow(one, two):
    one_g = cv2.cvtColor(one, cv2.COLOR_RGB2GRAY)
    two_g = cv2.cvtColor(two, cv2.COLOR_RGB2GRAY)
    hsv = np.zeros((120, 320, 3))
    # set saturation
    hsv[:,:,1] = cv2.cvtColor(two, cv2.COLOR_RGB2HSV)[:,:,1]
    # obtain dense optical flow paramters
    flow = cv2.calcOpticalFlowFarneback(one_g, two_g, flow=None,
                                        pyr_scale=0.5, levels=1, winsize=15,
                                        iterations=2,
                                        poly_n=5, poly_sigma=1.1, flags=0)
    # convert from cartesian to polar
    mag, ang = cv2.cartToPolar(flow[..., 0], flow[..., 1])
    # hue corresponds to direction
    hsv[:,:,0] = ang * (180/ np.pi / 2)
    # value corresponds to magnitude
    hsv[:,:,2] = cv2.normalize(mag,None,0,255,cv2.NORM_MINMAX)
    # convert HSV to int32's
    hsv = np.asarray(hsv, dtype= np.float32)
    rgb_flow = cv2.cvtColor(hsv,cv2.COLOR_HSV2RGB)
    return rgb_flow

The rgb_flow is a 3D array looks like this:

[[[0 0 0]
  [0 0 0]
  [0 0 0]
  ...
  [0 0 0]
  [0 0 0]
  [0 0 0]]

 [[0 0 0]
  [0 0 0]
  [0 0 0]
  ...
  [0 0 0]
  [0 0 0]
  [0 0 0]]


 ...


 [[0 0 0]
  [0 0 0]
  [0 0 0]
  ...
  [0 0 0]
  [0 0 0]
  [0 0 0]]]

And the mag matrix is 2D array like this:

[[3.2825139e-03 3.9561605e-03 4.8938910e-03 ... 3.7310597e-02
  3.2986153e-02 2.5520157e-02]
 [4.9569397e-03 6.3276174e-03 7.7017904e-03 ... 3.9564677e-02
  3.2582227e-02 2.6329078e-02]
 ...

 [6.9548332e-06 8.3683852e-05 6.0906638e-03 ... 8.3484064e-04
  6.4721738e-04 2.9505073e-04]]
edit retag flag offensive close merge delete

Comments

1

avg = np.mean(mag) ??

berak gravatar imageberak ( 2019-08-15 15:28:31 -0500 )edit

Yes, I did that. And then I average this over the whole shot frames. But the results are mehhh...Sometimes a more fast-moving scenes have less values than a more static ones.

Tina  J gravatar imageTina J ( 2019-08-15 15:50:57 -0500 )edit

if you think of it, it's the mean motion of a whole image, including all non-moving pixels (probably the majority)

so, the mean of the magnitude of the flow does not deliver, what you want, but what do you want ?

how fast objects inside those frames are moving.

can you explain ? what do you need it for ? do you need movement directions ? why the optflow ?

I'm trying to see how big different two given video frames are.

maybe much simpler measures like psnr would do better ?

berak gravatar imageberak ( 2019-08-15 16:02:25 -0500 )edit
1

No, my final aim is differentiate fast scenes with slow scenes. Fast camera or fast objects inside, both are OK. Like, a car racing vs a person just standing and talking.

Tina  J gravatar imageTina J ( 2019-08-15 16:11:17 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
0

answered 2020-11-19 20:23:12 -0500

crackwitz gravatar image

magnitudes are always positive. the noise in a static scene can produce noticeable flow vectors... albeit in random directions... which don't affect magnitudes.

you should sum up all the flow vectors instead. that will give you average motion and direction. the average will represent camera motion or full scene motion. it will only imperfectly track one object moving across the view.

if you want to ignore static parts of the image and only react to moving parts, make a mask for the static parts (magnitudes less than some threshold) and, for the summing/averaging, select only flow vectors that aren't in that mask.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2019-08-15 12:21:09 -0500

Seen: 485 times

Last updated: Nov 19