How to apply optical flow to initial image?

asked 2018-03-09 18:20:07 -0500

I found old posts online as well as a code sample in the master branch of opencv here

I will copy/paste the code for future reference:

 def warp_flow(img, flow):
    h, w = flow.shape[:2]
    flow = -flow
    flow[:,:,0] += np.arange(w)
    flow[:,:,1] += np.arange(h)[:,np.newaxis]
    res = cv.remap(img, flow, None, cv.INTER_LINEAR)
    return res

Say I have two grayscale images im1 and im2 with values between 0 and 1. I computed the flow with:

flow = cv2.calcOpticalFlowFarneback(im1g, im2g, None, 0.5, 3, 15, 3, 5, 1.2, 0)

I understood that the function warp_flow expects the original images with values in [0,255] and the computed flow computed on grayscale as input, so I called it with:

prediction = warp_flow(origim1, flow)

However, when I write the truth origim2 and the prediction to disk with cv2.imwrite, it turns out that the prediction is pretty much like origim1. So the flow is not modifying the input image.

I appreciate if you can explain what is happening or if you can provide a working example that I could try with my images. Is the warp_flow function up-to-date?

Thank you in advance,

edit retag flag offensive close merge delete


  • what are you trying to achieve here ?
  • "with values between 0 and 1" -- it expects input in the [0..255] range
  • " Is the warp_flow function up-to-date?" -- it is part of the "glitch" visualization, and might not be relevant at all for your purpose, which is ? (see 1. again)
berak gravatar imageberak ( 2018-03-09 21:02:48 -0500 )edit

Hi @berak, let me formulate the problem then. Given two images im1 and im2, I want to compute the velocity field vx and vy that transforms im1 to im2. Given this velocity field and im1, I would like to reconstruct im2. In a practical scenario, I will be applying the transformation multiple times with synthetic velocities to produce synthetic images, and finally a video.

juliohm gravatar imagejuliohm ( 2018-03-09 21:17:46 -0500 )edit