[Python] Real time image stabilization with Optical Flow

edit

Hi! I'm new here on this forum, and would love some help with a project I'm working on!

I'm trying to make a small image stabilization programme in Python, but I can't get it to work the way I want.

First, my test programme:

from stabilizer import Stabilizer
import cv2
import sys
from imutils.video import VideoStream
import time



imageCapture = cv2.VideoCapture(0)
imageCapture.open(0)
time.sleep(2.0)
frame=0
counter=0

stabilizer=Stabilizer()

while True:
    image=imageCapture.read()
    frame, result=stabilizer.stabilize(image, frame)

    cv2.imshow("Result", result)
    cv2.imshow("Image", image[1])
    key = cv2.waitKey(1) & 0xFF

    # if the `q` key was pressed, break from the loop
    if key == ord("q"):
        break
    counter+=1
    print counter

print("[INFO] cleaning up...")
cv2.destroyAllWindows()
imageCapture.release()

...and this is my actual stabilization programme:

import numpy as np
import imutils
import cv2



class Stabilizer:
    def stabilize(self,image, old_frame):


            # params for ShiTomasi corner detection
            feature_params = dict( maxCorners = 100,qualityLevel = 0.3,minDistance = 7,blockSize = 7 )

            # Parameters for lucas kanade optical flow
            lk_params = dict( winSize  = (15,15),
                              maxLevel = 2,
                              criteria = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))

            # Create some random colors
            color = np.random.randint(0,255,(100,3))

            # Take first frame and find corners in it
            try:
                if old_frame==0:
                    ret, old_frame = image
            except:
                print("tull")

            old_gray = cv2.cvtColor(old_frame, cv2.COLOR_BGR2GRAY)
            p0 = cv2.goodFeaturesToTrack(old_gray, mask = None, **feature_params)


            ret,frame = image
            frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

            # calculate optical flow
            p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)


            # Select good points
            good_new = p1[st==1]
            good_old = p0[st==1]


            # Make 3x3 matrix
            h=cv2.findHomography(good_old,good_new)
            #h=cv2.getPerspectiveTransform(good_old,good_new) #not working



            # Now update the previous frame and previous points
            #old_gray = frame_gray.copy()
            #p0 = good_new.reshape(-1,1,2)

            #cv2.destroyAllWindows()

            result=cv2.warpPerspective(frame,h[0], (frame.shape[1],frame.shape[0]))

            return frame, result

This is what I thought making this:

1. Catch one frame, finding points (p0) to match. The first time the old and new frame will be the same, but the next run it should be two different frames.
2. Calculate "Optical Flow" from these points.
3. Make 3x3 transformation matrix from this "Optical Flow"
4. Apply the transformation to the image

Is there any one who could help me with this one? Thanks!