Ask Your Question

Object recognition and replacement in opencv

asked 2018-11-22 17:38:45 -0500

updated 2018-11-24 01:12:47 -0500

berak gravatar image

Hello everyone,

I wanted to recognize an object in an image and then replace it with another image.

Could you help me understanding the replacement part? I have the following code from OpenCV docs that detects and marks the area to be replaced with another image:

import numpy as np
import cv2
from matplotlib import pyplot as plt

img1 = cv2.imread('pil.jpg',0) 
img2 = cv2.imread('pil_search.jpg',0)

orb = cv2.ORB_create()

kp1, des1 = orb.detectAndCompute(img1,None)
kp2, des2 = orb.detectAndCompute(img2,None)

index_params= dict(algorithm = FLANN_INDEX_LSH,
               table_number = 6, 
               key_size = 12,     
               multi_probe_level = 1) 
search_params = dict(checks = 50)

flann = cv2.FlannBasedMatcher(index_params, search_params)

matches = flann.knnMatch(des1,des2,k=2)

good = []
for m,n in matches:
if m.distance < 0.75*n.distance:

if len(good)>MIN_MATCH_COUNT:
    src_pts = np.float32([ kp1[m.queryIdx].pt for m in good ]).reshape(-1,1,2)
    dst_pts = np.float32([ kp2[m.trainIdx].pt for m in good ]).reshape(-1,1,2)

    M, mask = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC,5.0)
    matchesMask = mask.ravel().tolist()

    h,w = img1.shape
    pts = np.float32([ [0,0],[0,h-1],[w-1,h-1],[w-1,0] ]).reshape(-1,1,2)
    dst = cv2.perspectiveTransform(pts,M)

    img2 = cv2.polylines(img2,[np.int32(dst)],True,255,3, cv2.LINE_AA)
    cv2.imshow('Match', img2)

    print("Not enough matches are found - %d/%d" % (len(good), MIN_MATCH_COUNT))
    matchesMask = None

#draw our inliers (if successfully found the object) or matching keypoints (if failed).

draw_params = dict(matchColor = (0,255,0), # draw matches in green color
                   singlePointColor = None,
                   matchesMask = matchesMask, # draw only inliers
                   flags = 2)

img3 = cv2.drawMatches(img1,kp1,img2,kp2,good,None,**draw_params)

plt.imshow(img3, 'gray'),

image description

edit retag flag offensive close merge delete


note, that this is NOT "object recognition" at all. it has no idea about "objects".

all it does is find a homography between 2 known parts of a scene.

berak gravatar imageberak ( 2018-11-24 01:28:55 -0500 )edit

1 answer

Sort by ยป oldest newest most voted

answered 2018-11-24 02:28:09 -0500

berak gravatar image

updated 2018-11-24 06:10:03 -0500

I wanted to recognize an object in an image and then replace it with another image.

i dou't think, your approach will work here (there is no "object" detection, and the matching here won't allow you to tell, if there is no such thing in the image).

however, you can still warp another image (with the same size/channels !) into your scene using:

M, mask = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC,5.0)
scene = cv.warpPerspective(small_img, M, (scene.shape[1], scene.shape[0]), scene)
edit flag offensive delete link more

Question Tools

1 follower


Asked: 2018-11-22 17:38:45 -0500

Seen: 177 times

Last updated: Nov 24 '18