Detecting movement of the camera itself
The goal is to detect whether or not the computer camera is moving. Reaction needs to be fast. If people are moving in front of the camera, it shouldn't confuse the camera.
I tried using mobilenet SSD on the images, masking away people detected in the image, then using background subtraction to determine how much movement there is. It works okay, but it is slow. Problems ; mobilenet SSD is slow, and background subtraction isn't always that accurate (if I go stand in front of the window the change in lighting makes it look like there was a lot of movement).
I tried considering only corners but what's in the corners is usually considered background by the algorithm.
I tried doing feature matching to detect if keypoints all moved in a similar direction (in the corners of the image, for example).
More ideas I've yet to try ; optical flow (no experience with this), camera calibration with RANSAC (no experience either) and training my own machine learning model (based on 1000fps Human Segmentation with Deep Convolutional Neural Networks by C. Song).
Does anyone here have any other ideas that might increase the accuracy of the algorithm? Accuracy and speed of detection are equally as important.
why so ? can you try to explain ? what is your input ?
It's just the end goal to detect camera movement on any laptop, it's the point of the application. The input is the camera frames, with maybe 15FPS or so (maximum of 30). The initial approach (doing background subtraction on the input frames after masking out all persons detected with machine learning) worked okay, but the machine learning model is rather slow (max 6 frames per second) leading to frequent false positives.
@Peire C. You can't. How does the camera is moving? By hand, motor or pods(dolly or stick)=> that is the answer, Not camera itself.
What do you mean? It's an integrated camera, like the iSight of a Macbook. There's no motion sensor. I was wondering if you could analyse images to decide whether or not the camera is moving. It shouldn't be accurate in all cases. The initial approach worked okay to be honest, but the machine learning model makes it slow.
How do you mount on camera? I am not talking about sensor.
Not sure what you're asking ... I have a camera. It captures frames. I analyse those frames. The camera moves if the computer moves.
In my hope. I was consider about motion detection.
Any ideas how to tackle it? Dense optical flow is slow, optical flow is not robust, absdiff isn't either, background subtraction captures any sort of movement and it is hard to discriminate between camera movements and movement of objects in the environment, ... the best approach so far is single-shot-detection of people -> masking them away -> doing background subtraction on the rest. It only fails in one case ; when someone moves very close to the camera. I'm gonna try segmentation at higher FPS now, but it may just become one more wasted effort.
Unable help you. Because, I'm getting 28" monitor 4k UHD by tomorrow. I need more time.