Integration

As promised in the last blog post, in this short post I will be discussing how we managed to integrate the Computer Vision system with the rest of the components.

In order to make the detector easily usable in other classes and scripts, I moved the code into a MotionDetector class, with easily accessible methods and instance variables.

 Structuring the code this way proved easy for us to use it in the main application logic. In there, we instantiate the object, starting the camera & detection or stopping it whenever we need to.

Work in progress: Vision system III

The final approach I took in order to detect motion was to do Background substraction and tune the algorithm present on the openCV website, in order to filter out noise but also to make more resilient.

I convert the frame into grayscale, then apply the Background Substraction algorithm. The result of this should be only black & white values, but due to heavy light conditions ( where present ), we also encounter median values. Therefore, I also apply a binary threshold algorithm to the result, in order to filter out these values. I then use the openCV implementation findCountours in order to find all the countours in the image. I calculate the area of all of them, and filter the low-area ones. Finally, for viewing purposes, I draw the result.

In the next blog post, I will be talking about how we will be handling communication between the camera and the other components of our device.

Work in progress: Vision system II

The first attempt was to use Motion Detection Algorithms. Luckily, OpenCV has available implementations which you can find here. Detection is achieved using either the Lucas-Kanade algorithm or the Farnerback algorithm.

On the left, I have attached the result of the Farnerback algorithm, which sees better results overall than LK, but has the disadvantage of being very slow, since it relies on heavy computation. One solution for speedup was to use GPU programming.

Before actually doing this, I decided to have a look at other methods.


I started focusing only on skin detection, and not motion. One of the easiest ways of detecting the skin was to define a color range and then simply look for that color range in the frames.

As you can see, this approach is heavily dependent on lighting conditions and also background. For example, if wood material is present in the scene, then that also gets detected as skin, and represents a false positive.

Work in progress: Vision system I

This post is going to be slightly larger than my previous ones, since I haven't posted anything for a while now. Mainly I will cover Computer Vision related aspects, since that is something I have mainly worked on.

The idea we had for the project (initially), was to track smoke rings, suing the Kinect and OpenCV. The whole point of this was to project on them, without having a static location where we project. The main advantage the Kinect offers is depth recognition.

So far, so good.

Since I am using Ubuntu Linux, I managed to install the openKinect and freenect libraries on my machine, since the official SDK is available only on Microsoft Windows.

On the left you can see the depth representation from the Kinect camera. We encountered that while trying to shoot smoke rings, they would not get recognized as altering the depth of the image. Because of this problem, we decided to drop the Kinect but also to stop trying to recognize the smoke rings.

Instead, we started looking at how to recognize hand gestures solely in OpenCV.

In the next posts I will describe the techniques we used to build a reliable system.