MyRobotLab - Template Matching - Bot finds teet..

myrobotlab, open source, Java, service based framework, robotics and creative machine control

Logitech C600 Camera - PanTilt Rig with $16 BBB Arduino Clone - not super fancy but it works..

 


Template Matching is now available through MRL.

Template Matching is a process of matching a small sub image within a larger global image.  As an exercise I chose a wall socket since this could be the goal for a self charging robot. 

When the target is locked and centered, an event will fire off.  If this were a mobile platform and the goal was to mate with the socket, the next behavior would be to move closer to the socket and avoid obstacles.  Since, this is not a mobile platform, I have chosen to send the event to a Text To Speech service with the appropriate verbiage. 

The interface for creating a template can be programmed with coordinate numbers, or selected through the video feed.  To select a new template the Matching Template filter should be high-lighted, then simply select the top left and bottom right rectangle of the new template.  You will see the template image become visible in the Photo Reel section of the OpenCV gui.

Currently, I am using the Face Tracking service in MRL.  The Face Tracking service will soon be decomposed into a more generalized Tracking Service, which can be used to track a target with any sort of appropriate sensor data.  Previously I found tracking problematic.  The pan/tilt platform would bounce back and forth and overcompensate (Hysterisis).  The lag which video processing incurs makes the tracking difficult.  In an attempt to compensate this issue, I have recently combined a PID controller into the Tracking service, and have been very pleased with the results.  The tracking is bounces around much less, although there is still room for improvement.  

PID is a method (and artform) which allows error correction in complex systems.  Initially a set of values must be chosen for the specific system.  There are 3 major values.  

  • Kp = Proportional constant - dependent on present errors
  • Ki = Integral constant - is the accumulation of past errors 
  • Kd = Derivative constant - attempts to predict future errors

The video will show the initial setup.  This involves connecting an Arduino to a COM port, then connecting 2 Servos to theArduino (1 for pan & another for tilt).  After this is done, I begin selecting different templates to match as the test continues. The template match value in the upper left corner represents and represents the quality of matching.

The states which can occur

  • "I found my num num" - first time tracking
  • "I got my num num" - lock and centered on target
  • "I wan't dat" - not centered - but tracking to target
  • "boo who who - where is my num num?" - lost tracking

More to Come

In the video you can see when I switch off the lights the lock is lost.  Template matching is sensitive to scale, rotation, and light changes.  Haar object detection is more robust and less sensitive to scale and lighting changes.  The next step will be taking  the template and proceeding with Haar training. 

Kinect & Bag Of Words - http://en.wikipedia.org/wiki/Bag_of_words_model_in_computer_vision association


Update 2011.08.08

I was on vacation for a week, however, when I got back I wanted to make sure the latest (mrl 13) was cross platform compatible.
I had some problems with Fedora Core 15 / GNOME 4 desktop / Java & OpenCV
FC15 can install opencv version 2.2 but 2.3 is available for download.
I removed the 2.2 version - and did a clean install of mrl 13

The desktop is still acting a little "goofy" but after :

  • copying *.so's from the bin directory to /usr/lib (blech) 
  • ldconfig
  • loading opencv service - and refreshing the gui screen (blech)
  • using the Gray/PyramidDown/MatchTemplate filters

I got tempate matchin on the MAAHR brain going at 78 ms per frame running with debug logging. (almost 13 frames per second!)
it says 93ms because the screen-capture slows the process down. 

 

MAAHR is currently running on a 12V SLA
The CPU and power supply are cool 
None of the CPUs are over 60% and this should drop off significantly if the video feed was not being displayed.
MRL-13 has now been tested on Windows XP & 7, Fedora Core 13 & 15 -  The next version should come with opencv binaries compatible with an ARM-7, although Ro-Bot-X's Chumby appears down for some reason.....

 

 

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

excellant, I have it logging the keys I type! But is there any way to make it a string or have it recognise strings yet?

What?  
Download the latest r112M & look harder ! :) 

 All I'm seeing is this:

Did you remove myrobotlab.jar and download the latest?
Yes, that's how I got the keyboard service.

Your right...

Somethings fishy with the google "Otto" poster ...  Grrrr.... it says it updated it, and sent the file .. but something is very wonky...

Alright I did it manually - you should be able to try again - the label says "no really ! - it does have "send strings" "

Your right...

Somethings fishy with the google "Otto" poster ...  Grrrr.... it says it updated it, and sent the file .. but something is very wonky...

Alright I did it manually - you should be able to try again - the label says "no really ! - it does have "send strings" "

1. Sounds good 2. Iris system will be a prop that I am ordering. It will use xbee, IR, cables, and rf to send commands to various things. It just needs to receive a command. 3. Sorry about that, it was a little misleading, I meant: not an arduino. Either a prop. Or a picaxe. Also, in your oppinion, do you think it would be easier to add the computer to the communications cog (which will require adding an extra ID bit to the command and involve waiting until the board was ready (not already receiving), or to have an 08m monitor the port and take care of all that? 4. Thanks a lot for the help btw :)

1. Hmmm... MRL does have keyboard input but only in the context of previous robots.. where keyboard was used to do remote control.  I could make a keyboard service easy enough...

2. Ok, does this IRIS System already exist?  And can it do these things?  e.g. Lights 75%, Butler Bot Deliver drink 1 - Kitchen?

3. Not Arduino Prop or Picaxe?  You currently have something listening on a serial port?  Arduino is currently supported.. in that it has its own gui in MRL.  MRL can currently connects through the serial port to control an Arduino.  This could be done for anything which listens to the serial port.

 1. The text is from my keyboard.

 2. the text will be a genral command to the IRIS System. Anything from "Anthonys room - Lights - 75%" to "Butler Bot - Deliver: drink 1 - Kitchen".

 3. not arduino, prop. or picaxe (which do you think is easier to communicate with from myrobotlab?). When it gets the binary message, it will execute the command.