Let's Make Robots!

MyRobotLab - OpenCV - How to find your balls


 

 

This is a quick tutorial on one way to isolate an object in a video stream using myrobolab and OpenCV.  Once an object is isolated in the view, the computer can attempt to identify it, or locate it relative to some other mechanical extension, like grippers for example.  Using color to isolate objects is just one of many different methods.  Edge detection is another method which can be used.  In fact, these different processes can be combined together to derive more information from the video stream.

Hue, Saturation and Value (HSV) is another way to measure color.  In most video streams the color of each pixel is divided into Red, Blue, and Green values.  As light changes in the view, these values change considerably.  It was shocking to myself to see how much the values changed from one video frame to another !   Webcams are great sensors, but the data they give on a digital level is very noisy.  In order to limit some of this noise, or really organize it in a different way, the image is converted to Hue, Saturation and Value.   In simple form, Hue represents "what color",  Saturation represents "how much color", and value represents "how bright".   In this way looking for color objects is simplified such that looking for a particular color, means you are just looking for a specific value or range of Hue.

In myrobotlab you will need a compatible webcam and the OpenCV service, I usually name mine "camera", you can choose a name you prefer.

Add the filter InRange - this begins by converting your image to HSV.

In this demonstration I will be isolating the pink ball.

Highlight the newly named filter and dialog controls will appear.
First we will enable the Hue (color) filter.  When it is first enabled a white screen should appear.  This is because the default settings for Hue are set to allow everything through, and the results is a white mask which represents matching pixels.

I will now lower the max hue until the object I am interested in turns black.  This means the max value of hue is below what it needs to be for the pink ball.

I'll now move the slider back up to allow "pink" to go through the filter.

Now, I'll do the same thing with the min hue control, in an attempt to find the minimum and maximum "pink" color boundary.  As you can see there is little bits of "pink" in a lot of other objects.

We will use the other 2 filters to narrow and isolate the object of interest.  At this point the object is pretty "clean", there are a few speckles which flash on and off in the video stream, but not too bad.  You can see the highlight on the ball as a black spot since it does not come near our current values of "pink" 

I usually like to add a Dilate filter to smooth out the edges and remove any black speckles from the ball.

Now, would be the time to add a FindContours filter.  FindContours is a Blob Detector .   It computationally will group pixels together and define polygons.  It would be useful to find the relative location of the pink ball, its center, and size, even the number of edges.  However, this filter is computationally expensive.  It becomes very expensive depending on how many blobs are in view.  The more blobs the more time it will take to find them, so it's best to have a very clean and simple image before sending it to FindContours. It this case it took 225 milliseconds for all filters including FindContours to process. 

We can see now the Pink Ball is an 8 sided polygon (octagon), it's center is x 293, y 198,  it's bounding box is 103 x 103 pixels  giving a rough area of 10609 pixels

With configuration the polygons can be sent to some other Services, such as motors, servos or something else, which in turn might make a robot roll towards or attempt to grasp the pretty pink ball.  

GO GET THAT BALL!


 

 

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Yes! It worked! Looks like Microsoft Visual C++ had changed the PATH variable... screw them. Anyways, thanks for the good customer support ;) (joking off course).

attractive,,,

ill start using Myrobotlab ..

 

 

thanks GroG

Do you mean I have to add a new PATH variable as described in here: http://www.java.com/en/download/help/path.xml? Oh, and where to? Thanks

The PATH is a windows environment variable. It is set to a list of directory paths which windows will look for executables to run when requested. Somewhere on your computer exists a java.exe file. You want to find where this file is on your computer and add that location to your PATH variable. Google adding Java to windows system path

I get the "java is not recognised as an internal or external command, operable program or batch file." error. As to the log, only the zipped log.txt has text. It says the same thing as last time. The unzipped log.txt is blank.

This is helpful.  A few things:

1. Don't worry about the .sh files - they are for Linux (sort of .bat equivalents)
2. You will need the thirdParty.zip if you want to use the OpenCV stuff 
3. I have made a bug entry to look into the details of why you get the error when you double click the .jar
4. You are only suppose to need to double click the myrobotlab.bat - but since this is not working, lets try this. 
     First we want to see what is going on...  Maybe the java exe is not in your path - if not the following text should come up
      "java.exe" is not recognized as an internal or external command ... blah blah 

To confirm this - we need the black box to stay open:

  • Open a Command Prompt - This is usually found here  Start->Accessories->Command Prompt
  • in the box type "java" see if it comes up with the error above or 

C:\>java
Usage: java [-options] class [args...]
           (to execute a class)
   or  java [-options] -jar jarfile [args...]
           (to execute a jar file)

You don't want the "not recognized error" - if you do, you will have to find java.exe and put it in your path.

If the preferrable Usage: comes up then copy and paste the following in the command prompt when your in the myrobotlab-0001 directory

java -Djava.library.path=./bin  -cp "myrobotlab.jar;lib/*" com.myrobotlab.service.Invoker -service Invoker invoker GUIService gui > log.txt

send me the log.txt and any text which appears.

 

Yes It's like this. When I run the .hs it says it can't open it or needs a special programm and when I run .bat it opens the command prompt, but closes so fast I can only glimpse at "Invoker". When I open .jar it says cant find class com.myrobotlab.Invoker.service. I have copied the dll files into WINDOWS/system32 and I haven't downloaded third party files yet. The READ file only says svn 438 and the log is empty. That was in the unzipped folder. The zipped original folder's log has some error reports I guess:

service count 2

attempting to invoke : com.myrobotlab.service.Invoker invoker

0 [main] INFO class com.myrobotlab.comm.CommunicationManager - instanciating a com.myrobotlab.comm.CommObjectStreamOverTCPUDP

5 [main] DEBUG class com.myrobotlab.framework.Service - invoker registerServices

125 [main] INFO class com.myrobotlab.framework.Service - void main([Ljava.lang.Class;@29428e) adding as new MethodEntry

125 [main] INFO class com.myrobotlab.framework.Service - void help([Ljava.lang.Class;@d0a5d9) adding as new MethodEntry

126 [main] INFO class com.myrobotlab.framework.Service - void removeService([Ljava.lang.Class;@388993) adding as new MethodEntry

126 [main] INFO class com.myrobotlab.framework.Service - void loadDefaultConfiguration([Ljava.lang.Class;@1d04653) adding as new MethodEntry

126 [main] INFO class com.myrobotlab.framework.Service - void invokeCMDLine([Ljava.lang.Class;@b8f82d) adding as new MethodEntry

126 [main] INFO class com.myrobotlab.framework.Service - class [Ljava.lang.String; getServiceShortClassNames([Ljava.lang.Class;@1ad77a7) adding as new MethodEntry

127 [main] INFO class com.myrobotlab.framework.Service - void addService([Ljava.lang.Class;@18aaa1e) adding as new MethodEntry

127 [invoker_outbox] DEBUG com.myrobotlab.framework.Outbox - outbox run WAITING invoker_outbox

128 [invoker] DEBUG com.myrobotlab.framework.Inbox - inbox getMsg just before synchronized

128 [invoker] DEBUG com.myrobotlab.framework.Inbox - Inbox WAITING invoker

attempting to invoke : com.myrobotlab.service.GUIService gui

206 [main] INFO class com.myrobotlab.comm.CommunicationManager - instanciating a com.myrobotlab.comm.CommObjectStreamOverTCPUDP

206 [main] DEBUG class com.myrobotlab.framework.Service - gui registerServices

274 [main] INFO class com.myrobotlab.framework.Service - void run([Ljava.lang.Class;@d80be3) adding as new MethodEntry

275 [main] INFO class com.myrobotlab.framework.Service - void loadDefaultConfiguration([Ljava.lang.Class;@1f4689e) adding as new MethodEntry

275 [main] INFO class com.myrobotlab.framework.Service - void display([Ljava.lang.Class;@1006d75) adding as new MethodEntry

275 [main] INFO class com.myrobotlab.framework.Service - void registerServicesNotify([Ljava.lang.Class;@1125127) adding as new MethodEntry

276 [main] INFO class com.myrobotlab.framework.Service - class com.myrobotlab.service.data.IPAndPort noConnection([Ljava.lang.Class;@18dfef8) adding as new MethodEntry

276 [main] INFO class com.myrobotlab.framework.Service - class java.util.HashMap getServiceGUIMap([Ljava.lang.Class;@15e83f9) adding as new MethodEntry

276 [main] INFO class com.myrobotlab.framework.Service - class com.myrobotlab.control.ServiceTabPane loadTabPanels([Ljava.lang.Class;@2a5330) adding as new MethodEntry

276 [main] INFO class com.myrobotlab.framework.Service - void GUIUpdated([Ljava.lang.Class;@bb7465) adding as new MethodEntry

276 [main] INFO class com.myrobotlab.framework.Service - class java.lang.String setRemoteConnectionStatus([Ljava.lang.Class;@d6c16c) adding as new MethodEntry

277 [main] INFO class com.myrobotlab.framework.Service - class java.awt.image.BufferedImage processImage([Ljava.lang.Class;@134bed0) adding as new MethodEntry

2770 [gui_outbox] DEBUG com.myrobotlab.framework.Outbox - outbox run WAITING gui_outbox

2840 [gui] DEBUG com.myrobotlab.framework.Inbox - inbox getMsg just before synchronized

2840 [gui] DEBUG com.myrobotlab.framework.Inbox - Inbox WAITING gui

3167 [main] INFO com.myrobotlab.service.GUIService - gui

3446 [main] INFO com.myrobotlab.service.GUIService - invoker

Hi Markamas.
Did you follow the installation instructions?
Does your directory look like this?

Does a dos shell open up when you start the bat file?  Can you send me the log.txt ?

 

 I am trying to install this on vista and it says it cant find class com.myrobotlab.service.Invoker

Other than that my computer doesnt seem to be able to read .sh files

Do you have any solutions to this?

Cool! Same as RoboRealm.