Let's Make Robots!


image tracking/processing on android with bluetooth/arduino controls
Proposal.doc18.5 KB
ScreenCapture.zip3.44 MB
CotsBotsArd_V2.zip5.93 KB
CotsBotsArd_V1.zip4.73 KB

This is DroidBot!, I'm doing some AI research for my University and this is a test bot I put together to test some of my code. It uses an Arduino Romeo to control the motors/servos via bluetooth connected to the mounted droid. Using the camera to do some image processing, it searches for a predefined set of pixels and relates a command on location. Its not very intelligent on decision making yet but I'm working on a full bore image recognition and tracking app that I will update when I can.


Proposal.doc.txt - Project proposal for anyone more interested in what's the plan, how I intend to use a genetic algorithm, etc.

The CotsBotArd zip contains the arduino library for the robot(CotsBot). Sorry I haven't had the time to write up the readme for it, I'll update that soon. Read through the header file and comment out anything you don't have setup, for example the IR sensor. Then load the example .pde on to your board. Basically it will wait for a serial command (F,B,L,R) in the form of ascii hex (caps) and move based on those commands. The idea here is to keep the microcontroller stuff super simple and fast, while having the computational stuff being done on the phone.

The src zip contains the three java files neccesary for the phone, NOTE that you must enter in your bluetooth modules MAC address before it'll work correctly, or you can add some code to pick from paired devices if you'd like. The code doesn't do much yet, just sets up a camera preview, creates the bluetooth connecting and looks for a specific pixel and moves based on that pixels position. Over the next few months I'll be working on tracking/searching algorithm.

I AM NEW TO ANDROID AND AI! I have taught myself almost all of this by practice and reading example code so please leave comments and suggestions I enjoy them all.

EDIT: March 2, 2012: I have implemented OpenCV in my android app. It will search for an object based on color, it learns by the user touching the screen in the location of the object, the more the user tells the app where to look the better its color ranging gets.

The apk is too large for this site, I'll work on that so people can check it out. The screenshots.zip folder contains some screenshots of it in action.



I've posted a few pictures of the tracking in action, I will be updating this page more seriously/completely in a couple (more like 4) weeks when summer starts. I have some interresting/usefull things to post.





Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Where can I find the app? Is the .apk file in the OpenCVBot file I found at your web site?

Here is a link to a zip file containing the last two APK's I have for this project, sorry to everyone who has been waiting, summer job has me swomped. <http://www2.cs.uidaho.edu/~lamb3619/fwdlmrdroidbot.zip>

Hey konto89,

I haven't had any time to keep my site updated this summer to do work but its on my list. I'll check and see about the apk when I get home today. Its worth mentioning that I have the MAC address for the bluetooth module hard coded, not the best solution, but you'll have to change that if you want to connect to your own module.

Sorry its taken me so long to update this project, I've recently been putting together a web site for all my projects(ALOT!). Its not complete yet but I have a few of my updated projects and source code available at the moment. I'll be working on it and will be updating my LMR pages soon as well. Heres the link <http://www2.cs.uidaho.edu/~lamb3619/>.

An ambitious and interesting project.  I will especially read with interest anything you write about how you handle the tracking, edge detection and movement.

My robot Shrödinger is lacking in things like edge detection (no camera, only photodiode/IR pulse/ultrasonic sonar type of sensors) so he has a bit of a problem when he approaches a drop-off like the top of a set of stairs. If allowed to roll merrily forward he would tumble down the stairs. I could have a downward-looking sensor, but never have added such. He is a pretty tough little guy, but stairs can still be his downfall.  (grins)

I have had thoughts of working on a 3D camera system, but that is going to have to wait for another project some time.

Thank you, I'm learning a lot myself but I'm making progress. Definittely is nice to have a more sophisticated control thanks to image processing. I too was interested in 3D camera systems but for now I'm really trying to keep everything on the a generic android device, something most people have. I will be posting an update here in the next week or so about my implementation and progress.

AMENDED: I notice that in the (first) video, the robot is having trouble identifying obstacles.  With the new software upgrades it is now looking very good at locating and following its target.

And as I said before, I will continue watching with great interest.

Good luck with this project.


Am looking forward to seeing your progress with OpenCV and edge detection for obstacle avoidance. I am currently implementing a ball following bot in Python and OpenCV and once I have cracked that was going to move onto object avoidance using OpenCV as opposed to using sensors connected directly to an aurdino.

Thank you, I will be updating this post here in the next week or so with a project update on how I implement the color recognition and how I use that for the object tracking.

You went the hard way to learn Android/Java, my respect for it. I started similar but abandoned much earlier because of lack of time to learn Android/Java. Then I met the IOIO board. That was the missing gap I needed, an easy start point.

Continue the good work. I've looked at your programs, because I was interested in the camera control. I would also try this later. I've played around also with a facial recognition program. Does not look very difficult to use this example code.