Let's Make Robots!

Live soccer robots on LMR

Recently Gareth suggested to me that it would be good to have 2 teams of my bi-ped robots playing soccer. This was too good an idea to ignore.

After talking about this idea backstage this is where we are at:

On behalf of DAGU I will donate 6 Mini-Biped robots (plus spare parts) and an Arduino compatible IR transmitter to control them. Frits has volunteered to host the robot soccer match. Players would control the robots from their homes via the internet and the video would be streamed on LMR so the players can see what their robot is doing.

What we need is someone who can write some simple software to be loaded on the player and host computers.
The software for the the players will take keyboard commands (e.g. Cursor keys and spacebar) and send this information via the Internet to the host PC.
The software for the host computer (preferably mac compatible) will receive the data from the 6 player computers and send it to the Arduino compatible transmitter as serial data.
The data only needs to be a simple code such as the number 1-6 indicating which player has pressed a key followed by a letter A-E indicating which key was pressed.
A = forward       
B = backward
C = left
D = right
E = stop / kick
Thus the data stream would be something like 0D1A2A3E4D5C etc.
These robots are small, slow and clumsy so while the game will not be fast and furious I hope it will be fun to watch as well as to play.



Below is an explanation of how I will be programming the Arduino transmitter which will receive serial data from the host computer the same as it would using the serial monitor built into the Arduino enviroment. The Host computer will simply pass on the data it receives from the players computers.

Because the IR receiver needs to receive at least 14 pulses at 38KHz to recognise a 1, the Sony protocol uses 600uS as the width of a data bit and 1200uS as the width of the start bit. These times could be shortened but that could reduce reliability so for now I will stick with them. Beyond that I am ditching the Sony protocol which has 7 data bits and 5 device bits for a shorter, faster protocol.

I will write the Arduino code to send out 1 start bit, 3 player ID bits and 3 command bits. This will allow a maximum of 8 player and 8 commands. For now we will only have 6 players and 5 commands.

The Arduino will poll the player IDs thus the first 3 ID bits will count 0-5 repeatedly. The command sent out will be the last command received for that robot.

Each robot will be programmed with it's own ID so for example when the code 1A is sent out, all robots will receive the code but only robot 1 will respond and walk forward.

Because the slow IR transmission presents a bottle neck to the flow of data it would probably be best if the program running on the players computers limited the rate it sent command to 10 commands per seccond. This is fast enough to ensure a reasonable response time but will prevent the Arduino serial buffer from overflowing.

7 lovely red robot brains sitting on my desk. The main PCB is a simple little Arduino compatible board. The smaller PCB at the back is his "backpack" with speaker and IR comms hardware. An IR compound eye plugs into the front. I should have 7 little robots on their way to Frit's house next week. The seventh is a spare, just in case.

I've dubbed them the 7 dwarfs.
Here are some links to video of the prototype so you can get an idea what their soccer skill are like :P

RC chasing a ball: http://v.youku.com/v_show/id_XMTg0NDMxNTY4.html

Autonomous goal keeper: http://v.youku.com/v_show/id_XMTk5NDY1MTcy.html

Update: 20-09-2010

I have upgraded the Game Transmitter hardware and software to allow the pan/tilt assemblies to fire lasers (or other weapons) at the players. Lets be honest, no robot competition is complete without laser fire!

I have attached the code for both the game transmitter and the soccerbots. Currently the soccerbots will accept commands from both the game transmitter and a TV remote. The TV remote allows the host (Frits) to calibrate and test the players without a computer. The players ID can also be entered by TV remote so that if a player dies during a match Frits can quicky replace it with the spare robot just by changing the spare robots player ID to that of the dead robot.

Calibration settings and player ID are stored in the robots EEPROM so that the data isn't lost when the robot is turned off.

Update: 22-09-2010

As the famous saying goes, the devil is in the details! The seven dwarfs are not accepting commands from the game transmitter. As they work fine from the TV remote I have to assume the problem is with the transmitter. On the oscilloscope the transmitter seems to work fine with a nice modulated 38KHz signal.

Until I can sort this problem out the Dwarfs are staying in China :(



Update: 22-01-2011

Well they are finally in Frits's workshop. If nothing else Frits will learn a bit about Arduino. I did a lot of last minute stuff before I sent it all out and forgot the ball in the process. Now I need to get some instructions together for Frits.

I've taken some of Frits's photos and labelled them so every one knows what all the thingys are.

The Mini-Bipeds are normally sold pre-programmed for use by younger robot enthusiest. Therefore a programming cable was not produced. As Frits's will probably need to update the code a few times I quickly made up a cable. Experienced users can make a serial cable quite easily or use the ISP socket.


Soccerbot_transmitter_V2.zip2.35 KB
Soccerbot_V1.zip3.7 KB

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
nuumio's picture

I've been trying to find "low lag" alternatives to uStream. I did that real simple remote controller demo too, but since GroG's been working on that one and he's progressing well I haven't put more effort on that one. Of course I'll help if I can and there's a need.

About laggy video issues: So far I've learned that technology to overcome any lag issues already exists. H.264 video codec can be used to produce video stream almost real time. Its open source implementation x264 can already do it (or at least they say it can). In this case the problem is not only the real time video encoding on the host but also transferring the video stream to players and spectators. On IP level the solution would be IP multicast. IP multicast however works only if every piece of hardware between broadcaster and receiver(s) supports it and there's helluva lot of hardware in the Internet. I have never tried this but I think multicast wouldn't work. Like GroG said on SB: 'so there's a lot of "if"s between you and me'

Another solution to transfer video stream efficiently could be peercasting. In short, it's peer-to-peer (P2P) video casting. In peercasting we could try to get player peers as close to host as possible to give players the lowest possible lag. Spectators just need to join the P2P network at some node/peer to get the stream as lag is not a big problem for them. P2P doesn't need any special support from network hardware, it already works. Also, it might be even possible to get live stream to you, OddBot, in CN. This is the option I'm looking at right now. Interestingly a lot of proprietary P2P video software on this list seems to come from China. I'm concentrating on free/open source software. Unfortunately it doesn't look very promising. There's a lot of software that has been developed a few years ago but most of it isn't maintained anymore. Maybe there's not enough demand for P2P video streaming to make it interesting to develop. I'll keep looking anyway.

More on this later. I have to head to the campus.

GroG's picture

Hi nuumio,

I am re-investigating the issues MyRobotLab has for distributing video.  The current state is that it works over tcp/ip but for a very limited time, the stream becomes corrupt.

I have made a working test of MultiCasting on my local LAN ... sadly the definition of multicasting means its  a class D network and therefore not really available in any form on the public internet

I looked a little into peercasting - SwarmPlayer-P2P looked "almost" promising ... and it seemed like it was a worthy cause to align with Wikipedia for possible video distribution.  Unfortunately, I loaded it (Chrome is not supported had to use FF) and Firefox requested I load a bunch of python modules, which are not in my current repo's.....    So that seems out...

The details on my current implementation is I'm using TCP and Object Streams - When the stream becomes corrupted - the socket would have have to reattach, which is very problematic if the stream get corrupted every 10 seconds.

So my next approach will be using UDP and quite possibly compressing each frame - until I get sidetracked again (SQUIRREL!)

If you have any brilliant ideas let me know.

nuumio's picture

Not that many brilliant ideas on my mind right now but if I get any I'll post them. When I looked at peercast solutions earlier it looked like there were many "promising" projects but none of them was really working (on multiple platforms). Too bad...

When I did tests with VLC the problem was that after a while the stream started to get "cumulative lag" and it never caught up the "real time" feed. I used x264 as video encoder and it did its job quite nicely. Every once in a while video stream got a bit corrupted (those green sluggish things/distortions) but it got fixed as soon as IDR-frame (or keyframe) arrived. I'm not sure if the "cumulative lag" was caused by x264 or the carrier protocol (I tried http, rtp, asf and "raw udp").

Hmm... Now that I think of it there's one idea emerging at some dark corner of my brain (not sure if that's going to be brilliant): It's been a while (maybe 5 years, give or take a few) since the last time I used serialization in Java. But if I remember correctly you can do it "the simple way" by using Serializable interface or specify your own methods doing reading and writing by using Externalizable interface. If you use Externalizable interface it could be possible to do some kind of synchronization for example in every 10 frames or so. Maybe send a certain bit/byte pattern in every 10th frame. So the receiving end expects sync pattern every 10th frame and if it doesn't get it just waits until it gets it and gets back in sync. I'm not sure if this is doable with Object Streams.


GroG's picture

Very early on, I did serialization into encoded text after compressing into JPEG - Yikes - probably will never support that again, but it did fit into an XML request (SOAP developers would be proud of me)

As to your mention of Java serialization, I quicky switched to that, it seems simple and clean.  Remote communication into the framework is handled with Java's ObjectOutput/InputStream over TCP/IP - the only structure communicated this was is a "Message" - which again keeps things simple and clean.

I have used this method of moving a JPG compressed video frame using a Message over TCP/IP.  All control messages (e.g. motor commands, servo commands etc) are handled the same way.  

So my strategy now is to augment the system with an optional UDP method of sending the Message.  I have not worked with Datagrams much previously, but there is no ACK or other overhead - just fire and forget.  

I have high hopes .. the lag should be cut down, since its connectionless it should bypass my biggest issue which is the TCP/IP breakdown of the connection.

The only compression I'm doing now is JPG - but after getting UDP working I look into x264 more deeply.

Thanks for letting me bounce ideas off you.

nuumio's picture

Ah, so the stream got broken at TCP/IP level. I thought that your data just got out of sync/distorted and the only way to recover was to reconnect. And this happened in LAN too? Damn, I've never had that kind of problem. But then again I haven't done that much TCP/IP things with Java. I did some P2P and MMORPG try outs few years ago and I'm pretty sure I used TCP without any problems but my data rates were probably much lower than yours.

Just another thought: Maybe it's an Exception on Serialization/Object Stream (data corruption again, I seem to have developed a fixation to it ;-) that climbs up to Socket and breaks it. I'm just guessing here.


GroG's picture

I get a few different errors, but the usual exception thrown is StreamCorruptException ... some of the info coming back from this exception is "unexpected block" or "unknown type 74".  So its not actually breaking the TCP/IP connection, sorry if I mislead you.  But since I am using ObjectInputStream & ObjectOutputStream - once the stream is corrupt it is unusable at the receiving end.  I'm guessing it becomes much like any synchronous serial stream out of wack.  Maybe there is a way to signal the sender to reset.  I tried a few quick attempts of this, but did not get it to work. Then I thought  since video is typically acceptable if a few frames are dropped or out of sequence the transport could be UDP.  Since UDP is connectionless you don't need to reset the connection in order to receive a new Datagram, you just throw the corrupted one away.

nuumio's picture

Ok, now I think I got it now. But unfortunately I have no ideas how to fix that. I've never seen such exception myself.

Like you said, UDP might be better for the job because it's connectionless. Couple of things that came to my mind about UDP at this point (you might already be familiar with these): It's easy if your frame/message fits into single UDP datagram (practical limit with IPv4 is 65,507 bytes [Wikipedia]) and packets don't necessarily arrive in same order as they were sent (because it's connectionless and two sequential datagrams can get routed differently). Both issues can of course be dealt with by simple "message splitting protocol" and/or frame numbering (if needed).


GroG's picture

Woot !   Video over UDP !

It took me a long time (relatively speaking).  I've never used UDP before and the paradigm is very different, also I have to deal with complexities of a Applet client.  This confused the hell out of me, as it was very clear you cant start TCP server sockets on applets, strangely, you can start UDP sockets (there is no server socket) and recieve on a port.  Then there was getting it incorporated into the framework.

Right now I changed it so all messages go over UDP, with a little config I should be able to make it a configurable switch between UDP and TCP.  It all worked over UDP !  I'll also put in a 3rd strategy of communicating based on datatypes - where video will be the only UDP messages and all the rest (e.g. control messages) will be TCP.

1. I got 2 clients to receive video over UDP TCP connections lasted about 5 - 10 seconds
2. While sending to both clients I downloaded several large files, ran several youtube video, internet speed test, etc to get my network as crowded as possible - my server actually froze (might have been chrome being a memory hog) ... video stopped momentarily, until I closed chrome... then POOF video came back on again to both clients

Rockin !

So, now I got to make a little bot and see how other's fare on driving it around with hopefully less laggy video...
Would like to do sound too... but gotta keep focused...

GroG's picture

Things that don't currently work

  • Logging in / Authentication will take some work - Authentication works - simple file on the host computer name & passwords
  • Exported scoreboard & Player roster need to be implemented - to associate the login name with the player
  • Details of the serial stream still need to be worked on
  • Blue Team Red Team
  • Strange concurrency bug I've seen pop up occasionly - keepin my eye on it
  • Host ping - Turns out we don't need this
  • update of latency times
  • Clean up when a player either drops the connection or leaves - This was big - might have some bugs
  • Connectivity checks
  • Host gui's are not all updated  when events happen
  • Start testing with others here (no bots - but i can hook it up to a few servos, laundry machine, and some other junk) YAY! - thanks Gareth!
  • Found Testing with Gareth the video feed - connection is broken easily probably due to amount of data and glich moving over tcpip - Gareth reported that Gray & Small is speedy - gotta look into UDP, the complexity of re-establishing TCP and preserving all the necessary state is ... well, yucky
Gareth's picture

Experimentation - Shouting with Grog ........ oh yes the net is drawing closer